[26.0] Standardize agent API schemas and response metadata#21692
Draft
dannon wants to merge 12 commits intogalaxyproject:release_26.0from
Draft
[26.0] Standardize agent API schemas and response metadata#21692dannon wants to merge 12 commits intogalaxyproject:release_26.0from
dannon wants to merge 12 commits intogalaxyproject:release_26.0from
Conversation
Member
|
Did you want to target 26.0 ? |
dannon
force-pushed
the
agent-response-standardization
branch
from
b52bf0b to
1e7078a
Compare
github-actions
bot
changed the title
dannon
force-pushed
the
agent-response-standardization
branch
2 times, most recently
from
ac09883 to
abdc3cb
Compare
Added helper methods (_build_metadata, _build_response) to the base agent class so all agents produce consistent metadata with model name, token usage, and method tracking. The router now properly serializes handoff responses to preserve the delegated agent's metadata through the routing layer. Fixed ChatResponse schema - agent_response and exchange_id fields were being silently dropped by Pydantic, which is why suggestions weren't showing in the UI. Also removed the `as any` casts in ChatGXY.vue now that types are correct.
Updated ChatResponse to use typed AgentResponse instead of untyped dict, and added processing_time field. ChatPayload now supports a regenerate flag to bypass cached responses for job-based queries. Deprecated /api/ai/agents/query (duplicates /api/chat), removed dead code (routing_info, stream fields that were never used). Added save_exchange parameter to error-analysis and custom-tool endpoints for opt-in feedback tracking.
Fields with non-None defaults still generate as required in TypeScript. Changed to default=None and handle None as False in the API handler.
ActionSuggestions now validate required parameters based on action type at creation time, catching invalid suggestions before they reach the frontend. - TOOL_RUN requires tool_id - SAVE_TOOL requires tool_yaml - VIEW_EXTERNAL requires url - DOCUMENTATION, CONTACT_SUPPORT, REFINE_QUERY have no required params Also fixed error_analysis agent to use REFINE_QUERY instead of TOOL_RUN for solution guidance since these aren't specific tool invocations.
Extended mocked integration and unit tests to verify that agent responses include valid suggestions: - custom_tool: verify SAVE_TOOL suggestion has tool_yaml and tool_id - error_analysis: verify suggestions have action_type, description, confidence - Added unit tests for _create_suggestions validating ActionSuggestion params
ActionSuggestions should only be for concrete actions the frontend can execute, not general guidance. Error analysis now only emits CONTACT_SUPPORT when admin intervention is needed, with a pre-filled message containing error context. Removed REFINE_QUERY suggestions - if the agent needs clarification it should ask conversationally, not via an action button.
- base.py: removed keyword-matching heuristics that created invalid TOOL_RUN suggestions without tool_id - tools.py: validate tool exists in toolbox before creating TOOL_RUN suggestion in simple response parsing path - error_analysis: returns empty suggestions for now (CONTACT_SUPPORT not wired up)
Agents should ask clarifying questions conversationally, not via action buttons. VIEW_EXTERNAL and DOCUMENTATION kept as placeholders for future use.
Replace raw string confidence values with ConfidenceLevel enum everywhere — base validation response, error_analysis structured and simple paths, router handoff deserialization, and tool recommendation's verbose if/elif chains. The internal AgentResponse constructor already normalized strings, but callers should be explicit about types.
The simple text parsing path was always returning a CONTACT_SUPPORT suggestion regardless of the error, while the structured output path correctly returned empty suggestions. Aligned both paths to return empty suggestions -- support contact should only be suggested when it's actually the right course of action, not as a default fallback.
Converted all four direct AgentResponse constructions in tools.py to use _build_response(), so they get consistent metadata (model name, token usage, query_length). Added reasoning parameter to _build_response() to support it.
HandoffInfo was defined but never populated -- handoff data flows through metadata["handoff_info"] as a plain dict, which the frontend already reads. No reason to have a typed schema field that's always None.
dannon
force-pushed
the
agent-response-standardization
branch
from
abdc3cb to
f7a75d3
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Followup to #21580 — final cleanup/standardization pass on the agent system before release.
Goals: make agent responses consistent across all agents, fix schema issues causing data loss, and tighten suggestion validation.
_build_metadata()/_build_response()helpers so all agents produce consistent responsesChatResponse.agent_responseasOptional[AgentResponse]instead ofdict[str, Any]processing_time,regenerate,save_exchangefields to relevant schemas/api/ai/agents/query(duplicates/api/chat), removed deadrouting_info/streamfieldsmodel_validatoronActionSuggestionenforcing required params per action typeConfidenceLevelenum used consistently everywhere instead of raw stringsHandoffInfoschema model and preemptive CONTACT_SUPPORT suggestionsHow to test the changes?
License