();\n```\n\n## Decision Outcome\n\n### Response Type Options Decision\n\nOption 1.1 with the caveate that we cannot control the output of all agents. However, as far as possible we should have appropriate AIContext derived types for\nprogress updates so that TextContent is not used for these.\n\n### Custom Response Type Design Options Decision\n\nOption 2 chosen so that we can vary Agent responses independently of Chat Client.\n\n### StructuredOutputs Decision\n\nWe will not support structured output per run request, but individual agents are free to allow this on the concrete implementation or at construction time.\nWe will however add support for easily extracting a structured output type from the `AgentResponse`.\n\n## Addendum 1: AIContext Derived Types for different response types / Gap Analysis (Work in progress)\n\nWe need to decide what AIContent types, each agent response type will be mapped to.\n\n| Number | DataType | AIContent Type |\n|-|-|-|\n| 1. | General response messages to the user | TextContent + DataContent + UriContent |\n| 2. | Structured confirmation requests to the user | ? |\n| 3. | Function invocation activities executed (both local and remote). For information only. | FunctionCallContent + FunctionResultContent |\n| 4. | Tool invocation activities executed (both local and remote). For information only. | FunctionCallContent/FunctionResultContent/Custom ? |\n| 5. | Reasoning/Thinking output. For information only. | TextReasoningContent |\n| 6. | Handoffs / transitions from agent to agent. | ? |\n| 7. | An indication that the agent is responding (i.e. typing) as if it's a real human. | ? |\n| 8. | Complete messages in addition to updates, when streaming | TextContent |\n| 9. | Id for long running process that is launched | ? |\n| 10. | Memory storage / lookups (are these just traces?) | ? |\n| 11. | RAG indexing / lookups (are these just traces?) | ? |\n| 12. | General status updates for human consumption / Tracing | ? |\n| 13. | Unknown Type | AIContent |\n\n## Addendum 2: Other SDK feature comparison\n\n### Structured Outputs Support\n\n1. Configure Schema on Agent at Agent construction\n2. Pass schema at Agent invocation\n\n| SDK | Structured Outputs support |\n|-|-|\n| AutoGen | **Approach 1** Supports [configuring an agent](https://microsoft.github.io/autogen/stable/user-guide/agentchat-user-guide/tutorial/agents.html#structured-output) at agent creation. |\n| Google ADK | **Approach 1** Both [input and output schemas can be specified for LLM Agents](https://google.github.io/adk-docs/agents/llm-agents/#structuring-data-input_schema-output_schema-output_key) at construction time. This option is specific to this agent type and other agent types do not necessarily support |\n| AWS (Strands) | **Approach 2** Supports a special invocation method called [structured_output](https://strandsagents.com/docs/api/python/strands.agent.agent/) |\n| LangGraph | **Approach 1** Supports [configuring an agent](https://langchain-ai.github.io/langgraph/agents/agents/?h=structured#6-configure-structured-output) at agent construction time, and a [structured response](https://langchain-ai.github.io/langgraph/agents/run_agents/#output-format) can be retrieved as a special property on the agent response |\n| Agno | **Approach 1** Supports [configuring an agent](https://docs.agno.com/input-output/structured-output/agent) at agent construction time |\n| A2A | **Informal Approach 2** Doesn't formally support schema negotiation, but [hints can be provided via metadata](https://a2a-protocol.org/latest/specification/#97-structured-data-exchange-requesting-and-providing-json) at invocation time |\n| Protocol Activity | Supports returning [Complex types](https://github.com/microsoft/Agents/blob/main/specs/activity/protocol-activity.md#complex-types) but no support for requesting a type |\n\n### Response Reason Support\n\n| SDK | Response Reason support |\n|-|-|\n| AutoGen | Supports a [stop reason](https://microsoft.github.io/autogen/stable/reference/python/autogen_agentchat.base.html#autogen_agentchat.base.TaskResult.stop_reason) which is a freeform text string |\n| Google ADK | [No equivalent present](https://github.com/google/adk-python/blob/main/src/google/adk/events/event.py) |\n| AWS (Strands) | Exposes a [stop_reason](https://strandsagents.com/docs/api/python/strands.types.event_loop/) property on the [AgentResult](https://strandsagents.com/docs/api/python/strands.agent.agent_result/) class with options that are tied closely to LLM operations. |\n| LangGraph | No equivalent present, output contains only [messages](https://langchain-ai.github.io/langgraph/agents/run_agents/#output-format) |\n| Agno | [No equivalent present](https://docs.agno.com/reference/agents/run-response) |\n| A2A | No equivalent present, response only contains a [message](https://a2a-protocol.org/latest/specification/#64-message-object) or [task](https://a2a-protocol.org/latest/specification/#61-task-object). |\n| Protocol Activity | [No equivalent present.](https://github.com/microsoft/Agents/blob/main/specs/activity/protocol-activity.md) |\n"
},
{
"path": "docs/decisions/0002-agent-tools.md",
"content": "---\n# These are optional elements. Feel free to remove any of them.\nstatus: {proposed}\ncontact: {dmytrostruk}\ndate: {2025-06-23}\ndeciders: {stephentoub, markwallace-microsoft, RogerBarreto, westey-m}\nconsulted: {}\ninformed: {}\n---\n\n# Agent Tools\n\n## Context and Problem Statement\n\nAI agents increasingly rely on diverse tools like function calling, file search, and computer use, but integrating each tool often requires custom, inconsistent implementations. A unified abstraction for tool usage is essential to simplify development, ensure consistency, and enable scalable, reliable agent performance across varied tasks.\n\n## Decision Drivers\n\n- The abstraction must provide a consistent API for all tools to reduce complexity and improve developer experience.\n- The design should allow seamless integration of new tools without significant changes to existing implementations.\n- Robust mechanisms for managing tool-specific errors and timeouts are required for reliability.\n- The abstraction should support a fallback approach to directly use unsupported or custom tools, bypassing standard abstractions when necessary.\n\n## Considered Options\n\n### Option 1: Use ChatOptions.RawRepresentationFactory for Provider-Specific Tools\n\n#### Description\n\nUtilize the existing `ChatOptions.RawRepresentationFactory` to inject provider-specific tools (e.g., for an AI provider like Foundry) without extending the `AITool` abstract class from `Microsoft.Extensions.AI`.\n\n```csharp\nChatOptions options = new()\n{\n RawRepresentationFactory = _ => new ResponseCreationOptions()\n {\n Tools = { ... }, // backend-specific tools\n },\n};\n```\n\n#### Pros\n\n- No development work needed; leverages existing `Microsoft.Extensions.AI` functionality.\n- Flexible for integrating tools from any AI provider without modifying the `AITool`.\n- Minimal codebase changes, reducing the risk of introducing errors.\n\n#### Cons\n\n- Requires a separate mechanism to register tools, complicating the developer experience.\n- Developers must know the specific AI provider (via `IChatClient`) to configure tools, reducing abstraction.\n- Inconsistent with the `AITool` abstraction, leading to fragmented tool usage patterns.\n- Poor tool discoverability, as they are not integrated into the `AITool` ecosystem.\n\n### Option 2: Add Provider-Specific AITool-Derived Types in Provider Packages\n\n#### Description\n\nCreate provider-specific tool types that inherit from the `AITool` abstract class within each AI provider’s package (e.g., a Foundry package could include Foundry-specific tools). The provider’s `IChatClient` implementation would natively recognize and process these `AITool`-derived types, eliminating the need for a separate registration mechanism.\n\n#### Pros\n\n- Integrates with the `AITool` abstract class, providing a consistent developer experience within the `Microsoft.Extensions.AI`.\n- Eliminates the need for a special registration mechanism like `RawRepresentationFactory`.\n- Enhances type safety and discoverability for provider-specific tools.\n- Aligns with the standardized interface driver by leveraging `AITool` as the base class.\n\n#### Cons\n\n- Developers must know they are targeting a specific AI provider to select the appropriate `AITool`-derived types.\n- Increases maintenance overhead for each provider’s package to support and update these tool types.\n- Leads to fragmentation, as each provider requires its own set of `AITool`-derived types.\n- Potential for duplication if multiple providers implement similar tools with different `AITool` derivatives.\n\n### Option 3: Create Generic AITool-Derived Abstractions in M.E.AI.Abstractions\n\n#### Description\n\nDevelop generic tool abstractions that inherit from the `AITool` abstract class in the `M.E.AI.Abstractions` package (e.g., `HostedCodeInterpreterTool`, `HostedWebSearchTool`). These abstractions map to common tool concepts across multiple AI providers, with provider-specific implementations handled internally.\n\n#### Pros\n\n- Provides a standardized `AITool`-based interface across AI providers, improving consistency and developer experience.\n- Reduces the need for provider-specific knowledge by abstracting tool implementations.\n- Highly extensible, supporting new `AITool`-derived types for common tool concepts (e.g., server-side MCP tools).\n\n#### Cons\n\n- Complex mapping logic needed to support diverse provider implementations.\n- May not cover niche or provider-specific tools, necessitating a fallback mechanism.\n\n### Option 4: Hybrid Approach Combining Options 1, 2, and 3\n\n#### Description\n\nImplement a hybrid strategy where common tools use generic `AITool`-derived abstractions in `M.E.AI.Abstractions` (Option 3), provider-specific tools (e.g., for Foundry) are implemented as `AITool`-derived types in their respective provider packages (Option 2), and rare or unsupported tools fall back to `ChatOptions.RawRepresentationFactory` (Option 1).\n\n#### Pros\n\n- Balances developer experience and flexibility by using the best `AITool`-based approach for each tool type.\n- Supports standardized `AITool` interfaces for common tools while allowing provider-specific and breakglass mechanisms.\n- Extensible and scalable, accommodating both current and future tool requirements across AI providers.\n- Addresses ancillary and intermediate content (e.g., MCP permissions) with generic types.\n\n#### Cons\n\n- Increases complexity by managing multiple `AITool` integration approaches within the same system.\n- Requires clear documentation to guide developers on when to use each option.\n- Potential for inconsistency if boundaries between approaches are not well-defined.\n- Higher maintenance burden to support and test multiple tool integration paths.\n\n## More information\n\n### AI Agent Tool Types Availability\n\nTool Type | Azure AI Foundry Agent Service | OpenAI Assistant API | OpenAI ChatCompletion API | OpenAI Responses API | Amazon Bedrock Agents | Google | Anthropic | Description\n-- | -- | -- | -- | -- | -- | -- | -- | --\nFunction Calling | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | Enables custom, stateless functions to define specific agent behaviors.\nCode Interpreter | ✅ | ✅ | ❌ | ✅ | ✅ | ✅ | ✅ | Allows agents to execute code for tasks like data analysis or problem-solving.\nSearch and Retrieval | ✅ (File Search, Azure AI Search) | ✅ (File Search) | ❌ | ✅ (File Search) | ✅ (Knowledge Bases) | ✅ (Vertex AI Search) | ❌ | Enables agents to search and retrieve information from files, knowledge bases, or enterprise search systems.\nWeb Search | ✅ (Bing Search) | ❌ | ✅ | ✅ | ❌ | ✅ (Google Search) | ✅ | Provides real-time access to internet-based content using search engines or web APIs for dynamic, up-to-date information.\nRemote MCP Servers | ✅ | ❌ | ❌ | ✅ | ❌ | ✅ | ✅ | Gives the model access to new capabilities via Model Context Protocol servers.\nComputer Use | ❌ | ❌ | ❌ | ✅ | ✅ (ANTHROPIC.Computer) | ❌ | ✅ | Creates agentic workflows that enable a model to control a computer interface.\nOpenAPI Spec Tool | ✅ | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | Integrates existing OpenAPI specifications for service APIs.\nStateful Functions | ✅ (Azure Functions) | ❌ | ❌ | ❌ | ✅ (AWS Lambda) | ❌ | ❌ | Supports custom, stateful functions for complex agent actions.\nText Editor | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | ✅ | Allows agents to view and modify text files for debugging or editing purposes.\nAzure Logic Apps | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | Low-code/no-code solution to add workflows to AI agents.\nMicrosoft Fabric | ✅ | ❌ | ❌ | ❌ | ❌ | ❌ | ❌ | Enables agents to interact with data in Microsoft Fabric for insights.\nImage Generation | ❌ | ❌ | ❌ | ✅ | ❌ | ❌ | ❌ | Generates or edits images using GPT image.\n\n### API Comparison\n\n#### Function Calling\n\n Azure AI Foundry Agent Service
\n Source: https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/function-calling?pivots=rest\n\n Message Request:\n ```json\n {\n \"tools\": [\n { \n \"type\": \"function\",\n \"function\": {\n \"description\": \"{string}\",\n \"name\": \"{string}\",\n \"parameters\": \"{JSON Schema object}\"\n }\n }\n ]\n }\n ```\n\n Tool Call Response:\n ```json\n {\n \"tool_calls\": [\n {\n \"id\": \"{string}\",\n \"type\": \"function\",\n \"function\": {\n \"name\": \"{string}\",\n \"arguments\": \"{JSON object}\",\n }\n }\n ]\n }\n ```\n \n\n OpenAI Assistant API
\n Source: https://platform.openai.com/docs/assistants/tools/function-calling\n\n Message Request:\n ```json\n {\n \"tools\": [\n { \n \"type\": \"function\",\n \"function\": {\n \"description\": \"{string}\",\n \"name\": \"{string}\",\n \"parameters\": \"{JSON Schema object}\"\n }\n }\n ]\n }\n ```\n\n Tool Call Response:\n ```json\n {\n \"tool_calls\": [\n {\n \"id\": \"{string}\",\n \"type\": \"function\",\n \"function\": {\n \"name\": \"{string}\",\n \"arguments\": \"{JSON object}\",\n }\n }\n ]\n }\n ```\n \n\n OpenAI ChatCompletion API
\n Source: https://platform.openai.com/docs/guides/function-calling?api-mode=chat\n\n Message Request:\n ```json\n {\n \"tools\": [\n { \n \"type\": \"function\",\n \"function\": {\n \"description\": \"{string}\",\n \"name\": \"{string}\",\n \"parameters\": \"{JSON Schema object}\"\n }\n }\n ]\n }\n ```\n\n Tool Call Response:\n ```json\n [\n {\n \"id\": \"{string}\",\n \"type\": \"function\",\n \"function\": {\n \"name\": \"{string}\",\n \"arguments\": \"{JSON object}\",\n }\n }\n ]\n ```\n \n\n OpenAI Responses API
\n Source: https://platform.openai.com/docs/guides/function-calling?api-mode=responses\n\n Message Request:\n ```json\n {\n \"tools\": [\n { \n \"type\": \"function\",\n \"description\": \"{string}\",\n \"name\": \"{string}\",\n \"parameters\": \"{JSON Schema object}\"\n }\n ]\n }\n ```\n\n Tool Call Response:\n ```json\n [\n {\n \"id\": \"{string}\",\n \"call_id\": \"{string}\",\n \"type\": \"function_call\",\n \"name\": \"{string}\",\n \"arguments\": \"{JSON object}\"\n }\n ]\n ```\n \n\n Amazon Bedrock Agents
\n Source: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent_CreateAgentActionGroup.html#API_agent_CreateAgentActionGroup_RequestSyntax\n\n CreateAgentActionGroup Request:\n ```json\n {\n \"functionSchema\": {\n \"name\": \"{string}\",\n \"description\": \"{string}\",\n \"parameters\": {\n \"type\": \"{string | number | integer | boolean | array}\",\n \"description\": \"{string}\",\n \"required\": \"{boolean}\"\n }\n }\n }\n ```\n\n Tool Call Response:\n ```json\n {\n \"invocationInputs\": [\n {\n \"functionInvocationInput\": {\n \"actionGroup\": \"{string}\",\n \"function\": \"{string}\",\n \"parameters\": [\n {\n \"name\": \"{string}\",\n \"type\": \"{string | number | integer | boolean | array}\",\n \"value\": {}\n }\n ]\n }\n }\n ]\n }\n ```\n \n\n Google
\n Source: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling#rest\n\n Message Request:\n ```json\n {\n \"tools\": [\n {\n \"functionDeclarations\": [\n {\n \"name\": \"{string}\",\n \"description\": \"{string}\",\n \"parameters\": \"{JSON Schema object}\"\n }\n ]\n }\n ]\n }\n ```\n\n Tool Call Response:\n ```json\n {\n \"content\": {\n \"role\": \"model\",\n \"parts\": [\n {\n \"functionCall\": {\n \"name\": \"{string}\",\n \"args\": {\n \"{argument_name}\": {}\n }\n }\n }\n ]\n }\n }\n ```\n \n\n Anthropic
\n Source: https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/overview\n\n Message Request:\n ```json\n {\n \"tools\": [\n {\n \"name\": \"{string}\",\n \"description\": \"{string}\",\n \"input_schema\": \"{JSON Schema object}\"\n }\n ]\n }\n ```\n\n Tool Call Response:\n ```json\n {\n \"id\": \"{string}\",\n \"model\": \"{string}\",\n \"stop_reason\": \"tool_use\",\n \"role\": \"assistant\",\n \"content\": [\n {\n \"type\": \"text\",\n \"text\": \"{string}\"\n },\n {\n \"type\": \"tool_use\",\n \"id\": \"{string}\",\n \"name\": \"{string}\",\n \"input\": {\n \"argument_name\": {}\n }\n }\n ]\n }\n ```\n \n\n#### Commonalities\n\n- **Standardized Tool Definition**: All providers use a JSON-based structure for defining tools, including a `type` field (commonly \"function\") and a `function` object with `name`, `description`, and `parameters` (often following JSON Schema).\n- **Tool Call Response Structure**: Responses typically include a list of tool calls with an `id`, `type`, and details about the function called (e.g., `name` and `arguments`), enabling consistent handling of function invocations.\n- **JSON Schema for Parameters**: Parameters for functions are defined using JSON Schema objects across most providers, facilitating a unified approach to parameter validation and processing.\n- **Extensibility**: The structure allows for additional metadata or fields (e.g., `call_id`, `actionGroup`), suggesting potential for abstraction to support provider-specific extensions while maintaining core compatibility.\n\n
\n\n#### Code Interpreter\n\n Azure AI Foundry Agent Service
\n Source: https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/code-interpreter-samples?pivots=rest-api
\n\n .NET Support: ✅
\n\n Message Request:\n ```json\n {\n \"tools\": [\n { \n \"type\": \"code_interpreter\"\n }\n ],\n \"tool_resources\": {\n \"code_interpreter\": {\n \"file_ids\": [\"{string}\"],\n \"data_sources\": [\n {\n \"type\": {\n \"id_asset\": \"{string}\",\n \"uri_asset\": \"{string}\"\n },\n \"uri\": \"{string}\"\n }\n ]\n }\n }\n }\n ```\n\n Tool Call Response:\n ```json\n {\n \"tool_calls\": [\n {\n \"id\": \"{string}\",\n \"type\": \"code_interpreter\",\n \"code_interpreter\": {\n \"input\": \"{string}\",\n \"outputs\": [\n {\n \"type\": \"image\",\n \"file_id\": \"{string}\"\n },\n {\n \"type\": \"logs\",\n \"logs\": \"{string}\"\n }\n ]\n }\n }\n ]\n }\n ```\n \n\n OpenAI Assistant API
\n Source: https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/code-interpreter-samples?pivots=rest-api
\n\n .NET Support: ✅
\n\n Message Request:\n ```json\n {\n \"tools\": [\n { \n \"type\": \"code_interpreter\"\n }\n ],\n \"tool_resources\": {\n \"code_interpreter\": {\n \"file_ids\": [\"{string}\"]\n }\n }\n }\n ```\n\n Tool Call Response:\n ```json\n {\n \"tool_calls\": [\n {\n \"id\": \"{string}\",\n \"type\": \"code\",\n \"code\": {\n \"input\": \"{string}\",\n \"outputs\": [\n {\n \"type\": \"logs\",\n \"logs\": \"{string}\"\n }\n ]\n }\n }\n ]\n }\n ```\n \n\n OpenAI Responses API
\n Source: https://platform.openai.com/docs/guides/tools-code-interpreter
\n\n .NET Support: ❌ (currently in development: GitHub issue)
\n\n Message Request:\n ```json\n {\n \"tools\": [\n { \n \"type\": \"code_interpreter\",\n \"container\": { \"type\": \"auto\" }\n }\n ]\n }\n ```\n\n Tool Call Response:\n ```json\n [\n {\n \"id\": \"{string}\",\n \"code\": \"{string}\",\n \"type\": \"code_interpreter_call\",\n \"status\": \"{string}\",\n \"container_id\": \"{string}\",\n \"results\": [\n {\n \"type\": \"logs\",\n \"logs\": \"{string}\"\n },\n {\n \"type\": \"files\",\n \"files\": [\n {\n \"file_id\": \"{string}\",\n \"mime_type\": \"{string}\"\n }\n ]\n }\n ]\n }\n ]\n ```\n \n\n Amazon Bedrock Agents
\n Source: https://docs.aws.amazon.com/bedrock/latest/userguide/agents-enable-code-interpretation.html
\n\n .NET Support: ❌ (Amazon SDK has IChatClient implementation but lacks ChatOptions.RawRepresentationFactory)
\n\n CreateAgentActionGroup Request:\n ```json\n {\n \"actionGroupName\": \"{string}\",\n \"parentActionGroupSignature\": \"AMAZON.CodeInterpreter\",\n \"actionGroupState\": \"ENABLED\"\n }\n ```\n\n Tool Call Response:\n ```json\n {\n \"trace\": {\n \"orchestrationTrace\": {\n \"invocationInput\": {\n \"invocationType\": \"ACTION_GROUP_CODE_INTERPRETER\",\n \"codeInterpreterInvocationInput\": {\n \"code\": \"{string}\",\n \"files\": [\"{string}\"]\n }\n },\n \"observation\": {\n \"codeInterpreterInvocationOutput\": {\n \"executionError\": \"{string}\",\n \"executionOutput\": \"{string}\",\n \"executionTimeout\": \"{boolean}\",\n \"files\": [\"{string}\"],\n \"metadata\": {\n \"clientRequestId\": \"{string}\",\n \"endTime\": \"{timestamp}\",\n \"operationTotalTimeMs\": \"{long}\",\n \"startTime\": \"{timestamp}\",\n \"totalTimeMs\": \"{long}\",\n \"usage\": {\n \"inputTokens\": \"{integer}\",\n \"outputTokens\": \"{integer}\"\n }\n }\n }\n }\n }\n }\n }\n ```\n \n\n Google
\n Source: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/code-execution#googlegenaisdk_tools_code_exec_with_txt-drest
\n\n .NET Support: ❌ (official SDK lacks IChatClient implementation.)
\n\n Message Request:\n ```json\n {\n \"contents\": {\n \"role\": \"{string}\",\n \"parts\": { \n \"text\": \"{string}\" \n }\n },\n \"tools\": [\n {\n \"codeExecution\": {}\n }\n ]\n }\n ```\n\n Tool Call Response:\n ```json\n {\n \"content\": {\n \"role\": \"model\",\n \"parts\": [\n {\n \"executableCode\": {\n \"language\": \"{string}\",\n \"code\": \"{string}\"\n }\n },\n {\n \"codeExecutionResult\": {\n \"outcome\": \"{string}\",\n \"output\": \"{string}\"\n }\n }\n ]\n }\n }\n ```\n \n\n Anthropic
\n Source: https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/code-execution-tool
\n\n \n .NET Support: ❌
\n
\n - Anthropic.SDK - uses `code_interpreter` instead of `code_execution` and lacks a possibility to specify file id.
\n - Anthropic by tryAGI - has `code_execution` implementation, but it's in beta and can't be used as a tool.
\n
\n \n\n Message Request:\n ```json\n {\n \"tools\": [\n {\n \"name\": \"code_execution\",\n \"type\": \"code_execution_20250522\"\n }\n ]\n }\n ```\n\n Tool Call Response:\n ```json\n {\n \"role\": \"assistant\",\n \"container\": {\n \"id\": \"{string}\",\n \"expires_at\": \"{timestamp}\"\n },\n \"content\": [\n {\n \"type\": \"server_tool_use\",\n \"id\": \"{string}\",\n \"name\": \"code_execution\",\n \"input\": {\n \"code\": \"{string}\"\n }\n },\n {\n \"type\": \"code_execution_tool_result\",\n \"tool_use_id\": \"{string}\",\n \"content\": {\n \"type\": \"code_execution_result\",\n \"stdout\": \"{string}\",\n \"stderr\": \"{string}\",\n \"return_code\": \"{integer}\"\n }\n }\n ]\n }\n ```\n \n\n#### Commonalities\n\n- **Tool Type Specification**: Providers consistently define a `code_interpreter` tool type within the `tools` array, indicating support for code execution capabilities.\n- **Input and Output Handling**: Requests include mechanisms to specify code input (e.g., `input` or `code` fields), and responses return execution outputs, such as logs or files, in a structured format.\n- **File Resource Support**: Most providers allow associating files with the code interpreter (e.g., via `file_ids` or `files`), enabling data input/output for code execution.\n- **Execution Metadata**: Responses often include metadata about the execution process (e.g., `status`, `logs`, or `executionError`), which can be abstracted for standardized error handling and result processing.\n\n
\n\n#### Search and Retrieval\n\n Azure AI Foundry Agent Service
\n Source: https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/file-search-upload-files?pivots=rest\n\n File Search Request:\n ```json\n {\n \"tools\": [\n { \n \"type\": \"file_search\"\n }\n ],\n \"tool_resources\": { \n \"file_search\": {\n \"vector_store_ids\": [\"{string}\"],\n \"vector_stores\": [\n {\n \"name\": \"{string}\",\n \"configuration\": {\n \"data_sources\": [\n {\n \"type\": {\n \"id_asset\": \"{string}\",\n \"uri_asset\": \"{string}\"\n },\n \"uri\": \"{string}\"\n }\n ]\n }\n }\n ]\n }\n }\n }\n ```\n\n File Search Tool Call Response:\n ```json\n {\n \"tool_calls\": [\n {\n \"id\": \"{string}\",\n \"type\": \"file_search\",\n \"file_search\": {\n \"ranking_options\": {\n \"ranker\": \"{string}\",\n \"score_threshold\": \"{float}\"\n },\n \"results\": [\n {\n \"file_id\": \"{string}\",\n \"file_name\": \"{string}\",\n \"score\": \"{float}\",\n \"content\": [\n {\n \"text\": \"{string}\",\n \"type\": \"{string}\"\n }\n ]\n }\n ]\n }\n }\n ]\n }\n ```\n\n Azure AI Search Request:\n ```json\n {\n \"tools\": [\n { \n \"type\": \"azure_ai_search\"\n }\n ],\n \"tool_resources\": { \n \"azure_ai_search\": {\n \"indexes\": [\n {\n \"index_connection_id\": \"{string}\",\n \"index_name\": \"{string}\",\n \"query_type\": \"{string}\"\n }\n ]\n }\n }\n }\n ```\n\n Azure AI Search Tool Call Response:\n ```json\n {\n \"tool_calls\": [\n {\n \"id\": \"{string}\",\n \"type\": \"azure_ai_search\",\n \"azure_ai_search\": {} // From documentation: Reserved for future use\n }\n ]\n }\n ```\n \n\n OpenAI Assistant API
\n Source: https://platform.openai.com/docs/assistants/tools/file-search\n\n Message Request:\n ```json\n {\n \"tools\": [\n { \n \"type\": \"file_search\"\n }\n ],\n \"tool_resources\": { \n \"file_search\": {\n \"vector_store_ids\": [\"string\"]\n }\n }\n }\n ```\n\n Tool Call Response:\n ```json\n {\n \"tool_calls\": [\n {\n \"id\": \"{string}\",\n \"type\": \"file_search\",\n \"file_search\": {\n \"ranking_options\": {\n \"ranker\": \"{string}\",\n \"score_threshold\": \"{float}\"\n },\n \"results\": [\n {\n \"file_id\": \"{string}\",\n \"file_name\": \"{string}\",\n \"score\": \"{float}\",\n \"content\": [\n {\n \"text\": \"{string}\",\n \"type\": \"{string}\"\n }\n ]\n }\n ]\n }\n }\n ]\n }\n ```\n \n\n OpenAI Responses API
\n Source: https://platform.openai.com/docs/api-reference/responses/create\n\n Message Request:\n ```json\n {\n \"tools\": [\n { \n \"type\": \"file_search\"\n }\n ],\n \"tool_resources\": { \n \"file_search\": {\n \"vector_store_ids\": [\"string\"]\n }\n }\n }\n ```\n\n Tool Call Response:\n ```json\n {\n \"output\": [\n {\n \"id\": \"{string}\",\n \"queries\": [\"{string}\"],\n \"status\": \"{in_progress | searching | incomplete | failed | completed}\",\n \"type\": \"file_search_call\",\n \"results\": [\n {\n \"attributes\": {},\n \"file_id\": \"{string}\",\n \"filename\": \"{string}\",\n \"score\": \"{float}\",\n \"text\": \"{string}\"\n }\n ]\n }\n ]\n }\n ```\n \n\n Amazon Bedrock Agents
\n Source: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent-runtime_InvokeAgent.html\n\n Message Request:\n ```json\n {\n \"sessionState\": {\n \"knowledgeBaseConfigurations\": [\n { \n \"knowledgeBaseId\": \"{string}\",\n \"retrievalConfiguration\": { \n \"vectorSearchConfiguration\": { \n \"filter\": {},\n \"implicitFilterConfiguration\": { \n \"metadataAttributes\": [ \n { \n \"description\": \"{string}\",\n \"key\": \"{string}\",\n \"type\": \"{string}\"\n }\n ],\n \"modelArn\": \"{string}\"\n },\n \"numberOfResults\": \"{number}\",\n \"overrideSearchType\": \"{string}\",\n \"rerankingConfiguration\": { \n \"bedrockRerankingConfiguration\": { \n \"metadataConfiguration\": { \n \"selectionMode\": \"{string}\",\n \"selectiveModeConfiguration\": {}\n },\n \"modelConfiguration\": { \n \"additionalModelRequestFields\": { \n \"string\" : \"{JSON string}\"\n },\n \"modelArn\": \"{string}\"\n },\n \"numberOfRerankedResults\": \"{number}\"\n },\n \"type\": \"{string}\"\n }\n }\n }\n }\n ]\n }\n }\n ```\n\n Tool Call Response:\n ```json\n {\n \"trace\": {\n \"orchestrationTrace\": {\n \"invocationInput\": {\n \"invocationType\": \"KNOWLEDGE_BASE\",\n \"knowledgeBaseLookupInput\": {\n \"knowledgeBaseId\": \"{string}\",\n \"text\": \"{string}\"\n }\n },\n \"observation\": {\n \"type\": \"KNOWLEDGE_BASE\",\n \"knowledgeBaseLookupOutput\": {\n \"retrievedReferences\": [\n {\n \"metadata\": {},\n \"content\": {\n \"byteContent\": \"{string}\",\n \"row\": [\n {\n \"columnName\": \"{string}\",\n \"columnValue\": \"{string}\",\n \"type\": \"{BLOB | BOOLEAN | DOUBLE | NULL | LONG | STRING}\"\n }\n ],\n \"text\": \"{string}\",\n \"type\": \"{TEXT | IMAGE | ROW}\"\n }\n }\n ],\n \"metadata\": {\n \"clientRequestId\": \"{string}\",\n \"endTime\": \"{timestamp}\",\n \"operationTotalTimeMs\": \"{long}\",\n \"startTime\": \"{timestamp}\",\n \"totalTimeMs\": \"{long}\",\n \"usage\": {\n \"inputTokens\": \"{integer}\",\n \"outputTokens\": \"{integer}\"\n }\n }\n }\n }\n }\n }\n }\n ```\n \n\n Google
\n Source: https://cloud.google.com/vertex-ai/generative-ai/docs/grounding/grounding-with-vertex-ai-search\n\n Message Request:\n ```json\n {\n \"contents\": [\n {\n \"role\": \"user\",\n \"parts\": [\n {\n \"text\": \"{string}\"\n }\n ]\n }\n ],\n \"tools\": [\n {\n \"retrieval\": {\n \"vertexAiSearch\": {\n \"datastore\": \"{string}\"\n }\n }\n }\n ]\n }\n ```\n\n Tool Call Response:\n ```json\n {\n \"content\": {\n \"role\": \"model\",\n \"parts\": [\n {\n \"text\": \"{string}\"\n }\n ]\n },\n \"groundingMetadata\": {\n \"retrievalQueries\": [\n \"{string}\"\n ],\n \"groundingChunks\": [\n {\n \"retrievedContext\": {\n \"uri\": \"{string}\",\n \"title\": \"{string}\"\n }\n }\n ],\n \"groundingSupport\": [\n {\n \"segment\": {\n \"startIndex\": \"{number}\",\n \"endIndex\": \"{number}\"\n },\n \"segment_text\": \"{string}\",\n \"supportChunkIndices\": [\"{number}\"],\n \"confidenceScore\": [\"{number}\"]\n }\n ]\n }\n }\n ```\n \n\n#### Commonalities\n\n- **Vector Store Integration**: Providers like Azure and OpenAI use `vector_store_ids` or similar constructs to reference vector stores for file search, suggesting a common approach to retrieval-augmented generation.\n- **Search Configuration**: Requests include configurations for search (e.g., `vectorSearchConfiguration`, `ranking_options`), allowing customization of retrieval parameters like result count or ranking.\n- **Result Structure**: Responses contain a list of search results with fields like `file_id`, `score`, and `content` or `text`, enabling consistent processing of retrieved data.\n- **Metadata Inclusion**: Search responses often include metadata (e.g., `score`, `timestamp`, `usage`), which can be abstracted for unified analytics and performance tracking.\n\n
\n\n#### Web Search\n\n Azure AI Foundry Agent Service
\n Source: https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/bing-code-samples?pivots=rest\n\n Bing Search Message Request:\n ```json\n {\n \"tools\": [\n { \n \"type\": \"bing_grounding\",\n \"bing_grounding\": {\n \"search_configurations\": [\n {\n \"connection_id\": \"{string}\",\n \"count\": \"{number}\", \n \"market\": \"{string}\", \n \"set_lang\": \"{string}\", \n \"freshness\": \"{string}\",\n }\n ]\n }\n }\n ]\n }\n ```\n\n Bing Search Tool Call Response:\n ```json\n {\n \"tool_calls\": [\n {\n \"id\": \"{string}\",\n \"type\": \"function\",\n \"bing_grounding\": {} // From documentation: Reserved for future use\n }\n ]\n }\n ```\n \n\n OpenAI ChatCompletion API
\n Source: https://platform.openai.com/docs/guides/tools-web-search?api-mode=chat\n\n Message Request:\n ```json\n {\n \"web_search_options\": {},\n \"messages\": [\n {\n \"role\": \"user\",\n \"content\": \"{string}\"\n }\n ]\n }\n ```\n\n Tool Call Response:\n ```json\n [\n {\n \"index\": 0,\n \"message\": {\n \"role\": \"assistant\",\n \"content\": \"{string}\",\n \"annotations\": [\n {\n \"type\": \"url_citation\",\n \"url_citation\": {\n \"end_index\": \"{number}\",\n \"start_index\": \"{number}\",\n \"title\": \"{string}\",\n \"url\": \"{string}\"\n }\n }\n ]\n }\n }\n ]\n ```\n \n\n OpenAI Responses API
\n Source: https://platform.openai.com/docs/guides/tools-web-search?api-mode=responses\n\n Message Request:\n ```json\n {\n \"tools\": [\n {\n \"type\": \"web_search_preview\"\n }\n ],\n \"input\": \"{string}\"\n }\n ```\n\n Tool Call Response:\n ```json\n {\n \"output\": [\n {\n \"type\": \"web_search_call\",\n \"id\": \"{string}\",\n \"status\": \"{string}\"\n },\n {\n \"id\": \"{string}\",\n \"type\": \"message\",\n \"status\": \"{string}\",\n \"role\": \"assistant\",\n \"content\": [\n {\n \"type\": \"output_text\",\n \"text\": \"{string}\",\n \"annotations\": [\n {\n \"type\": \"url_citation\",\n \"start_index\": \"{number}\",\n \"end_index\": \"{string}\",\n \"url\": \"{string}\",\n \"title\": \"{string}\"\n }\n ]\n }\n ]\n }\n ]\n }\n ```\n \n\n Google
\n Source: https://cloud.google.com/vertex-ai/generative-ai/docs/grounding/grounding-with-google-search\n\n Message Request:\n ```json\n {\n \"contents\": [\n {\n \"role\": \"user\",\n \"parts\": [\n {\n \"text\": \"{string}\"\n }\n ]\n }\n ],\n \"tools\": [\n {\n \"googleSearch\": {}\n }\n ]\n }\n ```\n\n Tool Call Response:\n ```json\n {\n \"content\": {\n \"role\": \"model\",\n \"parts\": [\n {\n \"text\": \"{string}\"\n }\n ]\n },\n \"groundingMetadata\": {\n \"webSearchQueries\": [\n \"{string}\"\n ],\n \"searchEntryPoint\": {\n \"renderedContent\": \"{string}\"\n },\n \"groundingChunks\": [\n {\n \"web\": {\n \"uri\": \"{string}\",\n \"title\": \"{string}\",\n \"domain\": \"{string}\"\n }\n }\n ],\n \"groundingSupports\": [\n {\n \"segment\": {\n \"startIndex\": \"{number}\",\n \"endIndex\": \"{number}\",\n \"text\": \"{string}\"\n },\n \"groundingChunkIndices\": [\n \"{number}\"\n ],\n \"confidenceScores\": [\n \"{number}\"\n ]\n }\n ],\n \"retrievalMetadata\": {}\n }\n }\n ```\n \n\n Anthropic
\n Source: https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/web-search-tool\n\n Message Request:\n ```json\n {\n \"tools\": [\n {\n \"name\": \"web_search\",\n \"type\": \"web_search_20250305\",\n \"max_uses\": \"{number}\",\n \"allowed_domains\": [\"{string}\"],\n \"blocked_domains\": [\"{string}\"],\n \"user_location\": {\n \"type\": \"approximate\",\n \"city\": \"{string}\",\n \"region\": \"{string}\",\n \"country\": \"{string}\",\n \"timezone\": \"{string}\"\n }\n }\n ]\n }\n ```\n\n Tool Call Response:\n ```json\n {\n \"role\": \"assistant\",\n \"content\": [\n {\n \"type\": \"server_tool_use\",\n \"id\": \"{string}\",\n \"name\": \"web_search\",\n \"input\": {\n \"query\": \"{string}\"\n }\n },\n {\n \"type\": \"web_search_tool_result\",\n \"tool_use_id\": \"{string}\",\n \"content\": [\n {\n \"type\": \"web_search_result\",\n \"url\": \"{string}\",\n \"title\": \"{string}\",\n \"encrypted_content\": \"{string}\",\n \"page_age\": \"{string}\"\n }\n ]\n },\n {\n \"text\": \"{string}\",\n \"type\": \"text\",\n \"citations\": [\n {\n \"type\": \"web_search_result_location\",\n \"url\": \"{string}\",\n \"title\": \"{string}\",\n \"encrypted_index\": \"{string}\",\n \"cited_text\": \"{string}\"\n }\n ]\n }\n ]\n }\n ```\n \n\n#### Commonalities\n\n- **Tool-Based Activation**: Providers define web search as a tool (e.g., `web_search`, `bing_grounding`, `googleSearch`), typically within a `tools` array, allowing standardized activation of search capabilities.\n- **Query Input**: Requests support passing a search query (e.g., via `input`, `content`, or `query`), enabling a unified interface for initiating searches.\n- **Result Annotations**: Responses include search results with metadata like `url`, `title`, and sometimes `confidenceScores` or `citations`, which can be abstracted for consistent result presentation.\n- **Grounding Metadata**: Most providers include grounding metadata (e.g., `groundingMetadata`, `annotations`), facilitating traceability and validation of search results.\n\n
\n\n#### Remote MCP Servers\n\n OpenAI Responses API
\n Source: https://platform.openai.com/docs/guides/tools-remote-mcp\n\n Message Request:\n ```json\n {\n \"tools\": [\n { \n \"type\": \"mcp\",\n \"server_label\": \"{string}\",\n \"server_url\": \"{string}\",\n \"require_approval\": \"{string}\"\n }\n ]\n }\n ```\n\n Tool Call Response:\n ```json\n {\n \"output\": [\n {\n \"id\": \"{string}\",\n \"type\": \"mcp_list_tools\",\n \"server_label\": \"{string}\",\n \"tools\": [\n {\n \"name\": \"{string}\",\n \"input_schema\": \"{JSON Schema object}\"\n }\n ]\n },\n {\n \"id\": \"{string}\",\n \"type\": \"mcp_call\",\n \"approval_request_id\": \"{string}\",\n \"arguments\": \"{JSON string}\",\n \"error\": \"{string}\",\n \"name\": \"{string}\",\n \"output\": \"{string}\",\n \"server_label\": \"{string}\"\n }\n ]\n }\n ```\n \n\n Google
\n Source: https://google.github.io/adk-docs/tools/mcp-tools/#using-mcp-tools-in-your-own-agent-out-of-adk-web\n\n ```python\n async def get_agent_async():\n toolset = MCPToolset(\n tool_filter=['read_file', 'list_directory'] # Optional: filter specific tools\n connection_params=SseServerParams(url=\"http://remote-server:port/path\", headers={...})\n )\n\n # Use in an agent\n root_agent = LlmAgent(\n model='model', # Adjust model name if needed based on availability\n name='agent_name',\n instruction='agent_instructions',\n tools=[toolset], # Provide the MCP tools to the ADK agent\n )\n return root_agent, toolset\n ```\n \n\n Anthropic
\n Source: https://docs.anthropic.com/en/docs/agents-and-tools/mcp-connector\n\n Message Request:\n ```json\n {\n \"messages\": [\n {\n \"role\": \"user\", \n \"content\": \"{string}\"\n }\n ],\n \"mcp_servers\": [\n {\n \"type\": \"url\",\n \"url\": \"{string}\",\n \"name\": \"{string}\",\n \"tool_configuration\": {\n \"enabled\": true,\n \"allowed_tools\": [\"{string}\"]\n },\n \"authorization_token\": \"{string}\"\n }\n ]\n }\n ```\n\n Tool Use Response:\n ```json\n {\n \"type\": \"mcp_tool_use\",\n \"id\": \"{string}\",\n \"name\": \"{string}\",\n \"server_name\": \"{string}\",\n \"input\": { \"param1\": \"{object}\", \"param2\": \"{object}\" }\n }\n ```\n\n Tool Result Response:\n ```json\n {\n \"type\": \"mcp_tool_result\",\n \"tool_use_id\": \"{string}\",\n \"is_error\": \"{boolean}\",\n \"content\": [\n {\n \"type\": \"text\",\n \"text\": \"{string}\"\n }\n ]\n }\n ```\n \n\n#### Commonalities\n\n- **Server Configuration**: Providers specify remote servers via URL and metadata (e.g., `server_url`, `url`, `name`), enabling a standardized way to connect to external MCP services.\n- **Tool Integration**: MCP tools are integrated into the `tools` or `mcp_servers` array, allowing agents to interact with remote tools in a consistent manner.\n- **Input/Output Structure**: Requests and responses include structured input (e.g., `input`, `arguments`) and output (e.g., `output`, `content`), supporting abstraction for tool execution workflows.\n- **Authorization Support**: Most providers include mechanisms for authentication (e.g., `authorization_token`, `headers`), which can be abstracted for secure communication with remote servers.\n\n
\n\n#### Computer Use\n\n OpenAI Responses API
\n Source: https://platform.openai.com/docs/guides/tools-computer-use\n\n Message Request:\n ```json\n {\n \"tools\": [\n {\n \"type\": \"computer_use_preview\",\n \"display_width\": \"{number}\",\n \"display_height\": \"{number}\",\n \"environment\": \"{browser | mac | windows | ubuntu}\"\n }\n ]\n }\n ```\n\n Tool Call Response:\n ```json\n {\n \"output\": [\n {\n \"type\": \"reasoning\",\n \"id\": \"{string}\",\n \"summary\": [\n {\n \"type\": \"summary_text\",\n \"text\": \"{string}\"\n }\n ]\n },\n {\n \"type\": \"computer_call\",\n \"id\": \"{string}\",\n \"call_id\": \"{string}\",\n \"action\": {\n \"type\": \"{click | double_click | drag | keypress | move | screenshot | scroll | type | wait}\",\n // Other properties are associated with specific action type.\n },\n \"pending_safety_checks\": [],\n \"status\": \"{in_progress | completed | incomplete}\"\n }\n ]\n }\n ```\n \n\n Amazon Bedrock Agents
\n Source: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent_CreateAgentActionGroup.html#API_agent_CreateAgentActionGroup_RequestSyntax
\n Source: https://docs.aws.amazon.com/bedrock/latest/userguide/agent-computer-use-handle-tools.html\n\n CreateAgentActionGroup Request:\n ```json\n {\n \"actionGroupName\": \"{string}\",\n \"parentActionGroupSignature\": \"ANTHROPIC.Computer\",\n \"actionGroupState\": \"ENABLED\"\n }\n ```\n\n Tool Call Response:\n ```json\n {\n \"returnControl\": {\n \"invocationId\": \"{string}\",\n \"invocationInputs\": [\n {\n \"functionInvocationInput\": {\n \"actionGroup\": \"{string}\",\n \"actionInvocationType\": \"RESULT\",\n \"agentId\": \"{string}\",\n \"function\": \"{string}\",\n \"parameters\": [\n {\n \"name\": \"{string}\",\n \"type\": \"string\",\n \"value\": \"{string}\"\n }\n ]\n }\n }\n ]\n }\n }\n ```\n \n\n Anthropic
\n Source: https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/computer-use-tool\n\n Message Request:\n ```json\n {\n \"tools\": [\n {\n \"type\": \"computer_20250124\",\n \"name\": \"computer\",\n \"display_width_px\": \"{number}\",\n \"display_height_px\": \"{number}\",\n \"display_number\": \"{number}\"\n },\n ]\n }\n ```\n\n Tool Call Response:\n ```json\n {\n \"role\": \"assistant\",\n \"content\": [\n {\n \"type\": \"tool_use\",\n \"id\": \"{string}\",\n \"name\": \"{string}\",\n \"input\": \"{object}\"\n }\n ]\n }\n ```\n \n\n#### Commonalities\n\n- **Tool Type Definition**: Providers define a computer use tool (e.g., `computer_use_preview`, `computer_20250124`, `ANTHROPIC.Computer`) within the `tools` array, indicating support for computer interaction capabilities.\n- **Action Specification**: Responses include actions (e.g., `click`, `keypress`, `type`) with associated parameters, enabling standardized interaction with computer environments.\n- **Environment Configuration**: Requests allow specifying the environment (e.g., `browser`, `windows`, `display_width`), which can be abstracted for cross-platform compatibility.\n- **Status Tracking**: Responses include status indicators (e.g., `status`, `pending_safety_checks`), facilitating consistent monitoring of computer use tasks.\n\n
\n\n#### OpenAPI Spec Tool\n\n Azure AI Foundry Agent Service
\n Source: https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/openapi-spec-samples?pivots=rest-api
\n Source: https://learn.microsoft.com/en-us/rest/api/aifoundry/aiagents/run-steps/get-run-step?view=rest-aifoundry-aiagents-v1&tabs=HTTP#runstepopenapitoolcall\n\n Message Request:\n ```json\n {\n \"tools\": [\n { \n \"type\": \"openapi\",\n \"openapi\": {\n \"description\": \"{string}\",\n \"name\": \"{string}\",\n \"auth\": {\n \"type\": \"{string}\"\n },\n \"spec\": \"{OpenAPI specification object}\"\n }\n }\n ]\n }\n ```\n\n Tool Call Response:\n ```json\n {\n \"tool_calls\": [\n {\n \"id\": \"{string}\",\n \"type\": \"openapi\",\n \"openapi\": {} // From documentation: Reserved for future use\n }\n ]\n }\n ```\n \n\n Amazon Bedrock Agents
\n Source: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent_CreateAgentActionGroup.html#API_agent_CreateAgentActionGroup_RequestSyntax\n\n CreateAgentActionGroup Request:\n ```json\n {\n \"apiSchema\": {\n \"payload\": \"{JSON or YAML OpenAPI specification string}\"\n }\n }\n ```\n\n Tool Call Response:\n ```json\n {\n \"invocationInputs\": [\n {\n \"apiInvocationInput\": {\n \"actionGroup\": \"{string}\",\n \"apiPath\": \"{string}\",\n \"httpMethod\": \"{string}\",\n \"parameters\": [\n {\n \"name\": \"{string}\",\n \"type\": \"{string}\",\n \"value\": \"{string}\"\n }\n ]\n }\n }\n ]\n }\n ```\n \n\n#### Commonalities\n\n- **OpenAPI Specification**: Both providers support defining tools using OpenAPI specifications, either as a JSON/YAML payload or a structured `spec` object, enabling standardized API integration.\n- **Tool Type Identification**: The tool is identified as `openapi` or via an `apiSchema`, providing a clear entry point for OpenAPI-based tool usage.\n- **Parameter Handling**: Responses include parameters (e.g., `parameters`, `apiPath`, `httpMethod`) for API invocation, which can be abstracted for unified API call execution.\n\n
\n\n#### Stateful Functions\n\n Azure AI Foundry Agent Service
\n Source: https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/azure-functions-samples?pivots=rest\n\n Message Request:\n ```json\n {\n \"tools\": [\n { \n \"type\": \"azure_function\",\n \"azure_function\": {\n \"function\": {\n \"name\": \"{string}\",\n \"description\": \"{string}\",\n \"parameters\": \"{JSON Schema object}\"\n },\n \"input_binding\": {\n \"type\": \"storage_queue\",\n \"storage_queue\": {\n \"queue_service_endpoint\": \"{string}\",\n \"queue_name\": \"{string}\"\n }\n },\n \"output_binding\": {\n \"type\": \"storage_queue\",\n \"storage_queue\": {\n \"queue_service_endpoint\": \"{string}\",\n \"queue_name\": \"{string}\"\n }\n }\n }\n }\n ]\n }\n ```\n\n Tool Call Response: Not specified in the documentation.\n \n\n Amazon Bedrock Agents
\n Source: https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent_CreateAgentActionGroup.html#API_agent_CreateAgentActionGroup_RequestSyntax\n\n CreateAgentActionGroup Request:\n ```json\n {\n \"apiSchema\": {\n \"payload\": \"{JSON or YAML OpenAPI specification string}\"\n }\n }\n ```\n\n Tool Call Response:\n ```json\n {\n \"invocationInputs\": [\n {\n \"apiInvocationInput\": {\n \"actionGroup\": \"{string}\",\n \"apiPath\": \"{string}\",\n \"httpMethod\": \"{string}\",\n \"parameters\": [\n {\n \"name\": \"{string}\",\n \"type\": \"{string}\",\n \"value\": \"{string}\"\n }\n ]\n }\n }\n ]\n }\n ```\n \n\n#### Commonalities\n\n- **API-Driven Interaction**: Both providers use API-based structures (e.g., `apiSchema`, `azure_function`) to define stateful functions, enabling integration with external services.\n- **Parameter Specification**: Requests include parameter definitions (e.g., `parameters`, `JSON Schema object`), supporting standardized input handling.\n\n
\n\n#### Text Editor\n\n Anthropic
\n Source: https://docs.anthropic.com/en/docs/agents-and-tools/tool-use/text-editor-tool\n\n Message Request:\n ```json\n {\n \"tools\": [\n {\n \"type\": \"text_editor_20250429\",\n \"name\": \"str_replace_based_edit_tool\"\n }\n ]\n }\n ```\n\n Tool Call Response:\n ```json\n {\n \"role\": \"assistant\",\n \"content\": [\n {\n \"type\": \"tool_use\",\n \"id\": \"{string}\",\n \"name\": \"str_replace_based_edit_tool\",\n \"input\": {\n \"command\": \"{string}\",\n \"path\": \"{string}\"\n }\n }\n ]\n }\n ```\n \n\n
\n\n#### Microsoft Fabric\n\n Azure AI Foundry Agent Service
\n Source: https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/fabric?pivots=rest\n\n Message Request:\n ```json\n {\n \"tools\": [\n { \n \"type\": \"fabric_dataagent\",\n \"fabric_dataagent\": {\n \"connections\": [\n {\n \"connection_id\": \"{string}\"\n }\n ]\n }\n }\n ]\n }\n ```\n\n Tool Call Response: Not specified in the documentation.\n \n\n
\n\n#### Image Generation\n\n OpenAI Responses API
\n Source: https://platform.openai.com/docs/guides/tools-image-generation\n\n Message Request:\n ```json\n {\n \"tools\": [\n {\n \"type\": \"image_generation\"\n }\n ]\n }\n ```\n\n Tool Call Response:\n ```json\n {\n \"output\": [\n {\n \"type\": \"image_generation_call\",\n \"id\": \"{string}\",\n \"result\": \"{Base64 string}\",\n \"status\": \"{string}\"\n }\n ]\n }\n ```\n \n\n
\n\n## Decision Outcome\n\nTBD.\n"
},
{
"path": "docs/decisions/0003-agent-opentelemetry-instrumentation.md",
"content": "---\nstatus: proposed\ncontact: rogerbarreto\ndate: 2025-07-14\ndeciders: stephentoub, markwallace-microsoft, rogerbarreto, westey-m\ninformed: {}\n---\n\n# Agent OpenTelemetry Instrumentation\n\n## Context and Problem Statement\n\nCurrently, the Agent Framework lacks comprehensive observability and telemetry capabilities, making it difficult for developers to monitor agent performance, track usage patterns, debug issues, and gain insights into agent behavior in production environments. While the underlying ChatClient implementations may have their own telemetry, there is no standardized way to capture agent-specific metrics and traces that provide visibility into agent operations, token usage, response times, and error patterns at the agent abstraction level.\n\n## Decision Drivers\n\n- **Compliance**: The implementation should adhere to established OpenTelemetry semantic conventions for agents, ensuring consistency and interoperability with existing telemetry systems.\n- **Observability Requirements**: Developers need comprehensive telemetry to monitor agent performance, track usage patterns, and debug issues in production environments.\n- **Standardization**: The solution must follow established OpenTelemetry semantic conventions and integrate seamlessly with existing .NET telemetry infrastructure.\n- **Microsoft.Extensions.AI Alignment**: The implementation should follow the exact patterns and conventions established by Microsoft.Extensions.AI's OpenTelemetry instrumentation.\n- **Non-Intrusive Design**: Telemetry should be optional and not impact the core agent functionality or performance when disabled.\n- **Agent-Level Insights**: The telemetry should capture agent-specific operations without duplicating underlying ChatClient telemetry.\n- **Extensibility**: The solution should support future enhancements and additional telemetry scenarios.\n\n## Considered Options\n\n### Option 1: Direct Integration into Core Agent Classes\n\nEmbed OpenTelemetry instrumentation directly into the base `Agent` class and `ChatClientAgent` implementations.\n\n#### Pros\n- Automatic telemetry for all agent implementations\n- No additional wrapper classes needed\n- Consistent telemetry across all agents\n\n#### Cons\n- Violates single responsibility principle\n- Increases complexity of core agent classes\n- Makes telemetry mandatory rather than optional\n- Harder to test and maintain\n- Couples telemetry concerns with business logic\n\n### Option 2: Aspect-Oriented Programming (AOP) Approach\n\nUse interceptors or AOP frameworks to inject telemetry behavior into agent methods.\n\n#### Pros\n- Clean separation of concerns\n- Non-intrusive to existing code\n- Can be applied selectively\n\n#### Cons\n- Adds complexity with AOP framework dependencies\n- Runtime overhead for interception\n- Harder to debug and understand\n- Not consistent with Microsoft.Extensions.AI patterns\n\n### Option 3: OpenTelemetryAgent Wrapper Pattern\n\nCreate a delegating `OpenTelemetryAgent` wrapper class that implements the `Agent` interface and wraps any existing agent with telemetry instrumentation, following the exact pattern of Microsoft.Extensions.AI's `OpenTelemetryChatClient`.\n\n#### Pros\n- Follows established Microsoft.Extensions.AI patterns exactly\n- Clean separation of concerns\n- Optional and non-intrusive\n- Easy to test and maintain\n- Consistent with .NET telemetry conventions\n- Supports any agent implementation\n- Provides agent-level telemetry without duplicating ChatClient telemetry\n\n#### Cons\n- Requires explicit wrapping of agents\n- Additional object allocation for wrapper\n\n## Decision Outcome\n\nChosen option: \"OpenTelemetryAgent Wrapper Pattern\", because it follows the established Microsoft.Extensions.AI patterns exactly, provides clean separation of concerns, maintains optional telemetry, and offers the best balance of functionality, maintainability, and consistency with existing .NET telemetry infrastructure.\n\n### Implementation Details\n\nThe implementation includes:\n\n1. **OpenTelemetryAgent Wrapper Class**: A delegating agent that wraps any `Agent` implementation with telemetry instrumentation\n2. **AgentOpenTelemetryConsts**: Comprehensive constants for telemetry attribute names and metric definitions\n3. **Extension Methods**: `.WithOpenTelemetry()` extension method for easy agent wrapping\n4. **Comprehensive Test Suite**: Full test coverage following Microsoft.Extensions.AI testing patterns\n\n### Telemetry Data Captured\n\n**Activities/Spans:**\n- `agent.operation.name` (agent.run, agent.run_streaming)\n- `agent.request.id`, `agent.request.name`, `agent.request.instructions`\n- `agent.request.message_count`, `agent.request.thread_id`\n- `agent.response.id`, `agent.response.message_count`, `agent.response.finish_reason`\n- `agent.usage.input_tokens`, `agent.usage.output_tokens`\n- Error information and activity status codes\n\n**Metrics:**\n- Operation duration histogram with proper buckets\n- Token usage histogram (input/output tokens)\n- Request count counter\n- All metrics tagged with operation type and agent name\n\n### Consequences\n\n- **Good**: Provides comprehensive agent-level observability following established patterns\n- **Good**: Non-intrusive and optional implementation that doesn't affect core functionality\n- **Good**: Consistent with Microsoft.Extensions.AI telemetry conventions\n- **Good**: Easy to integrate with existing OpenTelemetry infrastructure\n- **Good**: Supports debugging, monitoring, and performance analysis\n- **Neutral**: Requires explicit wrapping of agents with `.WithOpenTelemetry()`\n- **Neutral**: Additional object allocation for telemetry wrapper\n\n## Validation\n\nThe implementation is validated through:\n\n1. **Comprehensive Unit Tests**: 16 test methods covering all scenarios including success, error, streaming, and edge cases\n2. **Integration Testing**: Step05 telemetry sample demonstrating real-world usage\n3. **Pattern Compliance**: Exact adherence to Microsoft.Extensions.AI OpenTelemetry patterns\n4. **Semantic Convention Compliance**: Follows OpenTelemetry semantic conventions for telemetry data\n\n## More Information\n\n### Usage Example\n\n```csharp\n// Create TracerProvider\nusing var tracerProvider = Sdk.CreateTracerProviderBuilder()\n .AddSource(AgentOpenTelemetryConsts.DefaultSourceName)\n .AddConsoleExporter()\n .Build();\n\n// Create and wrap agent with telemetry\nvar baseAgent = new ChatClientAgent(chatClient, options);\nusing var telemetryAgent = baseAgent.WithOpenTelemetry();\n\n// Use agent normally - telemetry is captured automatically\nvar response = await telemetryAgent.RunAsync(messages);\n```\n\n### Relationship to Microsoft.Extensions.AI\n\nThis implementation follows the exact patterns established by Microsoft.Extensions.AI's OpenTelemetry instrumentation, ensuring consistency across the AI ecosystem and leveraging proven patterns for telemetry integration.\n"
},
{
"path": "docs/decisions/0004-foundry-sdk-extensions.md",
"content": "---\n# These are optional elements. Feel free to remove any of them.\nstatus: proposed\ncontact: markwallace-microsoft\ndate: 2025-08-06\ndeciders: markwallace-microsoft, westey-m, quibitron, trrwilson\nconsulted: \ninformed: \n---\n\n# `Azure.AI.Agents.Persistent` package Extensions Methods for Agent Framework\n\n## Context and Problem Statement\n\nTo align the `Azure.AI.Agents.Persistent` package and Agent Framework a set of extensions methods have been created which allow a developer to create or retrieve an `AIAgent` using the `PersistentAgentsClient`.\nThe purpose of this ADR is to decide where these extension methods should live.\n\n## Decision Drivers\n\n- Provide the optimum experience for developers.\n- Avoid adding additional dependencies to the `Azure.AI.Agents.Persistent` package (and not in the future)\n\n## Considered Options\n\n- Add the extension methods to the `Azure.AI.Agents.Persistent` package and change it's dependencies\n- Add the extension methods to the `Azure.AI.Agents.Persistent` package without changing it's dependencies\n- Add the extension methods to a `Microsoft.Extensions.AI.Azure` package\n\n\n### Add the extension methods to the `Azure.AI.Agents.Persistent` package and change it's dependencies\n\n- `Azure.AI.Agents.Persistent` would depend on `Microsoft.Extensions.AI` instead of `Microsoft.Extensions.AI.Abstractions`\n\n- Good because, extension methods are in the `Azure.AI.Agents.Persistent` package and can be easily kept up-to-date\n- Good because, developers don't need to explicitly depend on a new package to get Agent Framework functionality\n- Bad because, it introduces additional dependencies which would possibly grow overtime\n\n\n### - Add the extension methods to the `Azure.AI.Agents.Persistent` package without changing it's dependencies\n\n- `Azure.AI.Agents.Persistent` would depend on `Microsoft.Extensions.AI.Abstractions` (as it currently does)\n- `ChatClientAgent` and `FunctionInvokingChatClient` would move to `Microsoft.Extensions.AI.Abstractions`\n\n- Good because, extension methods are in the `Azure.AI.Agents.Persistent` package and can be easily kept up-to-date\n- Good because, developers don't need to explicitly depend on a new package to get Agent Framework functionality\n- Good because, it introduces minimal additional dependencies\n- Bad because, it adds additional dependencies to `Microsoft.Extensions.AI.Abstractions` and these additional dependencies add up as transitive to `Azure`.AI.Agents.Persistent`\n\n\n### Add the extension methods to a `Microsoft.Extensions.AI.Azure` package\n\n- Introduce a new package called `Microsoft.Extensions.AI.Azure` where the extension methods would live\n- `Azure.AI.Agents.Persistent` does not change\n\n- Good because, it introduces no additional dependencies to `Azure.AI.Agents.Persistent` package\n- Bad because, extension methods are not in the `Azure.AI.Agents.Persistent` package and cannot be easily kept up-to-date\n- Bad because, developers need to explicitly depend on a new package to get Agent Framework functionality\n\n## Decision Outcome\n\nChosen option: \"Add the extension methods to a `Microsoft.Extensions.AI.Azure` package\", because\nit introduces no additional dependencies to `Azure.AI.Agents.Persistent` package.\n"
},
{
"path": "docs/decisions/0005-python-naming-conventions.md",
"content": "---\nstatus: accepted\ncontact: eavanvalkenburg\ndate: 2025-09-04\ndeciders: markwallace-microsoft, dmytrostruk, peterychang, ekzhu, sphenry\nconsulted: taochenosu, alliscode, moonbox3, johanste\n---\n\n# Python naming conventions and renames (ADR)\n\n## Context and Problem Statement\n\nThe project has a public .NET surface and a Python surface. During a cross-language alignment effort the community proposed renames to make the Python surface more idiomatic while preserving discoverability and mapping to the .NET names. This ADR captures the final naming decisions (or the proposed ones), the rationale, and the alternatives considered and rejected.\n\n## Decision drivers\n\n- Follow Python naming conventions (PEP 8) where appropriate (snake_case for functions and module-level variables, PascalCase for classes).\n- Preserve conceptual parity with .NET names to make it easy for developers reading both surfaces to correlate types and behaviors.\n- Avoid ambiguous or overloaded names in Python that could conflict with stdlib, common third-party packages, or existing package/module names.\n- Prefer clarity and discoverability in the public API surface over strict symmetry with .NET when Python conventions conflict.\n- Minimize churn and migration burden for existing Python users where backwards compatibility is feasible.\n\n## Principles applied\n\n- Map .NET PascalCase class names to PascalCase Python classes when they represent types.\n- Map .NET method/field names that are camelCase to snake_case in Python where they will be used as functions or module-level attributes.\n- When a .NET name is an acronym or initialism, use Python-friendly casing (e.g., `Http` -> `HTTP` in classes, but acronyms in function names should be lowercased per PEP 8 where sensible).\n- Avoid names that shadow common stdlib modules (e.g., `logging`, `asyncio`) or widely used third-party modules.\n- When multiple reasonable Python names exist, prefer the one that communicates intent most clearly to Python users, and record rejected alternatives in the table with justification.\n\n## Renaming table\n\nThe table below represents the majority of the naming changes discussed in issue #506. Each row has:\n- Original and/or .NET name — the canonical name used in dotnet or earlier Python variants.\n- New name — the chosen Python name.\n- Status — accepted if the new name differs from the original, rejected if unchanged.\n- Reasoning — short rationale why the new name was chosen.\n- Rejected alternatives — other candidate new names that were considered and rejected; include the rejected 'new name' values and the reason each was rejected.\n\n| Original and/or .NET name | New name (Python) | Status | Reasoning | Rejected alternatives (as \"new name\" + reason rejected) |\n|---|---|---|---|---|\n| AIAgent | AgentProtocol | accepted | The AI prefix is meaningless in the context of the Agent Framework, and the `protocol` suffix makes it very clear that this is a protocol, and not a concrete agent implementation. | - AgentLike, not seen in many other places, but was a frontrunner.
- Agent, as too generic.
- BaseAgent/AbstractAgent, it is not a base/ABC class and should not be treated as such.
|\n| ChatClientAgent | ChatAgent | accepted | Type name is shorter, while it is still clear that a ChatClient is used, also by virtue of the first parameter for initialization. | Agent, as too generic. |\n| ChatClient/IChatClient (in dotnet) | ChatClientProtocol | accepted | Keeping this protocol in sync with the AgentProtocol naming. | Similar as AgentProtocol. |\n| ChatClientBase | BaseChatClient | accepted | Following convention, serves as base class so, should be named accordingly. | None |\n| AITool | ToolProtocol | accepted | In line with other protocols. | Tool, too generic. |\n| AIToolBase | BaseTool | accepted | More descriptive than just Tool, while still concise. | AbstractTool/BaseTool, it is not an abstract/base class and should not be treated as such. |\n| ChatRole | Role | accepted | More concise while still clear in context. | None |\n| ChatFinishReason | FinishReason | accepted | More concise while still clear in context. | None |\n| AIContent | BaseContent | accepted | More accurate as it serves as the base class for all content types. | Content, too generic. |\n| AIContents | Contents | accepted | This is the annotated typing object that is the union of all concrete content types, so plural makes sense and since this is used as a type hint, the generic nature of the name is acceptable. | None |\n| AIAnnotations | Annotations | accepted | In sync with contents | None |\n| AIAnnotation | BaseAnnotation | accepted | In sync with contents | None |\n| *Mcp* & *Http* | *MCP* & *HTTP* | accepted | Acronyms should be uppercased in class names, according to PEP 8. | None |\n| `agent.run_streaming` | `agent.run_stream` | accepted | Shorter and more closely aligns with AutoGen and Semantic Kernel names for the same methods. | None |\n| `workflow.run_streaming` | `workflow.run_stream` | accepted | In sync with `agent.run_stream` and shorter and more closely aligns with AutoGen and Semantic Kernel names for the same methods. | None |\n| AgentResponse & AgentResponseUpdate | AgentResponse & AgentResponseUpdate | rejected | Rejected, because it is the response to a run invocation and AgentResponse is too generic. | None |\n| *Content | * | rejected | Rejected other content type renames (removing `Content` suffix) because it would reduce clarity and discoverability. | Item was also considered, but rejected as it is very similar to Content, but would be inconsistent with dotnet. |\n| ChatResponse & ChatResponseUpdate | Response & ResponseUpdate | rejected | Rejected, because Response is too generic. | None |\n\n## Naming guidance\nIn general Python tends to prefer shorter names, while .NET tends to prefer more descriptive names. The table above captures the specific renames agreed upon, but in general the following guidelines were applied:\n- Use [PEP 8](https://peps.python.org/pep-0008/) for generic naming conventions (snake_case for functions and module-level variables, PascalCase for classes).\n\nWhen mapping .NET names to Python:\n- Remove `AI` prefix when appropriate, as it is often redundant in the context of an AI SDK.\n- Remove `Chat` prefix when the context is clear (e.g., Role and FinishReason).\n- Use `Protocol` suffix for interfaces/protocols to clarify their purpose.\n- Use `Base` prefix for base classes that are not abstract but serve as a common ancestor for internal implementations.\n- When readability improves while it is still easy to understand what it does and how it maps to the .NET name, prefer the shorter name.\n"
},
{
"path": "docs/decisions/0006-userapproval.md",
"content": "---\n# These are optional elements. Feel free to remove any of them.\nstatus: accepted\ncontact: westey-m\ndate: 2025-09-12 {YYYY-MM-DD when the decision was last updated}\ndeciders: sergeymenshykh, markwallace-microsoft, rogerbarreto, dmytrostruk, westey-m, eavanvalkenburg, stephentoub, peterychang\nconsulted: \ninformed: \n---\n\n# Agent User Approvals Content Types and FunctionCall approvals Design\n\n## Context and Problem Statement\n\nWhen agents are operating on behalf of a user, there may be cases where the agent requires user approval to continue an operation.\nThis is complicated by the fact that an agent may be remote and the user may not immediately be available to provide the approval.\n\nInference services are also increasingly supporting built-in tools or service side MCP invocation, which may require user approval before the tool can be invoked.\n\nThis document aims to provide options and capture the decision on how to model this user approval interaction with the agent caller.\n\nSee various features that would need to be supported via this type of mechanism, plus how various other frameworks support this:\n\n- Also see [dotnet issue 6492](https://github.com/dotnet/extensions/issues/6492), which discusses the need for a similar pattern in the context of MCP approvals.\n- Also see [the openai human-in-the-loop guide](https://openai.github.io/openai-agents-js/guides/human-in-the-loop/#approval-requests).\n- Also see [the openai MCP guide](https://openai.github.io/openai-agents-js/guides/mcp/#optional-approval-flow).\n- Also see [MCP Approval Requests from OpenAI](https://platform.openai.com/docs/guides/tools-remote-mcp#approvals).\n- Also see [Azure AI Foundry MCP Approvals](https://learn.microsoft.com/en-us/azure/ai-foundry/agents/how-to/tools/model-context-protocol-samples?pivots=rest#submit-your-approval).\n- Also see [MCP Elicitation requests](https://modelcontextprotocol.io/specification/draft/client/elicitation)\n\n## Decision Drivers\n\n- Agents should encapsulate their internal logic and not leak it to the caller.\n- We need to support approvals for local actions as well as remote actions.\n- We need to support approvals for service-side tool use, such as remote MCP tool invocations\n- We should consider how other user input requests will be modeled, so that we can have a consistent approach for user input requests and approvals.\n\n## Considered Options\n\n### 1. Return a FunctionCallContent to the agent caller, that it executes\n\nThis introduces a manual function calling element to agents, where the caller of the agent is expected to invoke the function if the user approves it.\n\nThis approach is problematic for a number of reasons:\n\n- This may not work for remote agents (e.g. via A2A), where the function that the agent wants to call does not reside on the caller's machine.\n- The main value prop of an agent is to encapsulate the internal logic of the agent, but this leaks that logic to the caller, requiring the caller to know how to invoke the agent's function calls.\n- Inference services are introducing their own approval content types for server side tool or function invocation, and will not be addressed by this approach.\n\n### 2. Introduce an ApprovalCallback in AgentRunOptions and ChatOptions\n\nThis approach allows a caller to provide a callback that the agent can invoke when it requires user approval.\n\nThis approach is easy to use when the user and agent are in the same application context, such as a desktop application, where the application can show the approval request to the user and get their response from the callback before continuing the agent run.\n\nThis approach does not work well for cases where the agent is hosted in a remote service, and where there is no user available to provide the approval in the same application context.\nFor cases like this, the agent needs to be suspended, and a network response must be sent to the client app. After the user provides their approval, the client app must call the service that hosts the agent again, with the user's decision, and the agent needs to be resumed. However, with a callback, the agent is deep in the call stack and cannot be suspended or resumed like this.\n\n```csharp\nclass AgentRunOptions\n{\n public Func>? ApprovalCallback { get; set; }\n}\n\nagent.RunAsync(\"Please book me a flight for Friday to Paris.\", thread, new AgentRunOptions\n{\n ApprovalCallback = async (approvalRequest) =>\n {\n // Show the approval request to the user in the appropriate format.\n // The user can then approve or reject the request.\n // The optional FunctionCallContent can be used to show the user what function the agent wants to call with the parameter set:\n // approvalRequest.FunctionCall?.Arguments.\n\n // If the user approves:\n return true;\n }\n});\n```\n\n### 3. Introduce new ApprovalRequestContent and ApprovalResponseContent types\n\nThe agent would return an `ApprovalRequestContent` to the caller, which would then be responsible for getting approval from the user in whatever way is appropriate for the application.\nThe caller would then invoke the agent again with an `ApprovalResponseContent` to the agent containing the user decision.\n\nWhen an agent returns an `ApprovalRequestContent`, the run is finished for the time being, and to continue, the agent must be invoked again with an `ApprovalResponseContent` on the same thread as the original request. This doesn't of course have to be the exact same thread object, but it should have the equivalent contents as the original thread, since the agent would have stored the `ApprovalRequestContent` in its thread state.\n\nThe `ApprovalRequestContent` could contain an optional `FunctionCallContent` if the approval is for a function call, along with any additional information that the agent wants to provide to the user to help them make a decision.\n\nIt is up to the agent to decide when and if a user approval is required, and therefore when to return an `ApprovalRequestContent`.\n\n`ApprovalRequestContent` and `ApprovalResponseContent` will not necessarily always map to a supported content type for the underlying service or agent thread storage.\nSpecifically, when we are deciding in the IChatClient stack to ask for approval from the user, for a function call, this does not mean that the underlying ai service or\nservice side thread type (where applicable) supports the concept of a function call approval request. While we can store the approval requests and response in local\nthreads, service managed threads won't necessarily support this. For service managed threads, there will therefore be no long term record of the approval request in the chat history.\nWe should however log approvals so that there is a trace of this for debugging and auditing purposes.\n\nSuggested Types:\n\n```csharp\nclass ApprovalRequestContent : AIContent\n{\n // An ID to uniquely identify the approval request/response pair.\n public string Id { get; set; }\n\n // An optional user targeted message to explain what needs to be approved.\n public string? Text { get; set; }\n\n // Optional: If the approval is for a function call, this will contain the function call content.\n public FunctionCallContent? FunctionCall { get; set; }\n\n public ApprovalResponseContent CreateApproval()\n {\n return new ApprovalResponseContent\n {\n Id = this.Id,\n Approved = true,\n FunctionCall = this.FunctionCall\n };\n }\n\n public ApprovalResponseContent CreateRejection()\n {\n return new ApprovalResponseContent\n {\n Id = this.Id,\n Approved = false,\n FunctionCall = this.FunctionCall\n };\n }\n}\n\nclass ApprovalResponseContent : AIContent\n{\n // An ID to uniquely identify the approval request/response pair.\n public string Id { get; set; }\n\n // Indicates whether the user approved the request.\n public bool Approved { get; set; }\n\n // Optional: If the approval is for a function call, this will contain the function call content.\n public FunctionCallContent? FunctionCall { get; set; }\n}\n\nvar response = await agent.RunAsync(\"Please book me a flight for Friday to Paris.\", thread);\nwhile (response.ApprovalRequests.Count > 0)\n{\n List messages = new List();\n foreach (var approvalRequest in response.ApprovalRequests)\n {\n // Show the approval request to the user in the appropriate format.\n // The user can then approve or reject the request.\n // The optional FunctionCallContent can be used to show the user what function the agent wants to call with the parameter set:\n // approvalRequest.FunctionCall?.Arguments.\n // The Text property of the ApprovalRequestContent can also be used to show the user any additional textual context about the request.\n \n // If the user approves:\n messages.Add(new ChatMessage(ChatRole.User, [approvalRequest.CreateApproval()]));\n }\n\n // Get the next response from the agent.\n response = await agent.RunAsync(messages, thread);\n}\n\nclass AgentResponse\n{\n ...\n\n // A new property on AgentResponse to aggregate the ApprovalRequestContent items from\n // the response messages (Similar to the Text property).\n public IEnumerable ApprovalRequests { get; set; }\n\n ...\n}\n```\n\n### 4. Introduce new Container UserInputRequestContent and UserInputResponseContent types\n\nThis approach is similar to the `ApprovalRequestContent` and `ApprovalResponseContent` types, but is more generic and can be used for any type of user input request, not just approvals.\n\nThere is some ambiguity with this approach. When using an LLM based agent the LLM may return a text response about missing user input.\nE.g the LLM may need to invoke a function but the user did not supply all necessary information to fill out all arguments.\nTypically an LLM would just respond with a text message asking the user for the missing information.\nIn this case, the message is not distinguishable from any other result message, and therefore cannot be returned to the caller as a `UserInputRequestContent`, even though it is conceptually a type of unstructured user input request. Ultimately our types are modeled to make it easy for callers to decide on the right way to represent this to users. E.g. is it just a regular message to show to users, or do we need a special UX for it.\n\nSuggested Types:\n\n```csharp\nclass UserInputRequestContent : AIContent\n{\n // An ID to uniquely identify the approval request/response pair.\n public string ApprovalId { get; set; }\n\n // DecisionTarget could contain:\n // FunctionCallContent: The function call that the agent wants to invoke.\n // TextContent: Text that describes the question for that the user should answer.\n object? DecisionTarget { get; set; } // Anything else the user may need to make a decision about.\n\n // Possible InputFormat subclasses:\n // SchemaInputFormat: Contains a schema for the user input.\n // ApprovalInputFormat: Indicates that the user needs to approve something.\n // FreeformTextInputFormat: Indicates that the user can provide freeform text input.\n // Other formats can be added as needed, e.g. cards when using activity protocol.\n public InputFormat InputFormat { get; set; } // How the user should provide input (e.g., form, options, etc.).\n}\n\nclass UserInputResponseContent : AIContent\n{\n // An ID to uniquely identify the approval request/response pair.\n public string ApprovalId { get; set; }\n\n // Possible UserInputResult subclasses:\n // SchemaInputResult: Contains the structured data provided by the user.\n // ApprovalResult: Contains a bool with approved / rejected.\n // FreeformTextResult: Contains the freeform text input provided by the user.\n public UserInputResult Result { get; set; } // The user input.\n\n public object? DecisionTarget { get; set; } // A copy of the DecisionTarget from the UserInputRequestContent, if applicable.\n}\n\nvar response = await agent.RunAsync(\"Please book me a flight for Friday to Paris.\", thread);\nwhile (response.UserInputRequests.Any())\n{\n List messages = new List();\n foreach (var userInputRequest in response.UserInputRequests)\n {\n // Show the user input request to the user in the appropriate format.\n // The DecisionTarget can be used to show the user what function the agent wants to call with the parameter set.\n // The InputFormat property can be used to determine the type of UX when allowing users to provide input.\n\n if (userInputRequest.InputFormat is ApprovalInputFormat approvalInputFormat)\n {\n // Here we need to show the user an approval request.\n // We can use the DecisionTarget to show e.g. the function call that the agent wants to invoke.\n // The user can then approve or reject the request.\n \n // If the user approves:\n var approvalMessage = new ChatMessage(ChatRole.User, new UserInputResponseContent { \n ApprovalId = userInputRequest.ApprovalId,\n Result = new ApprovalResult { Approved = true },\n DecisionTarget = userInputRequest.DecisionTarget\n });\n messages.Add(approvalMessage);\n }\n else\n {\n throw new NotSupportedException(\"Unsupported InputFormat type.\");\n }\n }\n\n // Get the next response from the agent.\n response = await agent.RunAsync(messages, thread);\n}\n\nclass AgentResponse\n{\n ...\n\n // A new property on AgentResponse to aggregate the UserInputRequestContent items from\n // the response messages (Similar to the Text property).\n public IReadOnlyList UserInputRequests { get; set; }\n\n ...\n}\n```\n\n### 5. Introduce new Base UserInputRequestContent and UserInputResponseContent types\n\nThis approach is similar to option 4, but the `UserInputRequestContent` and `UserInputResponseContent` types are base classes rather than generic container types.\n\nSuggested Types:\n\n```csharp\nclass UserInputRequestContent : AIContent\n{\n // An ID to uniquely identify the approval request/response pair.\n public string Id { get; set; }\n}\n\nclass UserInputResponseContent : AIContent\n{\n // An ID to uniquely identify the approval request/response pair.\n public string Id { get; set; }\n}\n\n// -----------------------------------\n// Used for approving a function call.\nclass FunctionApprovalRequestContent : UserInputRequestContent\n{\n // Contains the function call that the agent wants to invoke.\n public FunctionCallContent FunctionCall { get; set; }\n\n public ApprovalResponseContent CreateApproval()\n {\n return new ApprovalResponseContent\n {\n Id = this.Id,\n Approved = true,\n FunctionCall = this.FunctionCall\n };\n }\n\n public ApprovalResponseContent CreateRejection()\n {\n return new ApprovalResponseContent\n {\n Id = this.Id,\n Approved = false,\n FunctionCall = this.FunctionCall\n };\n }\n}\nclass FunctionApprovalResponseContent : UserInputResponseContent\n{\n // Indicates whether the user approved the request.\n public bool Approved { get; set; }\n\n // Contains the function call that the agent wants to invoke.\n public FunctionCallContent FunctionCall { get; set; }\n}\n\n// --------------------------------------------------\n// Used for approving a request described using text.\nclass TextApprovalRequestContent : UserInputRequestContent\n{\n // A user targeted message to explain what needs to be approved.\n public string Text { get; set; }\n}\nclass TextApprovalResponseContent : UserInputResponseContent\n{\n // Indicates whether the user approved the request.\n public bool Approved { get; set; }\n}\n\n// ------------------------------------------------\n// Used for providing input in a structured format.\nclass StructuredDataInputRequestContent : UserInputRequestContent\n{\n // A user targeted message to explain what is being requested.\n public string? Text { get; set; }\n\n // Contains the schema for the user input.\n public JsonElement Schema { get; set; }\n}\nclass StructuredDataInputResponseContent : UserInputResponseContent\n{\n // Contains the structured data provided by the user.\n public JsonElement StructuredData { get; set; }\n}\n\nvar response = await agent.RunAsync(\"Please book me a flight for Friday to Paris.\", thread);\nwhile (response.UserInputRequests.Any())\n{\n List messages = new List();\n foreach (var userInputRequest in response.UserInputRequests)\n {\n if (userInputRequest is FunctionApprovalRequestContent approvalRequest)\n {\n // Here we need to show the user an approval request.\n // We can use the FunctionCall property to show e.g. the function call that the agent wants to invoke.\n // If the user approves:\n messages.Add(new ChatMessage(ChatRole.User, approvalRequest.CreateApproval()));\n }\n }\n\n // Get the next response from the agent.\n response = await agent.RunAsync(messages, thread);\n}\n\nclass AgentResponse\n{\n ...\n\n // A new property on AgentResponse to aggregate the UserInputRequestContent items from\n // the response messages (Similar to the Text property).\n public IEnumerable UserInputRequests { get; set; }\n\n ...\n}\n```\n\n## Decision Outcome\n\nChosen option 5.\n\n## Appendices\n\n### ChatClientAgent Approval Process Flow\n\n1. User passes a User message to the agent with a request.\n1. Agent calls IChatClient with any functions registered on the agent.\n (IChatClient has FunctionInvokingChatClient)\n1. Model responds with FunctionCallContent indicating function calls required.\n1. FunctionInvokingChatClient decorator identifies any function calls that require user approval and returns an FunctionApprovalRequestContent.\n (If there are multiple parallel function calls, all function calls will be returned as FunctionApprovalRequestContent even if only some require approval.)\n1. Agent updates the thread with the FunctionApprovalRequestContent (or this may have already been done by a service threaded agent).\n1. Agent returns the FunctionApprovalRequestContent to the caller which shows it to the user in the appropriate format.\n1. User (via caller) invokes the agent again with FunctionApprovalResponseContent.\n1. Agent adds the FunctionApprovalResponseContent to the thread.\n1. Agent calls IChatClient with the provided FunctionApprovalResponseContent.\n1. Agent invokes IChatClient with FunctionApprovalResponseContent and the FunctionInvokingChatClient decorator identifies the response as an approval for the function call.\n Any rejected approvals are converted to FunctionResultContent with a message indicating that the function invocation was denied.\n Any approved approvals are executed by the FunctionInvokingChatClient decorator.\n1. FunctionInvokingChatClient decorator passes the FunctionCallContent and FunctionResultContent for the approved and rejected function calls to the model.\n1. Model responds with the result.\n1. FunctionInvokingChatClient returns the FunctionCallContent, FunctionResultContent, and the result message to the agent.\n1. Agent responds to caller with the same messages and updates the thread with these as well.\n\n### CustomAgent Approval Process Flow\n\n1. User passes a User message to the agent with a request.\n1. Agent adds this message to the thread.\n1. Agent executes various steps.\n1. Agent encounters a step for which it requires user input to continue.\n1. Agent responds with an UserInputRequestContent and also adds it to its thread.\n1. User (via caller) invokes the agent again with UserInputResponseContent.\n1. Agent adds the UserInputResponseContent to the thread.\n1. Agent responds to caller with result message and thread is updated with the result message.\n\n### Sequence Diagram: FunctionInvokingChatClient with built in Approval Generation\n\nThis is a ChatClient Approval Stack option has been proven to work via a proof of concept implementation.\n\n```mermaid\n---\ntitle: Multiple Functions with partial approval\n---\n\nsequenceDiagram\n note right of Developer: Developer asks question with two functions.\n Developer->>+FunctionInvokingChatClient: What is the special soup today?
[GetMenu, GetSpecials]\n FunctionInvokingChatClient->>+ResponseChatClient: What is the special soup today?
[GetMenu, GetSpecials]\n\n ResponseChatClient-->>-FunctionInvokingChatClient: [FunctionCallContent(GetMenu)],
[FunctionCallContent(GetSpecials)]\n note right of FunctionInvokingChatClient: FICC turns FunctionCallContent
into FunctionApprovalRequestContent\n FunctionInvokingChatClient->>+Developer: [FunctionApprovalRequestContent(GetMenu)]
[FunctionApprovalRequestContent(GetSpecials)]\n\n note right of Developer:Developer asks user for approval\n Developer->>+FunctionInvokingChatClient: [FunctionApprovalRequestContent(GetMenu, approved=false)]
[FunctionApprovalRequestContent(GetSpecials, approved=true)]\n note right of FunctionInvokingChatClient:FunctionInvokingChatClient executes the approved
function and generates a failed FunctionResultContent
for the rejected one, before invoking the model again.\n FunctionInvokingChatClient->>+ResponseChatClient: What is the special soup today?
[FunctionCallContent(GetMenu)],
[FunctionCallContent(GetSpecials)],
[FunctionResultContent(GetMenu, Function invocation denied\")]
[FunctionResultContent(GetSpecials, \"Special Soup: Clam Chowder...\")]\n\n ResponseChatClient-->>-FunctionInvokingChatClient: [TextContent(\"The specials soup is...\")]\n FunctionInvokingChatClient->>+Developer: [FunctionCallContent(GetMenu)],
[FunctionCallContent(GetSpecials)],
[FunctionResultContent(GetMenu, Function invocation denied\")]
[FunctionResultContent(GetSpecials, \"Special Soup: Clam Chowder...\")]
[TextContent(\"The specials soup is...\")]\n```\n\n### Sequence Diagram: Post FunctionInvokingChatClient ApprovalGeneratingChatClient - Multiple function calls with partial approval\n\nThis is a discarded ChatClient Approval Stack option, but is included here for reference.\n\n```mermaid\n---\ntitle: Multiple Functions with partial approval\n---\n\nsequenceDiagram\n note right of Developer: Developer asks question with two functions.\n Developer->>+FunctionInvokingChatClient: What is the special soup today? [GetMenu, GetSpecials]\n FunctionInvokingChatClient->>+ApprovalGeneratingChatClient: What is the special soup today? [GetMenu, GetSpecials]\n ApprovalGeneratingChatClient->>+ResponseChatClient: What is the special soup today? [GetMenu, GetSpecials]\n\n ResponseChatClient-->>-ApprovalGeneratingChatClient: [FunctionCallContent(GetMenu)],
[FunctionCallContent(GetSpecials)]\n ApprovalGeneratingChatClient-->>-FunctionInvokingChatClient: [FunctionApprovalRequestContent(GetMenu)],
[FunctionApprovalRequestContent(GetSpecials)]\n FunctionInvokingChatClient-->>-Developer: [FunctionApprovalRequestContent(GetMenu)]
[FunctionApprovalRequestContent(GetSpecials)]\n\n note right of Developer: Developer approves one function call and rejects the other.\n Developer->>+FunctionInvokingChatClient: [FunctionApprovalResponseContent(GetMenu, approved=true)]
[FunctionApprovalResponseContent(GetSpecials, approved=false)]\n FunctionInvokingChatClient->>+ApprovalGeneratingChatClient: [FunctionApprovalResponseContent(GetMenu, approved=true)]
[FunctionApprovalResponseContent(GetSpecials, approved=false)]\n\n note right of FunctionInvokingChatClient: ApprovalGeneratingChatClient only returns FunctionCallContent
for approved FunctionApprovalResponseContent.\n ApprovalGeneratingChatClient-->>-FunctionInvokingChatClient: [FunctionCallContent(GetMenu)]\n note right of FunctionInvokingChatClient: FunctionInvokingChatClient has to also include all
FunctionApprovalResponseContent in the new downstream request.\n FunctionInvokingChatClient->>+ApprovalGeneratingChatClient: [FunctionResultContent(GetMenu, \"mains.... deserts...\")]
[FunctionApprovalResponseContent(GetMenu, approved=true)]
[FunctionApprovalResponseContent(GetSpecials, approved=false)]\n\n note right of ApprovalGeneratingChatClient: ApprovalGeneratingChatClient now throws away
approvals for executed functions, and creates
failed FunctionResultContent for denied function calls.\n ApprovalGeneratingChatClient->>+ResponseChatClient: [FunctionResultContent(GetMenu, \"mains.... deserts...\")]
[FunctionResultContent(GetSpecials, \"Function invocation denied\")]\n```\n\n### Sequence Diagram: Pre FunctionInvokingChatClient ApprovalGeneratingChatClient - Multiple function calls with partial approval\n\nThis is a discarded ChatClient Approval Stack option, but is included here for reference.\n\nIt doesn't work for the scenario where we have multiple function calls for the same function in serial with different arguments.\n\nFlow:\n\n- AGCC turns AIFunctions into AIFunctionDefinitions (not invocable) and FICC ignores these.\n- We get back a FunctionCall for one of these and it gets approved.\n- We invoke the FICC again, this time with an AIFunction.\n- We call the service with the FCC and FRC.\n- We get back a new Function call for the same function again with different arguments.\n- Since we were passed an AIFunction instead of an AIFunctionDefinition, we now incorrectly execute this FC without approval.\n\n```mermaid\n---\ntitle: Multiple Functions with partial approval\n---\n\nsequenceDiagram\n note right of Developer: Developer asks question with two functions.\n Developer->>+ApprovalGeneratingChatClient: What is the special soup today? [GetMenu, GetSpecials]\n note right of ApprovalGeneratingChatClient: AGCC marks functions as not-invocable\n ApprovalGeneratingChatClient->>+FunctionInvokingChatClient: What is the special soup today?
[GetMenu(invocable=false)]
[GetSpecials(invocable=false)]\n FunctionInvokingChatClient->>+ResponseChatClient: What is the special soup today?
[GetMenu(invocable=false)]
[GetSpecials(invocable=false)]\n\n ResponseChatClient-->>-FunctionInvokingChatClient: [FunctionCallContent(GetMenu)],
[FunctionCallContent(GetSpecials)]\n note right of FunctionInvokingChatClient: FICC doesn't invoke functions since they are not invocable.\n FunctionInvokingChatClient-->>-ApprovalGeneratingChatClient: [FunctionCallContent(GetMenu)],
[FunctionCallContent(GetSpecials)]\n note right of ApprovalGeneratingChatClient: AGCC turns functions into approval requests\n ApprovalGeneratingChatClient-->>-Developer: [FunctionApprovalRequestContent(GetMenu)]
[FunctionApprovalRequestContent(GetSpecials)]\n\n note right of Developer: Developer approves one function call and rejects the other.\n Developer->>+ApprovalGeneratingChatClient: [FunctionApprovalResponseContent(GetMenu, approved=true)]
[FunctionApprovalResponseContent(GetSpecials, approved=false)]\n note right of ApprovalGeneratingChatClient: AGCC turns turns approval requests
into FCC or failed function calls\n ApprovalGeneratingChatClient->>+FunctionInvokingChatClient: [FunctionCallContent(GetMenu)]
[FunctionCallContent(GetSpecials)
[FunctionResultContent(GetSpecials, \"Function invocation denied\"))]\n note right of FunctionInvokingChatClient: FICC invokes GetMenu since it's the only remaining one.\n FunctionInvokingChatClient->>+ResponseChatClient: [FunctionCallContent(GetMenu)]
[FunctionResultContent(GetMenu, \"mains.... deserts...\")]
[FunctionCallContent(GetSpecials)
[FunctionResultContent(GetSpecials, \"Function invocation denied\"))]\n\n ResponseChatClient-->>-FunctionInvokingChatClient: [FunctionCallContent(GetMenu)]
[FunctionResultContent(GetMenu, \"mains.... deserts...\")]
[FunctionCallContent(GetSpecials)
[FunctionResultContent(GetSpecials, \"Function invocation denied\"))]
[TextContent(\"The specials soup is...\")]\n FunctionInvokingChatClient-->>-ApprovalGeneratingChatClient: [FunctionCallContent(GetMenu)]
[FunctionResultContent(GetMenu, \"mains.... deserts...\")]
[FunctionCallContent(GetSpecials)
[FunctionResultContent(GetSpecials, \"Function invocation denied\"))]
[TextContent(\"The specials soup is...\")]\n ApprovalGeneratingChatClient-->>-Developer: [FunctionCallContent(GetMenu)]
[FunctionResultContent(GetMenu, \"mains.... deserts...\")]
[FunctionCallContent(GetSpecials)
[FunctionResultContent(GetSpecials, \"Function invocation denied\"))]
[TextContent(\"The specials soup is...\")]\n```\n"
},
{
"path": "docs/decisions/0007-agent-filtering-middleware.md",
"content": "---\nstatus: proposed\ncontact: rogerbarreto\ndate: 2025-09-15\ndeciders: markwallace-microsoft, rogerbarreto, westey-m, dmytrostruk, sergeymenshykh\ninformed: {}\n---\n\n# Agent Filtering Middleware Design\n\n## Context and Problem Statement\n\nThe current Agent Framework lacks a standardized, extensible mechanism for intercepting and processing agent execution. Developers need the ability to add custom filters/middleware to intercept and modify agent behavior at various stages of the execution pipeline. While the framework has basic agent abstractions with `RunAsync` and `RunStreamingAsync` methods, and standards like approval workflows, there is no middleware that allows developers to intercept and modify agent behavior at different agent execution contexts.\n\nThe challenge is to design an architecture that supports:\n- Multiple execution contexts (invocation, function calls, approval requests, error handling)\n- Support for both streaming and non-streaming scenarios\n- Dependency injection friendly setup\n\n## Decision Drivers\n\n- Agents should be able to intercept and modify agent behavior at various stages of the execution pipeline.\n- The design should be simple and intuitive for developers to understand and use.\n- The design should be extensible to support new execution contexts and scenarios.\n- The design should support both manual and dependency injection configuration.\n- The design should allow flexible custom behaviors provided by enough context information.\n- The design should be exception friendly and allow clear error handling and recovery mechanisms.\n\n## Other AI Agent Framework Analysis\n\nThis section provides an analysis of how other major AI agent frameworks handle filtering, middleware, hooks, or similar interception capabilities. The goal is to identify ubiquitous language, design patterns, and approaches that could inform our Agent Middleware design also providing valuable insights into achieving a more idiomatic designs.\n\n### Overview Comparison Table\n\n| Provider | Language | Supports (Y/N) | Naming | TL;DR Observation |\n|---------------------------|----------|----------------|---------------------------------|------------------------|\n| LangChain (Python) | Python | Y (read) | Callbacks (BaseCallbackHandler) | Uses observer pattern with event methods for interception (e.g., on_chain_start); supports agent actions and errors; handlers can read inputs/outputs and modification is limited to the parameters or by raising exceptions to influence flow. [Details](#langchain) |\n| LangChain (JS) | JS | Y (read/write) | Callbacks (BaseCallbackHandler) | Similar observer pattern to Python, with event methods adapted for JS async handling; supports chain/agent interception; handlers can read inputs/outputs and modify metadata or raise exceptions to influence flow. [Details](#langchain) |\n| LangChain | JS/Python/TS | Y (read/write) | Middleware | Middleware concept was recently introduced in LangChain 1.0 alpha; [Details](https://blog.langchain.com/agent-middleware/) |\n| LangGraph | Python | Y (read/write) | Hooks/Callbacks (inherited from LangChain) | Event-driven with runtime handlers; integrates callbacks for observability in graphs; inherits LangChain's ability to read/modify metadata or interrupt execution. [Details](#langgraph) |\n| AutoGen (Python) | Python | Y (read/write) | Reply Functions (register_reply) | Reply functions intercept and process messages; middleware-like for agent replies; can directly modify messages or replies before continuing. [Details](#autogen) |\n| AutoGen (C#) | C# | Y (read/write) | Middleware (MiddlewareAgent) | Decorator/wrapper with middleware delegates for message modification; delegates can read and alter message content or options. [Details](#autogen) |\n| Semantic Kernel (C#) | C# | Y (read/write) | Filters (IFunctionInvocationFilter, etc.) | Interface-based middleware pattern for function/prompt interception; filters can read and modify context, arguments, or results. [Details](#semantic-kernel) |\n| Semantic Kernel (Python) | Python | Y (read/write) | Filters (add_filter, @kernel.filter decorator) | Function and decorator-based for interception; no explicit interfaces like C#, focuses on async functions for filters; can read and modify context/arguments/results. [Details](#semantic-kernel) |\n| CrewAI | Python | Y (read) | Events/Callbacks (BaseEventListener) | Event-driven orchestration with listeners for workflows; listeners can observe events (e.g., read source/event data) but are primarily for logging/reactions without direct modification of workflow state. [Details](#crewai) |\n| LlamaIndex | Python | Y (read) | Callbacks (CallbackManager) | Observer pattern with event methods for queries and tools; handlers can observe events/payloads (e.g., read prompts/responses) but are designed for debugging/tracing without modifying execution context. [Details](#llamaindex) |\n| Haystack | Python | N (Pipeline-based interception) | N/A (Pipeline Components/Routers) | Relies on modular pipelines for implicit interception but lacks explicit middleware/filters; custom components can read/write data flow via routing/transformations, but this is compositional rather than hook-based interception. [Details](#haystack) |\n| OpenAI Swarm | Python | N | N/A | No explicit middleware/filters; interception requires custom wrappers or manual handling (e.g., function decorators, client subclassing), lacking native framework support for built-in components to accept such modifications. [Details](#openai-swarm) |\n| Atomic Agents | Python | N | N/A (Composable Components) | No explicit middleware/filters; modularity allows composable units but no dedicated interception hooks or callbacks for custom reading/modification mid-execution. [Details](#atomic-agents) |\n| Smolagents (Hugging Face)| Python | N | N/A | No explicit support; focuses on simple agent building without interception mechanisms or hooks for reading/modifying execution. [Details](#smolagents-hugging-face) |\n| Phidata (Agno) | Python | N | N/A | No explicit middleware/filters; agents use tools/memory but no interception hooks for custom reading/modification of calls. [Details](#phidata-agno) |\n| PromptFlow (Microsoft) | Python | N (Tracing only) | Tracing | Supports tracing for LLM interactions, acting as callbacks for debugging/iteration; tracing is read-only for observability/telemetry without options to modify context or intercept calls beyond logging. [Details](#promptflow-microsoft) |\n| n8n | JS/TS | Y (read/write) | Callbacks (inherited from LangChain) | AI Agent node uses LangChain under the hood, inheriting callbacks for observability; supports reading/modifying metadata or interrupting flow as in LangChain. [Details](#n8n) |\n\n## Considered Options\n\n### Option 1: Semantic Kernel Approach\n\nSimilar to the Semantic Kernel kernel filters this option involves exposing different interface and properties for each specialized filter.\n\n```csharp\n\nvar services = new ServiceCollection();\nservices.AddSingleton();\nservices.AddSingleton();\n\n// Using DI\nvar agent = new MyAgent(services.BuildServiceProvider());\n\n// Manual\nvar agent = new MyAgent();\nagent.RunFilters.Add(new MyAgentRunFilter());\nagent.FunctionCallFilters.Add(new MyAgentFunctionCallFilter());\n\npublic class MyAgentRunFilter : IAgentRunFilter\n{\n public async Task OnRunAsync(AgentRunContext context, Func next, CancellationToken cancellationToken = default)\n {\n // Pre-run logic\n\n await next(context);\n\n // Post-run logic\n }\n}\n\npublic interface IAgentRunFilter\n{\n Task OnRunAsync(AgentRunContext context, Func next, CancellationToken cancellationToken = default);\n}\n\npublic interface IAgentFunctionCallFilter\n{\n Task OnFunctionCallAsync(AgentFunctionCallContext context, Func next, CancellationToken cancellationToken = default);\n}\n\npublic class AIAgent\n{\n private readonly AgentFilterProcessor _filterProcessor;\n\n public AIAgent(AgentFilterProcessor? filterProcessor = null)\n {\n _filterProcessor = filterProcessor ?? new AgentFilterProcessor();\n }\n\n public AIAgent(IServiceProvider serviceProvider)\n {\n _filterProcessor = serviceProvider.GetService() ?? new AgentFilterProcessor();\n\n // Auto-register filters from DI\n var filters = serviceProvider.GetServices();\n foreach (var filter in filters)\n {\n _filterProcessor.AddFilter(filter);\n }\n }\n\n public async Task RunAsync(\n IReadOnlyCollection messages,\n AgentThread? thread = null,\n AgentRunOptions? options = null,\n CancellationToken cancellationToken = default)\n {\n var context = new AgentRunContext(messages, thread, options);\n\n // Process through filter pipeline using the same pattern as Semantic Kernel\n await _filterProcessor.ProcessAsync(context, async ctx =>\n {\n // Core agent logic - implement actual agent execution here\n var response = await this.ExecuteCoreLogicAsync(ctx.Messages, ctx.Thread, ctx.Options, cancellationToken);\n ctx.Response = response;\n }, cancellationToken);\n\n // Extract the response from the context\n return context.Response ?? throw new InvalidOperationException(\"Agent execution did not produce a response\");\n }\n\n protected abstract Task ExecuteCoreLogicAsync(\n IReadOnlyCollection messages,\n AgentThread? thread,\n AgentRunOptions? options,\n CancellationToken cancellationToken);\n}\n\n```\n#### Pros\n- Clean separation of concerns\n- Follows established patterns in Semantic Kernel and easy migration path\n- No resistance or complaints from the community when used in Semantic Kernel\n- Composable and reusable filter components\n\n#### Cons\n- Adding more filters may require adding more properties to the agent class.\n- Filters are not always used, and adding this responsibility to the `AIAgent` abstraction level, may be an overkill.\n\n### Option 2: Agent Filter Decorator Pattern\n\nSimilar to the `OpenTelemetryAgent` and the `DelegatingChatClient` in `Microsoft.Extensions.AI`, this option involves creating decorator agents that wrap the inner agent and allow interception of method calls. The current POC implementation demonstrates two approaches:\n\n#### 2a. Direct Decorator Implementation (GuardrailCallbackAgent)\n\n```csharp\n// Current POC implementation from samples\nvar agent = persistentAgentsClient.CreateAIAgent(model).AsBuilder()\n .Use((innerAgent) => new GuardrailCallbackAgent(innerAgent)) // Decoration based agent run handling\n .Use(async (context, next) => // Context based handling\n {\n // Guardrail: Filter input messages for PII\n context.Messages = context.Messages.Select(m => new ChatMessage(m.Role, FilterPii(m.Text))).ToList();\n Console.WriteLine($\"Pii Middleware - Filtered messages: {new ChatResponse(context.Messages).Text}\");\n\n await next(context);\n\n if (!context.IsStreaming)\n {\n // Guardrail: Filter output messages for PII\n context.Messages = context.Messages.Select(m => new ChatMessage(m.Role, FilterPii(m.Text))).ToList();\n }\n else\n {\n context.SetRawResponse(StreamingPiiDetectionAsync(context.RunStreamingResponse!));\n }\n })\n .Build();\n\n// Direct decorator implementation\ninternal sealed class GuardrailCallbackAgent : DelegatingAIAgent\n{\n private readonly string[] _forbiddenKeywords = { \"harmful\", \"illegal\", \"violence\" };\n\n public GuardrailCallbackAgent(AIAgent innerAgent) : base(innerAgent) { }\n\n public override async Task RunAsync(IEnumerable messages, AgentThread? thread = null, AgentRunOptions? options = null, CancellationToken cancellationToken = default)\n {\n var filteredMessages = this.FilterMessages(messages);\n Console.WriteLine($\"Guardrail Middleware - Filtered messages: {new ChatResponse(filteredMessages).Text}\");\n\n var response = await this.InnerAgent.RunAsync(filteredMessages, thread, options, cancellationToken);\n\n response.Messages = response.Messages.Select(m => new ChatMessage(m.Role, this.FilterContent(m.Text))).ToList();\n\n return response;\n }\n\n public override async IAsyncEnumerable RunStreamingAsync(IEnumerable messages, AgentThread? thread = null, AgentRunOptions? options = null, [EnumeratorCancellation] CancellationToken cancellationToken = default)\n {\n var filteredMessages = this.FilterMessages(messages);\n await foreach (var update in this.InnerAgent.RunStreamingAsync(filteredMessages, thread, options, cancellationToken))\n {\n if (update.Text != null)\n {\n yield return new AgentResponseUpdate(update.Role, this.FilterContent(update.Text));\n }\n else\n {\n yield return update;\n }\n }\n }\n\n private List FilterMessages(IEnumerable messages)\n {\n return messages.Select(m => new ChatMessage(m.Role, this.FilterContent(m.Text))).ToList();\n }\n\n private string FilterContent(string content)\n {\n foreach (var keyword in this._forbiddenKeywords)\n {\n if (content.Contains(keyword, StringComparison.OrdinalIgnoreCase))\n {\n return \"[REDACTED: Forbidden content]\";\n }\n }\n return content;\n }\n}\n```\n\n#### 2b. Context-Based Middleware (RunningCallbackHandlerAgent)\n\nThe POC also includes a context-based approach using `RunningCallbackHandlerAgent` that wraps the agent and provides a context object for middleware processing:\n\n```csharp\n// Internal implementation that supports the .Use() pattern\ninternal sealed class RunningCallbackHandlerAgent : DelegatingAIAgent\n{\n private readonly Func, Task> _func;\n\n internal RunningCallbackHandlerAgent(AIAgent innerAgent, Func, Task> func) : base(innerAgent)\n {\n this._func = func;\n }\n\n public override async Task RunAsync(IEnumerable messages, AgentThread? thread = null, AgentRunOptions? options = null, CancellationToken cancellationToken = default)\n {\n var context = new AgentInvokeCallbackContext(this, messages, thread, options, isStreaming: false, cancellationToken);\n\n async Task CoreLogicAsync(AgentInvokeCallbackContext ctx)\n {\n var response = await this.InnerAgent.RunAsync(ctx.Messages, ctx.Thread, ctx.Options, ctx.CancellationToken);\n ctx.SetRawResponse(response);\n }\n\n await this._func(context, CoreLogicAsync);\n\n return context.RunResponse!;\n }\n}\n```\n\n#### 2c. Function Invocation Filtering \n\nThe POC also demonstrates function invocation filtering using a similar decorator pattern:\n\n```csharp\n// Function invocation middleware using .Use() pattern\nvar agent = persistentAgentsClient.CreateAIAgent(model)\n .AsBuilder()\n .Use((functionInvocationContext, next, ct) =>\n {\n Console.WriteLine($\"IsStreaming: {functionInvocationContext!.IsStreaming}\");\n return next(functionInvocationContext.Arguments, ct);\n })\n .Use((functionInvocationContext, next, ct) =>\n {\n Console.WriteLine($\"City Name: {(functionInvocationContext!.Arguments.TryGetValue(\"location\", out var location) ? location : \"not provided\")}\");\n return next(functionInvocationContext.Arguments, ct);\n })\n .Build();\n```\n\nThis demonstrates that the current POC supports both agent-level and function-level filtering through consistent patterns.\n\n#### Pros\n- Clean separation of concerns\n- Follows established patterns in `Microsoft.Extensions.AI` (DelegatingChatClient, OpenTelemetryAgent)\n- Non-intrusive to existing agent implementations\n- Supports both manual and DI configuration through builder pattern\n- Context-specific processing middleware with `AgentInvokeCallbackContext`\n- Composable and reusable filter components\n- Flexible implementation allowing both direct decorators and context-based middleware\n- Seamless integration with builder pattern using `.Use()` method\n- Support for both streaming and non-streaming scenarios\n- Rich context object providing access to messages, thread, options, and response handling\n\n### Option 3: Dedicated Processor Component for Middleware\n\nThis approach involves creating a dedicated `CallbackMiddlewareProcessor` that manages collections of `ICallbackMiddleware` instances. The current POC implementation demonstrates this pattern with the `CallbackEnabledAgent` and processor architecture.\n\n#### Current POC Implementation\n\n```csharp\n// Current POC usage from samples\nvar agent = persistentAgentsClient.CreateAIAgent(model)\n .AsBuilder()\n .UseCallbacks(config =>\n {\n config.AddCallback(new PiiDetectionMiddleware());\n config.AddCallback(new GuardrailCallbackMiddleware());\n }).Build();\n\n// Middleware implementation\ninternal sealed class PiiDetectionMiddleware : CallbackMiddleware\n{\n public override async Task OnProcessAsync(AgentInvokeCallbackContext context, Func next, CancellationToken cancellationToken)\n {\n // Guardrail: Filter input messages for PII\n context.Messages = context.Messages.Select(m => new ChatMessage(m.Role, FilterPii(m.Text))).ToList();\n Console.WriteLine($\"Pii Middleware - Filtered messages: {new ChatResponse(context.Messages).Text}\");\n await next(context);\n\n if (!context.IsStreaming)\n {\n // Guardrail: Filter output messages for PII\n context.Messages = context.Messages.Select(m => new ChatMessage(m.Role, FilterPii(m.Text))).ToList();\n }\n else\n {\n context.SetRawResponse(StreamingPiiDetectionAsync(context.RunStreamingResponse!));\n }\n }\n\n private static string FilterPii(string content)\n {\n // PII detection logic...\n }\n}\n\ninternal sealed class GuardrailCallbackMiddleware : CallbackMiddleware\n{\n private readonly string[] _forbiddenKeywords = { \"harmful\", \"illegal\", \"violence\" };\n\n public override async Task OnProcessAsync(AgentInvokeCallbackContext context, Func next, CancellationToken cancellationToken)\n {\n // Guardrail: Filter input messages for forbidden content\n context.Messages = this.FilterMessages(context.Messages);\n Console.WriteLine($\"Guardrail Middleware - Filtered messages: {new ChatResponse(context.Messages).Text}\");\n\n await next(context);\n if (!context.IsStreaming)\n {\n // Guardrail: Filter output messages for forbidden content\n context.Messages = this.FilterMessages(context.Messages);\n }\n else\n {\n context.SetRawResponse(StreamingGuardRailAsync(context.RunStreamingResponse!));\n }\n }\n}\n```\n\n#### Function Invocation Filtering\n\nThe POC also demonstrates function invocation filtering using the processor pattern:\n\n```csharp\n// Processor-based function invocation middleware\nvar agent = persistentAgentsClient.CreateAIAgent(model)\n .AsBuilder()\n .UseCallbacks(config =>\n {\n config.AddCallback(new UsedApiFunctionInvocationCallback());\n config.AddCallback(new CityInformationFunctionInvocationCallback());\n }).Build();\n\ninternal sealed class UsedApiFunctionInvocationCallback : CallbackMiddleware\n{\n public override async Task OnProcessAsync(AgentFunctionInvocationCallbackContext context, Func next, CancellationToken cancellationToken)\n {\n Console.WriteLine($\"IsStreaming: {context!.IsStreaming}\");\n\n await next(context);\n }\n}\n\ninternal sealed class CityInformationFunctionInvocationCallback : CallbackMiddleware\n{\n public override async Task OnProcessAsync(AgentFunctionInvocationCallbackContext context, Func next, CancellationToken cancellationToken)\n {\n Console.WriteLine($\"City Name: {(context!.Arguments.TryGetValue(\"location\", out var location) ? location : \"not provided\")}\");\n await next(context);\n }\n}\n```\n\nThis demonstrates that the current POC supports both agent-level and function-level filtering through consistent patterns.\n\n#### Processor Implementation\n\nThe `CallbackMiddlewareProcessor` manages the filter pipeline and chain execution:\n\n```csharp\npublic sealed class CallbackMiddlewareProcessor\n{\n // For thread-safety when used as a Singleton\n private readonly ConcurrentBag _agentCallbacks = [];\n\n public CallbackMiddlewareProcessor(IEnumerable? callbacks = null)\n {\n if (callbacks is not null)\n {\n foreach (var callback in callbacks)\n {\n AddCallback(callback);\n }\n }\n }\n\n internal CallbackMiddlewareProcessor AddCallback(ICallbackMiddleware middleware)\n {\n switch (middleware)\n {\n case CallbackMiddleware:\n this._agentCallbacks.Add(middleware);\n break;\n default:\n throw new ArgumentException($\"The middleware type '{middleware.GetType().FullName}' is not supported.\", nameof(middleware));\n }\n\n return this;\n }\n\n public async Task ProcessAsync(TContext context, Func coreLogic, CancellationToken cancellationToken = default)\n where TContext : CallbackContext\n {\n var applicableCallbacks = this.GetApplicableCallbacks().ToList();\n await this.InvokeChainAsync(context, applicableCallbacks, 0, coreLogic, cancellationToken);\n }\n\n private IEnumerable GetApplicableCallbacks()\n where TContext : CallbackContext\n {\n return this._agentCallbacks.Where(callback => callback.CanProcess());\n }\n}\n```\n\n#### CallbackEnabledAgent Implementation\n\n```csharp\npublic sealed class CallbackEnabledAgent : DelegatingAIAgent\n{\n private readonly CallbackMiddlewareProcessor _callbacksProcessor;\n\n public CallbackEnabledAgent(AIAgent agent, CallbackMiddlewareProcessor? callbackMiddlewareProcessor) : base(agent)\n {\n this._callbacksProcessor = callbackMiddlewareProcessor ?? new();\n }\n\n public override async Task RunAsync(\n IEnumerable messages,\n AgentThread? thread = null,\n AgentRunOptions? options = null,\n CancellationToken cancellationToken = default)\n {\n AgentInvokeCallbackContext roamingContext = null!;\n\n async Task CoreLogic(AgentInvokeCallbackContext ctx)\n {\n roamingContext ??= ctx;\n var result = await this.InnerAgent.RunAsync(ctx.Messages, ctx.Thread, ctx.Options, ctx.CancellationToken);\n\n ctx.SetRawResponse(result);\n }\n\n await this._callbacksProcessor.ProcessAsync(\n new AgentInvokeCallbackContext(\n agent: this,\n messages: messages,\n thread,\n options,\n isStreaming: false,\n cancellationToken),\n CoreLogic,\n cancellationToken);\n\n return roamingContext.RunResponse!;\n }\n}\n```\n\n#### Pros\n- Flexibility: Use shared processor for multiple agents or create per-agent instances\n- Clean fluent configuration API with `.UseCallbacks()` builder method\n- Type-safe middleware registration with `CallbackMiddleware` base class\n- Thread-safe processor implementation using `ConcurrentBag`\n- Extensible context system with `AgentInvokeCallbackContext` providing rich execution context\n- Seamless integration with existing agent builder pattern\n- Support for both streaming and non-streaming scenarios in middleware\n- Clear separation between middleware logic and agent core functionality\n- Simplicity: Agents stay lean, middleware is externalized to processor\n- Extensibility: Add new contexts/filters without changing agent implementation\n\n#### Cons\n- Additional complexity with processor class and context management\n- Requires understanding of middleware lifecycle and context passing\n- Type switching in processor for different middleware types\n- Roaming context pattern needed to capture specialized contexts through middleware chain\n\n## APPENDIX 1: Proposed Middleware Contexts\n\nThe following context classes would be needed to support the filtering architecture:\n\n```csharp\npublic abstract class AgentContext\n{\n // For scenarios where the filter is processed by multiple agents sounds very desirable to provide access to the invoking agent\n public AIAgent Agent { get; } \n\n public AgentRunOptions? Options { get; set; } // Options are allowed to be set by filters\n\n protected AgentContext(AIAgent agent, AgentRunOptions? options)\n {\n Agent = agent;\n Options = options;\n }\n}\n\npublic class AgentRunContext : AgentContext\n{\n public IList Messages { get; set; }\n public AgentResponse? Response { get; set; }\n public AgentThread? Thread { get; }\n\n public AgentRunContext(AIAgent agent, IList messages, AgentThread? thread, AgentRunOptions? options)\n : base(agent, options)\n {\n Messages = messages;\n Thread = thread;\n }\n}\n\npublic class AgentFunctionInvocationContext : AgentToolContext\n{\n // Similar to MEAI.FunctionInvocationContext\n public AIFunction Function { get; set; }\n public AIFunctionArguments Arguments { get; set; }\n public FunctionCallContent CallContent { get; set; }\n public IList Messages { get; set; }\n public ChatOptions? Options { get; set; }\n public int Iteration { get; set; }\n public int FunctionCallIndex { get; set; }\n public int FunctionCount { get; set; }\n public bool Terminate { get; set; }\n public bool IsStreaming { get; set; }\n}\n\n```\n\n## APPENDIX 2: Setting Up Middleware Options\n\n### 1. Semantic Kernel Setup\n\nHas the benefit of clear separation of concerns, but this approach requires developers \nto manage and maintain separate collections for each filter type, increasing code complexity and maintenance overhead.\n\n```csharp\n// Use Case\nvar agent = new MyAgent();\nagent.RunFilters.Add(new MyAgentRunFilter());\nagent.RunFilters.Add(new MyMultipleFilterImplementation());\nagent.FunctionCallFilters.Add(new MyAgentFunctionCallFilter());\nagent.FunctionCallFilters.Add(new MyMultipleFilterImplementation());\nagent.AYZFilters.Add(new MyAgentAYZFilter());\nagent.AYZFilters.Add(new MyMultipleFilterImplementation());\n\n\n\n// Impl\ninterface IAgentRunFilter\n{\n Task OnRunAsync(AgentRunContext context, Func next, CancellationToken cancellationToken = default);\n}\ninterface IAgentFunctionCallFilter\n{\n Task OnFunctionCallAsync(AgentFunctionCallContext context, Func next, CancellationToken cancellationToken = default);\n}\n```\n\n#### Pros\n- Clean separation of concerns\n- Follows established patterns in Semantic Kernel and easy migration path\n- No resistance or complaints from the community when used in Semantic Kernel\n\n#### Cons\n- Adding more filters may require adding more properties to the agent/processor class.\n- Adding more filters requires bigger code changes downstream to callers.\n\n### 2. Setup with Generic Method\n\nInstead of properties, exposing as a method may be more appropriate while still maintaining those filters in separate buckets internally.\n\n```csharp\n// Use Case\nvar agent = new MyAgent();\nagent.AddFilters([new MyAgentRunFilter(), new MyMultipleFilterImplementation()]);\nagent.AddFilters([new MyAgentFunctionCallFilter(), new MyMultipleFilterImplementation()]);\nagent.AddFilters([new MyAgentAYZFilter(), new MyMultipleFilterImplementation()]);\n\n```\n\n#### Pros\n- Clean separation of concerns\n- Cleaner API for adding filters compared to option 1\n- No resistance or complaints from the community when used in Semantic Kernel\n\n#### Cons\n- Adding more filters may require adding more properties to the agent/processor class.\n- Adding more filters requires bigger code changes downstream to callers.\n\n### 3. Setup with Filter Hierarchy, Fully Generic Setup\n\nIn a more generic approach, filters can be grouped in the same bucket and processed based on the context.\nOne generic interface for all filters, with context-specific implementations. \nAllow simple grouping of filters in the same list and adding new filter types with low code-changes.\n\n```csharp\n// Use Case\nvar agent = new MyAgent();\nagent.Filters.Add(new MyAgentRunFilter());\nagent.Filters.Add(new MyAgentFunctionCallFilter());\nagent.Filters.Add(new MyAgentAYZFilter());\nagent.Filters.Add(new MyMultipleFilterImplementation());\n\n// OR Via constructor (Also DI Friendly)\nvar agent = new MyAgent(new List { \n new MyAgentRunFilter(), \n new MyAgentFunctionCallFilter(), \n new MyAgentAYZFilter(), \n new MyMultipleFilterImplementation() });\n\n// Impl\ninterface IAgentFilter\n{\n bool CanProcess(AgentContext context);\n Task OnProcessAsync(AgentContext context, Func next, CancellationToken cancellationToken = default);\n}\n\ninterface IAgentFilter : IAgentFilter where T : AgentContext\n{\n Task OnProcessAsync(T context, Func next, CancellationToken cancellationToken = default);\n}\n\nclass MySingleFilterImplementation : IAgentFilter\n{\n public bool CanProcess(AgentContext context)\n => context is AgentRunContext;\n\n public async Task OnProcessAsync(AgentContext context, Func next, CancellationToken cancellationToken = default)\n {\n Func wrappedNext = async ctx => await next(ctx);\n await OnProcessAsync((AgentRunContext)context, wrappedNext, cancellationToken);\n }\n\n public async Task OnProcessAsync(AgentRunContext context, Func next, CancellationToken cancellationToken = default)\n {\n // Pre-run logic\n await next(context);\n // Post-run logic\n }\n}\n\nclass MyMultipleFilterImplementation : IAgentFilter, IAgentFilter\n{\n public bool CanProcess(AgentContext context)\n => context is AgentRunContext or FunctionCallAgentContext;\n\n public async Task OnProcessAsync(AgentContext context, Func next, CancellationToken cancellationToken = default)\n {\n if (context is AgentRunContext runContext)\n {\n Func wrappedNext = async ctx => await next(ctx);\n await OnProcessAsync(runContext, wrappedNext, cancellationToken);\n return;\n }\n\n if (context is FunctionCallAgentContext callContext)\n {\n Func wrappedNext = async ctx => await next(ctx);\n await OnProcessAsync(callContext, wrappedNext, cancellationToken);\n return;\n }\n\n await next(context);\n }\n\n public async Task OnProcessAsync(AgentRunContext context, Func next, CancellationToken cancellationToken = default)\n {\n // Pre-run logic\n await next(context);\n // Post-run logic\n }\n\n public async Task OnProcessAsync(FunctionCallAgentContext context, Func next, CancellationToken cancellationToken = default)\n {\n // Pre-function call logic\n await next(context);\n // Post-function call logic\n }\n}\n```\n\n#### Pros\n- Simple grouping of filters in the same list, help with DI registration and filtering iteration\n- Lower maintenance and learning curve when adding new filter types\n- Can be combined with other patterns like the `AgentFilterProcessor`\n\n#### Cons\n- Less clear separation of concerns compared to dedicated filter types\n- Requires extra runtime type checking and casting for context-specific processing\n\n## Decision Outcome\n\n- **Option 2 (Decorator Pattern)** is the preferred approach for the following reasons:\n - Adding a processor pattern seems an overkill as we can achieve same results without introducing new abstractions and complexity.\n - Direct decorator on agents and tools for agent and function invocation middleware.\n - Support for Context-based middleware also leveraging closer patterns to Semantic Kernel filters.\n - Agent Builder pattern integration with `.Use()` method for fluent configuration\n\n**Key POC Insights**:\n1. Both patterns actually work\n2. The decorator pattern offers more direct control and simpler and more flexible implementation\n2. The processor seems an overkill compared to decorator as it adds more extra abstractions and complexity\n4. Function invocation filtering is supported in both patterns\n5. Streaming scenarios are well-supported in both approaches\n6. Function approval request filtering is supported in both patterns\n7. Builder pattern added as part of the POC is a must-have and mades both approaches developer-friendly\n\n## Appendix: Other AI Agent Framework Analysis Details\n\n#### LangChain\n\nLangChain uses callbacks for interception, which can be passed at runtime or during construction.\n\nNaming (Python): Callbacks (BaseCallbackHandler) \nSupports: Y (read/write) \nObservation: Uses observer pattern with event methods for interception (e.g., on_chain_start); supports agent actions and errors; handlers can read inputs/outputs and modify metadata or raise exceptions to influence flow.\n\n**Python Example:** For more details, see the official documentation: [Callbacks - Python LangChain](https://python.langchain.com/docs/concepts/callbacks/).\n\n```python\nfrom langchain_core.callbacks import BaseCallbackHandler\n\nclass MyHandler(BaseCallbackHandler):\n def on_chain_start(self, serialized, inputs, **kwargs):\n inputs['number'] += 1 # Modify inputs (write capability)\n print(\"Chain started!\")\n\nhandler = MyHandler()\n\n# Pass callback at runtime\nchain.invoke({\"number\": 25}, {\"callbacks\": [handler]})\n\n# Or at constructor time\nchain = SomeChain(callbacks=[handler])\nchain.invoke({\"number\": 25})\n```\n\nNaming (JS): Callbacks (BaseCallbackHandler) \nSupports: Y (read/write) \nObservation: Similar observer pattern to Python, with event methods adapted for JS async handling; supports chain/agent interception; handlers can read inputs/outputs and modify metadata or raise exceptions to influence flow.\n\n**JS Example:** For more details, see the official documentation: [Callbacks - LangChain.js](https://js.langchain.com/docs/concepts/callbacks/). (Adapted for async handling in JS.)\n\n```javascript\nimport { BaseCallbackHandler } from \"@langchain/core/callbacks/base\";\n\nclass MyHandler extends BaseCallbackHandler {\n name = \"my_handler\";\n\n async handleChainStart(chain, inputs) {\n inputs.number += 1; # Modify inputs (write capability)\n console.log(\"Chain started!\");\n }\n}\n\nconst handler = new MyHandler();\n\n// Pass callback at runtime\nawait chain.invoke({ number: 25 }, { callbacks: [handler] });\n\n// Or at constructor time\nconst chainWithHandler = new SomeChain({ callbacks: [handler] });\nawait chainWithHandler.invoke({ number: 25 });\n```\n\n#### LangGraph\n\nLangGraph inherits callbacks from LangChain and often uses them with handlers for observability (e.g., via Langfuse).\n\nNaming (Python): Hooks/Callbacks (inherited from LangChain) \nSupports: Y (read/write) \nObservation: Event-driven with runtime handlers; integrates callbacks for observability in graphs; inherits LangChain's ability to read/modify metadata or interrupt execution.\n\nFor more details, see the official documentation (inherited from LangChain): [Callbacks - Python LangChain](https://python.langchain.com/docs/concepts/callbacks/). Here's an example of streaming with a callback handler (Python):\n\n```python\nfrom langfuse.langchain import CallbackHandler\nfrom langchain_core.messages import HumanMessage\n\nclass MyLangfuseHandler(CallbackHandler):\n def on_chain_start(self, serialized, inputs, **kwargs):\n inputs['messages'][0].content += \" modified\" # Modify input messages (write capability)\n super().on_chain_start(serialized, inputs, **kwargs)\n\nlangfuse_handler = MyLangfuseHandler()\n\n# Stream with callback in config\nfor s in graph.stream(\n {\"messages\": [HumanMessage(content=\"What is Langfuse?\")]},\n config={\"callbacks\": [langfuse_handler]}\n):\n print(s)\n```\n\n#### AutoGen\n\nAutoGen supports middleware-like behavior in both languages.\n\nNaming (Python): Reply Functions (register_reply) \nSupports: Y (read/write) \nObservation: Reply functions intercept and process messages; middleware-like for agent replies; can directly modify messages or replies before continuing.\n\n**Python Example:** For more details, see the official documentation: [agentchat.conversable_agent | AutoGen 0.2](https://microsoft.github.io/autogen/0.2/docs/reference/agentchat/conversable_agent). Uses `register_reply` to add reply functions that intercept and process messages.\n\n```python\ndef print_messages(recipient, messages, sender, config): \n if \"callback\" in config and config[\"callback\"] is not None:\n callback = config[\"callback\"]\n callback(sender, recipient, messages[-1])\n messages[-1][\"content\"] += \" modified\" # Modify last message content (write capability)\n print(f\"Messages sent to: {recipient.name} | num messages: {len(messages)}\")\n return False, None # required to ensure the agent communication flow continues\n\nuser_proxy.register_reply(\n [autogen.Agent, None],\n reply_func=print_messages, \n config={\"callback\": None},\n)\n\nassistant.register_reply(\n [autogen.Agent, None],\n reply_func=print_messages, \n config={\"callback\": None},\n)\n```\n\nNaming (C#): Middleware (MiddlewareAgent) \nSupports: Y (read/write) \nObservation: Decorator/wrapper with middleware delegates for message modification; delegates can read and alter message content or options.\n\n**C# Example:** For more details, see the official documentation: [Use middleware in an agent - AutoGen for .NET](https://microsoft.github.io/autogen-for-net/articles/Middleware-overview.html). Registers middleware to modify messages.\n\n```csharp\n// Register middleware to modify messages\nvar middlewareAgent = new MiddlewareAgent(innerAgent: agent);\nmiddlewareAgent.Use(async (messages, options, agent, ct) =>\n{\n if (messages.Last() is TextMessage lastMessage && lastMessage.Content.Contains(\"Hello World\"))\n {\n lastMessage.Content = $\"[middleware] {lastMessage.Content}\"; # Modify message content (write capability)\n return lastMessage;\n }\n return await agent.GenerateReplyAsync(messages, options, ct);\n});\n```\n\n#### Semantic Kernel\n\nSemantic Kernel uses filters added to the kernel for interception during function invocation, prompt rendering, etc. Implementations differ by language: C# use interfaces, while Python uses functions and decorators.\n\nNaming (C#): Filters (IFunctionInvocationFilter, etc.) \nSupports: Y (read/write) \nObservation: Interface-based middleware for function/prompt interception; filters can read and modify context, arguments, or results.\n\n**C# Example:** For more details, see the official documentation: [Semantic Kernel Filters | Microsoft Learn](https://learn.microsoft.com/en-us/semantic-kernel/concepts/enterprise-readiness/filters). Adding a function invocation filter using interfaces.\n\n```csharp\nusing Microsoft.SemanticKernel;\n\nIKernelBuilder builder = Kernel.CreateBuilder();\nbuilder.Services.AddSingleton();\n\nKernel kernel = builder.Build();\n\n// Alternatively, add directly\nkernel.FunctionInvocationFilters.Add(new LoggingFilter(logger));\n\n// Define the filter\npublic sealed class LoggingFilter(ILogger logger) : IFunctionInvocationFilter\n{\n public async Task OnFunctionInvocationAsync(FunctionInvocationContext context, Func next)\n {\n context.Arguments[\"new_arg\"] = \"modified_value\"; # Modify arguments by adding a new key (write capability)\n logger.LogInformation(\"Invoking {FunctionName}\", context.Function.Name);\n await next(context);\n logger.LogInformation(\"Invoked {FunctionName}\", context.Function.Name);\n }\n}\n```\n\nNaming (Python): Filters (add_filter, @kernel.filter decorator) \nSupports: Y (read/write) \nObservation: Function and decorator-based for interception; no explicit interfaces like C#, focuses on async functions for filters; can read and modify context/arguments/results.\n\n**Python Example:** For more details, see the official documentation: [Semantic Kernel Filters | Microsoft Learn](https://learn.microsoft.com/en-us/semantic-kernel/concepts/enterprise-readiness/filters). Adding function invocation filters (one as a standalone function and one via decorator).\n\n```python\nimport logging\nfrom typing import Callable, Coroutine, Any\nfrom semantic_kernel import Kernel\nfrom semantic_kernel.filters import FilterTypes, FunctionInvocationContext\nfrom semantic_kernel.connectors.ai.open_ai import AzureChatCompletion\nfrom semantic_kernel.contents import ChatHistory\nfrom semantic_kernel.exceptions import OperationCancelledException\n\nlogger = logging.getLogger(__name__)\n\nasync def input_output_filter(\n context: FunctionInvocationContext,\n next: Callable[[FunctionInvocationContext], Coroutine[Any, Any, None]],\n) -> None:\n if context.function.plugin_name != \"chat\":\n await next(context)\n return\n try:\n user_input = input(\"User:> \")\n except (KeyboardInterrupt, EOFError) as exc:\n raise OperationCancelledException(\"User stopped the operation\") from exc\n if user_input == \"exit\":\n raise OperationCancelledException(\"User stopped the operation\")\n context.arguments[\"chat_history\"].add_user_message(user_input) # Modify arguments by adding message (write capability)\n\n await next(context)\n\n if context.result:\n logger.info(f\"Usage: {context.result.metadata.get('usage')}\")\n context.arguments[\"chat_history\"].add_message(context.result.value[0])\n print(f\"Mosscap:> {context.result!s}\")\n\nkernel = Kernel()\nkernel.add_service(AzureChatCompletion(service_id=\"chat-gpt\"))\n\n# Add filter as a standalone function\nkernel.add_filter(\"function_invocation\", input_output_filter)\n\n# Add filter via decorator\n@kernel.filter(filter_type=FilterTypes.FUNCTION_INVOCATION)\nasync def exception_catch_filter(\n context: FunctionInvocationContext, next: Coroutine[FunctionInvocationContext, Any, None]\n):\n try:\n await next(context)\n except Exception as e:\n logger.info(e)\n\n# Example invocation (assuming a \"chat\" plugin is added)\nhistory = ChatHistory()\nresult = await kernel.invoke(\n function_name=\"chat\",\n plugin_name=\"chat\",\n chat_history=history,\n)\n```\n\n#### CrewAI\n\nCrewAI uses event listeners for callbacks.\n\nNaming (Python): Events/Callbacks (BaseEventListener) \nSupports: Y (read) \nObservation: Event-driven orchestration with listeners for workflows; listeners can observe events (e.g., read source/event data) but are primarily for logging/reactions without direct modification of workflow state.\n\nFor more details, see the official documentation: [Event Listeners - CrewAI Documentation](https://docs.crewai.com/concepts/event-listener). Here's an example of setting up a custom listener (Python):\n\n```python\nfrom crewai.utilities.events import (\n CrewKickoffStartedEvent,\n BaseEventListener,\n crewai_event_bus\n)\n\nclass MyCustomListener(BaseEventListener):\n def setup_listeners(self, crewai_event_bus):\n @crewai_event_bus.on(CrewKickoffStartedEvent)\n def on_crew_started(source, event):\n print(f\"Crew '{event.crew_name}' started!\")\n\nmy_listener = MyCustomListener() # Automatically registers on init\n\n# Use in a crew\ncrew = Crew(agents=[...], tasks=[...])\n```\n\n#### LlamaIndex\n\nLlamaIndex uses callback managers with handlers.\n\nNaming (Python): Callbacks (CallbackManager, BaseCallbackHandler) \nSupports: Y (read) \nObservation: Observer pattern with event methods for queries and tools; handlers can observe events/payloads (e.g., read prompts/responses) but are designed for debugging/tracing without modifying execution context.\n\nFor more details, see the official documentation: [Callbacks - LlamaIndex](https://docs.llamaindex.ai/en/stable/module_guides/observability/callbacks/). Here's an example setup (Python):\n\n```python\nfrom llama_index.core.callbacks import CallbackManager, LlamaDebugHandler\n\ndebug_handler = LlamaDebugHandler() # Concrete handler subclassing BaseCallbackHandler\ncallback_manager = CallbackManager([debug_handler])\n\n# Assign to components, e.g., an index or query engine\nindex = VectorStoreIndex.from_documents(documents, callback_manager=callback_manager)\nquery_engine = index.as_query_engine()\nresponse = query_engine.query(\"What is this about?\")\n```\n\n#### Haystack\n\nHaystack does not support explicit middleware or filters like the others. Instead, it uses a modular pipeline architecture for interception via components (e.g., ConditionalRouter for routing based on conditions like tool calls) and observability through logging/tracing integrations (e.g., Langfuse).\n\nNaming (Python): N/A (Pipeline Components/Routers) \nSupports: N (Pipeline-based interception) \nObservation: Relies on modular pipelines for implicit interception but lacks explicit middleware/filters; custom components can read/write data flow via routing/transformations, but this is compositional rather than hook-based interception.\n\nFor more details, see the official documentation: [Pipelines - Haystack Documentation](https://docs.haystack.deepset.ai/docs/pipelines). Here's an example of pipeline-based interception with a custom collector component (Python):\n\n```python\nfrom haystack import Pipeline\nfrom haystack.components.generators.chat import OpenAIChatGenerator\nfrom haystack.components.routers import ConditionalRouter\nfrom haystack.components.tools import ToolInvoker\nfrom haystack.tools import ComponentTool\nfrom haystack.components.websearch import SerperDevWebSearch\nfrom haystack.dataclasses import ChatMessage\nfrom typing import Any, Dict, List\nfrom haystack import component\nfrom haystack.core.component.types import Variadic\n\n# Custom component to collect/observe messages (for interception/observation)\n@component()\nclass MessageCollector:\n def __init__(self):\n self._messages = []\n @component.output_types(messages=List[ChatMessage])\n def run(self, messages: Variadic[List[ChatMessage]]) -> Dict[str, Any]:\n self._messages.extend([msg for inner in messages for msg in inner])\n return {\"messages\": self._messages}\n def clear(self):\n self._messages = []\n\n# Define a tool\nweb_tool = ComponentTool(component=SerperDevWebSearch(top_k=3))\n\n# Define routes for filtering (e.g., check for tool calls)\nroutes = [\n {\n \"condition\": \"{{replies[0].tool_calls | length > 0}}\",\n \"output\": \"{{replies}}\",\n \"output_name\": \"there_are_tool_calls\",\n \"output_type\": List[ChatMessage],\n },\n {\n \"condition\": \"{{replies[0].tool_calls | length == 0}}\",\n \"output\": \"{{replies}}\",\n \"output_name\": \"final_replies\",\n \"output_type\": List[ChatMessage],\n },\n]\n\n# Build the pipeline\npipeline = Pipeline()\npipeline.add_component(\"generator\", OpenAIChatGenerator(model=\"gpt-4o-mini\"))\npipeline.add_component(\"router\", ConditionalRouter(routes=routes))\npipeline.add_component(\"tool_invoker\", ToolInvoker(tools=[web_tool]))\npipeline.add_component(\"message_collector\", MessageCollector())\n\n# Connect components (interception via routing and collection)\npipeline.connect(\"generator.replies\", \"router.replies\")\npipeline.connect(\"router.there_are_tool_calls\", \"tool_invoker.messages\")\npipeline.connect(\"tool_invoker.messages\", \"message_collector.messages\")\npipeline.connect(\"router.final_replies\", \"message_collector.messages\")\n\n# Run the pipeline (observes via collector, filters via router)\nresult = pipeline.run({\"generator\": {\"messages\": [ChatMessage.from_user(\"What's the weather in Berlin?\")]}})\nprint(result[\"message_collector\"][\"messages\"])\n```\n\n#### OpenAI Swarm\n\nOpenAI Swarm does not provide native support for middleware, filters, callbacks, or hooks. While interception can be achieved through custom implementations (e.g., function wrappers, client subclassing, or manual tool execution with `execute_tools=False`), this requires the caller to implement their own logic, which is not considered built-in framework support.\n\nNaming (Python): N/A \nSupports: N \nObservation: No explicit middleware/filters; interception requires custom wrappers or manual handling (e.g., function decorators, client subclassing), lacking native framework support for built-in components to accept such modifications.\n\nFor more details, see the official GitHub repository: [OpenAI Swarm GitHub](https://github.com/openai/swarm). No native code examples available for interception; custom approaches are possible but not framework-native.\n\n#### Atomic Agents\n\nAtomic Agents does not support explicit middleware, callbacks, hooks, or filters. Its modularity allows composable components, but no dedicated interception mechanisms are documented.\n\nNaming (Python): N/A (Composable Components) \nSupports: N \nObservation: No explicit middleware/filters; modularity allows composable units but no dedicated interception hooks or callbacks for custom reading/modification mid-execution.\n\nFor more details, see the official documentation: [Atomic Agents Docs](https://brainblend-ai.github.io/atomic-agents/). No specific code examples available for interception.\n\n#### Smolagents (Hugging Face)\n\nSmolagents does not support explicit middleware, callbacks, hooks, or filters; it focuses on simple agent building.\n\nNaming (Python): N/A \nSupports: N \nObservation: No explicit support; focuses on simple agent building without interception mechanisms or hooks for reading/modifying execution.\n\nFor more details, see the official documentation: [Smolagents Docs](https://huggingface.co/docs/smolagents/en/index). No specific code examples available for interception.\n\n#### Phidata (Agno)\n\nPhidata (Agno) does not support explicit middleware, callbacks, hooks, or filters; agents rely on tools and memory.\n\nNaming (Python): N/A \nSupports: N \nObservation: No explicit middleware/filters; agents use tools/memory but no interception hooks for custom reading/modification of calls.\n\nFor more details, see the official documentation: [Phidata Docs](https://docs.phidata.com/). No specific code examples available for interception.\n\n#### PromptFlow (Microsoft)\n\nPromptFlow supports tracing for LLM interactions, which acts like callbacks for debugging and iteration.\n\nNaming (Python): Tracing \nSupports: N (Tracing only) \nObservation: Supports tracing for LLM interactions, acting as callbacks for debugging/iteration; tracing is read-only for observability/telemetry without options to modify context or intercept calls beyond logging.\n\nFor more details, see the official documentation: [Tracing in PromptFlow](https://microsoft.github.io/promptflow/how-to-guides/tracing/index.html). No direct code examples in the browsed content, but tracing is integrated into flow debugging (Python).\n\n#### n8n\n\nn8n's AI Agent node inherits callbacks from LangChain for observability in workflows.\n\nNaming (JS/TS): Callbacks (inherited from LangChain) \nSupports: Y (read/write) \nObservation: AI Agent node uses LangChain under the hood, inheriting callbacks for observability; supports reading/modifying metadata or interrupting flow as in LangChain.\n\nFor more details, see the official documentation: [AI Agent Node Docs](https://docs.n8n.io/integrations/builtin/cluster-nodes/root-nodes/n8n-nodes-langchain.agent/). (Inherits from LangChain; refer to LangChain docs for callback examples.) No specific n8n-unique code in the content, but uses LangChain's observer pattern. Here's an adapted LangChain JS example for consistency:\n\n```javascript\nimport { BaseCallbackHandler } from \"@langchain/core/callbacks/base\";\n\nclass MyHandler extends BaseCallbackHandler {\n name = \"my_handler\";\n\n async handleChainStart(chain, inputs) {\n inputs.number += 1; # Modify inputs (write capability)\n console.log(\"Chain started!\");\n }\n}\n\nconst handler = new MyHandler();\n\n// Pass callback at runtime\nawait chain.invoke({ number: 25 }, { callbacks: [handler] });\n\n// Or at constructor time\nconst chainWithHandler = new SomeChain({ callbacks: [handler] });\nawait chainWithHandler.invoke({ number: 25 });\n```\n"
},
{
"path": "docs/decisions/0008-python-subpackages.md",
"content": "---\nstatus: accepted\ncontact: eavanvalkenburg\ndate: 2025-09-19\ndeciders: eavanvalkenburg, markwallace-microsoft, ekzhu, sphenry, alliscode\nconsulted: taochenosu, moonbox3, dmytrostruk, giles17\n---\n\n# Python Subpackages Design\n\n## Context and Problem Statement\n\nThe goal is to design a subpackage structure for the Python agent framework that balances ease of use, maintainability, and scalability. How can we organize the codebase to facilitate the development and integration of connectors while minimizing complexity for users?\n\n## Decision Drivers\n\n- Ease of use for developers\n- Maintainability of the codebase\n- User experience for installing and using the integrations\n- Clear lifecycle management for integrations\n- Minimize non-GA dependencies in the main package\n\n## Considered Options\n\n1. One subpackage per vendor, so a `google` package that contains all Google related connectors, such as `GoogleChatClient`, `BigQueryCollection`, etc.\n * Pros:\n - fewer packages to manage, publish and maintain\n - easier for users to find and install the right package.\n - users that work primarily with one platform have a single package to install.\n * Cons:\n - larger packages with more dependencies\n - larger installation sizes\n - more difficult to version, since some parts may be GA, while other are in preview.\n2. One subpackage per connector, so a i.e. `google_chat` package, a i.e. `google_bigquery` package, etc.\n * Pros:\n - smaller packages with fewer dependencies\n - smaller installation sizes\n - easy to version and do lifecycle management on\n * Cons:\n - more packages to manage, register, publish and maintain\n - more extras, means more difficult for users to find and install the right package.\n3. Group connectors by vendor and maturity, so that you can graduate something from the i.e. the `google-preview` package to the `google` package when it becomes GA.\n * Pros:\n - fewer packages to manage, publish and maintain\n - easier for users to find and install the right package.\n - users that work primarily with one platform have a single package to install.\n - clear what the status is based on extra name\n * Cons:\n - moving something from one to the other might be a breaking change\n - still larger packages with more dependencies\n It could be mitigated that the `google-preview` package is still imported from `agent_framework.google`, so that the import path does not change, when something graduates, but it is still a clear choice for users to make. And we could then have three extras on that package, `google`, `google-preview` and `google-all` to make it easy to install the right package or just all.\n4. Group connectors by vendor and type, so that you have a `google-chat` package, a `google-data` package, etc.\n * Pros:\n - smaller packages with fewer dependencies\n - smaller installation sizes\n * Cons:\n - more packages to manage, register, publish and maintain\n - more extras, means more difficult for users to find and install the right package.\n - still keeps the lifecycle more difficult, since some parts may be GA, while other are in preview.\n5. Add `meta`-extras, that combine different subpackages as one extra, so we could have a `google` extra that includes `google-chat`, `google-bigquery`, etc.\n * Pros:\n - easier for users on a single platform\n * Cons:\n - more packages to manage, register, publish and maintain\n - more extras, means more difficult for users to find and install the right package.\n - makes developer package management more complex, because that meta-extra will include both GA and non-GA packages, so during dev they could use that, but then during prod they have to figure out which one they actually need and make a change in their dependencies, leading to mismatches between dev and prod.\n6. Make all imports happen from `agent_framework.connectors` (or from two or three groups `agent_framework.chat_clients`, `agent_framework.context_providers`, or something similar) while the underlying code comes from different packages.\n * Pros:\n - best developer experience, since all imports are from the same place and it is easy to find what you need, and we can raise a meaningfull error with which extra to install.\n - easier for users to find and install the right package.\n * Cons:\n - larger overhead in maintaining the `__init__.py` files that do the lazy loading and error handling.\n - larger overhead in package management, since we have to ensure that the main package.\n7. Subpackage existence will be based off status of dependencies and/or possibilities of a external support mechanism. What this means is that:\n - Integrations that need non-GA dependencies will be subpackages, so that we can avoid having non-GA dependencies in the main package.\n - Integrations where the AF-code is still experimental, preview or release candidate will be subpackages, so that we can avoid having non-GA code in the main package and we can version those packages properly.\n - Integrations that are outside Microsoft and where we might not always be able to fast-follow breaking changes, will stay as subpackages, to provide some isolation and to be able to version them properly.\n - Integrations that are mature and that have released (GA) dependencies and or features on the service side will be moved into the main package, the dependencies of those packages will stay installable under the same `extra` name, so that users do not have to change anything, and we then remove the subpackage itself.\n - All subpackage imports in the code should be from a stable place, mostly vendor-based, so that when something moves from a subpackage to the main package, the import path does not change, so `from agent_framework.google import GoogleChatClient` will always work, even if it moves from the `agent-framework-google` package to the main `agent-framework` package.\n - The imports in those vendor namespaces (these won't be actual python namespaces, just the folders with a __init__.py file and any code) will do lazy loading and raise a meaningful error if the subpackage or dependencies are not installed, so that users know which extra to install with ease.\n - On a case by case basis we can decide to create additional `extras`, that combine multiple subpackages into one extra, so that users that work primarily with one platform can install everything they need with a single extra, for instance you can install with the `agent-framework[azure-purview]` extra that only implement a Azure Purview Middleware, or you can install with the `agent-framework[azure]` extra that includes all Azure related connectors, like `purview`, `content safety` and others (all examples, not actual packages (yet)), regardless of where the code sits, these should always be importable from `agent_framework.azure`.\n - Subpackage naming should also follow this, so in principle a package name is `-`, so `google-gemini`, `azure-purview`, `microsoft-copilotstudio`, etc. For smaller vendors, with less likely to have a multitude of connectors, we can skip the feature/brand part, so `mem0`, `redis`, etc.\n\n## Decision Outcome\n\nOption 7: This provides us a good balance between developer experience, user experience, package management and maintenance, while also allowing us to evolve the package structure over time as dependencies and features mature. And it ensures the main package, installed without extras does not include non-GA dependencies or code, extras do not carry that guarantee, for both the code and the dependencies.\n\n# Microsoft vs Azure packages\nAnother consideration is for Microsoft, since we have a lot of Azure services, but also other Microsoft services, such as Microsoft Copilot Studio, and potentially other services in the future, and maybe Foundry also will be marketed separate from Azure at some point. We could also have both a `microsoft` and an `azure` package, where the `microsoft` package contains all Microsoft services, excluding Azure, while the `azure` package only contains Azure services. Only applicable for the variants where we group by vendor, including with meta packages.\n\n## Decision Outcome\nAzure and Microsoft will be the two vendor folders for Microsoft services, so Copilot Studio will be imported from `agent_framework.microsoft`, while Foundry, Azure OpenAI and other Azure services will be imported from `agent_framework.azure`.\n"
},
{
"path": "docs/decisions/0009-support-long-running-operations.md",
"content": "---\nstatus: accepted\ncontact: sergeymenshykh\ndate: 2025-10-15\ndeciders: markwallace, rbarreto, westey-m, stephentoub\ninformed: {}\n---\n\n## Long-Running Operations Design\n\n## Context and Problem Statement\n\nThe Agent Framework currently supports synchronous request-response patterns for AI agent interactions, \nwhere agents process requests and return results immediately. Similarly, MEAI chat clients follow the same \nsynchronous pattern for AI interactions. However, many real-world AI scenarios involve complex tasks that \nrequire significant processing time, such as:\n- Code generation and analysis tasks\n- Complex reasoning and research operations \n- Image and content generation\n- Large document processing and summarization\n\nThe current Agent Framework architecture needs native support for long-running operations, as it is\nessential for handling these scenarios effectively. Additionally, as MEAI chat clients need to start supporting \nlong-running operations as well to be used together with AF agents, the design should consider integration \npatterns and consistency with the broader Microsoft.Extensions.AI ecosystem to provide a unified experience \nacross both agent and chat client scenarios.\n\n## Decision Drivers\n- Chat clients and agents should support long-running execution as well as quick prompts.\n- The design should be simple and intuitive for developers to use.\n- The design should be extensible to allow new long-running execution features to be added in the future.\n- The design should be additive rather than disruptive to allow existing chat clients to iteratively add \nsupport for long-running operations without breaking existing functionality.\n\n## Comparison of Long-Running Operation Features\n| Feature | OpenAI Responses | Foundry Agents | A2A |\n|-----------------------------|---------------------------|-------------------------------------|----------------------|\n| Initiated by | User (Background = true) | Long-running execution is always on | Agent |\n| Modeled as \t\t\t | Response | Run | Task |\n| Supported modes1 | Sync, Async | Async | Sync, Async |\n| Getting status support | ✅ | ✅ | ✅ |\n| Getting result support | ✅ | ✅ | ✅ |\n| Update support | ❌ | ❌ | ✅ |\n| Cancellation support | ✅ | ✅ | ✅ |\n| Delete support | ✅ | ❌ | ❌ |\n| Non-streaming support | ✅ | ✅ | ✅ |\n| Streaming support | ✅ | ✅ | ✅ |\n| Execution statuses | InProgress, Completed, Queued
Cancelled, Failed, Incomplete | InProgress, Completed, Queued
Cancelled, Failed, Cancelling,
RequiresAction, Expired | Working, Completed, Canceled,
Failed, Rejected, AuthRequired,
InputRequired, Submitted, Unknown |\n\n1 Sync is a regular message-based request/response communication pattern; Async is a pattern for long-running operations/tasks where the agent returns an ID for a run/task and allows polling for status and final results by the ID.\n\n**Note:** The names for new classes, interfaces, and their members used in the sections below are tentative and will be discussed in a dedicated section of this document.\n\n## Long-Running Operations Support for Chat Clients\n\nThis section describes different options for various aspects required to add long-running operations support to chat clients.\n\n### 1. Methods for Working with Long-Running Operations\n\nBased on the analysis of existing APIs that support long-running operations (such as OpenAI Responses, Azure AI Foundry Agents, and A2A), \nthe following operations are used for working with long-running operations:\n- Common operations:\n - **Start Long-Running Execution**: Initiates a long-running operation and returns its Id.\n - **Get Status of Long-Running Execution**: This method retrieves the status of a long-running operation.\n - **Get Result of Long-Running Execution**: Retrieves the result of a long-running operation.\n- Uncommon operations:\n - **Update Long-Running Execution**: This method updates a long-running operation, such as adding new messages or modifying existing ones.\n - **Cancel Long-Running Execution**: This method cancels a long-running operation.\n - **Delete Long-Running Execution**: This method deletes a long-running operation.\n\nTo support these operations by `IChatClient` implementations, the following options are available:\n- **1.1 New IAsyncChatClient Interface for All Long-Running Execution Operations**\n- **1.2 Get{Streaming}ResponseAsync for Common Operations & New IAsyncChatClient Interface for Uncommon Operations**\n- **1.3 Get{Streaming}ResponseAsync for Common Operations & New IAsyncChatClient Interface for Uncommon Operations & Capability Check**\n- **1.4 Get{Streaming}ResponseAsync for Common Operations & Individual Interface per Uncommon Operation**\n\n#### 1.1 New IAsyncChatClient Interface for All Long-Running Execution Operations\n\nThis option suggests adding a new interface `IAsyncChatClient` that some implementations of `IChatClient` may implement to support long-running operations.\n```csharp\npublic interface IAsyncChatClient\n{\n Task StartAsyncRunAsync(IList chatMessages, RunOptions? options = null, CancellationToken ct = default);\n Task GetAsyncRunStatusAsync(string runId, CancellationToken ct = default);\n Task GetAsyncRunResultAsync(string runId, CancellationToken ct = default);\n Task UpdateAsyncRunAsync(string runId, IList chatMessages, CancellationToken ct = default);\n Task CancelAsyncRunAsync(string runId, CancellationToken ct = default);\n Task DeleteAsyncRunAsync(string runId, CancellationToken ct = default);\n}\n\npublic class CustomChatClient : IChatClient, IAsyncChatClient\n{\n ...\n}\n```\n\nConsumer code example:\n```csharp\nIChatClient chatClient = new CustomChatClient();\n\nstring prompt = \"...\"\n\n// Determine if the prompt should be run as a long-running execution\nif(chatClient.GetService() is { } asyncChatClient && ShouldRunPromptAsynchronously(prompt)) \n{\n try\n {\n // Start a long-running execution\n AsyncRunResult result = await asyncChatClient.StartAsyncRunAsync(prompt);\n }\n catch (NotSupportedException)\n {\n Console.WriteLine(\"This chat client does not support long-running operations.\");\n throw;\n }\n\n AsyncRunContent? asyncRunContent = GetAsyncRunContent(result);\n \n // Poll for the status of the long-running execution\n while (asyncRunContent.Status is AsyncRunStatus.InProgress or AsyncRunStatus.Queued)\n {\n result = await asyncChatClient.GetAsyncRunStatusAsync(asyncRunContent.RunId);\n asyncRunContent = GetAsyncRunContent(result);\n }\n \n // Get the result of the long-running execution\n result = await asyncChatClient.GetAsyncRunStatusAsync(asyncRunContent.RunId);\n Console.WriteLine(result);\n}\nelse\n{\n // Complete a quick prompt\n ChatResponse response = await chatClient.GetResponseAsync(prompt);\n Console.WriteLine(response);\n}\n```\n\n**Pros:**\n- Not a breaking change: Existing chat clients are not affected.\n- Callers can determine if a chat client supports long-running operations by calling its `GetService()` method.\n\n**Cons:**\n- Not extensible: Adding new methods to the `IAsyncChatClient` interface after its release will break existing implementations of the interface.\n- Missing capability check: Callers cannot determine if chat clients support specific uncommon operations before attempting to use them.\n- Insufficient information: Callers may not have enough information to decide whether a prompt should run as a long-running operation.\n- The new method calls bypass existing decorators such as logging, telemetry, etc.\n- An alternative solution for decorating the new methods will have to be put in place because the new method calls bypass existing decorators \nsuch as logging, telemetry, etc.\n\n#### 1.2 Get{Streaming}ResponseAsync for Common Operations & New IAsyncChatClient Interface for Uncommon Operations\n\nThis option suggests using the existing `GetResponseAsync` and `GetStreamingResponseAsync` methods of the `IChatClient` interface to support \ncommon long-running operations, such as starting long-running operations, getting their status, their results, and potentially \nupdating them, in addition to their existing functionality of serving quick prompts. Methods for the uncommon operations, such as updating, \ncancelling, and deleting long-running operations, will be added to a new `IAsyncChatClient` interface that will be implemented by chat clients \nthat support them.\n\nThis option presumes that Option 3.2 (Have one method for getting long-running execution status and result) is selected.\n\n```csharp\npublic interface IAsyncChatClient\n{\n /// The update can be handled by GetResponseAsync method as well.\n Task UpdateAsyncRunAsync(string runId, IList chatMessages, CancellationToken ct = default);\n \n Task CancelAsyncRunAsync(string runId, CancellationToken ct = default);\n Task DeleteAsyncRunAsync(string runId, CancellationToken ct = default);\n}\n\npublic class ResponsesChatClient : IChatClient, IAsyncChatClient\n{\n public async Task GetResponseAsync(string prompt, ChatOptions? options = null, CancellationToken ct = default)\n {\n ClientResult? result = null;\n\n // If long-running execution mode is enabled, we run the prompt as a long-running execution\n if(enableLongRunningResponses)\n {\n // No RunId is provided, so we start a long-running execution\n if(options?.RunId is null)\n {\n result = await this._openAIResponseClient.CreateResponseAsync(prompt, new ResponseCreationOptions\n {\n Background = true,\n });\n }\n else // RunId is provided, so we get the status of a long-running execution\n {\n result = await this._openAIResponseClient.GetResponseAsync(options.RunId);\n }\n }\n else\n {\n // Handle the case when the prompt should be run as a quick prompt\n result = await this._openAIResponseClient.CreateResponseAsync(prompt, new ResponseCreationOptions\n {\n Background = false\n });\n }\n\n ...\n }\n\n public Task UpdateAsyncRunAsync(string runId, IList chatMessages, CancellationToken ct = default)\n {\n throw new NotSupportedException(\"This chat client does not support updating long-running operations.\");\n }\n\n public Task CancelAsyncRunAsync(string runId, CancellationToken cancellationToken = default)\n {\n return this._openAIResponseClient.CancelResponseAsync(runId, cancellationToken);\n }\n\n public Task DeleteAsyncRunAsync(string runId, CancellationToken cancellationToken = default)\n {\n return this._openAIResponseClient.DeleteResponseAsync(runId, cancellationToken);\n }\n}\n```\n\nConsumer code example:\n```csharp\nIChatClient chatClient = new ResponsesChatClient();\n\nChatResponse response = await chatClient.GetResponseAsync(\"\");\n\nif (GetAsyncRunContent(response) is AsyncRunContent asyncRunContent)\n{\n // Get result of the long-running execution\n response = await chatClient.GetResponseAsync([], new ChatOptions\n { \n RunId = asyncRunContent.RunId \n });\n\n // After some time\n\n // If it's still running, cancel and delete the run\n if (GetAsyncRunContent(response).Status is AsyncRunStatus.InProgress or AsyncRunStatus.Queued)\n {\n IAsyncChatClient? asyncChatClient = chatClient.GetService();\n\n try\n {\n await asyncChatClient?.CancelAsyncRunAsync(asyncRunContent.RunId);\n }\n catch (NotSupportedException)\n {\n Console.WriteLine(\"This chat client does not support cancelling long-running operations.\");\n }\n \n try\n {\n await asyncChatClient?.DeleteAsyncRunAsync(asyncRunContent.RunId);\n }\n catch (NotSupportedException)\n {\n Console.WriteLine(\"This chat client does not support deleting long-running operations.\");\n }\n }\n}\nelse\n{\n // Handle the case when the response is a quick prompt completion\n Console.WriteLine(response);\n}\n```\n\nThis option addresses the issue that the option above has with callers needing to know whether the prompt should \nbe run as a long-running operation or a quick prompt. It allows callers to simply call the existing `GetResponseAsync` method, \nand the chat client will decide whether to run the prompt as a long-running operation or a quick prompt. If control over \nthe execution mode is still needed, and the underlying API supports it, it will be possible for callers to set the mode at \nthe chat client invocation or configuration. More details about this are provided in one of the sections below about enabling long-running operation mode.\n \nAdditionally, it addresses another issue where the `GetResponseAsync` method may return a long-running\nexecution response and the `StartAsyncRunAsync` method may return a quick prompt response. Having one method that handles both cases\nallows callers to not worry about this behavior and simply check the type of the response to determine if it is a long-running operation\nor a quick prompt completion.\n\nWith the `GetResponseAsync` method becoming responsible for starting, getting status, getting results and updating long-running operations,\nthere are only a few operations left in the `IAsyncChatClient` interface - cancel and delete. As a result, the `IAsyncChatClient` interface\nname may not be the best fit, as it suggests that it is responsible for all long-running operations while it is not. Should \nthe interface be renamed to reflect the operations it supports? What should the new name be? Option 1.4 considers an alternative\nthat might solve the naming issue. \n\n**Pros:**\n- Delegation and control: Callers delegate the decision of whether to run a prompt as a long-running operation or quick prompt to chat clients,\nwhile still having the option to control the execution mode to determine how to handle prompts if needed.\n- Not a breaking change: Existing chat clients are not affected. \n \n**Cons:** \n- Not extensible: Adding new methods to the `IAsyncChatClient` interface after its release will break existing implementations of the interface. \n- Missing capability check: Callers cannot determine if chat clients support specific uncommon operations before attempting to use them.\n- An alternative solution for decorating the new methods will have to be put in place because the new method calls bypass existing decorators \nsuch as logging, telemetry, etc.\n\n#### 1.3 Get{Streaming}ResponseAsync for Common Operations & New IAsyncChatClient Interface for Uncommon Operations & Capability Check\n\nThis option extends the previous option with a way for callers to determine if a chat client supports uncommon operations before attempting to use them.\n\n```csharp\npublic interface IAsyncChatClient\n{\n bool CanUpdateAsyncRun { get; }\n bool CanCancelAsyncRun { get; } \n bool CanDeleteAsyncRun { get; } \n\n Task UpdateAsyncRunAsync(string runId, IList chatMessages, CancellationToken ct = default);\n Task CancelAsyncRunAsync(string runId, CancellationToken ct = default);\n Task DeleteAsyncRunAsync(string runId, CancellationToken ct = default);\n}\n\npublic class ResponsesChatClient : IChatClient, IAsyncChatClient\n{\n public async Task GetResponseAsync(string prompt, ChatOptions? options = null, CancellationToken ct = default)\n {\n ...\n }\n\n public bool CanUpdateAsyncRun => false; // This chat client does not support updating long-running operations.\n public bool CanCancelAsyncRun => true; // This chat client supports cancelling long-running operations.\n public bool CanDeleteAsyncRun => true; // This chat client supports deleting long-running operations.\n\n public Task UpdateAsyncRunAsync(string runId, IList chatMessages, CancellationToken ct = default)\n {\n throw new NotSupportedException(\"This chat client does not support updating long-running operations.\");\n }\n\n public Task CancelAsyncRunAsync(string runId, CancellationToken cancellationToken = default)\n {\n return this._openAIResponseClient.CancelResponseAsync(runId, cancellationToken);\n }\n\n public Task DeleteAsyncRunAsync(string runId, CancellationToken cancellationToken = default)\n {\n return this._openAIResponseClient.DeleteResponseAsync(runId, cancellationToken);\n }\n}\n```\n\nConsumer code example:\n```csharp\nIChatClient chatClient = new ResponsesChatClient();\n\nChatResponse response = await chatClient.GetResponseAsync(\"\");\n\nif (GetAsyncRunContent(response) is AsyncRunContent asyncRunContent)\n{\n // Get result of the long-running execution\n response = await chatClient.GetResponseAsync([], new ChatOptions\n { \n RunId = asyncRunContent.RunId \n });\n\n // After some time\n\n IAsyncChatClient? asyncChatClient = chatClient.GetService();\n\n // If it's still running, cancel and delete the run\n if (GetAsyncRunContent(response).Status is AsyncRunStatus.InProgress or AsyncRunStatus.Queued)\n {\n if(asyncChatClient?.CanCancelAsyncRun ?? false)\n {\n await asyncChatClient?.CancelAsyncRunAsync(asyncRunContent.RunId);\n }\n\n if(asyncChatClient?.CanDeleteAsyncRun ?? false)\n {\n await asyncChatClient?.DeleteAsyncRunAsync(asyncRunContent.RunId);\n } \n }\n}\nelse\n{\n // Handle the case when the response is a quick prompt completion\n Console.WriteLine(response);\n}\n```\n\n**Pros:**\n- Delegation and control: Callers delegate the decision of whether to run a prompt as a long-running execution or quick prompt to chat clients,\nwhile still having the option to control the execution mode to determine how to handle prompts if needed.\n- Not a breaking change: Existing chat clients are not affected. \n- Capability check: Callers can determine if the chat client supports an uncommon operation before attempting to use it.\n \n**Cons:** \n- Not extensible: Adding new members to the `IAsyncChatClient` interface after its release will break existing implementations of the interface. \n- An alternative solution for decorating the new methods will have to be put in place because the new method calls bypass existing decorators \nsuch as logging, telemetry, etc.\n\n#### 1.4 Get{Streaming}ResponseAsync for Common Operations & Individual Interface per Uncommon Operation\n\nThis option suggests using the existing `Get{Streaming}ResponseAsync` methods of the `IChatClient` interface to support \ncommon long-running operations, such as starting long-running operations, getting their status, and their results, and potentially \nupdating them, in addition to their existing functionality of serving quick prompts.\n\nThe uncommon operations that are not supported by all analyzed APIs, such as updating (which can be handled by `Get{Streaming}ResponseAsync`), cancelling, \nand deleting long-running operations, as well as future ones, will be added to their own interfaces that will be implemented by chat clients \nthat support them.\n\nThis option presumes that Option 3.2 (Have one method for getting long-running execution status and result) is selected.\n\nThe interfaces can inherit from `IChatClient` to allow callers to use an instance of `ICancelableChatClient`, `IUpdatableChatClient`, or `IDeletableChatClient` \nfor calling the `Get{Streaming}ResponseAsync` methods as well. However, those methods belong to a leaf chat client that, if obtained via the `GetService()` \nmethod, won't be decorated by existing decorators such as function invocation, logging, etc. As a result, an alternative solution (wrap the instance of the leaf \nchat client in a decorator at the `GetService` method call) will need to be applied not only to the new methods of one of the interfaces but also to the existing\n`Get{Streaming}ResponseAsync` ones.\n\n```csharp\npublic interface ICancelableChatClient\n{ \n Task CancelAsyncRunAsync(string runId, CancellationToken cancellationToken = default);\n}\n\npublic interface IUpdatableChatClient\n{ \n Task UpdateAsyncRunAsync(string runId, IList chatMessages, CancellationToken cancellationToken = default);\n}\n\npublic interface IDeletableChatClient\n{ \n Task DeleteAsyncRunAsync(string runId, CancellationToken cancellationToken = default);\n}\n\n// Responses chat client that supports standard long-running operations + cancellation and deletion\npublic class ResponsesChatClient : IChatClient, ICancelableChatClient, IDeletableChatClient\n{\n public async Task GetResponseAsync(string prompt, ChatOptions? options = null, CancellationToken ct = default)\n {\n ...\n }\n\n public Task CancelAsyncRunAsync(string runId, CancellationToken cancellationToken = default)\n {\n return this._openAIResponseClient.CancelResponseAsync(runId, cancellationToken);\n }\n\n public Task DeleteAsyncRunAsync(string runId, CancellationToken cancellationToken = default)\n {\n return this._openAIResponseClient.DeleteResponseAsync(runId, cancellationToken);\n }\n}\n```\n\nExample that starts a long-running operation, gets its status, and cancels and deletes it if it's not completed after some time:\n```csharp\nIChatClient chatClient = new ResponsesChatClient();\n\nChatResponse response = await chatClient.GetResponseAsync(\"\", new ChatOptions { AllowLongRunningResponses = true });\n\nif (GetAsyncRunContent(response) is AsyncRunContent asyncRunContent)\n{\n // Get result\n response = await chatClient.GetResponseAsync([], new ChatOptions\n { \n RunId = asyncRunContent.RunId \n });\n\n // After some time\n\n // If it's still running, cancel and delete the run\n if (GetAsyncRunContent(response).Status is AsyncRunStatus.InProgress or AsyncRunStatus.Queued)\n {\n if(chatClient.GetService() is {} cancelableChatClient)\n {\n await cancelableChatClient.CancelAsyncRunAsync(asyncRunContent.RunId);\n }\n\n if(chatClient.GetService() is {} deletableChatClient)\n {\n await deletableChatClient.DeleteAsyncRunAsync(asyncRunContent.RunId);\n }\n }\n}\n```\n\n**Pros:**\n- Extensible: New interfaces can be added and implemented to support new long-running operations without breaking \nexisting chat client implementations.\n- Not a breaking change: Existing chat clients that implement the `IChatClient` interface are not affected.\n- Delegation and control: Callers delegate the decision of whether to run a prompt as a long-running operation or quick prompt\nto chat clients, while still having the option to control the execution mode to determine how to handle prompts if needed.\n \n**Cons:** \n- Breaking changes: Changing the signatures of the methods of the operation-specific interfaces or adding new members to them will \nbreak existing implementations of those interfaces. However, the blast radius of this change is much smaller and limited to a subset\nof chat clients that implement the operation-specific interfaces. However, this is still a breaking change.\n\n### 2. Enabling Long-Running Operations\n\nBased on the API analysis, some APIs must be explicitly configured to run in long-running operation mode, \nwhile others don't need additional configuration because they either decide themselves whether a request\nshould run as a long-running operation, or they always operate in long-running operation mode or quick prompt mode:\n| Feature | OpenAI Responses | Foundry Agents | A2A |\n|-----------------------------|---------------------------|-------------------------------------|----------------------|\n| Long-running execution | User (Background = true) | Long-running execution is always on | Agent |\n\nThe options below consider how to enable long-running operation mode for chat clients that support both quick prompts and long-running operations.\n\n#### 2.1 Execution Mode per `Get{Streaming}ResponseAsync` Invocation\n\nThis option proposes adding a new nullable `AllowLongRunningResponses` property to the `ChatOptions` class.\nThe property value will be `true` if the caller requests a long-running operation, `false`, `null` or omitted otherwise.\n \nChat clients that work with APIs requiring explicit configuration per operation will use this property to determine whether to run the prompt as a long-running \noperation or quick prompt. Chat clients that work with APIs that don't require explicit configuration will ignore this property and operate according \nto their own logic/configuration.\n\n```csharp\npublic class ChatOptions\n{\n // Existing properties...\n public bool? AllowLongRunningResponses { get; set; }\n}\n\n// Consumer code example\nIChatClient chatClient = ...; // Get an instance of IChatClient\n\n// Start a long-running execution for the prompt if supported by the underlying API\nChatResponse response = await chatClient.GetResponseAsync(\"\", new ChatOptions { AllowLongRunningResponses = true });\n\n// Start a quick prompt\nChatResponse quickResponse = await chatClient.GetResponseAsync(\"\", new ChatOptions { AllowLongRunningResponses = false });\n```\n\n**Pros:** \n- Callers can switch between quick prompts and long-running operation per invocation of the `Get{Streaming}ResponseAsync` methods without \nchanging the client configuration.\n- Enables explicit control over the execution mode by callers per invocation, meaning that no caller site is broken if the agent is injected via DI, \nand the caller can turn on the long-running operation mode when it can handle it.\n\n**Con:** This may not be valuable for all callers, as they may not have enough information to decide whether the prompt should run as a long-running operation or quick prompt.\n\n#### 2.2 Execution Mode per `Get{Streaming}ResponseAsync` Invocation + Model Class\n\nThis option is similar to the previous one, but suggest using a model class `LongRunningResponsesOptions` for properties related to long-running operations.\n\n```csharp\npublic class LongRunningResponsesOptions\n{\n public bool? Allow { get; set; }\n //public PollingSettings? PollingSettings { get; set; } // Can be added leter if necessary\n}\n\npublic class ChatOptions\n{\n public LongRunningResponsesOptions? LongRunningResponsesOptions { get; set; }\n}\n\n// Consumer code example\nIChatClient chatClient = ...; // Get an instance of IChatClient\n\n// Start a long-running execution for the prompt if supported by the underlying API\nChatResponse response = await chatClient.GetResponseAsync(\"\", new ChatOptions { LongRunningResponsesOptions = new() { Allow = true } });\n```\n\n**Pros:** \n- Enables explicit control over the execution mode by callers per invocation, meaning that no caller site is broken if the agent is injected via DI, \nand the caller can turn on the long-running operation mode when it can handle it.\n- No proliferation of long-running operation-related properties in the `ChatOptions` class.\n\n**Con:** Slightly more complex initialization.\n\n#### 2.3 Execution Mode per Chat Client Instance\n\nThis option proposes adding a new `enableLongRunningResponses` parameter to constructors of chat clients that support both quick prompts and long-running operations.\nThe parameter value will be `true` if the chat client should operate in long-running operation mode, `false` if it should operate in quick prompt mode.\n\nChat clients that work with APIs requiring explicit configuration will use this parameter to determine whether to run prompts as long-running operations or quick prompts.\nChat clients that work with APIs that don't require explicit configuration won't have this parameter in their constructors and will operate according to their own \nlogic/configuration.\n\n```csharp\npublic class CustomChatClient : IChatClient\n{\n private readonly bool _enableLongRunningResponses;\n\n public CustomChatClient(bool enableLongRunningResponses)\n {\n this._enableLongRunningResponses = enableLongRunningResponses;\n }\n\n // Existing methods...\n}\n\n// Consumer code example\nIChatClient chatClient = new CustomChatClient(enableLongRunningResponses: true);\n\n// Start a long-running execution for the prompt\nChatResponse response = await chatClient.GetResponseAsync(\"\");\n```\n\nChat clients can be configured to always operate in long-running operation mode or quick prompt mode based on their role in a specific scenario.\nFor example, a chat client responsible for generating ideas for images can be configured for quick prompt mode, while a chat client responsible for image \ngeneration can be configured to always use long-running operation mode.\n\n**Pro:** Can be beneficial for scenarios where chat clients need to be configured upfront in accordance with their role in a scenario.\n\n**Con:** Less flexible than the previous option, as it requires configuring the chat client upfront at instantiation time. However, this flexibility might not be needed.\n\n#### 2.4 Combined Approach\n\nThis option proposes a combined approach that allows configuration per chat client instance and per `Get{Streaming}ResponseAsync` method invocation.\n\nThe chat client will use whichever configuration is provided, whether set in the chat client constructor or in the options for the `Get{Streaming}ResponseAsync` \nmethod invocation. If both are set, the one provided in the `Get{Streaming}ResponseAsync` method invocation takes precedence.\n\n```csharp\npublic class CustomChatClient : IChatClient\n{\n private readonly bool _enableLongRunningResponses;\n\n public CustomChatClient(bool enableLongRunningResponses)\n {\n this._enableLongRunningResponses = enableLongRunningResponses;\n }\n \n public async Task GetResponseAsync(string prompt, ChatOptions? options = null, CancellationToken ct = default)\n {\n bool enableLongRunningResponses = options?.AllowLongRunningResponses ?? this._enableLongRunningResponses;\n // Logic to handle the prompt based on enableLongRunningResponses...\n }\n}\n\n// Consumer code example\nIChatClient chatClient = new CustomChatClient(enableLongRunningResponses: true);\n\n// Start a long-running execution for the prompt\nChatResponse response = await chatClient.GetResponseAsync(\"\");\n\n// Start a quick prompt\nChatResponse quickResponse = await chatClient.GetResponseAsync(\"\", new ChatOptions { AllowLongRunningResponses = false });\n```\n\n**Pros:** Flexible approach that combines the benefits of both previous options.\n\n### 3. Getting Status and Result of Long-Running Execution\n\nThe explored APIs use different approaches for retrieving the status and results of long-running operations. Some are using\none method to retrieve both status and result, while others use two separate methods for each operation:\n| Feature | OpenAI Responses | Foundry Agents | A2A |\n|-------------------|-------------------------------|----------------------------------------------------|-----------------------|\n| API to Get Status | GetResponseAsync(responseId) | Runs.GetRunAsync(thread.Id, threadRun.Id) | GetTaskAsync(task.Id) |\n| API to Get Result | GetResponseAsync(responseId) | Messages.GetMessagesAsync(thread.Id, threadRun.Id) | GetTaskAsync(task.Id) |\n\nTaking into account the differences, the following options propose a few ways to model the API for getting the status and result of \nlong-running operations for the `AIAgent` interface implementations.\n\n#### 3.1 Two Separate Methods for Status and Result\n\nThis option suggests having two separate methods for getting the status and result of long-running operations:\n```csharp\npublic interface IAsyncChatClient\n{\n Task GetAsyncRunStatusAsync(string runId, CancellationToken ct = default);\n Task GetAsyncRunResultAsync(string runId, CancellationToken ct = default);\n}\n```\n\n**Pros:** Could be more intuitive for developers, as it clearly separates the concerns of checking the status and retrieving the result of a long-running operation.\n\n**Cons:** Creates inefficiency for chat clients that use APIs that return both status and result in a single call, \nas callers might make redundant calls to get the result after checking the status that already contains the result.\n\n#### 3.2 One Method to Get Status and Result\n\nThis option suggests having a single method for getting both the status and result of long-running operations:\n```csharp\npublic interface IAsyncChatClient\n{\n Task GetAsyncRunResultAsync(string runId, AgentThread? thread = null, CancellationToken ct = default);\n}\n```\n\nThis option will redirect the call to the appropriate method of the underlying API that uses one method to retrieve both.\nFor APIs that use two separate methods, the method will first get the status and if the status indicates that the \noperation is still running, it will return the status to the caller. If the status indicates that the operation is completed,\nit will then call the method to get the result of the long-running operation and return it together with the status.\n\n**Pros:**\n- Simplifies the API by providing a single, intuitive method for retrieving long-running operation information.\n- More optimal for chat clients that use APIs that return both status and result in a single call, as it avoids unnecessary API calls.\n\n### 4. Place For RunId, Status, and UpdateId of Long-Running Operations\n\nThis section considers different options for exposing the `RunId`, `Status`, and `UpdateId` properties of long-running operations.\n\n#### 4.1. As AIContent\n\nThe `AsyncRunContent` class will represent a long-running operation initiated and managed by an agent/LLM.\nItems of this content type will be returned in a chat message as part of the `AgentResponse` or `ChatResponse`\nresponse to represent the long-running operation.\n\nThe `AsyncRunContent` class has two properties: `RunId` and `Status`. The `RunId` identifies the \nlong-running operation, and the `Status` represents the current status of the operation. The class \ninherits from `AIContent`, which is a base class for all AI-related content in MEAI and AF.\n\nThe `AsyncRunStatus` class represents the status of a long-running operation. Initially, it will have \na set of predefined statuses that represent the possible statuses used by existing Agent/LLM APIs that support\nlong-running operations. It will be extended to support additional statuses as needed while also\nallowing custom, not-yet-defined statuses to propagate as strings from the underlying API to the callers.\n\nThe content class type can be used by both agents and chat clients to represent long-running operations.\nFor chat clients to use it, it should be declared in one of the MEAI packages.\n\n```csharp\npublic class AsyncRunContent : AIContent\n{\n public string RunId { get; }\n public AsyncRunStatus? Status { get; }\n}\n\npublic readonly struct AsyncRunStatus : IEquatable\n{\n public static AsyncRunStatus Queued { get; } = new(\"Queued\");\n public static AsyncRunStatus InProgress { get; } = new(\"InProgress\");\n public static AsyncRunStatus Completed { get; } = new(\"Completed\");\n public static AsyncRunStatus Cancelled { get; } = new(\"Cancelled\");\n public static AsyncRunStatus Failed { get; } = new(\"Failed\");\n public static AsyncRunStatus RequiresAction { get; } = new(\"RequiresAction\");\n public static AsyncRunStatus Expired { get; } = new(\"Expired\");\n public static AsyncRunStatus Rejected { get; } = new(\"Rejected\");\n public static AsyncRunStatus AuthRequired { get; } = new(\"AuthRequired\");\n public static AsyncRunStatus InputRequired { get; } = new(\"InputRequired\");\n public static AsyncRunStatus Unknown { get; } = new(\"Unknown\");\n\n public string Label { get; }\n\n public AsyncRunStatus(string label)\n {\n if (string.IsNullOrWhiteSpace(label))\n {\n throw new ArgumentException(\"Label cannot be null or whitespace.\", nameof(label));\n }\n\n this.Label = label;\n }\n\n /// Other members\n}\n````\n\nThe streaming API may return an UpdateId identifying a particular update within a streamed response. \nThis UpdateId should be available together with RunId to callers, allowing them to resume a long-running operation identified \nby the RunId from the last received update, identified by the UpdateId.\n\n#### 4.2. As Properties Of ChatResponse{Update}\n\nThis option suggests adding properties related to long-running operations directly to the `ChatResponse` and `ChatResponseUpdate` classes rather \nthan using a separate content class for that. See section \"6. Model To Support Long-Running Operations\" for more details.\n\n### 5. Streaming Support\n\nAll analyzed APIs that support long-running operations also support streaming. \n\nSome of them natively support resuming streaming from a specific point in the stream, while for others, this is either implementation-dependent or needs to be emulated:\n\n| API | Can Resume Streaming | Model |\n|-------------------------|--------------------------------------|------------------------------------------------------------------------------------------------------------|\n| OpenAI Responses | Yes | StreamingResponseUpdate.**SequenceNumber** + GetResponseStreamingAsync(responseId, **startingAfter**, ct) |\n| Azure AI Foundry Agents | Emulated2 | RunStep.**Id** + custom pseudo code: client.Runs.GetRunStepsAsync(...).AllStepsAfter(**stepId**) |\n| A2A | Implementation dependent1 | \t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t\t |\n\n1 The [A2A specification](https://github.com/a2aproject/A2A/blob/main/docs/topics/streaming-and-async.md#1-streaming-with-server-sent-events-sse)\nallows an A2A agent implementation to decide how to handle streaming resumption: _If a client's SSE connection breaks prematurely while \na task is still active (and the server hasn't sent a final: true event for that phase), the client can attempt to reconnect to the stream using the tasks/resubscribe RPC method. \nThe server's behavior regarding missed events during the disconnection period (e.g., whether it backfills or only sends new updates) is implementation-dependent._\n\n2 The Azure AI Foundry Agents API has an API to start a streaming run but does not have an API to resume streaming from a specific point in the stream.\nHowever, it has non-streaming APIs to access already started runs, which can be used to emulate streaming resumption by accessing a run and its steps and streaming all the steps after a specific step.\n\n#### Required Changes\n\nTo support streaming resumption, the following model changes are required:\n\n- The `ChatOptions` class needs to be extended with a new `StartAfter` property that will identify an update to resume streaming from and to start generating responses after.\n- The `ChatResponseUpdate` class needs to be extended with a new `SequenceNumber` property that will identify the update number within the stream.\n\nAll the chat clients supporting the streaming resumption will need to return the `SequenceNumber` property as part of the `ChatResponseUpdate` class and \nhonor the `StartAfter` property of the `ChatOptions` class.\n\n#### Function Calling\n\nFunction calls over streaming are communicated to chat clients through a series of updates. Chat clients accumulate these updates in their internal state to build\nthe function call content once the last update has been received. The completed function call content is then returned to the function-calling chat client, \nwhich eventually invokes it.\n\nSince chat clients keep function call updates in their internal state, resuming streaming from a specific update can be impossible if the resumption request \nis made using a chat client that does not have the previous updates stored. This situation can occur if a host suspends execution during an ongoing function call \nstream and later resumes from that particular update. Because chat clients' internal state is not persisted, they will lack the prior updates needed to continue \nthe function call, leading to a failure in resumption.\n\nTo address this issue, chat clients can only return sequence numbers for updates that are resumable. For updates that cannot be resumed from, chat clients can \nreturn the sequence number of the most recent update received before the non-resumable one. This allows callers to resume from that earlier update,\neven if it means re-processing some updates that have already been handled.\n\nChat clients will continue returning the sequence number of the last resumable update until a new resumable update becomes available. For example, a chat client might \nkeep returning sequence number 2, corresponding to the last resumable update received before an update for the first function call. Once **all** function call updates \nare received and processed, and the model returns a non-function call response, the chat client will then return a sequence number, say 10, which corresponds to the \nfirst non-function call update. \n\n##### Status of Streaming Updates\n\nDifferent APIs provide different statuses for streamed function call updates\n\nSequence of updates from OpenAI Responses API to answer the question \"What time is it?\" using a function call:\n| Id | SN | Update.Kind | Response.Status | ChatResponseUpdate.Status | Description |\n|--------|----|--------------------------|-----------------|---------------------------|---------------------------------------------------|\n| resp_1 | 0 | resp.created | Queued | Queued | |\n| resp_1 | 1 | resp.queued | Queued | Queued | |\n| resp_1 | 2 | resp.in_progress | InProgress | InProgress | |\n| resp_1 | 3 | resp.output_item.added | - | InProgress | |\n| resp_1 | 4 | resp.func_call.args.delta| - | InProgress | |\n| resp_1 | 5 | resp.func_call.args.done | - | InProgress | |\n| resp_1 | 6 | resp.output_item.done | - | InProgress | |\n| resp_1 | 7 | resp.completed | Completed | Complete | |\n| resp_1 | - | - | - | null | FunctionInvokingChatClient yields function result |\n| | | | OpenAI Responses created a new response to handle function call result |\n| resp_2 | 0 | resp.created | Queued | Queued | |\n| resp_2 | 1 | resp.queued | Queued | Queued | |\n| resp_2 | 2 | resp.in_progress | InProgress | InProgress | |\n| resp_2 | 3 | resp.output_item.added | - | InProgress | |\n| resp_2 | 4 | resp.cnt_part.added | - | InProgress | |\n| resp_2 | 5 | resp.output_text.delta | - | InProgress | |\n| resp_2 | 6 | resp.output_text.delta | - | InProgress | |\n| resp_2 | 7 | resp.output_text.delta | - | InProgress | |\n| resp_2 | 8 | resp.output_text.done | - | InProgress | |\n| resp_2 | 9 | resp.cnt_part.done | - | InProgress | |\n| resp_2 | 10 | resp.output_item.done | - | InProgress | |\n| resp_2 | 11 | resp.completed | Completed | Completed | |\n\nSequence of updates from Azure AI Foundry Agents API to answer the question \"What time is it?\" using a function call:\n| Id | SN | UpdateKind | Run.Status | Step.Status | Message.Status | ChatResponseUpdate.Status | Description |\n|--------|---------|-------------------|----------------|-------------|-----------------|---------------------------|---------------------------------------------------|\n| run_1 | - | RunCreated | Queued | - | - | Queued | |\n| run_1 | step_1 | - | RequiredAction | InProgress | - | RequiredAction | |\n| TBD\t | -\t | -\t\t\t\t | - | - | - | - | FunctionInvokingChatClient yields function result |\n| run_1 | - | RunStepCompleted | Completed | - | - | InProgress | |\n| run_1 | -\t | RunQueued | Queued\t\t | - | - | Queued | |\n| run_1 | -\t | RunInProgress | InProgress\t | - | - | InProgress | |\n| run_1 | step_2 | RunStepCreated | - | InProgress | - | InProgress | |\n| run_1 | step_2 | RunStepInProgress | - | InProgress | - | InProgress | |\n| run_1 | - | MessageCreated | - | - | InProgress | InProgress | |\n| run_1 | - | MessageInProgress | - | - | InProgress | InProgress | |\n| run_1 | - | MessageUpdated | - | - | - | InProgress | |\n| run_1 | - | MessageUpdated | - | - | - | InProgress | |\n| run_1 | - | MessageUpdated | - | - | - | InProgress | |\n| run_1 | - | MessageCompleted | - | - | Completed | InProgress | |\n| run_1 | step_2 | RunStepCompleted | Completed | - | - | InProgress | |\n| run_1 | - | RunCompleted | Completed | - | - | Completed | |\n\n### 6. Model To Support Long-Running Operations\n\nTo support long-running operations, the following values need to be returned by the GetResponseAsync and GetStreamingResponseAsync methods:\n- `ResponseId` - identifier of the long-running operation or an entity representing it, such as a task.\n- `ConversationId` - identifier of the conversation or thread the long-running operation is part of. Some APIs, like Azure AI Foundry Agents, use \n this identifier together with the ResponseId to identify a run.\n- `SequenceNumber` - identifier of an update within a stream of updates. This is required to support streaming resumption by the GetStreamingResponseAsync method only.\n- `Status` - status of the long-running operation: whether it is queued, running, failed, cancelled, completed, etc.\n\nThese values need to be supplied to subsequent calls of the GetResponseAsync and GetStreamingResponseAsync methods to get the status and result of long-running operations.\n\n#### 6.1 ChatOptions\n\nThe following options consider different ways of extending the `ChatOptions` class to include the following properties to support long-running operations:\n- `AllowLongRunningResponses` - a boolean property that indicates whether the caller allows the chat client to run in long-running operation mode if it's supported by the chat client.\n- `ResponseId` - a string property that represents the identifier of the long-running operation or an entity representing it. A non-null value of this property would indicate to chat clients\nthat callers want to get the status and result of an existing long-running operation, identified by the property value, rather than starting a new one.\n- `StartAfter` - a string property that represents the sequence number of an update within a stream of updates so that the chat client can resume streaming after the last received update.\n\n##### 6.1.1 Direct Properties in ChatOptions\n\n```csharp\npublic class ChatOptions\n{\n // Existing properties...\n /// Gets or sets an optional identifier used to associate a request with an existing conversation.\n public string? ConversationId { get; set; }\n ...\n\n // New properties...\n public bool? AllowLongRunningResponses { get; set; }\n public string? ResponseId { get; set; }\n public string? StartAfter { get; set; }\n}\n\n// Usage example\nvar response = await chatClient.GetResponseAsync(\"\", new ChatOptions { AllowLongRunningResponses = true });\n\n// If the response indicates a long-running operation, get its status and result\nif(response.Status is {} status)\n{\n response = await chatClient.GetResponseAsync([], new ChatOptions \n { \n AllowLongRunningResponses = true,\n ResponseId = response.ResponseId,\n ConversationId = response.ConversationId,\n //StartAfter = response.SequenceNumber // for GetStreamingResponseAsync only\n });\n}\n\n```\n\n**Con:** Proliferation of long-running operation properties in the `ChatOptions` class.\n\n##### 6.1.2 LongRunOptions Model Class\n\n```csharp\npublic class ChatOptions\n{\n // Existing properties...\n public string? ConversationId { get; set; } \n ...\n \n // New properties...\n public bool? AllowLongRunningResponses { get; set; }\n\n public LongRunOptions? LongRunOptions { get; set; }\n}\n\npublic class LongRunOptions\n{\n public string? ResponseId { get; set; }\n public string? ConversationId { get; set; } \n public string? StartAfter { get; set; }\n\n // Alternatively, ChatResponse can have an extension method ToLongRunOptions.\n public LongRunOptions FromChatResponse(ChatResponse response)\n {\n return new LongRunOptions\n {\n ResponseId = response.ResponseId,\n ConversationId = response.ConversationId,\n };\n }\n\n // Alternatively, ChatResponseUpdate can have an extension method ToLongRunOptions.\n public LongRunOptions FromChatResponseUpdate(ChatResponseUpdate update)\n {\n return new LongRunOptions\n {\n ResponseId = update.ResponseId,\n ConversationId = update.ConversationId,\n StartAfter = update.SequenceNumber,\n };\n }\n}\n\n// Usage example\nvar response = await chatClient.GetResponseAsync(\"\", new ChatOptions { AllowLongRunningResponses = true });\n\n// If the response indicates a long-running operation, get its status and result\nif(response.Status is {} status)\n{\n while(status != ResponseStatus.Completed)\n {\n response = await chatClient.GetResponseAsync([], new ChatOptions \n { \n AllowLongRunningResponses = true,\n LongRunOptions = LongRunOptions.FromChatResponse(response)\n // or extension method\n LongRunOptions = response.ToLongRunOptions()\n // or implicit conversion\n LongRunOptions = response\n });\n }\n}\n```\n\n**Pro:** No proliferation of long-running operation properties in the `ChatOptions` class.\n\n**Con:** Duplicated property `ConversationId`.\n\n##### 6.1.3 Continuation Token of System.ClientModel.ContinuationToken Type\n\nThis option suggests using `System.ClientModel.ContinuationToken` to encapsulate all properties required for long-running operations.\nThe continuation token will be returned by chat clients as part of the `ChatResponse` and `ChatResponseUpdate` responses to indicate that\nthe response is part of a long-running execution. A null value of the property will indicate that the response is not part of a long-running execution.\nChat clients will accept a non-null value of the property to indicate that callers want to get the status and result of an existing long-running operation.\n\nEach chat client will implement its own continuation token class that inherits from `ContinuationToken` to encapsulate properties required for long-running operations\nthat are specific to the underlying API the chat client works with. For example, for the OpenAI Responses API, the continuation token class will encapsulate\nthe `ResponseId` and `SequenceNumber` properties.\n\n```csharp\npublic class ChatOptions\n{\n // Existing properties...\n public string? ConversationId { get; set; } \n ...\n \n // New properties...\n public bool? AllowLongRunningResponses { get; set; }\n\n public ContinuationToken? ContinuationToken { get; set; }\n}\n\ninternal sealed class LongRunContinuationToken : ContinuationToken\n{\n public LongRunContinuationToken(string responseId)\n {\n this.ResponseId = responseId;\n }\n\n public string ResponseId { get; set; }\n\n public int? SequenceNumber { get; set; }\n\n public static LongRunContinuationToken FromToken(ContinuationToken token)\n {\n if (token is LongRunContinuationToken longRunContinuationToken)\n {\n return longRunContinuationToken;\n }\n\n BinaryData data = token.ToBytes();\n\n Utf8JsonReader reader = new(data);\n\n string responseId = null!;\n int? startAfter = null;\n\n reader.Read();\n\n // Reading functionality\n\n return new(responseId)\n {\n SequenceNumber = startAfter\n };\n }\n}\n\n// Usage example\nChatOptions options = new() { AllowLongRunningResponses = true };\n\nvar response = await chatClient.GetResponseAsync(\"\", options);\n\nwhile (response.ContinuationToken is { } token)\n{\n options.ContinuationToken = token;\n\n response = await chatClient.GetResponseAsync([], options);\n}\n\nConsole.WriteLine(response.Text);\n```\n\n**Pro:** No proliferation of long-running operation properties in the `ChatOptions` class, including the `Status` property.\n\n##### 6.1.4 Continuation Token of String Type\n\nThis options is similar to the previous one but suggests using a string type for the continuation token instead of the `System.ClientModel.ContinuationToken` type.\n\n```csharp\ninternal sealed class LongRunContinuationToken\n{\n public LongRunContinuationToken(string responseId)\n {\n this.ResponseId = responseId;\n }\n\n public string ResponseId { get; set; }\n\n public int? SequenceNumber { get; set; }\n\n public static LongRunContinuationToken Deserialize(string json)\n {\n Throw.IfNullOrEmpty(json);\n\n var token = JsonSerializer.Deserialize(json, OpenAIJsonContext2.Default.LongRunContinuationToken)\n ?? throw new InvalidOperationException(\"Failed to deserialize LongRunContinuationToken.\");\n\n return token;\n }\n\n public string Serialize()\n {\n return JsonSerializer.Serialize(this, OpenAIJsonContext2.Default.LongRunContinuationToken);\n }\n}\n\npublic class ChatOptions\n{\n public string? ContinuationToken { get; set; }\n}\n```\n\n**Pro:** No dependency on the `System.ClientModel` package.\n\n##### 6.1.5 Continuation Token of a Custom Type\n\nThe option is similar the the \"6.1.3 Continuation Token of System.ClientModel.ContinuationToken Type\" option but suggests using a \ncustom type for the continuation token instead of the `System.ClientModel.ContinuationToken` type.\n\n**Pros**\n- There is no dependency on the `System.ClientModel` package. \n- There is no ambiguity between extension methods for `IChatClient` that would occur if a new extension method, which accepts a continuation token of string type as the first parameter, is added.\n\n#### 6.2 Overloads of GetResponseAsync and GetStreamingResponseAsync\n\nThis option proposes introducing overloads of the `GetResponseAsync` and `GetStreamingResponseAsync` methods that will accept long-running operation parameters directly:\n\n```csharp\npublic interface ILongRunningChatClient\n{\n Task GetResponseAsync(\n IEnumerable messages,\n string responseId,\n ChatOptions? options = null,\n CancellationToken cancellationToken = default);\n\n IAsyncEnumerable GetStreamingResponseAsync(\n IEnumerable messages,\n string responseId,\n string? startAfter = null,\n ChatOptions? options = null,\n CancellationToken cancellationToken = default);\n}\n\npublic class CustomChatClient : IChatClient, ILongRunningChatClient\n{\n ...\n}\n\n// Usage example\nIChatClient chatClient = ...; // Get an instance of IChatClient\n\nChatResponse response = await chatClient.GetResponseAsync(\"\", new ChatOptions { AllowLongRunningResponses = true });\n\nif(response.Status is {} status && chatClient.GetService() is {} longRunningChatClient)\n{\n while(status != AsyncRunStatus.Completed)\n {\n response = await longRunningChatClient.GetResponseAsync([], response.ResponseId, new ChatOptions { ConversationId = response.ConversationId });\n }\n ...\n}\n\n```\n\n**Pros:**\n- No proliferation of long-running operation properties in the ChatOptions class, except for the new AllowLongRunningResponses property discussed in section 2.\n\n**Cons:**\n- Interface switching: Callers need to switch to the `ILongRunningChatClient` interface to get the status and result of long-running operations.\n- An alternative solution for decorating the new methods will have to be put in place.\n\n## Long-Running Operations Support for AF Agents\n\n### 1. Methods for Working with Long-Running Operations\n\nThe design for supporting long-running operations by agents is very similar to that for chat clients because it is based on \nthe same analysis of existing APIs and anticipated consumption patterns.\n\n#### 1.1 Run{Streaming}Async Methods for Common Operations and the Update Operation + New Method Per Uncommon Operation\n\nThis option suggests using the existing `Run{Streaming}Async` methods of the `AIAgent` interface implementations to start, get results, and update long-running operations.\n\nFor cancellation and deletion of long-running operations, new methods will be added to the `AIAgent` interface implementations.\n\n```csharp\npublic abstract class AIAgent\n{\n // Existing methods...\n public Task RunAsync(string message, AgentThread? thread = null, AgentRunOptions? options = null, CancellationToken cancellationToken = default) { ... }\n public IAsyncEnumerable RunStreamingAsync(string message, AgentThread? thread = null, AgentRunOptions? options = null, CancellationToken cancellationToken = default) { ... }\n\n // New methods for uncommon operations\n public virtual Task CancelRunAsync(string id, AgentCancelRunOptions? options = null, CancellationToken cancellationToken = default)\n {\n return Task.FromResult(null);\n }\n\n public virtual Task DeleteRunAsync(string id, AgentDeleteRunOptions? options = null, CancellationToken cancellationToken = default)\n {\n return Task.FromResult(null);\n }\n}\n\n// Agent that supports update and cancellation\npublic class CustomAgent : AIAgent\n{\n public override async Task CancelRunAsync(string id, AgentCancelRunOptions? options = null, CancellationToken cancellationToken = default)\n {\n var response = await this._client.CancelRunAsync(id, options?.Thread?.ConversationId);\n\n return ConvertToAgentResponse(response); \n }\n\n // No overload for DeleteRunAsync as it's not supported by the underlying API\n}\n\n// Usage\nAIAgent agent = new CustomAgent();\n\nAgentThread thread = agent.GetNewThread();\n\nAgentResponse response = await agent.RunAsync(\"What is the capital of France?\");\n\nresponse = await agent.CancelRunAsync(response.ResponseId, new AgentCancelRunOptions { Thread = thread });\n```\n\nIn case an agent supports either or both cancellation and deletion of long-running operations, it will override the corresponding methods.\nOtherwise, it won't override them, and the base implementations will return null by default.\n\nSome agents, for example Azure AI Foundry Agents, require the thread identifier to cancel a run. To accommodate this requirement, the `CancelRunAsync` method\naccepts an optional `AgentCancelRunOptions` parameter that allows callers to specify the thread associated with the run they want to cancel.\n\n```csharp\npublic class AgentCancelRunOptions\n{\n public AgentThread? Thread { get; set; }\n}\n```\n\nSimilar design considerations can be applied to the `DeleteRunAsync` method and the `AgentDeleteRunOptions` class.\n\nHaving options in the method signatures allows for future extensibility; however, they can be added later if needed to the method overloads.\n\n**Pros:**\n- Existing `Run{Streaming}Async` methods are reused for common operations.\n- New methods for uncommon operations can be added in a non-breaking way.\n\n### 2. Enabling Long-Running Operations\n\nThe options for enabling long-running operations are exactly the same as those discussed in section \"2. Enabling Long-Running Operations\" for chat clients:\n- Execution Mode per `Run{Streaming}Async` Invocation\n- Execution Mode per `Run{Streaming}Async` Invocation + Model Class\n- Execution Mode per agent instance\n- Combined Approach\n\nBelow are the details of the option selected for chat clients that is also selected for agents.\n\n#### 2.1 Execution Mode per `Run{Streaming}Async` Invocation\n\nThis option proposes adding a new nullable `AllowLongRunningResponses` property of bool type to the `AgentRunOptions` class.\nThe property value will be `true` if the caller requests a long-running operation, `false`, `null` or omitted otherwise.\n \nAI agents that work with APIs requiring explicit configuration per operation will use this property to determine whether to run the prompt as a long-running \noperation or quick prompt. Agents that work with APIs that don't require explicit configuration will ignore this property and operate according \nto their own logic/configuration.\n\n```csharp\npublic class AgentRunOptions\n{\n // Existing properties...\n public bool? AllowLongRunningResponses { get; set; }\n}\n\n// Consumer code example\nAIAgent agent = ...; // Get an instance of an AIAgent\n\n// Start a long-running execution for the prompt if supported by the underlying API\nAgentResponse response = await agent.RunAsync(\"\", new AgentRunOptions { AllowLongRunningResponses = true });\n\n// Start a quick prompt\nAgentResponse response = await agent.RunAsync(\"\");\n```\n\n**Pros:** \n- Callers can switch between quick prompts and long-running operations per invocation of the `Run{Streaming}Async` methods without \nchanging agent configuration.\n- Enables explicit control over the execution mode by callers per invocation, meaning that no caller site is broken if the agent is injected via DI, \nand the caller can turn on the long-running operation mode when it can handle it.\n\n**Con:** This may not be valuable for all callers, as they may not have enough information to decide whether the prompt should run as a long-running operation or quick prompt.\n\n### 3. Model To Support Long-Running Operations\n\nThe options for modeling long-running operations are exactly the same as those for chat clients discussed in section \"6. Model To Support Long-Running Operations\" above:\n- Direct Properties in ChatOptions\n- LongRunOptions Model Class\n- Continuation Token of System.ClientModel.ContinuationToken Type\n- Continuation Token of String Type\n- Continuation Token of a Custom Type\n\nBelow are the details of the option selected for chat clients that is also selected for agents.\n \n#### 3.1 Continuation Token of a Custom Type\n\nThis option suggests using `ContinuationToken` to encapsulate all properties representing a long-running operation. The continuation token will be returned by agents in the \n`ContinuationToken` property of the `AgentResponse` and `AgentResponseUpdate` responses to indicate that the response is part of a long-running operation. A null value \nof the property will indicate that the response is not part of a long-running operation or the long-running operation has been completed. Callers will set the token in the\n`ContinuationToken` property of the `AgentRunOptions` class in follow-up calls to the `Run{Streaming}Async` methods to indicate that they want to \"continue\" the long-running\noperation identified by the token.\n\nEach agent will implement its own continuation token class that inherits from `ContinuationToken` to encapsulate properties required for long-running operations that are\nspecific to the underlying API the agent works with. For example, for the A2A agent, the continuation token class will encapsulate the `TaskId` property.\n\n```csharp\ninternal sealed class A2AAgentContinuationToken : ResponseContinuationToken\n{\n public A2AAgentContinuationToken(string taskId)\n {\n this.TaskId = taskId;\n }\n\n public string TaskId { get; set; }\n\n public static LongRunContinuationToken FromToken(ContinuationToken token)\n {\n if (token is LongRunContinuationToken longRunContinuationToken)\n {\n return longRunContinuationToken;\n }\n\n ... // Deserialization logic\n }\n}\n\npublic class AgentRunOptions\n{\n public ResponseContinuationToken? ContinuationToken { get; set; }\n}\n\npublic class AgentResponse\n{\n public ResponseContinuationToken? ContinuationToken { get; }\n}\n \npublic class AgentResponseUpdate\n{\n public ResponseContinuationToken? ContinuationToken { get; }\n}\n\n// Usage example\nAgentResponse response = await agent.RunAsync(\"What is the capital of France?\");\n\nAgentRunOptions options = new() { ContinuationToken = response.ContinuationToken };\n\nwhile (response.ContinuationToken is { } token)\n{\n options.ContinuationToken = token;\n response = await agent.RunAsync([], options);\n}\n\nConsole.WriteLine(response.Text);\n```\n\n### 4. Continuation Token and Agent Thread\n\nThere are two types of agent threads: server-managed and client-managed. The server-managed threads live server-side and are identified by a conversation identifier, and \nagents use the identifier to associate runs with the threads. The client-managed threads live client-side and are represented by a collection of chat messages that agents maintain\nby adding user messages to them before sending the thread to the service and by adding the agent response back to the thread when received from the service.\n\nWhen long-running operations are enabled and an agent is configured with tools, the initial run response may contain a tool call that needs to be invoked by the agent. If the agent runs\nwith a server-managed thread, the tool call will be captured as part of the conversation history server-side and follow-up runs will have access to it, and as a result the agent will invoke the tool.\nHowever, if no thread is provided at the agent's initial run and a client-managed thread is provided for follow-up runs and the agent calls a tool, the tool call which the agent made \nat the initial run will not be added to the client-managed thread since the initial run was made with no thread, and as a result the agent will not be able to invoke the tool.\n\n#### 4.1 Require Thread for Long-Running Operations\n\nThis option suggests that AI agents require a thread to be provided when long-running operations are enabled. If no thread is provided, the agent will throw an exception.\n\n**Pro:** Ensures agent responses are always captured by client-managed threads when long-running operations are enabled, providing a consistent experience for callers.\n\n**Con:** May be inconvenient for callers to always provide a thread when long-running operations are enabled.\n\n#### 4.2 Don't Require Thread for Long-Running Operations\n\nThis option suggests that AI agents don't require a thread to be provided when long-running operations are enabled. According to this option, it's up to the caller to ensure that\nthe thread is provided with background operations consistently for all runs.\n\n**Pro:** Provides more flexibility to callers by not enforcing thread requirements.\n\n**Con:** May lead to an inconsistent experience for callers if they forget to provide the thread for initial or follow-up runs.\n\n## Decision Outcome\n\n### Long-Running Execution Support for Chat Clients\n- **Methods**: Option 1.4 - Use existing `Get{Streaming}ResponseAsync` for common operations; individual interfaces for uncommon operations (e.g., `ICancelableChatClient`)\n- **Enabling**: Option 2.1 - Execution mode per invocation via `ChatOptions.AllowLongRunningResponses`\n- **Status/Result**: Option 3.2 - Single method to get both status and result\n- **RunId/UpdateId**: Option 4.2 - As properties of `ChatResponse{Update}`\n- **Model**: Option 6.1.5 - Custom continuation token type\n\n### Long-Running Operations Support for AF Agents\n- **Methods**: Option 1.1 - Use existing `Run{Streaming}Async` for common operations; new methods for uncommon operations\n- **Enabling**: Option 2.1 - Execution mode per invocation via `AgentRunOptions.AllowLongRunningResponses`\n- **Model**: Option 3.1 - Custom continuation token type\n- **Thread Requirement**: Option 4.1 - Require thread for long-running operations\n\n## Addendum 1: APIs of Agents Supporting Long-Running Execution\n\nOpenAI Responses
\n\n- Create a background response and wait for it to complete using polling:\n ```csharp\n ClientResult result = await this._openAIResponseClient.CreateResponseAsync(\"What is SLM in AI?\", new ResponseCreationOptions\n {\n Background = true,\n });\n\n // InProgress, Completed, Cancelled, Queued, Incomplete, Failed\n while (result.Value.Status is (ResponseStatus.Queued or ResponseStatus.InProgress))\n {\n Thread.Sleep(500); // Wait for 0.5 seconds before checking the status again\n result = await this._openAIResponseClient.GetResponseAsync(result.Value.Id);\n }\n\n Console.WriteLine($\"Response Status: {result.Value.Status}\"); // Completed\n Console.WriteLine(result.Value.GetOutputText()); // SLM in the context of AI refers to ...\n ```\n\n- Cancel a background response:\n ```csharp\n ...\n ClientResult result = await this._openAIResponseClient.CreateResponseAsync(\"What is SLM in AI?\", new ResponseCreationOptions\n {\n Background = true,\n });\n\n result = await this._openAIResponseClient.CancelResponseAsync(result.Value.Id);\n\n Console.WriteLine($\"Response Status: {result.Value.Status}\"); // Cancelled\n ```\n\n- Delete a background response:\n ```csharp\n ClientResult result = await this._openAIResponseClient.CreateResponseAsync(\"What is SLM in AI?\", new ResponseCreationOptions\n {\n Background = true,\n });\n\n ClientResult deleteResult = await this._openAIResponseClient.DeleteResponseAsync(result.Value.Id);\n\n Console.WriteLine($\"Response Deleted: {deleteResult.Value.Deleted}\"); // True if the response was deleted successfully\n ```\n\n- Streaming a background response\n ```csharp\n await foreach (StreamingResponseUpdate update in this._openAIResponseClient.CreateResponseStreamingAsync(\"What is SLM in AI?\", new ResponseCreationOptions { Background = true }))\n {\n Console.WriteLine($\"Sequence Number: {update.SequenceNumber}\"); // 0, 1, 2, etc.\n\n switch (update)\n {\n case StreamingResponseCreatedUpdate createdUpdate:\n Console.WriteLine($\"Response Status: {createdUpdate.Response.Status}\"); // Queued\n break;\n case StreamingResponseQueuedUpdate queuedUpdate:\n Console.WriteLine($\"Response Status: {queuedUpdate.Response.Status}\"); // Queued\n break;\n case StreamingResponseInProgressUpdate inProgressUpdate:\n Console.WriteLine($\"Response Status: {inProgressUpdate.Response.Status}\"); // InProgress\n break;\n case StreamingResponseOutputItemAddedUpdate outputItemAddedUpdate:\n Console.WriteLine($\"Output index: {outputItemAddedUpdate.OutputIndex}\");\n Console.WriteLine($\"Item Id: {outputItemAddedUpdate.Item.Id}\");\n break;\n case StreamingResponseContentPartAddedUpdate contentPartAddedUpdate:\n Console.WriteLine($\"Output Index: {contentPartAddedUpdate.OutputIndex}\");\n Console.WriteLine($\"Item Id: {contentPartAddedUpdate.ItemId}\");\n Console.WriteLine($\"Content Index: {contentPartAddedUpdate.ContentIndex}\");\n break;\n case StreamingResponseOutputTextDeltaUpdate outputTextDeltaUpdate:\n Console.WriteLine($\"Output Index: {outputTextDeltaUpdate.OutputIndex}\");\n Console.WriteLine($\"Item Id: {outputTextDeltaUpdate.ItemId}\");\n Console.WriteLine($\"Content Index: {outputTextDeltaUpdate.ContentIndex}\");\n Console.WriteLine($\"Delta: {outputTextDeltaUpdate.Delta}\"); // SL>M> in> AI> typically>....\n break;\n case StreamingResponseOutputTextDoneUpdate outputTextDoneUpdate:\n Console.WriteLine($\"Output Index: {outputTextDoneUpdate.OutputIndex}\");\n Console.WriteLine($\"Item Id: {outputTextDoneUpdate.ItemId}\");\n Console.WriteLine($\"Content Index: {outputTextDoneUpdate.ContentIndex}\");\n Console.WriteLine($\"Text: {outputTextDoneUpdate.Text}\"); // SLM in the context of AI typically refers to ...\n break;\n case StreamingResponseContentPartDoneUpdate contentPartDoneUpdate:\n Console.WriteLine($\"Output Index: {contentPartDoneUpdate.OutputIndex}\");\n Console.WriteLine($\"Item Id: {contentPartDoneUpdate.ItemId}\");\n Console.WriteLine($\"Content Index: {contentPartDoneUpdate.ContentIndex}\");\n Console.WriteLine($\"Text: {contentPartDoneUpdate.Part.Text}\"); // SLM in the context of AI typically refers to ...\n break;\n case StreamingResponseOutputItemDoneUpdate outputItemDoneUpdate:\n Console.WriteLine($\"Output Index: {outputItemDoneUpdate.OutputIndex}\");\n Console.WriteLine($\"Item Id: {outputItemDoneUpdate.Item.Id}\");\n break;\n case StreamingResponseCompletedUpdate completedUpdate:\n Console.WriteLine($\"Response Status: {completedUpdate.Response.Status}\"); // Completed\n Console.WriteLine($\"Output: {completedUpdate.Response.GetOutputText()}\"); // SLM in the context of AI typically refers to ...\n break;\n default:\n Console.WriteLine($\"Unexpected update type: {update.GetType().Name}\");\n break;\n }\n }\n ```\n\n Docs: [OpenAI background mode](https://platform.openai.com/docs/guides/background)\n \n- Background Mode Disabled\n\n - Non-streaming API - returns the final result\n | Method Call | Status | Result | Notes |\n |-------------------------------------|-----------|---------------------------------|-------------------------------------|\n | CreateResponseAsync(msgs, opts, ct) | Completed | The capital of France is Paris. | |\n | GetResponseAsync(responseId, ct) | Completed | The capital of France is Paris. | response is less than 5 minutes old |\n | GetResponseAsync(responseId, ct) | Completed | The capital of France is Paris. | response is more than 5 minutes old |\n | GetResponseAsync(responseId, ct) | Completed | The capital of France is Paris. | response is more than 12 hours old |\n \n | Cancellation Method | Result |\n |---------------------|--------------------------------------|\n | CancelResponseAsync | Cannot cancel a synchronous response |\n\n - Streaming API - returns streaming updates callers can iterate over to get the result\n | Method Call | Status | Result |\n |----------------------------------------------|------------|----------------------------------------------------------------------------------|\n | CreateResponseStreamingAsync(msgs, opts, ct) | - | updates |\n | Iterating over updates | InProgress | - |\n | Iterating over updates | InProgress | - |\n | Iterating over updates | InProgress | The |\n | Iterating over updates | InProgress | capital |\n | Iterating over updates | InProgress | ... |\n | Iterating over updates | InProgress | Paris. |\n | Iterating over updates | Completed | The capital of France is Paris. |\n | GetStreamingResponseAsync(responseId, ct) | - | HTTP 400 - Response cannot be streamed, it was not created with background=true. |\n \n | Cancellation Method | Result |\n |---------------------|--------------------------------------|\n | CancelResponseAsync | Cannot cancel a synchronous response |\n \n- Background Mode Enabled\n \n - Non-streaming API - returns queued response immediately and allow polling for the status and result\n | Method Call | Status | Result | Notes |\n |-------------------------------------|-----------|---------------------------------|--------------------------------------------|\n | CreateResponseAsync(msgs, opts, ct) | Queued | responseId | |\n | GetResponseAsync(responseId, ct) | Queued | - | if called before the response is completed |\n | GetResponseAsync(responseId, ct) | Queued | - | if called before the response is completed |\n | GetResponseAsync(responseId, ct) | Completed | The capital of France is Paris. | response is less than 5 minutes old |\n | GetResponseAsync(responseId, ct) | Completed | The capital of France is Paris. | response is more than 5 minutes old |\n | GetResponseAsync(responseId, ct) | Completed | The capital of France is Paris. | response is more than 12 hours old |\n\n The response started in background mode runs server-side until it completes, fails, or is cancelled. The client can poll for\n the status of the response using its Id. If the client polls before the response is completed, it will get the latest status of the response.\n If the client polls after the response is completed, it will get the completed response with the result.\n \n | Cancellation Method | Result | Notes |\n |---------------------|-----------|----------------------------------------|\n | CancelResponseAsync | Cancelled | if cancelled before response completed |\n | CancelResponseAsync | Completed | if cancelled after response completed |\n | CancellationToken | No effect | it just cancels the client side call |\n\n - Streaming API - returns streaming updates callers can iterate over immediately or after dropping the stream and picking it up later\n | Method Call | Status | Result | Notes |\n |----------------------------------------------|------------|--------------------------------------------------------------------------------|-------------------------------------------|\n | CreateResponseStreamingAsync(msgs, opts, ct) | - | updates | |\n | Iterating over updates | Queued | - | |\n | Iterating over updates | Queued | - | |\n | Iterating over updates | InProgress | - | |\n | Iterating over updates | InProgress | - | |\n | Iterating over updates | InProgress | The | |\n | Iterating over updates | InProgress | capital | |\n | Iterating over updates | InProgress | ... | |\n | Iterating over updates | InProgress | Paris. | |\n | Iterating over updates | Completed | The capital of France is Paris. | |\n | GetStreamingResponseAsync(responseId, ct) | - | updates | response is less than 5 minutes old |\n | Iterating over updates | Queued | - | |\n | ... \t\t\t\t\t\t\t\t\t | ... | ... | |\n | GetStreamingResponseAsync(responseId, ct) | - | HTTP 400 - Response can no longer be streamed, it is more than 5 minutes old. | response is more than 5 minutes old |\n | GetResponseAsync(responseId, ct)\t | Completed | The capital of France is Paris. | accessing response that can't be streamed |\n \n The streamed response that is not available after 5 minutes can be retrieved using the non-streaming API `GetResponseAsync`.\n \n | Cancellation Method | Result | Notes |\n |---------------------|------------------------------------|----------------------------------------|\n | CancelResponseAsync | Canceled1 | if cancelled before response completed |\n | CancelResponseAsync | Cannot cancel a completed response | if cancelled after response completed |\n | CancellationToken | No effect | it just cancels the client side call |\n\n 1 The CancelResponseAsync method returns `Canceled` status, but a subsequent call to GetResponseStreamingAsync returns \n an enumerable that can be iterated over to get the rest of the response until it completes.\n \n \n\n\nAzure AI Foundry Agents
\n\n- Create a thread and run the agent against it and wait for it to complete using polling:\n ```csharp\n // Create a thread with a message.\n ThreadMessageOptions options = new(MessageRole.User, \"What is SLM in AI?\");\n thread = await this._persistentAgentsClient!.Threads.CreateThreadAsync([options]);\n\n // Run the agent on the thread.\n ThreadRun threadRun = await this._persistentAgentsClient.Runs.CreateRunAsync(thread.Id, agent.Id);\n\n // Poll for the run status.\n // InProgress, Completed, Cancelling, Cancelled, Queued, Failed, RequiresAction, Expired\n while (threadRun.Status == RunStatus.InProgress || threadRun.Status == RunStatus.Queued)\n {\n threadRun = await this._persistentAgentsClient.Runs.GetRunAsync(thread.Id, threadRun.Id);\n }\n\n // Access the run result.\n await foreach (PersistentThreadMessage msg in this._persistentAgentsClient.Messages.GetMessagesAsync(thread.Id, threadRun.Id))\n {\n foreach (MessageContent content in msg.ContentItems)\n {\n switch (content)\n {\n case MessageTextContent textItem:\n Console.WriteLine($\" Text: {textItem.Text}\");\n //M1: In the context of Artificial Intelligence (AI), **SLM** often ...\n //M2: What is SLM in AI?\n break;\n }\n }\n }\n ```\n\n- Cancel an agent run:\n ```csharp\n // Create a thread with a message.\n ThreadMessageOptions options = new(MessageRole.User, \"What is SLM in AI?\");\n thread = await this._persistentAgentsClient!.Threads.CreateThreadAsync([options]);\n\n // Run the agent on the thread.\n ThreadRun threadRun = await this._persistentAgentsClient.Runs.CreateRunAsync(thread.Id, agent.Id);\n\n Response cancellationResponse = await this._persistentAgentsClient.Runs.CancelRunAsync(thread.Id, threadRun.Id);\n ```\n\n- Other agent run operations:\n GetRunStepAsync\n\n \n\n\nA2A Agents
\n\n- Send message to agent and handle the response\n ```csharp\n // Send message to the A2A agent.\n A2AResponse response = await this.Client.SendMessageAsync(messageSendParams, cancellationToken).ConfigureAwait(false);\n\n // Handle task responses.\n if (response is AgentTask task)\n {\n while (task.Status.State == TaskState.Working)\n {\n task = await this.Client.GetTaskAsync(task.Id, cancellationToken).ConfigureAwait(false);\n }\n\n if (task.Artifacts != null && task.Artifacts.Count > 0)\n {\n foreach (var artifact in task.Artifacts)\n {\n foreach (var part in artifact.Parts)\n {\n if (part is TextPart textPart)\n {\n Console.WriteLine($\"Result: {textPart.Text}\");\n }\n }\n }\n Console.WriteLine();\n }\n }\n // Handle message responses.\n else if (response is Message message)\n {\n foreach (var part in message.Parts)\n {\n if (part is TextPart textPart)\n {\n Console.WriteLine($\"Result: {textPart.Text}\");\n }\n }\n }\n else\n {\n throw new InvalidOperationException(\"Unexpected response type from A2A client.\");\n }\n ```\n\n- Cancel task\n ```csharp\n // Send message to the A2A agent.\n A2AResponse response = await this.Client.SendMessageAsync(messageSendParams, cancellationToken).ConfigureAwait(false);\n\n // Cancel the task\n if (response is AgentTask task)\n {\n await this.Client.CancelTaskAsync(new TaskIdParams() { Id = task.Id }, cancellationToken).ConfigureAwait(false);\n }\n ```\n\n "
},
{
"path": "docs/decisions/0010-ag-ui-support.md",
"content": "---\nstatus: accepted\ncontact: javiercn\ndate: 2025-10-29\ndeciders: javiercn, DeagleGross, moonbox3, markwallace-microsoft\nconsulted: Agent Framework team\ninformed: .NET community\n---\n\n# AG-UI Protocol Support for .NET Agent Framework\n\n## Context and Problem Statement\n\nThe .NET Agent Framework needed a standardized way to enable communication between AI agents and user-facing applications with support for streaming, real-time updates, and bidirectional communication. Without AG-UI protocol support, .NET agents could not interoperate with the growing ecosystem of AG-UI-compatible frontends and agent frameworks (LangGraph, CrewAI, Pydantic AI, etc.), limiting the framework's adoption and utility.\n\nThe AG-UI (Agent-User Interaction) protocol is an open, lightweight, event-based protocol that addresses key challenges in agentic applications including streaming support for long-running agents, event-driven architecture for nondeterministic behavior, and protocol interoperability that complements MCP (tool/context) and A2A (agent-to-agent) protocols.\n\n## Decision Drivers\n\n- Need for streaming communication between agents and client applications\n- Requirement for protocol interoperability with other AI frameworks\n- Support for long-running, multi-turn conversation sessions\n- Real-time UI updates for nondeterministic agent behavior\n- Standardized approach to agent-to-UI communication\n- Framework abstraction to protect consumers from protocol changes\n\n## Considered Options\n\n1. **Implement AG-UI event types as public API surface** - Expose AG-UI event models directly to consumers\n2. **Use custom AIContent types for lifecycle events** - Create new content types (RunStartedContent, RunFinishedContent, RunErrorContent)\n3. **Current approach** - Internal event types with framework-native abstractions\n\n## Decision Outcome\n\nChosen option: \"Current approach with internal event types and framework-native abstractions\", because it:\n\n- Protects consumers from protocol changes by keeping AG-UI events internal\n- Maintains framework abstractions through conversion at boundaries\n- Uses existing framework types (AgentResponseUpdate, ChatMessage) for public API\n- Focuses on core text streaming functionality\n- Leverages existing properties (ConversationId, ResponseId, ErrorContent) instead of custom types\n- Provides bidirectional client and server support\n\n### Implementation Details\n\n**In Scope:**\n1. **Client-side AG-UI consumption** (`Microsoft.Agents.AI.AGUI` package)\n - `AGUIAgent` class for connecting to remote AG-UI servers\n - `AGUIAgentThread` for managing conversation threads\n - HTTP/SSE streaming support\n - Event-to-framework type conversion\n\n2. **Server-side AG-UI hosting** (`Microsoft.Agents.AI.Hosting.AGUI.AspNetCore` package)\n - `MapAGUIAgent` extension method for ASP.NET Core\n - Server-Sent Events (SSE) response formatting\n - Framework-to-event type conversion\n - Agent factory pattern for per-request instantiation\n\n3. **Text streaming events**\n - Lifecycle events: `RunStarted`, `RunFinished`, `RunError`\n - Text message events: `TextMessageStart`, `TextMessageContent`, `TextMessageEnd`\n - Thread and run ID management via `ConversationId` and `ResponseId`\n\n### Key Design Decisions\n\n1. **Event Models as Internal Types** - AG-UI event types are internal with conversion via extension methods; public API uses the existing types in Microsoft.Extensions.AI as those are the abstractions people are familiar with\n\n2. **No Custom Content Types** - Run lifecycle communicated through existing `ChatResponseUpdate` properties (`ConversationId`, `ResponseId`) and standard `ErrorContent` type\n\n3. **Agent Factory Pattern** - `MapAGUIAgent` uses factory function `(messages) => AIAgent` to allow request-specific agent configuration supporting multi-tenancy\n\n4. **Bidirectional Conversion Architecture** - Symmetric conversion logic in shared namespace compiled into both packages for server (`AgentResponseUpdate` → AG-UI events) and client (AG-UI events → `AgentResponseUpdate`)\n\n5. **Thread Management** - `AGUIAgentThread` stores only `ThreadId` with thread ID communicated via `ConversationId`; applications manage persistence for parity with other implementations and to be compliant with the protocol. Future extensions will support having the server manage the conversation.\n\n6. **Custom JSON Converter** - Uses custom polymorphic deserialization via `BaseEventJsonConverter` instead of built-in System.Text.Json support to handle AG-UI protocol's flexible discriminator positioning\n\n### Consequences\n\n**Positive:**\n- .NET developers can consume AG-UI servers from any framework\n- .NET agents accessible from any AG-UI-compatible client\n- Standardized streaming communication patterns\n- Protected from protocol changes through internal implementation\n- Symmetric conversion logic between client and server\n- Framework-native public API surface\n\n**Negative:**\n- Custom JSON converter required (internal implementation detail)\n- Shared code uses preprocessor directives (`#if ASPNETCORE`)\n- Additional abstraction layer between protocol and public API\n\n**Neutral:**\n- Initial implementation focused on text streaming\n- Applications responsible for thread persistence\n"
},
{
"path": "docs/decisions/0011-create-get-agent-api.md",
"content": "---\nstatus: proposed\ncontact: dmytrostruk\ndate: 2025-12-12\ndeciders: dmytrostruk, markwallace-microsoft, eavanvalkenburg, giles17\n---\n\n# Create/Get Agent API\n\n## Context and Problem Statement\n\nThere is a misalignment between the create/get agent API in the .NET and Python implementations.\n\nIn .NET, the `CreateAIAgent` method can create either a local instance of an agent or a remote instance if the backend provider supports it. For remote agents, once the agent is created, you can retrieve an existing remote agent by using the `GetAIAgent` method. If a backend provider doesn't support remote agents, `CreateAIAgent` just initializes a new local agent instance and `GetAIAgent` is not available. There is also a `BuildAIAgent` method, which is an extension for the `ChatClientBuilder` class from `Microsoft.Extensions.AI`. It builds pipelines of `IChatClient` instances with an `IServiceProvider`. This functionality does not exist in Python, so `BuildAIAgent` is out of scope.\n\nIn Python, there is only one `create_agent` method, which always creates a local instance of the agent. If the backend provider supports remote agents, the remote agent is created only on the first `agent.run()` invocation.\n\nBelow is a short summary of different providers and their APIs in .NET:\n\n| Package | Method | Behavior | Python support |\n|---|---|---|---|\n| Microsoft.Agents.AI | `CreateAIAgent` (based on `IChatClient`) | Creates a local instance of `ChatClientAgent`. | Yes (`create_agent` in `BaseChatClient`). |\n| Microsoft.Agents.AI.Anthropic | `CreateAIAgent` (based on `IBetaService` and `IAnthropicClient`) | Creates a local instance of `ChatClientAgent`. | Yes (`AnthropicClient` inherits `BaseChatClient`, which exposes `create_agent`). |\n| Microsoft.Agents.AI.AzureAI (V2) | `GetAIAgent` (based on `AIProjectClient` with `AgentReference`) | Creates a local instance of `ChatClientAgent`. | Partial (Python uses `create_agent` from `BaseChatClient`). |\n| Microsoft.Agents.AI.AzureAI (V2) | `GetAIAgent`/`GetAIAgentAsync` (with `Name`/`ChatClientAgentOptions`) | Fetches `AgentRecord` via HTTP, then creates a local `ChatClientAgent` instance. | No |\n| Microsoft.Agents.AI.AzureAI (V2) | `CreateAIAgent`/`CreateAIAgentAsync` (based on `AIProjectClient`) | Creates a remote agent first, then wraps it into a local `ChatClientAgent` instance. | No |\n| Microsoft.Agents.AI.AzureAI.Persistent (V1) | `GetAIAgent` (based on `PersistentAgentsClient` with `PersistentAgent`) | Creates a local instance of `ChatClientAgent`. | Partial (Python uses `create_agent` from `BaseChatClient`). |\n| Microsoft.Agents.AI.AzureAI.Persistent (V1) | `GetAIAgent`/`GetAIAgentAsync` (with `AgentId`) | Fetches `PersistentAgent` via HTTP, then creates a local `ChatClientAgent` instance. | No |\n| Microsoft.Agents.AI.AzureAI.Persistent (V1) | `CreateAIAgent`/`CreateAIAgentAsync` | Creates a remote agent first, then wraps it into a local `ChatClientAgent` instance. | No |\n| Microsoft.Agents.AI.OpenAI | `GetAIAgent` (based on `AssistantClient` with `Assistant`) | Creates a local instance of `ChatClientAgent`. | Partial (Python uses `create_agent` from `BaseChatClient`). |\n| Microsoft.Agents.AI.OpenAI | `GetAIAgent`/`GetAIAgentAsync` (with `AgentId`) | Fetches `Assistant` via HTTP, then creates a local `ChatClientAgent` instance. | No |\n| Microsoft.Agents.AI.OpenAI | `CreateAIAgent`/`CreateAIAgentAsync` (based on `AssistantClient`) | Creates a remote agent first, then wraps it into a local `ChatClientAgent` instance. | No |\n| Microsoft.Agents.AI.OpenAI | `CreateAIAgent` (based on `ChatClient`) | Creates a local instance of `ChatClientAgent`. | Yes (`create_agent` in `BaseChatClient`). |\n| Microsoft.Agents.AI.OpenAI | `CreateAIAgent` (based on `OpenAIResponseClient`) | Creates a local instance of `ChatClientAgent`. | Yes (`create_agent` in `BaseChatClient`). |\n\nAnother difference between Python and .NET implementation is that in .NET `CreateAIAgent`/`GetAIAgent` methods are implemented as extension methods based on underlying SDK client, like `AIProjectClient` from Azure AI or `AssistantClient` from OpenAI:\n\n```csharp\n// Definition\npublic static ChatClientAgent CreateAIAgent(\n this AIProjectClient aiProjectClient,\n string name,\n string model,\n string instructions,\n string? description = null,\n IList? tools = null,\n Func? clientFactory = null,\n IServiceProvider? services = null,\n CancellationToken cancellationToken = default)\n{ }\n\n// Usage\nAIProjectClient aiProjectClient = new(new Uri(endpoint), new AzureCliCredential()); // Initialization of underlying SDK client\n\nvar newAgent = await aiProjectClient.CreateAIAgentAsync(name: AgentName, model: deploymentName, instructions: AgentInstructions, tools: [tool]); // ChatClientAgent creation from underlying SDK client\n\n// Alternative usage (same as extension method, just explicit syntax)\nvar newAgent = await AzureAIProjectChatClientExtensions.CreateAIAgentAsync(\n aiProjectClient,\n name: AgentName,\n model: deploymentName,\n instructions: AgentInstructions,\n tools: [tool]);\n```\n\nPython doesn't support extension methods. Currently `create_agent` method is defined on `BaseChatClient`, but this method only creates a local instance of `ChatAgent` and it can't create remote agents for providers that support it for a couple of reasons:\n\n- It's defined as non-async.\n- `BaseChatClient` implementation is stateful for providers like Azure AI or OpenAI Assistants. The implementation stores agent/assistant metadata like `AgentId` and `AgentName`, so currently it's not possible to create different instances of `ChatAgent` from a single `BaseChatClient` in case if the implementation is stateful.\n\n## Decision Drivers\n\n- API should be aligned between .NET and Python.\n- API should be intuitive and consistent between backend providers in .NET and Python.\n\n## Considered Options\n\nAdd missing implementations on the Python side. This should include the following:\n\n### agent-framework-azure-ai (both V1 and V2)\n\n- Add a `get_agent` method that accepts an underlying SDK agent instance and creates a local instance of `ChatAgent`.\n- Add a `get_agent` method that accepts an agent identifier, performs an additional HTTP request to fetch agent data, and then creates a local instance of `ChatAgent`.\n- Override the `create_agent` method from `BaseChatClient` to create a remote agent instance and wrap it into a local `ChatAgent`.\n\n.NET:\n\n```csharp\nvar agent1 = new AIProjectClient(...).GetAIAgent(agentInstanceFromSdkType); // Creates a local ChatClientAgent instance from Azure.AI.Projects.OpenAI.AgentReference \nvar agent2 = new AIProjectClient(...).GetAIAgent(agentName); // Fetches agent data, creates a local ChatClientAgent instance\nvar agent3 = new AIProjectClient(...).CreateAIAgent(...); // Creates a remote agent, returns a local ChatClientAgent instance\n```\n\n### agent-framework-core (OpenAI Assistants)\n\n- Add a `get_agent` method that accepts an underlying SDK agent instance and creates a local instance of `ChatAgent`.\n- Add a `get_agent` method that accepts an agent name, performs an additional HTTP request to fetch agent data, and then creates a local instance of `ChatAgent`.\n- Override the `create_agent` method from `BaseChatClient` to create a remote agent instance and wrap it into a local `ChatAgent`.\n\n.NET:\n\n```csharp\nvar agent1 = new AssistantClient(...).GetAIAgent(agentInstanceFromSdkType); // Creates a local ChatClientAgent instance from OpenAI.Assistants.Assistant\nvar agent2 = new AssistantClient(...).GetAIAgent(agentId); // Fetches agent data, creates a local ChatClientAgent instance\nvar agent3 = new AssistantClient(...).CreateAIAgent(...); // Creates a remote agent, returns a local ChatClientAgent instance\n```\n\n### Possible Python implementations\n\nMethods like `create_agent` and `get_agent` should be implemented separately or defined on some stateless component that will allow to create multiple agents from the same instance/place.\n\nPossible options:\n\n#### Option 1: Module-level functions\n\nImplement free functions in the provider package that accept the underlying SDK client as the first argument (similar to .NET extension methods, but expressed in Python).\n\nExample:\n\n```python\nfrom agent_framework.azure import create_agent, get_agent\n\nai_project_client = AIProjectClient(...)\n\n# Creates a remote agent first, then returns a local ChatAgent wrapper\ncreated_agent = await create_agent(\n ai_project_client,\n name=\"\",\n instructions=\"\",\n tools=[tool],\n)\n\n# Gets an existing remote agent and returns a local ChatAgent wrapper\nfirst_agent = await get_agent(ai_project_client, agent_id=agent_id)\n\n# Wraps an SDK agent instance (no extra HTTP call)\nsecond_agent = get_agent(ai_project_client, agent_reference)\n```\n\nPros:\n\n- Naturally supports async `create_agent` / `get_agent`.\n- Supports multiple agents per SDK client.\n- Closest conceptual match to .NET extension methods while staying Pythonic.\n\nCons:\n\n- Discoverability is lower (users need to know where the functions live).\n- Verbose when creating multiple agents (client must be passed every time):\n\n ```python\n agent1 = await azure_agents.create_agent(client, name=\"Agent1\", ...)\n agent2 = await azure_agents.create_agent(client, name=\"Agent2\", ...)\n ```\n\n#### Option 2: Provider object\n\nIntroduce a dedicated provider type that is constructed from the underlying SDK client, and exposes async `create_agent` / `get_agent` methods.\n\nExample:\n\n```python\nfrom agent_framework.azure import AzureAIAgentProvider\n\nai_project_client = AIProjectClient(...)\nprovider = AzureAIAgentProvider(ai_project_client)\n\nagent = await provider.create_agent(\n name=\"\",\n instructions=\"\",\n tools=[tool],\n)\n\nagent = await provider.get_agent(agent_id=agent_id)\nagent = provider.get_agent(agent_reference=agent_reference)\n```\n\nPros:\n\n- High discoverability and clear grouping of related behavior.\n- Keeps SDK clients unchanged and supports multiple agents per SDK client.\n- Concise when creating multiple agents (client passed once):\n\n ```python\n provider = AzureAIAgentProvider(ai_project_client)\n agent1 = await provider.create_agent(name=\"Agent1\", ...)\n agent2 = await provider.create_agent(name=\"Agent2\", ...)\n ```\n\nCons:\n\n- Adds a new public concept/type for users to learn.\n\n#### Option 3: Inheritance (SDK client subclass)\n\nCreate a subclass of the underlying SDK client and add `create_agent` / `get_agent` methods.\n\nExample:\n\n```python\nclass ExtendedAIProjectClient(AIProjectClient):\n async def create_agent(self, *, name: str, model: str, instructions: str, **kwargs) -> ChatAgent:\n ...\n\n async def get_agent(self, *, agent_id: str | None = None, sdk_agent=None, **kwargs) -> ChatAgent:\n ...\n\nclient = ExtendedAIProjectClient(...)\nagent = await client.create_agent(name=\"\", instructions=\"\")\n```\n\nPros:\n\n- Discoverable and ergonomic call sites.\n- Mirrors the .NET “methods on the client” feeling.\n\nCons:\n\n- Many SDK clients are not designed for inheritance; SDK upgrades can break subclasses.\n- Users must opt into subclass everywhere.\n- Typing/initialization can be tricky if the SDK client has non-trivial constructors.\n\n#### Option 4: Monkey patching\n\nAttach `create_agent` / `get_agent` methods to an SDK client class (or instance) at runtime.\n\nExample:\n\n```python\ndef _create_agent(self, *, name: str, model: str, instructions: str, **kwargs) -> ChatAgent:\n ...\n\nAIProjectClient.create_agent = _create_agent # monkey patch\n```\n\nPros:\n\n- Produces “extension method-like” call sites without wrappers or subclasses.\n\nCons:\n\n- Fragile across SDK updates and difficult to type-check.\n- Surprising behavior (global side effects), potential conflicts across packages.\n- Harder to support/debug, especially in larger apps and test suites.\n\n## Decision Outcome\n\nImplement `create_agent`/`get_agent`/`as_agent` API via **Option 2: Provider object**.\n\n### Rationale\n\n| Aspect | Option 1 (Functions) | Option 2 (Provider) |\n|--------|----------------------|---------------------|\n| Multiple implementations | One package may contain V1, V2, and other agent types. Function names like `create_agent` become ambiguous - which agent type does it create? | Each provider class is explicit: `AzureAIAgentsProvider` vs `AzureAIProjectAgentProvider` |\n| Discoverability | Users must know to import specific functions from the package | IDE autocomplete on provider instance shows all available methods |\n| Client reuse | SDK client must be passed to every function call: `create_agent(client, ...)`, `get_agent(client, ...)` | SDK client passed once at construction: `provider = Provider(client)` |\n\n**Option 1 example:**\n```python\nfrom agent_framework.azure import create_agent, get_agent\nagent1 = await create_agent(client, name=\"Agent1\", ...) # Which agent type, V1 or V2?\nagent2 = await create_agent(client, name=\"Agent2\", ...) # Repetitive client passing\n```\n\n**Option 2 example:**\n```python\nfrom agent_framework.azure import AzureAIProjectAgentProvider\nprovider = AzureAIProjectAgentProvider(client) # Clear which service, client passed once\nagent1 = await provider.create_agent(name=\"Agent1\", ...)\nagent2 = await provider.create_agent(name=\"Agent2\", ...)\n```\n\n### Method Naming\n\n| Operation | Python | .NET | Async |\n|-----------|--------|------|-------|\n| Create on service | `create_agent()` | `CreateAIAgent()` | Yes |\n| Get from service | `get_agent(id=...)` | `GetAIAgent(agentId)` | Yes |\n| Wrap SDK object | `as_agent(reference)` | `AsAIAgent(agentInstance)` | No |\n\nThe method names (`create_agent`, `get_agent`) do not explicitly mention \"service\" or \"remote\" because:\n- In Python, the provider class name explicitly identifies the service (`AzureAIAgentsProvider`, `OpenAIAssistantProvider`), making additional qualifiers in method names redundant.\n- In .NET, these are extension methods on `AIProjectClient` or `AssistantClient`, which already imply service operations.\n\n### Provider Class Naming\n\n| Package | Provider Class | SDK Client | Service |\n|---------|---------------|------------|---------|\n| `agent_framework.azure` | `AzureAIProjectAgentProvider` | `AIProjectClient` | Azure AI Agent Service, based on Responses API (V2) |\n| `agent_framework.azure` | `AzureAIAgentsProvider` | `AgentsClient` | Azure AI Agent Service (V1) |\n| `agent_framework.openai` | `OpenAIAssistantProvider` | `AsyncOpenAI` | OpenAI Assistants API |\n\n> **Note:** Azure AI naming is temporary. Final naming will be updated according to Azure AI / Microsoft Foundry renaming decisions.\n\n### Usage Examples\n\n#### Azure AI Agent Service V2 (based on Responses API)\n\n```python\nfrom agent_framework.azure import AzureAIProjectAgentProvider\nfrom azure.ai.projects import AIProjectClient\n\nclient = AIProjectClient(endpoint, credential)\nprovider = AzureAIProjectAgentProvider(client)\n\n# Create new agent on service\nagent = await provider.create_agent(name=\"MyAgent\", model=\"gpt-4\", instructions=\"...\")\n\n# Get existing agent by name\nagent = await provider.get_agent(agent_name=\"MyAgent\")\n\n# Wrap already-fetched SDK object (no HTTP calls)\nagent_ref = await client.agents.get(\"MyAgent\")\nagent = provider.as_agent(agent_ref)\n```\n\n#### Azure AI Persistent Agents V1\n\n```python\nfrom agent_framework.azure import AzureAIAgentsProvider\nfrom azure.ai.agents import AgentsClient\n\nclient = AgentsClient(endpoint, credential)\nprovider = AzureAIAgentsProvider(client)\n\nagent = await provider.create_agent(name=\"MyAgent\", model=\"gpt-4\", instructions=\"...\")\nagent = await provider.get_agent(agent_id=\"persistent-agent-456\")\nagent = provider.as_agent(persistent_agent)\n```\n\n#### OpenAI Assistants\n\n```python\nfrom agent_framework.openai import OpenAIAssistantProvider\nfrom openai import OpenAI\n\nclient = OpenAI()\nprovider = OpenAIAssistantProvider(client)\n\nagent = await provider.create_agent(name=\"MyAssistant\", model=\"gpt-4\", instructions=\"...\")\nagent = await provider.get_agent(assistant_id=\"asst_123\")\nagent = provider.as_agent(assistant)\n```\n\n#### Local-Only Agents (No Provider)\n\nCurrent method `create_agent` (python) / `CreateAIAgent` (.NET) can be renamed to `as_agent` (python) / `AsAIAgent` (.NET) to emphasize the conversion logic rather than creation/initialization logic and to avoid collision with `create_agent` method for remote calls.\n\n```python\nfrom agent_framework import ChatAgent\nfrom agent_framework.openai import OpenAIChatClient\n\n# Convert chat client to ChatAgent (no remote service involved)\nclient = OpenAIChatClient(model=\"gpt-4\")\nagent = client.as_agent(name=\"LocalAgent\", instructions=\"...\") # instead of create_agent\n```\n\n### Adding New Agent Types\n\nPython:\n\n1. Create provider class in appropriate package.\n2. Implement `create_agent`, `get_agent`, `as_agent` as applicable.\n\n.NET:\n\n1. Create static class for extension methods.\n2. Implement `CreateAIAgentAsync`, `GetAIAgentAsync`, `AsAIAgent` as applicable.\n"
},
{
"path": "docs/decisions/0012-python-typeddict-options.md",
"content": "---\n# These are optional elements. Feel free to remove any of them.\nstatus: proposed\ncontact: eavanvalkenburg\ndate: 2026-01-08\ndeciders: eavanvalkenburg, markwallace-microsoft, sphenry, alliscode, johanst, brettcannon\nconsulted: taochenosu, moonbox3, dmytrostruk, giles17\n---\n\n# Leveraging TypedDict and Generic Options in Python Chat Clients\n\n## Context and Problem Statement\n\nThe Agent Framework Python SDK provides multiple chat client implementations for different providers (OpenAI, Anthropic, Azure AI, Bedrock, Ollama, etc.). Each provider has unique configuration options beyond the common parameters defined in `ChatOptions`. Currently, developers using these clients lack type safety and IDE autocompletion for provider-specific options, leading to runtime errors and a poor developer experience.\n\nHow can we provide type-safe, discoverable options for each chat client while maintaining a consistent API across all implementations?\n\n## Decision Drivers\n\n- **Type Safety**: Developers should get compile-time/static analysis errors when using invalid options\n- **IDE Support**: Full autocompletion and inline documentation for all available options\n- **Extensibility**: Users should be able to define custom options that extend provider-specific options\n- **Consistency**: All chat clients should follow the same pattern for options handling\n- **Provider Flexibility**: Each provider can expose its unique options without affecting the common interface\n\n## Considered Options\n\n- **Option 1: Status Quo - Class `ChatOptions` with `**kwargs`**\n- **Option 2: TypedDict with Generic Type Parameters**\n\n### Option 1: Status Quo - Class `ChatOptions` with `**kwargs`\n\nThe current approach uses a base `ChatOptions` Class with common parameters, and provider-specific options are passed via `**kwargs` or loosely typed dictionaries.\n\n```python\n# Current usage - no type safety for provider-specific options\nresponse = await client.get_response(\n messages=messages,\n temperature=0.7,\n top_k=40,\n random=42, # No validation\n)\n```\n\n**Pros:**\n- Simple implementation\n- Maximum flexibility\n\n**Cons:**\n- No type checking for provider-specific options\n- No IDE autocompletion for available options\n- Runtime errors for typos or invalid options\n- Documentation must be consulted for each provider\n\n### Option 2: TypedDict with Generic Type Parameters (Chosen)\n\nEach chat client is parameterized with a TypeVar bound to a provider-specific `TypedDict` that extends `ChatOptions`. This enables full type safety and IDE support.\n\n```python\n# Provider-specific TypedDict\nclass AnthropicChatOptions(ChatOptions, total=False):\n \"\"\"Anthropic-specific chat options.\"\"\"\n top_k: int\n thinking: ThinkingConfig\n # ... other Anthropic-specific options\n\n# Generic chat client\nclass AnthropicChatClient(ChatClientBase[TAnthropicChatOptions]):\n ...\n\nclient = AnthropicChatClient(...)\n\n# Usage with full type safety\nresponse = await client.get_response(\n messages=messages,\n options={\n \"temperature\": 0.7,\n \"top_k\": 40,\n \"random\": 42, # fails type checking and IDE would flag this\n }\n)\n\n# Users can extend for custom options\nclass MyAnthropicOptions(AnthropicChatOptions, total=False):\n custom_field: str\n\n\nclient = AnthropicChatClient[MyAnthropicOptions](...)\n\n# Usage of custom options with full type safety\nresponse = await client.get_response(\n messages=messages,\n options={\n \"temperature\": 0.7,\n \"top_k\": 40,\n \"custom_field\": \"value\",\n }\n)\n\n```\n\n**Pros:**\n- Full type safety with static analysis\n- IDE autocompletion for all options\n- Compile-time error detection\n- Self-documenting through type hints\n- Users can extend options for their specific needs or advances in models\n\n**Cons:**\n- More complex implementation\n- Some type: ignore comments needed for TypedDict field overrides\n- Minor: Requires TypeVar with default (Python 3.13+ or typing_extensions)\n\n> [NOTE!]\n> In .NET this is already achieved through overloads on the `GetResponseAsync` method for each provider-specific options class, e.g., `AnthropicChatOptions`, `OpenAIChatOptions`, etc. So this does not apply to .NET.\n\n### Implementation Details\n\n1. **Base Protocol**: `ChatClientProtocol[TOptions]` is generic over options type, with default set to `ChatOptions` (the new TypedDict)\n2. **Provider TypedDicts**: Each provider defines its options extending `ChatOptions`\n They can even override fields with type=None to indicate they are not supported.\n3. **TypeVar Pattern**: `TProviderOptions = TypeVar(\"TProviderOptions\", bound=TypedDict, default=ProviderChatOptions, contravariant=True)`\n4. **Option Translation**: Common options are kept in place,and explicitly documented in the Options class how they are used. (e.g., `user` → `metadata.user_id`) in `_prepare_options` (for Anthropic) to preserve easy use of common options.\n\n## Decision Outcome\n\nChosen option: **\"Option 2: TypedDict with Generic Type Parameters\"**, because it provides full type safety, excellent IDE support with autocompletion, and allows users to extend provider-specific options for their use cases. Extended this Generic to ChatAgents in order to also properly type the options used in agent construction and run methods.\n\nSee [typed_options.py](../../python/samples/02-agents/typed_options.py) for a complete example demonstrating the usage of typed options with custom extensions.\n"
},
{
"path": "docs/decisions/0013-python-get-response-simplification.md",
"content": "---\nstatus: Accepted\ncontact: eavanvalkenburg\ndate: 2026-01-06\ndeciders: markwallace-microsoft, dmytrostruk, taochenosu, alliscode, moonbox3, sphenry\nconsulted: sergeymenshykh, rbarreto, dmytrostruk, westey-m\ninformed:\n---\n\n# Simplify Python Get Response API into a single method\n\n## Context and Problem Statement\n\nCurrently chat clients must implement two separate methods to get responses, one for streaming and one for non-streaming. This adds complexity to the client implementations and increases the maintenance burden. This was likely done because the .NET version cannot do proper typing with a single method, in Python this is possible and this for instance is also how the OpenAI python client works, this would then also make it simpler to work with the Python version because there is only one method to learn about instead of two.\n\n## Implications of this change\n\n### Current Architecture Overview\n\nThe current design has **two separate methods** at each layer:\n\n| Layer | Non-streaming | Streaming |\n|-------|---------------|-----------|\n| **Protocol** | `get_response()` → `ChatResponse` | `get_streaming_response()` → `AsyncIterable[ChatResponseUpdate]` |\n| **BaseChatClient** | `get_response()` (public) | `get_streaming_response()` (public) |\n| **Implementation** | `_inner_get_response()` (private) | `_inner_get_streaming_response()` (private) |\n\n### Key Usage Areas Identified\n\n#### 1. **ChatAgent** (_agents.py)\n- `run()` → calls `self.chat_client.get_response()`\n- `run_stream()` → calls `self.chat_client.get_streaming_response()`\n\nThese are parallel methods on the agent, so consolidating the client methods would **not break** the agent API. You could keep `agent.run()` and `agent.run_stream()` unchanged while internally calling `get_response(stream=True/False)`.\n\n#### 2. **Function Invocation Decorator** (_tools.py)\nThis is **the most impacted area**. Currently:\n- `_handle_function_calls_response()` decorates `get_response`\n- `_handle_function_calls_streaming_response()` decorates `get_streaming_response`\n- The `use_function_invocation` class decorator wraps **both methods separately**\n\n**Impact**: The decorator logic is almost identical (~200 lines each) with small differences:\n- Non-streaming collects response, returns it\n- Streaming yields updates, returns async iterable\n\nWith a unified method, you'd need **one decorator** that:\n- Checks the `stream` parameter\n- Uses `@overload` to determine return type\n- Handles both paths with conditional logic\n- The new decorator could be applied just on the method, instead of the whole class.\n\nThis would **reduce code duplication** but add complexity to a single function.\n\n#### 3. **Observability/Instrumentation** (observability.py)\nSame pattern as function invocation:\n- `_trace_get_response()` wraps `get_response`\n- `_trace_get_streaming_response()` wraps `get_streaming_response`\n- `use_instrumentation` decorator applies both\n\n**Impact**: Would need consolidation into a single tracing wrapper.\n\n#### 4. **Chat Middleware** (_middleware.py)\nThe `use_chat_middleware` decorator also wraps both methods separately with similar logic.\n\n#### 5. **AG-UI Client** (_client.py)\nWraps both methods to unwrap server function calls:\n```python\noriginal_get_streaming_response = chat_client.get_streaming_response\noriginal_get_response = chat_client.get_response\n```\n\n#### 6. **Provider Implementations** (all subpackages)\nAll subclasses implement both `_inner_*` methods, except:\n- OpenAI Assistants Client (and similar clients, such as Foundry Agents V1) - it implements `_inner_get_response` by calling `_inner_get_streaming_response`\n\n### Implications of Consolidation\n\n| Aspect | Impact |\n|--------|--------|\n| **Type Safety** | Overloads work well: `@overload` with `Literal[True]` → `AsyncIterable`, `Literal[False]` → `ChatResponse`. Runtime return type based on `stream` param. |\n| **Breaking Change** | **Major breaking change** for anyone implementing custom chat clients. They'd need to update from 2 methods to 1 (or 2 inner methods to 1). |\n| **Decorator Complexity** | All 3 decorator systems (function invocation, middleware, observability) would need refactoring to handle both paths in one wrapper. |\n| **Code Reduction** | Significant reduction in _tools.py (~200 lines of near-duplicate code) and other decorators. |\n| **Samples/Tests** | Many samples call `get_streaming_response()` directly - would need updates. |\n| **Protocol Simplification** | `ChatClientProtocol` goes from 2 methods + 1 property to 1 method + 1 property. |\n\n### Recommendation\n\nThe consolidation makes sense architecturally, but consider:\n\n1. **The overload pattern with `stream: bool`** works well in Python typing:\n ```python\n @overload\n async def get_response(self, messages, *, stream: Literal[True] = True, ...) -> AsyncIterable[ChatResponseUpdate]: ...\n @overload\n async def get_response(self, messages, *, stream: Literal[False] = False, ...) -> ChatResponse: ...\n ```\n\n2. **The decorator complexity** is the biggest concern. The current approach of separate decorators for separate methods is cleaner than conditional logic inside one wrapper.\n\n## Decision Drivers\n\n- Reduce code needed to implement a Chat Client, simplify the public API for chat clients\n- Reduce code duplication in decorators and middleware\n- Maintain type safety and clarity in method signatures\n\n## Considered Options\n\n1. Status quo: Keep separate methods for streaming and non-streaming\n2. Consolidate into a single `get_response` method with a `stream` parameter\n3. Option 2 plus merging `agent.run` and `agent.run_stream` into a single method with a `stream` parameter as well\n\n## Option 1: Status Quo\n- Good: Clear separation of streaming vs non-streaming logic\n- Good: Aligned with .NET design, although it is already `run` for Python and `RunAsync` for .NET\n- Bad: Code duplication in decorators and middleware\n- Bad: More complex client implementations\n\n## Option 2: Consolidate into Single Method\n- Good: Simplified public API for chat clients\n- Good: Reduced code duplication in decorators\n- Good: Smaller API footprint for users to get familiar with\n- Good: People using OpenAI directly already expect this pattern\n- Bad: Increased complexity in decorators and middleware\n- Bad: Less alignment with .NET design (`get_response(stream=True)` vs `GetStreamingResponseAsync`)\n\n## Option 3: Consolidate + Merge Agent and Workflow Methods\n- Good: Further simplifies agent and workflow implementation\n- Good: Single method for all chat interactions\n- Good: Smaller API footprint for users to get familiar with\n- Good: People using OpenAI directly already expect this pattern\n- Good: Workflows internally already use a single method (_run_workflow_with_tracing), so would eliminate public API duplication as well, with hardly any code changes\n- Bad: More breaking changes for agent users\n- Bad: Increased complexity in agent implementation\n- Bad: More extensive misalignment with .NET design (`run(stream=True)` vs `RunStreamingAsync` in addition to `get_response` change)\n\n## Misc\n\nSmaller questions to consider:\n- Should default be `stream=False` or `stream=True`? (Current is False)\n - Default to `False` makes it simpler for new users, as non-streaming is easier to handle.\n - Default to `False` aligns with existing behavior.\n - Streaming tends to be faster, so defaulting to `True` could improve performance for common use cases.\n - Should this differ between ChatClient, Agent and Workflows? (e.g., Agent and Workflow defaults to streaming, ChatClient to non-streaming)\n\n## Decision Outcome\n\nChosen Option: **Option 3: Consolidate + Merge Agent and Workflow Methods**\n\nSince this is the most pythonic option and it reduces the API surface and code duplication the most, we will go with this option.\nWe will keep the default of `stream=False` for all methods to maintain backward compatibility and simplicity for new users.\n\n# Appendix\n## Code Samples for Consolidated Method\n\n### Python - Option 3: Direct ChatClient + Agent with Single Method\n\n```python\n# Copyright (c) Microsoft. All rights reserved.\n\nimport asyncio\nfrom random import randint\nfrom typing import Annotated\n\nfrom agent_framework import ChatAgent\nfrom agent_framework.openai import OpenAIChatClient\nfrom pydantic import Field\n\n\ndef get_weather(\n location: Annotated[str, Field(description=\"The location to get the weather for.\")],\n) -> str:\n \"\"\"Get the weather for a given location.\"\"\"\n conditions = [\"sunny\", \"cloudy\", \"rainy\", \"stormy\"]\n return f\"The weather in {location} is {conditions[randint(0, 3)]} with a high of {randint(10, 30)}°C.\"\n\n\nasync def main() -> None:\n # Example 1: Direct ChatClient usage with single method\n client = OpenAIChatClient()\n message = \"What's the weather in Amsterdam and in Paris?\"\n\n # Non-streaming usage\n print(f\"User: {message}\")\n response = await client.get_response(message, tools=get_weather)\n print(f\"Assistant: {response.text}\")\n\n # Streaming usage - same method, different parameter\n print(f\"\\nUser: {message}\")\n print(\"Assistant: \", end=\"\")\n async for chunk in client.get_response(message, tools=get_weather, stream=True):\n if chunk.text:\n print(chunk.text, end=\"\")\n print(\"\")\n\n # Example 2: Agent usage with single method\n agent = ChatAgent(\n chat_client=client,\n tools=get_weather,\n name=\"WeatherAgent\",\n instructions=\"You are a weather assistant.\",\n )\n thread = agent.get_new_thread()\n\n # Non-streaming agent\n print(f\"\\nUser: {message}\")\n result = await agent.run(message, thread=thread) # default would be stream=False\n print(f\"{agent.name}: {result.text}\")\n\n # Streaming agent - same method, different parameter\n print(f\"\\nUser: {message}\")\n print(f\"{agent.name}: \", end=\"\")\n async for update in agent.run(message, thread=thread, stream=True):\n if update.text:\n print(update.text, end=\"\")\n print(\"\")\n\n\nif __name__ == \"__main__\":\n asyncio.run(main())\n```\n\n### .NET - Current pattern for comparison\n\n```csharp\n// Copyright (c) Microsoft. All rights reserved.\n\nusing Azure.AI.OpenAI;\nusing Azure.Identity;\nusing Microsoft.Agents.AI;\nusing OpenAI.Chat;\n\nvar endpoint = Environment.GetEnvironmentVariable(\"AZURE_OPENAI_ENDPOINT\")\n ?? throw new InvalidOperationException(\"AZURE_OPENAI_ENDPOINT is not set.\");\nvar deploymentName = Environment.GetEnvironmentVariable(\"AZURE_OPENAI_DEPLOYMENT_NAME\") ?? \"gpt-4o-mini\";\n\nAIAgent agent = new AzureOpenAIClient(\n new Uri(endpoint),\n new AzureCliCredential())\n .GetChatClient(deploymentName)\n .CreateAIAgent(\n instructions: \"You are good at telling jokes about pirates.\",\n name: \"PirateJoker\");\n\n// Non-streaming: Returns a string directly\nConsole.WriteLine(\"=== Non-streaming ===\");\nstring result = await agent.RunAsync(\"Tell me a joke about a pirate.\");\nConsole.WriteLine(result);\n\n// Streaming: Returns IAsyncEnumerable\nConsole.WriteLine(\"\\n=== Streaming ===\");\nawait foreach (AgentUpdate update in agent.RunStreamingAsync(\"Tell me a joke about a pirate.\"))\n{\n Console.Write(update);\n}\nConsole.WriteLine();\n\n```\n"
},
{
"path": "docs/decisions/0014-feature-collections.md",
"content": "---\nstatus: accepted\ncontact: westey-m\ndate: 2025-01-21\ndeciders: sergeymenshykh, markwallace, rbarreto, westey-m, stephentoub\nconsulted: reubenbond\ninformed:\n---\n\n# Feature Collections\n\n## Context and Problem Statement\n\nWhen using agents, we often have cases where we want to pass some arbitrary services or data to an agent or some component in the agent execution stack.\nThese services or data are not necessarily known at compile time and can vary by the agent stack that the user has built.\nE.g., there may be an agent decorator or chat client decorator that was added to the stack by the user, and an arbitrary payload needs to be passed to that decorator.\n\nSince these payloads are related to components that are not integral parts of the agent framework, they cannot be added as strongly typed settings to the agent run options.\nHowever, the payloads could be added to the agent run options as loosely typed 'features', that can be retrieved as needed.\n\nIn some cases certain classes of agents may support the same capability, but not all agents do.\nHaving the configuration for such a capability on the main abstraction would advertise the functionality to all users, even if their chosen agent does not support it.\nThe user may type test for certain agent types, and call overloads on the appropriate agent types, with the strongly typed configuration.\nHaving a feature collection though, would be an alternative way of passing such configuration, without needing to type check the agent type.\nAll agents that support the functionality would be able to check for the configuration and use it, simplifying the user code.\nIf the agent does not support the capability, that configuration would be ignored.\n\n### Sample Scenario 1 - Per Run ChatMessageStore Override for hosting Libraries\n\nWe are building an agent hosting library, that can host any agent built using the agent framework.\nWhere an agent is not built on a service that uses in-service chat history storage, the hosting library wants to force the agent to use\nthe hosting library's chat history storage implementation.\nThis chat history storage implementation may be specifically tailored to the type of protocol that the hosting library uses, e.g. conversation id based storage or response id based storage.\nThe hosting library does not know what type of agent it is hosting, so it cannot provide a strongly typed parameter on the agent.\nInstead, it adds the chat history storage implementation to a feature collection, and if the agent supports custom chat history storage, it retrieves the implementation from the feature collection and uses it.\n\n```csharp\n// Pseudo-code for an agent hosting library that supports conversation id based hosting.\npublic async Task HandleConversationsBasedRequestAsync(AIAgent agent, string conversationId, string userInput)\n{\n var thread = await this._threadStore.GetOrCreateThread(conversationId);\n\n // The hosting library can set a per-run chat message store via Features that only applies for that run.\n // This message store will load and save messages under the conversation id provided.\n ConversationsChatMessageStore messageStore = new(this._dbClient, conversationId);\n var response = await agent.RunAsync(\n userInput,\n thread,\n options: new AgentRunOptions()\n {\n Features = new AgentFeatureCollection().WithFeature(messageStore)\n });\n\n await this._threadStore.SaveThreadAsync(conversationId, thread);\n return response.Text;\n}\n\n// Pseudo-code for an agent hosting library that supports response id based hosting.\npublic async Task<(string responseMessage, string responseId)> HandleResponseIdBasedRequestAsync(AIAgent agent, string previousResponseId, string userInput)\n{\n var thread = await this._threadStore.GetOrCreateThreadAsync(previousResponseId);\n\n // The hosting library can set a per-run chat message store via Features that only applies for that run.\n // This message store will buffer newly added messages until explicitly saved after the run.\n ResponsesChatMessageStore messageStore = new(this._dbClient, previousResponseId);\n\n var response = await agent.RunAsync(\n userInput,\n thread,\n options: new AgentRunOptions()\n {\n Features = new AgentFeatureCollection().WithFeature(messageStore)\n });\n\n // Since the message store may not actually have been used at all (if the agent's underlying chat client requires service-based chat history storage),\n // we may not have anything to save back to the database.\n // We still want to generate a new response id though, so that we can save the updated thread state under that id.\n // We should also use the same id to save any buffered messages in the message store if there are any.\n var newResponseId = this.GenerateResponseId();\n if (messageStore.HasBufferedMessages)\n {\n await messageStore.SaveBufferedMessagesAsync(newResponseId);\n }\n \n // Save the updated thread state under the new response id that was generated by the store.\n await this._threadStore.SaveThreadAsync(newResponseId, thread);\n return (response.Text, newResponseId);\n}\n```\n\n### Sample Scenario 2 - Structured output\n\nCurrently our base abstraction does not support structured output, since the capability is not supported by all agents.\nFor those agents that don't support structured output, we could add an agent decorator that takes the response from the underlying agent, and applies structured output parsing on top of it via an additional LLM call.\n\nIf we add structured output configuration as a feature, then any agent that supports structured output could retrieve the configuration from the feature collection and apply it, and where it is not supported, the configuration would simply be ignored.\n\nWe could add a simple StructuredOutputAgentFeature that can be added to the list of features and also be used to return the generated structured output.\n\n```csharp\ninternal class StructuredOutputAgentFeature\n{\n public Type? OutputType { get; set; }\n\n public JsonSerializerOptions? SerializerOptions { get; set; }\n\n public bool? UseJsonSchemaResponseFormat { get; set; }\n\n // Contains the result of the structured output parsing request.\n public ChatResponse? ChatResponse { get; set; }\n}\n```\n\nWe can add a simple decorator class that does the chat client invocation.\n\n```csharp\npublic class StructuredOutputAgent : DelegatingAIAgent\n{\n private readonly IChatClient _chatClient;\n public StructuredOutputAgent(AIAgent innerAgent, IChatClient chatClient)\n : base(innerAgent)\n {\n this._chatClient = Throw.IfNull(chatClient);\n }\n\n public override async Task RunAsync(\n IEnumerable messages,\n AgentThread? thread = null,\n AgentRunOptions? options = null,\n CancellationToken cancellationToken = default)\n {\n // Run the inner agent first, to get back the text response we want to convert.\n var response = await base.RunAsync(messages, thread, options, cancellationToken).ConfigureAwait(false);\n\n if (options?.Features?.TryGet(out var responseFormatFeature) is true\n && responseFormatFeature.OutputType is not null)\n {\n // Create the chat options to request structured output.\n ChatOptions chatOptions = new()\n {\n ResponseFormat = ChatResponseFormat.ForJsonSchema(responseFormatFeature.OutputType, responseFormatFeature.SerializerOptions)\n };\n\n // Invoke the chat client to transform the text output into structured data.\n // The feature is updated with the result.\n // The code can be simplified by adding a non-generic structured output GetResponseAsync\n // overload that takes Type as input.\n responseFormatFeature.ChatResponse = await this._chatClient.GetResponseAsync(\n messages: new[]\n {\n new ChatMessage(ChatRole.System, \"You are a json expert and when provided with any text, will convert it to the requested json format.\"),\n new ChatMessage(ChatRole.User, response.Text)\n },\n options: chatOptions,\n cancellationToken: cancellationToken).ConfigureAwait(false);\n }\n\n return response;\n }\n}\n```\n\nFinally, we can add an extension method on `AIAgent` that can add the feature to the run options and check the feature for the structured output result and add the deserialized result to the response.\n\n```csharp\npublic static async Task> RunAsync(\n this AIAgent agent,\n IEnumerable messages,\n AgentThread? thread = null,\n JsonSerializerOptions? serializerOptions = null,\n AgentRunOptions? options = null,\n bool? useJsonSchemaResponseFormat = null,\n CancellationToken cancellationToken = default)\n{\n // Create the structured output feature.\n var structuredOutputFeature = new StructuredOutputAgentFeature();\n structuredOutputFeature.OutputType = typeof(T);\n structuredOutputFeature.UseJsonSchemaResponseFormat = useJsonSchemaResponseFormat;\n\n // Run the agent.\n options ??= new AgentRunOptions();\n options.Features ??= new AgentFeatureCollection();\n options.Features.Set(structuredOutputFeature);\n\n var response = await agent.RunAsync(messages, thread, options, cancellationToken).ConfigureAwait(false);\n\n // Deserialize the JSON output.\n if (structuredOutputFeature.ChatResponse is not null)\n {\n var typed = new ChatResponse(structuredOutputFeature.ChatResponse, serializerOptions ?? AgentJsonUtilities.DefaultOptions);\n return new AgentRunResponse(response, typed.Result);\n }\n\n throw new InvalidOperationException(\"No structured output response was generated by the agent.\");\n}\n```\n\nWe can then use the extension method with any agent that supports structured output or that has\nbeen decorated with the `StructuredOutputAgent` decorator.\n\n```csharp\nagent = new StructuredOutputAgent(agent, chatClient);\n\nAgentRunResponse response = await agent.RunAsync([new ChatMessage(\n ChatRole.User,\n \"Please provide information about John Smith, who is a 35-year-old software engineer.\")]);\n```\n\n## Implementation Options\n\nThree options were considered for implementing feature collections:\n\n- **Option 1**: FeatureCollections similar to ASP.NET Core\n- **Option 2**: AdditionalProperties Dictionary\n- **Option 3**: IServiceProvider\n\nHere are some comparisons about their suitability for our use case:\n\n| Criteria | Feature Collection | Additional Properties | IServiceProvider |\n|------------------|--------------------|-----------------------|------------------|\n|Ease of use |✅ Good |❌ Bad |✅ Good |\n|User familiarity |❌ Bad |✅ Good |✅ Good |\n|Type safety |✅ Good |❌ Bad |✅ Good |\n|Ability to modify registered options when progressing down the stack|✅ Supported|✅ Supported|❌ Not-Supported (IServiceProvider is read-only)|\n|Already available in MEAI stack|❌ No|✅ Yes|❌ No|\n|Ambiguity with existing AdditionalProperties|❌ Yes|✅ No|❌ Yes|\n\n## IServiceProvider\n\nService Collections and Service Providers provide a very popular way to register and retrieve services by type and could be used as a way to pass features to agents and chat clients.\n\nHowever, since IServiceProvider is read-only, it is not possible to modify the registered services when progressing down the execution stack.\nE.g. an agent decorator cannot add additional services to the IServiceProvider passed to it when calling into the inner agent.\n\nIServiceProvider also does not expose a way to list all services contained in it, making it difficult to copy services from one provider to another.\n\nThis lack of mutability makes IServiceProvider unsuitable for our use case, since we will not be able to use it to build sample scenario 2.\n\n## AdditionalProperties dictionary\n\nThe AdditionalProperties dictionary is already available on various options classes in the agent framework as well as in the MEAI stack and\nallows storing arbitrary key/value pairs, where the key is a string and the value is an object.\n\nWhile FeatureCollection uses Type as a key, AdditionalProperties uses string keys.\nThis means that users need to agree on string keys to use for specific features, however it is also possible to use Type.FullName as a key by convention\nto avoid key collisions, which is an easy convention to follow.\n\nSince the value of AdditionalProperties is of type object, users need to cast the value to the expected type when retrieving it, which is also\na drawback, but when using the convention of using Type.FullName as a key, there is at least a clear expectation of what type to cast to.\n\n```csharp\n// Setting a feature\noptions.AdditionalProperties[typeof(MyFeature).FullName] = new MyFeature();\n\n// Retrieving a feature\nif (options.AdditionalProperties.TryGetValue(typeof(MyFeature).FullName, out var featureObj)\n && featureObj is MyFeature myFeature)\n{\n // Use myFeature\n}\n```\n\nIt would also be possible to add extension methods to simplify setting and getting features from AdditionalProperties.\nHaving a base class for features should help make this more feature rich.\n\n```csharp\n// Setting a feature, this can use Type.FullName as the key.\noptions.AdditionalProperties\n .WithFeature(new MyFeature());\n\n// Retrieving a feature, this can use Type.FullName as the key.\nif (options.AdditionalProperties.TryGetFeature(out var myFeature))\n{\n // Use myFeature\n}\n```\n\nIt would also be possible to add extension methods for a feature to simplify setting and getting features from AdditionalProperties.\n\n```csharp\n// Setting a feature\noptions.AdditionalProperties\n .WithMyFeature(new MyFeature());\n// Retrieving a feature\nif (options.AdditionalProperties.TryGetMyFeature(out var myFeature))\n{\n // Use myFeature\n}\n```\n\n## Feature Collection\n\nIf we choose the feature collection option, we need to decide on the design of the feature collection itself.\n\n### Feature Collections extension points\n\nWe need to decide the set of actions that feature collections would be supported for. Here is the suggested list of actions:\n\n**MAAI.AIAgent:**\n\n1. GetNewThread\n 1. E.g. this would allow passing an already existing storage id for the thread to use, or an initialized custom chat message store to use.\n1. DeserializeThread\n 1. E.g. this would allow passing an already existing storage id for the thread to use, or an initialized custom chat message store to use.\n1. Run / RunStreaming\n 1. E.g. this would allow passing an override chat message store just for that run, or a desired schema for a structured output middleware component.\n\n**MEAI.ChatClient:**\n\n1. GetResponse / GetStreamingResponse\n\n### Reconciling with existing AdditionalProperties\n\nIf we decide to add feature collections, separately from the existing AdditionalProperties dictionaries, we need to consider how to explain to users when to use each one.\nOne possible approach though is to have the one use the other under the hood.\nAdditionalProperties could be stored as a feature in the feature collection.\n\nUsers would be able to retrieve additional properties from the feature collection, in addition to retrieving it via a dedicated AdditionalProperties property.\nE.g. `features.Get()`\n\nOne challenge with this approach is that when setting a value in the AdditionalProperties dictionary, the feature collection would need to be created first if it does not already exist.\n\n```csharp\npublic class AgentRunOptions\n{\n public AdditionalPropertiesDictionary? AdditionalProperties { get; set; }\n public IAgentFeatureCollection? Features { get; set; }\n}\n\nvar options = new AgentRunOptions();\n// This would need to create the feature collection first, if it does not already exist.\noptions.AdditionalProperties ??= new AdditionalPropertiesDictionary();\n```\n\nSince IAgentFeatureCollection is an interface, AgentRunOptions would need to have a concrete implementation of the interface to create, meaning that the user cannot decide.\nIt also means that if the user doesn't realise that AdditionalProperties is implemented using feature collections, they may set a value on AdditionalProperties, and then later overwrite the entire feature collection, losing the AdditionalProperties feature.\n\nOptions to avoid these issues:\n\n1. Make `Features` readonly.\n 1. This would prevent the user from overwriting the feature collection after setting AdditionalProperties.\n 1. Since the user cannot set their own implementation of IAgentFeatureCollection, having an interface for it may not be necessary.\n\n### Feature Collection Implementation\n\nWe have two options for implementing feature collections:\n\n1. Create our own [IAgentFeatureCollection interface](https://github.com/microsoft/agent-framework/pull/2354/files#diff-9c42f3e60d70a791af9841d9214e038c6de3eebfc10e3997cb4cdffeb2f1246d) and [implementation](https://github.com/microsoft/agent-framework/pull/2354/files#diff-a435cc738baec500b8799f7f58c1538e3bb06c772a208afc2615ff90ada3f4ca).\n2. Reuse the asp.net [IFeatureCollection interface](https://github.com/dotnet/aspnetcore/blob/main/src/Extensions/Features/src/IFeatureCollection.cs) and [implementation](https://github.com/dotnet/aspnetcore/blob/main/src/Extensions/Features/src/FeatureCollection.cs).\n\n#### Roll our own\n\nAdvantages:\n\nCreating our own IAgentFeatureCollection interface and implementation has the advantage of being more clearly associated with the agent framework and allows us to\nimprove on some of the design decisions made in asp.net core's IFeatureCollection.\n\nDrawbacks:\n\nIt would mean a different implementation to maintain and test.\n\n#### Reuse asp.net IFeatureCollection\n\nAdvantages:\n\nReusing the asp.net IFeatureCollection has the advantage of being able to reuse the well-established and tested implementation from asp.net\ncore. Users who are using agents in an asp.net core application may be able to pass feature collections from asp.net core to the agent framework directly.\n\nDrawbacks:\n\nWhile the package name is `Microsoft.Extensions.Features`, the namespaces of the types are `Microsoft.AspNetCore.Http.Features`, which may create confusion for users of agent framework who are not building web applications or services.\nUsers may rightly ask: Why do I need to use a class from asp.net core when I'm not building a web application / service?\n\nThe current design has some design issues that would be good to avoid. E.g. it does not distinguish between a feature being \"not set\" and \"null\". Get returns both as null and there is no tryget method.\nSince the [default implementation](https://github.com/dotnet/aspnetcore/blob/main/src/Extensions/Features/src/FeatureCollection.cs) also supports value types, it throws for null values of value types.\nA TryGet method would be more appropriate.\n\n## Feature Layering\n\nOne possible scenario when adding support for feature collections is to allow layering of features by scope.\n\nThe following levels of scope could be supported:\n\n1. Application - Application wide features that apply to all agents / chat clients\n2. Artifact (Agent / ChatClient) - Features that apply to all runs of a specific agent or chat client instance\n3. Action (GetNewThread / Run / GetResponse) - Feature that apply to a single action only\n\nWhen retrieving a feature from the collection, the search would start from the most specific scope (Action) and progress to the least specific scope (Application), returning the first matching feature found.\n\nIntroducing layering adds some challenges:\n\n- There may be multiple feature collections at the same scope level, e.g. an Agent that uses a ChatClient where both have their own feature collections.\n - Do we layer the agent feature collection over the chat client feature collection (Application -> ChatClient -> Agent -> Run), or only use the agent feature collection in the agent (Application -> Agent -> Run), and the chat client feature collection in the chat client (Application -> ChatClient -> Run)?\n- The appropriate base feature collection may change when progressing down the stack, e.g. when an Agent calls a ChatClient, the action feature collection stays the same, but the artifact feature collection changes.\n- Who creates the feature collection hierarchy?\n - Since the hierarchy changes as it progresses down the execution stack, and the caller can only pass in the action level feature collection, the callee needs to combine it with its own artifact level feature collection and the application level feature collection. Each action will need to build the appropriate feature collection hierarchy, at the start of its execution.\n- For Artifact level features, it seems odd to pass them in as a bag of untyped features, when we are constructing a known artifact type and therefore can have typed settings.\n - E.g. today we have a strongly typed setting on ChatClientAgentOptions to configure a ChatMessageStore for the agent.\n- To avoid global statics for application level features, the user would need to pass in the application level feature collection to each artifact that they create.\n - This would be very odd if the user also already has to strongly typed settings for each feature that they want to set at the artifact level.\n\n### Layering Options\n\n1. No layering - only a single feature collection is supported per action (the caller can still create a layered collection if desired, but the callee does not do any layering automatically).\n 1. Fallback is to any features configured on the artifact via strongly typed settings.\n1. Full layering - support layering at all levels (Application -> Artifact -> Action).\n 1. Only apply applicable artifact level features when calling into that artifact.\n 1. Apply upstream artifact features when calling into downstream artifacts, e.g. Feature hierarchy in ChatClientAgent would be `Application -> Agent -> Run` and in ChatClient would be `Application -> ChatClient -> Agent -> Run` or `Application -> Agent -> ChatClient -> Run`\n 1. The user needs to provide the application level feature collection to each artifact that they create and artifact features are passed via strongly typed settings.\n\n### Accessing application level features Options\n\nWe need to consider how application level features would be accessed if supported.\n\n1. The user provides the application level feature collection to each artifact that the user constructs\n 1. Passing the application level feature collection to each artifact is tedious for the user.\n1. There is a static application level feature collection that can be accessed globally.\n 1. Statics create issues with testing and isolation.\n\n## Decisions\n\n- Feature Collections Container: Use AdditionalProperties\n- Feature Layering: No layering - only a single collection/dictionary is supported per action. Application layers can be added later if needed.\n"
},
{
"path": "docs/decisions/0015-agent-run-context.md",
"content": "---\nstatus: proposed\ncontact: westey-m\ndate: 2026-01-27\ndeciders: sergeymenshykh, markwallace, rbarreto, dmytrostruk, westey-m, eavanvalkenburg, stephentoub, lokitoth, alliscode, taochenosu, moonbox3\nconsulted: \ninformed: \n---\n\n# AgentRunContext for Agent Run\n\n## Context and Problem Statement\n\nDuring an agent run, various components involved in the execution (middleware, filters, tools, nested agents, etc.) may need access to contextual information about the current run, such as:\n\n1. The agent that is executing the run\n2. The session associated with the run\n3. The request messages passed to the agent\n4. The run options controlling the agent's behavior\n\nAdditionally, some components may need to modify this context during execution, for example:\n\n- Replacing the session with a different one\n- Modifying the request messages before they reach the agent core\n- Updating or replacing the run options entirely\n\nCurrently, there is no standardized way to access or modify this context from arbitrary code that executes during an agent run, especially from deeply nested call stacks where the context is not explicitly passed.\n\n## Sample Scenario\n\nWhen using an Agent as an AIFunction developers may want to pass context from the parent agent run to the child agent run. For example, the developer may want to copy chat history to the child agent, or share the same session across both agents.\n\nTo enable these scenarios, we need a way to access the parent agent run context, including e.g. the parent agent itself, the parent agent session, and the parent run options from function tool calls.\n\n```csharp\n public static AIFunction AsAIFunctionWithSessionPropagation(this ChatClientAgent agent, AIFunctionFactoryOptions? options = null)\n {\n Throw.IfNull(agent);\n\n [Description(\"Invoke an agent to retrieve some information.\")]\n async Task InvokeAgentAsync(\n [Description(\"Input query to invoke the agent.\")] string query,\n CancellationToken cancellationToken)\n {\n // Get the session from the parent agent and pass it to the child agent.\n var session = AIAgent.CurrentRunContext?.Session;\n\n // Alternatively, the developer may want to create a new session but copy over the chat history from the parent agent.\n // var parentChatHistory = AIAgent.CurrentRunContext?.Session?.GetService>();\n // if (parentChatHistory != null)\n // {\n // var chp = new InMemoryChatHistoryProvider();\n // foreach (var message in parentChatHistory)\n // {\n // chp.Add(message);\n // }\n // session = agent.GetNewSession(chp);\n // }\n\n var response = await agent.RunAsync(query, session: session, cancellationToken: cancellationToken).ConfigureAwait(false);\n return response.Text;\n }\n\n options ??= new();\n options.Name ??= SanitizeAgentName(agent.Name);\n options.Description ??= agent.Description;\n\n return AIFunctionFactory.Create(InvokeAgentAsync, options);\n }\n```\n\n## Decision Drivers\n\n- Components executing during an agent run need access to run context without explicit parameter passing through every layer\n- Context should flow naturally across async calls without manual propagation\n- The design should allow modification of context properties by agent decorators (e.g., replacing options or session)\n- Solution should be consistent with patterns used in similar frameworks (e.g., `FunctionInvokingChatClient.CurrentContext` `HttpContext.Current`, `Activity.Current`)\n\n## Considered Options\n\n- **Option 1**: Pass context explicitly through all method signatures\n- **Option 2**: Use `AsyncLocal` to provide ambient context accessible anywhere during the run\n- **Option 3**: Use a combination of explicit parameters for `RunCoreAsync` and `AsyncLocal` for ambient access\n\n## Decision Outcome\n\nChosen option: **Option 3** - Combination of explicit parameters and AsyncLocal ambient access.\n\nThis approach provides the best of both worlds:\n\n1. **Explicit parameters are passed to `RunCoreAsync`**: The core agent implementation receives the parameters explicitly, making it clear what data is available and enabling easy unit testing. Any modification of these in a decorator will require calling `RunAsync` on the inner agent with the updated parameters, which would result in the inner agent creating a new `AgentRunContext` instance.\n\n ```csharp\n public async Task RunAsync(\n IEnumerable messages,\n AgentSession? session = null,\n AgentRunOptions? options = null,\n CancellationToken cancellationToken = default)\n {\n\n CurrentRunContext = new(this, session, messages as IReadOnlyCollection ?? messages.ToList(), options);\n return await this.RunCoreAsync(messages, session, options, cancellationToken).ConfigureAwait(false);\n }\n ```\n\n2. **`AsyncLocal` for ambient access**: The context is stored in an `AsyncLocal` field, making it accessible from any code executing during the agent run via a static property.\n\n The main scenario for this is to allow deeply nested components (e.g., tools, chat client middleware) to access the context without needing to pass it through every method signature. These are external components that cannot easily be modified to accept additional parameters. For internal components, we prefer passing any parameters explicitly.\n\n ```csharp\n public static AgentRunContext? CurrentRunContext\n {\n get => s_currentContext.Value;\n protected set => s_currentContext.Value = value;\n }\n ```\n\n### AgentRunContext Design\n\nThe `AgentRunContext` class encapsulates all run-related state:\n\n```csharp\npublic class AgentRunContext\n{\n public AgentRunContext(\n AIAgent agent,\n AgentSession? session,\n IReadOnlyCollection requestMessages,\n AgentRunOptions? agentRunOptions)\n\n public AIAgent Agent { get; }\n public AgentSession? Session { get; }\n public IReadOnlyCollection RequestMessages { get; }\n public AgentRunOptions? RunOptions { get; }\n}\n```\n\nKey design decisions:\n\n- **All properties are read-only**: While some of the sub-properties on the provided properties (like `AgentRunOptions.AllowBackgroundResponses`) may be mutable, the `AgentRunContext` itself is immutable and we want to discourage anyone modifying the values in the context. Modifying the context is unlikely to result in the desired behavior, as the values will typically already have been used by the time any custom code accesses them.\n\n### Benefits\n\n1. **Ambient Access**: Any code executing during the run can access context via `AIAgent.CurrentRunContext` without needing explicit parameters\n2. **Async Flow**: `AsyncLocal` automatically flows across async/await boundaries\n3. **Modifiability**: Components can modify or replace session, messages, or options as needed\n4. **Testability**: The explicit parameter to `RunCoreAsync` makes unit testing straightforward\n"
},
{
"path": "docs/decisions/0016-python-context-middleware.md",
"content": "---\n# These are optional elements. Feel free to remove any of them.\nstatus: accepted\ncontact: eavanvalkenburg\ndate: 2026-02-09\ndeciders: eavanvalkenburg, markwallace-microsoft, sphenry, alliscode, johanst, brettcannon, westey-m\nconsulted: taochenosu, moonbox3, dmytrostruk, giles17\n---\n\n# Unifying Context Management with ContextPlugin\n\n## Context and Problem Statement\n\nThe Agent Framework Python SDK currently has multiple abstractions for managing conversation context:\n\n| Concept | Purpose | Location |\n|---------|---------|----------|\n| `ContextProvider` | Injects instructions, messages, and tools before/after invocations | `_memory.py` |\n| `ChatMessageStore` | Stores and retrieves conversation history | `_threads.py` |\n| `AgentThread` | Manages conversation state and coordinates storage | `_threads.py` |\n\nThis creates cognitive overhead for developers doing \"Context Engineering\" - the practice of dynamically managing what context (history, RAG results, instructions, tools) is sent to the model. Users must understand:\n- When to use `ContextProvider` vs `ChatMessageStore`\n- How `AgentThread` coordinates between them\n- Different lifecycle hooks (`invoking()`, `invoked()`, `thread_created()`)\n\n**How can we simplify context management into a single, composable pattern that handles all context-related concerns?**\n\n## Decision Drivers\n\n- **Simplicity**: Reduce the number of concepts users must learn\n- **Composability**: Enable multiple context sources to be combined flexibly\n- **Consistency**: Follow existing patterns in the framework\n- **Flexibility**: Support both stateless and session-specific context engineering\n- **Attribution**: Enable tracking which provider added which messages/tools\n- **Zero-config**: Simple use cases should work without configuration\n\n## Related Issues\n\nThis ADR addresses the following issues from the parent issue [#3575](https://github.com/microsoft/agent-framework/issues/3575):\n\n| Issue | Title | How Addressed |\n|-------|-------|---------------|\n| [#3587](https://github.com/microsoft/agent-framework/issues/3587) | Rename AgentThread to AgentSession | ✅ `AgentThread` → `AgentSession` (clean break, no alias). See [§7 Renaming](#7-renaming-thread--session). |\n| [#3588](https://github.com/microsoft/agent-framework/issues/3588) | Add get_new_session, get_session_by_id methods | ✅ `agent.create_session()` and `agent.get_session(service_session_id)`. See [§9 Session Management Methods](#9-session-management-methods). |\n| [#3589](https://github.com/microsoft/agent-framework/issues/3589) | Move serialize method into the agent | ✅ No longer needed. `AgentSession` provides `to_dict()`/`from_dict()` for serialization. Providers write JSON-serializable values to `session.state`. See [§8 Serialization](#8-session-serializationdeserialization). |\n| [#3590](https://github.com/microsoft/agent-framework/issues/3590) | Design orthogonal ChatMessageStore for service vs local | ✅ `HistoryProvider` works orthogonally: configure `load_messages=False` when service manages storage. Multiple history providers allowed. See [§3 Unified Storage](#3-unified-storage). |\n| [#3601](https://github.com/microsoft/agent-framework/issues/3601) | Rename ChatMessageStore to ChatHistoryProvider | 🔒 **Closed** - Superseded by this ADR. `ChatMessageStore` removed entirely, replaced by `StorageContextMiddleware`. |\n\n## Current State Analysis\n\n### ContextProvider (Current)\n\n```python\nclass ContextProvider(ABC):\n async def thread_created(self, thread_id: str | None) -> None:\n \"\"\"Called when a new thread is created.\"\"\"\n pass\n\n async def invoked(\n self,\n request_messages: ChatMessage | Sequence[ChatMessage],\n response_messages: ChatMessage | Sequence[ChatMessage] | None = None,\n invoke_exception: Exception | None = None,\n **kwargs: Any,\n ) -> None:\n \"\"\"Called after the agent receives a response.\"\"\"\n pass\n\n @abstractmethod\n async def invoking(self, messages: ChatMessage | MutableSequence[ChatMessage], **kwargs: Any) -> Context:\n \"\"\"Called before model invocation. Returns Context with instructions, messages, tools.\"\"\"\n pass\n```\n\n**Limitations:**\n- No clear way to compose multiple providers\n- No source attribution for debugging\n\n### ChatMessageStore (Current)\n\n```python\nclass ChatMessageStoreProtocol(Protocol):\n async def list_messages(self) -> list[ChatMessage]: ...\n async def add_messages(self, messages: Sequence[ChatMessage]) -> None: ...\n async def serialize(self, **kwargs: Any) -> dict[str, Any]: ...\n @classmethod\n async def deserialize(cls, state: MutableMapping[str, Any], **kwargs: Any) -> \"ChatMessageStoreProtocol\": ...\n```\n\n**Limitations:**\n- Only handles message storage, no context injection\n- Separate concept from `ContextProvider`\n- No control over what gets stored (RAG context vs user messages)\n- No control over which get's executed first, the Context Provider or the ChatMessageStore (ordering ambiguity), this is controlled by the framework\n\n### AgentThread (Current)\n\n```python\nclass AgentThread:\n def __init__(\n self,\n *,\n service_thread_id: str | None = None,\n message_store: ChatMessageStoreProtocol | None = None,\n context_provider: ContextProvider | None = None,\n ) -> None: ...\n```\n\n**Limitations:**\n- Coordinates storage and context separately\n- Only one `context_provider` and one `ChatMessageStore` (no composition)\n\n## Key Design Considerations\n\nThe following key decisions shape the ContextProvider design:\n\n| # | Decision | Rationale |\n|---|----------|-----------|\n| 1 | **Agent vs Session Ownership** | Agent owns provider instances; Session owns state as mutable dict. Providers shared across sessions, state isolated per session. |\n| 2 | **Execution Pattern** | **ContextProvider** with `before_run`/`after_run` methods (hooks pattern). Simpler mental model than wrapper/onion pattern. |\n| 3 | **State Management** | Whole state dict (`dict[str, Any]`) passed to each plugin. Dict is mutable, so no return value needed. |\n| 4 | **Default Storage at Runtime** | `InMemoryHistoryProvider` auto-added when no providers configured and `options.conversation_id` is set or `options.store` is True. Evaluated at runtime so users can modify pipeline first. |\n| 5 | **Multiple Storage Allowed** | Warn at session creation if multiple or zero history providers have `load_messages=True` (likely misconfiguration). |\n| 6 | **Single Storage Class** | One `HistoryProvider` configured for memory/audit/evaluation - no separate classes. |\n| 7 | **Mandatory source_id** | Required parameter forces explicit naming for attribution in `context_messages` dict. |\n| 8 | **Explicit Load Behavior** | `load_messages: bool = True` - explicit configuration with no automatic detection. For history, `before_run` is skipped entirely when `load_messages=False`. |\n| 9 | **Dict-based Context** | `context_messages: dict[str, list[ChatMessage]]` keyed by source_id maintains order and enables filtering. Messages can have an `attribution` marker in `additional_properties` for external filtering scenarios. |\n| 10 | **Selective Storage** | `store_context_messages` and `store_context_from` control what gets persisted from other plugins. |\n| 11 | **Tool Attribution** | `extend_tools()` automatically sets `tool.metadata[\"context_source\"] = source_id`. |\n| 12 | **Clean Break** | Remove `AgentThread`, old `ContextProvider`, `ChatMessageStore` completely; replace with new `ContextProvider` (hooks pattern), `HistoryProvider`, `AgentSession`. PR1 uses temporary names (`_ContextProviderBase`, `_HistoryProviderBase`) to coexist with old types; PR2 renames to final names after old types are removed. No compatibility shims (preview). |\n| 13 | **Plugin Ordering** | User-defined order; storage sees prior plugins (pre-processing) or all plugins (post-processing). |\n| 14 | **Session Serialization via `to_dict`/`from_dict`** | `AgentSession` provides `to_dict()` and `from_dict()` for round-tripping. Providers must ensure values they write to `session.state` are JSON-serializable. No `serialize()`/`restore()` methods on providers. |\n| 15 | **Session Management Methods** | `agent.create_session()` and `agent.get_session(service_session_id)` for clear lifecycle management. |\n\n## Considered Options\n\n### Option 1: Status Quo - Keep Separate Abstractions\n\nKeep `ContextProvider`, `ChatMessageStore`, and `AgentThread` as separate concepts. With updated naming and minor improvements, but no fundamental changes to the API or execution model.\n\n**Pros:**\n- No migration required\n- Familiar to existing users\n- Each concept has a focused responsibility\n- Existing documentation and examples remain valid\n\n**Cons:**\n- Cognitive overhead: three concepts to learn for context management\n- No composability: only one `ContextProvider` per thread\n- Inconsistent with middleware pattern used elsewhere in the framework\n- `invoking()`/`invoked()` split makes related pre/post logic harder to follow\n- No source attribution for debugging which provider added which context\n- `ChatMessageStore` and `ContextProvider` overlap conceptually but are separate APIs\n\n### Option 2: ContextMiddleware - Wrapper Pattern\n\nCreate a unified `ContextMiddleware` base class that uses the onion/wrapper pattern (like existing `AgentMiddleware`, `ChatMiddleware`) to handle all context-related concerns. This includes a `StorageContextMiddleware` subclass specifically for history persistence.\n\n**Class hierarchy:**\n- `ContextMiddleware` (base) - for general context injection (RAG, instructions, tools)\n- `StorageContextMiddleware(ContextMiddleware)` - for conversation history storage (in-memory, Redis, Cosmos, etc.)\n\n```python\nclass ContextMiddleware(ABC):\n def __init__(self, source_id: str, *, session_id: str | None = None):\n self.source_id = source_id\n self.session_id = session_id\n\n @abstractmethod\n async def process(self, context: SessionContext, next: ContextMiddlewareNext) -> None:\n \"\"\"Wrap the context flow - modify before next(), process after.\"\"\"\n # Pre-processing: add context, modify messages\n context.add_messages(self.source_id, [...])\n\n await next(context) # Call next middleware or terminal handler\n\n # Post-processing: log, store, react to response\n await self.store(context.response_messages)\n```\n\n**Pros:**\n- Single concept for all context engineering\n- Familiar pattern from other middleware in the framework (`AgentMiddleware`, `ChatMiddleware`)\n- Natural composition via pipeline with clear execution order\n- Pre/post processing in one method keeps related logic together\n- Source attribution built-in\n- Full control over the invocation chain (can short-circuit, retry, wrap with try/catch)\n- Exception handling naturally scoped to the middleware that caused it\n\n**Cons:**\n- Forgetting `await next(context)` silently breaks the chain\n- Stack depth increases with each middleware layer\n- Harder to implement middleware that only needs pre OR post processing\n- Streaming is more complicated\n\n### Option 3: ContextHooks - Pre/Post Pattern\n\nCreate a `ContextHooks` base class with explicit `before_run()` and `after_run()` methods, diverging from the wrapper pattern used by middleware. This includes a `HistoryContextHooks` subclass specifically for history persistence.\n\n**Class hierarchy:**\n- `ContextHooks` (base) - for general context injection (RAG, instructions, tools)\n- `HistoryContextHooks(ContextHooks)` - for conversation history storage (in-memory, Redis, Cosmos, etc.)\n\n```python\nclass ContextHooks(ABC):\n def __init__(self, source_id: str, *, session_id: str | None = None):\n self.source_id = source_id\n self.session_id = session_id\n\n async def before_run(self, context: SessionContext) -> None:\n \"\"\"Called before model invocation. Modify context here.\"\"\"\n pass\n\n async def after_run(self, context: SessionContext) -> None:\n \"\"\"Called after model invocation. React to response here.\"\"\"\n pass\n```\n\n> **Note on naming:** Both the class name (`ContextHooks`) and method names (`before_run`/`after_run`) are open for discussion. The names used throughout this ADR are placeholders pending a final decision. See alternative naming options below.\n\n**Alternative class naming options:**\n\n| Name | Rationale |\n|------|-----------|\n| `ContextHooks` | Emphasizes the hook-based nature, familiar from React/Git hooks |\n| `ContextHandler` | Generic term for something that handles context events |\n| `ContextInterceptor` | Common in Java/Spring, emphasizes interception points |\n| `ContextProcessor` | Emphasizes processing at defined stages |\n| `ContextPlugin` | Emphasizes extensibility, familiar from build tools |\n| `SessionHooks` | Ties to `AgentSession`, emphasizes session lifecycle |\n| `InvokeHooks` | Directly describes what's being hooked (the invoke call) |\n\n**Alternative method naming options:**\n\n| before / after | Rationale |\n|----------------|-----------|\n| `before_run` / `after_run` | Matches `agent.run()` terminology |\n| `before_invoke` / `after_invoke` | Emphasizes invocation lifecycle |\n| `invoking` / `invoked` | Matches current Python `ContextProvider` and .NET naming |\n| `pre_invoke` / `post_invoke` | Common prefix convention |\n| `on_invoking` / `on_invoked` | Event-style naming |\n| `prepare` / `finalize` | Action-oriented naming |\n\n**Example usage:**\n\n```python\nclass RAGHooks(ContextHooks):\n async def before_run(self, context: SessionContext) -> None:\n docs = await self.retrieve_documents(context.input_messages[-1].text)\n context.add_messages(self.source_id, [ChatMessage.system(f\"Context: {docs}\")])\n\n async def after_run(self, context: SessionContext) -> None:\n await self.store_interaction(context.input_messages, context.response_messages)\n\n\n# Pipeline execution is linear, not nested:\n# 1. hook1.before_run(context)\n# 2. hook2.before_run(context)\n# 3. \n# 4. hook2.after_run(context) # Reverse order for symmetry\n# 5. hook1.after_run(context)\n\nagent = ChatAgent(\n chat_client=client,\n context_hooks=[\n InMemoryStorageHooks(\"memory\"),\n RAGHooks(\"rag\"),\n ]\n)\n```\n\n**Pros:**\n- Simpler mental model: \"before\" runs before, \"after\" runs after - no nesting to understand\n- Clearer separation between what this does vs what Agent Middleware can do.\n- Impossible to forget calling `next()` - the framework handles sequencing\n- Easier to implement hooks that only need one phase (just override one method)\n- Lower cognitive overhead for developers new to middleware patterns\n- Clearer separation of concerns: pre-processing logic separate from post-processing\n- Easier to test: no need to mock `next` callable, just call methods directly\n- Flatter stack traces when debugging\n- More similar to the current `ContextProvider` API (`invoking`/`invoked`), easing migration\n- Explicit about what happens when: no hidden control flow\n\n**Cons:**\n- Diverges from the wrapper pattern used by `AgentMiddleware` and `ChatMiddleware`\n- Less powerful: cannot short-circuit the chain or implement retry logic (to mitigate, AgentMiddleware still exists and can be used for this scenario.)\n- No \"around\" advice: cannot wrap invocation in try/catch or timing block\n- Exception in `before_run` may leave state inconsistent if no cleanup in `after_run`\n- Two methods to implement instead of one (though both are optional)\n- Harder to share state between before/after (need instance variables, use state)\n- Cannot control whether subsequent hooks run (no early termination)\n\n## Detailed Design\n\nThis section covers the design decisions that apply to both approaches. Where the approaches differ, both are shown.\n\n### 1. Execution Pattern\n\nThe core difference between the two options is the execution model:\n\n**Option 2 - Middleware (Wrapper/Onion):**\n```python\nclass ContextMiddleware(ABC):\n @abstractmethod\n async def process(self, context: SessionContext, next: ContextMiddlewareNext) -> None:\n \"\"\"Abstract — subclasses must implement the full pre/invoke/post flow.\"\"\"\n ...\n\n# Subclass must implement process():\nclass RAGMiddleware(ContextMiddleware):\n async def process(self, context, next):\n context.add_messages(self.source_id, [...]) # Pre-processing\n await next(context) # Call next middleware\n await self.store(context.response_messages) # Post-processing\n```\n\n**Option 3 - Hooks (Linear):**\n```python\nclass ContextHooks:\n async def before_run(self, context: SessionContext) -> None:\n \"\"\"Default no-op. Override to add pre-invocation logic.\"\"\"\n pass\n\n async def after_run(self, context: SessionContext) -> None:\n \"\"\"Default no-op. Override to add post-invocation logic.\"\"\"\n pass\n\n# Subclass overrides only the hooks it needs:\nclass RAGHooks(ContextHooks):\n async def before_run(self, context):\n context.add_messages(self.source_id, [...])\n\n async def after_run(self, context):\n await self.store(context.response_messages)\n```\n\n**Execution flow comparison:**\n\n```\nMiddleware (Wrapper/Onion): Hooks (Linear):\n┌──────────────────────────┐ ┌─────────────────────────┐\n│ middleware1.process() │ │ hook1.before_run() │\n│ ┌───────────────────┐ │ │ hook2.before_run() │\n│ │ middleware2.process│ │ │ hook3.before_run() │\n│ │ ┌─────────────┐ │ │ ├─────────────────────────┤\n│ │ │ invoke │ │ │ vs │ │\n│ │ └─────────────┘ │ │ ├─────────────────────────┤\n│ │ (post-processing) │ │ │ hook3.after_run() │\n│ └───────────────────┘ │ │ hook2.after_run() │\n│ (post-processing) │ │ hook1.after_run() │\n└──────────────────────────┘ └─────────────────────────┘\n```\n\n### 2. Agent vs Session Ownership\n\nWhere provider instances live (agent-level vs session-level) is an orthogonal decision that applies to both execution patterns. Each combination has different consequences:\n\n| | **Agent owns instances** | **Session owns instances** |\n|--|--------------------------|---------------------------|\n| **Middleware (Option 2)** | Agent holds the middleware chain; all sessions share it. Per-session state must be externalized (e.g., passed via context). Pipeline ordering is fixed across sessions. | Each session gets its own middleware chain (via factories). Middleware can hold per-session state internally. Requires factory pattern to construct per-session instances. |\n| **Hooks (Option 3)** | Agent holds provider instances; all sessions share them. Per-session state lives in `session.state` dict. Simple flat iteration, no pipeline to construct. | Each session gets its own provider instances (via factories). Providers can hold per-session state internally. Adds factory complexity without the pipeline benefit. |\n\n**Key trade-offs:**\n\n- **Agent-owned + Middleware**: The nested call chain makes it awkward to share — each `process()` call captures `next` in its closure, which may carry session-specific assumptions. Externalizing state is harder when it's interleaved with the wrapping flow.\n- **Session-owned + Middleware**: Natural fit — each session gets its own chain with isolated state. But requires factories and heavier sessions.\n- **Agent-owned + Hooks**: Natural fit — `before_run`/`after_run` are stateless calls that receive everything they need as parameters (`session`, `context`, `state`). No pipeline to construct, lightweight sessions.\n- **Session-owned + Hooks**: Works but adds factory overhead without clear benefit — hooks don't need per-instance state since `session.state` handles isolation.\n\n### 3. Unified Storage\n\nInstead of separate `ChatMessageStore`, storage is a subclass of the base context type:\n\n**Middleware:**\n```python\nclass StorageContextMiddleware(ContextMiddleware):\n def __init__(\n self,\n source_id: str,\n *,\n load_messages: bool = True,\n store_inputs: bool = True,\n store_responses: bool = True,\n store_context_messages: bool = False,\n store_context_from: Sequence[str] | None = None,\n ): ...\n```\n\n**Hooks:**\n```python\nclass StorageContextHooks(ContextHooks):\n def __init__(\n self,\n source_id: str,\n *,\n load_messages: bool = True,\n store_inputs: bool = True,\n store_responses: bool = True,\n store_context_messages: bool = False,\n store_context_from: Sequence[str] | None = None,\n ): ...\n```\n\n**Load Behavior:**\n- `load_messages=True` (default): Load messages from storage in `before_run`/pre-processing\n- `load_messages=False`: Skip loading; for `StorageContextHooks`, the `before_run` hook is not called at all\n\n**Comparison to Current:**\n| Aspect | ChatMessageStore (Current) | Storage Middleware/Hooks (New) |\n|--------|---------------------------|------------------------------|\n| Load messages | Always via `list_messages()` | Configurable `load_messages` flag |\n| Store messages | Always via `add_messages()` | Configurable `store_*` flags |\n| What to store | All messages | Selective: inputs, responses, context |\n| Injected context | Not supported | `store_context_messages=True/False` + `store_context_from=[source_ids]` for filtering |\n\n### 4. Source Attribution via `source_id`\n\nBoth approaches require a `source_id` for attribution (identical implementation):\n\n```python\nclass SessionContext:\n context_messages: dict[str, list[ChatMessage]]\n\n def add_messages(self, source_id: str, messages: Sequence[ChatMessage]) -> None:\n if source_id not in self.context_messages:\n self.context_messages[source_id] = []\n self.context_messages[source_id].extend(messages)\n\n def get_messages(\n self,\n sources: Sequence[str] | None = None,\n exclude_sources: Sequence[str] | None = None,\n ) -> list[ChatMessage]:\n \"\"\"Get messages, optionally filtered by source.\"\"\"\n ...\n```\n\n**Benefits:**\n- Debug which middleware/hooks added which messages\n- Filter messages by source (e.g., exclude RAG from storage)\n- Multiple instances of same type distinguishable\n\n**Message-level Attribution:**\n\nIn addition to source-based filtering, individual `ChatMessage` objects should have an `attribution` marker in their `additional_properties` dict. This enables external scenarios to filter messages after the full list has been composed from input and context messages:\n\n```python\n# Setting attribution on a message\nmessage = ChatMessage(\n role=\"system\",\n text=\"Relevant context from knowledge base\",\n additional_properties={\"attribution\": \"knowledge_base\"}\n)\n\n# Filtering by attribution (external scenario)\nall_messages = context.get_all_messages(include_input=True)\nfiltered = [m for m in all_messages if m.additional_properties.get(\"attribution\") != \"ephemeral\"]\n```\n\nThis is useful for scenarios where filtering by `source_id` is not sufficient, such as when messages from the same source need different treatment.\n\n> **Note:** The `attribution` marker is intended for runtime filtering only and should **not** be propagated to storage. Storage middleware should strip `attribution` from `additional_properties` before persisting messages.\n\n### 5. Default Storage Behavior\n\nZero-config works out of the box (both approaches):\n\n```python\n# No middleware/hooks configured - still gets conversation history!\nagent = ChatAgent(chat_client=client, name=\"assistant\")\nsession = agent.create_session()\nresponse = await agent.run(\"Hello!\", session=session)\nresponse = await agent.run(\"What did I say?\", session=session) # Remembers!\n```\n\nDefault in-memory storage is added at runtime **only when**:\n- No `service_session_id` (service not managing storage)\n- `options.store` is not `True` (user not expecting service storage)\n- **No pipeline configured at all** (pipeline is empty or None)\n\n**Important:** If the user configures *any* middleware/hooks (even non-storage ones), the framework does **not** automatically add storage. This is intentional:\n- Once users start customizing the pipeline, we consider them a advanced user and they should know what they are doing, therefore they should explicitly configure storage\n- Automatic insertion would create ordering ambiguity\n- Explicit configuration is clearer than implicit behavior\n\n### 6. Instance vs Factory\n\nBoth approaches support shared instances and per-session factories:\n\n**Middleware:**\n```python\n# Instance (shared across sessions)\nagent = ChatAgent(context_middleware=[RAGContextMiddleware(\"rag\")])\n\n# Factory (new instance per session)\ndef create_cache(session_id: str | None) -> ContextMiddleware:\n return SessionCacheMiddleware(\"cache\", session_id=session_id)\n\nagent = ChatAgent(context_middleware=[create_cache])\n```\n\n**Hooks:**\n```python\n# Instance (shared across sessions)\nagent = ChatAgent(context_hooks=[RAGContextHooks(\"rag\")])\n\n# Factory (new instance per session)\ndef create_cache(session_id: str | None) -> ContextHooks:\n return SessionCacheHooks(\"cache\", session_id=session_id)\n\nagent = ChatAgent(context_hooks=[create_cache])\n```\n\n### 7. Renaming: Thread → Session\n\n`AgentThread` becomes `AgentSession` to better reflect its purpose:\n- \"Thread\" implies a sequence of messages\n- \"Session\" better captures the broader scope (state, pipeline, lifecycle)\n- Align with recent change in .NET SDK\n\n### 8. Session Serialization/Deserialization\n\nThere are two approaches to session serialization:\n\n**Option A: Direct serialization on `AgentSession`**\n\nThe session itself provides `to_dict()` and `from_dict()`. The caller controls when and where to persist:\n\n```python\n# Serialize\ndata = session.to_dict() # → {\"type\": \"session\", \"session_id\": ..., \"service_session_id\": ..., \"state\": {...}}\njson_str = json.dumps(data) # Store anywhere (database, file, cache, etc.)\n\n# Deserialize\ndata = json.loads(json_str)\nsession = AgentSession.from_dict(data) # Reconstructs session with all state intact\n```\n\n**Option B: Serialization through the agent**\n\nThe agent provides `save_session()`/`load_session()` methods that coordinate with providers (e.g., letting providers hook into the serialization process, or validating state before persisting). This adds flexibility but also complexity — providers would need lifecycle hooks for serialization, and the agent becomes responsible for persistence concerns.\n\n**Provider contract (both options):** Any values a provider writes to `session.state`/through lifecycle hooks **must be JSON-serializable** (dicts, lists, strings, numbers, booleans, None).\n\n**Comparison to Current:**\n| Aspect | Current (`AgentThread`) | New (`AgentSession`) |\n|--------|------------------------|---------------------|\n| Serialization | `ChatMessageStore.serialize()` + custom logic | `session.to_dict()` → plain dict |\n| Deserialization | `ChatMessageStore.deserialize()` + factory | `AgentSession.from_dict(data)` |\n| Provider state | Instance state, needs custom ser/deser | Plain dict values in `session.state` |\n\n### 9. Session Management Methods\n\nBoth approaches use identical agent methods:\n\n```python\nclass ChatAgent:\n def create_session(self, *, session_id: str | None = None) -> AgentSession:\n \"\"\"Create a new session.\"\"\"\n ...\n\n def get_session(self, service_session_id: str, *, session_id: str | None = None) -> AgentSession:\n \"\"\"Get a session for a service-managed session ID.\"\"\"\n ...\n```\n\n**Usage (identical for both):**\n```python\nsession = agent.create_session()\nsession = agent.create_session(session_id=\"custom-id\")\nsession = agent.get_session(\"existing-service-session-id\")\nsession = agent.get_session(\"existing-service-session-id\", session_id=\"custom-id\")\n```\n\n### 10. Accessing Context from Other Middleware/Hooks\n\nNon-storage middleware/hooks can read context added by others via `context.context_messages`. However, they should operate under the assumption that **only the current input messages are available** - there is no implicit conversation history.\n\nIf historical context is needed (e.g., RAG using last few messages), maintain a **self-managed buffer**, which would look something like this:\n\n**Middleware:**\n```python\nclass RAGWithBufferMiddleware(ContextMiddleware):\n def __init__(self, source_id: str, retriever: Retriever, *, buffer_window: int = 5):\n super().__init__(source_id)\n self._retriever = retriever\n self._buffer_window = buffer_window\n self._message_buffer: list[ChatMessage] = []\n\n async def process(self, context: SessionContext, next: ContextMiddlewareNext) -> None:\n # Use buffer + current input for retrieval\n recent = self._message_buffer[-self._buffer_window * 2:]\n query = self._build_query(recent + list(context.input_messages))\n docs = await self._retriever.search(query)\n context.add_messages(self.source_id, [ChatMessage.system(f\"Context: {docs}\")])\n\n await next(context)\n\n # Update buffer\n self._message_buffer.extend(context.input_messages)\n if context.response_messages:\n self._message_buffer.extend(context.response_messages)\n```\n\n**Hooks:**\n```python\nclass RAGWithBufferHooks(ContextHooks):\n def __init__(self, source_id: str, retriever: Retriever, *, buffer_window: int = 5):\n super().__init__(source_id)\n self._retriever = retriever\n self._buffer_window = buffer_window\n self._message_buffer: list[ChatMessage] = []\n\n async def before_run(self, context: SessionContext) -> None:\n recent = self._message_buffer[-self._buffer_window * 2:]\n query = self._build_query(recent + list(context.input_messages))\n docs = await self._retriever.search(query)\n context.add_messages(self.source_id, [ChatMessage.system(f\"Context: {docs}\")])\n\n async def after_run(self, context: SessionContext) -> None:\n self._message_buffer.extend(context.input_messages)\n if context.response_messages:\n self._message_buffer.extend(context.response_messages)\n```\n\n**Simple RAG (input only, no buffer):**\n\n```python\n# Middleware\nasync def process(self, context, next):\n query = \" \".join(msg.text for msg in context.input_messages if msg.text)\n docs = await self._retriever.search(query)\n context.add_messages(self.source_id, [ChatMessage.system(f\"Context: {docs}\")])\n await next(context)\n\n# Hooks\nasync def before_run(self, context):\n query = \" \".join(msg.text for msg in context.input_messages if msg.text)\n docs = await self._retriever.search(query)\n context.add_messages(self.source_id, [ChatMessage.system(f\"Context: {docs}\")])\n```\n\n### Migration Impact\n\n| Current | Middleware (Option 2) | Hooks (Option 3) |\n|---------|----------------------|------------------|\n| `ContextProvider` | `ContextMiddleware` | `ContextHooks` |\n| `invoking()` | Before `await next(context)` | `before_run()` |\n| `invoked()` | After `await next(context)` | `after_run()` |\n| `ChatMessageStore` | `StorageContextMiddleware` | `StorageContextHooks` |\n| `AgentThread` | `AgentSession` | `AgentSession` |\n\n### Example: Current vs New\n\n**Current:**\n```python\nclass MyContextProvider(ContextProvider):\n async def invoking(self, messages, **kwargs) -> Context:\n docs = await self.retrieve_documents(messages[-1].text)\n return Context(messages=[ChatMessage.system(f\"Context: {docs}\")])\n\n async def invoked(self, request, response, **kwargs) -> None:\n await self.store_interaction(request, response)\n\nthread = await agent.get_new_thread(message_store=ChatMessageStore())\nthread.context_provider = provider\nresponse = await agent.run(\"Hello\", thread=thread)\n```\n\n**New (Middleware):**\n```python\nclass RAGMiddleware(ContextMiddleware):\n async def process(self, context: SessionContext, next) -> None:\n docs = await self.retrieve_documents(context.input_messages[-1].text)\n context.add_messages(self.source_id, [ChatMessage.system(f\"Context: {docs}\")])\n await next(context)\n await self.store_interaction(context.input_messages, context.response_messages)\n\nagent = ChatAgent(\n chat_client=client,\n context_middleware=[InMemoryStorageMiddleware(\"memory\"), RAGMiddleware(\"rag\")]\n)\nsession = agent.create_session()\nresponse = await agent.run(\"Hello\", session=session)\n```\n\n**New (Hooks):**\n```python\nclass RAGHooks(ContextHooks):\n async def before_run(self, context: SessionContext) -> None:\n docs = await self.retrieve_documents(context.input_messages[-1].text)\n context.add_messages(self.source_id, [ChatMessage.system(f\"Context: {docs}\")])\n\n async def after_run(self, context: SessionContext) -> None:\n await self.store_interaction(context.input_messages, context.response_messages)\n\nagent = ChatAgent(\n chat_client=client,\n context_hooks=[InMemoryStorageHooks(\"memory\"), RAGHooks(\"rag\")]\n)\nsession = agent.create_session()\nresponse = await agent.run(\"Hello\", session=session)\n```\n### Instance Ownership Options (for reference)\n\n#### Option A: Instances in Session\n\nThe `AgentSession` owns the actual middleware/hooks instances. The pipeline is created when the session is created, and instances are stored in the session.\n\n```python\nclass AgentSession:\n \"\"\"Session owns the middleware instances.\"\"\"\n\n def __init__(\n self,\n *,\n session_id: str | None = None,\n context_pipeline: ContextMiddlewarePipeline | None = None, # Owns instances\n ):\n self._session_id = session_id or str(uuid.uuid4())\n self._context_pipeline = context_pipeline # Actual instances live here\n\n\nclass ChatAgent:\n def __init__(\n self,\n chat_client: ...,\n *,\n context_middleware: Sequence[ContextMiddlewareConfig] | None = None,\n ):\n self._context_middleware_config = list(context_middleware or [])\n\n def create_session(self, *, session_id: str | None = None) -> AgentSession:\n \"\"\"Create session with resolved middleware instances.\"\"\"\n resolved_id = session_id or str(uuid.uuid4())\n\n # Resolve factories and create actual instances\n pipeline = None\n if self._context_middleware_config:\n pipeline = ContextMiddlewarePipeline.from_config(\n self._context_middleware_config,\n session_id=resolved_id,\n )\n\n return AgentSession(\n session_id=resolved_id,\n context_pipeline=pipeline, # Session owns the instances\n )\n\n async def run(self, input: str, *, session: AgentSession) -> AgentResponse:\n # Session's pipeline executes\n context = await session.run_context_pipeline(input_messages)\n # ... invoke model ...\n```\n\n**Pros:**\n- Self-contained session - all state and behavior together\n- Middleware can maintain per-session instance state naturally\n- Session given to another agent will work the same way\n\n**Cons:**\n- Session becomes heavier (instances + state)\n- Complicated serialization - serialization needs to deal with instances, which might include non-serializable things like clients or connections\n- Harder to share stateless middleware across sessions efficiently\n- Factories must be re-resolved for each session\n\n#### Option B: Instances in Agent, State in Session (CHOSEN)\n\nThe agent owns and manages the middleware/hooks instances. The `AgentSession` only stores state data that middleware reads/writes. The agent's runner executes the pipeline using the session's state.\n\nTwo variants exist for how state is stored in the session:\n\n##### Option B1: Simple Dict State (CHOSEN)\n\nThe session stores state as a simple `dict[str, Any]`. Each plugin receives the **whole state dict**, and since dicts are mutable in Python, plugins can modify it in place without needing to return a value.\n\n```python\nclass AgentSession:\n \"\"\"Session only holds state as a simple dict.\"\"\"\n\n def __init__(self, *, session_id: str | None = None):\n self._session_id = session_id or str(uuid.uuid4())\n self.service_session_id: str | None = None\n self.state: dict[str, Any] = {} # Mutable state dict\n\n\nclass ChatAgent:\n def __init__(\n self,\n chat_client: ...,\n *,\n context_providers: Sequence[ContextProvider] | None = None,\n ):\n # Agent owns the actual plugin instances\n self._context_providers = list(context_providers or [])\n\n def create_session(self, *, session_id: str | None = None) -> AgentSession:\n \"\"\"Create lightweight session with just state.\"\"\"\n return AgentSession(session_id=session_id)\n\n async def run(self, input: str, *, session: AgentSession) -> AgentResponse:\n context = SessionContext(\n session_id=session.session_id,\n input_messages=[...],\n )\n\n # Before-run plugins\n for plugin in self._context_providers:\n # Skip before_run for HistoryProviders that don't load messages\n if isinstance(plugin, HistoryProvider) and not plugin.load_messages:\n continue\n await plugin.before_run(self, session, context, session.state)\n\n # assemble final input messages from context\n\n # ... actual running, i.e. `get_response` for ChatAgent ...\n\n # After-run plugins (reverse order)\n for plugin in reversed(self._context_providers):\n await plugin.after_run(self, session, context, session.state)\n\n\n# Plugin that maintains state - modifies dict in place\nclass InMemoryHistoryProvider(ContextProvider):\n async def before_run(\n self,\n agent: \"SupportsAgentRun\",\n session: AgentSession,\n context: SessionContext,\n state: dict[str, Any],\n ) -> None:\n # Read from state (use source_id as key for namespace)\n my_state = state.get(self.source_id, {})\n messages = my_state.get(\"messages\", [])\n context.extend_messages(self.source_id, messages)\n\n async def after_run(\n self,\n agent: \"SupportsAgentRun\",\n session: AgentSession,\n context: SessionContext,\n state: dict[str, Any],\n ) -> None:\n # Modify state dict in place - no return needed\n my_state = state.setdefault(self.source_id, {})\n messages = my_state.get(\"messages\", [])\n my_state[\"messages\"] = [\n *messages,\n *context.input_messages,\n *(context.response.messages or []),\n ]\n\n\n# Stateless plugin - ignores state\nclass TimeContextProvider(ContextProvider):\n async def before_run(\n self,\n agent: \"SupportsAgentRun\",\n session: AgentSession,\n context: SessionContext,\n state: dict[str, Any],\n ) -> None:\n context.extend_instructions(self.source_id, f\"Current time: {datetime.now()}\")\n\n async def after_run(\n self,\n agent: \"SupportsAgentRun\",\n session: AgentSession,\n context: SessionContext,\n state: dict[str, Any],\n ) -> None:\n pass # No state, nothing to do after\n```\n\n##### Option B2: SessionState Object\n\nThe session stores state in a dedicated `SessionState` object. Each hook receives its own state slice through a mutable wrapper that writes back automatically.\n\n```python\nclass HookState:\n \"\"\"Mutable wrapper for a single hook's state.\n\n Changes are written back to the session state automatically.\n \"\"\"\n\n def __init__(self, session_state: dict[str, dict[str, Any]], source_id: str):\n self._session_state = session_state\n self._source_id = source_id\n if source_id not in session_state:\n session_state[source_id] = {}\n\n def get(self, key: str, default: Any = None) -> Any:\n return self._session_state[self._source_id].get(key, default)\n\n def set(self, key: str, value: Any) -> None:\n self._session_state[self._source_id][key] = value\n\n def update(self, values: dict[str, Any]) -> None:\n self._session_state[self._source_id].update(values)\n\n\nclass SessionState:\n \"\"\"Structured state container for a session.\"\"\"\n\n def __init__(self, session_id: str):\n self.session_id = session_id\n self.service_session_id: str | None = None\n self._hook_state: dict[str, dict[str, Any]] = {} # source_id -> state\n\n def get_hook_state(self, source_id: str) -> HookState:\n \"\"\"Get mutable state wrapper for a specific hook.\"\"\"\n return HookState(self._hook_state, source_id)\n\n\nclass AgentSession:\n \"\"\"Session holds a SessionState object.\"\"\"\n\n def __init__(self, *, session_id: str | None = None):\n self._session_id = session_id or str(uuid.uuid4())\n self._state = SessionState(self._session_id)\n\n @property\n def state(self) -> SessionState:\n return self._state\n\n\nclass ContextHooksRunner:\n \"\"\"Agent-owned runner that executes hooks with session state.\"\"\"\n\n def __init__(self, hooks: Sequence[ContextHooks]):\n self._hooks = list(hooks)\n\n async def run_before(\n self,\n context: SessionContext,\n session_state: SessionState,\n ) -> None:\n \"\"\"Run before_run for all hooks.\"\"\"\n for hook in self._hooks:\n my_state = session_state.get_hook_state(hook.source_id)\n await hook.before_run(context, my_state)\n\n async def run_after(\n self,\n context: SessionContext,\n session_state: SessionState,\n ) -> None:\n \"\"\"Run after_run for all hooks in reverse order.\"\"\"\n for hook in reversed(self._hooks):\n my_state = session_state.get_hook_state(hook.source_id)\n await hook.after_run(context, my_state)\n\n\n# Hook uses HookState wrapper - no return needed\nclass InMemoryStorageHooks(ContextHooks):\n async def before_run(\n self,\n context: SessionContext,\n state: HookState, # Mutable wrapper\n ) -> None:\n messages = state.get(\"messages\", [])\n context.add_messages(self.source_id, messages)\n\n async def after_run(\n self,\n context: SessionContext,\n state: HookState, # Mutable wrapper\n ) -> None:\n messages = state.get(\"messages\", [])\n state.set(\"messages\", [\n *messages,\n *context.input_messages,\n *(context.response_messages or []),\n ])\n\n\n# Stateless hook - state wrapper provided but not used\nclass TimeContextHooks(ContextHooks):\n async def before_run(\n self,\n context: SessionContext,\n state: HookState,\n ) -> None:\n context.add_instructions(self.source_id, f\"Current time: {datetime.now()}\")\n\n async def after_run(\n self,\n context: SessionContext,\n state: HookState,\n ) -> None:\n pass # Nothing to do\n```\n\n**Option B Pros (both variants):**\n- Lightweight sessions - just data, serializable via `to_dict()`/`from_dict()`\n- Plugin instances shared across sessions (more memory efficient)\n- Clearer separation: agent = behavior, session = state\n\n**Option B Cons (both variants):**\n- More complex execution model (agent + session coordination)\n- Plugins must explicitly read/write state (no implicit instance variables)\n- Session given to another agent may not work (different plugins configuration)\n\n**B1 vs B2:**\n\n| Aspect | B1: Simple Dict (CHOSEN) | B2: SessionState Object |\n|--------|-----------------|-------------------------|\n| Simplicity | Simpler, less abstraction | More structure, helper methods |\n| State passing | Whole dict passed, mutate in place | Mutable wrapper, no return needed |\n| Type safety | `dict[str, Any]` - loose | Can add type hints on methods |\n| Extensibility | Add keys as needed | Can add methods/validation |\n| Serialization | Direct JSON serialization | Need custom serialization |\n\n#### Comparison\n\n| Aspect | Option A: Instances in Session | Option B: Instances in Agent (CHOSEN) |\n|--------|-------------------------------|------------------------------|\n| Session weight | Heavier (instances + state) | Lighter (state only) |\n| Plugin sharing | Per-session instances | Shared across sessions |\n| Instance state | Natural (instance variables) | Explicit (state dict) |\n| Serialization | Serialize session + plugins | `session.to_dict()`/`AgentSession.from_dict()` |\n| Factory handling | Resolved at session creation | Not needed (state dict handles per-session needs) |\n| Signature | `before_run(context)` | `before_run(agent, session, context, state)` |\n| Session portability | Works with any agent | Tied to agent's plugins config |\n\n#### Factories Not Needed with Option B\n\nWith Option B (instances in agent, state in session), the plugins are shared across sessions and the explicit state dict handles per-session needs. Therefore, **factory support is not needed**:\n\n- State is externalized to the session's `state: dict[str, Any]`\n- If a plugin needs per-session initialization, it can do so in `before_run` on first call (checking if state is empty)\n- All plugins are shared across sessions (more memory efficient)\n- Plugins use `state.setdefault(self.source_id, {})` to namespace their state\n\n---\n## Decision Outcome\n\n### Decision 1: Execution Pattern\n\n**Chosen: Option 3 - Hooks (Pre/Post Pattern)** with the following naming:\n- **Class name:** `ContextProvider` (emphasizes extensibility, familiar from build tools, and does not favor reading or writing)\n- **Method names:** `before_run` / `after_run` (matches `agent.run()` terminology)\n\nRationale:\n- Simpler mental model: \"before\" runs before, \"after\" runs after - no nesting to understand\n- Easier to implement plugins that only need one phase (just override one method)\n- More similar to the current `ContextProvider` API (`invoking`/`invoked`), easing migration\n- Clearer separation between what this does vs what Agent Middleware can do\n\nBoth options share the same:\n- Agent vs Session ownership model\n- `source_id` attribution\n- Natively serializable sessions (state dict is JSON-serializable)\n- Session management methods (`create_session`, `get_session`)\n- Renaming `AgentThread` → `AgentSession`\n\n### Decision 2: Instance Ownership (Orthogonal)\n\n**Chosen: Option B1 - Instances in Agent, State in Session (Simple Dict)**\n\nThe agent (any `SupportsAgentRun` implementation) owns and manages the `ContextProvider` instances. The `AgentSession` only stores state as a mutable `dict[str, Any]`. Each plugin receives the **whole state dict** (not just its own slice), and since a dict is mutable, no return value is needed - plugins modify the dict in place.\n\nRationale for B over A:\n- Lightweight sessions - just data, serializable via `to_dict()`/`from_dict()`\n- Plugin instances shared across sessions (more memory efficient)\n- Clearer separation: agent = behavior, session = state\n- Factories not needed - state dict handles per-session needs\n\nRationale for B1 over B2: Simpler is better. The whole state dict is passed to each plugin, and since Python dicts are mutable, plugins can modify state in place without returning anything. This is the most Pythonic approach.\n\n> **Note on trust:** Since all `ContextProvider` instances reason over conversation messages (which may contain sensitive user data), they should be **trusted by default**. This is also why we allow all plugins to see all state - if a plugin is untrusted, it shouldn't be in the pipeline at all. The whole state dict is passed rather than isolated slices because plugins that handle messages already have access to the full conversation context.\n\n\n### Addendum (2026-02-17): Provider-scoped hook state and default source IDs\n\nThis addendum introduces a **breaking change** that supersedes earlier references in this ADR where hooks received the\nentire `session.state` object as their `state` parameter.\n\n#### Hook state contract\n\n- `before_run` and `after_run` now receive a **provider-scoped** mutable state dict.\n- The framework passes `session.state.setdefault(provider.source_id, {})` to hook `state`.\n- Cross-provider/global inspection remains available through `session.state` on `AgentSession`.\n\n#### Session requirement and fallback behavior\n\n- Provider hooks must use session-backed scoped state; there is no ad-hoc `{}` fallback state.\n- If providers run without a caller-supplied session, the framework creates an internal run-scoped `AgentSession` and\n passes provider-scoped state from that session.\n\n#### Migration guidance\n\nMigrate provider implementations and samples from nested access to scoped access:\n\n- `state[self.source_id][\"key\"]` → `state[\"key\"]`\n- `state.setdefault(self.source_id, {})[\"key\"]` → `state[\"key\"]`\n\n#### DEFAULT_SOURCE_ID standardization\n\nAligned with and extending [PR #3944](https://github.com/microsoft/agent-framework/pull/3944), all built-in/connector\nproviders in this surface now define a `DEFAULT_SOURCE_ID` and allow constructor override via `source_id`.\n\nNaming convention:\n\n- snake_case\n- close to the provider class name\n- history providers may use `*_memory` where differentiation is useful\n\nDefaults introduced by this change:\n\n- `InMemoryHistoryProvider.DEFAULT_SOURCE_ID = \"in_memory\"`\n- `Mem0ContextProvider.DEFAULT_SOURCE_ID = \"mem0\"`\n- `RedisContextProvider.DEFAULT_SOURCE_ID = \"redis\"`\n- `RedisHistoryProvider.DEFAULT_SOURCE_ID = \"redis_memory\"`\n- `AzureAISearchContextProvider.DEFAULT_SOURCE_ID = \"azure_ai_search\"`\n- `FoundryMemoryProvider.DEFAULT_SOURCE_ID = \"foundry_memory\"`\n\n\n## Comparison to .NET Implementation\n\nThe .NET Agent Framework provides equivalent functionality through a different structure. Both implementations achieve the same goals using idioms natural to their respective languages.\n\n### Concept Mapping\n\n| .NET Concept | Python (Chosen) |\n|--------------|-----------------|\n| `AIContextProvider` (abstract base) | `ContextProvider` |\n| `ChatHistoryProvider` (abstract base) | `HistoryProvider` |\n| `AIContext` (return from `InvokingAsync`) | `SessionContext` (mutable, passed through) |\n| `AgentSession` / `ChatClientAgentSession` | `AgentSession` |\n| `InMemoryChatHistoryProvider` | `InMemoryHistoryProvider` |\n| `ChatClientAgentOptions` factory delegates | Not needed - state dict handles per-session needs |\n\n### Feature Equivalence\n\nBoth platforms provide the same core capabilities:\n\n| Capability | .NET | Python |\n|------------|------|--------|\n| Inject context before invocation | `AIContextProvider.InvokingAsync()` → returns `AIContext` with `Instructions`, `Messages`, `Tools` | `ContextProvider.before_run()` → mutates `SessionContext` in place |\n| React after invocation | `AIContextProvider.InvokedAsync()` | `ContextProvider.after_run()` |\n| Load conversation history | `ChatHistoryProvider.InvokingAsync()` → returns `IEnumerable` | `HistoryProvider.before_run()` → calls `context.extend_messages()` |\n| Store conversation history | `ChatHistoryProvider.InvokedAsync()` | `HistoryProvider.after_run()` → calls `save_messages()` |\n| Session serialization | `Serialize()` on providers → `JsonElement` | `session.to_dict()`/`AgentSession.from_dict()` — providers write JSON-serializable values to `session.state` |\n| Factory-based creation | `Func>` delegates on `ChatClientAgentOptions` | Not needed - state dict handles per-session needs |\n| Default storage | Auto-injects `InMemoryChatHistoryProvider` when no `ChatHistoryProvider` or `ConversationId` set | Auto-injects `InMemoryHistoryProvider` when no providers and `conversation_id` or `store=True` |\n| Service-managed history | `ConversationId` property (mutually exclusive with `ChatHistoryProvider`) | `service_session_id` on `AgentSession` |\n| Message reduction | `IChatReducer` on `InMemoryChatHistoryProvider` | Not yet designed (see Open Discussion: Context Compaction) |\n\n### Implementation Differences\n\nThe implementations differ in ways idiomatic to each language:\n\n| Aspect | .NET Approach | Python Approach |\n|--------|---------------|-----------------|\n| **Context providers** | Separate `AIContextProvider` and `ChatHistoryProvider` (one of each per session) | Unified list of `ContextProvider` (multiple) |\n| **Composition** | One of each provider type per session | Unlimited providers in pipeline |\n| **Context passing** | `InvokingAsync()` returns `AIContext` (instructions + messages + tools) | `before_run()` mutates `SessionContext` in place |\n| **Response access** | `InvokedContext` carries response messages | `SessionContext.response` carries full `AgentResponse` (messages, response_id, usage_details, etc.) |\n| **Type system** | Strict abstract classes, compile-time checks | Duck typing, protocols, runtime flexibility |\n| **Configuration** | Factory delegates on `ChatClientAgentOptions` | Direct instantiation, list of instances |\n| **State management** | Instance state in providers, serialized via `JsonElement` | Explicit state dict in session, serialized via `session.to_dict()` |\n| **Default storage** | Auto-injects `InMemoryChatHistoryProvider` when neither `ChatHistoryProvider` nor `ConversationId` is set | Auto-injects `InMemoryHistoryProvider` when no providers and `conversation_id` or `store=True` |\n| **Source tracking** | Limited - `message.source_id` in observability/DevUI only | Built-in `source_id` on every provider, keyed in `context_messages` dict |\n| **Service discovery** | `GetService()` on providers and sessions | Not applicable - Python uses direct references |\n\n### Design Trade-offs\n\nEach approach has trade-offs that align with language conventions:\n\n**.NET's separate provider types:**\n- Clearer separation between context injection and history storage\n- Easier to detect \"missing storage\" and auto-inject defaults (checks for `ChatHistoryProvider` or `ConversationId`)\n- Type system enforces single provider of each type\n- `AIContext` return type makes it clear what context is being added (instructions vs messages vs tools)\n- `GetService()` pattern enables provider discovery without tight coupling\n\n**Python's unified pipeline:**\n- Single abstraction for all context concerns\n- Multiple instances of same type (e.g., multiple storage backends with different `source_id`s)\n- More explicit - customization means owning full configuration\n- `source_id` enables filtering/debugging across all sources\n- Mutable `SessionContext` avoids allocating return objects\n- Explicit state dict makes serialization trivial (no `JsonElement` layer)\n\nNeither approach is inherently better - they reflect different language philosophies while achieving equivalent functionality. The Python design embraces the \"we're all consenting adults\" philosophy, while .NET provides more compile-time guardrails.\n\n---\n\n## Open Discussion: Context Compaction\n\n### Problem Statement\n\nA common need for long-running agents is **context compaction** - automatically summarizing or truncating conversation history when approaching token limits. This is particularly important for agents that make many tool calls in succession (10s or 100s), where the context can grow unboundedly.\n\nCurrently, this is challenging because:\n- `ChatMessageStore.list_messages()` is only called once at the start of `agent.run()`, not during the tool loop\n- `ChatMiddleware` operates on a copy of messages, so modifications don't persist across tool loop iterations\n- The function calling loop happens deep within the `ChatClient`, which is below the agent level\n\n### Design Question\n\nShould `ContextPlugin` be invoked:\n1. **Only at agent invocation boundaries** (current proposal) - before/after each `agent.run()` call\n2. **During the tool loop** - before/after each model call within a single `agent.run()`\n\n### Boundary vs In-Run Compaction\n\nWhile boundary and in-run compaction could potentially use the same mechanism, they have **different goals and behaviors**:\n\n**Boundary compaction** (before/after `agent.run()`):\n- **Before run**: Keep context manageable - load a compacted view of history\n- **After run**: Keep storage compact - summarize/truncate before persisting\n- Useful for maintaining reasonable context sizes across conversation turns\n- One reason to have **multiple storage plugins**: persist compacted history for use during runs, while also storing the full uncompacted history for auditing and evaluations\n\n**In-run compaction** (during function calling loops):\n- Relevant for **function calling scenarios** where many tool calls accumulate\n- Typically **in-memory only** - no need to persist intermediate compaction and only useful when the conversation/session is _not_ managed by the service\n- Different strategies apply:\n - Remove old function call/result pairs entirely/Keep only the most recent N tool interactions\n - Replace call/result pairs with a single summary message (with a different role)\n - Summarize several function call/result pairs into one larger context message\n\n### Service-Managed vs Local Storage\n\n**Important:** In-run compaction is relevant only for **non-service-managed histories**. When using service-managed storage (`service_session_id` is set):\n- The service handles history management internally\n- Only the new calls and results are sent to/from the service each turn\n- The service is responsible for its own compaction strategy, but we do not control that\n\nFor local storage, a full message list is sent to the model each time, making compaction the client's responsibility.\n\n### Options\n\n**Option A: Invocation-boundary only (current proposal)**\n- Simpler mental model\n- Consistent with `AgentMiddleware` pattern\n- In-run compaction would need to happen via a separate mechanism (e.g., `ChatMiddleware` at the client level)\n- Risk: Different compaction mechanisms at different layers could be confusing\n\n**Option B: Also during tool loops**\n- Single mechanism for all context manipulation\n- More powerful but more complex\n- Requires coordination with `ChatClient` internals\n- Risk: Performance overhead if plugins are expensive\n\n**Option C: Unified approach across layers**\n- Define a single context compaction abstraction that works at both agent and client levels\n- `ContextPlugin` could delegate to `ChatMiddleware` for mid-loop execution\n- Requires deeper architectural thought\n\n### Potential Extension Points (for any option)\n\nRegardless of the chosen approach, these extension points could support compaction:\n- A `CompactionStrategy` that can be shared between plugins and function calling configuration\n- Hooks for `ChatClient` to notify the agent layer when context limits are approaching\n- A unified `ContextManager` that coordinates compaction across layers\n- **Message-level attribution**: The `attribution` marker in `ChatMessage.additional_properties` can be used during compaction to identify messages that should be preserved (e.g., `attribution: \"important\"`) or that are safe to remove (e.g., `attribution: \"ephemeral\"`). This prevents accidental filtering of critical context during aggressive compaction.\n\n> **Note:** The .NET SDK currently has a `ChatReducer` interface for context reduction/compaction. We should consider adopting similar naming in Python (e.g., `ChatReducer` or `ContextReducer`) for cross-platform consistency.\n\n**This section requires further discussion.**\n\n## Implementation Plan\n\nSee **Appendix A** for class hierarchy, API signatures, and user experience examples.\nSee the **Workplan** at the end for PR breakdown and reference implementation.\n\n---\n\n## Appendix A: API Overview\n\n### Class Hierarchy\n\n```\nContextProvider (base - hooks pattern)\n├── HistoryProvider (storage subclass)\n│ ├── InMemoryHistoryProvider (built-in)\n│ ├── RedisHistoryProvider (packages/redis)\n│ └── CosmosHistoryProvider (packages/azure-ai)\n├── AzureAISearchContextProvider (packages/azure-ai-search)\n├── Mem0ContextProvider (packages/mem0)\n└── (custom user providers)\n\nAgentSession (lightweight state container)\n\nSessionContext (per-invocation state)\n```\n\n### ContextProvider\n\n```python\nclass ContextProvider(ABC):\n \"\"\"Base class for context providers (hooks pattern).\n\n Context providers participate in the context engineering pipeline,\n adding context before model invocation and processing responses after.\n\n Attributes:\n source_id: Unique identifier for this provider instance (required).\n Used for message/tool attribution so other providers can filter.\n \"\"\"\n\n def __init__(self, source_id: str):\n self.source_id = source_id\n\n async def before_run(\n self,\n agent: \"SupportsAgentRun\",\n session: AgentSession,\n context: SessionContext,\n state: dict[str, Any],\n ) -> None:\n \"\"\"Called before model invocation. Override to add context.\"\"\"\n pass\n\n async def after_run(\n self,\n agent: \"SupportsAgentRun\",\n session: AgentSession,\n context: SessionContext,\n state: dict[str, Any],\n ) -> None:\n \"\"\"Called after model invocation. Override to process response.\"\"\"\n pass\n```\n\n> **Serialization contract:** Any values a provider writes to `state` must be JSON-serializable. Sessions are serialized via `session.to_dict()` and restored via `AgentSession.from_dict()`.\n\n> **Agent-agnostic:** The `agent` parameter is typed as `SupportsAgentRun` (the base protocol), not `ChatAgent`. Context providers work with any agent implementation.\n\n### HistoryProvider\n\n```python\nclass HistoryProvider(ContextProvider):\n \"\"\"Base class for conversation history storage providers.\n\n Subclasses only need to implement get_messages() and save_messages().\n The default before_run/after_run handle loading and storing based on\n configuration flags. Override them for custom behavior.\n\n A single class configured for different use cases:\n - Primary memory storage (loads + stores messages)\n - Audit/logging storage (stores only, doesn't load)\n - Evaluation storage (stores only for later analysis)\n\n Loading behavior:\n - `load_messages=True` (default): Load messages from storage in before_run\n - `load_messages=False`: Agent skips `before_run` entirely (audit/logging mode)\n\n Storage behavior:\n - `store_inputs`: Store input messages (default True)\n - `store_responses`: Store response messages (default True)\n - `store_context_messages`: Also store context from other providers (default False)\n - `store_context_from`: Only store from specific source_ids (default None = all)\n \"\"\"\n\n def __init__(\n self,\n source_id: str,\n *,\n load_messages: bool = True,\n store_inputs: bool = True,\n store_responses: bool = True,\n store_context_messages: bool = False,\n store_context_from: Sequence[str] | None = None,\n ): ...\n\n # --- Subclasses implement these ---\n\n @abstractmethod\n async def get_messages(self, session_id: str | None) -> list[ChatMessage]:\n \"\"\"Retrieve stored messages for this session.\"\"\"\n ...\n\n @abstractmethod\n async def save_messages(self, session_id: str | None, messages: Sequence[ChatMessage]) -> None:\n \"\"\"Persist messages for this session.\"\"\"\n ...\n\n # --- Default implementations (override for custom behavior) ---\n\n async def before_run(self, agent, session, context, state) -> None:\n \"\"\"Load history into context. Skipped by the agent when load_messages=False.\"\"\"\n history = await self.get_messages(context.session_id)\n context.extend_messages(self.source_id, history)\n\n async def after_run(self, agent, session, context, state) -> None:\n \"\"\"Store messages based on store_* configuration flags.\"\"\"\n messages_to_store: list[ChatMessage] = []\n # Optionally include context from other providers\n if self.store_context_messages:\n if self.store_context_from:\n messages_to_store.extend(context.get_messages(sources=self.store_context_from))\n else:\n messages_to_store.extend(context.get_messages(exclude_sources=[self.source_id]))\n if self.store_inputs:\n messages_to_store.extend(context.input_messages)\n if self.store_responses and context.response.messages:\n messages_to_store.extend(context.response.messages)\n if messages_to_store:\n await self.save_messages(context.session_id, messages_to_store)\n```\n\n### SessionContext\n\n```python\nclass SessionContext:\n \"\"\"Per-invocation state passed through the context provider pipeline.\n\n Created fresh for each agent.run() call. Providers read from and write to\n the mutable fields to add context before invocation and process responses after.\n\n Attributes:\n session_id: The ID of the current session\n service_session_id: Service-managed session ID (if present)\n input_messages: New messages being sent to the agent (set by caller)\n context_messages: Dict mapping source_id -> messages added by that provider.\n Maintains insertion order (provider execution order).\n instructions: Additional instructions - providers can append here\n tools: Additional tools - providers can append here\n response (property): After invocation, contains the full AgentResponse (set by agent).\n Includes response.messages, response.response_id, response.agent_id,\n response.usage_details, etc. Read-only property - use AgentMiddleware to modify.\n options: Options passed to agent.run() - READ-ONLY, for reflection only\n metadata: Shared metadata dictionary for cross-provider communication\n \"\"\"\n\n def __init__(\n self,\n *,\n session_id: str | None = None,\n service_session_id: str | None = None,\n input_messages: list[ChatMessage],\n context_messages: dict[str, list[ChatMessage]] | None = None,\n instructions: list[str] | None = None,\n tools: list[ToolProtocol] | None = None,\n options: dict[str, Any] | None = None,\n metadata: dict[str, Any] | None = None,\n ): ...\n self._response: \"AgentResponse | None\" = None\n\n @property\n def response(self) -> \"AgentResponse | None\":\n \"\"\"The agent's response. Set by the framework after invocation, read-only for providers.\"\"\"\n ...\n\n def extend_messages(self, source_id: str, messages: Sequence[ChatMessage]) -> None:\n \"\"\"Add context messages from a specific source.\"\"\"\n ...\n\n def extend_instructions(self, source_id: str, instructions: str | Sequence[str]) -> None:\n \"\"\"Add instructions to be prepended to the conversation.\"\"\"\n ...\n\n def extend_tools(self, source_id: str, tools: Sequence[ToolProtocol]) -> None:\n \"\"\"Add tools with source attribution in tool.metadata.\"\"\"\n ...\n\n def get_messages(\n self,\n *,\n sources: Sequence[str] | None = None,\n exclude_sources: Sequence[str] | None = None,\n include_input: bool = False,\n include_response: bool = False,\n ) -> list[ChatMessage]:\n \"\"\"Get context messages, optionally filtered and optionally including input/response.\n\n Returns messages in provider execution order (dict insertion order),\n with input and response appended if requested.\n \"\"\"\n ...\n```\n\n### AgentSession (Decision B1)\n\n```python\nclass AgentSession:\n \"\"\"A conversation session with an agent.\n\n Lightweight state container. Provider instances are owned by the agent,\n not the session. The session only holds session IDs and a mutable state dict.\n \"\"\"\n\n def __init__(self, *, session_id: str | None = None):\n self._session_id = session_id or str(uuid.uuid4())\n self.service_session_id: str | None = None\n self.state: dict[str, Any] = {}\n\n @property\n def session_id(self) -> str:\n return self._session_id\n\n def to_dict(self) -> dict[str, Any]:\n \"\"\"Serialize session to a plain dict.\"\"\"\n return {\n \"type\": \"session\",\n \"session_id\": self._session_id,\n \"service_session_id\": self.service_session_id,\n \"state\": self.state,\n }\n\n @classmethod\n def from_dict(cls, data: dict[str, Any]) -> \"AgentSession\":\n \"\"\"Restore session from a dict.\"\"\"\n session = cls(session_id=data[\"session_id\"])\n session.service_session_id = data.get(\"service_session_id\")\n session.state = data.get(\"state\", {})\n return session\n```\n\n### ChatAgent Integration\n\n```python\nclass ChatAgent:\n def __init__(\n self,\n chat_client: ...,\n *,\n context_providers: Sequence[ContextProvider] | None = None,\n ):\n self._context_providers = list(context_providers or [])\n\n def create_session(self, *, session_id: str | None = None) -> AgentSession:\n \"\"\"Create a new lightweight session.\"\"\"\n return AgentSession(session_id=session_id)\n\n def get_session(self, service_session_id: str, *, session_id: str | None = None) -> AgentSession:\n \"\"\"Get or create a session for a service-managed session ID.\"\"\"\n session = AgentSession(session_id=session_id)\n session.service_session_id = service_session_id\n return session\n\n async def run(self, input: str, *, session: AgentSession, options: dict[str, Any] | None = None) -> AgentResponse:\n options = options or {}\n\n # Auto-add InMemoryHistoryProvider when no providers and conversation_id/store requested\n if not self._context_providers and (options.get(\"conversation_id\") or options.get(\"store\") is True):\n self._context_providers.append(InMemoryHistoryProvider(\"memory\"))\n\n context = SessionContext(session_id=session.session_id, input_messages=[...])\n\n # Before-run providers (forward order, skip HistoryProviders with load_messages=False)\n for provider in self._context_providers:\n if isinstance(provider, HistoryProvider) and not provider.load_messages:\n continue\n await provider.before_run(self, session, context, session.state)\n\n # ... assemble messages, invoke model ...\n context._response = response # Set the full AgentResponse for after_run access\n\n # After-run providers (reverse order)\n for provider in reversed(self._context_providers):\n await provider.after_run(self, session, context, session.state)\n```\n\n### Message/Tool Attribution\n\nThe `SessionContext` provides explicit methods for adding context:\n\n```python\n# Adding messages (keyed by source_id in context_messages dict)\ncontext.extend_messages(self.source_id, messages)\n\n# Adding instructions (flat list, source_id for debugging)\ncontext.extend_instructions(self.source_id, \"Be concise and helpful.\")\ncontext.extend_instructions(self.source_id, [\"Instruction 1\", \"Instruction 2\"])\n\n# Adding tools (source attribution added to tool.metadata automatically)\ncontext.extend_tools(self.source_id, [my_tool, another_tool])\n\n# Getting all context messages in provider execution order\nall_context = context.get_messages()\n\n# Including input and response messages too\nfull_conversation = context.get_messages(include_input=True, include_response=True)\n\n# Filtering by source\nmemory_messages = context.get_messages(sources=[\"memory\"])\nnon_rag_messages = context.get_messages(exclude_sources=[\"rag\"])\n\n# Direct access to check specific sources\nif \"memory\" in context.context_messages:\n history = context.context_messages[\"memory\"]\n```\n\n---\n\n## User Experience Examples\n\n### Example 0: Zero-Config Default (Simplest Use Case)\n\n```python\nfrom agent_framework import ChatAgent\n\n# No providers configured - but conversation history still works!\nagent = ChatAgent(\n chat_client=client,\n name=\"assistant\",\n # No context_providers specified\n)\n\n# Create session - automatically gets InMemoryHistoryProvider when conversation_id or store=True\nsession = agent.create_session()\nresponse = await agent.run(\"Hello, my name is Alice!\", session=session)\n\n# Conversation history is preserved automatically\nresponse = await agent.run(\"What's my name?\", session=session)\n# Agent remembers: \"Your name is Alice!\"\n\n# With service-managed session - no default storage added (service handles it)\nservice_session = agent.create_session(service_session_id=\"thread_abc123\")\n\n# With store=True in options - user expects service storage, no default added\nresponse = await agent.run(\"Hello!\", session=session, options={\"store\": True})\n```\n\n### Example 1: Explicit Memory Storage\n\n```python\nfrom agent_framework import ChatAgent, InMemoryHistoryProvider\n\n# Explicit provider configuration (same behavior as default, but explicit)\nagent = ChatAgent(\n chat_client=client,\n name=\"assistant\",\n context_providers=[\n InMemoryHistoryProvider(source_id=\"memory\")\n ]\n)\n\n# Create session and chat\nsession = agent.create_session()\nresponse = await agent.run(\"Hello!\", session=session)\n\n# Messages are automatically stored and loaded on next invocation\nresponse = await agent.run(\"What did I say before?\", session=session)\n```\n\n### Example 2: RAG + Memory + Audit (All HistoryProvider)\n\n```python\nfrom agent_framework import ChatAgent\nfrom agent_framework.azure import CosmosHistoryProvider, AzureAISearchContextProvider\nfrom agent_framework.redis import RedisHistoryProvider\n\n# RAG provider that injects relevant documents\nsearch_provider = AzureAISearchContextProvider(\n source_id=\"rag\",\n endpoint=\"https://...\",\n index_name=\"documents\",\n)\n\n# Primary memory storage (loads + stores)\n# load_messages=True (default) - loads and stores messages\nmemory_provider = RedisHistoryProvider(\n source_id=\"memory\",\n redis_url=\"redis://...\",\n)\n\n# Audit storage - SAME CLASS, different configuration\n# load_messages=False = never loads, just stores for audit\naudit_provider = CosmosHistoryProvider(\n source_id=\"audit\",\n connection_string=\"...\",\n load_messages=False, # Don't load - just store for audit\n)\n\nagent = ChatAgent(\n chat_client=client,\n name=\"assistant\",\n context_providers=[\n memory_provider, # First: loads history\n search_provider, # Second: adds RAG context\n audit_provider, # Third: stores for audit (no load)\n ]\n)\n```\n\n### Example 3: Custom Context Providers\n\n```python\nfrom agent_framework import ContextProvider, SessionContext\n\nclass TimeContextProvider(ContextProvider):\n \"\"\"Adds current time to the context.\"\"\"\n\n async def before_run(self, agent, session, context, state) -> None:\n from datetime import datetime\n context.extend_instructions(\n self.source_id,\n f\"Current date and time: {datetime.now().isoformat()}\"\n )\n\n\nclass UserPreferencesProvider(ContextProvider):\n \"\"\"Tracks and applies user preferences from conversation.\"\"\"\n\n async def before_run(self, agent, session, context, state) -> None:\n prefs = state.get(self.source_id, {}).get(\"preferences\", {})\n if prefs:\n context.extend_instructions(\n self.source_id,\n f\"User preferences: {json.dumps(prefs)}\"\n )\n\n async def after_run(self, agent, session, context, state) -> None:\n # Extract preferences from response and store in session state\n for msg in context.response.messages or []:\n if \"preference:\" in msg.text.lower():\n my_state = state.setdefault(self.source_id, {})\n my_state.setdefault(\"preferences\", {})\n # ... extract and store preference\n\n\n# Compose providers - each with mandatory source_id\nagent = ChatAgent(\n chat_client=client,\n context_providers=[\n InMemoryHistoryProvider(source_id=\"memory\"),\n TimeContextProvider(source_id=\"time\"),\n UserPreferencesProvider(source_id=\"prefs\"),\n ]\n)\n```\n\n### Example 4: Filtering by Source (Using Dict-Based Context)\n\n```python\nclass SelectiveContextProvider(ContextProvider):\n \"\"\"Provider that only processes messages from specific sources.\"\"\"\n\n async def before_run(self, agent, session, context, state) -> None:\n # Check what sources have added messages so far\n print(f\"Sources so far: {list(context.context_messages.keys())}\")\n\n # Get messages excluding RAG context\n non_rag_messages = context.get_messages(exclude_sources=[\"rag\"])\n\n # Or get only memory messages\n if \"memory\" in context.context_messages:\n memory_only = context.context_messages[\"memory\"]\n\n # Do something with filtered messages...\n # e.g., sentiment analysis, topic extraction\n\n\nclass RAGContextProvider(ContextProvider):\n \"\"\"Provider that adds RAG context.\"\"\"\n\n async def before_run(self, agent, session, context, state) -> None:\n # Search for relevant documents based on input\n relevant_docs = await self._search(context.input_messages)\n\n # Add RAG context using explicit method\n rag_messages = [\n ChatMessage(role=\"system\", text=f\"Relevant info: {doc}\")\n for doc in relevant_docs\n ]\n context.extend_messages(self.source_id, rag_messages)\n```\n\n### Example 5: Explicit Storage Configuration for Service-Managed Sessions\n\n```python\n# HistoryProvider uses explicit configuration - no automatic detection.\n# load_messages=True (default): Load messages from storage\n# load_messages=False: Skip loading (useful for audit-only storage)\n\nagent = ChatAgent(\n chat_client=client,\n context_providers=[\n RedisHistoryProvider(\n source_id=\"memory\",\n redis_url=\"redis://...\",\n # load_messages=True is the default\n )\n ]\n)\n\nsession = agent.create_session()\n\n# Normal run - loads and stores messages\nresponse = await agent.run(\"Hello!\", session=session)\n\n# For service-managed sessions, configure storage explicitly:\n# - Use load_messages=False when service handles history\nservice_storage = RedisHistoryProvider(\n source_id=\"audit\",\n redis_url=\"redis://...\",\n load_messages=False, # Don't load - service manages history\n)\n\nagent_with_service = ChatAgent(\n chat_client=client,\n context_providers=[service_storage]\n)\nservice_session = agent_with_service.create_session(service_session_id=\"thread_abc123\")\nresponse = await agent_with_service.run(\"Hello!\", session=service_session)\n# History provider stores for audit but doesn't load (service handles history)\n```\n\n### Example 6: Multiple Instances of Same Provider Type\n\n```python\n# You can have multiple instances of the same provider class\n# by using different source_ids\n\nagent = ChatAgent(\n chat_client=client,\n context_providers=[\n # Primary storage for conversation history\n RedisHistoryProvider(\n source_id=\"conversation_memory\",\n redis_url=\"redis://primary...\",\n load_messages=True, # This one loads\n ),\n # Secondary storage for audit (different Redis instance)\n RedisHistoryProvider(\n source_id=\"audit_log\",\n redis_url=\"redis://audit...\",\n load_messages=False, # This one just stores\n ),\n ]\n)\n# Warning will NOT be logged because only one has load_messages=True\n```\n\n### Example 7: Provider Ordering - RAG Before vs After Memory\n\nThe order of providers determines what context each one can see. This is especially important for RAG, which may benefit from seeing conversation history.\n\n```python\nfrom agent_framework import ChatAgent\nfrom agent_framework.context import InMemoryHistoryProvider, ContextProvider, SessionContext\n\nclass RAGContextProvider(ContextProvider):\n \"\"\"RAG provider that retrieves relevant documents based on available context.\"\"\"\n\n async def before_run(self, agent, session, context, state) -> None:\n # Build query from what we can see\n query_parts = []\n\n # We can always see the current input\n for msg in context.input_messages:\n query_parts.append(msg.text)\n\n # Can we see history? Depends on provider order!\n history = context.get_messages() # Gets context from providers that ran before us\n if history:\n # Include recent history for better RAG context\n recent = history[-3:] # Last 3 messages\n for msg in recent:\n query_parts.append(msg.text)\n\n query = \" \".join(query_parts)\n documents = await self._retrieve_documents(query)\n\n # Add retrieved documents as context\n rag_messages = [ChatMessage.system(f\"Relevant context:\\n{doc}\") for doc in documents]\n context.extend_messages(self.source_id, rag_messages)\n\n async def _retrieve_documents(self, query: str) -> list[str]:\n # ... vector search implementation\n return [\"doc1\", \"doc2\"]\n\n\n# =============================================================================\n# SCENARIO A: RAG runs BEFORE Memory\n# =============================================================================\n# RAG only sees the current input message - no conversation history\n# Use when: RAG should be based purely on the current query\n\nagent_rag_first = ChatAgent(\n chat_client=client,\n context_providers=[\n RAGContextProvider(\"rag\"), # Runs first - only sees input_messages\n InMemoryHistoryProvider(\"memory\"), # Runs second - loads/stores history\n ]\n)\n\n# Flow:\n# 1. RAG.before_run():\n# - context.input_messages = [\"What's the weather?\"]\n# - context.get_messages() = [] (empty - memory hasn't run yet)\n# - RAG query based on: \"What's the weather?\" only\n# - Adds: context_messages[\"rag\"] = [retrieved docs]\n#\n# 2. Memory.before_run():\n# - Loads history: context_messages[\"memory\"] = [previous conversation]\n#\n# 3. Agent invocation with: history + rag docs + input\n#\n# 4. Memory.after_run():\n# - Stores: input + response (not RAG docs by default)\n#\n# 5. RAG.after_run():\n# - (nothing to do)\n\n\n# =============================================================================\n# SCENARIO B: RAG runs AFTER Memory\n# =============================================================================\n# RAG sees conversation history - can use it for better retrieval\n# Use when: RAG should consider conversation context for better results\n\nagent_memory_first = ChatAgent(\n chat_client=client,\n context_providers=[\n InMemoryHistoryProvider(\"memory\"), # Runs first - loads history\n RAGContextProvider(\"rag\"), # Runs second - sees history + input\n ]\n)\n\n# Flow:\n# 1. Memory.before_run():\n# - Loads history: context_messages[\"memory\"] = [previous conversation]\n#\n# 2. RAG.before_run():\n# - context.input_messages = [\"What's the weather?\"]\n# - context.get_messages() = [previous conversation] (sees history!)\n# - RAG query based on: recent history + \"What's the weather?\"\n# - Better retrieval because RAG understands conversation context\n# - Adds: context_messages[\"rag\"] = [more relevant docs]\n#\n# 3. Agent invocation with: history + rag docs + input\n#\n# 4. RAG.after_run():\n# - (nothing to do)\n#\n# 5. Memory.after_run():\n# - Stores: input + response\n\n\n# =============================================================================\n# SCENARIO C: RAG after Memory, with selective storage\n# =============================================================================\n# Memory first for better RAG, plus separate audit that stores RAG context\n\nagent_full_context = ChatAgent(\n chat_client=client,\n context_providers=[\n InMemoryHistoryProvider(\"memory\"), # Primary history storage\n RAGContextProvider(\"rag\"), # Gets history context for better retrieval\n PersonaContextProvider(\"persona\"), # Adds persona instructions\n # Audit storage - stores everything including RAG results\n CosmosHistoryProvider(\n \"audit\",\n load_messages=False, # Don't load (memory handles that)\n store_context_messages=True, # Store RAG + persona context too\n ),\n ]\n)\n```\n\n---\n\n### Workplan\n\nThe implementation is split into 2 PRs to limit scope and simplify review.\n\n```\nPR1 (New Types) ──► PR2 (Agent Integration + Cleanup)\n```\n\n#### PR 1: New Types\n\n**Goal:** Create all new types. No changes to existing code yet. Because the old `ContextProvider` class (in `_memory.py`) still exists during this PR, the new base class uses the **temporary name `_ContextProviderBase`** to avoid import collisions. All new provider implementations reference `_ContextProviderBase` / `_HistoryProviderBase` in PR1.\n\n**Core Package - `packages/core/agent_framework/_sessions.py`:**\n- [ ] `SessionContext` class with explicit add/get methods\n- [ ] `_ContextProviderBase` base class with `before_run()`/`after_run()` (temporary name; renamed to `ContextProvider` in PR2)\n- [ ] `_HistoryProviderBase(_ContextProviderBase)` derived class with load_messages/store flags (temporary; renamed to `HistoryProvider` in PR2)\n- [ ] `AgentSession` class with `state: dict[str, Any]`, `to_dict()`, `from_dict()`\n- [ ] `InMemoryHistoryProvider(_HistoryProviderBase)`\n\n**External Packages (new classes alongside existing ones, temporary `_` prefix):**\n- [ ] `packages/azure-ai-search/` - create `_AzureAISearchContextProvider(_ContextProviderBase)` — constructor keeps existing params, adds `source_id` (see compatibility notes below)\n- [ ] `packages/redis/` - create `_RedisHistoryProvider(_HistoryProviderBase)` — constructor keeps existing `RedisChatMessageStore` connection params, adds `source_id` + storage flags\n- [ ] `packages/redis/` - create `_RedisContextProvider(_ContextProviderBase)` — constructor keeps existing `RedisProvider` vector/search params, adds `source_id`\n- [ ] `packages/mem0/` - create `_Mem0ContextProvider(_ContextProviderBase)` — constructor keeps existing params, adds `source_id`\n\n**Constructor Compatibility Notes:**\n\nThe existing provider constructors can be preserved with minimal additions:\n\n| Existing Class | New Class (PR1 temporary name) | Constructor Changes |\n|---|---|---|\n| `AzureAISearchContextProvider(ContextProvider)` | `_AzureAISearchContextProvider(_ContextProviderBase)` | Add `source_id: str` (required). All existing params (`endpoint`, `index_name`, `api_key`, `mode`, `top_k`, etc.) stay the same. `invoking()` → `before_run()`, `invoked()` → `after_run()`. |\n| `Mem0Provider(ContextProvider)` | `_Mem0ContextProvider(_ContextProviderBase)` | Add `source_id: str` (required). All existing params (`mem0_client`, `api_key`, `agent_id`, `user_id`, etc.) stay the same. `scope_to_per_operation_thread_id` → maps to session_id scoping via `before_run`. |\n| `RedisChatMessageStore` | `_RedisHistoryProvider(_HistoryProviderBase)` | Add `source_id: str` (required) + `load_messages`, `store_inputs`, `store_responses` flags. Keep connection params (`redis_url`, `credential_provider`, `host`, `port`, `ssl`). Drop `thread_id` (now from `context.session_id`), `messages` (state managed via `session.state`), `max_messages` (→ message reduction concern). |\n| `RedisProvider(ContextProvider)` | `_RedisContextProvider(_ContextProviderBase)` | Add `source_id: str` (required). Keep vector/search params (`redis_url`, `index_name`, `redis_vectorizer`, etc.). Drop `thread_id` scoping (now from `context.session_id`). |\n\n**Testing:**\n- [ ] Unit tests for `SessionContext` methods (extend_messages, get_messages, extend_instructions, extend_tools)\n- [ ] Unit tests for `_HistoryProviderBase` load/store flags\n- [ ] Unit tests for `InMemoryHistoryProvider` state persistence via session.state\n- [ ] Unit tests for source attribution (mandatory source_id)\n\n---\n\n#### PR 2: Agent Integration + Cleanup\n\n**Goal:** Wire up new types into `ChatAgent` and remove old types.\n\n**Changes to `ChatAgent`:**\n- [ ] Replace `thread` parameter with `session` in `agent.run()`\n- [ ] Add `context_providers` parameter to `ChatAgent.__init__()`\n- [ ] Add `create_session()` method\n- [ ] Verify `session.to_dict()`/`AgentSession.from_dict()` round-trip in integration tests\n- [ ] Wire up provider iteration (before_run forward, after_run reverse)\n- [ ] Add validation warning if multiple/zero history providers have `load_messages=True`\n- [ ] Wire up default `InMemoryHistoryProvider` behavior (auto-add when no providers and `conversation_id` or `store=True`)\n\n**Remove Legacy Types:**\n- [ ] `packages/core/agent_framework/_memory.py` - remove old `ContextProvider` class\n- [ ] `packages/core/agent_framework/_threads.py` - remove `ChatMessageStore`, `ChatMessageStoreProtocol`, `AgentThread`\n- [ ] Remove old provider classes from `azure-ai-search`, `redis`, `mem0`\n\n**Rename Temporary Types → Final Names:**\n- [ ] `_ContextProviderBase` → `ContextProvider` in `_sessions.py`\n- [ ] `_HistoryProviderBase` → `HistoryProvider` in `_sessions.py`\n- [ ] `_AzureAISearchContextProvider` → `AzureAISearchContextProvider` in `packages/azure-ai-search/`\n- [ ] `_Mem0ContextProvider` → `Mem0ContextProvider` in `packages/mem0/`\n- [ ] `_RedisHistoryProvider` → `RedisHistoryProvider` in `packages/redis/`\n- [ ] `_RedisContextProvider` → `RedisContextProvider` in `packages/redis/`\n- [ ] Update all imports across packages and `__init__.py` exports to use final names\n\n**Public API (root package exports):**\n\nAll base classes and `InMemoryHistoryProvider` are exported from the root package:\n```python\nfrom agent_framework import (\n ContextProvider,\n HistoryProvider,\n InMemoryHistoryProvider,\n SessionContext,\n AgentSession,\n)\n```\n\n**Documentation & Samples:**\n- [ ] Update all samples in `samples/` to use new API\n- [ ] Write migration guide\n- [ ] Update API documentation\n\n**Testing:**\n- [ ] Unit tests for provider execution order (before_run forward, after_run reverse)\n- [ ] Unit tests for validation warnings (multiple/zero loaders)\n- [ ] Unit tests for session serialization (`session.to_dict()`/`AgentSession.from_dict()` round-trip)\n- [ ] Integration test: agent with `context_providers` + `session` works\n- [ ] Integration test: full conversation with memory persistence\n- [ ] Ensure all existing tests still pass (with updated API)\n- [ ] Verify no references to removed types remain\n\n---\n\n#### CHANGELOG (single entry for release)\n\n- **[BREAKING]** Replaced `ContextProvider` with new `ContextProvider` (hooks pattern with `before_run`/`after_run`)\n- **[BREAKING]** Replaced `ChatMessageStore` with `HistoryProvider`\n- **[BREAKING]** Replaced `AgentThread` with `AgentSession`\n- **[BREAKING]** Replaced `thread` parameter with `session` in `agent.run()`\n- Added `SessionContext` for invocation state with source attribution\n- Added `InMemoryHistoryProvider` for conversation history\n- `AgentSession` provides `to_dict()`/`from_dict()` for serialization (no special serialize/restore on providers)\n\n---\n\n#### Estimated Sizes\n\n| PR | New Lines | Modified Lines | Risk |\n|----|-----------|----------------|------|\n| PR1 | ~500 | ~0 | Low |\n| PR2 | ~150 | ~400 | Medium |\n\n---\n\n#### Implementation Detail: Decorator-based Providers\n\nFor simple use cases, a class-based provider can be verbose. A decorator API allows registering plain functions as `before_run` or `after_run` hooks for a more Pythonic setup:\n\n```python\nfrom agent_framework import ChatAgent, before_run, after_run\n\nagent = ChatAgent(chat_client=client)\n\n@before_run(agent)\nasync def add_system_prompt(agent, session, context, state):\n \"\"\"Inject a system prompt before every invocation.\"\"\"\n context.extend_messages(\"system\", [ChatMessage(role=\"system\", content=\"You are helpful.\")])\n\n@after_run(agent)\nasync def log_response(agent, session, context, state):\n \"\"\"Log the response after every invocation.\"\"\"\n print(f\"Response: {context.response.text}\")\n```\n\nUnder the hood, the decorators create a `ContextProvider` instance wrapping the function and append it to `agent._context_providers`:\n\n```python\ndef before_run(agent: ChatAgent, *, source_id: str = \"decorated\"):\n def decorator(fn):\n provider = _FunctionContextProvider(source_id=source_id, before_fn=fn)\n agent._context_providers.append(provider)\n return fn\n return decorator\n\ndef after_run(agent: ChatAgent, *, source_id: str = \"decorated\"):\n def decorator(fn):\n provider = _FunctionContextProvider(source_id=source_id, after_fn=fn)\n agent._context_providers.append(provider)\n return fn\n return decorator\n```\n\nThis is a convenience layer — the class-based API remains the primary interface for providers that need configuration, state, or both hooks.\n\n---\n\n#### Reference Implementation\n\nFull implementation code for the chosen design (hooks pattern, Decision B1).\n\n##### SessionContext\n\n```python\n# Copyright (c) Microsoft. All rights reserved.\n\nfrom abc import ABC, abstractmethod\nfrom collections.abc import Awaitable, Callable, Sequence\nfrom typing import Any\n\nfrom ._types import ChatMessage\nfrom ._tools import ToolProtocol\n\n\nclass SessionContext:\n \"\"\"Per-invocation state passed through the context provider pipeline.\n\n Created fresh for each agent.run() call. Providers read from and write to\n the mutable fields to add context before invocation and process responses after.\n\n Attributes:\n session_id: The ID of the current session\n service_session_id: Service-managed session ID (if present, service handles storage)\n input_messages: The new messages being sent to the agent (read-only, set by caller)\n context_messages: Dict mapping source_id -> messages added by that provider.\n Maintains insertion order (provider execution order). Use extend_messages()\n to add messages with proper source attribution.\n instructions: Additional instructions - providers can append here\n tools: Additional tools - providers can append here\n response (property): After invocation, contains the full AgentResponse (set by agent).\n Includes response.messages, response.response_id, response.agent_id,\n response.usage_details, etc.\n Read-only property - use AgentMiddleware to modify responses.\n options: Options passed to agent.run() - READ-ONLY, for reflection only\n metadata: Shared metadata dictionary for cross-provider communication\n\n Note:\n - `options` is read-only; changes will NOT be merged back into the agent run\n - `response` is a read-only property; use AgentMiddleware to modify responses\n - `instructions` and `tools` are merged by the agent into the run options\n - `context_messages` values are flattened in order when building the final input\n \"\"\"\n\n def __init__(\n self,\n *,\n session_id: str | None = None,\n service_session_id: str | None = None,\n input_messages: list[ChatMessage],\n context_messages: dict[str, list[ChatMessage]] | None = None,\n instructions: list[str] | None = None,\n tools: list[ToolProtocol] | None = None,\n options: dict[str, Any] | None = None,\n metadata: dict[str, Any] | None = None,\n ):\n self.session_id = session_id\n self.service_session_id = service_session_id\n self.input_messages = input_messages\n self.context_messages: dict[str, list[ChatMessage]] = context_messages or {}\n self.instructions: list[str] = instructions or []\n self.tools: list[ToolProtocol] = tools or []\n self._response: AgentResponse | None = None\n self.options = options or {} # READ-ONLY - for reflection only\n self.metadata = metadata or {}\n\n @property\n def response(self) -> AgentResponse | None:\n \"\"\"The agent's response. Set by the framework after invocation, read-only for providers.\"\"\"\n return self._response\n\n def extend_messages(self, source_id: str, messages: Sequence[ChatMessage]) -> None:\n \"\"\"Add context messages from a specific source.\n\n Messages are stored keyed by source_id, maintaining insertion order\n based on provider execution order.\n\n Args:\n source_id: The provider source_id adding these messages\n messages: The messages to add\n \"\"\"\n if source_id not in self.context_messages:\n self.context_messages[source_id] = []\n self.context_messages[source_id].extend(messages)\n\n def extend_instructions(self, source_id: str, instructions: str | Sequence[str]) -> None:\n \"\"\"Add instructions to be prepended to the conversation.\n\n Instructions are added to a flat list. The source_id is recorded\n in metadata for debugging but instructions are not keyed by source.\n\n Args:\n source_id: The provider source_id adding these instructions\n instructions: A single instruction string or sequence of strings\n \"\"\"\n if isinstance(instructions, str):\n instructions = [instructions]\n self.instructions.extend(instructions)\n\n def extend_tools(self, source_id: str, tools: Sequence[ToolProtocol]) -> None:\n \"\"\"Add tools to be available for this invocation.\n\n Tools are added with source attribution in their metadata.\n\n Args:\n source_id: The provider source_id adding these tools\n tools: The tools to add\n \"\"\"\n for tool in tools:\n if hasattr(tool, 'metadata') and isinstance(tool.metadata, dict):\n tool.metadata[\"context_source\"] = source_id\n self.tools.extend(tools)\n\n def get_messages(\n self,\n *,\n sources: Sequence[str] | None = None,\n exclude_sources: Sequence[str] | None = None,\n include_input: bool = False,\n include_response: bool = False,\n ) -> list[ChatMessage]:\n \"\"\"Get context messages, optionally filtered and including input/response.\n\n Returns messages in provider execution order (dict insertion order),\n with input and response appended if requested.\n\n Args:\n sources: If provided, only include context messages from these sources\n exclude_sources: If provided, exclude context messages from these sources\n include_input: If True, append input_messages after context\n include_response: If True, append response.messages at the end\n\n Returns:\n Flattened list of messages in conversation order\n \"\"\"\n result: list[ChatMessage] = []\n for source_id, messages in self.context_messages.items():\n if sources is not None and source_id not in sources:\n continue\n if exclude_sources is not None and source_id in exclude_sources:\n continue\n result.extend(messages)\n if include_input and self.input_messages:\n result.extend(self.input_messages)\n if include_response and self.response:\n result.extend(self.response.messages)\n return result\n```\n\n##### ContextProvider\n\n```python\nclass ContextProvider(ABC):\n \"\"\"Base class for context providers (hooks pattern).\n\n Context providers participate in the context engineering pipeline,\n adding context before model invocation and processing responses after.\n\n Attributes:\n source_id: Unique identifier for this provider instance (required).\n Used for message/tool attribution so other providers can filter.\n \"\"\"\n\n def __init__(self, source_id: str):\n \"\"\"Initialize the provider.\n\n Args:\n source_id: Unique identifier for this provider instance.\n Used for message/tool attribution.\n \"\"\"\n self.source_id = source_id\n\n async def before_run(\n self,\n agent: \"SupportsAgentRun\",\n session: AgentSession,\n context: SessionContext,\n state: dict[str, Any],\n ) -> None:\n \"\"\"Called before model invocation.\n\n Override to add context (messages, instructions, tools) to the\n SessionContext before the model is invoked.\n\n Args:\n agent: The agent running this invocation\n session: The current session\n context: The invocation context - add messages/instructions/tools here\n state: The session's mutable state dict\n \"\"\"\n pass\n\n async def after_run(\n self,\n agent: \"SupportsAgentRun\",\n session: AgentSession,\n context: SessionContext,\n state: dict[str, Any],\n ) -> None:\n \"\"\"Called after model invocation.\n\n Override to process the response (store messages, extract info, etc.).\n The context.response.messages will be populated at this point.\n\n Args:\n agent: The agent that ran this invocation\n session: The current session\n context: The invocation context with response populated\n state: The session's mutable state dict\n \"\"\"\n pass\n```\n\n> **Serialization contract:** Any values a provider writes to `state` must be JSON-serializable.\n> Sessions are serialized via `session.to_dict()` and restored via `AgentSession.from_dict()`.\n```\n\n##### HistoryProvider\n\n```python\nclass HistoryProvider(ContextProvider):\n \"\"\"Base class for conversation history storage providers.\n\n A single class that can be configured for different use cases:\n - Primary memory storage (loads + stores messages)\n - Audit/logging storage (stores only, doesn't load)\n - Evaluation storage (stores only for later analysis)\n\n Loading behavior (when to add messages to context_messages[source_id]):\n - `load_messages=True` (default): Load messages from storage\n - `load_messages=False`: Agent skips `before_run` entirely (audit/logging mode)\n\n Storage behavior:\n - `store_inputs`: Store input messages (default True)\n - `store_responses`: Store response messages (default True)\n - Storage always happens unless explicitly disabled, regardless of load_messages\n\n Warning: At session creation time, a warning is logged if:\n - Multiple history providers have `load_messages=True` (likely duplicate loading)\n - Zero history providers have `load_messages=True` (likely missing primary storage)\n\n Examples:\n # Primary memory - loads and stores\n memory = InMemoryHistoryProvider(source_id=\"memory\")\n\n # Audit storage - stores only, doesn't add to context\n audit = RedisHistoryProvider(\n source_id=\"audit\",\n load_messages=False,\n redis_url=\"redis://...\",\n )\n\n # Full audit - stores everything including RAG context\n full_audit = CosmosHistoryProvider(\n source_id=\"full_audit\",\n load_messages=False,\n store_context_messages=True,\n )\n \"\"\"\n\n def __init__(\n self,\n source_id: str,\n *,\n load_messages: bool = True,\n store_responses: bool = True,\n store_inputs: bool = True,\n store_context_messages: bool = False,\n store_context_from: Sequence[str] | None = None,\n ):\n super().__init__(source_id)\n self.load_messages = load_messages\n self.store_responses = store_responses\n self.store_inputs = store_inputs\n self.store_context_messages = store_context_messages\n self.store_context_from = list(store_context_from) if store_context_from else None\n\n @abstractmethod\n async def get_messages(self, session_id: str | None) -> list[ChatMessage]:\n \"\"\"Retrieve stored messages for this session.\"\"\"\n pass\n\n @abstractmethod\n async def save_messages(\n self,\n session_id: str | None,\n messages: Sequence[ChatMessage]\n ) -> None:\n \"\"\"Persist messages for this session.\"\"\"\n pass\n\n def _get_context_messages_to_store(self, context: SessionContext) -> list[ChatMessage]:\n \"\"\"Get context messages that should be stored based on configuration.\"\"\"\n if not self.store_context_messages:\n return []\n if self.store_context_from is not None:\n return context.get_messages(sources=self.store_context_from)\n else:\n return context.get_messages(exclude_sources=[self.source_id])\n\n async def before_run(self, agent, session, context, state) -> None:\n \"\"\"Load history into context. Skipped by the agent when load_messages=False.\"\"\"\n history = await self.get_messages(context.session_id)\n context.extend_messages(self.source_id, history)\n\n async def after_run(self, agent, session, context, state) -> None:\n \"\"\"Store messages based on configuration.\"\"\"\n messages_to_store: list[ChatMessage] = []\n messages_to_store.extend(self._get_context_messages_to_store(context))\n if self.store_inputs:\n messages_to_store.extend(context.input_messages)\n if self.store_responses and context.response.messages:\n messages_to_store.extend(context.response.messages)\n if messages_to_store:\n await self.save_messages(context.session_id, messages_to_store)\n```\n\n##### AgentSession\n\n```python\nimport uuid\nimport warnings\nfrom collections.abc import Sequence\n\n\nclass AgentSession:\n \"\"\"A conversation session with an agent.\n\n Lightweight state container. Provider instances are owned by the agent,\n not the session. The session only holds session IDs and a mutable state dict.\n\n Attributes:\n session_id: Unique identifier for this session\n service_session_id: Service-managed session ID (if using service-side storage)\n state: Mutable state dict shared with all providers\n \"\"\"\n\n def __init__(\n self,\n *,\n session_id: str | None = None,\n service_session_id: str | None = None,\n ):\n \"\"\"Initialize the session.\n\n Note: Prefer using agent.create_session() instead of direct construction.\n\n Args:\n session_id: Optional session ID (generated if not provided)\n service_session_id: Optional service-managed session ID\n \"\"\"\n self._session_id = session_id or str(uuid.uuid4())\n self.service_session_id = service_session_id\n self.state: dict[str, Any] = {}\n\n @property\n def session_id(self) -> str:\n \"\"\"The unique identifier for this session.\"\"\"\n return self._session_id\n\n def to_dict(self) -> dict[str, Any]:\n \"\"\"Serialize session to a plain dict for storage/transfer.\"\"\"\n return {\n \"type\": \"session\",\n \"session_id\": self._session_id,\n \"service_session_id\": self.service_session_id,\n \"state\": self.state,\n }\n\n @classmethod\n def from_dict(cls, data: dict[str, Any]) -> \"AgentSession\":\n \"\"\"Restore session from a previously serialized dict.\"\"\"\n session = cls(\n session_id=data[\"session_id\"],\n service_session_id=data.get(\"service_session_id\"),\n )\n session.state = data.get(\"state\", {})\n return session\nclass ChatAgent:\n def __init__(\n self,\n chat_client: ...,\n *,\n context_providers: Sequence[ContextProvider] | None = None,\n ):\n self._context_providers = list(context_providers or [])\n\n def create_session(\n self,\n *,\n session_id: str | None = None,\n ) -> AgentSession:\n \"\"\"Create a new lightweight session.\n\n Args:\n session_id: Optional session ID (generated if not provided)\n \"\"\"\n return AgentSession(session_id=session_id)\n\n def get_session(\n self,\n service_session_id: str,\n *,\n session_id: str | None = None,\n ) -> AgentSession:\n \"\"\"Get or create a session for a service-managed session ID.\n\n Args:\n service_session_id: Service-managed session ID\n session_id: Optional session ID (generated if not provided)\n \"\"\"\n session = AgentSession(session_id=session_id)\n session.service_session_id = service_session_id\n return session\n\n def _ensure_default_storage(self, session: AgentSession, options: dict[str, Any]) -> None:\n \"\"\"Add default InMemoryHistoryProvider if needed.\n\n Default storage is added when ALL of these are true:\n - A session is provided (always the case here)\n - No context_providers configured\n - Either options.conversation_id is set or options.store is True\n \"\"\"\n if self._context_providers:\n return\n if options.get(\"conversation_id\") or options.get(\"store\") is True:\n self._context_providers.append(InMemoryHistoryProvider(\"memory\"))\n\n def _validate_providers(self) -> None:\n \"\"\"Warn if history provider configuration looks like a mistake.\"\"\"\n storage_providers = [\n p for p in self._context_providers\n if isinstance(p, HistoryProvider)\n ]\n if not storage_providers:\n return\n loaders = [p for p in storage_providers if p.load_messages is True]\n if len(loaders) > 1:\n warnings.warn(\n f\"Multiple history providers configured to load messages: \"\n f\"{[p.source_id for p in loaders]}. \"\n f\"This may cause duplicate messages in context.\",\n UserWarning\n )\n elif len(loaders) == 0:\n warnings.warn(\n f\"History providers configured but none have load_messages=True: \"\n f\"{[p.source_id for p in storage_providers]}. \"\n f\"No conversation history will be loaded.\",\n UserWarning\n )\n\n async def run(self, input: str, *, session: AgentSession, options: dict[str, Any] | None = None) -> ...:\n \"\"\"Run the agent with the given input.\"\"\"\n options = options or {}\n\n # Ensure default storage on first run\n self._ensure_default_storage(session, options)\n self._validate_providers()\n\n context = SessionContext(\n session_id=session.session_id,\n service_session_id=session.service_session_id,\n input_messages=[...],\n options=options,\n )\n\n # Before-run providers (forward order, skip HistoryProviders with load_messages=False)\n for provider in self._context_providers:\n if isinstance(provider, HistoryProvider) and not provider.load_messages:\n continue\n await provider.before_run(self, session, context, session.state)\n\n # ... assemble final messages from context, invoke model ...\n\n # After-run providers (reverse order)\n for provider in reversed(self._context_providers):\n await provider.after_run(self, session, context, session.state)\n\n\n# Session serialization is trivial — session.state is a plain dict:\n#\n# # Serialize\n# data = {\n# \"session_id\": session.session_id,\n# \"service_session_id\": session.service_session_id,\n# \"state\": session.state,\n# }\n# json_str = json.dumps(data)\n#\n# # Deserialize\n# data = json.loads(json_str)\n# session = AgentSession(session_id=data[\"session_id\"], service_session_id=data.get(\"service_session_id\"))\n# session.state = data[\"state\"]\n```\n"
},
{
"path": "docs/decisions/0016-structured-output.md",
"content": "---\nstatus: proposed\ncontact: sergeymenshykh\ndate: 2026-01-22\ndeciders: rbarreto, westey-m, stephentoub\ninformed: {}\n---\n\n# Structured Output\n\nStructured output is a valuable aspect of any agent system, since it forces an agent to produce output in a required format that may include required fields.\nThis allows easily turning unstructured data into structured data using a general-purpose language model.\n\n## Context and Problem Statement\n\nStructured output is currently supported only by `ChatClientAgent` and can be configured in two ways:\n\n**Approach 1: ResponseFormat + Deserialize**\n\nSpecify the SO type schema via the `ChatClientAgent{Run}Options.ChatOptions.ResponseFormat` property at agent creation or invocation time, then use `JsonSerializer.Deserialize` to extract the structured data from the response text.\n\n\t```csharp\n\t// SO type can be provided at agent creation time\n\tChatClientAgent agent = chatClient.AsAIAgent(new ChatClientAgentOptions()\n\t{\n\t\tName = \"...\",\n\t\tChatOptions = new() { ResponseFormat = ChatResponseFormat.ForJsonSchema() }\n\t});\n\n\tAgentResponse response = await agent.RunAsync(\"...\");\n\n\tPersonInfo personInfo = response.Deserialize(JsonSerializerOptions.Web);\n\n\tConsole.WriteLine($\"Name: {personInfo.Name}\");\n\tConsole.WriteLine($\"Age: {personInfo.Age}\");\n\tConsole.WriteLine($\"Occupation: {personInfo.Occupation}\");\n\n\t// Alternatively, SO type can be provided at agent invocation time\n\tresponse = await agent.RunAsync(\"...\", new ChatClientAgentRunOptions()\n\t{\n\t\tChatOptions = new() { ResponseFormat = ChatResponseFormat.ForJsonSchema() }\n\t});\n\n\tpersonInfo = response.Deserialize(JsonSerializerOptions.Web);\n\n\tConsole.WriteLine($\"Name: {personInfo.Name}\");\n\tConsole.WriteLine($\"Age: {personInfo.Age}\");\n\tConsole.WriteLine($\"Occupation: {personInfo.Occupation}\");\n\t```\n\n**Approach 2: Generic RunAsync**\n\nSupply the SO type as a generic parameter to `RunAsync` and access the parsed result directly via the `Result` property.\n\n\t```csharp\n\tChatClientAgent agent = ...;\n\t\n\tAgentResponse response = await agent.RunAsync(\"...\");\n\n\tConsole.WriteLine($\"Name: {response.Result.Name}\");\n\tConsole.WriteLine($\"Age: {response.Result.Age}\");\n\tConsole.WriteLine($\"Occupation: {response.Result.Occupation}\");\n\t```\n\tNote: `RunAsync` is an instance method of `ChatClientAgent` and not part of the `AIAgent` base class since not all agents support structured output.\n\nApproach 1 is perceived as cumbersome by the community, as it requires additional effort when using primitive or collection types - the SO schema may need to be wrapped in an artificial JSON object. Otherwise, the caller will encounter an error like _Invalid schema for response_format 'Movie': schema must be a JSON Schema of 'type: \"object\"', got 'type: \"array\"'_. \nThis occurs because OpenAI and compatible APIs require a JSON object as the root schema.\n\nApproach 1 is also necessary in scenarios where (a) agents can only be configured with SO at creation time (such as with `AIProjectClient`), (b) the SO type is not known at compile time, or (c) the JSON schema is represented as text (for declarative agents) or as a `JsonElement`.\n\nApproach 2 is more convenient and works seamlessly with primitives and collections. However, it requires the SO type to be known at compile time, making it less flexible.\n\nAdditionally, since the `RunAsync` methods are instance methods of `ChatClientAgent` and are not part of the `AIAgent` base class, applying decorators like `OpenTelemetryAgent` on top of `ChatClientAgent` prevents users from accessing `RunAsync`, meaning structured output is not available with decorated agents.\n\nGiven the different scenarios above in which structured output can be used, there is no one-size-fits-all solution. Each approach has its own advantages and limitations,\nand the two can complement each other to provide a comprehensive structured output experience across various use cases.\n\n## Approaches Overview\n\n1. SO usage via `ResponseFormat` property\n2. SO usage via `RunAsync` generic method\n\n## 1. SO usage via `ResponseFormat` property\n\nThis approach should be used in the following scenarios:\n - 1.1 SO result as text is sufficient as is, and deserialization is not required\n - 1.2 SO for inter-agent collaboration\n - 1.3 SO can only be configured at agent creation time (such as with `AIProjectClient`)\n - 1.4 SO type is not known at compile time and represented by System.Type\n - 1.5 SO is represented by JSON schema and there's no corresponding .NET type either at compile time or at runtime\n - 1.6 SO in streaming scenarios, where the SO response is produced in parts\n\n**Note: Primitives and arrays are not supported by this approach.**\n\nWhen a caller provides a schema via `ResponseFormat`, they are explicitly telling the framework what schema to use. The framework passes that schema through as-is and\nis not responsible for transforming it. Because the framework does not own the schema, it cannot wrap primitives or arrays into a JSON object to satisfy API requirements,\nnor can it unwrap the response afterward - the caller controls the schema and is responsible for ensuring it is compatible with the underlying API.\n\nThis is in contrast to the `RunAsync` approach (section 2), where the caller provides a type `T` and says \"make it work.\" In that case, the caller does not\ndictate the schema - the framework infers the schema from `T`, owns the end-to-end pipeline (schema generation, API invocation, and deserialization), and can\ntherefore wrap and unwrap primitives and arrays transparently.\n\nAdditionally, in streaming scenarios (1.6), the framework cannot reliably unwrap a response it did not wrap, since it has no way of knowing whether the caller wrapped the schema.Wrapping and unwrapping can only be done safely when the framework owns the entire lifecycle - from schema creation through deserialization — which is only the case with `RunAsync`.\n\nIf a caller needs to work with primitives or arrays via the `ResponseFormat` approach, they can easily create a wrapper type around them:\n\n```csharp\npublic class MovieListWrapper\n{\n public List Movies { get; set; }\n}\n```\n\n### 1.1 SO result as text is sufficient as is, and deserialization is not required\n\nIn this scenario, the caller only needs the raw JSON text returned by the model and does not need to deserialize it into a .NET type.\nThe SO schema is specified via `ResponseFormat` at agent creation or invocation time, and the response text is consumed directly from the `AgentResponse`.\n\n```csharp\nAIAgent agent = chatClient.AsAIAgent();\n\nAgentRunOptions runOptions = new()\n{\n ResponseFormat = ChatResponseFormat.ForJsonSchema()\n};\n\nAgentResponse response = await agent.RunAsync(\"...\", options: runOptions);\n\nConsole.WriteLine(response.Text);\n```\n\n### 1.2 SO for inter-agent collaboration\n\nThis scenario assumes a multi-agent setup where agents collaborate by passing messages to each other.\nOne agent produces structured output as text that is then passed directly as input to the next agent, without intermediate deserialization.\n\n```csharp\n// First agent extracts structured data from unstructured input\nAIAgent extractionAgent = chatClient.AsAIAgent(new ChatClientAgentOptions()\n{\n Name = \"ExtractionAgent\",\n ChatOptions = new() \n { \n Instructions = \"Extract person information from the provided text.\",\n ResponseFormat = ChatResponseFormat.ForJsonSchema() \n }\n});\n\nAgentResponse extractionResponse = await extractionAgent.RunAsync(\"John Smith is a 35-year-old software engineer.\");\n\n// Pass the message with structured output text directly to the next agent\nChatMessage soMessage = extractionResponse.Messages.Last();\n\nAIAgent summaryAgent = chatClient.AsAIAgent(new ChatClientAgentOptions()\n{\n Name = \"SummaryAgent\",\n ChatOptions = new() { Instructions = \"Given the following structured person data, write a short professional bio.\" }\n});\n\nAgentResponse summaryResponse = await summaryAgent.RunAsync(soMessage);\n\nConsole.WriteLine(summaryResponse);\n```\n\n### 1.3 SO configured at agent creation time\n\nIn this scenario, the SO schema can only be configured at agent creation time (such as with `AIProjectClient`) and cannot be changed on a per-run basis.\nThe caller specifies the `ResponseFormat` when creating the agent, and all subsequent invocations use the same schema.\n\n```csharp\nAIProjectClient client = ...;\n\nAIAgent agent = await client.CreateAIAgentAsync(model: \"\", new ChatClientAgentOptions()\n{\n Name = \"...\",\n ChatOptions = new() { ResponseFormat = ChatResponseFormat.ForJsonSchema() }\n});\n\nAgentResponse response = await agent.RunAsync(\"Please provide information about John Smith.\");\n\nPersonInfo personInfo = JsonSerializer.Deserialize(response.Text, JsonSerializerOptions.Web)!;\n\nConsole.WriteLine($\"Name: {personInfo.Name}\");\nConsole.WriteLine($\"Age: {personInfo.Age}\");\nConsole.WriteLine($\"Occupation: {personInfo.Occupation}\");\n```\n\n### 1.4 SO type not known at compile time and represented by System.Type\n\nIn this scenario, the SO type is not known at compile time and is provided as a `System.Type` at runtime. This is useful for dynamic scenarios where the schema is determined programmatically, \nsuch as when building tooling or frameworks that work with user-defined types.\n\n```csharp\nType soType = GetStructuredOutputTypeFromConfiguration(); // e.g., typeof(PersonInfo)\n\nChatResponseFormat responseFormat = ChatResponseFormat.ForJsonSchema(soType);\n\nAgentResponse response = await agent.RunAsync(\"...\", new ChatClientAgentRunOptions()\n{\n ChatOptions = new() { ResponseFormat = responseFormat }\n});\n\nPersonInfo personInfo = (PersonInfo)JsonSerializer.Deserialize(response.Text, soType, JsonSerializerOptions.Web)!;\n```\n\n### 1.5 SO represented by JSON schema with no corresponding .NET type\n\nIn this scenario, the SO schema is represented as raw JSON schema text or a `JsonElement`, and there is no corresponding .NET type available at compile time or runtime.\nThis is typical for declarative agents or scenarios where schemas are loaded from external configuration.\n\n```csharp\n// JSON schema provided as a string, e.g., loaded from a configuration file\nstring jsonSchema = \"\"\"\n{\n \"type\": \"object\",\n \"properties\": {\n \"name\": { \"type\": \"string\" },\n \"age\": { \"type\": \"integer\" },\n \"occupation\": { \"type\": \"string\" }\n },\n \"required\": [\"name\", \"age\", \"occupation\"]\n}\n\"\"\";\n\nChatResponseFormat responseFormat = ChatResponseFormat.ForJsonSchema(\n jsonSchemaName: \"PersonInfo\",\n jsonSchema: BinaryData.FromString(jsonSchema));\n\nAgentResponse response = await agent.RunAsync(\"...\", new ChatClientAgentRunOptions()\n{\n ChatOptions = new() { ResponseFormat = responseFormat }\n});\n\n// Consume the SO result as text since there's no .NET type to deserialize into\nConsole.WriteLine(response.Text);\n```\n\n### 1.6 SO in streaming scenarios\n\nIn this scenario, the SO response is produced incrementally in parts via streaming. The caller specifies the `ResponseFormat` and consumes the response chunks as they arrive.\nDeserialization is performed after all chunks have been received.\n\n```csharp\nAIAgent agent = chatClient.AsAIAgent(new ChatClientAgentOptions()\n{\n Name = \"HelpfulAssistant\",\n ChatOptions = new()\n {\n Instructions = \"You are a helpful assistant.\",\n ResponseFormat = ChatResponseFormat.ForJsonSchema()\n }\n});\n\nIAsyncEnumerable updates = agent.RunStreamingAsync(\"Please provide information about John Smith, who is a 35-year-old software engineer.\");\n\nAgentResponse response = await updates.ToAgentResponseAsync();\n\n// Deserialize the complete SO result after streaming is finished\nPersonInfo personInfo = JsonSerializer.Deserialize(response.Text)!;\n```\n\n## 2. SO usage via `RunAsync` generic method\n\nThis approach provides a convenient way to work with structured output on a per-run basis when the target type is known at compile time and a typed instance of the result\nis required.\n\n### Decision Drivers\n\n1. Support arrays and primitives as SO types\n2. Support complex types as SO types\n3. Work with `AIAgent` decorators (e.g., `OpenTelemetryAgent`)\n4. Enable SO for all AI agents, regardless of whether they natively support it\n\n### Considered Options\n\n1. `RunAsync` as an instance method of `AIAgent` class delegating to virtual `RunCoreAsync`\n2. `RunAsync` as an extension method using feature collection\n3. `RunAsync![\"Watch](\"https://img.youtube.com/vi/AAgdMhftj8w/hqdefault.jpg\"\n)
\n \n![\"See](\"https://img.youtube.com/vi/mOAaGY4WPvc/hqdefault.jpg\")
\n \n