[ { "path": ".github/workflows/black.yml", "content": "name: Lint\n\non: [push, pull_request]\n\njobs:\n lint:\n runs-on: ubuntu-latest\n steps:\n - uses: actions/checkout@v3\n - uses: psf/black@stable" }, { "path": ".github/workflows/run_pytest.yml", "content": "name: Lint\n\non: [push, pull_request]\n\njobs:\n build_and_test:\n runs-on: ubuntu-latest\n strategy:\n matrix:\n python-version: [ \"3.10\", \"3.11\", \"3.12\" ]\n steps:\n - uses: actions/checkout@v4\n - name: Set up Python ${{ matrix.python-version }}\n uses: actions/setup-python@v5\n with:\n python-version: ${{ matrix.python-version }}\n - name: Install dependencies\n run: |\n python -m pip install --upgrade pip\n pip install poetry\n poetry install --all-extras --with test\n - name: Test with pytest\n run: poetry run pytest -m \"not integration\"\n\n" }, { "path": ".gitignore", "content": "# Python\n__pycache__/\n*.py[cod]\n*$py.class\n*.so\n.Python\nbuild/\ndevelop-eggs/\ndist/\ndownloads/\neggs/\n.eggs/\nlib/\nlib64/\nparts/\nsdist/\nvar/\nwheels/\n*.egg-info/\n.installed.cfg\n*.egg\nMANIFEST\n.env\n.venv\nenv/\nvenv/\nENV/\n*.whl\n\n# Node/TypeScript\nnode_modules/\nnpm-debug.log*\nyarn-debug.log*\nyarn-error.log*\n.npm\n.env.local\n.env.*.local\ndist/\ncoverage/\n*.tsbuildinfo\n\n# IDEs and editors\n.idea/\n.vscode/\n*.swp\n*.swo\n.DS_Store\n**/.DS_Store\n*.sublime-workspace\n*.sublime-project\n\n# Jupyter Notebook\n.ipynb_checkpoints\n*/.ipynb_checkpoints/*\n\n# Testing\n.coverage\nhtmlcov/\n.pytest_cache/\ncoverage/\n.nyc_output/\n\n# Cloud credentials\n.google-adc\n\n# Logs\nlogs\n*.log\n\n# Python version\n.python-version\n" }, { "path": ".pre-commit-config.yaml", "content": "repos:\n # Using this mirror lets us use mypyc-compiled black, which is about 2x faster\n - repo: https://github.com/psf/black-pre-commit-mirror\n rev: 24.4.2\n hooks:\n - id: black\n # It is recommended to specify the latest version of Python\n # supported by your project here, or alternatively use\n # pre-commit's default_language_version, see\n # https://pre-commit.com/#top_level-default_language_version\n language_version: python3.12\n" }, { "path": "CONTRIBUTING.md", "content": "\n# Contributing to aisuite\n\nFirst off, thanks for taking the time to contribute!\n\nAll types of contributions are encouraged and valued. See the [Table of Contents](#table-of-contents)\nfor different ways to help and details about how this project handles them. Please make sure to read\nthe relevant section before making your contribution. It will make it a lot easier for us maintainers\nand smooth out the experience for all involved. The community looks forward to your contributions.\n\n> And if you like the project, but just don't have time to contribute, that's fine. There are other easy\n> ways to support the project and show your appreciation, which we would also be very happy about:\n> - Star the project\n> - Tweet about it\n> - Refer this project in your project's readme\n> - Mention the project at local meetups and tell your friends/colleagues\n\n\n## Table of Contents\n\n- [I Have a Question](#i-have-a-question)\n- [I Want To Contribute](#i-want-to-contribute)\n - [Reporting Bugs](#reporting-bugs)\n - [Suggesting Enhancements](#suggesting-enhancements)\n - [Your First Code Contribution](#your-first-code-contribution)\n - [Improving The Documentation](#improving-the-documentation)\n- [Styleguides](#styleguides)\n - [Commit Messages](#commit-messages)\n\n\n\n\n## I Have a Question\n\n> If you want to ask a question, we assume that you have read the available\n> [Documentation](https://github.com/andrewyng/aisuite/blob/main/README.md).\n\nBefore you ask a question, it is best to search for existing [Issues](https://github.com/andrewyng/aisuite/issues)\nthat might help you. If you find a relevant issue that already exists and still need clarification, please add your question to that existing issue. We also recommend reaching out to the community in the aisuite [Discord](https://discord.gg/T6Nvn8ExSb) server.\n\nIf you then still feel the need to ask a question and need clarification, we recommend the following:\n\n- Open an [Issue](https://github.com/andrewyng/aisuite/issues/new).\n- Provide as much context as you can about what you're running into.\n- Provide project and platform versions (python, OS, etc.), depending on what seems relevant.\n\nWe (or someone in the community) will then take care of the issue as soon as possible.\n\n\n## I Want To Contribute\n\n> ### Legal Notice \n> When contributing to this project, you must agree that you have authored 100% of the content, that\n> you have the necessary rights to the content and that the content you contribute may be provided\n> under the project license.\n\n### Reporting Bugs\n\n\n#### Before Submitting a Bug Report\n\nA good bug report shouldn't leave others needing to chase you up for more information. Therefore, we ask\nyou to investigate carefully, collect information and describe the issue in detail in your report. Please\ncomplete the following steps in advance to help us fix any potential bug as fast as possible.\n\n- Make sure that you are using the latest version.\n- Determine if your bug is really a bug and not an error on your side e.g. using incompatible environment \n components/versions (Make sure that you have read the [documentation](https://github.com/andrewyng/aisuite/blob/main/README.md).\n If you are looking for support, you might want to check [this section](#i-have-a-question)).\n- To see if other users have experienced (and potentially already solved) the same issue you are having,\n check if there is not already a bug report existing for your bug or error in the [bug tracker](https://github.com/andrewyng/aisuite?q=label%3Abug).\n- Also make sure to search the internet (including Stack Overflow) to see if users outside of the GitHub\n community have discussed the issue.\n- Collect information about the bug:\n - Stack trace (Traceback)\n - OS, Platform and Version (Windows, Linux, macOS, x86, ARM)\n - Version of the interpreter, compiler, SDK, runtime environment, package manager, depending on\n what seems relevant.\n - Possibly your input and the output\n - Can you reliably reproduce the issue? And can you also reproduce it with older versions?\n\n\n#### How Do I Submit a Good Bug Report?\n\n> You must never report security related issues, vulnerabilities or bugs including sensitive information to\n> the issue tracker, or elsewhere in public. Instead sensitive bugs must be sent by email to .\n\n\nWe use GitHub issues to track bugs and errors. If you run into an issue with the project:\n\n- Open an [Issue](https://github.com/andrewyng/aisuite/issues/new). (Since we can't be sure at\n this point whether it is a bug or not, we ask you not to talk about a bug yet and not to label the issue.)\n- Explain the behavior you would expect and the actual behavior.\n- Please provide as much context as possible and describe the *reproduction steps* that someone else can\n follow to recreate the issue on their own. This usually includes your code. For good bug reports you\n should isolate the problem and create a reduced test case.\n- Provide the information you collected in the previous section.\n\nOnce it's filed:\n\n- The project team will label the issue accordingly.\n- A team member will try to reproduce the issue with your provided steps. If there are no reproduction \n steps or no obvious way to reproduce the issue, the team will ask you for those steps and mark the\n issue as `needs-repro`. Bugs with the `needs-repro` tag will not be addressed until they are reproduced.\n- If the team is able to reproduce the issue, it will be marked `needs-fix`, as well as possibly other\n tags (such as `critical`), and the issue will be left to be\n [implemented by someone](#your-first-code-contribution).\n\nPlease use the issue templates provided.\n\n\n### Suggesting Enhancements\n\nThis section guides you through submitting an enhancement suggestion for aisuite,\n**including completely new features and minor improvements to existing functionality**. Following these\nguidelines will help maintainers and the community to understand your suggestion and find related suggestions.\n\n\n#### Before Submitting an Enhancement\n\n- Make sure that you are using the latest version.\n- Read the [documentation](https://github.com/andrewyng/aisuite/blob/main/README.md) carefully\n and find out if the functionality is already covered, maybe by an individual configuration.\n- Perform a [search](https://github.com/andrewyng/aisuite/issues) to see if the enhancement has\n already been suggested. If it has, add a comment to the existing issue instead of opening a new one.\n- Find out whether your idea fits with the scope and aims of the project. It's up to you to make a strong\n case to convince the project's developers of the merits of this feature. Keep in mind that we want features that will be useful to the majority of our users and not just a small subset. If you're just targeting a minority of users, consider writing an add-on/plugin library.\n\n\n#### How Do I Submit a Good Enhancement Suggestion?\n\nEnhancement suggestions are tracked as [GitHub issues](https://github.com/andrewyng/aisuite/issues).\n\n- Use a **clear and descriptive title** for the issue to identify the suggestion.\n- Provide a **step-by-step description of the suggested enhancement** in as many details as possible.\n- **Describe the current behavior** and **explain which behavior you expected to see instead** and why.\n At this point you can also tell which alternatives do not work for you.\n- **Explain why this enhancement would be useful** to most aisuite users. You may also want to\n point out the other projects that solved it better and which could serve as inspiration.\n\n\n### Your First Code Contribution\n\n#### Pre-requisites\n\nYou should first [fork](https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/fork-a-repo)\nthe `aisuite` repository and then clone your forked repository:\n\n```bash\ngit clone https://github.com//aisuite.git\n```\n\n\n\nOnce in the cloned repository directory, make a branch on the forked repository with your username and\ndescription of PR:\n```bash\ngit checkout -B /\n```\n\nPlease install the development and test dependencies:\n```bash\npoetry install --with dev,test\n```\n\n`aisuite` uses pre-commit to ensure the formatting is consistent:\n```bash\npre-commit install\n```\n\n**Make suggested changes**\n\nAfterwards, our suite of formatting tests will run automatically before each `git commit`. You can also\nrun these manually:\n```bash\npre-commit run --all-files\n```\n\nIf a formatting test fails, it will fix the modified code in place and abort the `git commit`. After looking\nover the changes, you can `git add ` and then repeat the previous git commit command.\n\n**Note**: a github workflow will check the files with the same formatter and reject the PR if it doesn't\npass, so please make sure it passes locally.\n\n\n#### Testing\n`aisuite` tracks unit tests. Pytest is used to execute said unit tests in `tests/`:\n\n```bash\npoetry run pytest tests\n```\n\nIf your code changes implement a new function, please make a corresponding unit test to the `test/*` files.\n\n#### Contributing Workflow\nWe actively welcome your pull requests.\n\n1. Create your new branch from main in your forked repo, with your username and a name describing the work\n you're completing e.g. user-123/add-feature-x.\n2. If you've added code that should be tested, add tests. Ensure all tests pass. See the testing section\n for more information.\n3. If you've changed APIs, update the documentation.\n4. Make sure your code lints.\n\n\n\n### Improving The Documentation\nWe welcome valuable contributions in the form of new documentation or revised documentation that provide\nfurther clarity or accuracy. Each function should be clearly documented. Well-documented code is easier\nto review and understand/extend.\n\n## Styleguides\nFor code documentation, please follow the [Google styleguide](https://github.com/google/styleguide/blob/gh-pages/pyguide.md#38-comments-and-docstrings).\n" }, { "path": "LICENSE", "content": "MIT License\n\nCopyright (c) 2024 Andrew Ng\n\nPermission is hereby granted, free of charge, to any person obtaining a copy of this software and\nassociated documentation files (the \"Software\"), to deal in the Software without restriction, including\nwithout limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell\ncopies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the\nfollowing conditions:\n\nThe above copyright notice and this permission notice shall be included in all copies or substantial\nportions of the Software.\n\nTHE SOFTWARE IS PROVIDED \"AS IS\", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT\nLIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.\nIN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,\nWHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE\nSOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.\n" }, { "path": "README.md", "content": "# aisuite\n\n[![PyPI](https://img.shields.io/pypi/v/aisuite)](https://pypi.org/project/aisuite/)\n[![Code style: black](https://img.shields.io/badge/code%20style-black-000000.svg)](https://github.com/psf/black)\n\n`aisuite` is a lightweight Python library that provides a **unified API for working with multiple Generative AI providers**. \nIt offers a consistent interface for models from *OpenAI, Anthropic, Google, Hugging Face, AWS, Cohere, Mistral, Ollama*, and others—abstracting away SDK differences, authentication details, and parameter variations. \nIts design is modeled after OpenAI’s API style, making it instantly familiar and easy to adopt.\n\n`aisuite` lets developers build and **run LLM-based or agentic applications across providers** with minimal setup. \nWhile it’s not a full-blown agents framework, it includes simple abstractions for creating standalone, lightweight agents. \nIt’s designed for low learning curve — so you can focus on building AI systems, not integrating APIs.\n\n---\n\n## Key Features\n\n`aisuite` is designed to eliminate the complexity of working with multiple LLM providers while keeping your code simple and portable. Whether you're building a chatbot, an agentic application, or experimenting with different models, `aisuite` provides the abstractions you need without getting in your way.\n\n* **Unified API for multiple model providers** – Write your code once and run it with any supported provider. Switch between OpenAI, Anthropic, Google, and others with a single parameter change.\n* **Easy agentic app or agent creation** – Build multi-turn agentic applications using a single parameter `max_turns`. No need to manually manage tool execution loops.\n* **Pass Tool calls easily** – Pass real Python functions instead of JSON specs; aisuite handles schema generation and execution automatically.\n* **MCP tools** – Connect to MCP-based tools without writing boilerplate; aisuite handles connection, schema and execution seamlessly.\n* **Modular and extensible provider architecture** – Add support for new providers with minimal code. The plugin-style architecture makes extensions straightforward.\n\n---\n\n## Installation\n\nYou can install just the base `aisuite` package, or install a provider's package along with `aisuite`.\n\nInstall just the base package without any provider SDKs:\n\n```shell\npip install aisuite\n```\n\nInstall aisuite with a specific provider (e.g., Anthropic):\n\n```shell\npip install 'aisuite[anthropic]'\n```\n\nInstall aisuite with all provider libraries:\n\n```shell\npip install 'aisuite[all]'\n```\n\n## Setup\n\nTo get started, you will need API Keys for the providers you intend to use. You'll need to\ninstall the provider-specific library either separately or when installing aisuite.\n\nThe API Keys can be set as environment variables, or can be passed as config to the aisuite Client constructor.\nYou can use tools like [`python-dotenv`](https://pypi.org/project/python-dotenv/) or [`direnv`](https://direnv.net/) to set the environment variables manually. Please take a look at the `examples` folder to see usage.\n\nHere is a short example of using `aisuite` to generate chat completion responses from gpt-4o and claude-3-5-sonnet.\n\nSet the API keys.\n\n```shell\nexport OPENAI_API_KEY=\"your-openai-api-key\"\nexport ANTHROPIC_API_KEY=\"your-anthropic-api-key\"\n```\n\nUse the python client.\n\n```python\nimport aisuite as ai\nclient = ai.Client()\n\nmodels = [\"openai:gpt-4o\", \"anthropic:claude-3-5-sonnet-20240620\"]\n\nmessages = [\n {\"role\": \"system\", \"content\": \"Respond in Pirate English.\"},\n {\"role\": \"user\", \"content\": \"Tell me a joke.\"},\n]\n\nfor model in models:\n response = client.chat.completions.create(\n model=model,\n messages=messages,\n temperature=0.75\n )\n print(response.choices[0].message.content)\n\n```\n\nNote that the model name in the create() call uses the format - `:`.\n`aisuite` will call the appropriate provider with the right parameters based on the provider value.\nFor a list of provider values, you can look at the directory - `aisuite/providers/`. The list of supported providers are of the format - `_provider.py` in that directory. We welcome providers to add support to this library by adding an implementation file in this directory. Please see section below for how to contribute.\n\nFor more examples, check out the `examples` directory where you will find several notebooks that you can run to experiment with the interface.\n\n---\n\n## Chat Completions\n\nThe chat API provides a high-level abstraction for model interactions. It supports all core parameters (`temperature`, `max_tokens`, `tools`, etc.) in a provider-agnostic way.\n\n```python\nresponse = client.chat.completions.create(\n model=\"google:gemini-pro\",\n messages=[{\"role\": \"user\", \"content\": \"Summarize this paragraph.\"}],\n)\nprint(response.choices[0].message.content)\n```\n\n`aisuite` standardizes request and response structures so you can focus on logic rather than SDK differences.\n\n---\n\n## Tool Calling & Agentic apps\n\n`aisuite` provides a simple abstraction for tool/function calling that works across supported providers. This is in addition to the regular abstraction of passing JSON spec of the tool to the `tools` parameter. The tool calling abstraction makes it easy to use tools with different LLMs without changing your code.\n\nThere are two ways to use tools with `aisuite`:\n\n### 1. Manual Tool Handling\n\nThis is the default behavior when `max_turns` is not specified. In this mode, you have full control over the tool execution flow. You pass tools using the standard OpenAI JSON schema format, and `aisuite` returns the LLM's tool call requests in the response. You're then responsible for executing the tools, processing results, and sending them back to the model in subsequent requests.\n\nThis approach is useful when you need:\n- Fine-grained control over tool execution logic\n- Custom error handling or validation before executing tools\n- The ability to selectively execute or skip certain tool calls\n- Integration with existing tool execution pipelines\n\nYou can pass tools in the OpenAI tool format:\n\n```python\ndef will_it_rain(location: str, time_of_day: str):\n \"\"\"Check if it will rain in a location at a given time today.\n \n Args:\n location (str): Name of the city\n time_of_day (str): Time of the day in HH:MM format.\n \"\"\"\n return \"YES\"\n\ntools = [{\n \"type\": \"function\",\n \"function\": {\n \"name\": \"will_it_rain\",\n \"description\": \"Check if it will rain in a location at a given time today\",\n \"parameters\": {\n \"type\": \"object\",\n \"properties\": {\n \"location\": {\n \"type\": \"string\",\n \"description\": \"Name of the city\"\n },\n \"time_of_day\": {\n \"type\": \"string\",\n \"description\": \"Time of the day in HH:MM format.\"\n }\n },\n \"required\": [\"location\", \"time_of_day\"]\n }\n }\n}]\n\nresponse = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=messages,\n tools=tools\n)\n```\n\n### 2. Automatic Tool Execution\n\nWhen `max_turns` is specified, you can pass a list of callable Python functions as the `tools` parameter. `aisuite` will automatically handle the tool calling flow:\n\n```python\ndef will_it_rain(location: str, time_of_day: str):\n \"\"\"Check if it will rain in a location at a given time today.\n \n Args:\n location (str): Name of the city\n time_of_day (str): Time of the day in HH:MM format.\n \"\"\"\n return \"YES\"\n\nclient = ai.Client()\nmessages = [{\n \"role\": \"user\",\n \"content\": \"I live in San Francisco. Can you check for weather \"\n \"and plan an outdoor picnic for me at 2pm?\"\n}]\n\n# Automatic tool execution with max_turns\nresponse = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=messages,\n tools=[will_it_rain],\n max_turns=2 # Maximum number of back-and-forth tool calls\n)\nprint(response.choices[0].message.content)\n```\n\nWhen `max_turns` is specified, `aisuite` will:\n1. Send your message to the LLM\n2. Execute any tool calls the LLM requests\n3. Send the tool results back to the LLM\n4. Repeat until the conversation is complete or max_turns is reached\n\nIn addition to `response.choices[0].message`, there is an additional field `response.choices[0].intermediate_messages` which contains the list of all messages including tool interactions used. This can be used to continue the conversation with the model.\nFor more detailed examples of tool calling, check out the `examples/tool_calling_abstraction.ipynb` notebook.\n\n### Model Context Protocol (MCP) Integration\n\n`aisuite` natively supports **MCP**, a standard protocol that allows LLMs to securely call external tools and access data. You can connect to MCP servers—such as a filesystem or database—and expose their tools directly to your model.\nRead more about MCP here - https://modelcontextprotocol.io/docs/getting-started/intro\n\nInstall aisuite with MCP support:\n\n```shell\npip install 'aisuite[mcp]'\n```\n\nYou'll also need an MCP server. For example, to use the filesystem server:\n\n```shell\nnpm install -g @modelcontextprotocol/server-filesystem\n```\n\nThere are two ways to use MCP tools with aisuite:\n\n#### Option 1: Config Dict Format (Recommended for Simple Use Cases)\n\n```python\nimport aisuite as ai\n\nclient = ai.Client()\nresponse = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"List the files in the current directory\"}],\n tools=[{\n \"type\": \"mcp\",\n \"name\": \"filesystem\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@modelcontextprotocol/server-filesystem\", \"/path/to/directory\"]\n }],\n max_turns=3\n)\n\nprint(response.choices[0].message.content)\n```\n\n#### Option 2: Explicit MCPClient (Recommended for Advanced Use Cases)\n\n```python\nimport aisuite as ai\nfrom aisuite.mcp import MCPClient\n\n# Create MCP client once, reuse across requests\nmcp = MCPClient(\n command=\"npx\",\n args=[\"-y\", \"@modelcontextprotocol/server-filesystem\", \"/path/to/directory\"]\n)\n\n# Use with aisuite\nclient = ai.Client()\nresponse = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"List the files\"}],\n tools=mcp.get_callable_tools(),\n max_turns=3\n)\n\nprint(response.choices[0].message.content)\nmcp.close() # Clean up\n```\n\nFor detailed usage (security filters, tool prefixing, and `MCPClient` management), see [docs/mcp-tools.md](docs/mcp-tools.md).\nFor detailed examples, see `examples/mcp_tools_example.ipynb`.\n\n---\n\n## Extending aisuite: Adding a Provider\n\nNew providers can be added by implementing a lightweight adapter. The system uses a naming convention for discovery:\n\n| Element | Convention |\n| --------------- | ---------------------------------- |\n| **Module file** | `_provider.py` |\n| **Class name** | `Provider` (capitalized) |\n\nExample:\n\n```python\n# providers/openai_provider.py\nclass OpenaiProvider(BaseProvider):\n ...\n```\n\nThis convention ensures consistency and enables automatic loading of new integrations.\n\n---\n\n## Contributing\n\nContributions are welcome. Please review the [Contributing Guide](https://github.com/andrewyng/aisuite/blob/main/CONTRIBUTING.md) and join our [Discord](https://discord.gg/T6Nvn8ExSb) for discussions.\n\n---\n\n## License\n\nReleased under the **MIT License** — free for commercial and non-commercial use.\n\n---\n" }, { "path": "aisuite/__init__.py", "content": "from .client import Client\nfrom .framework.message import Message\nfrom .utils.tools import Tools\n" }, { "path": "aisuite/client.py", "content": "from .provider import ProviderFactory\nimport os\nfrom .utils.tools import Tools\nfrom typing import Union, BinaryIO, Optional, Any, Literal\nfrom contextlib import ExitStack\nfrom .framework.message import (\n TranscriptionResponse,\n)\nfrom .framework.asr_params import ParamValidator\n\n# Import MCP utilities for config dict support\ntry:\n from .mcp.config import is_mcp_config\n from .mcp.client import MCPClient\n\n MCP_AVAILABLE = True\nexcept ImportError:\n MCP_AVAILABLE = False\n\n\nclass Client:\n def __init__(\n self,\n provider_configs: dict = {},\n extra_param_mode: Literal[\"strict\", \"warn\", \"permissive\"] = \"warn\",\n ):\n \"\"\"\n Initialize the client with provider configurations.\n Use the ProviderFactory to create provider instances.\n\n Args:\n provider_configs (dict): A dictionary containing provider configurations.\n Each key should be a provider string (e.g., \"google\" or \"aws-bedrock\"),\n and the value should be a dictionary of configuration options for that provider.\n For example:\n {\n \"openai\": {\"api_key\": \"your_openai_api_key\"},\n \"aws-bedrock\": {\n \"aws_access_key\": \"your_aws_access_key\",\n \"aws_secret_key\": \"your_aws_secret_key\",\n \"aws_region\": \"us-west-2\"\n }\n }\n extra_param_mode (str): How to handle unknown ASR parameters.\n - \"strict\": Raise ValueError on unknown params (production)\n - \"warn\": Log warning on unknown params (default, development)\n - \"permissive\": Allow all params without validation (testing)\n \"\"\"\n self.providers = {}\n self.provider_configs = provider_configs\n self.extra_param_mode = extra_param_mode\n self.param_validator = ParamValidator(extra_param_mode)\n self._chat = None\n self._audio = None\n\n def _initialize_providers(self):\n \"\"\"Helper method to initialize or update providers.\"\"\"\n for provider_key, config in self.provider_configs.items():\n provider_key = self._validate_provider_key(provider_key)\n self.providers[provider_key] = ProviderFactory.create_provider(\n provider_key, config\n )\n\n def _validate_provider_key(self, provider_key):\n \"\"\"\n Validate if the provider key corresponds to a supported provider.\n \"\"\"\n supported_providers = ProviderFactory.get_supported_providers()\n\n if provider_key not in supported_providers:\n raise ValueError(\n f\"Invalid provider key '{provider_key}'. Supported providers: {supported_providers}. \"\n \"Make sure the model string is formatted correctly as 'provider:model'.\"\n )\n\n return provider_key\n\n def configure(self, provider_configs: Optional[dict] = None):\n \"\"\"\n Configure the client with provider configurations.\n \"\"\"\n if provider_configs is None:\n return\n\n self.provider_configs.update(provider_configs)\n # Providers will be lazily initialized when needed\n\n @property\n def chat(self):\n \"\"\"Return the chat API interface.\"\"\"\n if not self._chat:\n self._chat = Chat(self)\n return self._chat\n\n @property\n def audio(self):\n \"\"\"Return the audio API interface.\"\"\"\n if not self._audio:\n self._audio = Audio(self)\n return self._audio\n\n\nclass Chat:\n def __init__(self, client: \"Client\"):\n self.client = client\n self._completions = Completions(self.client)\n\n @property\n def completions(self):\n \"\"\"Return the completions interface.\"\"\"\n return self._completions\n\n\nclass Completions:\n def __init__(self, client: \"Client\"):\n self.client = client\n\n def _process_mcp_configs(self, tools: list) -> tuple[list, list]:\n \"\"\"\n Process tools list and convert MCP config dicts to callable tools.\n\n This method:\n 1. Detects MCP config dicts ({\"type\": \"mcp\", ...})\n 2. Creates MCPClient instances from configs\n 3. Extracts callable tools with filtering and prefixing\n 4. Mixes MCP tools with regular callable tools\n 5. Returns both processed tools and MCP clients for cleanup\n\n Args:\n tools: List of tools (mix of callables and MCP configs)\n\n Returns:\n Tuple of (processed_tools, mcp_clients):\n - processed_tools: List of callable tools only\n - mcp_clients: List of MCPClient instances to be cleaned up\n\n Example:\n >>> tools = [\n ... my_function,\n ... {\"type\": \"mcp\", \"name\": \"fs\", \"command\": \"npx\", \"args\": [...]},\n ... another_function\n ... ]\n >>> callable_tools, mcp_clients = self._process_mcp_configs(tools)\n >>> # Returns: ([my_function, fs_tool1, fs_tool2, ..., another_function], [mcp_client])\n \"\"\"\n if not MCP_AVAILABLE:\n # If MCP not installed, check if user is trying to use it\n if any(is_mcp_config(tool) for tool in tools if isinstance(tool, dict)):\n raise ImportError(\n \"MCP tools require the 'mcp' package. \"\n \"Install it with: pip install 'aisuite[mcp]' or pip install mcp\"\n )\n return tools, []\n\n processed_tools = []\n mcp_clients = []\n\n for tool in tools:\n if isinstance(tool, dict) and is_mcp_config(tool):\n # It's an MCP config dict - convert to callable tools\n try:\n mcp_client = MCPClient.from_config(tool)\n mcp_clients.append(mcp_client)\n\n # Get tools with config settings\n mcp_tools = mcp_client.get_callable_tools(\n allowed_tools=tool.get(\"allowed_tools\"),\n use_tool_prefix=tool.get(\"use_tool_prefix\", False),\n )\n\n processed_tools.extend(mcp_tools)\n except Exception as e:\n raise ValueError(\n f\"Failed to create MCP client from config: {e}\\n\"\n f\"Config: {tool}\"\n )\n else:\n # Regular callable tool - pass through\n processed_tools.append(tool)\n\n return processed_tools, mcp_clients\n\n def _extract_thinking_content(self, response):\n \"\"\"\n Extract content between tags if present and store it in reasoning_content.\n\n Args:\n response: The response object from the provider\n\n Returns:\n Modified response object\n \"\"\"\n if hasattr(response, \"choices\") and response.choices:\n message = response.choices[0].message\n if hasattr(message, \"content\") and message.content:\n content = message.content.strip()\n if content.startswith(\"\") and \"\" in content:\n # Extract content between think tags\n start_idx = len(\"\")\n end_idx = content.find(\"\")\n thinking_content = content[start_idx:end_idx].strip()\n\n # Store the thinking content\n message.reasoning_content = thinking_content\n\n # Remove the think tags from the original content\n message.content = content[end_idx + len(\"\") :].strip()\n\n return response\n\n def _tool_runner(\n self,\n provider,\n model_name: str,\n messages: list,\n tools: Any,\n max_turns: int,\n **kwargs,\n ):\n \"\"\"\n Handle tool execution loop for max_turns iterations.\n\n Args:\n provider: The provider instance to use for completions\n model_name: Name of the model to use\n messages: List of conversation messages\n tools: Tools instance or list of callable tools\n max_turns: Maximum number of tool execution turns\n **kwargs: Additional arguments to pass to the provider\n\n Returns:\n The final response from the model with intermediate responses and messages\n \"\"\"\n # Handle tools validation and conversion\n if isinstance(tools, Tools):\n tools_instance = tools\n kwargs[\"tools\"] = tools_instance.tools()\n else:\n # Check if passed tools are callable\n if not all(callable(tool) for tool in tools):\n raise ValueError(\"One or more tools is not callable\")\n tools_instance = Tools(tools)\n kwargs[\"tools\"] = tools_instance.tools()\n\n turns = 0\n intermediate_responses = [] # Store intermediate responses\n intermediate_messages = [] # Store all messages including tool interactions\n\n while turns < max_turns:\n # Make the API call\n response = provider.chat_completions_create(model_name, messages, **kwargs)\n response = self._extract_thinking_content(response)\n\n # Store intermediate response\n intermediate_responses.append(response)\n\n # Check if there are tool calls in the response\n tool_calls = (\n getattr(response.choices[0].message, \"tool_calls\", None)\n if hasattr(response, \"choices\")\n else None\n )\n\n # Store the model's message\n intermediate_messages.append(response.choices[0].message)\n\n if not tool_calls:\n # Set the intermediate data in the final response\n response.intermediate_responses = intermediate_responses[\n :-1\n ] # Exclude final response\n response.choices[0].intermediate_messages = intermediate_messages\n return response\n\n # Execute tools and get results\n results, tool_messages = tools_instance.execute_tool(tool_calls)\n\n # Add tool messages to intermediate messages\n intermediate_messages.extend(tool_messages)\n\n # Add the assistant's response and tool results to messages\n messages.extend([response.choices[0].message, *tool_messages])\n\n turns += 1\n\n # Set the intermediate data in the final response\n response.intermediate_responses = intermediate_responses[\n :-1\n ] # Exclude final response\n response.choices[0].intermediate_messages = intermediate_messages\n return response\n\n def create(self, model: str, messages: list, **kwargs):\n \"\"\"\n Create chat completion based on the model, messages, and any extra arguments.\n Supports automatic tool execution when max_turns is specified.\n \"\"\"\n # Check that correct format is used\n if \":\" not in model:\n raise ValueError(\n f\"Invalid model format. Expected 'provider:model', got '{model}'\"\n )\n\n # Extract the provider key from the model identifier, e.g., \"google:gemini-xx\"\n provider_key, model_name = model.split(\":\", 1)\n\n # Validate if the provider is supported\n supported_providers = ProviderFactory.get_supported_providers()\n if provider_key not in supported_providers:\n raise ValueError(\n f\"Invalid provider key '{provider_key}'. Supported providers: {supported_providers}. \"\n \"Make sure the model string is formatted correctly as 'provider:model'.\"\n )\n\n # Initialize provider if not already initialized\n # TODO: Add thread-safe provider initialization with lock to prevent race conditions\n # when multiple threads try to initialize the same provider simultaneously.\n if provider_key not in self.client.providers:\n config = self.client.provider_configs.get(provider_key, {})\n self.client.providers[provider_key] = ProviderFactory.create_provider(\n provider_key, config\n )\n\n provider = self.client.providers.get(provider_key)\n if not provider:\n raise ValueError(f\"Could not load provider for '{provider_key}'.\")\n\n # Extract tool-related parameters\n max_turns = kwargs.pop(\"max_turns\", None)\n tools = kwargs.pop(\"tools\", None)\n\n # Use ExitStack to manage MCP client cleanup automatically\n with ExitStack() as stack:\n # Convert MCP config dicts to callable tools and get MCP clients\n mcp_clients = []\n if tools is not None:\n tools, mcp_clients = self._process_mcp_configs(tools)\n # Register all MCP clients for automatic cleanup\n for mcp_client in mcp_clients:\n stack.enter_context(mcp_client)\n\n # Check environment variable before allowing multi-turn tool execution\n if max_turns is not None and tools is not None:\n return self._tool_runner(\n provider,\n model_name,\n messages.copy(),\n tools,\n max_turns,\n **kwargs,\n )\n\n # Default behavior without tool execution\n # Delegate the chat completion to the correct provider's implementation\n response = provider.chat_completions_create(model_name, messages, **kwargs)\n return self._extract_thinking_content(response)\n\n\nclass Audio:\n \"\"\"Audio API interface.\"\"\"\n\n def __init__(self, client: \"Client\"):\n self.client = client\n self._transcriptions = Transcriptions(self.client)\n\n @property\n def transcriptions(self):\n \"\"\"Return the transcriptions interface.\"\"\"\n return self._transcriptions\n\n\nclass Transcriptions:\n \"\"\"Transcriptions API interface.\"\"\"\n\n def __init__(self, client: \"Client\"):\n self.client = client\n\n def create(\n self,\n *,\n model: str,\n file: Union[str, BinaryIO],\n **kwargs,\n ) -> TranscriptionResponse:\n \"\"\"\n Create audio transcription with parameter validation.\n\n This method uses a pass-through approach with validation:\n - Common parameters (OpenAI-style) are auto-mapped to provider equivalents\n - Provider-specific parameters are passed through directly\n - Unknown parameters are handled based on extra_param_mode\n\n Args:\n model: Provider and model in format 'provider:model' (e.g., 'openai:whisper-1')\n file: Audio file to transcribe (file path or file-like object)\n **kwargs: Transcription parameters (provider-specific or common)\n Common parameters (portable across providers):\n - language: Language code (e.g., \"en\")\n - prompt: Context for the transcription\n - temperature: Sampling temperature (0-1, OpenAI only)\n Provider-specific parameters are passed through directly.\n See provider documentation for valid parameters.\n\n Returns:\n TranscriptionResponse: Unified response (batch or streaming)\n\n Raises:\n ValueError: If model format invalid, provider not supported,\n or unknown params in strict mode\n\n Examples:\n # Portable code (OpenAI-style params)\n >>> result = client.audio.transcriptions.create(\n ... model=\"openai:whisper-1\",\n ... file=\"audio.mp3\",\n ... language=\"en\"\n ... )\n\n # Provider-specific features\n >>> result = client.audio.transcriptions.create(\n ... model=\"deepgram:nova-2\",\n ... file=\"audio.mp3\",\n ... language=\"en\", # Common param\n ... punctuate=True, # Deepgram-specific\n ... diarize=True # Deepgram-specific\n ... )\n \"\"\"\n # Validate model format\n if \":\" not in model:\n raise ValueError(\n f\"Invalid model format. Expected 'provider:model', got '{model}'\"\n )\n\n # Extract provider and model name\n provider_key, model_name = model.split(\":\", 1)\n\n # Validate provider is supported\n supported_providers = ProviderFactory.get_supported_providers()\n if provider_key not in supported_providers:\n raise ValueError(\n f\"Invalid provider key '{provider_key}'. \"\n f\"Supported providers: {supported_providers}\"\n )\n\n # Validate and map parameters\n validated_params = self.client.param_validator.validate_and_map(\n provider_key, kwargs\n )\n\n # Initialize provider if not already initialized\n if provider_key not in self.client.providers:\n config = self.client.provider_configs.get(provider_key, {})\n try:\n self.client.providers[provider_key] = ProviderFactory.create_provider(\n provider_key, config\n )\n except ImportError as e:\n raise ValueError(f\"Provider '{provider_key}' is not available: {e}\")\n\n provider = self.client.providers.get(provider_key)\n if not provider:\n raise ValueError(f\"Could not load provider for '{provider_key}'.\")\n\n # Check if provider supports audio transcription\n if not hasattr(provider, \"audio\") or provider.audio is None:\n raise ValueError(\n f\"Provider '{provider_key}' does not support audio transcription.\"\n )\n\n # Determine if streaming is requested\n should_stream = validated_params.get(\"stream\", False)\n\n # Delegate to provider implementation\n try:\n if should_stream:\n # Check if provider supports output streaming\n if hasattr(provider.audio, \"transcriptions\") and hasattr(\n provider.audio.transcriptions, \"create_stream_output\"\n ):\n return provider.audio.transcriptions.create_stream_output(\n model_name, file, **validated_params\n )\n else:\n raise ValueError(\n f\"Provider '{provider_key}' does not support streaming transcription.\"\n )\n else:\n # Non-streaming (batch) transcription\n if hasattr(provider.audio, \"transcriptions\") and hasattr(\n provider.audio.transcriptions, \"create\"\n ):\n return provider.audio.transcriptions.create(\n model_name, file, **validated_params\n )\n else:\n raise ValueError(\n f\"Provider '{provider_key}' does not support audio transcription.\"\n )\n except NotImplementedError:\n raise ValueError(\n f\"Provider '{provider_key}' does not support audio transcription.\"\n )\n" }, { "path": "aisuite/design-notes/asr-parameter-design-motivation.md", "content": "# ASR - API Parameter Design Philosophy\n\n## Design Goal: Portable Code with Provider Flexibility\n\nThe ASR parameter system is designed around a core principle: **developers should write portable code that works across providers, while retaining the ability to use provider-specific features when needed**. This document explains the rationale behind our parameter classification and validation approach.\n\n---\n\n## Mandatory Parameters and Common Mappings\n\n### The Foundation: Minimal Requirements\n\nEvery transcription needs just two things:\n- **`model`**: Which model/provider to use\n- **`file`**: What audio to transcribe\n\nBy keeping mandatory parameters minimal, we maximize compatibility and reduce the barrier to getting started.\n\n### Common Parameters: Write Once, Run Anywhere\n\nBeyond the basics, there are concepts that exist across providers but use different names or formats. We handle three common parameters that auto-map to each provider's native API:\n\n**Example: Same code, different providers**\n\n```python\n# Works with OpenAI\nresult = client.audio.transcriptions.create(\n model=\"openai:whisper-1\",\n file=\"meeting.mp3\",\n language=\"en\",\n prompt=\"discussion about API design\"\n)\n\n# Exact same code works with Deepgram\nresult = client.audio.transcriptions.create(\n model=\"deepgram:nova-2\",\n file=\"meeting.mp3\",\n language=\"en\",\n prompt=\"discussion about API design\"\n)\n```\n\nBehind the scenes:\n- **`language`** passes through as `language` for both OpenAI and Deepgram, but expands to `language_code: \"en-US\"` for Google\n- **`prompt`** passes as `prompt` to OpenAI, transforms to `keywords: [\"discussion\", \"about\", \"API\", \"design\"]` for Deepgram, and becomes `speech_contexts: [{\"phrases\": [\"discussion about API design\"]}]` for Google\n- **`temperature`** passes through to OpenAI (which supports it) and is silently ignored by Deepgram and Google (which don't)\n\n**Why auto-mapping?** Developers shouldn't need to remember that Google uses `language_code` while others use `language`, or that Deepgram expects a list of keywords. The framework handles these provider quirks transparently, letting you write portable code.\n\n---\n\n## Provider-Specific Features: Pass-Through for Power Users\n\nEach provider has unique features that give them competitive advantages. We don't limit you to the \"lowest common denominator\" - if you need provider-specific functionality, it's available:\n\n**Deepgram's advanced features:**\n```python\nresult = client.audio.transcriptions.create(\n model=\"deepgram:nova-2\",\n file=\"meeting.mp3\",\n language=\"en\",\n punctuate=True, # Deepgram-specific\n diarize=True, # Deepgram-specific\n sentiment=True, # Deepgram-specific\n smart_format=True # Deepgram-specific\n)\n```\n\n**Google's speech contexts:**\n```python\nresult = client.audio.transcriptions.create(\n model=\"google:latest_long\",\n file=\"meeting.mp3\",\n language_code=\"en-US\",\n enable_automatic_punctuation=True, # Google-specific\n max_alternatives=3, # Google-specific\n speech_contexts=[{\"phrases\": [\"API\", \"SDK\", \"REST\"]}] # Google-specific\n)\n```\n\nThese provider-specific parameters pass through directly to the provider's SDK. The framework validates them based on your configured mode (see next section), but doesn't block access to unique features.\n\n---\n\n## Progressive Validation: Safety When You Need It\n\nThe validation system supports three modes to match different development stages:\n\n### Development Mode: `\"warn\"` (Default)\n```python\nclient = Client(extra_param_mode=\"warn\")\n```\nUnknown parameters trigger warnings but continue execution. Perfect for exploration and prototyping. You see *\"OpenAI doesn't support 'punctuate'\"* but your code keeps running.\n\n### Strict Mode: `\"strict\"`\n```python\nclient = Client(extra_param_mode=\"strict\")\n```\nUnknown parameters raise errors immediately. Use in production to catch typos, configuration mistakes, or provider API changes early. Ensures no silent failures.\n\n### Permissive Mode: `\"permissive\"`\n```python\nclient = Client(extra_param_mode=\"permissive\")\n```\nAll parameters pass through without validation. Use for beta features, experimental parameters, or when providers add new capabilities faster than framework updates.\n\n**Progressive workflow:**\n1. **Develop** with `warn` - explore freely, see warnings\n2. **Refactor** - fix warnings to make code portable\n3. **Deploy** with `strict` - ensure production safety\n\n---\n\n## Developer Experience Benefits\n\n### 1. Write Portable Code Naturally\nThe same parameter names work across providers. Switch from OpenAI to Deepgram by changing one word: the model identifier.\n\n### 2. Progressive Enhancement\nStart with portable common parameters. Add provider-specific features only where you need them. Your core logic remains portable even when using advanced features for specific providers.\n\n### 3. Zero Framework Lock-in\nParameter names come directly from provider APIs, not framework abstractions. If you need to remove the framework, you already know the native API - the names are identical.\n\n### 4. Validation That Adapts to You\nChoose your safety level based on context. Strict for production, warn for development, permissive for bleeding-edge features. The framework supports your workflow rather than constraining it.\n\n### 5. No Documentation Friction\nCopy parameters from provider docs directly. No need to learn our abstraction layer or figure out mappings - we handle the common cases, you use native names for everything else.\n\n---\n\n## Alternative Design Considered\n\nWe considered creating a unified options object (`TranscriptionOptions`) that explicitly defines all parameters with framework-specific names. We chose pass-through instead because:\n\n1. **Provider APIs evolve faster than frameworks** - New parameters appear frequently. Pass-through lets developers use them immediately (in permissive mode) without waiting for framework updates.\n\n2. **Provider features don't map cleanly** - Deepgram's sentiment analysis, Google's complex speech contexts, OpenAI's timestamp granularities - each is unique. A unified object means either losing functionality or creating complex provider-specific abstractions.\n\n3. **Direct API access reduces friction** - Developers already know their provider's API from official docs. They can use parameter names directly rather than learning another abstraction layer.\n\nThe pass-through approach with progressive validation provides the best of both worlds: portability for common cases, power for advanced features, and safety when you need it.\n\n---\n\n## Design Principles Summary\n\n- **Mandatory Minimal**: Only `model` and `file` required\n- **Common Auto-Mapped**: Frequent cross-provider concepts map transparently\n- **Provider-Specific Pass-Through**: Unique features remain accessible\n- **Progressive Validation**: Three modes for different development stages\n- **Zero Abstraction Tax**: Use provider APIs directly with optional safety nets\n\nThis design prioritizes developer experience through portability without sacrificing power, validation without blocking experimentation, and simplicity without limiting functionality.\n" }, { "path": "aisuite/framework/__init__.py", "content": "from .provider_interface import ProviderInterface\nfrom .chat_completion_response import ChatCompletionResponse\nfrom .message import Message\n" }, { "path": "aisuite/framework/asr_params.py", "content": "\"\"\"\nASR parameter registry and validation.\n\nThis module provides a unified parameter validation system for audio transcription\nacross different providers. It supports:\n- Common parameters (OpenAI-style) that are auto-mapped to provider equivalents\n- Provider-specific parameters that are passed through directly\n- Three validation modes: strict, warn, and permissive\n\"\"\"\n\nfrom typing import Dict, Set, Any, Optional, Literal\nimport logging\n\nlogger = logging.getLogger(__name__)\n\n\n# Common parameters that get auto-mapped across providers\n# These follow OpenAI's API conventions for maximum portability\nCOMMON_PARAMS: Dict[str, Dict[str, Optional[str]]] = {\n \"language\": {\n \"openai\": \"language\",\n \"deepgram\": \"language\",\n \"google\": \"language_code\",\n \"huggingface\": None, # Not supported by Inference API\n },\n \"prompt\": {\n \"openai\": \"prompt\",\n \"deepgram\": \"keywords\",\n \"google\": \"speech_contexts\",\n \"huggingface\": None, # Not supported\n },\n \"temperature\": {\n \"openai\": \"temperature\",\n \"deepgram\": None, # Not supported\n \"google\": None, # Not supported\n \"huggingface\": \"temperature\", # Supported as generation param\n },\n}\n\n\n# Valid provider-specific parameters\n# Each provider has its own set of supported parameters\nPROVIDER_PARAMS: Dict[str, Set[str]] = {\n \"openai\": {\n # Basic parameters\n \"language\",\n \"prompt\",\n \"temperature\",\n # Output format\n \"response_format\", # \"json\" | \"text\" | \"srt\" | \"verbose_json\" | \"vtt\"\n \"timestamp_granularities\", # [\"word\"] | [\"segment\"] | [\"word\", \"segment\"]\n # Streaming\n \"stream\", # Boolean\n },\n \"deepgram\": {\n # Basic parameters\n \"language\",\n \"model\",\n # Text enhancement\n \"punctuate\", # Auto-add punctuation\n \"diarize\", # Speaker diarization\n \"utterances\", # Sentence-level timestamps\n \"paragraphs\", # Paragraph segmentation\n \"smart_format\", # Format numbers, dates, etc.\n \"profanity_filter\", # Filter profanity\n # Advanced features\n \"search\", # Search for keywords: [\"keyword1\", \"keyword2\"]\n \"replace\", # Replace words: {\"um\": \"\", \"uh\": \"\"}\n \"keywords\", # Boost keywords: [\"important\", \"technical\"]\n \"numerals\", # Format numerals\n \"measurements\", # Format measurements\n # AI features\n \"sentiment\", # Sentiment analysis\n \"topics\", # Topic detection\n \"intents\", # Intent recognition\n \"summarize\", # Auto-summarization\n # Audio format\n \"encoding\", # \"linear16\" | \"mp3\" | \"flac\"\n \"sample_rate\", # Integer (Hz)\n \"channels\", # Integer\n # Quality and alternatives\n \"confidence\", # Include confidence scores\n \"alternatives\", # Number of alternative transcripts\n # Streaming\n \"interim_results\", # Get interim results while streaming\n },\n \"google\": {\n # Basic parameters\n \"language_code\", # BCP-47 code like \"en-US\"\n \"model\", # \"latest_long\" | \"latest_short\" | \"default\"\n # Audio format\n \"encoding\", # \"LINEAR16\" | \"FLAC\" | \"MP3\"\n \"sample_rate_hertz\", # Integer\n \"audio_channel_count\", # Integer\n # Text enhancement\n \"enable_automatic_punctuation\", # Boolean\n \"profanity_filter\", # Boolean\n \"enable_spoken_punctuation\", # Boolean\n \"enable_spoken_emojis\", # Boolean\n # Speaker features\n \"enable_speaker_diarization\", # Boolean\n \"diarization_speaker_count\", # Integer (max speakers)\n \"min_speaker_count\", # Integer\n # Metadata\n \"enable_word_time_offsets\", # Word-level timestamps\n \"enable_word_confidence\", # Word-level confidence\n \"max_alternatives\", # Number of alternatives\n # Context\n \"speech_contexts\", # [{\"phrases\": [...], \"boost\": float}]\n \"boost\", # Float (phraseHint boost)\n # Streaming\n \"interim_results\", # Boolean\n \"single_utterance\", # Boolean (stop after one utterance)\n },\n \"huggingface\": {\n # Basic parameters\n \"model\", # Model ID on Hugging Face Hub\n \"temperature\", # Generation temperature\n # API options\n \"return_timestamps\", # Boolean or \"word\" or \"char\"\n \"use_cache\", # Boolean: use cached inference\n \"wait_for_model\", # Boolean: wait if model is loading\n # Generation parameters\n \"top_k\", # Integer: top-k sampling\n \"top_p\", # Float: nucleus sampling\n \"max_length\", # Integer: maximum output length\n \"do_sample\", # Boolean: enable sampling\n },\n}\n\n\n# Language code expansion for Google (2-letter to locale codes)\nGOOGLE_LANGUAGE_MAP = {\n \"en\": \"en-US\",\n \"es\": \"es-ES\",\n \"fr\": \"fr-FR\",\n \"de\": \"de-DE\",\n \"it\": \"it-IT\",\n \"pt\": \"pt-BR\",\n \"ja\": \"ja-JP\",\n \"ko\": \"ko-KR\",\n \"zh\": \"zh-CN\",\n \"ar\": \"ar-SA\",\n \"hi\": \"hi-IN\",\n \"ru\": \"ru-RU\",\n \"nl\": \"nl-NL\",\n \"pl\": \"pl-PL\",\n \"sv\": \"sv-SE\",\n \"da\": \"da-DK\",\n \"no\": \"nb-NO\",\n \"fi\": \"fi-FI\",\n \"tr\": \"tr-TR\",\n \"th\": \"th-TH\",\n \"vi\": \"vi-VN\",\n}\n\n\nclass ParamValidator:\n \"\"\"\n Validates and maps ASR parameters for different providers.\n\n This class handles three types of parameters:\n 1. Common parameters (OpenAI-style) - auto-mapped to provider equivalents\n 2. Provider-specific parameters - passed through with validation\n 3. Unknown parameters - handled based on extra_param_mode\n \"\"\"\n\n def __init__(self, extra_param_mode: Literal[\"strict\", \"warn\", \"permissive\"]):\n \"\"\"\n Initialize the parameter validator.\n\n Args:\n extra_param_mode: How to handle unknown parameters\n - \"strict\": Raise ValueError on unknown params\n - \"warn\": Log warning on unknown params (default)\n - \"permissive\": Allow all params without validation\n \"\"\"\n self.extra_param_mode = extra_param_mode\n\n def validate_and_map(\n self, provider_key: str, params: Dict[str, Any]\n ) -> Dict[str, Any]:\n \"\"\"\n Validate and map parameters for the given provider.\n\n This method:\n 1. Maps common parameters to provider-specific equivalents\n 2. Validates provider-specific parameters\n 3. Handles unknown parameters based on extra_param_mode\n\n Args:\n provider_key: Provider identifier (e.g., \"openai\", \"deepgram\")\n params: Raw parameters from user\n\n Returns:\n Validated and mapped parameters ready for provider API\n\n Raises:\n ValueError: If extra_param_mode=\"strict\" and unknown params found\n \"\"\"\n result = {}\n unknown_params = []\n provider_params = PROVIDER_PARAMS.get(provider_key, set())\n\n for key, value in params.items():\n # Check if it's a common param that needs mapping\n if key in COMMON_PARAMS:\n mapped_key = COMMON_PARAMS[key].get(provider_key)\n\n # Provider doesn't support this common param\n if mapped_key is None:\n logger.debug(\n f\"Parameter '{key}' not supported by {provider_key}, ignoring\"\n )\n continue\n\n # Transform value if needed (e.g., \"en\" -> \"en-US\" for Google)\n mapped_value = self._transform_value(provider_key, key, value)\n result[mapped_key] = mapped_value\n\n # Check if it's a valid provider-specific param\n elif key in provider_params:\n result[key] = value\n\n # Unknown parameter\n else:\n unknown_params.append(key)\n\n # Handle unknown parameters based on mode\n if unknown_params:\n self._handle_unknown(provider_key, unknown_params)\n\n # In permissive mode, still pass them through\n if self.extra_param_mode == \"permissive\":\n for key in unknown_params:\n result[key] = params[key]\n\n return result\n\n def _transform_value(self, provider_key: str, param_key: str, value: Any) -> Any:\n \"\"\"\n Transform parameter values during mapping.\n\n This handles provider-specific transformations like:\n - Google: Expanding \"en\" to \"en-US\"\n - Google: Wrapping prompt in speech_contexts structure\n - Deepgram: Converting prompt string to keywords list\n\n Args:\n provider_key: Provider identifier\n param_key: Parameter name (from COMMON_PARAMS)\n value: Parameter value to transform\n\n Returns:\n Transformed parameter value\n \"\"\"\n # Google: Expand 2-letter language codes to locale codes\n if provider_key == \"google\" and param_key == \"language\":\n if isinstance(value, str) and len(value) == 2:\n return GOOGLE_LANGUAGE_MAP.get(value, f\"{value}-US\")\n\n # Google: Wrap prompt in speech_contexts structure\n if provider_key == \"google\" and param_key == \"prompt\":\n return [{\"phrases\": [value]}]\n\n # Deepgram: Split prompt into keywords list\n if provider_key == \"deepgram\" and param_key == \"prompt\":\n if isinstance(value, str):\n return value.split()\n return value\n\n return value\n\n def _handle_unknown(self, provider_key: str, unknown_params: list):\n \"\"\"\n Handle unknown parameters based on extra_param_mode.\n\n Args:\n provider_key: Provider identifier\n unknown_params: List of unknown parameter names\n\n Raises:\n ValueError: If extra_param_mode=\"strict\"\n \"\"\"\n msg = (\n f\"Unknown parameters for {provider_key}: {unknown_params}. \"\n f\"See {provider_key} documentation for valid parameters.\"\n )\n\n if self.extra_param_mode == \"strict\":\n raise ValueError(msg)\n elif self.extra_param_mode == \"warn\":\n import warnings\n\n warnings.warn(msg, UserWarning)\n # permissive mode: do nothing\n" }, { "path": "aisuite/framework/chat_completion_response.py", "content": "\"\"\"Defines the ChatCompletionResponse class.\"\"\"\n\nfrom typing import Optional\n\nfrom aisuite.framework.choice import Choice\nfrom aisuite.framework.message import CompletionUsage\n\n\n# pylint: disable=too-few-public-methods\nclass ChatCompletionResponse:\n \"\"\"Used to conform to the response model of OpenAI.\"\"\"\n\n def __init__(self):\n \"\"\"Initializes the ChatCompletionResponse.\"\"\"\n self.choices = [Choice()] # Adjust the range as needed for more choices\n self.usage: Optional[CompletionUsage] = None\n" }, { "path": "aisuite/framework/choice.py", "content": "from aisuite.framework.message import Message\nfrom typing import Literal, Optional, List\n\n\nclass Choice:\n def __init__(self):\n self.finish_reason: Optional[Literal[\"stop\", \"tool_calls\"]] = None\n self.message = Message(\n content=None,\n tool_calls=None,\n role=\"assistant\",\n refusal=None,\n reasoning_content=None,\n )\n self.intermediate_messages: List[Message] = []\n" }, { "path": "aisuite/framework/message.py", "content": "\"\"\"\nInterface to hold contents of api responses when they do not confirm\nto the OpenAI style response.\n\"\"\"\n\nfrom typing import Literal, Optional, List, AsyncGenerator, Union, Dict, Any\nfrom pydantic import BaseModel\nfrom dataclasses import dataclass, field\n\n\nclass Function(BaseModel):\n \"\"\"Represents a function call.\"\"\"\n\n arguments: str\n name: str\n\n\nclass ChatCompletionMessageToolCall(BaseModel):\n \"\"\"Represents a tool call in a chat completion message.\"\"\"\n\n id: str\n function: Function\n type: Literal[\"function\"]\n\n\nclass Message(BaseModel):\n \"\"\"Represents a message in a chat completion.\"\"\"\n\n content: Optional[str] = None\n reasoning_content: Optional[str] = None\n tool_calls: Optional[List[ChatCompletionMessageToolCall]] = None\n role: Optional[Literal[\"user\", \"assistant\", \"system\", \"tool\"]] = None\n refusal: Optional[str] = None\n\n\nclass CompletionTokensDetails(BaseModel):\n \"\"\"Details about the tokens used in a completion.\"\"\"\n\n accepted_prediction_tokens: Optional[int] = None\n \"\"\"\n When using Predicted Outputs, the number of tokens in the prediction that\n appeared in the completion.\n \"\"\"\n\n audio_tokens: Optional[int] = None\n \"\"\"Audio input tokens generated by the model.\"\"\"\n\n reasoning_tokens: Optional[int] = None\n \"\"\"Tokens generated by the model for reasoning.\"\"\"\n\n rejected_prediction_tokens: Optional[int] = None\n \"\"\"\n When using Predicted Outputs, the number of tokens in the prediction that did\n not appear in the completion. However, like reasoning tokens, these tokens are\n still counted in the total completion tokens for purposes of billing, output,\n and context window limits.\n \"\"\"\n\n\nclass PromptTokensDetails(BaseModel):\n \"\"\"Details about the tokens used in a prompt.\"\"\"\n\n text_tokens: Optional[int] = None\n \"\"\"Tokens generated by the model for text.\"\"\"\n\n audio_tokens: Optional[int] = None\n \"\"\"Audio input tokens present in the prompt.\"\"\"\n\n cached_tokens: Optional[int] = None\n \"\"\"Cached tokens present in the prompt.\"\"\"\n\n\nclass CompletionUsage(BaseModel):\n \"\"\"Represents the token usage for a completion.\"\"\"\n\n completion_tokens: Optional[int] = None\n \"\"\"Number of tokens in the generated completion.\"\"\"\n\n prompt_tokens: Optional[int] = None\n \"\"\"Number of tokens in the prompt.\"\"\"\n\n total_tokens: Optional[int] = None\n \"\"\"Total number of tokens used in the request (prompt + completion).\"\"\"\n\n completion_tokens_details: Optional[CompletionTokensDetails] = None\n \"\"\"Breakdown of tokens used in a completion.\"\"\"\n\n prompt_tokens_details: Optional[PromptTokensDetails] = None\n \"\"\"Breakdown of tokens used in the prompt.\"\"\"\n\n\nclass Word(BaseModel):\n \"\"\"Represents a single word with timing information.\"\"\"\n\n word: str\n start: float\n end: float\n confidence: Optional[float] = None # Common across Deepgram, Azure, AWS\n speaker: Optional[int] = None # Speaker diarization (Deepgram, Azure, AWS)\n speaker_confidence: Optional[float] = None # Speaker identification confidence\n punctuated_word: Optional[str] = None # Word with punctuation (some providers)\n\n\nclass Segment(BaseModel):\n \"\"\"Represents a segment of transcribed text with detailed information.\"\"\"\n\n id: int\n seek: int\n start: float\n end: float\n text: str\n # OpenAI Whisper specific fields\n tokens: Optional[List[int]] = None\n temperature: Optional[float] = None\n avg_logprob: Optional[float] = None\n compression_ratio: Optional[float] = None\n no_speech_prob: Optional[float] = None\n # Common ASR provider fields\n confidence: Optional[float] = None # Segment-level confidence\n speaker: Optional[int] = None # Primary speaker for this segment\n speaker_confidence: Optional[float] = None # Speaker identification confidence\n words: Optional[List[Word]] = None # Words within this segment\n\n\nclass Alternative(BaseModel):\n \"\"\"Represents an alternative transcription hypothesis (common in many ASR APIs).\"\"\"\n\n transcript: str\n confidence: Optional[float] = None\n words: Optional[List[Word]] = None\n\n\nclass Channel(BaseModel):\n \"\"\"Represents a single audio channel (for multi-channel audio).\"\"\"\n\n alternatives: List[Alternative]\n search: Optional[List[dict]] = None # Search results if keyword search enabled\n\n\nclass TranscriptionResult(BaseModel):\n \"\"\"\n Unified transcription result format supporting multiple ASR providers.\n Based on OpenAI Whisper API but extended for common ASR features.\n \"\"\"\n\n # Core fields (supported by most providers)\n text: str\n language: Optional[str] = None\n confidence: Optional[float] = None # Overall transcription confidence\n\n # OpenAI Whisper specific fields\n task: Optional[str] = None # \"transcribe\" or \"translate\"\n duration: Optional[float] = None\n segments: Optional[List[Segment]] = None\n words: Optional[List[Word]] = None\n\n # Multi-channel and alternatives support (Deepgram, Azure, etc.)\n channels: Optional[List[Channel]] = None\n alternatives: Optional[List[Alternative]] = None\n\n # Advanced features (various providers)\n utterances: Optional[List[dict]] = None # Speaker utterances\n paragraphs: Optional[List[dict]] = None # Paragraph detection\n topics: Optional[List[dict]] = None # Topic detection\n intents: Optional[List[dict]] = None # Intent recognition\n sentiment: Optional[dict] = None # Sentiment analysis\n summary: Optional[dict] = None # Auto-summarization\n\n # Metadata\n metadata: Optional[dict] = None # Provider-specific metadata\n model_info: Optional[dict] = None # Model information\n\n\nclass StreamingTranscriptionChunk(BaseModel):\n \"\"\"Represents a single chunk of streaming transcription data.\"\"\"\n\n text: str\n is_final: bool\n confidence: Optional[float] = None\n start_time: Optional[float] = None\n end_time: Optional[float] = None\n speaker_id: Optional[int] = None\n speaker_confidence: Optional[float] = None\n words: Optional[List[Word]] = None\n sequence_number: Optional[int] = None\n channel: Optional[int] = None\n provider_data: Optional[dict] = None\n\n\n# Type alias for streaming transcription responses\nStreamingTranscriptionResponse = AsyncGenerator[StreamingTranscriptionChunk, None]\n\n# Union type for both batch and streaming responses\nTranscriptionResponse = Union[TranscriptionResult, StreamingTranscriptionResponse]\n\n\n@dataclass\nclass TranscriptionOptions:\n \"\"\"Unified transcription options for ASR providers.\"\"\"\n\n # Core parameters\n language: Optional[str] = None\n\n # Audio format parameters\n audio_format: Optional[str] = None\n sample_rate: Optional[int] = None\n channels: Optional[int] = None\n encoding: Optional[str] = None # Audio encoding type\n\n # Output format\n response_format: Optional[str] = None\n include_word_timestamps: Optional[bool] = None\n include_segment_timestamps: Optional[bool] = None\n timestamp_granularities: Optional[List[str]] = None # OpenAI: [\"word\", \"segment\"]\n\n # Context and guidance\n prompt: Optional[str] = None\n context_phrases: Optional[List[str]] = None\n boost_phrases: Optional[List[str]] = None\n\n # Speaker features\n enable_speaker_diarization: Optional[bool] = None\n max_speakers: Optional[int] = None\n min_speakers: Optional[int] = None\n\n # Text processing\n enable_automatic_punctuation: Optional[bool] = None\n enable_profanity_filter: Optional[bool] = None\n enable_smart_formatting: Optional[bool] = None\n enable_word_confidence: Optional[bool] = None\n enable_spoken_punctuation: Optional[bool] = None\n enable_spoken_emojis: Optional[bool] = None\n\n # Advanced features\n enable_sentiment_analysis: Optional[bool] = None\n enable_topic_detection: Optional[bool] = None\n enable_intent_recognition: Optional[bool] = None\n enable_summarization: Optional[bool] = None\n enable_translation: Optional[bool] = None\n translation_target_language: Optional[str] = None\n\n # Confidence and alternatives\n include_confidence_scores: Optional[bool] = None\n max_alternatives: Optional[int] = None\n\n # Processing options\n temperature: Optional[float] = None\n interim_results: Optional[bool] = None\n vad_sensitivity: Optional[float] = None\n stream: Optional[bool] = None # Enable streaming output\n\n # Custom parameters\n custom_parameters: Dict[str, Any] = field(default_factory=dict)\n\n def __post_init__(self):\n \"\"\"Validate parameters and constraints.\"\"\"\n # Validate constraints\n if self.temperature is not None and not (0.0 <= self.temperature <= 1.0):\n raise ValueError(\"temperature must be between 0.0 and 1.0\")\n\n if self.max_speakers is not None and self.max_speakers < 1:\n raise ValueError(\"max_speakers must be at least 1\")\n\n if self.min_speakers is not None and self.min_speakers < 1:\n raise ValueError(\"min_speakers must be at least 1\")\n\n if (\n self.max_speakers is not None\n and self.min_speakers is not None\n and self.min_speakers > self.max_speakers\n ):\n raise ValueError(\"min_speakers cannot be greater than max_speakers\")\n\n if self.vad_sensitivity is not None and not (\n 0.0 <= self.vad_sensitivity <= 1.0\n ):\n raise ValueError(\"vad_sensitivity must be between 0.0 and 1.0\")\n\n def has_any_parameters(self) -> bool:\n \"\"\"Check if any parameters are set.\"\"\"\n for field_name, field_value in self.__dict__.items():\n if field_name == \"custom_parameters\":\n if field_value:\n return True\n elif field_value is not None:\n return True\n return False\n\n def get_set_parameters(self) -> Dict[str, Any]:\n \"\"\"Get only the parameters that are set.\"\"\"\n set_params = {}\n for field_name, field_value in self.__dict__.items():\n if field_name == \"custom_parameters\":\n if field_value:\n set_params[field_name] = field_value\n elif field_value is not None:\n set_params[field_name] = field_value\n return set_params\n" }, { "path": "aisuite/framework/parameter_mapper.py", "content": "\"\"\"\nParameter mapping utilities for ASR providers.\nMaps unified TranscriptionOptions to provider-specific parameters.\n\"\"\"\n\nfrom typing import Dict, Any, List, TYPE_CHECKING\n\nif TYPE_CHECKING:\n from .message import TranscriptionOptions\n\n\nclass ParameterMapper:\n \"\"\"Maps unified TranscriptionOptions to provider-specific parameters.\"\"\"\n\n # OpenAI Whisper API parameter mapping\n OPENAI_MAPPING = {\n \"language\": \"language\",\n \"response_format\": \"response_format\",\n \"temperature\": \"temperature\",\n \"prompt\": \"prompt\",\n \"stream\": \"stream\",\n \"timestamp_granularities\": \"timestamp_granularities\",\n }\n\n # Deepgram API parameter mapping\n DEEPGRAM_MAPPING = {\n \"language\": \"language\",\n \"enable_automatic_punctuation\": \"punctuate\",\n \"enable_smart_formatting\": \"smart_format\",\n \"enable_speaker_diarization\": \"diarize\",\n \"include_word_timestamps\": \"utterances\",\n \"include_segment_timestamps\": \"paragraphs\",\n \"context_phrases\": \"keywords\",\n \"enable_profanity_filter\": \"profanity_filter\",\n \"enable_sentiment_analysis\": \"sentiment\",\n \"enable_topic_detection\": \"topics\",\n \"enable_intent_recognition\": \"intents\",\n \"enable_summarization\": \"summarize\",\n \"interim_results\": \"interim_results\",\n \"channels\": \"channels\",\n \"sample_rate\": \"sample_rate\",\n \"include_confidence_scores\": \"confidence\",\n \"enable_word_confidence\": \"confidence\",\n \"max_alternatives\": \"alternatives\",\n \"stream\": \"interim_results\",\n \"encoding\": \"encoding\",\n # timestamp_granularities is handled specially for Deepgram\n }\n\n # Google API parameter mapping\n GOOGLE_MAPPING = {\n \"language\": \"language_code\",\n \"sample_rate\": \"sample_rate_hertz\",\n \"channels\": \"audio_channel_count\",\n \"enable_automatic_punctuation\": \"enable_automatic_punctuation\",\n \"enable_speaker_diarization\": \"enable_speaker_diarization\",\n \"max_speakers\": \"diarization_speaker_count\",\n \"min_speakers\": \"min_speaker_count\",\n \"include_word_timestamps\": \"enable_word_time_offsets\",\n \"include_confidence_scores\": \"enable_word_confidence\",\n \"enable_word_confidence\": \"enable_word_confidence\",\n \"context_phrases\": \"speech_contexts\",\n \"enable_profanity_filter\": \"profanity_filter\",\n \"max_alternatives\": \"max_alternatives\",\n \"boost_phrases\": \"speech_contexts\",\n \"audio_format\": \"encoding\",\n \"encoding\": \"encoding\",\n \"interim_results\": \"interim_results\",\n \"stream\": \"interim_results\",\n \"enable_spoken_punctuation\": \"enable_spoken_punctuation\",\n \"enable_spoken_emojis\": \"enable_spoken_emojis\",\n }\n\n @classmethod\n def map_to_openai(cls, options: \"TranscriptionOptions\") -> Dict[str, Any]:\n \"\"\"Map TranscriptionOptions to OpenAI Whisper API parameters.\"\"\"\n params = {}\n\n # Handle timestamp granularities\n timestamp_granularities = []\n if options.include_word_timestamps:\n timestamp_granularities.append(\"word\")\n if options.include_segment_timestamps:\n timestamp_granularities.append(\"segment\")\n if timestamp_granularities:\n params[\"timestamp_granularities\"] = timestamp_granularities\n\n # Map other parameters\n for opt_key, api_key in cls.OPENAI_MAPPING.items():\n if hasattr(options, opt_key):\n value = getattr(options, opt_key)\n if value is not None and not opt_key.startswith(\"include_\"):\n params[api_key] = value\n\n # Handle custom parameters\n cls._apply_custom_parameters(params, options.custom_parameters, \"openai\")\n\n return params\n\n @classmethod\n def map_to_deepgram(cls, options: \"TranscriptionOptions\") -> Dict[str, Any]:\n \"\"\"Map TranscriptionOptions to Deepgram API parameters.\"\"\"\n params = {}\n\n for opt_key, api_key in cls.DEEPGRAM_MAPPING.items():\n if hasattr(options, opt_key):\n value = getattr(options, opt_key)\n if value is not None:\n params[api_key] = value\n\n # Handle special cases\n if options.context_phrases:\n params[\"keywords\"] = options.context_phrases\n\n # Handle timestamp_granularities conversion for Deepgram\n if (\n hasattr(options, \"timestamp_granularities\")\n and options.timestamp_granularities\n ):\n if \"word\" in options.timestamp_granularities:\n params[\"utterances\"] = True\n if \"segment\" in options.timestamp_granularities:\n params[\"paragraphs\"] = True\n\n # Handle custom parameters\n cls._apply_custom_parameters(params, options.custom_parameters, \"deepgram\")\n\n return params\n\n @classmethod\n def map_to_google(cls, options: \"TranscriptionOptions\") -> Dict[str, Any]:\n \"\"\"Map TranscriptionOptions to Google Speech-to-Text API parameters.\"\"\"\n params = {}\n\n for opt_key, api_key in cls.GOOGLE_MAPPING.items():\n if hasattr(options, opt_key):\n value = getattr(options, opt_key)\n if value is not None:\n if opt_key == \"context_phrases\" or opt_key == \"boost_phrases\":\n if \"speech_contexts\" not in params:\n params[\"speech_contexts\"] = []\n params[\"speech_contexts\"].append({\"phrases\": value})\n elif opt_key == \"language\":\n # Handle language code conversion for Google\n # Google expects BCP-47 locale codes like \"en-US\", not just \"en\"\n if len(value) == 2: # Convert \"en\" to \"en-US\"\n language_map = {\n \"en\": \"en-US\",\n \"es\": \"es-ES\",\n \"fr\": \"fr-FR\",\n \"de\": \"de-DE\",\n \"it\": \"it-IT\",\n \"pt\": \"pt-BR\", # Portuguese -> Brazilian Portuguese\n \"ja\": \"ja-JP\",\n \"ko\": \"ko-KR\",\n \"zh\": \"zh-CN\", # Chinese -> Simplified Chinese\n \"ar\": \"ar-SA\", # Arabic -> Saudi Arabia\n \"hi\": \"hi-IN\", # Hindi -> India\n \"ru\": \"ru-RU\", # Russian -> Russia\n \"nl\": \"nl-NL\", # Dutch -> Netherlands\n \"pl\": \"pl-PL\", # Polish -> Poland\n \"sv\": \"sv-SE\", # Swedish -> Sweden\n \"da\": \"da-DK\", # Danish -> Denmark\n \"no\": \"nb-NO\", # Norwegian -> Norway\n \"fi\": \"fi-FI\", # Finnish -> Finland\n \"tr\": \"tr-TR\", # Turkish -> Turkey\n \"th\": \"th-TH\", # Thai -> Thailand\n \"vi\": \"vi-VN\", # Vietnamese -> Vietnam\n }\n params[api_key] = language_map.get(value, f\"{value}-US\")\n else:\n params[api_key] = value\n else:\n params[api_key] = value\n\n # Handle audio encoding mapping\n if options.audio_format:\n encoding_map = {\n \"wav\": \"LINEAR16\",\n \"flac\": \"FLAC\",\n \"mp3\": \"MP3\",\n \"ogg\": \"OGG_OPUS\",\n \"webm\": \"WEBM_OPUS\",\n }\n params[\"encoding\"] = encoding_map.get(\n options.audio_format.lower(), \"LINEAR16\"\n )\n\n # Handle timestamp_granularities conversion for Google\n if (\n hasattr(options, \"timestamp_granularities\")\n and options.timestamp_granularities\n ):\n if \"word\" in options.timestamp_granularities:\n params[\"enable_word_time_offsets\"] = True\n\n # Handle custom parameters\n cls._apply_custom_parameters(params, options.custom_parameters, \"google\")\n\n return params\n\n @classmethod\n def _apply_custom_parameters(\n cls, params: Dict[str, Any], custom_params: Dict[str, Any], provider: str\n ):\n \"\"\"\n Apply custom parameters for the specific provider.\n\n Only provider-namespaced parameters are supported.\n Parameters not under a provider key are IGNORED.\n \"\"\"\n if not custom_params:\n return\n\n # Provider-specific namespacing ONLY\n # Users MUST structure custom_parameters like:\n # {\n # \"openai\": {\"response_format\": \"srt\", \"temperature\": 0.2},\n # \"deepgram\": {\"search\": [\"keyword\"], \"numerals\": True},\n # \"google\": {\"use_enhanced\": True, \"adaptation\": {...}}\n # }\n if provider in custom_params:\n params.update(custom_params[provider])\n # Note: Any parameters not under a provider key are ignored\n" }, { "path": "aisuite/framework/provider_interface.py", "content": "\"\"\"The shared interface for model providers.\"\"\"\n\n\n# TODO(rohit): Remove this. This interface is obsolete in favor of Provider.\nclass ProviderInterface:\n \"\"\"Defines the expected behavior for provider-specific interfaces.\"\"\"\n\n def chat_completion_create(self, messages=None, model=None, temperature=0) -> None:\n \"\"\"Create a chat completion using the specified messages, model, and temperature.\n\n This method must be implemented by subclasses to perform completions.\n\n Args:\n ----\n messages (list): The chat history.\n model (str): The identifier of the model to be used in the completion.\n temperature (float): The temperature to use in the completion.\n\n Raises:\n ------\n NotImplementedError: If this method has not been implemented by a subclass.\n\n \"\"\"\n raise NotImplementedError(\n \"Provider Interface has not implemented chat_completion_create()\"\n )\n" }, { "path": "aisuite/mcp/__init__.py", "content": "\"\"\"\nMCP (Model Context Protocol) integration for aisuite.\n\nThis module provides support for using MCP servers and their tools with aisuite's\nunified interface for AI providers.\n\nMCP allows AI applications to connect to external data sources and tools through\na standardized protocol. This integration makes MCP tools available as Python\ncallables that work seamlessly with aisuite's existing tool calling infrastructure.\n\nExample:\n >>> from aisuite import Client\n >>> from aisuite.mcp import MCPClient\n >>>\n >>> # Connect to an MCP server\n >>> mcp = MCPClient(\n ... command=\"npx\",\n ... args=[\"-y\", \"@modelcontextprotocol/server-filesystem\", \"/docs\"]\n ... )\n >>>\n >>> # Use MCP tools with any provider\n >>> client = Client()\n >>> response = client.chat.completions.create(\n ... model=\"openai:gpt-4o\",\n ... messages=[{\"role\": \"user\", \"content\": \"Read README.md\"}],\n ... tools=mcp.get_callable_tools(),\n ... max_turns=2\n ... )\n\"\"\"\n\nfrom .client import MCPClient\n\n__all__ = [\"MCPClient\"]\n" }, { "path": "aisuite/mcp/client.py", "content": "\"\"\"\nMCP Client for aisuite.\n\nThis module provides the MCPClient class that connects to MCP servers and\nexposes their tools as Python callables compatible with aisuite's tool system.\n\"\"\"\n\nimport asyncio\nimport json\nfrom typing import Any, Callable, Dict, List, Optional\nfrom contextlib import contextmanager\n\ntry:\n from mcp import ClientSession, StdioServerParameters\n from mcp.client.stdio import stdio_client\n import httpx\nexcept ImportError as e:\n if \"mcp\" in str(e):\n raise ImportError(\n \"MCP support requires the 'mcp' package. \"\n \"Install it with: pip install 'aisuite[mcp]' or pip install mcp\"\n )\n elif \"httpx\" in str(e):\n raise ImportError(\n \"HTTP transport requires the 'httpx' package. \"\n \"Install it with: pip install httpx\"\n )\n raise\n\nfrom .tool_wrapper import create_mcp_tool_wrapper\nfrom .config import MCPConfig, validate_mcp_config, get_transport_type\n\n\nclass MCPClient:\n \"\"\"\n Client for connecting to MCP servers and using their tools with aisuite.\n\n This class manages the connection to an MCP server, discovers available tools,\n and creates Python callable wrappers that work seamlessly with aisuite's\n existing tool calling infrastructure.\n\n Example:\n >>> # Connect to an MCP server\n >>> mcp = MCPClient(\n ... command=\"npx\",\n ... args=[\"-y\", \"@modelcontextprotocol/server-filesystem\", \"/path\"]\n ... )\n >>>\n >>> # Get tools and use with aisuite\n >>> import aisuite as ai\n >>> client = ai.Client()\n >>> response = client.chat.completions.create(\n ... model=\"openai:gpt-4o\",\n ... messages=[{\"role\": \"user\", \"content\": \"List files\"}],\n ... tools=mcp.get_callable_tools(),\n ... max_turns=2\n ... )\n\n The MCPClient handles:\n - Starting and managing the MCP server process\n - Performing the MCP handshake\n - Discovering available tools\n - Creating callable wrappers for tools\n - Executing tool calls via the MCP protocol\n \"\"\"\n\n def __init__(\n self,\n command: Optional[str] = None,\n args: Optional[List[str]] = None,\n env: Optional[Dict[str, str]] = None,\n server_url: Optional[str] = None,\n headers: Optional[Dict[str, str]] = None,\n timeout: float = 30.0,\n name: Optional[str] = None,\n ):\n \"\"\"\n Initialize the MCP client and connect to an MCP server.\n\n Supports both stdio and HTTP transports. Provide either stdio parameters\n (command) OR HTTP parameters (server_url), but not both.\n\n Args:\n command: Command to start the MCP server (e.g., \"npx\", \"python\") - for stdio transport\n args: Arguments to pass to the command (e.g., [\"-y\", \"server-package\"]) - for stdio transport\n env: Optional environment variables for the server process - for stdio transport\n server_url: Base URL of the MCP server (e.g., \"http://localhost:8000\") - for HTTP transport\n headers: Optional HTTP headers (e.g., for authentication) - for HTTP transport\n timeout: Request timeout in seconds - for HTTP transport (default: 30.0)\n name: Optional name for this MCP client (used for logging and prefixing)\n\n Raises:\n ImportError: If the mcp or httpx package is not installed\n ValueError: If both stdio and HTTP parameters are provided, or neither\n RuntimeError: If connection to the MCP server fails\n \"\"\"\n # Validate transport parameters\n has_stdio = command is not None\n has_http = server_url is not None\n\n if not (has_stdio ^ has_http):\n raise ValueError(\n \"Must provide exactly one transport: either 'command' (stdio) or 'server_url' (HTTP).\"\n )\n\n # Store parameters based on transport type\n if has_stdio:\n self.server_params = StdioServerParameters(\n command=command,\n args=args or [],\n env=env,\n )\n self.name = name or command\n # Stdio-specific state\n self._session: Optional[ClientSession] = None\n self._read = None\n self._write = None\n self._stdio_context = None\n else: # HTTP\n self.server_url = server_url\n self.headers = headers or {}\n self.timeout = timeout\n self.name = name or server_url\n # HTTP-specific state (initialized in _async_connect_http)\n self._http_client = None\n self._request_id = 0\n self._session_id: Optional[str] = None # MCP session ID from server\n\n # Shared state\n self._tools_cache: Optional[List[Dict[str, Any]]] = None\n self._event_loop: Optional[asyncio.AbstractEventLoop] = None\n\n # Initialize connection\n self._connect()\n\n @classmethod\n def from_config(cls, config: Dict[str, Any]) -> \"MCPClient\":\n \"\"\"\n Create an MCPClient from a configuration dictionary.\n\n This method validates the config and creates an MCPClient instance.\n It supports both stdio and HTTP transports.\n\n Args:\n config: MCP configuration dictionary\n\n Returns:\n MCPClient instance\n\n Raises:\n ValueError: If configuration is invalid\n\n Example (stdio):\n >>> config = {\n ... \"type\": \"mcp\",\n ... \"name\": \"filesystem\",\n ... \"command\": \"npx\",\n ... \"args\": [\"-y\", \"@modelcontextprotocol/server-filesystem\", \"/docs\"]\n ... }\n >>> mcp = MCPClient.from_config(config)\n\n Example (HTTP):\n >>> config = {\n ... \"type\": \"mcp\",\n ... \"name\": \"api-server\",\n ... \"server_url\": \"http://localhost:8000\",\n ... \"headers\": {\"Authorization\": \"Bearer token\"}\n ... }\n >>> mcp = MCPClient.from_config(config)\n \"\"\"\n # Validate and normalize config\n validated_config = validate_mcp_config(config)\n\n # Determine transport type\n transport = get_transport_type(validated_config)\n\n if transport == \"stdio\":\n return cls(\n command=validated_config[\"command\"],\n args=validated_config.get(\"args\", []),\n env=validated_config.get(\"env\"),\n name=validated_config[\"name\"],\n )\n else: # http\n return cls(\n server_url=validated_config[\"server_url\"],\n headers=validated_config.get(\"headers\"),\n timeout=validated_config.get(\"timeout\", 30.0),\n name=validated_config[\"name\"],\n )\n\n @staticmethod\n def get_tools_from_config(config: Dict[str, Any]) -> List[Callable]:\n \"\"\"\n Convenience method to create MCPClient and get callable tools from config.\n\n This is a helper that combines from_config() and get_callable_tools()\n in a single call. It respects the config's allowed_tools and use_tool_prefix\n settings.\n\n Args:\n config: MCP configuration dictionary\n\n Returns:\n List of callable tool wrappers\n\n Example:\n >>> config = {\n ... \"type\": \"mcp\",\n ... \"name\": \"filesystem\",\n ... \"command\": \"npx\",\n ... \"args\": [\"...\"],\n ... \"allowed_tools\": [\"read_file\"],\n ... \"use_tool_prefix\": True\n ... }\n >>> tools = MCPClient.get_tools_from_config(config)\n >>> # Returns callable tools filtered and prefixed per config\n \"\"\"\n # Validate config first\n validated_config = validate_mcp_config(config)\n\n # Create client\n client = MCPClient.from_config(validated_config)\n\n # Get tools with config settings\n tools = client.get_callable_tools(\n allowed_tools=validated_config.get(\"allowed_tools\"),\n use_tool_prefix=validated_config.get(\"use_tool_prefix\", False),\n )\n\n return tools\n\n def _connect(self):\n \"\"\"\n Establish connection to the MCP server.\n\n This method:\n 1. Creates an event loop if needed\n 2. Detects transport type (stdio or HTTP)\n 3. Establishes connection via appropriate transport\n 4. Performs the MCP initialization handshake\n 5. Caches the available tools\n\n Note: Automatically handles Jupyter/IPython environments where an event loop\n is already running by using nest_asyncio.\n \"\"\"\n # Get or create event loop\n try:\n self._event_loop = asyncio.get_running_loop()\n except RuntimeError:\n self._event_loop = asyncio.new_event_loop()\n asyncio.set_event_loop(self._event_loop)\n\n # Enable nested event loops for Jupyter/IPython compatibility\n # This allows run_until_complete() to work in environments where\n # an event loop is already running (like Jupyter notebooks)\n try:\n import nest_asyncio\n\n nest_asyncio.apply()\n except ImportError:\n # nest_asyncio not available - will work fine in regular Python\n # but may fail in Jupyter. User should install: pip install nest-asyncio\n pass\n\n # Detect transport type and run appropriate async connection\n if hasattr(self, \"server_url\"):\n # HTTP transport\n self._event_loop.run_until_complete(self._async_connect_http())\n else:\n # Stdio transport\n self._event_loop.run_until_complete(self._async_connect())\n\n async def _async_connect(self):\n \"\"\"Async connection initialization for stdio transport.\"\"\"\n # Start the MCP server and store the context manager\n self._stdio_context = stdio_client(self.server_params)\n self._read, self._write = await self._stdio_context.__aenter__()\n\n # Create session\n self._session = ClientSession(self._read, self._write)\n await self._session.__aenter__()\n\n # Initialize connection\n await self._session.initialize()\n\n # List available tools and cache them\n tools_result = await self._session.list_tools()\n\n # Convert Tool objects to dicts for easier handling\n if hasattr(tools_result, \"tools\"):\n self._tools_cache = [\n {\n \"name\": tool.name,\n \"description\": (\n tool.description if hasattr(tool, \"description\") else \"\"\n ),\n \"inputSchema\": (\n tool.inputSchema if hasattr(tool, \"inputSchema\") else {}\n ),\n }\n for tool in tools_result.tools\n ]\n else:\n self._tools_cache = []\n\n async def _parse_sse_response(\n self, response: httpx.Response, request_id: int\n ) -> Dict[str, Any]:\n \"\"\"\n Parse SSE stream and extract JSON-RPC response.\n\n SSE format per spec:\n data: {\"jsonrpc\": \"2.0\", \"id\": 1, \"result\": {...}}\n\n data: {\"jsonrpc\": \"2.0\", \"method\": \"notification\", ...}\n\n The server may send multiple events (notifications, requests) before\n sending the final response. We collect events until we find the\n response matching our request_id.\n\n Args:\n response: HTTP response with text/event-stream content type\n request_id: The JSON-RPC request ID to match\n\n Returns:\n Response result dictionary\n\n Raises:\n RuntimeError: If server returns an error or no matching response found\n \"\"\"\n result = None\n\n async for line in response.aiter_lines():\n line = line.strip()\n\n # Skip empty lines and comments\n if not line or line.startswith(\":\"):\n continue\n\n # Parse SSE data field\n if line.startswith(\"data: \"):\n data = line[6:] # Remove 'data: ' prefix\n\n try:\n message = json.loads(data)\n\n # Check if this is the response to our request\n if message.get(\"id\") == request_id:\n if \"error\" in message:\n error = message[\"error\"]\n raise RuntimeError(\n f\"MCP server error: {error.get('message', 'Unknown error')} \"\n f\"(code: {error.get('code', 'unknown')})\"\n )\n result = message.get(\"result\", {})\n # Found our response, can stop parsing\n break\n\n # Note: Server may send other notifications/requests\n # which we ignore for now (future enhancement for bidirectional comms)\n\n except json.JSONDecodeError:\n # Invalid JSON in SSE data, skip this event\n continue\n\n if result is None:\n raise RuntimeError(\n f\"No response received in SSE stream for request {request_id}\"\n )\n\n return result\n\n async def _send_http_request(\n self, method: str, params: Optional[Dict[str, Any]] = None\n ) -> Dict[str, Any]:\n \"\"\"\n Send JSON-RPC request to MCP server via HTTP.\n\n Args:\n method: JSON-RPC method name\n params: Optional parameters\n\n Returns:\n Response result\n\n Raises:\n RuntimeError: If HTTP request fails or server returns an error\n \"\"\"\n # Increment request ID\n self._request_id += 1\n\n # Build JSON-RPC 2.0 request\n request_data = {\n \"jsonrpc\": \"2.0\",\n \"id\": self._request_id,\n \"method\": method,\n }\n\n if params:\n request_data[\"params\"] = params\n\n # Use the exact server URL provided by the user\n url = self.server_url.rstrip(\"/\")\n\n # Build headers: MCP requires Accept header with both content types\n # Merge with any user-provided headers and session ID\n request_headers = {\n \"Accept\": \"application/json, text/event-stream\",\n }\n if self._session_id:\n request_headers[\"Mcp-Session-Id\"] = self._session_id\n if self.headers:\n request_headers.update(self.headers)\n\n try:\n response = await self._http_client.post(\n url, json=request_data, headers=request_headers\n )\n response.raise_for_status()\n\n # Check for MCP session ID in response headers\n if \"Mcp-Session-Id\" in response.headers and not self._session_id:\n self._session_id = response.headers[\"Mcp-Session-Id\"]\n\n # Check Content-Type to determine response format\n content_type = response.headers.get(\"content-type\", \"\").lower()\n\n if \"application/json\" in content_type:\n # Handle JSON response (simple request-response)\n result = response.json()\n\n # Check for JSON-RPC error\n if \"error\" in result:\n error = result[\"error\"]\n raise RuntimeError(\n f\"MCP server error: {error.get('message', 'Unknown error')} \"\n f\"(code: {error.get('code', 'unknown')})\"\n )\n\n return result.get(\"result\", {})\n\n elif \"text/event-stream\" in content_type:\n # Handle SSE stream response\n return await self._parse_sse_response(response, request_data[\"id\"])\n\n else:\n raise RuntimeError(\n f\"Unexpected Content-Type from MCP server: {content_type}\"\n )\n\n except httpx.HTTPError as e:\n raise RuntimeError(\n f\"HTTP request to MCP server failed: {type(e).__name__}: {str(e)}\"\n )\n\n async def _send_notification(\n self, method: str, params: Optional[Dict[str, Any]] = None\n ):\n \"\"\"\n Send a JSON-RPC notification (no response expected).\n\n Notifications are JSON-RPC messages without an ID field.\n Per the spec, the server should not send a response.\n\n Args:\n method: JSON-RPC method name\n params: Optional parameters\n \"\"\"\n # Build JSON-RPC notification (no id field)\n notification = {\n \"jsonrpc\": \"2.0\",\n \"method\": method,\n }\n\n if params:\n notification[\"params\"] = params\n\n # Build headers\n url = self.server_url.rstrip(\"/\")\n request_headers = {\n \"Accept\": \"application/json, text/event-stream\",\n }\n if self._session_id:\n request_headers[\"Mcp-Session-Id\"] = self._session_id\n if self.headers:\n request_headers.update(self.headers)\n\n try:\n # Send notification - don't wait for/expect a response\n await self._http_client.post(\n url, json=notification, headers=request_headers\n )\n # Note: We don't check response for notifications\n except httpx.HTTPError:\n # Notifications may timeout or fail, which is acceptable\n pass\n\n async def _async_connect_http(self):\n \"\"\"Async connection initialization for HTTP transport.\"\"\"\n # Create HTTP client\n self._http_client = httpx.AsyncClient(timeout=self.timeout)\n\n # Send initialize request\n init_params = {\n \"protocolVersion\": \"2024-11-05\",\n \"capabilities\": {\"roots\": {\"listChanged\": True}, \"sampling\": {}},\n \"clientInfo\": {\"name\": \"aisuite-mcp-client\", \"version\": \"1.0.0\"},\n }\n\n await self._send_http_request(\"initialize\", init_params)\n\n # Send initialized notification (required by MCP spec)\n await self._send_notification(\"notifications/initialized\")\n\n # List available tools\n tools_result = await self._send_http_request(\"tools/list\")\n\n # Cache tools\n self._tools_cache = [\n {\n \"name\": tool[\"name\"],\n \"description\": tool.get(\"description\", \"\"),\n \"inputSchema\": tool.get(\"inputSchema\", {}),\n }\n for tool in tools_result.get(\"tools\", [])\n ]\n\n def list_tools(self) -> List[Dict[str, Any]]:\n \"\"\"\n List all available tools from the MCP server.\n\n Returns:\n List of tool schemas in MCP format\n\n Example:\n >>> tools = mcp.list_tools()\n >>> for tool in tools:\n ... print(tool['name'], '-', tool['description'])\n \"\"\"\n if self._tools_cache is None:\n raise RuntimeError(\"Not connected to MCP server\")\n return self._tools_cache\n\n def get_callable_tools(\n self,\n allowed_tools: Optional[List[str]] = None,\n use_tool_prefix: bool = False,\n ) -> List[Callable]:\n \"\"\"\n Get all MCP tools as Python callables compatible with aisuite.\n\n This is the primary method for using MCP tools with aisuite. It returns\n a list of callable wrappers that can be passed directly to the `tools`\n parameter of `client.chat.completions.create()`.\n\n Args:\n allowed_tools: Optional list of tool names to include. If None, all tools are included.\n use_tool_prefix: If True, prefix tool names with \"{client_name}__\"\n\n Returns:\n List of callable tool wrappers\n\n Example:\n >>> # Get all tools\n >>> mcp_tools = mcp.get_callable_tools()\n >>>\n >>> # Get specific tools only\n >>> mcp_tools = mcp.get_callable_tools(allowed_tools=[\"read_file\"])\n >>>\n >>> # Get tools with name prefixing\n >>> mcp_tools = mcp.get_callable_tools(use_tool_prefix=True)\n >>> # Tools will be named \"filesystem__read_file\", etc.\n \"\"\"\n all_tools = self.list_tools()\n\n # Filter tools if allowed_tools is specified\n if allowed_tools is not None:\n all_tools = [t for t in all_tools if t[\"name\"] in allowed_tools]\n\n # Create wrappers\n wrappers = []\n for tool in all_tools:\n wrapper = create_mcp_tool_wrapper(self, tool[\"name\"], tool)\n\n # Apply prefix if requested\n if use_tool_prefix:\n original_name = wrapper.__name__\n wrapper.__name__ = f\"{self.name}__{original_name}\"\n\n wrappers.append(wrapper)\n\n return wrappers\n\n def get_tool(self, tool_name: str) -> Optional[Callable]:\n \"\"\"\n Get a specific MCP tool by name as a Python callable.\n\n Args:\n tool_name: Name of the tool to retrieve\n\n Returns:\n Callable wrapper for the tool, or None if not found\n\n Example:\n >>> read_file = mcp.get_tool(\"read_file\")\n >>> write_file = mcp.get_tool(\"write_file\")\n >>> tools = [read_file, write_file]\n \"\"\"\n tools = self.list_tools()\n for tool in tools:\n if tool[\"name\"] == tool_name:\n return create_mcp_tool_wrapper(self, tool_name, tool)\n return None\n\n def call_tool(self, tool_name: str, arguments: Dict[str, Any]) -> Any:\n \"\"\"\n Execute an MCP tool call.\n\n This method is called by MCPToolWrapper when the LLM requests a tool.\n It handles the async MCP protocol communication and returns the result.\n Automatically routes to the appropriate transport (stdio or HTTP).\n\n Args:\n tool_name: Name of the tool to call\n arguments: Tool arguments as a dictionary\n\n Returns:\n The result from the MCP tool execution\n\n Raises:\n RuntimeError: If not connected or tool call fails\n \"\"\"\n # Detect transport type and route to appropriate method\n if hasattr(self, \"_http_client\") and self._http_client is not None:\n # HTTP transport\n if self._http_client is None:\n raise RuntimeError(\"Not connected to MCP server (HTTP)\")\n result = self._event_loop.run_until_complete(\n self._async_call_tool_http(tool_name, arguments)\n )\n else:\n # Stdio transport\n if self._session is None:\n raise RuntimeError(\"Not connected to MCP server (stdio)\")\n result = self._event_loop.run_until_complete(\n self._async_call_tool(tool_name, arguments)\n )\n return result\n\n async def _async_call_tool(self, tool_name: str, arguments: Dict[str, Any]) -> Any:\n \"\"\"\n Async implementation of tool calling for stdio transport.\n\n Args:\n tool_name: Name of the tool\n arguments: Tool arguments\n\n Returns:\n Tool execution result\n \"\"\"\n result = await self._session.call_tool(tool_name, arguments)\n\n # Extract content from MCP result\n # MCP returns results in various formats, we try to extract the most useful content\n if hasattr(result, \"content\"):\n if isinstance(result.content, list) and len(result.content) > 0:\n # Get first content item\n content_item = result.content[0]\n if hasattr(content_item, \"text\"):\n return content_item.text\n elif hasattr(content_item, \"data\"):\n return content_item.data\n return str(content_item)\n return result.content\n\n # If no content attribute, return the whole result\n return str(result)\n\n async def _async_call_tool_http(\n self, tool_name: str, arguments: Dict[str, Any]\n ) -> Any:\n \"\"\"\n Async implementation of tool calling for HTTP transport.\n\n Args:\n tool_name: Name of the tool\n arguments: Tool arguments\n\n Returns:\n Tool execution result\n \"\"\"\n params = {\"name\": tool_name, \"arguments\": arguments}\n\n result = await self._send_http_request(\"tools/call\", params)\n\n # Extract content from MCP result (HTTP format)\n # Similar to stdio, but result is already a dict\n if \"content\" in result:\n content = result[\"content\"]\n if isinstance(content, list) and len(content) > 0:\n # Get first content item\n content_item = content[0]\n if isinstance(content_item, dict):\n if \"text\" in content_item:\n return content_item[\"text\"]\n elif \"data\" in content_item:\n return content_item[\"data\"]\n return str(content_item)\n return content\n\n # If no content field, return the whole result\n return json.dumps(result)\n\n def close(self):\n \"\"\"\n Close the connection to the MCP server.\n\n Works for both stdio and HTTP transports. It's recommended to use\n the MCPClient as a context manager to ensure proper cleanup, but\n this method can be called manually if needed.\n\n Example:\n >>> mcp = MCPClient(command=\"npx\", args=[\"server\"])\n >>> try:\n ... # Use mcp\n ... pass\n ... finally:\n ... mcp.close()\n \"\"\"\n # Check if we need to cleanup (either stdio or HTTP)\n needs_cleanup = (hasattr(self, \"_session\") and self._session is not None) or (\n hasattr(self, \"_http_client\") and self._http_client is not None\n )\n\n if needs_cleanup:\n self._event_loop.run_until_complete(self._async_close())\n\n async def _async_close(self):\n \"\"\"Async cleanup for both stdio and HTTP transports.\"\"\"\n # Cleanup stdio transport\n try:\n if hasattr(self, \"_session\") and self._session:\n await self._session.__aexit__(None, None, None)\n except RuntimeError as e:\n # Suppress anyio cancel scope errors that occur in Jupyter/nest_asyncio environments\n # This is a known incompatibility between nest_asyncio and anyio task groups\n if \"cancel scope\" not in str(e).lower():\n raise\n except Exception:\n pass # Ignore other errors during session cleanup\n\n try:\n if hasattr(self, \"_stdio_context\") and self._stdio_context:\n await self._stdio_context.__aexit__(None, None, None)\n except RuntimeError as e:\n # Suppress anyio cancel scope errors that occur in Jupyter/nest_asyncio environments\n # This is a known incompatibility between nest_asyncio and anyio task groups\n if \"cancel scope\" not in str(e).lower():\n raise\n except Exception:\n pass # Ignore other errors during stdio cleanup\n\n # Cleanup HTTP transport\n try:\n if hasattr(self, \"_http_client\") and self._http_client:\n await self._http_client.aclose()\n except Exception:\n pass # Ignore errors during HTTP client cleanup\n\n def __enter__(self):\n \"\"\"Context manager entry.\"\"\"\n return self\n\n def __exit__(self, exc_type, exc_val, exc_tb):\n \"\"\"Context manager exit.\"\"\"\n self.close()\n return False\n\n def __repr__(self) -> str:\n \"\"\"String representation.\"\"\"\n num_tools = len(self._tools_cache) if self._tools_cache else 0\n if hasattr(self, \"server_url\"):\n return f\"MCPClient(server_url={self.server_url!r}, tools={num_tools})\"\n else:\n return (\n f\"MCPClient(command={self.server_params.command!r}, tools={num_tools})\"\n )\n" }, { "path": "aisuite/mcp/config.py", "content": "\"\"\"\nMCP configuration validation and normalization.\n\nThis module provides utilities for validating and normalizing MCP tool\nconfiguration dictionaries passed to aisuite's chat completion API.\n\"\"\"\n\nfrom typing import Any, Dict, List, Literal, Optional, TypedDict\n\n\nclass MCPConfig(TypedDict, total=False):\n \"\"\"Type definition for MCP tool configuration.\"\"\"\n\n # Required fields\n type: Literal[\"mcp\"]\n name: str\n\n # Transport: stdio\n command: str\n args: List[str]\n env: Dict[str, str]\n cwd: str\n\n # Transport: http\n server_url: str\n headers: Dict[str, str]\n\n # Tool filtering\n allowed_tools: List[str]\n\n # Namespacing\n use_tool_prefix: bool\n\n # Safety limits\n timeout_seconds: int\n response_bytes_cap: int\n\n # Connection behavior\n lazy_connect: bool\n\n\n# Default values\nDEFAULT_TIMEOUT_SECONDS = 30\nDEFAULT_RESPONSE_BYTES_CAP = 10 * 1024 * 1024 # 10 MB\nDEFAULT_USE_TOOL_PREFIX = False\nDEFAULT_LAZY_CONNECT = False\n\n\ndef validate_mcp_config(config: Dict[str, Any]) -> MCPConfig:\n \"\"\"\n Validate and normalize an MCP tool configuration.\n\n This function:\n 1. Validates required fields are present\n 2. Auto-detects transport type (stdio vs http)\n 3. Validates transport-specific required fields\n 4. Sets defaults for optional fields\n 5. Returns a normalized config dict\n\n Args:\n config: Raw MCP configuration dictionary\n\n Returns:\n Validated and normalized MCP configuration\n\n Raises:\n ValueError: If configuration is invalid\n\n Example:\n >>> config = {\n ... \"type\": \"mcp\",\n ... \"name\": \"filesystem\",\n ... \"command\": \"npx\",\n ... \"args\": [\"-y\", \"@modelcontextprotocol/server-filesystem\", \"/docs\"]\n ... }\n >>> validated = validate_mcp_config(config)\n >>> validated['timeout_seconds']\n 30\n \"\"\"\n # Check type field\n if config.get(\"type\") != \"mcp\":\n raise ValueError(f\"Invalid config type: {config.get('type')}. Expected 'mcp'\")\n\n # Check name field (required)\n if \"name\" not in config:\n raise ValueError(\n \"MCP config must have 'name' field. \"\n \"Example: {'type': 'mcp', 'name': 'my_server', ...}\"\n )\n\n name = config[\"name\"]\n if not isinstance(name, str) or not name.strip():\n raise ValueError(f\"MCP 'name' must be a non-empty string, got: {name}\")\n\n # Auto-detect transport type\n has_stdio = \"command\" in config\n has_http = \"server_url\" in config\n\n if not (has_stdio ^ has_http):\n raise ValueError(\n \"MCP config must have either 'command' or 'server_url'.\"\n \"Use one or the other to specify transport type.\"\n )\n\n # Validate stdio transport\n if has_stdio:\n if not isinstance(config[\"command\"], str):\n raise ValueError(\n f\"MCP 'command' must be a string, got: {type(config['command'])}\"\n )\n\n # args is optional but should be a list if present\n if \"args\" in config and not isinstance(config[\"args\"], list):\n raise ValueError(f\"MCP 'args' must be a list, got: {type(config['args'])}\")\n\n # env is optional but should be a dict if present\n if \"env\" in config and not isinstance(config[\"env\"], dict):\n raise ValueError(f\"MCP 'env' must be a dict, got: {type(config['env'])}\")\n\n # Validate http transport\n if has_http:\n if not isinstance(config[\"server_url\"], str):\n raise ValueError(\n f\"MCP 'server_url' must be a string, got: {type(config['server_url'])}\"\n )\n\n # Validate URL format\n server_url = config[\"server_url\"]\n if not (server_url.startswith(\"http://\") or server_url.startswith(\"https://\")):\n raise ValueError(\n f\"MCP 'server_url' must start with http:// or https://, got: {server_url}\"\n )\n\n # headers is optional but should be a dict if present\n if \"headers\" in config and not isinstance(config[\"headers\"], dict):\n raise ValueError(\n f\"MCP 'headers' must be a dict, got: {type(config['headers'])}\"\n )\n\n # timeout is optional but should be a number if present\n if \"timeout\" in config:\n if not isinstance(config[\"timeout\"], (int, float)):\n raise ValueError(\n f\"MCP 'timeout' must be a number, got: {type(config['timeout'])}\"\n )\n if config[\"timeout\"] <= 0:\n raise ValueError(\n f\"MCP 'timeout' must be positive, got: {config['timeout']}\"\n )\n\n # Validate optional fields\n if \"allowed_tools\" in config:\n if not isinstance(config[\"allowed_tools\"], list):\n raise ValueError(\n f\"MCP 'allowed_tools' must be a list, got: {type(config['allowed_tools'])}\"\n )\n if not all(isinstance(t, str) for t in config[\"allowed_tools\"]):\n raise ValueError(\"MCP 'allowed_tools' must be a list of strings\")\n\n if \"use_tool_prefix\" in config:\n if not isinstance(config[\"use_tool_prefix\"], bool):\n raise ValueError(\n f\"MCP 'use_tool_prefix' must be a boolean, got: {type(config['use_tool_prefix'])}\"\n )\n\n if \"timeout_seconds\" in config:\n if not isinstance(config[\"timeout_seconds\"], (int, float)):\n raise ValueError(\n f\"MCP 'timeout_seconds' must be a number, got: {type(config['timeout_seconds'])}\"\n )\n if config[\"timeout_seconds\"] <= 0:\n raise ValueError(\n f\"MCP 'timeout_seconds' must be positive, got: {config['timeout_seconds']}\"\n )\n\n if \"response_bytes_cap\" in config:\n if not isinstance(config[\"response_bytes_cap\"], int):\n raise ValueError(\n f\"MCP 'response_bytes_cap' must be an integer, got: {type(config['response_bytes_cap'])}\"\n )\n if config[\"response_bytes_cap\"] <= 0:\n raise ValueError(\n f\"MCP 'response_bytes_cap' must be positive, got: {config['response_bytes_cap']}\"\n )\n\n # Create normalized config with defaults\n normalized: MCPConfig = {\n \"type\": \"mcp\",\n \"name\": config[\"name\"],\n }\n\n # Copy transport fields\n if has_stdio:\n normalized[\"command\"] = config[\"command\"]\n normalized[\"args\"] = config.get(\"args\", [])\n if \"env\" in config:\n normalized[\"env\"] = config[\"env\"]\n if \"cwd\" in config:\n normalized[\"cwd\"] = config[\"cwd\"]\n else: # has_http\n normalized[\"server_url\"] = config[\"server_url\"]\n if \"headers\" in config:\n normalized[\"headers\"] = config[\"headers\"]\n if \"timeout\" in config:\n normalized[\"timeout\"] = config[\"timeout\"]\n\n # Copy optional fields with defaults\n if \"allowed_tools\" in config:\n normalized[\"allowed_tools\"] = config[\"allowed_tools\"]\n\n normalized[\"use_tool_prefix\"] = config.get(\n \"use_tool_prefix\", DEFAULT_USE_TOOL_PREFIX\n )\n normalized[\"timeout_seconds\"] = config.get(\n \"timeout_seconds\", DEFAULT_TIMEOUT_SECONDS\n )\n normalized[\"response_bytes_cap\"] = config.get(\n \"response_bytes_cap\", DEFAULT_RESPONSE_BYTES_CAP\n )\n normalized[\"lazy_connect\"] = config.get(\"lazy_connect\", DEFAULT_LAZY_CONNECT)\n\n return normalized\n\n\ndef is_mcp_config(obj: Any) -> bool:\n \"\"\"\n Check if an object is an MCP config dictionary.\n\n Args:\n obj: Object to check\n\n Returns:\n True if obj is a dict with type=\"mcp\", False otherwise\n\n Example:\n >>> is_mcp_config({\"type\": \"mcp\", \"name\": \"test\"})\n True\n >>> is_mcp_config(lambda: None)\n False\n \"\"\"\n return isinstance(obj, dict) and obj.get(\"type\") == \"mcp\"\n\n\ndef get_transport_type(config: MCPConfig) -> Literal[\"stdio\", \"http\"]:\n \"\"\"\n Determine the transport type from a validated MCP config.\n\n Args:\n config: Validated MCP configuration\n\n Returns:\n \"stdio\" or \"http\"\n \"\"\"\n if \"command\" in config:\n return \"stdio\"\n else:\n return \"http\"\n" }, { "path": "aisuite/mcp/schema_converter.py", "content": "\"\"\"\nSchema conversion utilities for MCP tools.\n\nThis module provides functionality to convert MCP JSON Schema tool definitions\nto Python type annotations that are compatible with aisuite's existing Tools class.\n\"\"\"\n\nfrom typing import Any, Dict, List, Optional, Union, get_args, get_origin\nimport inspect\n\n\ndef json_schema_to_python_type(schema: Dict[str, Any]) -> type:\n \"\"\"\n Convert a JSON Schema type definition to a Python type annotation.\n\n Args:\n schema: JSON Schema type definition (e.g., {\"type\": \"string\"})\n\n Returns:\n Python type annotation (e.g., str, int, List[str], etc.)\n \"\"\"\n schema_type = schema.get(\"type\")\n\n # Handle null/None\n if schema_type == \"null\":\n return type(None)\n\n # Handle basic types\n type_mapping = {\n \"string\": str,\n \"number\": float,\n \"integer\": int,\n \"boolean\": bool,\n \"object\": dict,\n \"array\": list,\n }\n\n if schema_type in type_mapping:\n base_type = type_mapping[schema_type]\n\n # Handle arrays with item type\n if schema_type == \"array\" and \"items\" in schema:\n item_type = json_schema_to_python_type(schema[\"items\"])\n return List[item_type]\n\n return base_type\n\n # Handle anyOf/oneOf (union types)\n if \"anyOf\" in schema or \"oneOf\" in schema:\n union_schemas = schema.get(\"anyOf\", schema.get(\"oneOf\", []))\n types = [json_schema_to_python_type(s) for s in union_schemas]\n if len(types) == 1:\n return types[0]\n return Union[tuple(types)]\n\n # Default to Any if we can't determine the type\n return Any\n\n\ndef mcp_schema_to_annotations(input_schema: Dict[str, Any]) -> Dict[str, type]:\n \"\"\"\n Convert MCP tool input schema to Python type annotations.\n\n MCP tools use JSON Schema for their input parameters. This function\n converts those schemas to Python type annotations that can be used\n by aisuite's Tools class.\n\n Args:\n input_schema: MCP tool input schema (JSON Schema format)\n\n Returns:\n Dictionary mapping parameter names to Python types\n\n Example:\n >>> schema = {\n ... \"type\": \"object\",\n ... \"properties\": {\n ... \"location\": {\"type\": \"string\"},\n ... \"count\": {\"type\": \"integer\"}\n ... },\n ... \"required\": [\"location\"]\n ... }\n >>> annotations = mcp_schema_to_annotations(schema)\n >>> annotations\n {'location': , 'count': typing.Optional[int]}\n \"\"\"\n annotations = {}\n\n if input_schema.get(\"type\") != \"object\":\n return annotations\n\n properties = input_schema.get(\"properties\", {})\n required = input_schema.get(\"required\", [])\n\n for param_name, param_schema in properties.items():\n param_type = json_schema_to_python_type(param_schema)\n\n # Make optional if not in required list\n if param_name not in required:\n param_type = Optional[param_type]\n\n annotations[param_name] = param_type\n\n return annotations\n\n\ndef create_function_signature(\n func_name: str, annotations: Dict[str, type], docstring: Optional[str] = None\n) -> inspect.Signature:\n \"\"\"\n Create a function signature from parameter annotations.\n\n Args:\n func_name: Name of the function\n annotations: Dictionary mapping parameter names to types\n docstring: Optional docstring for the function\n\n Returns:\n inspect.Signature object\n \"\"\"\n parameters = []\n\n for param_name, param_type in annotations.items():\n # Check if it's an Optional type\n if get_origin(param_type) is Union:\n args = get_args(param_type)\n if type(None) in args:\n # It's Optional, set default to None\n parameters.append(\n inspect.Parameter(\n param_name,\n inspect.Parameter.KEYWORD_ONLY,\n default=None,\n annotation=param_type,\n )\n )\n else:\n parameters.append(\n inspect.Parameter(\n param_name,\n inspect.Parameter.KEYWORD_ONLY,\n annotation=param_type,\n )\n )\n else:\n # Required parameter\n parameters.append(\n inspect.Parameter(\n param_name,\n inspect.Parameter.KEYWORD_ONLY,\n annotation=param_type,\n )\n )\n\n return inspect.Signature(parameters)\n\n\ndef extract_parameter_descriptions(input_schema: Dict[str, Any]) -> Dict[str, str]:\n \"\"\"\n Extract parameter descriptions from MCP schema.\n\n Args:\n input_schema: MCP tool input schema\n\n Returns:\n Dictionary mapping parameter names to their descriptions\n \"\"\"\n descriptions = {}\n properties = input_schema.get(\"properties\", {})\n\n for param_name, param_schema in properties.items():\n if \"description\" in param_schema:\n descriptions[param_name] = param_schema[\"description\"]\n\n return descriptions\n\n\ndef build_docstring(\n tool_description: str, parameter_descriptions: Dict[str, str]\n) -> str:\n \"\"\"\n Build a Python docstring from MCP tool description and parameter descriptions.\n\n Args:\n tool_description: Overall description of the tool\n parameter_descriptions: Dictionary of parameter descriptions\n\n Returns:\n Formatted docstring\n \"\"\"\n lines = [tool_description, \"\"]\n\n if parameter_descriptions:\n lines.append(\"Args:\")\n for param_name, param_desc in parameter_descriptions.items():\n lines.append(f\" {param_name}: {param_desc}\")\n\n return \"\\n\".join(lines)\n" }, { "path": "aisuite/mcp/tool_wrapper.py", "content": "\"\"\"\nMCP Tool Wrapper for aisuite.\n\nThis module provides the MCPToolWrapper class, which creates Python callable\nwrappers around MCP tools that are compatible with aisuite's existing tool\ncalling infrastructure.\n\"\"\"\n\nfrom typing import Any, Callable, Dict, Optional\nimport asyncio\nimport inspect\nfrom .schema_converter import (\n mcp_schema_to_annotations,\n extract_parameter_descriptions,\n build_docstring,\n)\n\n\nclass MCPToolWrapper:\n \"\"\"\n A callable wrapper around an MCP tool that makes it compatible with aisuite.\n\n This class wraps an MCP tool and exposes it as a Python callable with proper\n type annotations and docstrings that aisuite's Tools class can inspect and use.\n\n The wrapper sets the following attributes that aisuite's Tools class reads:\n - __name__: The tool name\n - __doc__: The tool description and parameter documentation\n - __annotations__: Python type annotations for parameters\n\n When called, the wrapper executes the MCP tool via the MCP protocol.\n\n Example:\n >>> wrapper = MCPToolWrapper(mcp_client, \"read_file\", tool_schema)\n >>> result = wrapper(path=\"/path/to/file\")\n \"\"\"\n\n def __init__(\n self,\n mcp_client: \"MCPClient\", # Forward reference to avoid circular import\n tool_name: str,\n tool_schema: Dict[str, Any],\n ):\n \"\"\"\n Initialize the MCP tool wrapper.\n\n Args:\n mcp_client: The MCPClient instance that manages the connection\n tool_name: Name of the MCP tool\n tool_schema: MCP tool schema definition\n \"\"\"\n self.mcp_client = mcp_client\n self.tool_name = tool_name\n self.schema = tool_schema\n\n # Set attributes that aisuite's Tools class will inspect\n self.__name__ = tool_name\n\n # Build docstring from MCP schema\n description = tool_schema.get(\"description\", \"\")\n input_schema = tool_schema.get(\"inputSchema\", {})\n param_descriptions = extract_parameter_descriptions(input_schema)\n self.__doc__ = build_docstring(description, param_descriptions)\n\n # Convert MCP JSON Schema to Python type annotations\n self.__annotations__ = mcp_schema_to_annotations(input_schema)\n\n # Create a proper signature for inspect.signature() to read\n # This allows aisuite's Tools class to introspect the parameters\n self.__signature__ = self._create_signature(input_schema)\n\n # Store the original MCP inputSchema for direct use by Tools class\n # This avoids lossy round-trip conversion through Python type annotations\n # and preserves all JSON Schema details (arrays, nested objects, etc.)\n self.__mcp_input_schema__ = input_schema\n\n def _create_signature(self, input_schema: Dict[str, Any]) -> inspect.Signature:\n \"\"\"\n Create a signature for this wrapper based on MCP tool schema.\n\n This allows inspect.signature() to see the proper parameters with\n type annotations, rather than just **kwargs.\n \"\"\"\n properties = input_schema.get(\"properties\", {})\n required = input_schema.get(\"required\", [])\n\n parameters = []\n for param_name, annotation in self.__annotations__.items():\n # Create parameter with annotation and default\n if param_name in required:\n # Required parameter (no default)\n param = inspect.Parameter(\n param_name,\n inspect.Parameter.POSITIONAL_OR_KEYWORD,\n annotation=annotation,\n )\n else:\n # Optional parameter (with None default)\n param = inspect.Parameter(\n param_name,\n inspect.Parameter.POSITIONAL_OR_KEYWORD,\n default=None,\n annotation=annotation,\n )\n parameters.append(param)\n\n return inspect.Signature(parameters, return_annotation=Any)\n\n def __call__(self, **kwargs) -> Any:\n \"\"\"\n Execute the MCP tool with the given arguments.\n\n This method is called by aisuite's tool execution loop when the LLM\n requests this tool.\n\n Args:\n **kwargs: Tool arguments as keyword arguments\n\n Returns:\n The result from the MCP tool execution\n \"\"\"\n # Filter out None values - only pass parameters that have actual values\n # This prevents passing null to MCP tools that expect specific types\n # (e.g., a tool expecting number won't accept null, it wants the param omitted)\n filtered_kwargs = {k: v for k, v in kwargs.items() if v is not None}\n\n # Call the MCP client's tool execution method\n # The MCP client handles the async MCP protocol communication\n return self.mcp_client.call_tool(self.tool_name, filtered_kwargs)\n\n def __repr__(self) -> str:\n \"\"\"Return a string representation of the wrapper.\"\"\"\n return f\"MCPToolWrapper(name={self.tool_name!r})\"\n\n\ndef create_mcp_tool_wrapper(\n mcp_client: \"MCPClient\",\n tool_name: str,\n tool_schema: Dict[str, Any],\n) -> Callable:\n \"\"\"\n Factory function to create an MCP tool wrapper.\n\n Args:\n mcp_client: The MCPClient instance\n tool_name: Name of the tool\n tool_schema: MCP tool schema\n\n Returns:\n Callable wrapper for the MCP tool\n \"\"\"\n return MCPToolWrapper(mcp_client, tool_name, tool_schema)\n" }, { "path": "aisuite/provider.py", "content": "from abc import ABC, abstractmethod\nfrom pathlib import Path\nimport importlib\nimport os\nimport functools\nfrom typing import Union, BinaryIO, Optional\n\n\nclass LLMError(Exception):\n \"\"\"Custom exception for LLM errors.\"\"\"\n\n def __init__(self, message):\n super().__init__(message)\n\n\nclass ASRError(Exception):\n \"\"\"Custom exception for ASR errors.\"\"\"\n\n def __init__(self, message):\n super().__init__(message)\n\n\nclass Provider(ABC):\n def __init__(self):\n \"\"\"Initialize provider with optional audio functionality.\"\"\"\n self.audio: Optional[Audio] = None\n\n @abstractmethod\n def chat_completions_create(self, model, messages):\n \"\"\"Abstract method for chat completion calls, to be implemented by each provider.\"\"\"\n pass\n\n\nclass ProviderFactory:\n \"\"\"Factory to dynamically load provider instances based on naming conventions.\"\"\"\n\n PROVIDERS_DIR = Path(__file__).parent / \"providers\"\n\n @classmethod\n def create_provider(cls, provider_key, config):\n \"\"\"Dynamically load and create an instance of a provider based on the naming convention.\"\"\"\n # Convert provider_key to the expected module and class names\n provider_class_name = f\"{provider_key.capitalize()}Provider\"\n provider_module_name = f\"{provider_key}_provider\"\n\n module_path = f\"aisuite.providers.{provider_module_name}\"\n\n # Lazily load the module\n try:\n module = importlib.import_module(module_path)\n except ImportError as e:\n raise ImportError(\n f\"Could not import module {module_path}: {str(e)}. Please ensure the provider is supported by doing ProviderFactory.get_supported_providers()\"\n )\n\n # Instantiate the provider class\n provider_class = getattr(module, provider_class_name)\n return provider_class(**config)\n\n @classmethod\n @functools.cache\n def get_supported_providers(cls):\n \"\"\"List all supported provider names based on files present in the providers directory.\"\"\"\n provider_files = Path(cls.PROVIDERS_DIR).glob(\"*_provider.py\")\n return {file.stem.replace(\"_provider\", \"\") for file in provider_files}\n\n\nclass Audio:\n \"\"\"Base class for all audio functionality.\"\"\"\n\n def __init__(self):\n self.transcriptions: Optional[\"Audio.Transcription\"] = None\n\n class Transcription(ABC):\n \"\"\"Base class for audio transcription functionality.\"\"\"\n\n def create(\n self,\n model: str,\n file: Union[str, BinaryIO],\n options=None,\n **kwargs,\n ):\n \"\"\"Create audio transcription.\"\"\"\n raise NotImplementedError(\"Transcription not supported by this provider\")\n\n async def create_stream_output(\n self,\n model: str,\n file: Union[str, BinaryIO],\n options=None,\n **kwargs,\n ):\n \"\"\"Create streaming audio transcription.\"\"\"\n raise NotImplementedError(\n \"Streaming transcription not supported by this provider\"\n )\n" }, { "path": "aisuite/providers/__init__.py", "content": "" }, { "path": "aisuite/providers/anthropic_provider.py", "content": "# Anthropic provider\n# Links:\n# Tool calling docs - https://docs.anthropic.com/en/docs/build-with-claude/tool-use\n\nimport anthropic\nimport json\nfrom aisuite.provider import Provider\nfrom aisuite.framework import ChatCompletionResponse\nfrom aisuite.framework.message import (\n Message,\n ChatCompletionMessageToolCall,\n Function,\n CompletionUsage,\n PromptTokensDetails,\n)\n\n# Define a constant for the default max_tokens value\nDEFAULT_MAX_TOKENS = 4096\n\n\nclass AnthropicMessageConverter:\n # Role constants\n ROLE_USER = \"user\"\n ROLE_ASSISTANT = \"assistant\"\n ROLE_TOOL = \"tool\"\n ROLE_SYSTEM = \"system\"\n\n # Finish reason mapping\n FINISH_REASON_MAPPING = {\n \"end_turn\": \"stop\",\n \"max_tokens\": \"length\",\n \"tool_use\": \"tool_calls\",\n }\n\n def convert_request(self, messages):\n \"\"\"Convert framework messages to Anthropic format.\"\"\"\n system_message = self._extract_system_message(messages)\n converted_messages = [self._convert_single_message(msg) for msg in messages]\n return system_message, converted_messages\n\n def convert_response(self, response):\n \"\"\"Normalize the response from the Anthropic API to match OpenAI's response format.\"\"\"\n normalized_response = ChatCompletionResponse()\n normalized_response.choices[0].finish_reason = self._get_finish_reason(response)\n normalized_response.usage = self._get_completion_usage(response)\n normalized_response.choices[0].message = self._get_message(response)\n return normalized_response\n\n def _convert_single_message(self, msg):\n \"\"\"Convert a single message to Anthropic format.\"\"\"\n if isinstance(msg, dict):\n return self._convert_dict_message(msg)\n return self._convert_message_object(msg)\n\n def _convert_dict_message(self, msg):\n \"\"\"Convert a dictionary message to Anthropic format.\"\"\"\n if msg[\"role\"] == self.ROLE_TOOL:\n return self._create_tool_result_message(msg[\"tool_call_id\"], msg[\"content\"])\n elif msg[\"role\"] == self.ROLE_ASSISTANT and \"tool_calls\" in msg:\n return self._create_assistant_tool_message(\n msg[\"content\"], msg[\"tool_calls\"]\n )\n return {\"role\": msg[\"role\"], \"content\": msg[\"content\"]}\n\n def _convert_message_object(self, msg):\n \"\"\"Convert a Message object to Anthropic format.\"\"\"\n if msg.role == self.ROLE_TOOL:\n return self._create_tool_result_message(msg.tool_call_id, msg.content)\n elif msg.role == self.ROLE_ASSISTANT and msg.tool_calls:\n return self._create_assistant_tool_message(msg.content, msg.tool_calls)\n return {\"role\": msg.role, \"content\": msg.content}\n\n def _create_tool_result_message(self, tool_call_id, content):\n \"\"\"Create a tool result message in Anthropic format.\"\"\"\n return {\n \"role\": self.ROLE_USER,\n \"content\": [\n {\n \"type\": \"tool_result\",\n \"tool_use_id\": tool_call_id,\n \"content\": content,\n }\n ],\n }\n\n def _create_assistant_tool_message(self, content, tool_calls):\n \"\"\"Create an assistant message with tool calls in Anthropic format.\"\"\"\n message_content = []\n if content:\n message_content.append({\"type\": \"text\", \"text\": content})\n\n for tool_call in tool_calls:\n tool_input = (\n tool_call[\"function\"][\"arguments\"]\n if isinstance(tool_call, dict)\n else tool_call.function.arguments\n )\n message_content.append(\n {\n \"type\": \"tool_use\",\n \"id\": (\n tool_call[\"id\"] if isinstance(tool_call, dict) else tool_call.id\n ),\n \"name\": (\n tool_call[\"function\"][\"name\"]\n if isinstance(tool_call, dict)\n else tool_call.function.name\n ),\n \"input\": json.loads(tool_input),\n }\n )\n\n return {\"role\": self.ROLE_ASSISTANT, \"content\": message_content}\n\n def _extract_system_message(self, messages):\n \"\"\"Extract system message if present, otherwise return empty list.\"\"\"\n # TODO: This is a temporary solution to extract the system message.\n # User can pass multiple system messages, which can mingled with other messages.\n # This needs to be fixed to handle this case.\n if messages and messages[0][\"role\"] == \"system\":\n system_message = messages[0][\"content\"]\n messages.pop(0)\n return system_message\n return []\n\n def _get_finish_reason(self, response):\n \"\"\"Get the normalized finish reason.\"\"\"\n return self.FINISH_REASON_MAPPING.get(response.stop_reason, \"stop\")\n\n def _get_completion_usage(self, response):\n \"\"\"Get the usage statistics.\"\"\"\n return CompletionUsage(\n completion_tokens=response.usage.output_tokens,\n prompt_tokens=response.usage.input_tokens,\n total_tokens=response.usage.input_tokens + response.usage.output_tokens,\n prompt_tokens_details=PromptTokensDetails(\n cached_tokens=response.usage.cache_read_input_tokens,\n ),\n )\n\n def _get_message(self, response):\n \"\"\"Get the appropriate message based on response type.\"\"\"\n # Check if response contains any tool use blocks (regardless of stop_reason)\n has_tool_use = any(content.type == \"tool_use\" for content in response.content)\n\n if has_tool_use:\n tool_message = self.convert_response_with_tool_use(response)\n if tool_message:\n return tool_message\n\n # Safely extract text content from any position in content blocks\n text_content = next(\n (content.text for content in response.content if content.type == \"text\"),\n \"\",\n )\n\n return Message(\n content=text_content or None,\n role=\"assistant\",\n tool_calls=None,\n refusal=None,\n )\n\n def convert_response_with_tool_use(self, response):\n \"\"\"Convert Anthropic tool use response to the framework's format.\"\"\"\n tool_call = next(\n (content for content in response.content if content.type == \"tool_use\"),\n None,\n )\n\n if tool_call:\n function = Function(\n name=tool_call.name, arguments=json.dumps(tool_call.input)\n )\n tool_call_obj = ChatCompletionMessageToolCall(\n id=tool_call.id, function=function, type=\"function\"\n )\n text_content = next(\n (\n content.text\n for content in response.content\n if content.type == \"text\"\n ),\n \"\",\n )\n\n return Message(\n content=text_content or None,\n tool_calls=[tool_call_obj] if tool_call else None,\n role=\"assistant\",\n refusal=None,\n )\n return None\n\n def convert_tool_spec(self, openai_tools):\n \"\"\"Convert OpenAI tool specification to Anthropic format.\"\"\"\n anthropic_tools = []\n\n for tool in openai_tools:\n if tool.get(\"type\") != \"function\":\n continue\n\n function = tool[\"function\"]\n anthropic_tool = {\n \"name\": function[\"name\"],\n \"description\": function[\"description\"],\n \"input_schema\": {\n \"type\": \"object\",\n \"properties\": function[\"parameters\"][\"properties\"],\n \"required\": function[\"parameters\"].get(\"required\", []),\n },\n }\n anthropic_tools.append(anthropic_tool)\n\n return anthropic_tools\n\n\nclass AnthropicProvider(Provider):\n def __init__(self, **config):\n \"\"\"Initialize the Anthropic provider with the given configuration.\"\"\"\n self.client = anthropic.Anthropic(**config)\n self.converter = AnthropicMessageConverter()\n\n def chat_completions_create(self, model, messages, **kwargs):\n \"\"\"Create a chat completion using the Anthropic API.\"\"\"\n kwargs = self._prepare_kwargs(kwargs)\n system_message, converted_messages = self.converter.convert_request(messages)\n\n response = self.client.messages.create(\n model=model, system=system_message, messages=converted_messages, **kwargs\n )\n return self.converter.convert_response(response)\n\n def _prepare_kwargs(self, kwargs):\n \"\"\"Prepare kwargs for the API call.\"\"\"\n kwargs = kwargs.copy()\n kwargs.setdefault(\"max_tokens\", DEFAULT_MAX_TOKENS)\n\n if \"tools\" in kwargs:\n kwargs[\"tools\"] = self.converter.convert_tool_spec(kwargs[\"tools\"])\n\n return kwargs\n" }, { "path": "aisuite/providers/aws_provider.py", "content": "\"\"\"AWS Bedrock provider for the aisuite.\"\"\"\n\nimport os\nimport json\nfrom typing import List, Dict, Any, Tuple, Optional\n\nimport boto3\nimport botocore\n\nfrom aisuite.provider import Provider, LLMError\nfrom aisuite.framework import ChatCompletionResponse\nfrom aisuite.framework.message import Message, CompletionUsage\n\n\n# pylint: disable=too-few-public-methods\nclass BedrockConfig:\n \"\"\"Configuration for the AWS Bedrock provider.\"\"\"\n\n INFERENCE_PARAMETERS = [\"maxTokens\", \"temperature\", \"topP\", \"stopSequences\"]\n\n def __init__(self, **config):\n \"\"\"Initialize the BedrockConfig.\"\"\"\n self.region_name = config.get(\n \"region_name\", os.getenv(\"AWS_REGION\", \"us-west-2\")\n )\n\n def create_client(self):\n \"\"\"Create a Bedrock runtime client.\"\"\"\n return boto3.client(\"bedrock-runtime\", region_name=self.region_name)\n\n\n# AWS Bedrock API Example -\n# https://docs.aws.amazon.com/bedrock/latest/userguide/tool-use-inference-call.html\n# https://docs.aws.amazon.com/bedrock/latest/userguide/tool-use-examples.html\nclass BedrockMessageConverter:\n \"\"\"Converts messages between OpenAI and AWS Bedrock formats.\"\"\"\n\n @staticmethod\n def convert_request(\n messages: List[Dict[str, Any]],\n ) -> Tuple[List[Dict], List[Dict]]:\n \"\"\"Convert messages to AWS Bedrock format.\"\"\"\n # Convert all messages to dicts if they're Message objects\n messages = [\n message.model_dump() if hasattr(message, \"model_dump\") else message\n for message in messages\n ]\n\n # Handle system message\n system_message = []\n if messages and messages[0][\"role\"] == \"system\":\n system_message = [{\"text\": messages[0][\"content\"]}]\n messages = messages[1:]\n\n formatted_messages = []\n for message in messages:\n # Skip any additional system messages\n if message[\"role\"] == \"system\":\n continue\n\n if message[\"role\"] == \"tool\":\n bedrock_message = BedrockMessageConverter.convert_tool_result(message)\n if bedrock_message:\n formatted_messages.append(bedrock_message)\n elif message[\"role\"] == \"assistant\":\n bedrock_message = BedrockMessageConverter.convert_assistant(message)\n if bedrock_message:\n formatted_messages.append(bedrock_message)\n else: # user messages\n formatted_messages.append(\n {\n \"role\": message[\"role\"],\n \"content\": [{\"text\": message[\"content\"]}],\n }\n )\n\n return system_message, formatted_messages\n\n @staticmethod\n def convert_response_tool_call(\n response: Dict[str, Any],\n ) -> Optional[Dict[str, Any]]:\n \"\"\"Convert AWS Bedrock tool call response to OpenAI format.\"\"\"\n if response.get(\"stopReason\") != \"tool_use\":\n return None\n\n tool_calls = []\n for content in response[\"output\"][\"message\"][\"content\"]:\n if \"toolUse\" in content:\n tool = content[\"toolUse\"]\n tool_calls.append(\n {\n \"type\": \"function\",\n \"id\": tool[\"toolUseId\"],\n \"function\": {\n \"name\": tool[\"name\"],\n \"arguments\": json.dumps(tool[\"input\"]),\n },\n }\n )\n\n if not tool_calls:\n return None\n\n return {\n \"role\": \"assistant\",\n \"content\": None,\n \"tool_calls\": tool_calls,\n \"refusal\": None,\n }\n\n @staticmethod\n def convert_tool_result(message: Dict[str, Any]) -> Optional[Dict[str, Any]]:\n \"\"\"Convert OpenAI tool result format to AWS Bedrock format.\"\"\"\n if message[\"role\"] != \"tool\" or \"content\" not in message:\n return None\n\n tool_call_id = message.get(\"tool_call_id\")\n if not tool_call_id:\n raise LLMError(\"Tool result message must include tool_call_id\")\n\n try:\n content_json = json.loads(message[\"content\"])\n content = [{\"json\": content_json}]\n except json.JSONDecodeError:\n content = [{\"text\": message[\"content\"]}]\n\n return {\n \"role\": \"user\",\n \"content\": [\n {\"toolResult\": {\"toolUseId\": tool_call_id, \"content\": content}}\n ],\n }\n\n @staticmethod\n def convert_assistant(message: Dict[str, Any]) -> Optional[Dict[str, Any]]:\n \"\"\"Convert OpenAI assistant format to AWS Bedrock format.\"\"\"\n if message[\"role\"] != \"assistant\":\n return None\n\n content = []\n\n if message.get(\"content\"):\n content.append({\"text\": message[\"content\"]})\n\n if message.get(\"tool_calls\"):\n for tool_call in message[\"tool_calls\"]:\n if tool_call[\"type\"] == \"function\":\n try:\n input_json = json.loads(tool_call[\"function\"][\"arguments\"])\n except json.JSONDecodeError:\n input_json = tool_call[\"function\"][\"arguments\"]\n\n content.append(\n {\n \"toolUse\": {\n \"toolUseId\": tool_call[\"id\"],\n \"name\": tool_call[\"function\"][\"name\"],\n \"input\": input_json,\n }\n }\n )\n\n return {\"role\": \"assistant\", \"content\": content} if content else None\n\n @staticmethod\n def convert_response(response: Dict[str, Any]) -> ChatCompletionResponse:\n \"\"\"Normalize the response from the Bedrock API to match OpenAI's response format.\"\"\"\n norm_response = ChatCompletionResponse()\n\n # Check if the model is requesting tool use\n if response.get(\"stopReason\") == \"tool_use\":\n tool_message = BedrockMessageConverter.convert_response_tool_call(response)\n if tool_message:\n norm_response.choices[0].message = Message(**tool_message)\n norm_response.choices[0].finish_reason = \"tool_calls\"\n return norm_response\n\n # Handle regular text response\n norm_response.choices[0].message.content = response[\"output\"][\"message\"][\n \"content\"\n ][0][\"text\"]\n\n # Map Bedrock stopReason to OpenAI finish_reason\n stop_reason = response.get(\"stopReason\")\n if stop_reason == \"complete\":\n norm_response.choices[0].finish_reason = \"stop\"\n elif stop_reason == \"max_tokens\":\n norm_response.choices[0].finish_reason = \"length\"\n else:\n norm_response.choices[0].finish_reason = stop_reason\n\n # Conditionally parse usage data if it exists.\n if usage_data := response.get(\"usage\"):\n norm_response.usage = BedrockMessageConverter.get_completion_usage(\n usage_data\n )\n\n return norm_response\n\n @staticmethod\n def get_completion_usage(usage_data: dict):\n \"\"\"Get the usage statistics from a usage data dictionary.\"\"\"\n return CompletionUsage(\n completion_tokens=usage_data.get(\"outputTokens\"),\n prompt_tokens=usage_data.get(\"inputTokens\"),\n total_tokens=usage_data.get(\"totalTokens\"),\n )\n\n\nclass AwsProvider(Provider):\n \"\"\"Provider for AWS Bedrock.\"\"\"\n\n def __init__(self, **config):\n \"\"\"Initialize the AWS Bedrock provider with the given configuration.\"\"\"\n self.config = BedrockConfig(**config)\n self.client = self.config.create_client()\n self.transformer = BedrockMessageConverter()\n\n def convert_response(self, response: Dict[str, Any]) -> ChatCompletionResponse:\n \"\"\"Normalize the response from the Bedrock API to match OpenAI's response format.\"\"\"\n return self.transformer.convert_response(response)\n\n def _convert_tool_spec(self, kwargs: Dict[str, Any]) -> Optional[Dict[str, Any]]:\n \"\"\"Convert tool specifications to Bedrock format.\"\"\"\n if \"tools\" not in kwargs:\n return None\n\n tool_config = {\n \"tools\": [\n {\n \"toolSpec\": {\n \"name\": tool[\"function\"][\"name\"],\n \"description\": tool[\"function\"].get(\"description\", \" \"),\n \"inputSchema\": {\"json\": tool[\"function\"][\"parameters\"]},\n }\n }\n for tool in kwargs[\"tools\"]\n ]\n }\n return tool_config\n\n def _prepare_request_config(self, kwargs: Dict[str, Any]) -> Dict[str, Any]:\n \"\"\"Prepare the configuration for the Bedrock API request.\"\"\"\n # Convert tools and remove from kwargs\n tool_config = self._convert_tool_spec(kwargs)\n kwargs.pop(\"tools\", None) # Remove tools from kwargs if present\n\n inference_config = {\n key: kwargs[key]\n for key in BedrockConfig.INFERENCE_PARAMETERS\n if key in kwargs\n }\n\n additional_fields = {\n key: value\n for key, value in kwargs.items()\n if key not in BedrockConfig.INFERENCE_PARAMETERS\n }\n\n request_config = {\n \"inferenceConfig\": inference_config,\n \"additionalModelRequestFields\": additional_fields,\n }\n\n if tool_config is not None:\n request_config[\"toolConfig\"] = tool_config\n\n return request_config\n\n def chat_completions_create(\n self, model: str, messages: List[Dict[str, Any]], **kwargs\n ) -> ChatCompletionResponse:\n \"\"\"Create a chat completion request to AWS Bedrock.\"\"\"\n system_message, formatted_messages = self.transformer.convert_request(messages)\n request_config = self._prepare_request_config(kwargs)\n\n try:\n response = self.client.converse(\n modelId=model,\n messages=formatted_messages,\n system=system_message,\n **request_config,\n )\n except botocore.exceptions.ClientError as e:\n if e.response[\"Error\"][\"Code\"] == \"ValidationException\":\n error_message = e.response[\"Error\"][\"Message\"]\n raise LLMError(error_message) from e\n raise\n\n return self.convert_response(response)\n" }, { "path": "aisuite/providers/azure_provider.py", "content": "import urllib.request\nimport json\nimport os\n\nfrom aisuite.provider import Provider\nfrom aisuite.framework import ChatCompletionResponse\nfrom aisuite.framework.message import Message, ChatCompletionMessageToolCall, Function\n\n# Azure provider is based on the documentation here -\n# https://learn.microsoft.com/en-us/azure/machine-learning/reference-model-inference-api?view=azureml-api-2&source=recommendations&tabs=python\n# Azure AI Model Inference API is used.\n# From the documentation -\n# \"\"\"\n# The Azure AI Model Inference is an API that exposes a common set of capabilities for foundational models\n# and that can be used by developers to consume predictions from a diverse set of models in a uniform and consistent way.\n# Developers can talk with different models deployed in Azure AI Foundry portal without changing the underlying code they are using.\n#\n# The Azure AI Model Inference API is available in the following models:\n#\n# Models deployed to serverless API endpoints:\n# Cohere Embed V3 family of models\n# Cohere Command R family of models\n# Meta Llama 2 chat family of models\n# Meta Llama 3 instruct family of models\n# Mistral-Small\n# Mistral-Large\n# Jais family of models\n# Jamba family of models\n# Phi-3 family of models\n#\n# Models deployed to managed inference:\n# Meta Llama 3 instruct family of models\n# Phi-3 family of models\n# Mixtral famility of models\n#\n# The API is compatible with Azure OpenAI model deployments.\n# \"\"\"\n\n\nclass AzureMessageConverter:\n @staticmethod\n def convert_request(messages):\n \"\"\"Convert messages to Azure format.\"\"\"\n transformed_messages = []\n for message in messages:\n if isinstance(message, Message):\n transformed_messages.append(message.model_dump(mode=\"json\"))\n else:\n transformed_messages.append(message)\n return transformed_messages\n\n @staticmethod\n def convert_response(resp_json) -> ChatCompletionResponse:\n \"\"\"Normalize the response from the Azure API to match OpenAI's response format.\"\"\"\n completion_response = ChatCompletionResponse()\n choice = resp_json[\"choices\"][0]\n message = choice[\"message\"]\n\n # Set basic message content\n completion_response.choices[0].message.content = message.get(\"content\")\n completion_response.choices[0].message.role = message.get(\"role\", \"assistant\")\n\n # Handle tool calls if present\n if \"tool_calls\" in message and message[\"tool_calls\"] is not None:\n tool_calls = []\n for tool_call in message[\"tool_calls\"]:\n new_tool_call = ChatCompletionMessageToolCall(\n id=tool_call[\"id\"],\n type=tool_call[\"type\"],\n function={\n \"name\": tool_call[\"function\"][\"name\"],\n \"arguments\": tool_call[\"function\"][\"arguments\"],\n },\n )\n tool_calls.append(new_tool_call)\n completion_response.choices[0].message.tool_calls = tool_calls\n\n return completion_response\n\n\nclass AzureProvider(Provider):\n def __init__(self, **config):\n self.base_url = config.get(\"base_url\") or os.getenv(\"AZURE_BASE_URL\")\n self.api_key = config.get(\"api_key\") or os.getenv(\"AZURE_API_KEY\")\n self.api_version = config.get(\"api_version\") or os.getenv(\"AZURE_API_VERSION\")\n if not self.api_key:\n raise ValueError(\"For Azure, api_key is required.\")\n if not self.base_url:\n raise ValueError(\n \"For Azure, base_url is required. Check your deployment page for a URL like this - https://..models.ai.azure.com\"\n )\n self.transformer = AzureMessageConverter()\n\n def chat_completions_create(self, model, messages, **kwargs):\n url = f\"{self.base_url}/chat/completions\"\n\n if self.api_version:\n url = f\"{url}?api-version={self.api_version}\"\n\n # Remove 'stream' from kwargs if present\n kwargs.pop(\"stream\", None)\n\n # Transform messages using converter\n transformed_messages = self.transformer.convert_request(messages)\n\n # Prepare the request payload\n data = {\"messages\": transformed_messages}\n\n # Add tools if provided\n if \"tools\" in kwargs:\n data[\"tools\"] = kwargs[\"tools\"]\n kwargs.pop(\"tools\")\n\n # Add tool_choice if provided\n if \"tool_choice\" in kwargs:\n data[\"tool_choice\"] = kwargs[\"tool_choice\"]\n kwargs.pop(\"tool_choice\")\n\n # Add remaining kwargs\n data.update(kwargs)\n\n body = json.dumps(data).encode(\"utf-8\")\n headers = {\"Content-Type\": \"application/json\", \"Authorization\": self.api_key}\n\n try:\n req = urllib.request.Request(url, body, headers)\n with urllib.request.urlopen(req) as response:\n result = response.read()\n resp_json = json.loads(result)\n return self.transformer.convert_response(resp_json)\n\n except urllib.error.HTTPError as error:\n error_message = f\"The request failed with status code: {error.code}\\n\"\n error_message += f\"Headers: {error.info()}\\n\"\n error_message += error.read().decode(\"utf-8\", \"ignore\")\n raise Exception(error_message)\n" }, { "path": "aisuite/providers/cerebras_provider.py", "content": "\"\"\"Cerebras provider for the aisuite.\"\"\"\n\nimport cerebras.cloud.sdk as cerebras\nfrom aisuite.provider import Provider, LLMError\nfrom aisuite.providers.message_converter import OpenAICompliantMessageConverter\n\n\nclass CerebrasMessageConverter(OpenAICompliantMessageConverter):\n \"\"\"\n Cerebras-specific message converter if needed.\n \"\"\"\n\n\n# pylint: disable=too-few-public-methods\nclass CerebrasProvider(Provider):\n \"\"\"Provider for Cerebras.\"\"\"\n\n def __init__(self, **config):\n self.client = cerebras.Cerebras(**config)\n self.transformer = CerebrasMessageConverter()\n\n def chat_completions_create(self, model, messages, **kwargs):\n \"\"\"\n Makes a request to the Cerebras chat completions endpoint using the official client.\n \"\"\"\n try:\n response = self.client.chat.completions.create(\n model=model,\n messages=messages,\n **kwargs, # Pass any additional arguments to the Cerebras API.\n )\n return self.transformer.convert_response(response.model_dump())\n\n # Re-raise Cerebras API-specific exceptions.\n except cerebras.PermissionDeniedError:\n raise\n except cerebras.AuthenticationError:\n raise\n except cerebras.RateLimitError:\n raise\n\n # Wrap all other exceptions in LLMError.\n except Exception as e:\n raise LLMError(f\"An error occurred: {e}\") from e\n" }, { "path": "aisuite/providers/cohere_provider.py", "content": "import os\nimport cohere\nimport json\nfrom aisuite.framework import ChatCompletionResponse\nfrom aisuite.framework.message import Message, ChatCompletionMessageToolCall, Function\nfrom aisuite.provider import Provider, LLMError\n\n\nclass CohereMessageConverter:\n \"\"\"\n Cohere-specific message converter\n \"\"\"\n\n def convert_request(self, messages):\n \"\"\"Convert framework messages to Cohere format.\"\"\"\n converted_messages = []\n\n for message in messages:\n if isinstance(message, dict):\n role = message.get(\"role\")\n content = message.get(\"content\")\n tool_calls = message.get(\"tool_calls\")\n tool_plan = message.get(\"tool_plan\")\n else:\n role = message.role\n content = message.content\n tool_calls = message.tool_calls\n tool_plan = getattr(message, \"tool_plan\", None)\n\n # Convert to Cohere's format\n if role == \"tool\":\n # Handle tool response messages\n converted_message = {\n \"role\": role,\n \"tool_call_id\": (\n message.get(\"tool_call_id\")\n if isinstance(message, dict)\n else message.tool_call_id\n ),\n \"content\": self._convert_tool_content(content),\n }\n elif role == \"assistant\" and tool_calls:\n # Handle assistant messages with tool calls\n converted_message = {\n \"role\": role,\n \"tool_calls\": [\n {\n \"id\": tc.id if not isinstance(tc, dict) else tc[\"id\"],\n \"function\": {\n \"name\": (\n tc.function.name\n if not isinstance(tc, dict)\n else tc[\"function\"][\"name\"]\n ),\n \"arguments\": (\n tc.function.arguments\n if not isinstance(tc, dict)\n else tc[\"function\"][\"arguments\"]\n ),\n },\n \"type\": \"function\",\n }\n for tc in tool_calls\n ],\n \"tool_plan\": tool_plan,\n }\n if content:\n converted_message[\"content\"] = content\n else:\n # Handle regular messages\n converted_message = {\"role\": role, \"content\": content}\n\n converted_messages.append(converted_message)\n\n return converted_messages\n\n def _convert_tool_content(self, content):\n \"\"\"Convert tool response content to Cohere's expected format.\"\"\"\n if isinstance(content, str):\n try:\n # Try to parse as JSON first\n data = json.loads(content)\n return [{\"type\": \"document\", \"document\": {\"data\": json.dumps(data)}}]\n except json.JSONDecodeError:\n # If not JSON, return as plain text\n return content\n elif isinstance(content, list):\n # If content is already in Cohere's format, return as is\n return content\n else:\n # For other types, convert to string\n return str(content)\n\n @staticmethod\n def convert_response(response_data) -> ChatCompletionResponse:\n \"\"\"Convert Cohere's response to our standard format.\"\"\"\n normalized_response = ChatCompletionResponse()\n\n # Set usage information\n normalized_response.usage = {\n \"prompt_tokens\": response_data.usage.tokens.input_tokens,\n \"completion_tokens\": response_data.usage.tokens.output_tokens,\n \"total_tokens\": response_data.usage.tokens.input_tokens\n + response_data.usage.tokens.output_tokens,\n }\n\n # Handle tool calls\n if response_data.finish_reason == \"TOOL_CALL\":\n tool_call = response_data.message.tool_calls[0]\n function = Function(\n name=tool_call.function.name, arguments=tool_call.function.arguments\n )\n tool_call_obj = ChatCompletionMessageToolCall(\n id=tool_call.id, function=function, type=\"function\"\n )\n normalized_response.choices[0].message = Message(\n content=response_data.message.tool_plan, # Use tool_plan as content\n tool_calls=[tool_call_obj],\n role=\"assistant\",\n refusal=None,\n )\n normalized_response.choices[0].finish_reason = \"tool_calls\"\n else:\n # Handle regular text response\n normalized_response.choices[0].message.content = (\n response_data.message.content[0].text\n )\n normalized_response.choices[0].finish_reason = \"stop\"\n\n return normalized_response\n\n\nclass CohereProvider(Provider):\n def __init__(self, **config):\n \"\"\"\n Initialize the Cohere provider with the given configuration.\n Pass the entire configuration dictionary to the Cohere client constructor.\n \"\"\"\n # Ensure API key is provided either in config or via environment variable\n config.setdefault(\"api_key\", os.getenv(\"CO_API_KEY\"))\n if not config[\"api_key\"]:\n raise ValueError(\n \"Cohere API key is missing. Please provide it in the config or set the CO_API_KEY environment variable.\"\n )\n self.client = cohere.ClientV2(**config)\n self.transformer = CohereMessageConverter()\n\n def chat_completions_create(self, model, messages, **kwargs):\n \"\"\"\n Makes a request to Cohere using the official client.\n \"\"\"\n try:\n # Transform messages using converter\n transformed_messages = self.transformer.convert_request(messages)\n\n # Make the request to Cohere\n response = self.client.chat(\n model=model, messages=transformed_messages, **kwargs\n )\n\n return self.transformer.convert_response(response)\n except Exception as e:\n raise LLMError(f\"An error occurred: {e}\")\n" }, { "path": "aisuite/providers/deepgram_provider.py", "content": "import os\nimport json\nimport numpy as np\nimport queue\nimport threading\nimport time\nfrom typing import Union, BinaryIO, AsyncGenerator\n\nfrom aisuite.provider import Provider, ASRError, Audio\nfrom aisuite.framework.message import (\n TranscriptionResult,\n Segment,\n Word,\n Alternative,\n Channel,\n StreamingTranscriptionChunk,\n)\n\n\nclass DeepgramProvider(Provider):\n \"\"\"Deepgram ASR provider.\"\"\"\n\n def __init__(self, **config):\n \"\"\"Initialize the Deepgram provider with the given configuration.\"\"\"\n super().__init__()\n\n # Ensure API key is provided either in config or via environment variable\n self.api_key = config.get(\"api_key\") or os.getenv(\"DEEPGRAM_API_KEY\")\n if not self.api_key:\n raise ValueError(\n \"Deepgram API key is missing. Please provide it in the config or set the DEEPGRAM_API_KEY environment variable.\"\n )\n\n # Initialize Deepgram client (v5.0.0+)\n try:\n from deepgram import DeepgramClient\n\n self.client = DeepgramClient(api_key=self.api_key)\n except ImportError:\n raise ImportError(\n \"Deepgram SDK is required. Install it with: pip install deepgram-sdk\"\n )\n\n # Initialize audio functionality\n self.audio = DeepgramAudio(self.client)\n\n def chat_completions_create(self, model, messages):\n \"\"\"Deepgram does not support chat completions.\"\"\"\n raise NotImplementedError(\n \"Deepgram provider only supports audio transcription, not chat completions.\"\n )\n\n\n# Audio Classes\nclass DeepgramAudio(Audio):\n \"\"\"Deepgram Audio functionality container.\"\"\"\n\n def __init__(self, client):\n super().__init__()\n self.transcriptions = self.Transcriptions(client)\n\n class Transcriptions(Audio.Transcription):\n \"\"\"Deepgram Audio Transcriptions functionality.\"\"\"\n\n def __init__(self, client):\n self.client = client\n\n def create(\n self,\n model: str,\n file: Union[str, BinaryIO],\n **kwargs,\n ) -> TranscriptionResult:\n \"\"\"\n Create audio transcription using Deepgram SDK v5.\n\n All parameters are already validated and mapped by the Client layer.\n This is a simple pass-through to the Deepgram API.\n \"\"\"\n try:\n # Add model to params and set defaults\n kwargs[\"model\"] = model\n kwargs.setdefault(\"smart_format\", True)\n kwargs.setdefault(\"punctuate\", True)\n kwargs.setdefault(\"language\", \"en\")\n\n # Get audio bytes\n audio_bytes = self._prepare_audio_payload(file)\n\n # Use v5 API: client.listen.v1.media.transcribe_file()\n # All parameters passed as kwargs, no PrerecordedOptions needed\n response = self.client.listen.v1.media.transcribe_file(\n request=audio_bytes, **kwargs\n )\n\n # Convert Pydantic model to dict (v5 uses Pydantic v2)\n if hasattr(response, \"model_dump\"):\n response_dict = response.model_dump()\n elif hasattr(response, \"to_dict\"):\n response_dict = response.to_dict()\n elif hasattr(response, \"dict\"):\n response_dict = response.dict()\n else:\n response_dict = response\n\n return self._parse_deepgram_response(response_dict)\n\n except Exception as e:\n raise ASRError(f\"Deepgram transcription error: {e}\") from e\n\n async def create_stream_output(\n self,\n model: str,\n file: Union[str, BinaryIO],\n chunk_size_minutes: float = 3.0,\n **kwargs,\n ) -> AsyncGenerator[StreamingTranscriptionChunk, None]:\n \"\"\"\n Create streaming audio transcription using Deepgram SDK v5 with chunked processing.\n\n All parameters are already validated and mapped by the Client layer.\n This implementation handles audio chunking and streaming.\n \"\"\"\n try:\n # Load and prepare audio\n audio_data, sample_rate = await self._load_and_prepare_audio(file)\n\n # Calculate chunking strategy\n duration_seconds = len(audio_data) / sample_rate\n chunk_duration_seconds = chunk_size_minutes * 60\n\n if duration_seconds <= chunk_duration_seconds:\n chunks = [audio_data]\n else:\n chunk_size_samples = int(chunk_duration_seconds * sample_rate)\n chunks = []\n num_chunks = int(np.ceil(duration_seconds / chunk_duration_seconds))\n for i in range(num_chunks):\n start_sample = i * chunk_size_samples\n end_sample = min(\n start_sample + chunk_size_samples, len(audio_data)\n )\n chunks.append(audio_data[start_sample:end_sample])\n\n # Setup API parameters for v5\n kwargs[\"model\"] = model\n kwargs.setdefault(\"smart_format\", \"true\")\n kwargs.setdefault(\"punctuate\", \"true\")\n kwargs.setdefault(\"language\", \"en\")\n kwargs[\"interim_results\"] = (\n \"true\" # Enable interim results for streaming\n )\n\n # Remove parameters not supported by streaming\n kwargs.pop(\"utterances\", None)\n\n # Add critical audio format parameters (as strings for v5)\n kwargs[\"encoding\"] = \"linear16\" # PCM16 format\n kwargs[\"sample_rate\"] = \"16000\" # Match our target sample rate\n kwargs[\"channels\"] = \"1\" # Mono audio\n\n # Use thread-safe queue for cross-thread communication\n transcript_queue = queue.Queue()\n connection_closed = threading.Event()\n\n def on_message(*args, **message_kwargs):\n \"\"\"Handle transcript events\"\"\"\n # Extract result from args or kwargs\n result = None\n if len(args) >= 2:\n result = args[1]\n elif \"result\" in message_kwargs:\n result = message_kwargs[\"result\"]\n else:\n return\n\n if hasattr(result, \"channel\") and result.channel.alternatives:\n alt = result.channel.alternatives[0]\n if alt.transcript:\n chunk = StreamingTranscriptionChunk(\n text=alt.transcript,\n is_final=getattr(result, \"is_final\", False),\n confidence=getattr(alt, \"confidence\", None),\n )\n transcript_queue.put(chunk)\n\n def on_error(*args, **error_kwargs):\n \"\"\"Handle error events\"\"\"\n error = None\n if len(args) >= 2:\n error = args[1]\n elif \"error\" in error_kwargs:\n error = error_kwargs[\"error\"]\n\n if error:\n transcript_queue.put(\n ASRError(f\"Deepgram streaming error: {error}\")\n )\n\n def on_close(*args, **close_kwargs):\n \"\"\"Handle connection close events\"\"\"\n connection_closed.set()\n\n # Use v5 streaming API with context manager\n from deepgram.core.events import EventType\n\n async with self.client.listen.v1.connect(**kwargs) as connection:\n # Register event handlers\n connection.on(EventType.Transcript, on_message)\n connection.on(EventType.Error, on_error)\n connection.on(EventType.Close, on_close)\n\n # Send all chunks through connection\n for audio_chunk in chunks:\n self._send_audio_chunk(connection, audio_chunk)\n\n # Send CloseStream message to signal end\n close_stream_message = json.dumps({\"type\": \"CloseStream\"})\n connection.send(close_stream_message)\n\n # Yield results until connection closes\n while not connection_closed.is_set():\n try:\n chunk = transcript_queue.get(timeout=0.1)\n if isinstance(chunk, Exception):\n raise chunk\n yield chunk\n except queue.Empty:\n continue\n\n # Get any remaining results\n while not transcript_queue.empty():\n try:\n chunk = transcript_queue.get_nowait()\n if isinstance(chunk, Exception):\n raise chunk\n yield chunk\n except queue.Empty:\n break\n\n except Exception as e:\n raise ASRError(f\"Deepgram streaming transcription error: {e}\")\n\n def _prepare_audio_payload(self, file: Union[str, BinaryIO]) -> bytes:\n \"\"\"Prepare audio payload for Deepgram API v5.\n\n Returns raw bytes instead of dict payload (v5 API change).\n \"\"\"\n if isinstance(file, str):\n with open(file, \"rb\") as audio_file:\n buffer_data = audio_file.read()\n else:\n if hasattr(file, \"read\"):\n buffer_data = file.read()\n else:\n raise ValueError(\n \"File must be a file path string or file-like object\"\n )\n return buffer_data\n\n async def _load_and_prepare_audio(\n self, file: Union[str, BinaryIO]\n ) -> tuple[np.ndarray, int]:\n \"\"\"Load and prepare audio file for streaming.\n\n Conversions performed only when necessary:\n - Stereo to mono: Required for multi-channel audio\n - Sample rate conversion: Required when input != 16kHz\n - Other formats: Error out as unsupported\n \"\"\"\n try:\n try:\n import soundfile as sf\n except ImportError:\n raise ASRError(\n \"soundfile is required for audio processing. Install with: pip install soundfile\"\n )\n\n if isinstance(file, str):\n audio_data, original_sample_rate = sf.read(file)\n else:\n audio_data, original_sample_rate = sf.read(file)\n\n audio_data = np.asarray(audio_data, dtype=np.float32)\n\n # Convert to mono if stereo\n if len(audio_data.shape) > 1:\n if audio_data.shape[1] == 2:\n audio_data = np.mean(audio_data, axis=1)\n else:\n raise ASRError(\n f\"Unsupported audio format: {audio_data.shape[1]} channels. Only mono and stereo are supported.\"\n )\n\n # Resample to 16kHz if needed\n target_sample_rate = 16000\n if original_sample_rate != target_sample_rate:\n try:\n from scipy import signal\n\n num_samples = int(\n len(audio_data) * target_sample_rate / original_sample_rate\n )\n audio_data = signal.resample(audio_data, num_samples)\n except ImportError:\n raise ASRError(\n f\"Audio resampling required but scipy not available. \"\n f\"Input is {original_sample_rate}Hz, need {target_sample_rate}Hz. \"\n f\"Install scipy or provide audio at {target_sample_rate}Hz.\"\n )\n\n return np.asarray(audio_data, dtype=np.float32), target_sample_rate\n\n except Exception as e:\n if isinstance(e, ASRError):\n raise\n raise ASRError(f\"Error loading audio file: {e}\")\n\n def _send_audio_chunk(self, connection, audio_chunk: np.ndarray) -> None:\n \"\"\"Send audio chunk data through the connection.\"\"\"\n streaming_chunk_size = 8000 # Match reference BLOCKSIZE (~0.5s @16kHz mono)\n send_delay = 0.01\n\n for i in range(0, len(audio_chunk), streaming_chunk_size):\n piece = audio_chunk[i : i + streaming_chunk_size]\n\n if len(piece) < streaming_chunk_size:\n piece = np.pad(\n piece, (0, streaming_chunk_size - len(piece)), mode=\"constant\"\n )\n\n pcm16 = (piece * 32767).astype(np.int16).tobytes()\n connection.send(pcm16)\n time.sleep(send_delay) # Use synchronous sleep like reference\n\n def _parse_deepgram_response(self, response_dict: dict) -> TranscriptionResult:\n \"\"\"Convert Deepgram API response to unified TranscriptionResult.\"\"\"\n try:\n results = response_dict.get(\"results\", {})\n channels = results.get(\"channels\", [])\n\n if not channels or not channels[0].get(\"alternatives\"):\n return TranscriptionResult(\n text=\"\", language=None, confidence=None, task=\"transcribe\"\n )\n\n best_alternative = channels[0][\"alternatives\"][0]\n text = best_alternative.get(\"transcript\", \"\")\n confidence = best_alternative.get(\"confidence\", None)\n\n words = [\n Word(\n word=word_data.get(\"word\", \"\"),\n start=word_data.get(\"start\", None),\n end=word_data.get(\"end\", None),\n confidence=word_data.get(\"confidence\", None),\n )\n for word_data in best_alternative.get(\"words\", [])\n ]\n\n segments = []\n paragraphs = results.get(\"paragraphs\", {}).get(\"paragraphs\", [])\n for para in paragraphs:\n for sentence in para.get(\"sentences\", []):\n segments.append(\n Segment(\n id=len(segments),\n seek=0,\n start=sentence.get(\"start\", None),\n end=sentence.get(\"end\", None),\n text=sentence.get(\"text\", \"\"),\n tokens=[],\n temperature=0.0,\n avg_logprob=0.0,\n compression_ratio=0.0,\n no_speech_prob=0.0,\n )\n )\n\n alternatives_list = [\n Alternative(\n transcript=alt.get(\"transcript\", \"\"),\n confidence=alt.get(\"confidence\", None),\n )\n for alt in channels[0][\"alternatives\"][1:]\n ]\n\n channels_list = [\n Channel(\n alternatives=[\n Alternative(\n transcript=alt.get(\"transcript\", \"\"),\n confidence=alt.get(\"confidence\", None),\n )\n for alt in channel.get(\"alternatives\", [])\n ]\n )\n for channel in channels\n ]\n\n metadata = response_dict.get(\"metadata\", {})\n\n return TranscriptionResult(\n text=text,\n language=results.get(\"language\", None),\n confidence=confidence,\n task=\"transcribe\",\n duration=metadata.get(\"duration\", None) if metadata else None,\n segments=segments or None,\n words=words or None,\n channels=channels_list or None,\n alternatives=alternatives_list or None,\n utterances=results.get(\"utterances\", []),\n paragraphs=results.get(\"paragraphs\", None),\n topics=results.get(\"topics\", []),\n intents=results.get(\"intents\", []),\n sentiment=results.get(\"sentiment\", None),\n summary=results.get(\"summary\", None),\n metadata=metadata,\n )\n\n except (KeyError, TypeError, IndexError) as e:\n raise ASRError(f\"Error parsing Deepgram response: {e}\")\n" }, { "path": "aisuite/providers/deepseek_provider.py", "content": "\"\"\"Deepseek provider for the aisuite.\"\"\"\n\nimport os\nimport openai\nfrom aisuite.provider import Provider, LLMError\nfrom aisuite.providers.message_converter import OpenAICompliantMessageConverter\n\n\n# pylint: disable=too-few-public-methods\nclass DeepseekProvider(Provider):\n \"\"\"Provider for Deepseek.\"\"\"\n\n def __init__(self, **config):\n \"\"\"\n Initialize the DeepSeek provider with the given configuration.\n Pass the entire configuration dictionary to the OpenAI client constructor.\n \"\"\"\n # Ensure API key is provided either in config or via environment variable\n config.setdefault(\"api_key\", os.getenv(\"DEEPSEEK_API_KEY\"))\n if not config[\"api_key\"]:\n raise ValueError(\n \"DeepSeek API key is missing. Please provide it in the config or \"\n \"set the OPENAI_API_KEY environment variable.\"\n )\n config[\"base_url\"] = \"https://api.deepseek.com\"\n\n # NOTE: We could choose to remove above lines for api_key since OpenAI will automatically\n # infer certain values from the environment variables.\n # Eg: OPENAI_API_KEY, OPENAI_ORG_ID, OPENAI_PROJECT_ID. Except for\n # OPEN_AI_BASE_URL which has to be the deepseek url\n\n # Pass the entire config to the OpenAI client constructor\n self.client = openai.OpenAI(**config)\n # Using OpenAICompliantMessageConverter since DeepSeek's response format is\n # the same as OpenAI's.\n self.transformer = OpenAICompliantMessageConverter()\n\n def chat_completions_create(self, model, messages, **kwargs):\n # Any exception raised by OpenAI will be returned to the caller.\n # Maybe we should catch them and raise a custom LLMError.\n try:\n response = self.client.chat.completions.create(\n model=model,\n messages=messages,\n **kwargs, # Pass any additional arguments to the OpenAI API\n )\n return self.transformer.convert_response(response.model_dump())\n except Exception as e:\n raise LLMError(f\"An error occurred: {e}\") from e\n" }, { "path": "aisuite/providers/fireworks_provider.py", "content": "import os\nimport httpx\nimport json\nfrom aisuite.provider import Provider, LLMError\nfrom aisuite.framework import ChatCompletionResponse\nfrom aisuite.framework.message import Message, ChatCompletionMessageToolCall\n\n\nclass FireworksMessageConverter:\n @staticmethod\n def convert_request(messages):\n \"\"\"Convert messages to Fireworks format.\"\"\"\n transformed_messages = []\n for message in messages:\n if isinstance(message, Message):\n message_dict = message.model_dump(mode=\"json\")\n message_dict.pop(\"refusal\", None) # Remove refusal field if present\n transformed_messages.append(message_dict)\n else:\n transformed_messages.append(message)\n return transformed_messages\n\n @staticmethod\n def convert_response(resp_json) -> ChatCompletionResponse:\n \"\"\"Normalize the response from the Fireworks API to match OpenAI's response format.\"\"\"\n completion_response = ChatCompletionResponse()\n choice = resp_json[\"choices\"][0]\n message = choice[\"message\"]\n\n # Set basic message content\n completion_response.choices[0].message.content = message.get(\"content\")\n completion_response.choices[0].message.role = message.get(\"role\", \"assistant\")\n\n # Handle tool calls if present\n if \"tool_calls\" in message and message[\"tool_calls\"] is not None:\n tool_calls = []\n for tool_call in message[\"tool_calls\"]:\n new_tool_call = ChatCompletionMessageToolCall(\n id=tool_call[\"id\"],\n type=tool_call[\"type\"],\n function={\n \"name\": tool_call[\"function\"][\"name\"],\n \"arguments\": tool_call[\"function\"][\"arguments\"],\n },\n )\n tool_calls.append(new_tool_call)\n completion_response.choices[0].message.tool_calls = tool_calls\n\n return completion_response\n\n\n# Models that support tool calls:\n# [As of 01/20/2025 from https://docs.fireworks.ai/guides/function-calling]\n# Llama 3.1 405B Instruct\n# Llama 3.1 70B Instruct\n# Qwen 2.5 72B Instruct\n# Mixtral MoE 8x22B Instruct\n# Firefunction-v2: Latest and most performant model, optimized for complex function calling scenarios (on-demand only)\n# Firefunction-v1: Previous generation, Mixtral-based function calling model optimized for fast routing and structured output (on-demand only)\nclass FireworksProvider(Provider):\n \"\"\"\n Fireworks AI Provider using httpx for direct API calls.\n \"\"\"\n\n BASE_URL = \"https://api.fireworks.ai/inference/v1/chat/completions\"\n\n def __init__(self, **config):\n \"\"\"\n Initialize the Fireworks provider with the given configuration.\n The API key is fetched from the config or environment variables.\n \"\"\"\n self.api_key = config.get(\"api_key\", os.getenv(\"FIREWORKS_API_KEY\"))\n if not self.api_key:\n raise ValueError(\n \"Fireworks API key is missing. Please provide it in the config or set the FIREWORKS_API_KEY environment variable.\"\n )\n\n # Optionally set a custom timeout (default to 30s)\n self.timeout = config.get(\"timeout\", 30)\n self.transformer = FireworksMessageConverter()\n\n def chat_completions_create(self, model, messages, **kwargs):\n \"\"\"\n Makes a request to the Fireworks AI chat completions endpoint using httpx.\n \"\"\"\n # Remove 'stream' from kwargs if present\n kwargs.pop(\"stream\", None)\n\n # Transform messages using converter\n transformed_messages = self.transformer.convert_request(messages)\n\n # Prepare the request payload\n data = {\n \"model\": model,\n \"messages\": transformed_messages,\n }\n\n # Add tools if provided\n if \"tools\" in kwargs:\n data[\"tools\"] = kwargs[\"tools\"]\n kwargs.pop(\"tools\")\n\n # Add tool_choice if provided\n if \"tool_choice\" in kwargs:\n data[\"tool_choice\"] = kwargs[\"tool_choice\"]\n kwargs.pop(\"tool_choice\")\n\n # Add remaining kwargs\n data.update(kwargs)\n\n headers = {\n \"Authorization\": f\"Bearer {self.api_key}\",\n \"Content-Type\": \"application/json\",\n }\n\n try:\n # Make the request to Fireworks AI endpoint.\n response = httpx.post(\n self.BASE_URL, json=data, headers=headers, timeout=self.timeout\n )\n response.raise_for_status()\n return self.transformer.convert_response(response.json())\n except httpx.HTTPStatusError as error:\n error_message = (\n f\"The request failed with status code: {error.status_code}\\n\"\n )\n error_message += f\"Headers: {error.headers}\\n\"\n error_message += error.response.text\n raise LLMError(error_message)\n except Exception as e:\n raise LLMError(f\"An error occurred: {e}\")\n\n def _normalize_response(self, response_data):\n \"\"\"\n Normalize the response to a common format (ChatCompletionResponse).\n \"\"\"\n normalized_response = ChatCompletionResponse()\n normalized_response.choices[0].message.content = response_data[\"choices\"][0][\n \"message\"\n ][\"content\"]\n return normalized_response\n" }, { "path": "aisuite/providers/google_provider.py", "content": "\"\"\"The interface to Google's Vertex AI.\"\"\"\n\nimport os\nimport json\nfrom typing import List, Dict, Any, Optional, Union, BinaryIO, AsyncGenerator\n\nimport vertexai\nfrom vertexai.generative_models import (\n GenerativeModel,\n GenerationConfig,\n Content,\n Part,\n Tool,\n FunctionDeclaration,\n)\nimport pprint\n\nfrom aisuite.framework import ChatCompletionResponse, Message\nfrom aisuite.framework.message import (\n TranscriptionResult,\n Word,\n Segment,\n Alternative,\n StreamingTranscriptionChunk,\n)\nfrom aisuite.provider import Provider, ASRError, Audio\n\n\nDEFAULT_TEMPERATURE = 0.7\nENABLE_DEBUG_MESSAGES = False\n\n# Links.\n# https://codelabs.developers.google.com/codelabs/gemini-function-calling#6\n# https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/function-calling#chat-samples\n\n\nclass GoogleMessageConverter:\n @staticmethod\n def convert_user_role_message(message: Dict[str, Any]) -> Content:\n \"\"\"Convert user or system messages to Google Vertex AI format.\"\"\"\n parts = [Part.from_text(message[\"content\"])]\n return Content(role=\"user\", parts=parts)\n\n @staticmethod\n def convert_assistant_role_message(message: Dict[str, Any]) -> Content:\n \"\"\"Convert assistant messages to Google Vertex AI format.\"\"\"\n if \"tool_calls\" in message and message[\"tool_calls\"]:\n # Handle function calls\n tool_call = message[\"tool_calls\"][\n 0\n ] # Assuming single function call for now\n function_call = tool_call[\"function\"]\n\n # Create a Part from the function call\n parts = [\n Part.from_dict(\n {\n \"function_call\": {\n \"name\": function_call[\"name\"],\n # \"arguments\": json.loads(function_call[\"arguments\"])\n }\n }\n )\n ]\n # return Content(role=\"function\", parts=parts)\n else:\n # Handle regular text messages\n parts = [Part.from_text(message[\"content\"])]\n # return Content(role=\"model\", parts=parts)\n\n return Content(role=\"model\", parts=parts)\n\n @staticmethod\n def convert_tool_role_message(message: Dict[str, Any]) -> Part:\n \"\"\"Convert tool messages to Google Vertex AI format.\"\"\"\n if \"content\" not in message:\n raise ValueError(\"Tool result message must have a content field\")\n\n try:\n content_json = json.loads(message[\"content\"])\n part = Part.from_function_response(\n name=message[\"name\"], response=content_json\n )\n # TODO: Return Content instead of Part. But returning Content is not working.\n return part\n except json.JSONDecodeError:\n raise ValueError(\"Tool result message must be valid JSON\")\n\n @staticmethod\n def convert_request(messages: List[Dict[str, Any]]) -> List[Content]:\n \"\"\"Convert messages to Google Vertex AI format.\"\"\"\n # Convert all messages to dicts if they're Message objects\n messages = [\n message.model_dump() if hasattr(message, \"model_dump\") else message\n for message in messages\n ]\n\n formatted_messages = []\n for message in messages:\n if message[\"role\"] == \"tool\":\n vertex_message = GoogleMessageConverter.convert_tool_role_message(\n message\n )\n if vertex_message:\n formatted_messages.append(vertex_message)\n elif message[\"role\"] == \"assistant\":\n formatted_messages.append(\n GoogleMessageConverter.convert_assistant_role_message(message)\n )\n else: # user or system role\n formatted_messages.append(\n GoogleMessageConverter.convert_user_role_message(message)\n )\n\n return formatted_messages\n\n @staticmethod\n def convert_response(response) -> ChatCompletionResponse:\n \"\"\"Normalize the response from Vertex AI to match OpenAI's response format.\"\"\"\n openai_response = ChatCompletionResponse()\n\n if ENABLE_DEBUG_MESSAGES:\n print(\"Dumping the response\")\n pprint.pprint(response)\n\n # TODO: We need to go through each part, because function call may not be the first part.\n # Currently, we are only handling the first part, but this is not enough.\n #\n # This is a valid response:\n # candidates {\n # content {\n # role: \"model\"\n # parts {\n # text: \"The current temperature in San Francisco is 72 degrees Celsius. \\n\\n\"\n # }\n # parts {\n # function_call {\n # name: \"is_it_raining\"\n # args {\n # fields {\n # key: \"location\"\n # value {\n # string_value: \"San Francisco\"\n # }\n # }\n # }\n # }\n # }\n # }\n # finish_reason: STOP\n\n # Check if the response contains function calls\n # Note: Just checking if the function_call attribute exists is not enough,\n # it is important to check if the function_call is not None.\n if (\n hasattr(response.candidates[0].content.parts[0], \"function_call\")\n and response.candidates[0].content.parts[0].function_call\n ):\n function_call = response.candidates[0].content.parts[0].function_call\n\n # args is a MapComposite.\n # Convert the MapComposite to a dictionary\n args_dict = {}\n # Another way to try is: args_dict = dict(function_call.args)\n for key, value in function_call.args.items():\n args_dict[key] = value\n if ENABLE_DEBUG_MESSAGES:\n print(\"Dumping the args_dict\")\n pprint.pprint(args_dict)\n\n openai_response.choices[0].message = {\n \"role\": \"assistant\",\n \"content\": None,\n \"tool_calls\": [\n {\n \"type\": \"function\",\n \"id\": f\"call_{hash(function_call.name)}\", # Generate a unique ID\n \"function\": {\n \"name\": function_call.name,\n \"arguments\": json.dumps(args_dict),\n },\n }\n ],\n \"refusal\": None,\n }\n openai_response.choices[0].message = Message(\n **openai_response.choices[0].message\n )\n openai_response.choices[0].finish_reason = \"tool_calls\"\n else:\n # Handle regular text response\n openai_response.choices[0].message.content = (\n response.candidates[0].content.parts[0].text\n )\n openai_response.choices[0].finish_reason = \"stop\"\n\n return openai_response\n\n\nclass GoogleProvider(Provider):\n \"\"\"Implements the ProviderInterface for interacting with Google's Vertex AI.\"\"\"\n\n def __init__(self, **config):\n \"\"\"Set up the Google AI client with a project ID.\"\"\"\n super().__init__()\n\n self.project_id = config.get(\"project_id\") or os.getenv(\"GOOGLE_PROJECT_ID\")\n self.location = config.get(\"region\") or os.getenv(\"GOOGLE_REGION\")\n self.app_creds_path = config.get(\"application_credentials\") or os.getenv(\n \"GOOGLE_APPLICATION_CREDENTIALS\"\n )\n\n if not self.project_id or not self.location or not self.app_creds_path:\n raise EnvironmentError(\n \"Missing one or more required Google environment variables: \"\n \"GOOGLE_PROJECT_ID, GOOGLE_REGION, GOOGLE_APPLICATION_CREDENTIALS. \"\n \"Please refer to the setup guide: /guides/google.md.\"\n )\n\n vertexai.init(project=self.project_id, location=self.location)\n\n self.transformer = GoogleMessageConverter()\n\n # Initialize Speech client lazily\n self._speech_client = None\n\n # Initialize audio functionality\n self.audio = GoogleAudio(self)\n\n def chat_completions_create(self, model, messages, **kwargs):\n \"\"\"Request chat completions from the Google AI API.\n\n Args:\n ----\n model (str): Identifies the specific provider/model to use.\n messages (list of dict): A list of message objects in chat history.\n kwargs (dict): Optional arguments for the Google AI API.\n\n Returns:\n -------\n The ChatCompletionResponse with the completion result.\n\n \"\"\"\n\n # Set the temperature if provided, otherwise use the default\n temperature = kwargs.get(\"temperature\", DEFAULT_TEMPERATURE)\n\n # Convert messages to Vertex AI format\n message_history = self.transformer.convert_request(messages)\n\n # Handle tools if provided\n tools = None\n if \"tools\" in kwargs:\n tools = [\n Tool(\n function_declarations=[\n FunctionDeclaration(\n name=tool[\"function\"][\"name\"],\n description=tool[\"function\"].get(\"description\", \"\"),\n parameters={\n \"type\": \"object\",\n \"properties\": {\n param_name: {\n \"type\": param_info.get(\"type\", \"string\"),\n \"description\": param_info.get(\n \"description\", \"\"\n ),\n **(\n {\"enum\": param_info[\"enum\"]}\n if \"enum\" in param_info\n else {}\n ),\n }\n for param_name, param_info in tool[\"function\"][\n \"parameters\"\n ][\"properties\"].items()\n },\n \"required\": tool[\"function\"][\"parameters\"].get(\n \"required\", []\n ),\n },\n )\n for tool in kwargs[\"tools\"]\n ]\n )\n ]\n\n # Create the GenerativeModel\n model = GenerativeModel(\n model,\n generation_config=GenerationConfig(temperature=temperature),\n tools=tools,\n )\n\n if ENABLE_DEBUG_MESSAGES:\n print(\"Dumping the message_history\")\n pprint.pprint(message_history)\n\n # Start chat and get response\n chat = model.start_chat(history=message_history[:-1])\n last_message = message_history[-1]\n\n # If the last message is a function response, send the Part object directly\n # Otherwise, send just the text content\n message_to_send = (\n Content(role=\"function\", parts=[last_message])\n if isinstance(last_message, Part)\n else last_message.parts[0].text\n )\n # response = chat.send_message(message_to_send)\n response = chat.send_message(message_to_send)\n\n # Convert and return the response\n return self.transformer.convert_response(response)\n\n @property\n def speech_client(self):\n \"\"\"Lazy initialization of Google Cloud Speech client.\"\"\"\n if self._speech_client is None:\n try:\n from google.cloud import speech\n\n self._speech_client = speech.SpeechClient()\n except ImportError:\n raise ImportError(\n \"google-cloud-speech is required for ASR functionality. \"\n \"Install it with: pip install google-cloud-speech\"\n )\n return self._speech_client\n\n\n# Audio Classes\nclass GoogleAudio(Audio):\n \"\"\"Google Audio functionality container.\"\"\"\n\n def __init__(self, provider):\n super().__init__()\n self.provider = provider\n self.transcriptions = self.Transcriptions(provider)\n\n class Transcriptions(Audio.Transcription):\n \"\"\"Google Audio Transcriptions functionality.\"\"\"\n\n def __init__(self, provider):\n self.provider = provider\n\n def create(\n self,\n model: str,\n file: Union[str, BinaryIO],\n **kwargs,\n ) -> TranscriptionResult:\n \"\"\"\n Create audio transcription using Google Cloud Speech-to-Text API.\n\n All parameters are already validated and mapped by the Client layer.\n This is a simple pass-through to the Google API.\n \"\"\"\n try:\n from google.cloud import speech\n\n # Set defaults\n kwargs[\"model\"] = model if model != \"default\" else \"latest_long\"\n kwargs.setdefault(\"sample_rate_hertz\", 16000)\n kwargs.setdefault(\"enable_automatic_punctuation\", True)\n\n audio_data = self._read_audio_data(file)\n audio = speech.RecognitionAudio(content=audio_data)\n config = self._build_recognition_config(kwargs, speech, file)\n\n response = self.provider.speech_client.recognize(\n config=config, audio=audio\n )\n return self._parse_google_response(response)\n\n except ImportError:\n raise ASRError(\n \"google-cloud-speech is required for ASR functionality. \"\n \"Install it with: pip install google-cloud-speech\"\n )\n except Exception as e:\n raise ASRError(f\"Google Speech-to-Text error: {e}\") from e\n\n async def create_stream_output(\n self,\n model: str,\n file: Union[str, BinaryIO],\n **kwargs,\n ) -> AsyncGenerator[StreamingTranscriptionChunk, None]:\n \"\"\"\n Create streaming audio transcription using Google Cloud Speech-to-Text API.\n\n All parameters are already validated and mapped by the Client layer.\n This implementation handles streaming with Google's API.\n \"\"\"\n try:\n from google.cloud import speech\n\n # Set defaults\n kwargs[\"model\"] = model if model != \"default\" else \"latest_long\"\n kwargs.setdefault(\"sample_rate_hertz\", 16000)\n kwargs.setdefault(\"enable_automatic_punctuation\", True)\n\n config = self._build_recognition_config(kwargs, speech, file)\n streaming_config = speech.StreamingRecognitionConfig(\n config=config, interim_results=True, single_utterance=False\n )\n\n audio_data = self._read_audio_data(file)\n request_generator = self._create_streaming_requests(\n speech, streaming_config, audio_data\n )\n\n responses = self.provider.speech_client.streaming_recognize(\n config=streaming_config, requests=request_generator\n )\n\n for response in responses:\n for result in response.results:\n if result.alternatives:\n alternative = result.alternatives[0]\n yield StreamingTranscriptionChunk(\n text=alternative.transcript,\n is_final=result.is_final,\n confidence=getattr(alternative, \"confidence\", None),\n )\n\n except ImportError:\n raise ASRError(\n \"google-cloud-speech is required for ASR functionality. \"\n \"Install it with: pip install google-cloud-speech\"\n )\n except Exception as e:\n raise ASRError(f\"Google Speech-to-Text streaming error: {e}\") from e\n\n def _read_audio_data(self, file: Union[str, BinaryIO]) -> bytes:\n \"\"\"Read audio data from file or file-like object.\"\"\"\n if isinstance(file, str):\n with open(file, \"rb\") as audio_file:\n return audio_file.read()\n else:\n return file.read()\n\n def _detect_audio_encoding(self, file: Union[str, BinaryIO], speech):\n \"\"\"Detect audio encoding based on file extension or content.\"\"\"\n if isinstance(file, str):\n # File path - detect by extension\n file_lower = file.lower()\n if file_lower.endswith(\".mp3\"):\n return speech.RecognitionConfig.AudioEncoding.MP3\n elif file_lower.endswith(\".flac\"):\n return speech.RecognitionConfig.AudioEncoding.FLAC\n elif file_lower.endswith(\".wav\"):\n return speech.RecognitionConfig.AudioEncoding.LINEAR16\n elif file_lower.endswith(\".ogg\"):\n return speech.RecognitionConfig.AudioEncoding.OGG_OPUS\n elif file_lower.endswith(\".webm\"):\n return speech.RecognitionConfig.AudioEncoding.WEBM_OPUS\n\n # Default to LINEAR16 for unknown formats\n return speech.RecognitionConfig.AudioEncoding.LINEAR16\n\n def _build_recognition_config(\n self, params: dict, speech, file: Union[str, BinaryIO]\n ):\n \"\"\"Build Google Speech RecognitionConfig from parameters.\"\"\"\n # Auto-detect encoding if not specified\n encoding = params.get(\"encoding\")\n if encoding is None:\n encoding = self._detect_audio_encoding(file, speech)\n\n config_params = {\n \"encoding\": encoding,\n \"sample_rate_hertz\": params.get(\"sample_rate_hertz\", 16000),\n \"language_code\": params.get(\"language_code\", \"en-US\"),\n \"enable_word_time_offsets\": True,\n \"enable_word_confidence\": True,\n \"enable_automatic_punctuation\": params.get(\n \"enable_automatic_punctuation\", True\n ),\n \"model\": params[\"model\"],\n }\n\n for param in [\"max_alternatives\", \"profanity_filter\", \"speech_contexts\"]:\n if param in params:\n config_params[param] = params[param]\n\n return speech.RecognitionConfig(**config_params)\n\n def _create_streaming_requests(\n self, speech, streaming_config, audio_data: bytes\n ):\n \"\"\"Create streaming requests generator for Google Speech API.\"\"\"\n\n def request_generator():\n chunk_size = 8192\n for i in range(0, len(audio_data), chunk_size):\n chunk = audio_data[i : i + chunk_size]\n yield speech.StreamingRecognizeRequest(audio_content=chunk)\n\n return request_generator()\n\n def _parse_google_response(self, response) -> TranscriptionResult:\n \"\"\"Convert Google Speech-to-Text response to unified TranscriptionResult.\"\"\"\n if not response.results or not response.results[0].alternatives:\n return TranscriptionResult(\n text=\"\", language=None, confidence=None, task=\"transcribe\"\n )\n\n best_result = response.results[0]\n best_alternative = best_result.alternatives[0]\n text = best_alternative.transcript\n confidence = getattr(best_alternative, \"confidence\", None)\n\n words = []\n if hasattr(best_alternative, \"words\") and best_alternative.words:\n words = [\n Word(\n word=word.word,\n start=(\n word.start_time.total_seconds()\n if hasattr(word, \"start_time\")\n else 0.0\n ),\n end=(\n word.end_time.total_seconds()\n if hasattr(word, \"end_time\")\n else 0.0\n ),\n confidence=getattr(word, \"confidence\", None),\n )\n for word in best_alternative.words\n ]\n\n alternatives = [\n Alternative(\n transcript=alt.transcript,\n confidence=getattr(alt, \"confidence\", None),\n )\n for alt in best_result.alternatives\n ]\n\n segments = []\n if words:\n segments = [\n Segment(\n id=0,\n seek=0,\n start=words[0].start,\n end=words[-1].end,\n text=text,\n tokens=[],\n temperature=0.0,\n avg_logprob=0.0,\n compression_ratio=0.0,\n no_speech_prob=0.0,\n )\n ]\n\n return TranscriptionResult(\n text=text,\n language=None,\n confidence=confidence,\n task=\"transcribe\",\n words=words or None,\n alternatives=alternatives or None,\n segments=segments or None,\n )\n" }, { "path": "aisuite/providers/groq_provider.py", "content": "import os\nimport groq\nfrom aisuite.provider import Provider, LLMError\nfrom aisuite.providers.message_converter import OpenAICompliantMessageConverter\n\n# Implementation of Groq provider.\n# Groq's message format is same as OpenAI's.\n# Tool calling specification is also exactly the same as OpenAI's.\n# Links:\n# https://console.groq.com/docs/tool-use\n# Groq supports tool calling for the following models, as of 16th Nov 2024:\n# llama3-groq-70b-8192-tool-use-preview\n# llama3-groq-8b-8192-tool-use-preview\n# llama-3.1-70b-versatile\n# llama-3.1-8b-instant\n# llama3-70b-8192\n# llama3-8b-8192\n# mixtral-8x7b-32768 (parallel tool use not supported)\n# gemma-7b-it (parallel tool use not supported)\n# gemma2-9b-it (parallel tool use not supported)\n\n\nclass GroqMessageConverter(OpenAICompliantMessageConverter):\n \"\"\"\n Groq-specific message converter if needed\n \"\"\"\n\n pass\n\n\nclass GroqProvider(Provider):\n def __init__(self, **config):\n \"\"\"\n Initialize the Groq provider with the given configuration.\n Pass the entire configuration dictionary to the Groq client constructor.\n \"\"\"\n # Ensure API key is provided either in config or via environment variable\n self.api_key = config.get(\"api_key\", os.getenv(\"GROQ_API_KEY\"))\n if not self.api_key:\n raise ValueError(\n \"Groq API key is missing. Please provide it in the config or set the GROQ_API_KEY environment variable.\"\n )\n config[\"api_key\"] = self.api_key\n self.client = groq.Groq(**config)\n self.transformer = GroqMessageConverter()\n\n def chat_completions_create(self, model, messages, **kwargs):\n \"\"\"\n Makes a request to the Groq chat completions endpoint using the official client.\n \"\"\"\n try:\n # Transform messages using converter\n transformed_messages = self.transformer.convert_request(messages)\n\n response = self.client.chat.completions.create(\n model=model,\n messages=transformed_messages,\n **kwargs, # Pass any additional arguments to the Groq API\n )\n return self.transformer.convert_response(response.model_dump())\n except Exception as e:\n raise LLMError(f\"An error occurred: {e}\")\n" }, { "path": "aisuite/providers/huggingface_provider.py", "content": "import os\nimport json\nimport time\nfrom typing import Union, BinaryIO\nimport requests\nfrom huggingface_hub import InferenceClient\nfrom aisuite.provider import Provider, LLMError, ASRError, Audio\nfrom aisuite.framework import ChatCompletionResponse\nfrom aisuite.framework.message import Message, TranscriptionResult, Word\n\n\nclass HuggingfaceProvider(Provider):\n \"\"\"\n HuggingFace Provider using the official InferenceClient.\n This provider supports calls to HF serverless Inference Endpoints\n which use Text Generation Inference (TGI) as the backend.\n TGI is OpenAI protocol compliant.\n https://huggingface.co/inference-endpoints/\n \"\"\"\n\n def __init__(self, **config):\n \"\"\"\n Initialize the provider with the given configuration.\n The token is fetched from the config or environment variables.\n \"\"\"\n # Ensure API key is provided either in config or via environment variable\n self.token = (\n config.get(\"token\")\n or os.getenv(\"HF_TOKEN\")\n or os.getenv(\"HUGGINGFACE_API_KEY\")\n )\n if not self.token:\n raise ValueError(\n \"Hugging Face token is missing. Please provide it in the config or set the HF_TOKEN or HUGGINGFACE_API_KEY environment variable.\"\n )\n\n # Initialize the InferenceClient with the specified model and timeout if provided\n self.model = config.get(\"model\")\n self.timeout = config.get(\"timeout\", 30)\n self.client = InferenceClient(\n token=self.token, model=self.model, timeout=self.timeout\n )\n\n # Initialize audio functionality\n super().__init__()\n self.audio = HuggingfaceAudio(self.token, self.timeout)\n\n def chat_completions_create(self, model, messages, **kwargs):\n \"\"\"\n Makes a request to the Inference API endpoint using InferenceClient.\n \"\"\"\n # Validate and transform messages\n transformed_messages = []\n for message in messages:\n if isinstance(message, Message):\n transformed_message = self.transform_from_message(message)\n elif isinstance(message, dict):\n transformed_message = message\n else:\n raise ValueError(f\"Invalid message format: {message}\")\n\n # Ensure 'content' is a non-empty string\n if (\n \"content\" not in transformed_message\n or transformed_message[\"content\"] is None\n ):\n transformed_message[\"content\"] = \"\"\n\n transformed_messages.append(transformed_message)\n\n try:\n # Prepare the payload\n payload = {\n \"messages\": transformed_messages,\n **kwargs, # Include other parameters like temperature, max_tokens, etc.\n }\n\n # Make the API call using the client\n response = self.client.chat_completion(model=model, **payload)\n\n return self._normalize_response(response)\n\n except Exception as e:\n raise LLMError(f\"An error occurred: {e}\")\n\n def transform_from_message(self, message: Message):\n \"\"\"Transform framework Message to a format that HuggingFace understands.\"\"\"\n # Ensure content is a string\n content = message.content if message.content is not None else \"\"\n\n # Transform the message\n transformed_message = {\n \"role\": message.role,\n \"content\": content,\n }\n\n # Include tool_calls if present\n if message.tool_calls:\n transformed_message[\"tool_calls\"] = [\n {\n \"id\": tool_call.id,\n \"function\": {\n \"name\": tool_call.function.name,\n \"arguments\": tool_call.function.arguments,\n },\n \"type\": tool_call.type,\n }\n for tool_call in message.tool_calls\n ]\n\n return transformed_message\n\n def transform_to_message(self, message_dict: dict):\n \"\"\"Transform HuggingFace message (dict) to a format that the framework Message understands.\"\"\"\n # Ensure required fields are present\n message_dict.setdefault(\"content\", \"\") # Set empty string if content is missing\n message_dict.setdefault(\"refusal\", None) # Set None if refusal is missing\n message_dict.setdefault(\"tool_calls\", None) # Set None if tool_calls is missing\n\n # Handle tool calls if present and not None\n if message_dict.get(\"tool_calls\"):\n for tool_call in message_dict[\"tool_calls\"]:\n if \"function\" in tool_call:\n # Ensure function arguments are stringified\n if isinstance(tool_call[\"function\"].get(\"arguments\"), dict):\n tool_call[\"function\"][\"arguments\"] = json.dumps(\n tool_call[\"function\"][\"arguments\"]\n )\n\n return Message(**message_dict)\n\n def _normalize_response(self, response_data):\n \"\"\"\n Normalize the response to a common format (ChatCompletionResponse).\n \"\"\"\n normalized_response = ChatCompletionResponse()\n message_data = response_data[\"choices\"][0][\"message\"]\n normalized_response.choices[0].message = self.transform_to_message(message_data)\n return normalized_response\n\n\n# Audio Classes\nclass HuggingfaceAudio(Audio):\n \"\"\"Hugging Face Audio functionality container.\"\"\"\n\n def __init__(self, token, timeout=120):\n super().__init__()\n self.transcriptions = self.Transcriptions(token, timeout)\n\n class Transcriptions(Audio.Transcription):\n \"\"\"Hugging Face Audio Transcriptions functionality.\"\"\"\n\n def __init__(self, token, timeout=120):\n self.token = token\n self.timeout = timeout\n\n def create(\n self,\n model: str,\n file: Union[str, BinaryIO],\n **kwargs,\n ) -> TranscriptionResult:\n \"\"\"\n Create audio transcription using Hugging Face Inference API.\n\n All parameters are already validated and mapped by the Client layer.\n This makes an HTTP POST request to the Hugging Face Inference API.\n\n Note: Whisper-based models have a 30-second processing window.\n For longer audio, users should deploy custom Inference Endpoints.\n \"\"\"\n try:\n # Extract model ID from format \"huggingface:model-id\"\n model_id = model.split(\":\", 1)[1] if \":\" in model else model\n\n # Prepare API endpoint\n url = f\"https://api-inference.huggingface.co/models/{model_id}\"\n\n # Prepare audio data\n if isinstance(file, str):\n with open(file, \"rb\") as audio_file:\n audio_bytes = audio_file.read()\n content_type = self._detect_content_type(file)\n else:\n audio_bytes = file.read()\n # Default to wav for file-like objects\n content_type = \"audio/wav\"\n\n # Prepare headers\n headers = {\n \"Authorization\": f\"Bearer {self.token}\",\n \"Content-Type\": content_type,\n }\n\n # First attempt without wait_for_model\n try:\n response = requests.post(\n url,\n headers=headers,\n data=audio_bytes,\n timeout=self.timeout,\n )\n response.raise_for_status()\n except requests.exceptions.HTTPError as e:\n # If 503 (model loading), retry with x-wait-for-model header\n if e.response.status_code == 503:\n headers[\"x-wait-for-model\"] = \"true\"\n response = requests.post(\n url,\n headers=headers,\n data=audio_bytes,\n timeout=self.timeout,\n )\n response.raise_for_status()\n else:\n raise\n\n # Parse response\n response_data = response.json()\n return self._parse_huggingface_response(response_data, model_id)\n\n except requests.exceptions.RequestException as e:\n raise ASRError(f\"Hugging Face transcription error: {e}\") from e\n except Exception as e:\n raise ASRError(f\"Hugging Face transcription error: {e}\") from e\n\n def _detect_content_type(self, file_path: str) -> str:\n \"\"\"Detect audio content type from file extension.\"\"\"\n if file_path.lower().endswith(\".wav\"):\n return \"audio/wav\"\n elif file_path.lower().endswith(\".mp3\"):\n return \"audio/mpeg\" # HF API requires audio/mpeg for MP3\n elif file_path.lower().endswith(\".flac\"):\n return \"audio/flac\"\n else:\n # Default to wav if unknown\n return \"audio/wav\"\n\n def _parse_huggingface_response(\n self, response_data, model_id: str\n ) -> TranscriptionResult:\n \"\"\"\n Parse Hugging Face API response into TranscriptionResult.\n\n Response format can vary:\n - Standard: {\"text\": \"...\", \"chunks\": [...]}\n - Text only: {\"text\": \"...\"}\n - Some models may use different keys\n \"\"\"\n try:\n # Extract text\n if isinstance(response_data, dict):\n text = response_data.get(\"text\", \"\")\n elif isinstance(response_data, str):\n # Some models return plain string\n text = response_data\n else:\n text = str(response_data)\n\n # Extract words from chunks if available\n words = None\n if isinstance(response_data, dict) and \"chunks\" in response_data:\n chunks = response_data[\"chunks\"]\n if chunks:\n words = []\n for chunk in chunks:\n if isinstance(chunk, dict):\n word_text = chunk.get(\"text\", \"\")\n timestamp = chunk.get(\"timestamp\")\n\n # timestamp can be [start, end] or (start, end)\n start, end = None, None\n if timestamp and len(timestamp) >= 2:\n start, end = timestamp[0], timestamp[1]\n\n words.append(\n Word(\n word=word_text,\n start=start,\n end=end,\n confidence=None, # HF doesn't provide confidence\n )\n )\n\n return TranscriptionResult(\n text=text,\n language=None, # HF API doesn't return language\n confidence=None, # HF API doesn't return confidence\n words=words,\n task=\"transcribe\",\n )\n\n except (KeyError, TypeError, IndexError) as e:\n raise ASRError(f\"Error parsing Hugging Face response: {e}\")\n" }, { "path": "aisuite/providers/inception_provider.py", "content": "import openai\nimport os\nfrom aisuite.provider import Provider, LLMError\n\n\nclass InceptionProvider(Provider):\n def __init__(self, **config):\n \"\"\"\n Initialize the Inception provider with the given configuration.\n Pass the entire configuration dictionary to the Inception client constructor using openai.\n \"\"\"\n # Ensure API key is provided either in config or via environment variable\n config.setdefault(\"api_key\", os.getenv(\"INCEPTION_API_KEY\"))\n if not config[\"api_key\"]:\n raise ValueError(\n \"Inception API key is missing. Please provide it in the config or set the INCEPTION_API_KEY environment variable.\"\n )\n config[\"base_url\"] = \"https://api.inceptionlabs.ai/v1\"\n\n # Pass the entire config to the Inception client constructor using openai\n self.client = openai.OpenAI(**config)\n\n def chat_completions_create(self, model, messages, **kwargs):\n # Any exception raised by Inception will be returned to the caller.\n # Maybe we should catch them and raise a custom LLMError.\n try:\n response = self.client.chat.completions.create(\n model=model,\n messages=messages,\n **kwargs, # Pass any additional arguments to the Inception API\n )\n return response\n except Exception as e:\n raise LLMError(f\"An error occurred: {e}\")\n" }, { "path": "aisuite/providers/lmstudio_provider.py", "content": "import os\nimport httpx\nfrom aisuite.provider import Provider, LLMError\nfrom aisuite.framework import ChatCompletionResponse\n\n\nclass LmstudioProvider(Provider):\n \"\"\"\n LM Studio Provider that makes HTTP calls. Inspired by OllamaProvider in aisuite.\n It uses the /v1/chat/completions endpoint.\n Read more here - https://lmstudio.ai/docs/api and on your local instance in the \"Developer\" tab.\n If LMSTUDIO_API_URL is not set and not passed in config, then it will default to \"http://localhost:1234\"\n \"\"\"\n\n _CHAT_COMPLETION_ENDPOINT = \"/v1/chat/completions\"\n _CONNECT_ERROR_MESSAGE = \"LM Studio is likely not running. Start LM Studio by running `ollama serve` on your host.\"\n\n def __init__(self, **config):\n \"\"\"\n Initialize the LM Studio provider with the given configuration.\n \"\"\"\n self.url = config.get(\"api_url\") or os.getenv(\n \"LMSTUDIO_API_URL\", \"http://localhost:1234\"\n )\n\n # Optionally set a custom timeout (default to 300s)\n self.timeout = config.get(\"timeout\", 300)\n\n def chat_completions_create(self, model, messages, **kwargs):\n \"\"\"\n Makes a request to the chat completions endpoint using httpx.\n \"\"\"\n kwargs[\"stream\"] = False\n data = {\n \"model\": model,\n \"messages\": messages,\n **kwargs, # Pass any additional arguments to the API\n }\n\n try:\n response = httpx.post(\n self.url.rstrip(\"/\") + self._CHAT_COMPLETION_ENDPOINT,\n json=data,\n timeout=self.timeout,\n )\n response.raise_for_status()\n except httpx.ConnectError: # Handle connection errors\n raise LLMError(f\"Connection failed: {self._CONNECT_ERROR_MESSAGE}\")\n except httpx.HTTPStatusError as http_err:\n raise LLMError(f\"LM Studio request failed: {http_err}\")\n except Exception as e:\n raise LLMError(f\"An error occurred: {e}\")\n\n # Return the normalized response\n return self._normalize_response(response.json())\n\n def _normalize_response(self, response_data):\n \"\"\"\n Normalize the API response to a common format (ChatCompletionResponse).\n \"\"\"\n normalized_response = ChatCompletionResponse()\n normalized_response.choices[0].message.content = response_data[\"choices\"][0][\n \"message\"\n ][\"content\"]\n\n return normalized_response\n" }, { "path": "aisuite/providers/message_converter.py", "content": "\"\"\"Base message converter for OpenAI-compliant providers.\"\"\"\n\nfrom aisuite.framework import ChatCompletionResponse\nfrom aisuite.framework.message import (\n Message,\n ChatCompletionMessageToolCall,\n CompletionUsage,\n)\n\n\nclass OpenAICompliantMessageConverter:\n \"\"\"\n Base class for message converters that are compatible with OpenAI's API.\n \"\"\"\n\n # Class variable that derived classes can override\n tool_results_as_strings = False\n\n @staticmethod\n def convert_request(messages):\n \"\"\"Convert messages to OpenAI-compatible format.\"\"\"\n transformed_messages = []\n for message in messages:\n tmsg = None\n if isinstance(message, Message):\n message_dict = message.model_dump(mode=\"json\")\n message_dict.pop(\"refusal\", None) # Remove refusal field if present\n tmsg = message_dict\n else:\n tmsg = message\n # Check if tmsg is a dict, otherwise get role attribute\n role = tmsg[\"role\"] if isinstance(tmsg, dict) else tmsg.role\n if role == \"tool\":\n if OpenAICompliantMessageConverter.tool_results_as_strings:\n # Handle both dict and object cases for content\n if isinstance(tmsg, dict):\n tmsg[\"content\"] = str(tmsg[\"content\"])\n else:\n tmsg.content = str(tmsg.content)\n\n transformed_messages.append(tmsg)\n return transformed_messages\n\n def convert_response(self, response_data) -> ChatCompletionResponse:\n \"\"\"Normalize the response to match OpenAI's response format.\"\"\"\n completion_response = ChatCompletionResponse()\n choice = response_data[\"choices\"][0]\n message = choice[\"message\"]\n\n # Set basic message content\n completion_response.choices[0].message.content = message[\"content\"]\n completion_response.choices[0].message.role = message.get(\"role\", \"assistant\")\n # Conditionally parse usage data if it exists.\n if usage_data := response_data.get(\"usage\"):\n completion_response.usage = self.get_completion_usage(usage_data)\n\n # Handle tool calls if present\n if \"tool_calls\" in message and message[\"tool_calls\"] is not None:\n tool_calls = []\n for tool_call in message[\"tool_calls\"]:\n tool_calls.append(\n ChatCompletionMessageToolCall(\n id=tool_call.get(\"id\"),\n type=\"function\", # Always set to \"function\" as it's the only valid value\n function=tool_call.get(\"function\"),\n )\n )\n completion_response.choices[0].message.tool_calls = tool_calls\n\n return completion_response\n\n def get_completion_usage(self, usage_data: dict):\n \"\"\"Get the usage statistics from a usage data dictionary.\"\"\"\n return CompletionUsage(\n completion_tokens=usage_data.get(\"completion_tokens\"),\n prompt_tokens=usage_data.get(\"prompt_tokens\"),\n total_tokens=usage_data.get(\"total_tokens\"),\n prompt_tokens_details=usage_data.get(\"prompt_tokens_details\"),\n completion_tokens_details=usage_data.get(\"completion_tokens_details\"),\n )\n" }, { "path": "aisuite/providers/mistral_provider.py", "content": "\"\"\"Mistral provider for the aisuite.\"\"\"\n\nimport os\nfrom mistralai import Mistral\nfrom aisuite.framework import ChatCompletionResponse\nfrom aisuite.provider import Provider, LLMError\nfrom aisuite.providers.message_converter import OpenAICompliantMessageConverter\n\n\n# Implementation of Mistral provider.\n# Mistral's message format is the same as OpenAI's. Just different class names,\n# but fully cross-compatible.\n# Links:\n# https://docs.mistral.ai/capabilities/function_calling/\n\n\nclass MistralMessageConverter(OpenAICompliantMessageConverter):\n \"\"\"\n Mistral-specific message converter\n \"\"\"\n\n def convert_response(self, response_data) -> ChatCompletionResponse:\n \"\"\"Convert Mistral's response to our standard format.\"\"\"\n # Convert Mistral's response object to dict format\n response_dict = response_data.model_dump()\n return super().convert_response(response_dict)\n\n\n# Function calling is available for the following models:\n# [As of 01/19/2025 from https://docs.mistral.ai/capabilities/function_calling/]\n# Mistral Large\n# Mistral Small\n# Codestral 22B\n# Ministral 8B\n# Ministral 3B\n# Pixtral 12B\n# Mixtral 8x22B\n# Mistral Nemo\n# pylint: disable=too-few-public-methods\nclass MistralProvider(Provider):\n \"\"\"\n Mistral AI Provider using the official Mistral client.\n \"\"\"\n\n def __init__(self, **config):\n \"\"\"\n Initialize the Mistral provider with the given configuration.\n Pass the entire configuration dictionary to the Mistral client constructor.\n \"\"\"\n # Ensure API key is provided either in config or via environment variable\n config.setdefault(\"api_key\", os.getenv(\"MISTRAL_API_KEY\"))\n if not config[\"api_key\"]:\n raise ValueError(\n \"Mistral API key is missing. Please provide it in the config or set the \"\n \"MISTRAL_API_KEY environment variable.\"\n )\n self.client = Mistral(**config)\n self.transformer = MistralMessageConverter()\n\n def chat_completions_create(self, model, messages, **kwargs):\n \"\"\"\n Makes a request to Mistral using the official client.\n \"\"\"\n try:\n # Transform messages using converter\n transformed_messages = self.transformer.convert_request(messages)\n\n # Make the request to Mistral\n response = self.client.chat.complete(\n model=model, messages=transformed_messages, **kwargs\n )\n\n return self.transformer.convert_response(response)\n except Exception as e:\n raise LLMError(f\"An error occurred: {e}\") from e\n" }, { "path": "aisuite/providers/nebius_provider.py", "content": "import os\nfrom aisuite.provider import Provider\nfrom openai import Client\n\n\nBASE_URL = \"https://api.studio.nebius.ai/v1\"\n\n\n# TODO(rohitcp): This needs to be added to our internal testbed. Tool calling not tested.\nclass NebiusProvider(Provider):\n def __init__(self, **config):\n \"\"\"\n Initialize the Nebius AI Studio provider with the given configuration.\n Pass the entire configuration dictionary to the OpenAI client constructor.\n \"\"\"\n # Ensure API key is provided either in config or via environment variable\n config.setdefault(\"api_key\", os.getenv(\"NEBIUS_API_KEY\"))\n if not config[\"api_key\"]:\n raise ValueError(\n \"Nebius AI Studio API key is missing. Please provide it in the config or set the NEBIUS_API_KEY environment variable. You can get your API key at https://studio.nebius.ai/settings/api-keys\"\n )\n\n config[\"base_url\"] = BASE_URL\n # Pass the entire config to the OpenAI client constructor\n self.client = Client(**config)\n\n def chat_completions_create(self, model, messages, **kwargs):\n return self.client.chat.completions.create(\n model=model,\n messages=messages,\n **kwargs # Pass any additional arguments to the Nebius API\n )\n" }, { "path": "aisuite/providers/ollama_provider.py", "content": "import os\nimport httpx\nfrom aisuite.provider import Provider, LLMError\nfrom aisuite.framework import ChatCompletionResponse\n\n\nclass OllamaProvider(Provider):\n \"\"\"\n Ollama Provider that makes HTTP calls instead of using SDK.\n It uses the /api/chat endpoint.\n Read more here - https://github.com/ollama/ollama/blob/main/docs/api.md#generate-a-chat-completion\n If OLLAMA_API_URL is not set and not passed in config, then it will default to \"http://localhost:11434\"\n \"\"\"\n\n _CHAT_COMPLETION_ENDPOINT = \"/api/chat\"\n _CONNECT_ERROR_MESSAGE = \"Ollama is likely not running. Start Ollama by running `ollama serve` on your host.\"\n\n def __init__(self, **config):\n \"\"\"\n Initialize the Ollama provider with the given configuration.\n \"\"\"\n self.url = config.get(\"api_url\") or os.getenv(\n \"OLLAMA_API_URL\", \"http://localhost:11434\"\n )\n\n # Optionally set a custom timeout (default to 30s)\n self.timeout = config.get(\"timeout\", 30)\n\n def chat_completions_create(self, model, messages, **kwargs):\n \"\"\"\n Makes a request to the chat completions endpoint using httpx.\n \"\"\"\n kwargs[\"stream\"] = False\n data = {\n \"model\": model,\n \"messages\": messages,\n **kwargs, # Pass any additional arguments to the API\n }\n\n try:\n response = httpx.post(\n self.url.rstrip(\"/\") + self._CHAT_COMPLETION_ENDPOINT,\n json=data,\n timeout=self.timeout,\n )\n response.raise_for_status()\n except httpx.ConnectError: # Handle connection errors\n raise LLMError(f\"Connection failed: {self._CONNECT_ERROR_MESSAGE}\")\n except httpx.HTTPStatusError as http_err:\n raise LLMError(f\"Ollama request failed: {http_err}\")\n except Exception as e:\n raise LLMError(f\"An error occurred: {e}\")\n\n # Return the normalized response\n return self._normalize_response(response.json())\n\n def _normalize_response(self, response_data):\n \"\"\"\n Normalize the API response to a common format (ChatCompletionResponse).\n \"\"\"\n normalized_response = ChatCompletionResponse()\n normalized_response.choices[0].message.content = response_data[\"message\"][\n \"content\"\n ]\n return normalized_response\n" }, { "path": "aisuite/providers/openai_provider.py", "content": "import openai\nimport os\nfrom typing import Union, BinaryIO, AsyncGenerator\nfrom aisuite.provider import Provider, LLMError, ASRError, Audio\nfrom aisuite.providers.message_converter import OpenAICompliantMessageConverter\nfrom aisuite.framework.message import (\n TranscriptionResult,\n Segment,\n Word,\n StreamingTranscriptionChunk,\n)\n\n\nclass OpenaiProvider(Provider):\n def __init__(self, **config):\n \"\"\"\n Initialize the OpenAI provider with the given configuration.\n Pass the entire configuration dictionary to the OpenAI client constructor.\n \"\"\"\n # Ensure API key is provided either in config or via environment variable\n config.setdefault(\"api_key\", os.getenv(\"OPENAI_API_KEY\"))\n if not config[\"api_key\"]:\n raise ValueError(\n \"OpenAI API key is missing. Please provide it in the config or set the OPENAI_API_KEY environment variable.\"\n )\n\n # NOTE: We could choose to remove above lines for api_key since OpenAI will automatically\n # infer certain values from the environment variables.\n # Eg: OPENAI_API_KEY, OPENAI_ORG_ID, OPENAI_PROJECT_ID, OPENAI_BASE_URL, etc.\n\n # Pass the entire config to the OpenAI client constructor\n self.client = openai.OpenAI(**config)\n self.transformer = OpenAICompliantMessageConverter()\n\n # Initialize audio functionality\n super().__init__()\n self.audio = OpenAIAudio(self.client)\n\n def chat_completions_create(self, model, messages, **kwargs):\n # Any exception raised by OpenAI will be returned to the caller.\n # Maybe we should catch them and raise a custom LLMError.\n try:\n transformed_messages = self.transformer.convert_request(messages)\n response = self.client.chat.completions.create(\n model=model,\n messages=transformed_messages,\n **kwargs, # Pass any additional arguments to the OpenAI API\n )\n return response\n except Exception as e:\n raise LLMError(f\"An error occurred: {e}\")\n\n\n# Audio Classes\nclass OpenAIAudio(Audio):\n \"\"\"OpenAI Audio functionality container.\"\"\"\n\n def __init__(self, client):\n super().__init__()\n self.transcriptions = self.Transcriptions(client)\n\n class Transcriptions(Audio.Transcription):\n \"\"\"OpenAI Audio Transcriptions functionality.\"\"\"\n\n def __init__(self, client):\n self.client = client\n\n def create(\n self,\n model: str,\n file: Union[str, BinaryIO],\n **kwargs,\n ) -> TranscriptionResult:\n \"\"\"\n Create audio transcription using OpenAI Whisper API.\n\n All parameters are already validated and mapped by the Client layer.\n This is a simple pass-through to the OpenAI API.\n \"\"\"\n try:\n # Handle TranscriptionOptions object if passed\n if \"options\" in kwargs:\n options = kwargs.pop(\"options\")\n # Extract all non-None attributes from options object\n if hasattr(options, \"__dict__\"):\n for key, value in options.__dict__.items():\n if value is not None and key not in kwargs:\n kwargs[key] = value\n\n # Handle timestamp_granularities requirement\n if \"timestamp_granularities\" in kwargs:\n # OpenAI requires verbose_json format for timestamp_granularities\n kwargs[\"response_format\"] = \"verbose_json\"\n\n # Handle file input\n if isinstance(file, str):\n with open(file, \"rb\") as audio_file:\n response = self.client.audio.transcriptions.create(\n file=audio_file, model=model, **kwargs\n )\n else:\n response = self.client.audio.transcriptions.create(\n file=file, model=model, **kwargs\n )\n\n return self._parse_openai_response(response)\n\n except Exception as e:\n raise ASRError(f\"OpenAI transcription error: {e}\") from e\n\n async def create_stream_output(\n self,\n model: str,\n file: Union[str, BinaryIO],\n **kwargs,\n ) -> AsyncGenerator[StreamingTranscriptionChunk, None]:\n \"\"\"\n Create streaming audio transcription using OpenAI Whisper API.\n\n All parameters are already validated and mapped by the Client layer.\n This is a simple pass-through to the OpenAI API with streaming enabled.\n \"\"\"\n try:\n # Handle TranscriptionOptions object if passed\n if \"options\" in kwargs:\n options = kwargs.pop(\"options\")\n # Extract all non-None attributes from options object\n if hasattr(options, \"__dict__\"):\n for key, value in options.__dict__.items():\n if value is not None and key not in kwargs:\n kwargs[key] = value\n\n # Enable streaming\n kwargs[\"stream\"] = True\n\n # Handle timestamp_granularities requirement\n if \"timestamp_granularities\" in kwargs:\n # OpenAI requires verbose_json format for timestamp_granularities\n if (\n \"response_format\" in kwargs\n and kwargs[\"response_format\"] != \"verbose_json\"\n ):\n raise ASRError(\n f\"OpenAI timestamp_granularities requires response_format='verbose_json', \"\n f\"but got '{kwargs['response_format']}'. \"\n f\"Either remove timestamp_granularities or use response_format='verbose_json'.\"\n )\n else:\n kwargs[\"response_format\"] = \"verbose_json\"\n\n try:\n if isinstance(file, str):\n with open(file, \"rb\") as audio_file:\n response_stream = self.client.audio.transcriptions.create(\n file=audio_file, model=model, **kwargs\n )\n else:\n response_stream = self.client.audio.transcriptions.create(\n file=file, model=model, **kwargs\n )\n\n # Process streaming response - handle event types\n for event in response_stream:\n # Handle TranscriptionTextDeltaEvent (incremental text)\n if (\n hasattr(event, \"type\")\n and event.type == \"transcript.text.delta\"\n ):\n if hasattr(event, \"delta\") and event.delta:\n yield StreamingTranscriptionChunk(\n text=event.delta,\n is_final=False, # Delta events are interim\n confidence=getattr(event, \"confidence\", None),\n )\n # Handle TranscriptionTextDoneEvent (final complete text)\n elif (\n hasattr(event, \"type\")\n and event.type == \"transcript.text.done\"\n ):\n if hasattr(event, \"text\") and event.text:\n yield StreamingTranscriptionChunk(\n text=event.text,\n is_final=True, # Done event is final\n confidence=getattr(event, \"confidence\", None),\n )\n\n except Exception as stream_error:\n raise ASRError(\n f\"OpenAI streaming transcription error: {stream_error}\"\n ) from stream_error\n\n except Exception as e:\n raise ASRError(f\"OpenAI streaming transcription error: {e}\") from e\n\n def _parse_openai_response(self, response) -> TranscriptionResult:\n \"\"\"Parse OpenAI API response into TranscriptionResult.\"\"\"\n text = response.text if hasattr(response, \"text\") else \"\"\n language = getattr(response, \"language\", \"unknown\")\n\n # Parse segments if available\n segments = []\n if hasattr(response, \"segments\") and response.segments:\n for seg in response.segments:\n words = []\n if hasattr(seg, \"words\") and seg.words:\n for word in seg.words:\n words.append(\n Word(\n word=word.word,\n start=word.start,\n end=word.end,\n confidence=getattr(word, \"confidence\", None),\n )\n )\n\n segments.append(\n Segment(\n id=getattr(seg, \"id\", 0),\n seek=getattr(seg, \"seek\", 0),\n text=seg.text,\n start=seg.start,\n end=seg.end,\n words=words,\n confidence=getattr(seg, \"avg_logprob\", None),\n )\n )\n\n return TranscriptionResult(\n text=text,\n language=language,\n confidence=getattr(response, \"confidence\", None),\n segments=segments,\n )\n" }, { "path": "aisuite/providers/sambanova_provider.py", "content": "import os\nfrom aisuite.provider import Provider, LLMError\nfrom openai import OpenAI\nfrom aisuite.providers.message_converter import OpenAICompliantMessageConverter\n\n\nclass SambanovaMessageConverter(OpenAICompliantMessageConverter):\n \"\"\"\n SambaNova-specific message converter.\n \"\"\"\n\n pass\n\n\nclass SambanovaProvider(Provider):\n \"\"\"\n SambaNova Provider using OpenAI client for API calls.\n \"\"\"\n\n def __init__(self, **config):\n \"\"\"\n Initialize the SambaNova provider with the given configuration.\n Pass the entire configuration dictionary to the OpenAI client constructor.\n \"\"\"\n # Ensure API key is provided either in config or via environment variable\n self.api_key = config.get(\"api_key\", os.getenv(\"SAMBANOVA_API_KEY\"))\n if not self.api_key:\n raise ValueError(\n \"Sambanova API key is missing. Please provide it in the config or set the SAMBANOVA_API_KEY environment variable.\"\n )\n\n config[\"api_key\"] = self.api_key\n config[\"base_url\"] = \"https://api.sambanova.ai/v1/\"\n # Pass the entire config to the OpenAI client constructor\n self.client = OpenAI(**config)\n self.transformer = SambanovaMessageConverter()\n\n def chat_completions_create(self, model, messages, **kwargs):\n \"\"\"\n Makes a request to the SambaNova chat completions endpoint using the OpenAI client.\n \"\"\"\n try:\n # Transform messages using converter\n transformed_messages = self.transformer.convert_request(messages)\n\n response = self.client.chat.completions.create(\n model=model,\n messages=transformed_messages,\n **kwargs, # Pass any additional arguments to the Sambanova API\n )\n return self.transformer.convert_response(response.model_dump())\n except Exception as e:\n raise LLMError(f\"An error occurred: {e}\")\n" }, { "path": "aisuite/providers/together_provider.py", "content": "import os\nimport httpx\nfrom aisuite.provider import Provider, LLMError\nfrom aisuite.providers.message_converter import OpenAICompliantMessageConverter\n\n\nclass TogetherMessageConverter(OpenAICompliantMessageConverter):\n \"\"\"\n Together-specific message converter if needed\n \"\"\"\n\n pass\n\n\nclass TogetherProvider(Provider):\n \"\"\"\n Together AI Provider using httpx for direct API calls.\n \"\"\"\n\n BASE_URL = \"https://api.together.xyz/v1/chat/completions\"\n\n def __init__(self, **config):\n \"\"\"\n Initialize the Together provider with the given configuration.\n The API key is fetched from the config or environment variables.\n \"\"\"\n self.api_key = config.get(\"api_key\", os.getenv(\"TOGETHER_API_KEY\"))\n if not self.api_key:\n raise ValueError(\n \"Together API key is missing. Please provide it in the config or set the TOGETHER_API_KEY environment variable.\"\n )\n\n # Optionally set a custom timeout (default to 30s)\n self.timeout = config.get(\"timeout\", 30)\n self.transformer = TogetherMessageConverter()\n\n def chat_completions_create(self, model, messages, **kwargs):\n \"\"\"\n Makes a request to the Together AI chat completions endpoint using httpx.\n \"\"\"\n # Transform messages using converter\n transformed_messages = self.transformer.convert_request(messages)\n\n headers = {\n \"Authorization\": f\"Bearer {self.api_key}\",\n \"Content-Type\": \"application/json\",\n }\n\n data = {\n \"model\": model,\n \"messages\": transformed_messages,\n **kwargs, # Pass any additional arguments to the API\n }\n\n try:\n # Make the request to Together AI endpoint.\n response = httpx.post(\n self.BASE_URL, json=data, headers=headers, timeout=self.timeout\n )\n response.raise_for_status()\n return self.transformer.convert_response(response.json())\n except httpx.HTTPStatusError as http_err:\n raise LLMError(f\"Together AI request failed: {http_err}\")\n except Exception as e:\n raise LLMError(f\"An error occurred: {e}\")\n" }, { "path": "aisuite/providers/watsonx_provider.py", "content": "from aisuite.provider import Provider\nimport os\nfrom ibm_watsonx_ai import Credentials\nfrom ibm_watsonx_ai.foundation_models import ModelInference\nfrom aisuite.framework import ChatCompletionResponse\n\n\nclass WatsonxProvider(Provider):\n def __init__(self, **config):\n self.service_url = config.get(\"service_url\") or os.getenv(\"WATSONX_SERVICE_URL\")\n self.api_key = config.get(\"api_key\") or os.getenv(\"WATSONX_API_KEY\")\n self.project_id = config.get(\"project_id\") or os.getenv(\"WATSONX_PROJECT_ID\")\n\n if not self.service_url or not self.api_key or not self.project_id:\n raise EnvironmentError(\n \"Missing one or more required WatsonX environment variables: \"\n \"WATSONX_SERVICE_URL, WATSONX_API_KEY, WATSONX_PROJECT_ID. \"\n \"Please refer to the setup guide: /guides/watsonx.md.\"\n )\n\n def chat_completions_create(self, model, messages, **kwargs):\n model = ModelInference(\n model_id=model,\n credentials=Credentials(\n api_key=self.api_key,\n url=self.service_url,\n ),\n project_id=self.project_id,\n )\n\n res = model.chat(messages=messages, params=kwargs)\n return self.normalize_response(res)\n\n def normalize_response(self, response):\n openai_response = ChatCompletionResponse()\n openai_response.choices[0].message.content = response[\"choices\"][0][\"message\"][\n \"content\"\n ]\n return openai_response\n" }, { "path": "aisuite/providers/xai_provider.py", "content": "import os\nimport httpx\nfrom aisuite.provider import Provider, LLMError\nfrom aisuite.framework import ChatCompletionResponse\nfrom aisuite.providers.message_converter import OpenAICompliantMessageConverter\n\n\nclass XaiMessageConverter(OpenAICompliantMessageConverter):\n \"\"\"\n xAI-specific message converter if needed\n \"\"\"\n\n pass\n\n\nclass XaiProvider(Provider):\n \"\"\"\n xAI Provider using httpx for direct API calls.\n \"\"\"\n\n BASE_URL = \"https://api.x.ai/v1/chat/completions\"\n\n def __init__(self, **config):\n \"\"\"\n Initialize the xAI provider with the given configuration.\n The API key is fetched from the config or environment variables.\n \"\"\"\n self.api_key = config.get(\"api_key\", os.getenv(\"XAI_API_KEY\"))\n if not self.api_key:\n raise ValueError(\n \"xAI API key is missing. Please provide it in the config or set the XAI_API_KEY environment variable.\"\n )\n\n # Optionally set a custom timeout (default to 30s)\n self.timeout = config.get(\"timeout\", 30)\n self.transformer = XaiMessageConverter()\n\n def chat_completions_create(self, model, messages, **kwargs):\n \"\"\"\n Makes a request to the xAI chat completions endpoint using httpx.\n \"\"\"\n # Transform messages using converter\n transformed_messages = self.transformer.convert_request(messages)\n\n headers = {\n \"Authorization\": f\"Bearer {self.api_key}\",\n \"Content-Type\": \"application/json\",\n }\n\n data = {\n \"model\": model,\n \"messages\": transformed_messages,\n **kwargs, # Pass any additional arguments to the API\n }\n\n try:\n # Make the request to xAI endpoint.\n response = httpx.post(\n self.BASE_URL, json=data, headers=headers, timeout=self.timeout\n )\n response.raise_for_status()\n return self.transformer.convert_response(response.json())\n except httpx.HTTPStatusError as http_err:\n raise LLMError(f\"xAI request failed: {http_err}\")\n except Exception as e:\n raise LLMError(f\"An error occurred: {e}\")\n" }, { "path": "aisuite/utils/tools.py", "content": "from typing import Callable, Dict, Any, Type, Optional, get_origin, get_args, Union\nfrom pydantic import BaseModel, create_model, Field, ValidationError\nimport inspect\nimport json\nfrom docstring_parser import parse\n\n\nclass Tools:\n def __init__(self, tools: list[Callable] = None):\n self._tools = {}\n if tools:\n for tool in tools:\n self._add_tool(tool)\n\n # Add a tool function with or without a Pydantic model.\n def _add_tool(self, func: Callable, param_model: Optional[Type[BaseModel]] = None):\n \"\"\"Register a tool function with metadata. If no param_model is provided, infer from function signature.\"\"\"\n # Check if this is an MCP tool with original schema\n if hasattr(func, \"__mcp_input_schema__\") and func.__mcp_input_schema__:\n # Use the original MCP schema directly to preserve all JSON Schema details\n tool_spec = self._convert_mcp_schema_to_tool_spec(func)\n # Create Pydantic model from MCP schema for validation\n param_model = self._create_pydantic_model_from_mcp_schema(func)\n elif param_model:\n tool_spec = self._convert_to_tool_spec(func, param_model)\n else:\n tool_spec, param_model = self.__infer_from_signature(func)\n\n self._tools[func.__name__] = {\n \"function\": func,\n \"param_model\": param_model,\n \"spec\": tool_spec,\n }\n\n # Return tools in the specified format (default OpenAI).\n def tools(self, format=\"openai\") -> list:\n \"\"\"Return tools in the specified format (default OpenAI).\"\"\"\n if format == \"openai\":\n return self.__convert_to_openai_format()\n return [tool[\"spec\"] for tool in self._tools.values()]\n\n def _unwrap_optional(self, field_type: Type) -> tuple[Type, bool]:\n \"\"\"\n Unwrap Optional[T] to get the base type T.\n\n Returns:\n tuple: (base_type, is_optional)\n \"\"\"\n # Check if it's Optional (Union with None)\n origin = get_origin(field_type)\n if origin is Union:\n args = get_args(field_type)\n # Optional[T] is Union[T, None]\n if type(None) in args:\n # Get the non-None type\n non_none_types = [arg for arg in args if arg is not type(None)]\n if len(non_none_types) == 1:\n return non_none_types[0], True\n return field_type, False\n\n # Convert the function and its Pydantic model to a unified tool specification.\n def _convert_to_tool_spec(\n self, func: Callable, param_model: Type[BaseModel]\n ) -> Dict[str, Any]:\n \"\"\"Convert the function and its Pydantic model to a unified tool specification.\"\"\"\n type_mapping = {str: \"string\", int: \"integer\", float: \"number\", bool: \"boolean\"}\n\n properties = {}\n for field_name, field in param_model.model_fields.items():\n field_type = field.annotation\n\n # Unwrap Optional[T] to get base type T\n field_type, is_optional = self._unwrap_optional(field_type)\n\n # Handle enum types\n if hasattr(field_type, \"__members__\"): # Check if it's an enum\n enum_values = [\n member.value if hasattr(member, \"value\") else member.name\n for member in field_type\n ]\n properties[field_name] = {\n \"type\": \"string\",\n \"enum\": enum_values,\n \"description\": field.description or \"\",\n }\n # Convert enum default value to string if it exists\n if str(field.default) != \"PydanticUndefined\":\n properties[field_name][\"default\"] = (\n field.default.value\n if hasattr(field.default, \"value\")\n else field.default\n )\n else:\n properties[field_name] = {\n \"type\": type_mapping.get(field_type, str(field_type)),\n \"description\": field.description or \"\",\n }\n # Add default if it exists and isn't PydanticUndefined\n if str(field.default) != \"PydanticUndefined\":\n properties[field_name][\"default\"] = field.default\n\n return {\n \"name\": func.__name__,\n \"description\": func.__doc__ or \"\",\n \"parameters\": {\n \"type\": \"object\",\n \"properties\": properties,\n \"required\": [\n name\n for name, field in param_model.model_fields.items()\n if field.is_required and str(field.default) == \"PydanticUndefined\"\n ],\n },\n }\n\n def __extract_param_descriptions(self, func: Callable) -> dict[str, str]:\n \"\"\"Extract parameter descriptions from function docstring.\n\n Args:\n func: The function to extract parameter descriptions from\n\n Returns:\n Dictionary mapping parameter names to their descriptions\n \"\"\"\n docstring = inspect.getdoc(func) or \"\"\n parsed_docstring = parse(docstring)\n\n param_descriptions = {}\n for param in parsed_docstring.params:\n param_descriptions[param.arg_name] = param.description or \"\"\n\n return param_descriptions\n\n def _convert_mcp_schema_to_tool_spec(self, func: Callable) -> Dict[str, Any]:\n \"\"\"\n Convert MCP tool with original inputSchema to tool spec.\n\n This preserves the original JSON Schema from MCP without round-trip conversion,\n avoiding information loss for complex types like arrays and nested objects.\n\n Args:\n func: MCP tool wrapper with __mcp_input_schema__ attribute\n\n Returns:\n Tool specification compatible with OpenAI format\n \"\"\"\n input_schema = func.__mcp_input_schema__\n\n return {\n \"name\": func.__name__,\n \"description\": func.__doc__ or \"\",\n \"parameters\": input_schema, # Use original schema directly!\n }\n\n def _create_pydantic_model_from_mcp_schema(self, func: Callable) -> Type[BaseModel]:\n \"\"\"\n Create a Pydantic model from MCP inputSchema for parameter validation.\n\n This is needed for the execute() method to validate tool call arguments.\n\n Args:\n func: MCP tool wrapper with __mcp_input_schema__ attribute\n\n Returns:\n Pydantic model for parameter validation\n \"\"\"\n from ..mcp.schema_converter import mcp_schema_to_annotations\n\n input_schema = func.__mcp_input_schema__\n properties = input_schema.get(\"properties\", {})\n required = input_schema.get(\"required\", [])\n\n # Get type annotations from MCP schema\n annotations = mcp_schema_to_annotations(input_schema)\n\n fields = {}\n for param_name, param_type in annotations.items():\n param_schema = properties.get(param_name, {})\n description = param_schema.get(\"description\", \"\")\n\n if param_name in required:\n fields[param_name] = (param_type, Field(..., description=description))\n else:\n fields[param_name] = (\n param_type,\n Field(default=None, description=description),\n )\n\n return create_model(f\"{func.__name__.capitalize()}Params\", **fields)\n\n def __infer_from_signature(\n self, func: Callable\n ) -> tuple[Dict[str, Any], Type[BaseModel]]:\n \"\"\"Infer parameters(required and optional) and requirements directly from the function signature.\"\"\"\n signature = inspect.signature(func)\n fields = {}\n required_fields = []\n\n # Get function's docstring and parse parameter descriptions\n param_descriptions = self.__extract_param_descriptions(func)\n docstring = inspect.getdoc(func) or \"\"\n\n # Parse the docstring to get the main function description\n parsed_docstring = parse(docstring)\n function_description = parsed_docstring.short_description or \"\"\n if parsed_docstring.long_description:\n function_description += \"\\n\\n\" + parsed_docstring.long_description\n\n for param_name, param in signature.parameters.items():\n # Check if a type annotation is missing\n if param.annotation == inspect._empty:\n raise TypeError(\n f\"Parameter '{param_name}' in function '{func.__name__}' must have a type annotation.\"\n )\n\n # Determine field type and optionality\n param_type = param.annotation\n description = param_descriptions.get(param_name, \"\")\n\n if param.default == inspect._empty:\n fields[param_name] = (param_type, Field(..., description=description))\n required_fields.append(param_name)\n else:\n fields[param_name] = (\n param_type,\n Field(default=param.default, description=description),\n )\n\n # Dynamically create a Pydantic model based on inferred fields\n param_model = create_model(f\"{func.__name__.capitalize()}Params\", **fields)\n\n # Convert inferred model to a tool spec format\n tool_spec = self._convert_to_tool_spec(func, param_model)\n\n # Update the tool spec with the parsed function description instead of raw docstring\n tool_spec[\"description\"] = function_description\n\n return tool_spec, param_model\n\n def __convert_to_openai_format(self) -> list:\n \"\"\"Convert tools to OpenAI's format.\"\"\"\n return [\n {\"type\": \"function\", \"function\": tool[\"spec\"]}\n for tool in self._tools.values()\n ]\n\n def results_to_messages(self, results: list, message: any) -> list:\n \"\"\"Converts results to messages.\"\"\"\n # if message is empty return empty list\n if not message or len(results) == 0:\n return []\n\n messages = []\n # Iterate over results and match with tool calls from the message\n for result in results:\n # Find matching tool call from message.tool_calls\n for tool_call in message.tool_calls:\n if tool_call.id == result[\"tool_call_id\"]:\n messages.append(\n {\n \"role\": \"tool\",\n \"name\": result[\"name\"],\n \"content\": json.dumps(result[\"content\"]),\n \"tool_call_id\": tool_call.id,\n }\n )\n break\n\n return messages\n\n def execute(self, tool_calls) -> list:\n \"\"\"Executes registered tools based on the tool calls from the model.\n\n Args:\n tool_calls: List of tool calls from the model\n\n Returns:\n List of results from executing each tool call\n \"\"\"\n results = []\n\n # Handle single tool call or list of tool calls\n if not isinstance(tool_calls, list):\n tool_calls = [tool_calls]\n\n for tool_call in tool_calls:\n # Handle both dictionary and object-style tool calls\n if isinstance(tool_call, dict):\n tool_name = tool_call[\"function\"][\"name\"]\n arguments = tool_call[\"function\"][\"arguments\"]\n else:\n tool_name = tool_call.function.name\n arguments = tool_call.function.arguments\n\n # Ensure arguments is a dict\n if isinstance(arguments, str):\n arguments = json.loads(arguments)\n\n if tool_name not in self._tools:\n raise ValueError(f\"Tool '{tool_name}' not registered.\")\n\n tool = self._tools[tool_name]\n tool_func = tool[\"function\"]\n param_model = tool[\"param_model\"]\n\n # Validate and parse the arguments with Pydantic if a model exists\n try:\n validated_args = param_model(**arguments)\n result = tool_func(**validated_args.model_dump())\n results.append(result)\n except ValidationError as e:\n raise ValueError(f\"Error in tool '{tool_name}' parameters: {e}\")\n\n return results\n\n def execute_tool(self, tool_calls) -> tuple[list, list]:\n \"\"\"Executes registered tools based on the tool calls from the model.\n\n Args:\n tool_calls: List of tool calls from the model\n\n Returns:\n List of tuples containing (result, result_message) for each tool call\n \"\"\"\n results = []\n messages = []\n\n # Handle single tool call or list of tool calls\n if not isinstance(tool_calls, list):\n tool_calls = [tool_calls]\n\n for tool_call in tool_calls:\n # Handle both dictionary and object-style tool calls\n if isinstance(tool_call, dict):\n tool_name = tool_call[\"function\"][\"name\"]\n arguments = tool_call[\"function\"][\"arguments\"]\n tool_call_id = tool_call[\"id\"]\n else:\n tool_name = tool_call.function.name\n arguments = tool_call.function.arguments\n tool_call_id = tool_call.id\n\n # Ensure arguments is a dict\n if isinstance(arguments, str):\n arguments = json.loads(arguments)\n\n if tool_name not in self._tools:\n raise ValueError(f\"Tool '{tool_name}' not registered.\")\n\n tool = self._tools[tool_name]\n tool_func = tool[\"function\"]\n param_model = tool[\"param_model\"]\n\n # Validate and parse the arguments with Pydantic if a model exists\n try:\n validated_args = param_model(**arguments)\n result = tool_func(**validated_args.model_dump())\n results.append(result)\n messages.append(\n {\n \"role\": \"tool\",\n \"name\": tool_name,\n \"content\": json.dumps(result),\n \"tool_call_id\": tool_call_id,\n }\n )\n except ValidationError as e:\n raise ValueError(f\"Error in tool '{tool_name}' parameters: {e}\")\n\n return results, messages\n" }, { "path": "aisuite/utils/utils.py", "content": "\"\"\"Utility functions for aisuite.\"\"\"\n\nimport json\nfrom unittest.mock import MagicMock\nfrom pydantic import BaseModel\n\n\n# pylint: disable=too-few-public-methods\nclass Utils:\n \"\"\"\n Utility functions for debugging and inspecting objects.\n \"\"\"\n\n @staticmethod\n def spew(obj):\n \"\"\"\n Recursively inspects a Python object and prints its contents as a\n nicely formatted JSON string. Handles Pydantic models, nested objects,\n lists, and circular references.\n \"\"\"\n visited = set()\n\n # pylint: disable=too-many-return-statements\n def default_encoder(o):\n # Handle MagicMock objects to prevent circular reference errors in tests\n if isinstance(o, MagicMock):\n try:\n # Attempt to get a descriptive name for the mock\n # pylint: disable=protected-access\n name = o._extract_mock_name()\n # pylint: disable=broad-exception-caught\n except Exception:\n name = \"unknown\"\n return f''\n\n # Handle other circular references\n obj_id = id(o)\n if obj_id in visited:\n return f\"\"\n visited.add(obj_id)\n\n # Handle Pydantic models\n if isinstance(o, BaseModel):\n return o.model_dump()\n\n # Handle general objects by converting their __dict__\n if hasattr(o, \"__dict__\"):\n return o.__dict__\n\n # Handle sets\n if isinstance(o, set):\n return list(o)\n\n # Fallback for other types\n try:\n return str(o)\n # pylint: disable=broad-exception-caught\n except Exception:\n return f\"\"\n\n print(json.dumps(obj, default=default_encoder, indent=2))\n" }, { "path": "aisuite-js/README.md", "content": "# AISuite\n\nAISuite is a unified TypeScript library that provides a single, consistent interface for interacting with multiple Large Language Model (LLM) providers. The library uses OpenAI's API format as the standard interface while supporting OpenAI and Anthropic Claude.\n\nnpm pacakge - `npm i aisuite`\n\n## Features\n\n- **Unified API**: Single interface compatible with OpenAI's API structure\n- **Multi-Provider Support**: Currently supports OpenAI and Anthropic\n- **Provider Selection**: Use `provider:model` format (e.g., `openai:gpt-4o`, `anthropic:claude-3-haiku-20240307`)\n- **Tool Calling**: Transparent tool/function calling across all providers\n- **Streaming**: Real-time streaming responses with consistent API\n- **Type Safety**: Full TypeScript support with comprehensive type definitions\n- **Error Handling**: Unified error handling across providers\n- **Speech-to-Text**: Automatic Speech Recognition (ASR) support with multiple providers (OpenAI Whisper, Deepgram)\n\n## Installation\n\n```bash\nnpm install aisuite\n```\n\n## Quick Start\n\n```typescript\nimport { Client } from 'aisuite';\n\nconst client = new Client({\n openai: { \n apiKey: process.env.OPENAI_API_KEY, \n },\n anthropic: { apiKey: process.env.ANTHROPIC_API_KEY },\n deepgram: { apiKey: process.env.DEEPGRAM_API_KEY },\n});\n\n// Use any provider with identical interface\nconst response = await client.chat.completions.create({\n model: 'openai:gpt-4o',\n messages: [\n { role: 'system', content: 'You are a helpful assistant.' },\n { role: 'user', content: 'Hello!' }\n ],\n});\n\nconsole.log(response.choices[0].message.content);\n```\n\n## Usage Examples\n\n### Basic Chat Completion\n\n```typescript\n// OpenAI\nconst openaiResponse = await client.chat.completions.create({\n model: 'openai:gpt-4o',\n messages: [\n { role: 'system', content: 'You are a helpful assistant.' },\n { role: 'user', content: 'What is TypeScript?' }\n ],\n temperature: 0.7,\n max_tokens: 1000,\n});\n\n// Anthropic - exact same interface\nconst anthropicResponse = await client.chat.completions.create({\n model: 'anthropic:claude-3-haiku-20240307',\n messages: [\n { role: 'system', content: 'You are a helpful assistant.' },\n { role: 'user', content: 'What is TypeScript?' }\n ],\n temperature: 0.7,\n max_tokens: 1000,\n});\n```\n\n### Tool/Function Calling\n\n```typescript\nconst tools = [\n {\n type: 'function' as const,\n function: {\n name: 'get_weather',\n description: 'Get current weather for a location',\n parameters: {\n type: 'object',\n properties: {\n location: { type: 'string', description: 'City name' }\n },\n required: ['location']\n }\n }\n }\n];\n\n// Works identically across all providers\nconst response = await client.chat.completions.create({\n model: 'anthropic:claude-3-haiku-20240307',\n messages: [{ role: 'user', content: 'What\\'s the weather in NYC?' }],\n tools,\n tool_choice: 'auto'\n});\n\nif (response.choices[0].message.tool_calls) {\n console.log('Tool calls:', response.choices[0].message.tool_calls);\n}\n```\n\n### Streaming Responses\n\n```typescript\nconst stream = await client.chat.completions.create({\n model: 'openai:gpt-4o',\n messages: [{ role: 'user', content: 'Tell me a story' }],\n stream: true\n});\n\n// TypeScript: cast to AsyncIterable\nfor await (const chunk of stream as AsyncIterable) {\n process.stdout.write(chunk.choices[0]?.delta?.content || '');\n}\n```\n\n### Streaming with Abort Controller\n\n```typescript\nconst controller = new AbortController();\n\n// Abort after 5 seconds\nsetTimeout(() => controller.abort(), 5000);\n\nconst stream = await client.chat.completions.create({\n model: 'anthropic:claude-3-haiku-20240307',\n messages: [{ role: 'user', content: 'Write a long story' }],\n stream: true\n}, { signal: controller.signal });\n\ntry {\n for await (const chunk of stream as AsyncIterable) {\n process.stdout.write(chunk.choices[0]?.delta?.content || '');\n }\n} catch (error) {\n if (error.name === 'AbortError') {\n console.log('Stream aborted');\n }\n}\n```\n\n### Speech-to-Text Transcription\n\n```typescript\n// Initialize client with audio support for OpenAI\nconst client = new Client({\n openai: { \n apiKey: process.env.OPENAI_API_KEY, \n },\n deepgram: { apiKey: process.env.DEEPGRAM_API_KEY }\n});\n\n// Using Deepgram\nconst deepgramResponse = await client.audio.transcriptions.create({\n model: 'deepgram:nova-2',\n file: audioBuffer, // Buffer containing audio data\n language: 'en-US',\n timestamps: true,\n word_confidence: true,\n speaker_labels: true,\n});\n\n// Using OpenAI Whisper\nconst openaiResponse = await client.audio.transcriptions.create({\n model: 'openai:whisper-1',\n file: audioBuffer,\n language: 'en',\n response_format: 'verbose_json',\n temperature: 0,\n timestamps: true,\n});\n\nconsole.log('Transcribed Text:', openaiResponse.text);\nconsole.log('Words with timestamps:', openaiResponse.words);\n```\n\n### Error Handling\n\n```typescript\nimport { AISuiteError, ProviderNotConfiguredError } from 'aisuite';\n\ntry {\n const response = await client.chat.completions.create({\n model: 'invalid:model',\n messages: [{ role: 'user', content: 'Hello' }]\n });\n} catch (error) {\n if (error instanceof ProviderNotConfiguredError) {\n console.error('Provider not configured:', error.message);\n } else if (error instanceof AISuiteError) {\n console.error('AISuite error:', error.message, error.provider);\n } else {\n console.error('Unknown error:', error);\n }\n}\n```\n\n## API Reference\n\n### Client Configuration\n\n```typescript\nconst client = new Client({\n openai?: {\n apiKey: string;\n baseURL?: string;\n organization?: string; \n },\n anthropic?: {\n apiKey: string;\n baseURL?: string;\n },\n deepgram?: {\n apiKey: string;\n baseURL?: string;\n }\n});\n```\n\n### Chat Completion Request\n\nAll providers use the standard OpenAI chat completion format:\n\n```typescript\ninterface ChatCompletionRequest {\n model: string; // \"provider:model\" format\n messages: ChatMessage[];\n tools?: Tool[];\n tool_choice?: ToolChoice;\n temperature?: number;\n max_tokens?: number;\n stop?: string | string[];\n stream?: boolean;\n}\n```\n\n### Transcription Request\n\nAll ASR providers use a standard transcription request format with additional provider-specific parameters:\n\n```typescript\ninterface TranscriptionRequest {\n model: string; // \"provider:model\" format\n file: Buffer; // Audio file as Buffer\n language?: string; // Language code (e.g., \"en\", \"en-US\")\n timestamps?: boolean; // Include word-level timestamps\n [key: string]: any; // Additional provider-specific parameters:\n // For OpenAI: See https://platform.openai.com/docs/api-reference/audio/createTranscription\n // For Deepgram: See https://developers.deepgram.com/reference/speech-to-text-api/listen \n}\n```\n\n### Helper Methods\n\n```typescript\n// List all configured providers (including ASR)\nclient.listProviders(); // ['openai', 'anthropic']\nclient.listASRProviders(); // ['deepgram', 'openai']\n\n// Check if a provider is configured\nclient.isProviderConfigured('openai'); // true\nclient.isASRProviderConfigured('deepgram'); // true\n```\n\n## Current Limitations\n\n- Only OpenAI and Anthropic providers are currently supported for chat (Gemini, Mistral, and Bedrock coming soon)\n- Tool calling requires handling tool responses manually\n- Streaming tool calls require manual accumulation of arguments\n- ASR support is limited to OpenAI Whisper (requires explicit audio configuration) and Deepgram\n- Some provider-specific ASR features might require using provider-specific parameters\n\n## Development\n\n```bash\n# Install dependencies\nnpm install\n\n# Build the project\nnpm run build\n\n# Run tests\nnpm test\n\n# Run examples\n#Run basic usage example only:\nnpm run example:basic\n# Run tool calling example only:\nnpm run example:tools\n# Run the full test suite:\nnpm run test:examples\n```\n\n## License\n\nMIT\n" }, { "path": "aisuite-js/examples/basic-usage.ts", "content": "import 'dotenv/config';\nimport { Client } from '../src';\n\nasync function main() {\n // Initialize the client with API keys\n const client = new Client({\n openai: { apiKey: process.env.OPENAI_API_KEY! },\n anthropic: { apiKey: process.env.ANTHROPIC_API_KEY! },\n });\n\n console.log('Available providers:', client.listProviders());\n\n // Example 1: OpenAI Chat Completion\n console.log('\\n--- OpenAI Example ---');\n try {\n const openaiResponse = await client.chat.completions.create({\n model: 'openai:gpt-4o-mini',\n messages: [\n { role: 'system', content: 'You are a helpful assistant.' },\n { role: 'user', content: 'What is TypeScript in one sentence?' }\n ],\n temperature: 0.7,\n max_tokens: 100,\n });\n\n console.log('OpenAI Response:', openaiResponse.choices[0].message.content);\n console.log('Usage:', openaiResponse.usage);\n console.log('Full response:', JSON.stringify(openaiResponse, null, 2));\n } catch (error) {\n console.error('OpenAI Error:', error);\n }\n\n // Example 2: Anthropic Chat Completion\n console.log('\\n--- Anthropic Example ---');\n try {\n const anthropicResponse = await client.chat.completions.create({\n model: 'anthropic:claude-3-haiku-20240307',\n messages: [\n { role: 'system', content: 'You are a helpful assistant.' },\n { role: 'user', content: 'What is TypeScript in one sentence?' }\n ],\n temperature: 0.7,\n max_tokens: 100,\n });\n\n console.log('Anthropic Response:', anthropicResponse.choices[0].message.content);\n console.log('Usage:', anthropicResponse.usage);\n console.log('Full response:', JSON.stringify(anthropicResponse, null, 2));\n } catch (error) {\n console.error('Anthropic Error:', error);\n }\n\n // Example 3: Error handling - invalid provider\n console.log('\\n--- Error Handling Example ---');\n try {\n await client.chat.completions.create({\n model: 'invalid:model',\n messages: [{ role: 'user', content: 'Hello' }]\n });\n } catch (error) {\n console.error('Expected error:', error);\n }\n}\n\n// Run the examples\nmain().catch(console.error);" }, { "path": "aisuite-js/examples/chat-app/.eslintrc.cjs", "content": "module.exports = {\n root: true,\n env: { browser: true, es2020: true },\n extends: [\n 'eslint:recommended',\n '@typescript-eslint/recommended',\n 'plugin:react-hooks/recommended',\n ],\n ignorePatterns: ['dist', '.eslintrc.cjs'],\n parser: '@typescript-eslint/parser',\n plugins: ['react-refresh'],\n rules: {\n 'react-refresh/only-export-components': [\n 'warn',\n { allowConstantExport: true },\n ],\n },\n} " }, { "path": "aisuite-js/examples/chat-app/.gitignore", "content": "# Logs\nlogs\n*.log\nnpm-debug.log*\nyarn-debug.log*\nyarn-error.log*\npnpm-debug.log*\nlerna-debug.log*\n\nnode_modules\ndist\ndist-ssr\n*.local\n\n# Editor directories and files\n.vscode/*\n!.vscode/extensions.json\n.idea\n.DS_Store\n*.suo\n*.ntvs*\n*.njsproj\n*.sln\n*.sw?\n\n# Environment variables\n.env\n.env.local\n.env.development.local\n.env.test.local\n.env.production.local " }, { "path": "aisuite-js/examples/chat-app/README.md", "content": "# AISuite Chat App\n\nA modern React TypeScript chat application built with AISuite, allowing you to chat with multiple AI models and compare their responses in real-time.\n\n## Features\n\n- **Multi-Provider Support**: Chat with OpenAI, Anthropic, Groq, and Mistral models\n- **Comparison Mode**: Compare responses from two different AI models side-by-side\n- **Modern UI**: Clean, responsive interface built with React and Tailwind CSS\n- **Real-time Chat**: Instant messaging with AI models\n- **API Key Management**: Secure storage and management of API keys\n- **Error Handling**: Comprehensive error handling and user feedback\n- **TypeScript**: Full type safety throughout the application\n\n## Prerequisites\n\n- Node.js 18+ \n- npm or yarn\n- API keys for the AI providers you want to use:\n - OpenAI API key\n - Anthropic API key\n - Groq API key\n - Mistral API key\n\n## Installation\n\n1. Clone the repository and navigate to the chat app directory:\n```bash\ncd aisuite-js/chat-app\n```\n\n2. Install dependencies:\n```bash\nnpm install\n```\n\n3. Start the development server:\n```bash\nnpm run dev\n```\n\n4. Open your browser and navigate to `http://localhost:3000`\n\n## Configuration\n\n### API Keys\n\n1. Click the \"Configure API Keys\" button in the header\n2. Enter your API keys for the providers you want to use\n3. Click \"Save\" to store the configuration\n\nThe app will automatically save your API keys to localStorage for future use.\n\n### Supported Models\n\nThe app comes pre-configured with the following models:\n\n**OpenAI:**\n- GPT-4o\n- GPT-4o Mini\n\n**Anthropic:**\n- Claude 3.5 Sonnet\n- Claude 3 Haiku\n\n**Groq:**\n- Llama 3.1 8B\n- Mixtral 8x7B\n\n**Mistral:**\n- Mistral 7B\n- Mistral Large\n\n## Usage\n\n### Basic Chat\n\n1. Configure your API keys\n2. Select a model from the dropdown\n3. Type your message and press Enter or click Send\n4. View the AI response\n\n### Comparison Mode\n\n1. Enable \"Comparison Mode\" checkbox\n2. Select two different models\n3. Send a message to see responses from both models side-by-side\n4. Compare the different responses and capabilities\n\n### Chat Management\n\n- **Reset Chat**: Click the reset button to clear all chat history\n- **Model Switching**: Change models at any time during the conversation\n- **Error Handling**: The app displays clear error messages for API issues\n\n## Sample Queries\n\nTry these sample queries to test the different models:\n\n```\n\"What is the weather in Tokyo?\"\n```\n\n```\n\"Write a poem about the weather in Tokyo.\"\n```\n\n```\n\"Write a python program to print the fibonacci sequence.\"\n```\n\n```\n\"Write test cases for this program.\"\n```\n\n## Development\n\n### Project Structure\n\n```\nsrc/\n├── components/ # React components\n│ ├── ApiKeyModal.tsx\n│ ├── ChatContainer.tsx\n│ ├── ChatInput.tsx\n│ ├── ChatMessage.tsx\n│ └── ModelSelector.tsx\n├── config/ # Configuration files\n│ └── llm-config.ts\n├── services/ # Business logic\n│ └── aisuite-service.ts\n├── types/ # TypeScript type definitions\n│ └── chat.ts\n├── App.tsx # Main application component\n├── main.tsx # Application entry point\n└── index.css # Global styles\n```\n\n### Available Scripts\n\n- `npm run dev` - Start development server\n- `npm run build` - Build for production\n- `npm run preview` - Preview production build\n- `npm run lint` - Run ESLint\n\n### Adding New Models\n\nTo add new models, edit `src/config/llm-config.ts`:\n\n```typescript\nexport const configuredLLMs: LLMConfig[] = [\n // ... existing models\n {\n name: \"Your New Model\",\n provider: \"provider-name\",\n model: \"model-name\"\n }\n];\n```\n\n### Styling\n\nThe app uses Tailwind CSS for styling. The design system includes:\n\n- Light and dark mode support\n- Responsive design\n- Custom scrollbars\n- Loading animations\n- Error states\n\n## Technologies Used\n\n- **React 18** - UI framework\n- **TypeScript** - Type safety\n- **Vite** - Build tool and dev server\n- **Tailwind CSS** - Styling\n- **Lucide React** - Icons\n- **AISuite** - AI provider abstraction\n\n## Browser Support\n\n- Chrome 90+\n- Firefox 88+\n- Safari 14+\n- Edge 90+\n\n## Contributing\n\n1. Fork the repository\n2. Create a feature branch\n3. Make your changes\n4. Add tests if applicable\n5. Submit a pull request\n\n## License\n\nMIT License - see the main repository for details.\n\n## Support\n\nFor issues and questions:\n- Check the [AISuite documentation](https://github.com/andrewyng/aisuite)\n- Open an issue in the repository\n- Check the console for error messages\n\n## Security Notes\n\n- API keys are stored in localStorage (client-side only)\n- No API keys are sent to any server except the AI providers\n- Consider using environment variables for production deployments " }, { "path": "aisuite-js/examples/chat-app/index.html", "content": "\n\n \n \n \n \n AISuite Chat App\n \n \n
\n \n \n " }, { "path": "aisuite-js/examples/chat-app/package.json", "content": "{\n \"name\": \"aisuite-chat-app\",\n \"version\": \"1.0.0\",\n \"description\": \"A React TypeScript chat application using AISuite\",\n \"private\": true,\n \"scripts\": {\n \"dev\": \"vite\",\n \"build\": \"tsc && vite build\",\n \"preview\": \"vite preview\",\n \"lint\": \"eslint . --ext ts,tsx --report-unused-disable-directives --max-warnings 0\"\n },\n \"dependencies\": {\n \"react\": \"^18.2.0\",\n \"react-dom\": \"^18.2.0\", \n \"lucide-react\": \"^0.263.1\",\n \"clsx\": \"^2.0.0\",\n \"tailwind-merge\": \"^1.14.0\"\n },\n \"devDependencies\": {\n \"@types/react\": \"^18.2.15\",\n \"@types/react-dom\": \"^18.2.7\",\n \"@typescript-eslint/eslint-plugin\": \"^6.0.0\",\n \"@typescript-eslint/parser\": \"^6.0.0\",\n \"@vitejs/plugin-react\": \"^4.0.3\",\n \"autoprefixer\": \"^10.4.14\",\n \"eslint\": \"^8.45.0\",\n \"eslint-plugin-react-hooks\": \"^4.6.0\",\n \"eslint-plugin-react-refresh\": \"^0.4.3\",\n \"postcss\": \"^8.4.27\",\n \"tailwindcss\": \"^3.3.3\",\n \"typescript\": \"^5.0.2\",\n \"vite\": \"^4.4.5\"\n }\n} " }, { "path": "aisuite-js/examples/chat-app/postcss.config.js", "content": "export default {\n plugins: {\n tailwindcss: {},\n autoprefixer: {},\n },\n} " }, { "path": "aisuite-js/examples/chat-app/src/App.tsx", "content": "import React, { useState, useEffect } from 'react';\nimport { Settings, AlertCircle } from 'lucide-react';\nimport { Message, AISuiteConfig } from './types/chat';\nimport { configuredLLMs, getLLMConfigByName } from './config/llm-config';\nimport { aiSuiteService } from './services/aisuite-service';\nimport { ChatContainer } from './components/ChatContainer';\nimport { ChatInput } from './components/ChatInput';\nimport { ModelSelector } from './components/ModelSelector';\nimport { ProviderSelector } from './components/ProviderSelector';\nimport { ApiKeyModal } from './components/ApiKeyModal';\n\nfunction App() {\n const [chatHistory1, setChatHistory1] = useState([]);\n const [chatHistory2, setChatHistory2] = useState([]);\n const [isProcessing, setIsProcessing] = useState(false);\n const [useComparisonMode, setUseComparisonMode] = useState(false);\n const [selectedProvider, setSelectedProvider] = useState('');\n const [selectedModel1, setSelectedModel1] = useState('');\n const [selectedModel2, setSelectedModel2] = useState('');\n const [showApiKeyModal, setShowApiKeyModal] = useState(false);\n const [apiConfig, setApiConfig] = useState({});\n const [error, setError] = useState(null);\n\n // Initialize AISuite service when API config changes\n useEffect(() => {\n if (Object.keys(apiConfig).length > 0) {\n try {\n aiSuiteService.initialize(apiConfig);\n setError(null);\n } catch (err) {\n setError('Failed to initialize AISuite client');\n }\n }\n }, [apiConfig]);\n\n // Load API config from localStorage on mount\n useEffect(() => {\n const savedConfig = localStorage.getItem('aisuite-config');\n if (savedConfig) {\n try {\n const config = JSON.parse(savedConfig);\n setApiConfig(config);\n } catch (err) {\n console.error('Failed to load saved config');\n }\n }\n }, []);\n\n const handleSendMessage = async (message: string) => {\n if (!message.trim()) return;\n\n // Check if provider is selected\n if (!selectedProvider) {\n setError('Please select a provider first');\n return;\n }\n\n // Check if API key is configured for the selected provider\n if (!apiConfig[selectedProvider as keyof AISuiteConfig]?.apiKey) {\n setError(`API key for ${selectedProvider} is not configured. Please configure it first.`);\n setShowApiKeyModal(true);\n return;\n }\n\n const userMessage: Message = {\n role: 'user',\n content: message,\n timestamp: new Date()\n };\n\n setIsProcessing(true);\n setError(null);\n\n try {\n // Add user message to both chat histories\n setChatHistory1(prev => [...prev, userMessage]);\n if (useComparisonMode) {\n setChatHistory2(prev => [...prev, userMessage]);\n }\n\n // Get model configurations\n const modelConfig1 = getLLMConfigByName(selectedModel1);\n if (!modelConfig1) {\n throw new Error(`Model ${selectedModel1} not found`);\n }\n\n // Query first model\n const response1 = await aiSuiteService.queryLLM(modelConfig1, [...chatHistory1, userMessage]);\n const assistantMessage1: Message = {\n role: 'assistant',\n content: response1,\n timestamp: new Date()\n };\n setChatHistory1(prev => [...prev, assistantMessage1]);\n\n // Query second model if in comparison mode\n if (useComparisonMode) {\n const modelConfig2 = getLLMConfigByName(selectedModel2);\n if (!modelConfig2) {\n throw new Error(`Model ${selectedModel2} not found`);\n }\n\n const response2 = await aiSuiteService.queryLLM(modelConfig2, [...chatHistory2, userMessage]);\n const assistantMessage2: Message = {\n role: 'assistant',\n content: response2,\n timestamp: new Date()\n };\n setChatHistory2(prev => [...prev, assistantMessage2]);\n }\n } catch (err) {\n setError(err instanceof Error ? err.message : 'An error occurred');\n } finally {\n setIsProcessing(false);\n }\n };\n\n const handleResetChat = () => {\n setChatHistory1([]);\n setChatHistory2([]);\n setError(null);\n };\n\n const handleSaveApiConfig = (config: AISuiteConfig) => {\n setApiConfig(config);\n localStorage.setItem('aisuite-config', JSON.stringify(config));\n };\n\n // Get all available providers (show all by default)\n const allProviders = ['openai', 'anthropic', 'groq', 'mistral'];\n const availableProviders = allProviders;\n \n // Get configured providers (those with API keys)\n const configuredProviders = Object.keys(apiConfig).filter(provider => \n apiConfig[provider as keyof AISuiteConfig]?.apiKey\n );\n\n // Get models for the selected provider\n const availableModels = selectedProvider \n ? configuredLLMs.filter(model => model.provider === selectedProvider)\n : [];\n\n // Reset model selections when provider changes\n useEffect(() => {\n if (selectedProvider) {\n const providerModels = configuredLLMs.filter(model => model.provider === selectedProvider);\n if (providerModels.length > 0) {\n setSelectedModel1(providerModels[0].name);\n if (useComparisonMode && providerModels.length > 1) {\n setSelectedModel2(providerModels[1].name);\n } else {\n setSelectedModel2('');\n }\n } else {\n setSelectedModel1('');\n setSelectedModel2('');\n }\n } else {\n setSelectedModel1('');\n setSelectedModel2('');\n }\n }, [selectedProvider, useComparisonMode]);\n\n const hasConfiguredProviders = Object.keys(apiConfig).length > 0;\n\n return (\n
\n {/* Header */}\n
\n
\n
\n

AISuite Chat

\n setShowApiKeyModal(true)}\n className=\"flex items-center gap-2 rounded-lg border border-input bg-background px-3 py-2 text-sm font-medium ring-offset-background placeholder:text-muted-foreground hover:bg-accent hover:text-accent-foreground focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2\"\n >\n \n Configure API Keys\n \n
\n
\n
\n\n {/* Main Content */}\n
\n {!hasConfiguredProviders ? (\n
\n
\n \n

No API Keys Configured

\n

\n Please configure your API keys to start chatting with AI models.\n

\n setShowApiKeyModal(true)}\n className=\"rounded-lg bg-primary px-4 py-2 text-sm font-medium text-primary-foreground hover:bg-primary/90\"\n >\n Configure API Keys\n \n
\n
\n ) : (\n
\n {/* Error Display */}\n {error && (\n
\n
\n \n {error}\n
\n
\n )}\n\n {/* Controls */}\n
\n {/* Provider Selection */}\n
\n \n
\n \n
\n
\n\n {/* Model Selection - Only show if provider is selected */}\n {selectedProvider && (\n
\n \n {useComparisonMode && availableModels.length > 1 && (\n \n )}\n
\n )}\n
\n\n {/* Chat Containers */}\n {selectedProvider && selectedModel1 && (\n
\n
\n
\n

{selectedModel1}

\n
\n
\n \n
\n
\n\n {useComparisonMode && selectedModel2 && (\n
\n
\n

{selectedModel2}

\n
\n
\n \n
\n
\n )}\n
\n )}\n\n {/* No Provider Selected State */}\n {!selectedProvider && hasConfiguredProviders && (\n
\n
\n \n

Select a Provider

\n

\n Please select an AI provider to start chatting.\n

\n
\n
\n )}\n\n {/* Chat Input */}\n \n
\n )}\n
\n\n {/* API Key Modal */}\n setShowApiKeyModal(false)}\n onSave={handleSaveApiConfig}\n initialConfig={apiConfig}\n />\n
\n );\n}\n\nexport default App; " }, { "path": "aisuite-js/examples/chat-app/src/components/ApiKeyModal.tsx", "content": "import React, { useState } from 'react';\nimport { X, Eye, EyeOff } from 'lucide-react';\nimport { AISuiteConfig } from '../types/chat';\n\ninterface ApiKeyModalProps {\n isOpen: boolean;\n onClose: () => void;\n onSave: (config: AISuiteConfig) => void;\n initialConfig?: AISuiteConfig;\n}\n\nexport const ApiKeyModal: React.FC = ({\n isOpen,\n onClose,\n onSave,\n initialConfig = {}\n}) => {\n const [config, setConfig] = useState(initialConfig);\n const [showKeys, setShowKeys] = useState>({});\n\n const toggleKeyVisibility = (provider: string) => {\n setShowKeys(prev => ({\n ...prev,\n [provider]: !prev[provider]\n }));\n };\n\n const handleSave = () => {\n // Filter out empty API keys\n const filteredConfig: AISuiteConfig = {};\n Object.entries(config).forEach(([provider, providerConfig]) => {\n if (providerConfig?.apiKey?.trim()) {\n providerConfig.dangerouslyAllowBrowser = true;\n filteredConfig[provider as keyof AISuiteConfig] = providerConfig;\n }\n });\n onSave(filteredConfig);\n onClose();\n };\n\n const updateConfig = (provider: string, field: string, value: string) => {\n setConfig(prev => ({\n ...prev,\n [provider]: {\n ...prev[provider as keyof AISuiteConfig],\n [field]: value\n }\n }));\n };\n\n if (!isOpen) return null;\n\n return (\n
\n
\n
\n

Configure API Keys

\n \n \n \n
\n\n
\n {/* OpenAI */}\n
\n \n
\n updateConfig('openai', 'apiKey', e.target.value)}\n className=\"w-full rounded-lg border border-input bg-background px-3 py-2 text-sm ring-offset-background placeholder:text-muted-foreground focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2\"\n />\n toggleKeyVisibility('openai')}\n className=\"absolute right-2 top-1/2 -translate-y-1/2 text-muted-foreground hover:text-foreground\"\n >\n {showKeys.openai ? : }\n \n
\n
\n\n {/* Anthropic */}\n
\n \n
\n updateConfig('anthropic', 'apiKey', e.target.value)}\n className=\"w-full rounded-lg border border-input bg-background px-3 py-2 text-sm ring-offset-background placeholder:text-muted-foreground focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2\"\n />\n toggleKeyVisibility('anthropic')}\n className=\"absolute right-2 top-1/2 -translate-y-1/2 text-muted-foreground hover:text-foreground\"\n >\n {showKeys.anthropic ? : }\n \n
\n
\n\n {/* Groq */}\n
\n \n
\n updateConfig('groq', 'apiKey', e.target.value)}\n className=\"w-full rounded-lg border border-input bg-background px-3 py-2 text-sm ring-offset-background placeholder:text-muted-foreground focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2\"\n />\n toggleKeyVisibility('groq')}\n className=\"absolute right-2 top-1/2 -translate-y-1/2 text-muted-foreground hover:text-foreground\"\n >\n {showKeys.groq ? : }\n \n
\n
\n\n {/* Mistral */}\n
\n \n
\n updateConfig('mistral', 'apiKey', e.target.value)}\n className=\"w-full rounded-lg border border-input bg-background px-3 py-2 text-sm ring-offset-background placeholder:text-muted-foreground focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2\"\n />\n toggleKeyVisibility('mistral')}\n className=\"absolute right-2 top-1/2 -translate-y-1/2 text-muted-foreground hover:text-foreground\"\n >\n {showKeys.mistral ? : }\n \n
\n
\n
\n\n
\n \n Cancel\n \n \n Save\n \n
\n
\n
\n );\n}; " }, { "path": "aisuite-js/examples/chat-app/src/components/ChatContainer.tsx", "content": "import React, { useRef, useEffect } from 'react';\nimport { Message } from '../types/chat';\nimport { ChatMessage } from './ChatMessage';\n\ninterface ChatContainerProps {\n messages: Message[];\n modelName: string;\n isLoading?: boolean;\n}\n\nexport const ChatContainer: React.FC = ({ \n messages, \n modelName, \n isLoading = false \n}) => {\n const messagesEndRef = useRef(null);\n\n const scrollToBottom = () => {\n messagesEndRef.current?.scrollIntoView({ behavior: 'smooth' });\n };\n\n useEffect(() => {\n scrollToBottom();\n }, [messages]);\n\n return (\n
\n
\n
\n {messages.length === 0 ? (\n
\n
\n
No messages yet
\n
Start a conversation with {modelName}
\n
\n
\n ) : (\n messages.map((message, index) => (\n \n ))\n )}\n \n {isLoading && (\n
\n
\n
\n
\n
\n
\n
\n {modelName}\n
\n
\n
\n
\n
\n
\n
\n Thinking...\n
\n
\n
\n
\n )}\n \n
\n
\n
\n
\n );\n}; " }, { "path": "aisuite-js/examples/chat-app/src/components/ChatInput.tsx", "content": "import React, { useState, KeyboardEvent } from 'react';\nimport { Send, RotateCcw } from 'lucide-react';\n\ninterface ChatInputProps {\n onSendMessage: (message: string) => void;\n onResetChat: () => void;\n isLoading: boolean;\n placeholder?: string;\n disabled?: boolean;\n}\n\nexport const ChatInput: React.FC = ({\n onSendMessage,\n onResetChat,\n isLoading,\n placeholder = \"Enter your query...\",\n disabled = false\n}) => {\n const [message, setMessage] = useState('');\n\n const handleSend = () => {\n if (message.trim() && !isLoading && !disabled) {\n onSendMessage(message.trim());\n setMessage('');\n }\n };\n\n const handleKeyPress = (e: KeyboardEvent) => {\n if (e.key === 'Enter' && !e.shiftKey) {\n e.preventDefault();\n handleSend();\n }\n };\n\n return (\n
\n
\n
\n setMessage(e.target.value)}\n onKeyPress={handleKeyPress}\n placeholder={placeholder}\n disabled={isLoading || disabled}\n className=\"w-full resize-none rounded-lg border border-input bg-background px-3 py-2 text-sm ring-offset-background placeholder:text-muted-foreground focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50\"\n rows={3}\n style={{ minHeight: '60px', maxHeight: '120px' }}\n />\n
\n \n
\n \n \n \n \n \n \n \n
\n
\n \n {isLoading && (\n
\n
\n Processing...\n
\n )}\n
\n );\n}; " }, { "path": "aisuite-js/examples/chat-app/src/components/ChatMessage.tsx", "content": "import React from 'react';\nimport { Message } from '../types/chat';\nimport { User, Bot } from 'lucide-react';\n\ninterface ChatMessageProps {\n message: Message;\n modelName?: string;\n}\n\nexport const ChatMessage: React.FC = ({ message, modelName }) => {\n const isUser = message.role === 'user';\n const roleDisplay = isUser ? 'User' : modelName || 'Assistant';\n\n return (\n
\n {!isUser && (\n
\n \n
\n )}\n \n
\n
\n
\n {roleDisplay}\n
\n
\n {message.content}\n
\n
\n {message.timestamp && (\n
\n {message.timestamp.toLocaleTimeString()}\n
\n )}\n
\n\n {isUser && (\n
\n \n
\n )}\n
\n );\n}; " }, { "path": "aisuite-js/examples/chat-app/src/components/ModelSelector.tsx", "content": "import React from 'react';\nimport { LLMConfig } from '../types/chat';\n\ninterface ModelSelectorProps {\n selectedModel: string;\n onModelChange: (modelName: string) => void;\n availableModels: LLMConfig[];\n label: string;\n disabled?: boolean;\n}\n\nexport const ModelSelector: React.FC = ({\n selectedModel,\n onModelChange,\n availableModels,\n label,\n disabled = false\n}) => {\n return (\n
\n \n onModelChange(e.target.value)}\n disabled={disabled}\n className=\"w-full rounded-lg border border-input bg-background px-3 py-2 text-sm ring-offset-background placeholder:text-muted-foreground focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50\"\n >\n {availableModels.map((model) => (\n \n ))}\n \n
\n );\n}; " }, { "path": "aisuite-js/examples/chat-app/src/components/ProviderSelector.tsx", "content": "import React from 'react';\n\ninterface ProviderSelectorProps {\n selectedProvider: string;\n onProviderChange: (provider: string) => void;\n availableProviders: string[];\n configuredProviders: string[];\n label: string;\n disabled?: boolean;\n}\n\nexport const ProviderSelector: React.FC = ({\n selectedProvider,\n onProviderChange,\n availableProviders,\n configuredProviders,\n label,\n disabled = false\n}) => {\n const providerNames = {\n openai: 'OpenAI',\n anthropic: 'Anthropic',\n groq: 'Groq',\n mistral: 'Mistral'\n };\n\n return (\n
\n \n onProviderChange(e.target.value)}\n disabled={disabled}\n className=\"w-full rounded-lg border border-input bg-background px-3 py-2 text-sm ring-offset-background placeholder:text-muted-foreground focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-ring focus-visible:ring-offset-2 disabled:cursor-not-allowed disabled:opacity-50\"\n >\n \n {availableProviders.map((provider) => {\n const isConfigured = configuredProviders.includes(provider);\n const displayName = providerNames[provider as keyof typeof providerNames] || provider;\n return (\n \n );\n })}\n \n
\n );\n}; " }, { "path": "aisuite-js/examples/chat-app/src/config/llm-config.ts", "content": "import { LLMConfig } from '../types/chat';\n\nexport const configuredLLMs: LLMConfig[] = [\n {\n name: \"OpenAI GPT-4o\",\n provider: \"openai\",\n model: \"gpt-4o\"\n },\n {\n name: \"OpenAI GPT-4o Mini\",\n provider: \"openai\",\n model: \"gpt-4o-mini\"\n },\n {\n name: \"Anthropic Claude 3.5 Sonnet\",\n provider: \"anthropic\",\n model: \"claude-3-5-sonnet-20241022\"\n },\n {\n name: \"Anthropic Claude 3 Haiku\",\n provider: \"anthropic\",\n model: \"claude-3-haiku-20240307\"\n },\n {\n name: \"Groq Llama 3.3-70b Versatile\",\n provider: \"groq\",\n model: \"llama-3.3-70b-versatile\"\n },\n {\n name: \"Groq Mixtral 24B\",\n provider: \"groq\",\n model: \"mistral-saba-24b\"\n },\n {\n name: \"Groq Gemma 2 9B\",\n provider: \"groq\",\n model: \"gemma2-9b-it\"\n },\n {\n name: \"Mistral Medium\",\n provider: \"mistral\",\n model: \"mistral-medium\"\n },\n {\n name: \"Mistral Large\",\n provider: \"mistral\",\n model: \"mistral-large-latest\"\n }\n];\n\nexport const getLLMConfigByName = (name: string): LLMConfig | undefined => {\n return configuredLLMs.find(llm => llm.name === name);\n};\n\nexport const getLLMConfigByProviderAndModel = (provider: string, model: string): LLMConfig | undefined => {\n return configuredLLMs.find(llm => llm.provider === provider && llm.model === model);\n}; " }, { "path": "aisuite-js/examples/chat-app/src/index.css", "content": "@tailwind base;\n@tailwind components;\n@tailwind utilities;\n\n@layer base {\n :root {\n --background: 0 0% 100%;\n --foreground: 222.2 84% 4.9%;\n --card: 0 0% 100%;\n --card-foreground: 222.2 84% 4.9%;\n --popover: 0 0% 100%;\n --popover-foreground: 222.2 84% 4.9%;\n --primary: 222.2 47.4% 11.2%;\n --primary-foreground: 210 40% 98%;\n --secondary: 210 40% 96%;\n --secondary-foreground: 222.2 84% 4.9%;\n --muted: 210 40% 96%;\n --muted-foreground: 215.4 16.3% 46.9%;\n --accent: 210 40% 96%;\n --accent-foreground: 222.2 84% 4.9%;\n --destructive: 0 84.2% 60.2%;\n --destructive-foreground: 210 40% 98%;\n --border: 214.3 31.8% 91.4%;\n --input: 214.3 31.8% 91.4%;\n --ring: 222.2 84% 4.9%;\n --radius: 0.5rem;\n --chart-1: 12 76% 61%;\n --chart-2: 173 58% 39%;\n --chart-3: 197 37% 24%;\n --chart-4: 43 74% 66%;\n --chart-5: 27 87% 67%;\n }\n\n .dark {\n --background: 222.2 84% 4.9%;\n --foreground: 210 40% 98%;\n --card: 222.2 84% 4.9%;\n --card-foreground: 210 40% 98%;\n --popover: 222.2 84% 4.9%;\n --popover-foreground: 210 40% 98%;\n --primary: 210 40% 98%;\n --primary-foreground: 222.2 47.4% 11.2%;\n --secondary: 217.2 32.6% 17.5%;\n --secondary-foreground: 210 40% 98%;\n --muted: 217.2 32.6% 17.5%;\n --muted-foreground: 215 20.2% 65.1%;\n --accent: 217.2 32.6% 17.5%;\n --accent-foreground: 210 40% 98%;\n --destructive: 0 62.8% 30.6%;\n --destructive-foreground: 210 40% 98%;\n --border: 217.2 32.6% 17.5%;\n --input: 217.2 32.6% 17.5%;\n --ring: 212.7 26.8% 83.9%;\n --chart-1: 220 70% 50%;\n --chart-2: 160 60% 45%;\n --chart-3: 30 80% 55%;\n --chart-4: 280 65% 60%;\n --chart-5: 340 75% 55%;\n }\n}\n\n@layer base {\n * {\n @apply border-border;\n }\n body {\n @apply bg-background text-foreground;\n }\n}\n\n/* Custom scrollbar styles */\n.custom-scrollbar::-webkit-scrollbar {\n width: 6px;\n}\n\n.custom-scrollbar::-webkit-scrollbar-track {\n background: transparent;\n}\n\n.custom-scrollbar::-webkit-scrollbar-thumb {\n background: hsl(var(--muted-foreground) / 0.3);\n border-radius: 3px;\n}\n\n.custom-scrollbar::-webkit-scrollbar-thumb:hover {\n background: hsl(var(--muted-foreground) / 0.5);\n} " }, { "path": "aisuite-js/examples/chat-app/src/main.tsx", "content": "import React from 'react'\nimport ReactDOM from 'react-dom/client'\nimport App from './App.tsx'\nimport './index.css'\n\nReactDOM.createRoot(document.getElementById('root')!).render(\n \n \n ,\n) " }, { "path": "aisuite-js/examples/chat-app/src/services/aisuite-service.ts", "content": "import { Client } from '../../../../src/client';\nimport { Message, LLMConfig, AISuiteConfig } from '../types/chat';\n\nclass AISuiteService {\n private client: Client | null = null;\n private config: AISuiteConfig | null = null;\n\n initialize(config: AISuiteConfig) {\n this.config = config;\n this.client = new Client(config);\n }\n\n async queryLLM(modelConfig: LLMConfig, messages: Message[]): Promise {\n if (!this.client) {\n throw new Error('AISuite client not initialized. Please check your API keys.');\n }\n\n try {\n const model = `${modelConfig.provider}:${modelConfig.model}`;\n const response = await this.client.chat.completions.create({\n model,\n messages: messages.map(msg => ({\n role: msg.role,\n content: msg.content\n })),\n temperature: 0.7,\n max_tokens: 1000,\n stream: false, // Explicitly set stream to false to get ChatCompletionResponse\n });\n\n // Type guard to ensure we have a ChatCompletionResponse\n if ('choices' in response && Array.isArray(response.choices)) {\n return response.choices[0].message.content || 'No response from model';\n } else {\n throw new Error('Unexpected response format from model');\n }\n } catch (error) {\n console.error(`Error querying ${modelConfig.name}:`, error);\n throw new Error(`Error with ${modelConfig.name}: ${error instanceof Error ? error.message : 'Unknown error'}`);\n }\n }\n\n getAvailableProviders(): string[] {\n if (!this.client) {\n return [];\n }\n return this.client.listProviders();\n }\n\n isProviderConfigured(provider: string): boolean {\n if (!this.client) {\n return false;\n }\n return this.client.isProviderConfigured(provider);\n }\n\n getConfig(): AISuiteConfig | null {\n return this.config;\n }\n}\n\nexport const aiSuiteService = new AISuiteService(); " }, { "path": "aisuite-js/examples/chat-app/src/types/chat.ts", "content": "export interface Message {\n role: 'user' | 'assistant' | 'system';\n content: string;\n timestamp?: Date;\n}\n\nexport interface ChatHistory {\n id: string;\n messages: Message[];\n modelName: string;\n createdAt: Date;\n updatedAt: Date;\n}\n\nexport interface LLMConfig {\n name: string;\n provider: string;\n model: string;\n}\n\nexport interface ChatState {\n chatHistory1: Message[];\n chatHistory2: Message[];\n isProcessing: boolean;\n useComparisonMode: boolean;\n selectedModel1: string;\n selectedModel2: string;\n}\n\nexport interface AISuiteConfig {\n openai?: {\n apiKey: string;\n baseURL?: string;\n organization?: string;\n };\n anthropic?: {\n apiKey: string;\n baseURL?: string;\n };\n groq?: {\n apiKey: string;\n baseURL?: string;\n dangerouslyAllowBrowser?: boolean;\n };\n mistral?: {\n apiKey: string;\n baseURL?: string;\n };\n} " }, { "path": "aisuite-js/examples/chat-app/src/utils/cn.ts", "content": "import { type ClassValue, clsx } from \"clsx\"\nimport { twMerge } from \"tailwind-merge\"\n\nexport function cn(...inputs: ClassValue[]) {\n return twMerge(clsx(inputs))\n} " }, { "path": "aisuite-js/examples/chat-app/tailwind.config.js", "content": "/** @type {import('tailwindcss').Config} */\nexport default {\n content: [\n \"./index.html\",\n \"./src/**/*.{js,ts,jsx,tsx}\",\n ],\n theme: {\n extend: {\n colors: {\n background: \"hsl(var(--background))\",\n foreground: \"hsl(var(--foreground))\",\n card: {\n DEFAULT: \"hsl(var(--card))\",\n foreground: \"hsl(var(--card-foreground))\",\n },\n popover: {\n DEFAULT: \"hsl(var(--popover))\",\n foreground: \"hsl(var(--popover-foreground))\",\n },\n primary: {\n DEFAULT: \"hsl(var(--primary))\",\n foreground: \"hsl(var(--primary-foreground))\",\n },\n secondary: {\n DEFAULT: \"hsl(var(--secondary))\",\n foreground: \"hsl(var(--secondary-foreground))\",\n },\n muted: {\n DEFAULT: \"hsl(var(--muted))\",\n foreground: \"hsl(var(--muted-foreground))\",\n },\n accent: {\n DEFAULT: \"hsl(var(--accent))\",\n foreground: \"hsl(var(--accent-foreground))\",\n },\n destructive: {\n DEFAULT: \"hsl(var(--destructive))\",\n foreground: \"hsl(var(--destructive-foreground))\",\n },\n border: \"hsl(var(--border))\",\n input: \"hsl(var(--input))\",\n ring: \"hsl(var(--ring))\",\n chart: {\n \"1\": \"hsl(var(--chart-1))\",\n \"2\": \"hsl(var(--chart-2))\",\n \"3\": \"hsl(var(--chart-3))\",\n \"4\": \"hsl(var(--chart-4))\",\n \"5\": \"hsl(var(--chart-5))\",\n },\n },\n borderRadius: {\n lg: \"var(--radius)\",\n md: \"calc(var(--radius) - 2px)\",\n sm: \"calc(var(--radius) - 4px)\",\n },\n },\n },\n plugins: [],\n} " }, { "path": "aisuite-js/examples/chat-app/tsconfig.json", "content": "{\n \"compilerOptions\": {\n \"target\": \"ES2020\",\n \"useDefineForClassFields\": true,\n \"lib\": [\"ES2020\", \"DOM\", \"DOM.Iterable\"],\n \"module\": \"ESNext\",\n \"skipLibCheck\": true,\n\n /* Bundler mode */\n \"moduleResolution\": \"bundler\",\n \"allowImportingTsExtensions\": true,\n \"resolveJsonModule\": true,\n \"isolatedModules\": true,\n \"noEmit\": true,\n \"jsx\": \"react-jsx\",\n\n /* Linting */\n \"strict\": true,\n \"noUnusedLocals\": true,\n \"noUnusedParameters\": true,\n \"noFallthroughCasesInSwitch\": true\n },\n \"include\": [\"src\"],\n \"references\": [{ \"path\": \"./tsconfig.node.json\" }]\n} " }, { "path": "aisuite-js/examples/chat-app/tsconfig.node.json", "content": "{\n \"compilerOptions\": {\n \"composite\": true,\n \"skipLibCheck\": true,\n \"module\": \"ESNext\",\n \"moduleResolution\": \"bundler\",\n \"allowSyntheticDefaultImports\": true\n },\n \"include\": [\"vite.config.ts\"]\n} " }, { "path": "aisuite-js/examples/chat-app/vite.config.ts", "content": "import { defineConfig } from 'vite'\nimport react from '@vitejs/plugin-react'\n\n// https://vitejs.dev/config/\nexport default defineConfig({\n plugins: [react()],\n server: {\n port: 3000,\n open: true\n }\n}) " }, { "path": "aisuite-js/examples/deepgram.ts", "content": "import { Client } from \"../src\";\nimport * as fs from \"fs\";\nimport * as path from \"path\";\n\nasync function main() {\n // Initialize the client with Deepgram configuration\n // Using Deepgram SDK v4.11.2 with the new createClient API\n const client = new Client({\n deepgram: {\n apiKey: process.env.DEEPGRAM_API_KEY || \"your-deepgram-api-key\",\n },\n });\n\n console.log(\"Available ASR providers:\", client.listASRProviders());\n\n // Example: Transcribe an audio file\n try {\n // Create a simple test audio file (you would replace this with your actual audio file)\n const testAudioPath = path.join(\"test-audio.wav\");\n\n // Check if test file exists, if not create a placeholder\n if (!fs.existsSync(testAudioPath)) {\n console.log(\n \"Test audio file not found. Please provide a valid audio file for transcription.\"\n );\n console.log(\"Expected path:\", testAudioPath);\n return;\n }\n\n // Read the file as a buffer\n const audioBuffer = fs.readFileSync(testAudioPath);\n\n // Create the transcription request with the audio buffer\n const result = await client.audio.transcriptions.create({\n model: \"deepgram:general\",\n file: audioBuffer,\n language: \"en-US\",\n timestamps: true,\n word_confidence: true,\n speaker_labels: true,\n });\n\n console.log(\"Transcription Result:\");\n console.log(\"Text:\", result.text);\n console.log(\"Language:\", result.language);\n console.log(\"Confidence:\", result.confidence);\n\n if (result.words && result.words.length > 0) {\n console.log(\"\\nWords with timestamps:\");\n result.words.slice(0, 5).forEach((word, index) => {\n console.log(\n `${index + 1}. \"${word.text}\" (${word.start}s - ${\n word.end\n }s, confidence: ${word.confidence})`\n );\n });\n }\n\n if (result.segments && result.segments.length > 0) {\n console.log(\"\\nSegments:\");\n result.segments.forEach((segment, index) => {\n console.log(\n `${index + 1}. [${segment.start}s - ${segment.end}s] ${segment.text}`\n );\n });\n }\n } catch (error) {\n console.error(\"Error during transcription:\", error);\n }\n}\n\nmain().catch(console.error);\n\n" }, { "path": "aisuite-js/examples/groq.ts", "content": "import \"dotenv/config\";\nimport { Client, ChatCompletionResponse, ChatMessage } from \"../src\";\n\n// Mock function for weather\nfunction getWeather(location: string, unit: 'celsius' | 'fahrenheit' = 'celsius') {\n // Mock implementation\n return {\n location,\n temperature: unit === 'celsius' ? 22 : 72,\n condition: 'sunny',\n unit\n };\n}\n\n// Available Groq models\nconst AVAILABLE_MODELS = {\n MIXTRAL: \"groq:mistral-saba-24b\",\n LLAMA2: \"groq:llama-3.3-70b-versatile\",\n GEMMA: \"groq:gemma2-9b-it\",\n};\n\nasync function main() {\n const client = new Client({\n groq: { apiKey: process.env.GROQ_API_KEY! },\n });\n\n console.log(\"\\n🚀 Groq Chat Examples\\n\");\n\n // Example 1: Basic chat completion with Mixtral\n console.log(\"--- Basic Chat Completion with Mixtral ---\");\n try {\n const response = (await client.chat.completions.create({\n model: AVAILABLE_MODELS.MIXTRAL,\n messages: [\n { role: \"system\", content: \"You are a helpful assistant.\" },\n { role: \"user\", content: \"What is TypeScript in one sentence?\" },\n ],\n temperature: 0.7,\n max_tokens: 100,\n stream: false,\n })) as ChatCompletionResponse;\n\n console.log(\"Response:\", response.choices[0].message.content);\n console.log(\"Usage:\", response.usage);\n console.log(\"Full response:\", JSON.stringify(response, null, 2));\n } catch (error) {\n console.error(\"Error:\", error);\n }\n\n // Example 2: Streaming with LLaMA2\n console.log(\"\\n--- Streaming Example with LLaMA2 ---\");\n try {\n const stream = await client.chat.completions.create({\n model: AVAILABLE_MODELS.LLAMA2,\n messages: [\n { role: \"system\", content: \"You are a helpful assistant.\" },\n {\n role: \"user\",\n content: \"Write a haiku about artificial intelligence.\",\n },\n ],\n stream: true,\n temperature: 0.7,\n max_tokens: 100,\n });\n\n console.log(\"Response:\");\n let fullContent = \"\";\n for await (const chunk of stream as AsyncIterable) {\n const content = chunk.choices[0]?.delta?.content || \"\";\n process.stdout.write(content);\n fullContent += content;\n }\n console.log(\"\\n\");\n } catch (error) {\n console.error(\"Streaming error:\", error);\n }\n\n // Example 3: Chat completion with Gemma\n console.log(\"\\n--- Chat Completion with Gemma ---\");\n try {\n const response = (await client.chat.completions.create({\n model: AVAILABLE_MODELS.GEMMA,\n messages: [\n { role: \"system\", content: \"You are a helpful assistant.\" },\n {\n role: \"user\",\n content: \"Explain how machine learning can be used in healthcare.\",\n },\n ],\n temperature: 0.5,\n max_tokens: 200,\n stream: false,\n })) as ChatCompletionResponse;\n\n console.log(\"Response:\", response.choices[0].message.content);\n console.log(\"Usage:\", response.usage);\n } catch (error) {\n console.error(\"Error:\", error);\n }\n\n // Example 4: Conversation with context\n console.log(\"\\n--- Conversation with Context ---\");\n try {\n const conversation = [\n { role: \"system\", content: \"You are a helpful assistant.\" },\n { role: \"user\", content: \"What is quantum computing?\" },\n {\n role: \"assistant\",\n content:\n \"Quantum computing is a type of computing that uses quantum mechanical phenomena like superposition and entanglement to perform calculations.\",\n },\n { role: \"user\", content: \"Can you give a practical example?\" },\n ] as ChatMessage[];\n\n const response = (await client.chat.completions.create({\n model: AVAILABLE_MODELS.MIXTRAL,\n messages: conversation,\n temperature: 0.7,\n max_tokens: 150,\n stream: false,\n })) as ChatCompletionResponse;\n\n console.log(\"Response:\", response.choices[0].message.content);\n console.log(\"Usage:\", response.usage);\n } catch (error) {\n console.error(\"Error:\", error);\n }\n\n // Example 5: Tool calling with Groq\n console.log(\"\\n--- Tool Calling Example with Groq ---\");\n try {\n // Define tools in OpenAI format\n const tools = [\n {\n type: 'function' as const,\n function: {\n name: 'get_weather',\n description: 'Get the current weather for a location',\n parameters: {\n type: 'object' as const,\n properties: {\n location: {\n type: 'string',\n description: 'The city and state, e.g. San Francisco, CA'\n },\n unit: {\n type: 'string',\n enum: ['celsius', 'fahrenheit'],\n description: 'The temperature unit'\n }\n },\n required: ['location']\n }\n }\n }\n ];\n\n // Step 1: Initial request with tools\n const response = (await client.chat.completions.create({\n model: AVAILABLE_MODELS.MIXTRAL,\n messages: [\n { role: 'system', content: 'You are a helpful weather assistant.' },\n { role: 'user', content: \"What's the weather like in London?\" }\n ],\n tools,\n tool_choice: 'auto'\n })) as ChatCompletionResponse;\n\n const message = response.choices[0]?.message;\n console.log('Step 1 - Initial response:', JSON.stringify(message, null, 2));\n\n if (message?.tool_calls) {\n // Step 2: Execute tool calls and send results back\n const messages: ChatMessage[] = [\n { role: 'system', content: 'You are a helpful weather assistant.' },\n { role: 'user', content: \"What's the weather like in London?\" },\n message // The assistant's message with tool calls\n ];\n\n console.log('\\nTool calls detected:');\n for (const toolCall of message.tool_calls) {\n console.log(`- Function: ${toolCall.function.name}`);\n console.log(` Arguments: ${toolCall.function.arguments}`);\n \n // Execute the function\n const args = JSON.parse(toolCall.function.arguments);\n const result = getWeather(args.location, args.unit);\n console.log(` Result:`, result);\n\n // Add tool result to messages\n messages.push({\n role: 'tool',\n tool_call_id: toolCall.id,\n content: JSON.stringify(result)\n });\n }\n\n // Step 3: Get final response with tool results\n console.log('\\nStep 2 - Sending tool results back...');\n const finalResponse = (await client.chat.completions.create({\n model: AVAILABLE_MODELS.MIXTRAL,\n messages,\n temperature: 0.7,\n max_tokens: 200\n })) as ChatCompletionResponse;\n\n console.log('\\nStep 3 - Final response:', finalResponse.choices[0].message.content);\n }\n } catch (error) {\n console.error(\"Tool calling error:\", error);\n }\n}\n\nmain().catch(console.error);\n" }, { "path": "aisuite-js/examples/mistral.ts", "content": "import \"dotenv/config\";\nimport { Client, ChatCompletionResponse, ChatMessage } from \"../src\";\n\n// Sample data store\nconst data = {\n transactionId: [\"T1001\", \"T1002\", \"T1003\", \"T1004\", \"T1005\"],\n customerId: [\"C001\", \"C002\", \"C003\", \"C002\", \"C001\"],\n paymentAmount: [125.5, 89.99, 120.0, 54.3, 210.2],\n paymentDate: [\n \"2021-10-05\",\n \"2021-10-06\",\n \"2021-10-07\",\n \"2021-10-05\",\n \"2021-10-08\",\n ],\n paymentStatus: [\"Paid\", \"Unpaid\", \"Paid\", \"Paid\", \"Pending\"],\n};\n\n/**\n * Retrieves the payment status for a given transaction\n */\nfunction retrievePaymentStatus({ data, transactionId }) {\n const transactionIndex = data.transactionId.indexOf(transactionId);\n if (transactionIndex !== -1) {\n return JSON.stringify({ status: data.paymentStatus[transactionIndex] });\n }\n return JSON.stringify({ status: \"error - transaction id not found\" });\n}\n\n/**\n * Retrieves the payment date for a given transaction\n */\nfunction retrievePaymentDate({ data, transactionId }) {\n const transactionIndex = data.transactionId.indexOf(transactionId);\n if (transactionIndex !== -1) {\n return JSON.stringify({ date: data.paymentDate[transactionIndex] });\n }\n return JSON.stringify({ date: \"error - transaction id not found\" });\n}\n\n// Map function names to their implementations\nconst namesToFunctions = {\n retrievePaymentStatus: (transactionId) =>\n retrievePaymentStatus({ data, ...transactionId }),\n retrievePaymentDate: (transactionId) =>\n retrievePaymentDate({ data, ...transactionId }),\n};\n\n// Define available tools (functions) for the model\nconst TOOLS = [\n {\n type: \"function\",\n function: {\n name: \"retrievePaymentStatus\",\n description: \"Get payment status of a transaction id\",\n parameters: {\n type: \"object\",\n required: [\"transactionId\"],\n properties: {\n transactionId: { type: \"string\", description: \"The transaction id.\" },\n },\n },\n },\n },\n {\n type: \"function\",\n function: {\n name: \"retrievePaymentDate\",\n description: \"Get payment date of a transaction id\",\n parameters: {\n type: \"object\",\n required: [\"transactionId\"],\n properties: {\n transactionId: { type: \"string\", description: \"The transaction id.\" },\n },\n },\n },\n },\n];\n\nasync function main() {\n const client = new Client({\n mistral: { apiKey: process.env.MISTRAL_API_KEY! },\n });\n\n console.log(\"\\n🔮 Mistral Chat Examples\\n\");\n\n // Example 1: Basic chat completion\n console.log(\"--- Basic Chat Completion ---\");\n try {\n const response = (await client.chat.completions.create({\n model: \"mistral:mistral-medium\",\n messages: [\n { role: \"system\", content: \"You are a helpful assistant.\" },\n { role: \"user\", content: \"What is TypeScript in one sentence?\" },\n ],\n temperature: 0.7,\n max_tokens: 100,\n stream: false,\n })) as ChatCompletionResponse;\n\n console.log(\"Response:\", response.choices[0].message.content);\n console.log(\"Usage:\", response.usage);\n console.log(\"Full response:\", JSON.stringify(response, null, 2));\n } catch (error) {\n console.error(\"Error:\", error);\n }\n\n // Example 2: Streaming\n console.log(\"\\n--- Streaming Example ---\");\n try {\n const stream = await client.chat.completions.create({\n model: \"mistral:mistral-medium\",\n messages: [\n { role: \"system\", content: \"You are a helpful assistant.\" },\n {\n role: \"user\",\n content: \"Write a haiku about artificial intelligence.\",\n },\n ],\n stream: true,\n temperature: 0.7,\n max_tokens: 100,\n });\n\n console.log(\"Response:\");\n let fullContent = \"\";\n for await (const chunk of stream as AsyncIterable) {\n const content = chunk.choices[0]?.delta?.content || \"\";\n process.stdout.write(content);\n fullContent += content;\n }\n } catch (error) {\n console.error(\"Streaming error:\", error);\n }\n\n // Example 3: Tool calling\n console.log(\"\\n\\n--- Tool Calling Example ---\");\n try {\n const tools = [\n {\n type: \"function\",\n function: {\n name: \"retrievePaymentStatus\",\n description: \"Get payment status of a transaction id\",\n parameters: {\n type: \"object\",\n required: [\"transactionId\"],\n properties: {\n transactionId: {\n type: \"string\",\n description: \"The transaction id.\",\n },\n },\n },\n },\n },\n {\n type: \"function\",\n function: {\n name: \"retrievePaymentDate\",\n description: \"Get payment date of a transaction id\",\n parameters: {\n type: \"object\",\n required: [\"transactionId\"],\n properties: {\n transactionId: {\n type: \"string\",\n description: \"The transaction id.\",\n },\n },\n },\n },\n },\n ];\n const model = \"mistral:mistral-large-latest\";\n\n let messages: ChatMessage[] = [\n { role: \"user\", content: \"What's the status of my transaction?\" },\n ];\n\n // First interaction - Model asks for transaction ID\n let response = (await client.chat.completions.create({\n model,\n messages: [\n { role: \"user\", content: \"What's the status of my transaction?\" },\n ],\n tools: TOOLS as any, // Type assertion for Mistral's string-based tools\n })) as ChatCompletionResponse;\n\n messages.push({\n role: \"assistant\",\n content: response.choices[0].message.content as string,\n });\n\n // User provides transaction ID\n messages.push({ role: \"user\", content: \"My transaction ID is T1001.\" });\n\n // Second interaction - Model uses functions to get information\n response = (await client.chat.completions.create({\n model,\n messages,\n tools: TOOLS as any, // Type assertion for Mistral's string-based tools\n })) as ChatCompletionResponse;\n\n messages.push(response.choices[0].message);\n\n // Process tool calls\n const toolCalls = response.choices[0].message.tool_calls || [];\n for (const toolCall of toolCalls) {\n const functionName = toolCall.function.name;\n const functionParams = JSON.parse(toolCall.function.arguments);\n\n console.log(`Calling function: ${functionName}`);\n console.log(`Parameters: ${toolCall.function.arguments}`);\n\n const functionResult = namesToFunctions[functionName](functionParams);\n\n messages.push({\n role: \"tool\",\n name: functionName,\n content: functionResult,\n tool_call_id: toolCall.id,\n });\n }\n\n // Final response with the information\n response = (await client.chat.completions.create({\n model,\n messages,\n tools: TOOLS as any, // Type assertion for Mistral's string-based tools\n })) as ChatCompletionResponse;\n\n console.log(\"Final response:\", response.choices[0].message.content);\n } catch (error) {\n console.error(\"Tool calling error:\", error);\n }\n}\n\nmain().catch(console.error);\n" }, { "path": "aisuite-js/examples/openai-asr.ts", "content": "import { Client } from \"../src\";\nimport * as fs from \"fs\";\nimport * as path from \"path\";\n\nasync function main() {\n // Initialize the client with OpenAI configuration\n const client = new Client({\n openai: {\n apiKey: process.env.OPENAI_API_KEY!, \n },\n });\n\n console.log(\"Available ASR providers:\", client.listASRProviders());\n\n // Example: Transcribe an audio file\n try {\n // Path to your audio file\n const testAudioPath = path.join(\"test-audio.wav\");\n\n // Check if test file exists\n if (!fs.existsSync(testAudioPath)) {\n console.log(\n \"Test audio file not found. Please provide a valid audio file for transcription.\"\n );\n console.log(\"Expected path:\", testAudioPath);\n return;\n }\n\n const audioBuffer = fs.readFileSync(testAudioPath);\n\n // Transcribe using OpenAI Whisper model\n const result = await client.audio.transcriptions.create({\n model: \"openai:whisper-1\",\n file: audioBuffer,\n language: \"en\",\n response_format: \"verbose_json\",\n temperature: 0,\n timestamps: true,\n });\n\n console.log(\"Transcription Result:\");\n console.log(\"Text:\", result.text);\n console.log(\"Language:\", result.language);\n console.log(\"Confidence:\", result.confidence);\n\n if (result.words && result.words.length > 0) {\n console.log(\"\\nWords with timestamps:\");\n result.words.slice(0, 5).forEach((word, index) => {\n console.log(\n `${index + 1}. \"${word.text}\" (${word.start}s - ${word.end}s, confidence: ${word.confidence})`\n );\n });\n }\n\n if (result.segments && result.segments.length > 0) {\n console.log(\"\\nSegments:\");\n result.segments.slice(0, 3).forEach((segment, index) => {\n console.log(\n `${index + 1}. \"${segment.text}\" (${segment.start}s - ${segment.end}s)`\n );\n });\n }\n } catch (error) {\n console.error(\"Error:\", error);\n }\n}\n\nmain().catch(console.error);\n" }, { "path": "aisuite-js/examples/streaming.ts", "content": "import 'dotenv/config';\nimport { Client, ChatCompletionChunk } from '../src';\n\nasync function main() {\n const client = new Client({\n openai: { apiKey: process.env.OPENAI_API_KEY! },\n anthropic: { apiKey: process.env.ANTHROPIC_API_KEY! },\n });\n\n console.log('🚀 AISuite Streaming Examples\\n');\n\n // Example 1: Basic OpenAI Streaming\n console.log('--- OpenAI Streaming ---');\n try {\n const stream = await client.chat.completions.create({\n model: 'openai:gpt-3.5-turbo',\n messages: [\n { role: 'system', content: 'You are a helpful assistant.' },\n { role: 'user', content: 'Write a haiku about TypeScript' }\n ],\n stream: true,\n temperature: 0.7,\n max_tokens: 100,\n }) as AsyncIterable;\n\n console.log('Response: ');\n let fullContent = '';\n for await (const chunk of stream) {\n const content = chunk.choices[0]?.delta?.content || '';\n process.stdout.write(content);\n fullContent += content;\n }\n console.log('\\n\\nFull response:', fullContent);\n } catch (error) {\n console.error('OpenAI streaming error:', error);\n }\n\n // Example 2: Basic Anthropic Streaming\n console.log('\\n--- Anthropic Streaming ---');\n try {\n const stream = await client.chat.completions.create({\n model: 'anthropic:claude-3-haiku-20240307',\n messages: [\n { role: 'system', content: 'You are a helpful assistant.' },\n { role: 'user', content: 'Write a haiku about JavaScript' }\n ],\n stream: true,\n temperature: 0.7,\n max_tokens: 100,\n }) as AsyncIterable;\n\n console.log('Response: ');\n let fullContent = '';\n for await (const chunk of stream) {\n const content = chunk.choices[0]?.delta?.content || '';\n process.stdout.write(content);\n fullContent += content;\n }\n console.log('\\n\\nFull response:', fullContent);\n } catch (error) {\n console.error('Anthropic streaming error:', error);\n }\n\n // Example 3: Streaming with Progress Indicator\n console.log('\\n--- Streaming with Progress ---');\n try {\n const stream = await client.chat.completions.create({\n model: 'openai:gpt-3.5-turbo',\n messages: [\n { role: 'user', content: 'Count from 1 to 10 slowly' }\n ],\n stream: true,\n temperature: 0,\n max_tokens: 100,\n }) as AsyncIterable;\n\n console.log('Response: ');\n let charCount = 0;\n for await (const chunk of stream) {\n const content = chunk.choices[0]?.delta?.content || '';\n process.stdout.write(content);\n charCount += content.length;\n \n // Show progress in title (if supported)\n if (process.stdout.isTTY) {\n process.stdout.write(`\\x1b]0;Streaming: ${charCount} chars\\x07`);\n }\n }\n console.log(`\\n\\nTotal characters: ${charCount}`);\n } catch (error) {\n console.error('Streaming error:', error);\n }\n\n // Example 4: Abort Controller\n console.log('\\n--- Streaming with Abort Controller ---');\n try {\n const controller = new AbortController();\n \n // Abort after 2 seconds\n const timeout = setTimeout(() => {\n console.log('\\n\\n⏹️ Aborting stream...');\n controller.abort();\n }, 2000);\n\n const stream = await client.chat.completions.create({\n model: 'anthropic:claude-3-haiku-20240307',\n messages: [\n { role: 'user', content: 'Tell me a very long story about a programmer' }\n ],\n stream: true,\n temperature: 0.7,\n max_tokens: 500,\n }, { signal: controller.signal }) as AsyncIterable;\n\n console.log('Response (will abort after 2 seconds): ');\n try {\n for await (const chunk of stream) {\n const content = chunk.choices[0]?.delta?.content || '';\n process.stdout.write(content);\n }\n } catch (error: any) {\n if (error.name === 'AbortError') {\n console.log('\\n\\n✅ Stream successfully aborted');\n } else {\n throw error;\n }\n } finally {\n clearTimeout(timeout);\n }\n } catch (error) {\n console.error('Abort controller error:', error);\n }\n\n // Example 5: Streaming with Tool Calls\n console.log('\\n--- Streaming with Tool Calls ---');\n try {\n const tools = [{\n type: 'function' as const,\n function: {\n name: 'get_weather',\n description: 'Get current weather',\n parameters: {\n type: 'object',\n properties: {\n location: { type: 'string' }\n },\n required: ['location']\n }\n }\n }];\n\n const stream = await client.chat.completions.create({\n model: 'openai:gpt-4o-mini',\n messages: [\n { role: 'user', content: 'What\\'s the weather in Tokyo?' }\n ],\n tools,\n tool_choice: 'auto',\n stream: true,\n }) as AsyncIterable;\n\n console.log('Streaming response with potential tool calls:\\n');\n \n let currentToolCall: any = null;\n let toolCalls: any[] = [];\n \n for await (const chunk of stream) {\n // Handle text content\n const content = chunk.choices[0]?.delta?.content;\n if (content) {\n process.stdout.write(content);\n }\n \n // Handle tool calls\n const deltaToolCalls = chunk.choices[0]?.delta?.tool_calls;\n if (deltaToolCalls) {\n for (const toolCall of deltaToolCalls) {\n if (toolCall.id) {\n // New tool call\n currentToolCall = {\n id: toolCall.id,\n type: toolCall.type,\n function: {\n name: toolCall.function?.name || '',\n arguments: toolCall.function?.arguments || ''\n }\n };\n toolCalls.push(currentToolCall);\n } else if (currentToolCall && toolCall.function?.arguments) {\n // Accumulate arguments\n currentToolCall.function.arguments += toolCall.function.arguments;\n }\n }\n }\n \n // Check if we're done\n if (chunk.choices[0]?.finish_reason === 'tool_calls') {\n console.log('\\n\\nTool calls detected:');\n for (const tc of toolCalls) {\n console.log(`- ${tc.function.name}: ${tc.function.arguments}`);\n }\n }\n }\n } catch (error) {\n console.error('Streaming with tools error:', error);\n }\n\n console.log('\\n\\n✨ Streaming examples completed!');\n}\n\n// Run examples\nmain().catch(console.error);" }, { "path": "aisuite-js/examples/test-suite.ts", "content": "import 'dotenv/config';\nimport { Client, AISuiteError, ProviderNotConfiguredError, ChatCompletionChunk } from '../src';\n\n// Test configuration\nconst ENABLE_OPENAI_TESTS = !!process.env.OPENAI_API_KEY && process.env.OPENAI_API_KEY !== 'your-openai-api-key-here';\nconst ENABLE_ANTHROPIC_TESTS = !!process.env.ANTHROPIC_API_KEY && process.env.ANTHROPIC_API_KEY !== 'your-anthropic-api-key-here';\n\nconsole.log('🧪 AISuite Test Suite\\n');\nconsole.log(`OpenAI tests: ${ENABLE_OPENAI_TESTS ? '✅ Enabled' : '❌ Disabled (no API key)'}`);\nconsole.log(`Anthropic tests: ${ENABLE_ANTHROPIC_TESTS ? '✅ Enabled' : '❌ Disabled (no API key)'}`);\nconsole.log('');\n\n// Helper function to run a test\nasync function runTest(name: string, fn: () => Promise) {\n try {\n await fn();\n console.log(`✅ ${name}`);\n } catch (error) {\n console.error(`❌ ${name}`);\n console.error(` Error: ${error instanceof Error ? error.message : error}`);\n }\n}\n\nasync function main() {\n // Initialize client\n const client = new Client({\n ...(ENABLE_OPENAI_TESTS && { openai: { apiKey: process.env.OPENAI_API_KEY! } }),\n ...(ENABLE_ANTHROPIC_TESTS && { anthropic: { apiKey: process.env.ANTHROPIC_API_KEY! } }),\n });\n\n console.log('📋 Available providers:', client.listProviders());\n console.log('');\n\n // Test 1: Basic functionality\n console.log('🔍 Testing basic functionality...\\n');\n \n await runTest('Model parser - valid format', async () => {\n const { parseModel } = await import('../src');\n const result = parseModel('openai:gpt-4');\n if (result.provider !== 'openai' || result.model !== 'gpt-4') {\n throw new Error('Model parser failed');\n }\n });\n\n await runTest('Model parser - invalid format', async () => {\n const { parseModel } = await import('../src');\n try {\n parseModel('invalid-format');\n throw new Error('Should have thrown error');\n } catch (error) {\n if (!(error instanceof Error) || !error.message.includes('Invalid model format')) {\n throw error;\n }\n }\n });\n\n await runTest('Provider not configured error', async () => {\n try {\n await client.chat.completions.create({\n model: 'invalid:model',\n messages: [{ role: 'user', content: 'test' }]\n });\n throw new Error('Should have thrown error');\n } catch (error) {\n if (!(error instanceof ProviderNotConfiguredError)) {\n throw error;\n }\n }\n });\n\n // Test 2: OpenAI Provider\n if (ENABLE_OPENAI_TESTS) {\n console.log('\\n🤖 Testing OpenAI provider...\\n');\n\n await runTest('OpenAI - Basic chat completion', async () => {\n const response = await client.chat.completions.create({\n model: 'openai:gpt-3.5-turbo',\n messages: [\n { role: 'system', content: 'You are a test assistant. Respond with exactly: \"Test successful\"' },\n { role: 'user', content: 'Hello' }\n ],\n temperature: 0,\n max_tokens: 50,\n });\n\n if (!response.choices[0]?.message?.content) {\n throw new Error('No response content');\n }\n console.log(` Response: ${response.choices[0].message.content.trim()}`);\n });\n\n await runTest('OpenAI - Multiple messages', async () => {\n const response = await client.chat.completions.create({\n model: 'openai:gpt-3.5-turbo',\n messages: [\n { role: 'system', content: 'You are a helpful assistant.' },\n { role: 'user', content: 'Say \"A\"' },\n { role: 'assistant', content: 'A' },\n { role: 'user', content: 'Say \"B\"' }\n ],\n temperature: 0,\n max_tokens: 10,\n });\n\n if (!response.choices[0]?.message?.content?.includes('B')) {\n throw new Error('Multi-turn conversation failed');\n }\n });\n\n await runTest('OpenAI - Tool calling', async () => {\n const response = await client.chat.completions.create({\n model: 'openai:gpt-3.5-turbo',\n messages: [\n { role: 'user', content: 'What is the weather in San Francisco?' }\n ],\n tools: [{\n type: 'function',\n function: {\n name: 'get_weather',\n description: 'Get weather for a location',\n parameters: {\n type: 'object',\n properties: {\n location: { type: 'string' }\n },\n required: ['location']\n }\n }\n }],\n tool_choice: 'auto'\n });\n\n const toolCalls = response.choices[0]?.message?.tool_calls;\n if (!toolCalls || toolCalls.length === 0) {\n throw new Error('No tool calls in response');\n }\n console.log(` Tool called: ${toolCalls[0].function.name}`);\n });\n\n await runTest('OpenAI - Streaming support', async () => {\n const stream = await client.chat.completions.create({\n model: 'openai:gpt-3.5-turbo',\n messages: [{ role: 'user', content: 'Say exactly: \"Stream test\"' }],\n stream: true,\n temperature: 0,\n max_tokens: 20,\n }) as AsyncIterable;\n\n let content = '';\n for await (const chunk of stream) {\n content += chunk.choices[0]?.delta?.content || '';\n }\n \n if (!content.toLowerCase().includes('stream')) {\n throw new Error('Streaming response did not contain expected content');\n }\n console.log(` Streamed: ${content.trim()}`);\n });\n }\n\n // Test 3: Anthropic Provider\n if (ENABLE_ANTHROPIC_TESTS) {\n console.log('\\n🔮 Testing Anthropic provider...\\n');\n\n await runTest('Anthropic - Basic chat completion', async () => {\n const response = await client.chat.completions.create({\n model: 'anthropic:claude-3-haiku-20240307',\n messages: [\n { role: 'system', content: 'You are a test assistant. Respond with exactly: \"Test successful\"' },\n { role: 'user', content: 'Hello' }\n ],\n temperature: 0,\n max_tokens: 50,\n });\n\n if (!response.choices[0]?.message?.content) {\n throw new Error('No response content');\n }\n console.log(` Response: ${response.choices[0].message.content.trim()}`);\n });\n\n await runTest('Anthropic - System message handling', async () => {\n const response = await client.chat.completions.create({\n model: 'anthropic:claude-3-haiku-20240307',\n messages: [\n { role: 'system', content: 'Always respond in uppercase.' },\n { role: 'system', content: 'Also end with an exclamation mark.' },\n { role: 'user', content: 'say hello' }\n ],\n temperature: 0,\n max_tokens: 50,\n });\n\n const content = response.choices[0]?.message?.content || '';\n if (!content.includes('!') || content !== content.toUpperCase()) {\n console.log(` Warning: System message may not be properly handled. Response: ${content}`);\n }\n });\n\n await runTest('Anthropic - Tool calling', async () => {\n const response = await client.chat.completions.create({\n model: 'anthropic:claude-3-haiku-20240307',\n messages: [\n { role: 'user', content: 'What is the weather in Paris?' }\n ],\n tools: [{\n type: 'function',\n function: {\n name: 'get_weather',\n description: 'Get weather for a location',\n parameters: {\n type: 'object',\n properties: {\n location: { type: 'string', description: 'The city name' }\n },\n required: ['location']\n }\n }\n }],\n tool_choice: 'auto'\n });\n\n const toolCalls = response.choices[0]?.message?.tool_calls;\n if (!toolCalls || toolCalls.length === 0) {\n throw new Error('No tool calls in response');\n }\n console.log(` Tool called: ${toolCalls[0].function.name}`);\n console.log(` Arguments: ${toolCalls[0].function.arguments}`);\n });\n\n await runTest('Anthropic - Streaming support', async () => {\n const stream = await client.chat.completions.create({\n model: 'anthropic:claude-3-haiku-20240307',\n messages: [{ role: 'user', content: 'Say exactly: \"Stream works\"' }],\n stream: true,\n temperature: 0,\n max_tokens: 20,\n }) as AsyncIterable;\n\n let content = '';\n for await (const chunk of stream) {\n content += chunk.choices[0]?.delta?.content || '';\n }\n \n if (!content.toLowerCase().includes('stream')) {\n throw new Error('Streaming response did not contain expected content');\n }\n console.log(` Streamed: ${content.trim()}`);\n });\n }\n\n // Test 4: Cross-provider compatibility\n if (ENABLE_OPENAI_TESTS && ENABLE_ANTHROPIC_TESTS) {\n console.log('\\n🔄 Testing cross-provider compatibility...\\n');\n\n await runTest('Same prompt - different providers', async () => {\n const prompt = {\n messages: [\n { role: 'system', content: 'You are a helpful assistant. Be concise.' },\n { role: 'user', content: 'What is 2+2?' }\n ] as const,\n temperature: 0,\n max_tokens: 50,\n };\n\n const openaiResponse = await client.chat.completions.create({\n ...prompt,\n model: 'openai:gpt-3.5-turbo',\n });\n\n const anthropicResponse = await client.chat.completions.create({\n ...prompt,\n model: 'anthropic:claude-3-haiku-20240307',\n });\n\n console.log(` OpenAI: ${openaiResponse.choices[0].message.content?.trim()}`);\n console.log(` Anthropic: ${anthropicResponse.choices[0].message.content?.trim()}`);\n \n // Both should mention \"4\" in their response\n if (!openaiResponse.choices[0].message.content?.includes('4') || \n !anthropicResponse.choices[0].message.content?.includes('4')) {\n throw new Error('Providers gave inconsistent results');\n }\n });\n }\n\n console.log('\\n✨ Test suite completed!\\n');\n}\n\n// Run tests\nmain().catch(error => {\n console.error('\\n💥 Test suite failed:', error);\n process.exit(1);\n});" }, { "path": "aisuite-js/examples/tool-calling.ts", "content": "import 'dotenv/config';\nimport { Client, ChatMessage } from '../src';\n\n// Mock function for weather\nfunction getWeather(location: string, unit: 'celsius' | 'fahrenheit' = 'celsius') {\n // Mock implementation\n return {\n location,\n temperature: unit === 'celsius' ? 22 : 72,\n condition: 'sunny',\n unit\n };\n}\n\nasync function main() {\n const client = new Client({\n openai: { apiKey: process.env.OPENAI_API_KEY! },\n anthropic: { apiKey: process.env.ANTHROPIC_API_KEY! },\n });\n\n // Define tools in OpenAI format\n const tools = [\n {\n type: 'function' as const,\n function: {\n name: 'get_weather',\n description: 'Get the current weather for a location',\n parameters: {\n type: 'object',\n properties: {\n location: {\n type: 'string',\n description: 'The city and state, e.g. San Francisco, CA'\n },\n unit: {\n type: 'string',\n enum: ['celsius', 'fahrenheit'],\n description: 'The temperature unit'\n }\n },\n required: ['location']\n }\n }\n }\n ];\n\n // Example 1: OpenAI Tool Calling\n console.log('--- OpenAI Tool Calling ---');\n try {\n // Step 1: Initial request with tools\n const response = await client.chat.completions.create({\n model: 'openai:gpt-4o-mini',\n messages: [\n { role: 'system', content: 'You are a helpful weather assistant.' },\n { role: 'user', content: \"What's the weather like in San Francisco?\" }\n ],\n tools,\n tool_choice: 'auto'\n });\n\n const message = response.choices[0]?.message;\n console.log('Step 1 - Initial response:', JSON.stringify(message, null, 2));\n\n if (message?.tool_calls) {\n // Step 2: Execute tool calls and send results back\n const messages: ChatMessage[] = [\n { role: 'system', content: 'You are a helpful weather assistant.' },\n { role: 'user', content: \"What's the weather like in San Francisco?\" },\n message // The assistant's message with tool calls\n ];\n\n console.log('\\nTool calls detected:');\n for (const toolCall of message.tool_calls) {\n console.log(`- Function: ${toolCall.function.name}`);\n console.log(` Arguments: ${toolCall.function.arguments}`);\n \n // Execute the function\n const args = JSON.parse(toolCall.function.arguments);\n const result = getWeather(args.location, args.unit);\n console.log(` Result:`, result);\n\n // Add tool result to messages\n messages.push({\n role: 'tool',\n tool_call_id: toolCall.id,\n content: JSON.stringify(result)\n });\n }\n\n // Step 3: Get final response with tool results\n console.log('\\nStep 2 - Sending tool results back...');\n const finalResponse = await client.chat.completions.create({\n model: 'openai:gpt-4o-mini',\n messages,\n temperature: 0.7,\n max_tokens: 200\n });\n\n console.log('\\nStep 3 - Final response:', finalResponse.choices[0].message.content);\n }\n } catch (error) {\n console.error('OpenAI Tool Calling Error:', error);\n }\n\n // Example 2: Anthropic Tool Calling\n console.log('\\n--- Anthropic Tool Calling ---');\n try {\n // Step 1: Initial request with tools\n const response = await client.chat.completions.create({\n model: 'anthropic:claude-3-haiku-20240307',\n messages: [\n { role: 'system', content: 'You are a helpful weather assistant.' },\n { role: 'user', content: \"What's the weather like in New York?\" }\n ],\n tools,\n tool_choice: 'auto'\n });\n\n const message = response.choices[0]?.message;\n console.log('Step 1 - Initial response:', JSON.stringify(message, null, 2));\n\n if (message?.tool_calls) {\n // Step 2: Execute tool calls and send results back\n const messages: ChatMessage[] = [\n { role: 'system', content: 'You are a helpful weather assistant.' },\n { role: 'user', content: \"What's the weather like in New York?\" },\n message // The assistant's message with tool calls\n ];\n\n console.log('\\nTool calls detected:');\n for (const toolCall of message.tool_calls) {\n console.log(`- Function: ${toolCall.function.name}`);\n console.log(` Arguments: ${toolCall.function.arguments}`);\n \n // Execute the function\n const args = JSON.parse(toolCall.function.arguments);\n const result = getWeather(args.location, args.unit);\n console.log(` Result:`, result);\n\n // Add tool result to messages\n messages.push({\n role: 'tool',\n tool_call_id: toolCall.id,\n content: JSON.stringify(result)\n });\n }\n\n // Step 3: Get final response with tool results\n console.log('\\nStep 2 - Sending tool results back...');\n const finalResponse = await client.chat.completions.create({\n model: 'anthropic:claude-3-haiku-20240307',\n messages,\n temperature: 0.7,\n max_tokens: 200\n });\n\n console.log('\\nStep 3 - Final response:', finalResponse.choices[0].message.content);\n }\n } catch (error) {\n console.error('Anthropic Tool Calling Error:', error);\n }\n}\n\nmain().catch(console.error);" }, { "path": "aisuite-js/jest.config.ts", "content": "export default {\n preset: 'ts-jest',\n testEnvironment: 'node',\n roots: ['/tests'],\n testMatch: [\n '**/__tests__/**/*.ts',\n '**/?(*.)+(spec|test).ts'\n ],\n transform: {\n '^.+\\\\.ts$': 'ts-jest',\n '^.+\\\\.js$': 'babel-jest',\n },\n transformIgnorePatterns: [\n 'node_modules/(?!(@mistralai|@anthropic-ai|groq-sdk|openai)/)'\n ],\n extensionsToTreatAsEsm: ['.ts'],\n globals: {\n 'ts-jest': {\n useESM: true,\n },\n },\n collectCoverageFrom: [\n 'src/**/*.ts',\n '!src/**/*.d.ts',\n ],\n coverageDirectory: 'coverage',\n coverageReporters: ['text', 'lcov', 'html'],\n moduleNameMapper: {\n '^@/(.*)$': '/src/$1',\n },\n setupFilesAfterEnv: [],\n testTimeout: 10000,\n};" }, { "path": "aisuite-js/package.json", "content": "{\n \"name\": \"aisuite\",\n \"version\": \"0.1.1\",\n \"description\": \"Unified TypeScript library for multiple LLM providers\",\n \"main\": \"dist/index.js\",\n \"types\": \"dist/index.d.ts\",\n \"scripts\": {\n \"build\": \"tsc\",\n \"test\": \"jest --config=jest.config.ts\",\n \"test:examples\": \"tsx examples/test-suite.ts\",\n \"example:basic\": \"tsx examples/basic-usage.ts\",\n \"example:tools\": \"tsx examples/tool-calling.ts\",\n \"example:streaming\": \"tsx examples/streaming.ts\",\n \"example:mistral\": \"tsx examples/mistral.ts\",\n \"example:groq\": \"tsx examples/groq.ts\",\n \"example:deepgram\": \"tsx examples/deepgram.ts\",\n \"example:openai-asr\": \"tsx examples/openai-asr.ts\",\n \"lint\": \"eslint src/**/*.ts\",\n \"prepublishOnly\": \"npm run build\",\n \"dev\": \"tsc --watch\"\n },\n \"dependencies\": {\n \"@anthropic-ai/sdk\": \"^0.56.0\",\n \"@deepgram/sdk\": \"^4.11.2\",\n \"@mistralai/mistralai\": \"^0.1.3\",\n \"groq-sdk\": \"^0.29.0\",\n \"openai\": \"^4.0.0\"\n },\n \"peerDependencies\": {\n \"typescript\": \">=4.5.0\"\n },\n \"devDependencies\": {\n \"@types/jest\": \"^29.0.0\",\n \"@types/node\": \"^20.0.0\",\n \"@typescript-eslint/eslint-plugin\": \"^6.0.0\",\n \"@typescript-eslint/parser\": \"^6.0.0\",\n \"dotenv\": \"^16.0.0\",\n \"eslint\": \"^8.0.0\",\n \"jest\": \"^29.0.0\",\n \"ts-jest\": \"^29.0.0\",\n \"tsx\": \"^4.0.0\",\n \"typescript\": \"^5.0.0\"\n },\n \"keywords\": [\n \"llm\",\n \"openai\",\n \"anthropic\",\n \"claude\",\n \"gpt\",\n \"ai\",\n \"typescript\"\n ],\n \"author\": \"Andrew Ng and Rohit P\",\n \"license\": \"MIT\",\n \"engines\": {\n \"node\": \">=16.0.0\"\n },\n \"files\": [\n \"dist/**/*\",\n \"README.md\",\n \"LICENSE\"\n ],\n \"repository\": {\n \"type\": \"git\",\n \"url\": \"git+https://github.com/andrewyng/aisuite.git\"\n },\n \"bugs\": {\n \"url\": \"https://github.com/andrewyng/aisuite/issues\"\n },\n \"homepage\": \"https://github.com/andrewyng/aisuite#readme\",\n \"type\": \"module\"\n}\n" }, { "path": "aisuite-js/src/asr-providers/deepgram/adapters.ts", "content": "import { TranscriptionResult, Word, Segment } from \"../../types\";\n\nexport function adaptResponse(response: any): TranscriptionResult {\n const words: Word[] = [];\n const segments: Segment[] = [];\n\n // Handle Deepgram response structure\n if (response.results?.channels?.[0]?.alternatives?.[0]) {\n const alternative = response.results.channels[0].alternatives[0];\n\n // Extract words with timestamps and confidence\n if (alternative.words) {\n alternative.words.forEach((word: any) => {\n words.push({\n text: word.word,\n start: word.start,\n end: word.end,\n confidence: word.confidence,\n speaker: word.speaker?.toString(),\n });\n });\n }\n\n // Extract utterances/segments\n if (response.results.utterances) {\n response.results.utterances.forEach((utterance: any) => {\n segments.push({\n text: utterance.transcript,\n start: utterance.start,\n end: utterance.end,\n speaker: utterance.speaker?.toString(),\n });\n });\n }\n\n return {\n text: alternative.transcript,\n language: response.metadata?.language || \"unknown\",\n confidence: alternative.confidence,\n words,\n segments,\n };\n }\n\n // Fallback for unexpected response structure\n return {\n text: response.transcript || \"\",\n language: \"unknown\",\n confidence: undefined,\n words: [],\n segments: [],\n };\n}\n" }, { "path": "aisuite-js/src/asr-providers/deepgram/index.ts", "content": "export { DeepgramASRProvider } from \"./provider\";\nexport type { DeepgramConfig } from \"./types\";\n" }, { "path": "aisuite-js/src/asr-providers/deepgram/provider.ts", "content": "import { createClient, DeepgramClient } from \"@deepgram/sdk\";\nimport { ASRProvider } from \"../../core/base-asr-provider\";\nimport {\n TranscriptionRequest,\n TranscriptionResult,\n RequestOptions,\n} from \"../../types\";\nimport { DeepgramConfig } from \"./types\";\nimport { adaptResponse } from \"./adapters\";\nimport { AISuiteError } from \"../../core/errors\";\nimport * as fs from \"fs\";\n\nexport class DeepgramASRProvider implements ASRProvider {\n public readonly name = \"deepgram\";\n private client: DeepgramClient;\n\n constructor(config: DeepgramConfig) {\n // Use the new createClient API instead of the deprecated Deepgram constructor\n this.client = createClient({\n key: config.apiKey,\n ...(config.baseURL && { baseUrl: config.baseURL }),\n });\n }\n\n validateParams(params: { [key: string]: any }): void {\n if (!params.model) {\n throw new AISuiteError(\n \"Model parameter is required\",\n this.name,\n \"MODEL_PARAMETER_REQUIRED\"\n );\n }\n\n if (!params.file) {\n throw new AISuiteError(\n \"File parameter is required\",\n this.name,\n \"MODEL_PARAMETER_REQUIRED\"\n );\n }\n }\n\n translateParams(params: { [key: string]: any }): { [key: string]: any } {\n const { model: _, file: __, ...rest } = params;\n return Object.entries(rest).reduce((translated, [key, value]) => {\n switch (key) {\n case \"timestamps\":\n if (value) translated.utterances = true;\n break;\n default:\n translated[key] = value;\n }\n return translated;\n }, {} as { [key: string]: any });\n }\n\n async transcribe(\n request: TranscriptionRequest,\n options?: RequestOptions\n ): Promise {\n try {\n // Extract parameters excluding model and file\n const { model, file, ...params } = request;\n this.validateParams(request);\n const translatedParams = this.translateParams(params);\n\n // Handle different input types\n let audioData: Buffer;\n if (typeof request.file === \"string\") {\n audioData = fs.readFileSync(request.file);\n } else if (Buffer.isBuffer(request.file)) {\n audioData = request.file;\n } else if (request.file instanceof Uint8Array) {\n audioData = Buffer.from(request.file);\n } else {\n throw new AISuiteError(\n \"Unsupported audio input type\",\n this.name,\n \"INVALID_INPUT\"\n );\n }\n\n // Set up transcription options for v4 SDK format\n const transcriptionOptions = {\n model: request.model,\n ...translatedParams\n };\n\n // Use the v4 SDK format for transcription\n const response = await this.client.listen.prerecorded\n .transcribeFile(audioData, {\n ...transcriptionOptions\n });\n\n // Check for errors in the response\n if (response.error) {\n throw new AISuiteError(\n `Deepgram API error: ${response.error.message}`,\n this.name,\n \"API_ERROR\"\n );\n }\n\n return adaptResponse(response.result);\n } catch (error) {\n if (error instanceof AISuiteError) {\n throw error;\n }\n throw new AISuiteError(\n `Deepgram ASR error: ${\n error instanceof Error ? error.message : \"Unknown error\"\n }`,\n this.name,\n \"API_ERROR\"\n );\n }\n }\n}\n" }, { "path": "aisuite-js/src/asr-providers/deepgram/types.ts", "content": "export interface DeepgramConfig {\n apiKey: string;\n baseURL?: string;\n}\n" }, { "path": "aisuite-js/src/asr-providers/index.ts", "content": "export { DeepgramASRProvider } from \"./deepgram\";\nexport type { DeepgramConfig } from \"../types\";\n" }, { "path": "aisuite-js/src/client.ts", "content": "import {\n ChatCompletionRequest,\n ChatCompletionResponse,\n ChatCompletionChunk,\n ProviderConfigs,\n RequestOptions,\n TranscriptionRequest,\n TranscriptionResult,\n} from \"./types\";\nimport { Provider } from \"./core/base-provider\";\nimport { ASRProvider } from \"./core/base-asr-provider\";\nimport { parseModel } from \"./core/model-parser\";\nimport { ProviderNotConfiguredError } from \"./core/errors\";\nimport { OpenAIProvider } from \"./providers/openai\";\nimport { AnthropicProvider } from \"./providers/anthropic\";\nimport { MistralProvider } from \"./providers/mistral\";\nimport { GroqProvider } from \"./providers/groq\";\nimport { DeepgramASRProvider } from \"./asr-providers/deepgram\";\n\nexport class Client {\n private chatProviders: Map = new Map();\n private asrProviders: Map = new Map();\n\n constructor(config: ProviderConfigs) {\n this.initializeProviders(config);\n }\n\n private initializeProviders(config: ProviderConfigs): void {\n if (config.openai) {\n const openaiProvider = new OpenAIProvider(config.openai);\n this.chatProviders.set(\"openai\", openaiProvider);\n this.asrProviders.set(\"openai\", openaiProvider);\n }\n\n if (config.anthropic) {\n this.chatProviders.set(\n \"anthropic\",\n new AnthropicProvider(config.anthropic)\n );\n }\n\n if (config.mistral) {\n this.chatProviders.set(\"mistral\", new MistralProvider(config.mistral));\n }\n\n if (config.groq) {\n this.chatProviders.set(\"groq\", new GroqProvider(config.groq));\n }\n\n if (config.deepgram) {\n this.asrProviders.set(\n \"deepgram\",\n new DeepgramASRProvider(config.deepgram)\n );\n }\n }\n\n public chat = {\n completions: {\n create: async (\n request: ChatCompletionRequest,\n options?: RequestOptions\n ): Promise<\n ChatCompletionResponse | AsyncIterable\n > => {\n const { provider, model } = parseModel(request.model);\n const providerInstance = this.chatProviders.get(provider);\n\n if (!providerInstance) {\n throw new ProviderNotConfiguredError(\n provider,\n Array.from(this.chatProviders.keys())\n );\n }\n\n const requestWithParsedModel = {\n ...request,\n model, // Just the model name without provider prefix\n };\n\n if (request.stream) {\n return providerInstance.streamChatCompletion(\n requestWithParsedModel,\n options\n );\n } else {\n return providerInstance.chatCompletion(\n requestWithParsedModel,\n options\n );\n }\n },\n },\n };\n\n public audio = {\n transcriptions: {\n create: async (\n request: TranscriptionRequest,\n options?: RequestOptions\n ): Promise => {\n const { provider, model } = parseModel(request.model);\n const providerInstance = this.asrProviders.get(provider);\n\n if (!providerInstance) {\n throw new ProviderNotConfiguredError(\n provider,\n Array.from(this.asrProviders.keys())\n );\n }\n\n const requestWithParsedModel = {\n ...request,\n model, // Just the model name without provider prefix\n };\n\n return providerInstance.transcribe(requestWithParsedModel, options);\n },\n },\n };\n\n public listProviders(): string[] {\n return Array.from(this.chatProviders.keys());\n }\n\n public listASRProviders(): string[] {\n return Array.from(this.asrProviders.keys());\n }\n\n public isProviderConfigured(provider: string): boolean {\n return this.chatProviders.has(provider);\n }\n\n public isASRProviderConfigured(provider: string): boolean {\n return this.asrProviders.has(provider);\n }\n}\n" }, { "path": "aisuite-js/src/core/base-asr-provider.ts", "content": "import { \n TranscriptionRequest, \n TranscriptionResult,\n RequestOptions \n} from '../types';\n\nexport interface ASRProvider {\n readonly name: string;\n \n transcribe(\n request: TranscriptionRequest,\n options?: RequestOptions\n ): Promise;\n \n validateParams( \n params: { [key: string]: any }\n ): void;\n \n translateParams( \n params: { [key: string]: any }\n ): { [key: string]: any };\n}" }, { "path": "aisuite-js/src/core/base-provider.ts", "content": "import { \n ChatCompletionRequest, \n ChatCompletionResponse, \n ChatCompletionChunk,\n RequestOptions \n} from '../types';\n\nexport interface Provider {\n readonly name: string;\n \n chatCompletion(\n request: ChatCompletionRequest,\n options?: RequestOptions\n ): Promise;\n \n streamChatCompletion(\n request: ChatCompletionRequest,\n options?: RequestOptions\n ): AsyncIterable;\n}" }, { "path": "aisuite-js/src/core/errors.ts", "content": "export class AISuiteError extends Error {\n constructor(\n message: string,\n public provider: string,\n public code?: string,\n public statusCode?: number\n ) {\n super(message);\n this.name = \"AISuiteError\";\n }\n}\n\nexport class ProviderNotConfiguredError extends AISuiteError {\n constructor(provider: string, availableProviders: string[]) {\n super(\n `Provider '${provider}' not configured. Available: ${availableProviders.join(\n \", \"\n )}`,\n provider,\n \"PROVIDER_NOT_CONFIGURED\"\n );\n }\n}\n\nexport class InvalidModelFormatError extends AISuiteError {\n constructor(model: string) {\n super(\n `Invalid model format: ${model}. Expected \"provider:model\"`,\n \"unknown\",\n \"INVALID_MODEL_FORMAT\"\n );\n }\n}\n\nexport class ToolCallError extends AISuiteError {\n constructor(message: string, provider: string) {\n super(message, provider, \"TOOL_CALL_ERROR\");\n }\n}\n\nexport class AudioProcessingError extends AISuiteError {\n constructor(message: string, provider: string) {\n super(message, provider, \"AUDIO_PROCESSING_ERROR\");\n this.name = \"AudioProcessingError\";\n }\n}\n\nexport class UnsupportedParameterError extends AISuiteError {\n constructor(parameter: string, provider: string) {\n super(\n `Parameter '${parameter}' is not supported by provider '${provider}'`,\n provider,\n 'UNSUPPORTED_PARAMETER'\n );\n this.name = 'UnsupportedParameterError';\n }\n}\n" }, { "path": "aisuite-js/src/core/model-parser.ts", "content": "import { InvalidModelFormatError } from './errors';\n\nexport interface ParsedModel {\n provider: string;\n model: string;\n}\n\nexport function parseModel(model: string): ParsedModel {\n if (!model || typeof model !== 'string') {\n throw new InvalidModelFormatError(model);\n }\n\n const [provider, ...modelParts] = model.split(':');\n \n if (!provider || modelParts.length === 0) {\n throw new InvalidModelFormatError(model);\n }\n \n return {\n provider,\n model: modelParts.join(':') // Handle cases like \"openai:gpt-4:vision\"\n };\n}" }, { "path": "aisuite-js/src/index.ts", "content": "export { Client } from \"./client\";\nexport * from \"./types\";\nexport * from \"./core/errors\";\nexport { parseModel } from \"./core/model-parser\";\n\n// Re-export providers for advanced usage\nexport {\n OpenAIProvider,\n AnthropicProvider,\n GroqProvider,\n MistralProvider,\n} from \"./providers\";\n\nexport { DeepgramASRProvider } from \"./asr-providers\";\n" }, { "path": "aisuite-js/src/providers/anthropic/adapters.ts", "content": "import { \n ChatCompletionRequest, \n ChatCompletionResponse, \n ChatCompletionChunk,\n ChatMessage,\n Tool,\n ToolCall\n} from '../../types';\nimport type { \n Message, \n MessageCreateParams,\n MessageStreamEvent\n} from '@anthropic-ai/sdk/resources/messages';\nimport { generateId, createChunk } from '../../utils/streaming';\n\nexport function adaptRequest(request: ChatCompletionRequest): MessageCreateParams {\n const { systemMessage, userMessages } = transformMessages(request.messages);\n \n // Don't pass stream parameter to avoid accidental streaming\n const params: MessageCreateParams = {\n model: request.model,\n max_tokens: request.max_tokens || 1024,\n messages: userMessages,\n temperature: request.temperature,\n top_p: request.top_p,\n stop_sequences: Array.isArray(request.stop) ? request.stop : request.stop ? [request.stop] : undefined,\n };\n\n if (systemMessage) {\n params.system = systemMessage;\n }\n\n if (request.tools) {\n params.tools = request.tools.map(adaptTool);\n }\n\n return params;\n}\n\nfunction transformMessages(messages: ChatMessage[]) {\n const systemMessages = messages.filter(msg => msg.role === 'system');\n const otherMessages = messages.filter(msg => msg.role !== 'system');\n \n const systemMessage = systemMessages.map(msg => msg.content).join('\\n') || undefined;\n \n const userMessages = otherMessages.map(msg => {\n if (msg.role === 'tool') {\n // Transform tool response to user message with tool_result\n return {\n role: 'user' as const,\n content: [\n {\n type: 'tool_result' as const,\n tool_use_id: msg.tool_call_id!,\n content: msg.content!,\n }\n ]\n };\n }\n \n if (msg.role === 'assistant' && msg.tool_calls) {\n // Transform assistant message with tool calls\n const content: any[] = [];\n \n if (msg.content) {\n content.push({\n type: 'text',\n text: msg.content\n });\n }\n \n msg.tool_calls.forEach(toolCall => {\n content.push({\n type: 'tool_use',\n id: toolCall.id,\n name: toolCall.function.name,\n input: JSON.parse(toolCall.function.arguments)\n });\n });\n \n return {\n role: 'assistant' as const,\n content\n };\n }\n \n return {\n role: msg.role as 'user' | 'assistant',\n content: msg.content!,\n };\n });\n \n return {\n systemMessage,\n userMessages,\n };\n}\n\nfunction adaptTool(tool: Tool): any {\n return {\n name: tool.function.name,\n description: tool.function.description,\n input_schema: {\n type: 'object',\n properties: tool.function.parameters.properties,\n required: tool.function.parameters.required,\n },\n };\n}\n\nexport function adaptResponse(response: Message, originalModel: string): ChatCompletionResponse {\n const content = Array.isArray(response.content) \n ? response.content.find(block => block.type === 'text')?.text || ''\n : response.content;\n\n const toolCalls: ToolCall[] = [];\n if (Array.isArray(response.content)) {\n response.content.forEach(block => {\n if (block.type === 'tool_use') {\n toolCalls.push({\n id: block.id,\n type: 'function',\n function: {\n name: block.name,\n arguments: JSON.stringify(block.input),\n },\n });\n }\n });\n }\n\n return {\n id: response.id,\n object: 'chat.completion',\n created: Math.floor(Date.now() / 1000),\n model: originalModel,\n choices: [{\n index: 0,\n message: {\n role: 'assistant',\n content,\n tool_calls: toolCalls.length > 0 ? toolCalls : undefined,\n },\n finish_reason: response.stop_reason || 'stop',\n }],\n usage: {\n prompt_tokens: response.usage.input_tokens,\n completion_tokens: response.usage.output_tokens,\n total_tokens: response.usage.input_tokens + response.usage.output_tokens,\n },\n };\n}\n\nexport function adaptStreamEvent(\n event: MessageStreamEvent, \n streamId: string, \n originalModel: string\n): ChatCompletionChunk | null {\n switch (event.type) {\n case 'content_block_delta':\n if (event.delta.type === 'text_delta') {\n return createChunk(streamId, originalModel, event.delta.text);\n }\n break;\n \n case 'content_block_start':\n if (event.content_block.type === 'tool_use') {\n return createChunk(streamId, originalModel, undefined, undefined, [{\n id: event.content_block.id,\n type: 'function',\n function: {\n name: event.content_block.name,\n arguments: '',\n },\n }]);\n }\n break;\n \n case 'message_stop':\n return createChunk(streamId, originalModel, undefined, 'stop');\n \n default:\n return null;\n }\n \n return null;\n}" }, { "path": "aisuite-js/src/providers/anthropic/index.ts", "content": "export { AnthropicProvider } from './provider';\nexport type { AnthropicConfig } from './types';" }, { "path": "aisuite-js/src/providers/anthropic/provider.ts", "content": "import Anthropic from '@anthropic-ai/sdk';\nimport { Provider } from '../../core/base-provider';\nimport { \n ChatCompletionRequest, \n ChatCompletionResponse, \n ChatCompletionChunk,\n RequestOptions \n} from '../../types';\nimport { AnthropicConfig } from './types';\nimport { adaptRequest, adaptResponse, adaptStreamEvent } from './adapters';\nimport { AISuiteError } from '../../core/errors';\nimport { generateId } from '../../utils/streaming';\n\nexport class AnthropicProvider implements Provider {\n public readonly name = 'anthropic';\n private client: Anthropic;\n\n constructor(config: AnthropicConfig) {\n this.client = new Anthropic({\n apiKey: config.apiKey,\n baseURL: config.baseURL,\n });\n }\n\n async chatCompletion(\n request: ChatCompletionRequest,\n options?: RequestOptions\n ): Promise {\n try {\n // For now, we don't support streaming in non-streaming method\n if (request.stream) {\n throw new AISuiteError(\n 'Streaming is not yet supported. Set stream: false or use streamChatCompletion method.',\n this.name,\n 'STREAMING_NOT_SUPPORTED'\n );\n }\n\n const anthropicRequest = adaptRequest(request);\n const message = await this.client.messages.create(\n anthropicRequest,\n options\n ) as any; // Type assertion needed because Anthropic SDK returns a union type\n\n return adaptResponse(message, request.model);\n } catch (error) {\n if (error instanceof AISuiteError) {\n throw error;\n }\n throw new AISuiteError(\n `Anthropic API error: ${error instanceof Error ? error.message : 'Unknown error'}`,\n this.name,\n 'API_ERROR'\n );\n }\n }\n\n async *streamChatCompletion(\n request: ChatCompletionRequest,\n options?: RequestOptions\n ): AsyncIterable {\n try {\n const anthropicRequest = adaptRequest(request);\n const stream = await this.client.messages.create(\n {\n ...anthropicRequest,\n stream: true,\n },\n options\n );\n\n const streamId = generateId();\n\n // Handle abort signal\n if (options?.signal) {\n options.signal.addEventListener('abort', () => {\n if (stream && typeof (stream as any).controller?.abort === 'function') {\n (stream as any).controller.abort();\n }\n });\n }\n\n for await (const event of stream) {\n const chunk = adaptStreamEvent(event, streamId, request.model);\n if (chunk) {\n yield chunk;\n }\n }\n } catch (error) {\n throw new AISuiteError(\n `Anthropic streaming error: ${error instanceof Error ? error.message : 'Unknown error'}`,\n this.name,\n 'STREAMING_ERROR'\n );\n }\n }\n}" }, { "path": "aisuite-js/src/providers/anthropic/types.ts", "content": "import { AnthropicConfig } from '../../types';\n\nexport { AnthropicConfig };\n\n// Re-export Anthropic types that we need\nexport type { \n Message,\n MessageCreateParams,\n MessageStreamEvent\n} from '@anthropic-ai/sdk/resources/messages';" }, { "path": "aisuite-js/src/providers/groq/adapters.ts", "content": "import type { ChatCompletion as GroqChatCompletion } from \"groq-sdk/resources/chat/completions\";\nimport {\n ChatCompletionRequest,\n ChatCompletionResponse,\n ChatCompletionChunk as AISuiteChatCompletionChunk,\n Usage,\n} from \"../../types\";\n\nexport function adaptRequest(request: ChatCompletionRequest): any {\n return {\n model: request.model.replace(\"groq:\", \"\"),\n messages: request.messages,\n temperature: request.temperature,\n max_tokens: request.max_tokens,\n stream: request.stream,\n tools: request.tools,\n tool_choice: request.tool_choice,\n };\n}\n\nexport function adaptResponse(\n response: GroqChatCompletion\n): ChatCompletionResponse {\n return {\n id: response.id,\n object: response.object,\n created: response.created,\n model: `groq:${response.model}`,\n choices: response.choices.map((choice) => ({\n index: choice.index,\n message: choice.message,\n finish_reason: choice.finish_reason,\n })),\n usage: response.usage ?? {\n prompt_tokens: 0,\n completion_tokens: 0,\n total_tokens: 0,\n },\n };\n}\n\nexport function adaptStreamResponse(\n chunk: any,\n streamId: string\n): AISuiteChatCompletionChunk {\n return {\n id: streamId,\n object: \"chat.completion.chunk\",\n created: Date.now(),\n model: `groq:${chunk.model}`,\n choices: chunk.choices.map((choice: any) => ({\n index: choice.index,\n delta: choice.delta,\n finish_reason: choice.finish_reason,\n })),\n };\n}\n" }, { "path": "aisuite-js/src/providers/groq/index.ts", "content": "export { GroqProvider } from \"./provider\";\nexport type { GroqConfig } from \"./types\";\n" }, { "path": "aisuite-js/src/providers/groq/provider.ts", "content": "import Groq from \"groq-sdk\";\nimport { Provider } from \"../../core/base-provider\";\nimport {\n ChatCompletionRequest,\n ChatCompletionResponse,\n ChatCompletionChunk,\n RequestOptions,\n} from \"../../types\";\nimport { GroqConfig } from \"./types\";\nimport { adaptRequest, adaptResponse, adaptStreamResponse } from \"./adapters\";\nimport { AISuiteError } from \"../../core/errors\";\nimport { generateId } from \"../../utils/streaming\";\n\nexport class GroqProvider implements Provider {\n public readonly name = \"groq\";\n private client: Groq;\n\n constructor(config: GroqConfig) {\n this.client = new Groq({\n apiKey: config.apiKey,\n dangerouslyAllowBrowser: config.dangerouslyAllowBrowser || false, // Allow browser usage for chat app\n });\n if (config.baseURL) {\n (this.client as any).baseURL = config.baseURL;\n }\n }\n\n async chatCompletion(\n request: ChatCompletionRequest,\n options?: RequestOptions\n ): Promise {\n try {\n if (request.stream) {\n throw new AISuiteError(\n \"Streaming is not supported in non-streaming method. Set stream: false or use streamChatCompletion method.\",\n this.name,\n \"STREAMING_NOT_SUPPORTED\"\n );\n }\n\n const groqRequest = adaptRequest(request);\n const completion = await this.client.chat.completions.create(groqRequest);\n\n return adaptResponse(completion);\n } catch (error) {\n if (error instanceof AISuiteError) {\n throw error;\n }\n throw new AISuiteError(\n `Groq API error: ${\n error instanceof Error ? error.message : \"Unknown error\"\n }`,\n this.name,\n \"API_ERROR\"\n );\n }\n }\n\n async *streamChatCompletion(\n request: ChatCompletionRequest,\n options?: RequestOptions\n ): AsyncIterable {\n try {\n const groqRequest = adaptRequest(request);\n const stream = await this.client.chat.completions.create(groqRequest);\n const streamId = generateId();\n\n // Handle abort signal\n if (options?.signal) {\n options.signal.addEventListener(\"abort\", () => {\n if (\n stream &&\n typeof (stream as any).controller?.abort === \"function\"\n ) {\n (stream as any).controller.abort();\n }\n });\n }\n\n for await (const chunk of stream as any) {\n yield adaptStreamResponse(chunk, streamId);\n }\n } catch (error) {\n throw new AISuiteError(\n `Groq streaming error: ${\n error instanceof Error ? error.message : \"Unknown error\"\n }`,\n this.name,\n \"STREAMING_ERROR\"\n );\n }\n }\n}\n" }, { "path": "aisuite-js/src/providers/groq/types.ts", "content": "export interface GroqConfig {\n apiKey: string;\n baseURL?: string;\n dangerouslyAllowBrowser?: boolean;\n}\n" }, { "path": "aisuite-js/src/providers/index.ts", "content": "export { OpenAIProvider } from \"./openai\";\nexport { AnthropicProvider } from \"./anthropic\";\nexport { MistralProvider } from \"./mistral\";\nexport { GroqProvider } from \"./groq\";\nexport type {\n OpenAIConfig,\n AnthropicConfig,\n MistralConfig,\n GroqConfig, \n} from \"../types\";\n" }, { "path": "aisuite-js/src/providers/mistral/adapters.ts", "content": "import {\n ChatCompletionRequest,\n ChatCompletionResponse,\n ChatCompletionChunk,\n ChatMessage,\n} from \"../../types\";\nimport type {\n ChatCompletionResponse as MistralResponse,\n ChatCompletionResponseChunk as MistralStreamResponse,\n} from \"@mistralai/mistralai\";\n\nexport function adaptRequest(request: ChatCompletionRequest): any {\n // Transform the request into Mistral's format\n const tools =\n Array.isArray(request.tools) && request.tools.length > 0\n ? request.tools\n : undefined;\n\n return {\n model: request.model,\n messages: request.messages.map(adaptMessage),\n tools,\n temperature: request.temperature,\n max_tokens: request.max_tokens,\n top_p: request.top_p,\n stream: request.stream,\n };\n}\n\nfunction adaptMessage(message: ChatMessage): any {\n return {\n role: message.role,\n content: message.content,\n tool_calls: message.tool_calls,\n };\n}\n\nexport function adaptResponse(\n response: MistralResponse\n): ChatCompletionResponse {\n return {\n id: response.id,\n object: \"chat.completion\",\n created: Math.floor(Date.now() / 1000),\n model: response.model,\n choices: [\n {\n index: 0,\n message: {\n role: \"assistant\",\n content: response.choices[0].message.content,\n },\n finish_reason: response.choices[0].finish_reason,\n },\n ],\n usage: {\n prompt_tokens: response.usage.prompt_tokens,\n completion_tokens: response.usage.completion_tokens,\n total_tokens: response.usage.total_tokens,\n },\n };\n}\n\nexport function adaptStreamResponse(\n response: MistralStreamResponse,\n streamId: string\n): ChatCompletionChunk {\n return {\n id: streamId,\n object: \"chat.completion.chunk\",\n created: Math.floor(Date.now() / 1000),\n model: response.model,\n choices: [\n {\n index: 0,\n delta: {\n role: \"assistant\",\n content: response.choices[0].delta.content,\n tool_calls: response.choices[0].delta.tool_calls,\n },\n finish_reason: response.choices[0].finish_reason,\n },\n ],\n };\n}\n" }, { "path": "aisuite-js/src/providers/mistral/index.ts", "content": "export { MistralProvider } from \"./provider\";\nexport type { MistralConfig } from \"./types\";\n" }, { "path": "aisuite-js/src/providers/mistral/provider.ts", "content": "import MistralClient from \"@mistralai/mistralai\";\nimport { Provider } from \"../../core/base-provider\";\nimport {\n ChatCompletionRequest,\n ChatCompletionResponse,\n ChatCompletionChunk,\n RequestOptions,\n} from \"../../types\";\nimport { MistralConfig } from \"./types\";\nimport { adaptRequest, adaptResponse, adaptStreamResponse } from \"./adapters\";\nimport { AISuiteError } from \"../../core/errors\";\nimport { generateId } from \"../../utils/streaming\";\n\nexport class MistralProvider implements Provider {\n public readonly name = \"mistral\";\n private client: MistralClient;\n\n constructor(config: MistralConfig) {\n this.client = new MistralClient(config.apiKey);\n if (config.baseURL) {\n (this.client as any).baseURL = config.baseURL;\n }\n }\n\n async chatCompletion(\n request: ChatCompletionRequest,\n options?: RequestOptions\n ): Promise {\n try {\n if (request.stream) {\n throw new AISuiteError(\n \"Streaming is not supported in non-streaming method. Set stream: false or use streamChatCompletion method.\",\n this.name,\n \"STREAMING_NOT_SUPPORTED\"\n );\n }\n\n const mistralRequest = adaptRequest(request);\n const completion = await this.client.chat(mistralRequest);\n\n return adaptResponse(completion);\n } catch (error) {\n if (error instanceof AISuiteError) {\n throw error;\n }\n throw new AISuiteError(\n `Mistral API error: ${\n error instanceof Error ? error.message : \"Unknown error\"\n }`,\n this.name,\n \"API_ERROR\"\n );\n }\n }\n\n async *streamChatCompletion(\n request: ChatCompletionRequest,\n options?: RequestOptions\n ): AsyncIterable {\n try {\n const mistralRequest = adaptRequest(request);\n const stream = await this.client.chatStream(mistralRequest);\n const streamId = generateId();\n\n // Handle abort signal\n if (options?.signal) {\n options.signal.addEventListener(\"abort\", () => {\n if (\n stream &&\n typeof (stream as any).controller?.abort === \"function\"\n ) {\n (stream as any).controller.abort();\n }\n });\n }\n\n for await (const chunk of stream) {\n yield adaptStreamResponse(chunk, streamId);\n }\n } catch (error) {\n throw new AISuiteError(\n `Mistral streaming error: ${\n error instanceof Error ? error.message : \"Unknown error\"\n }`,\n this.name,\n \"STREAMING_ERROR\"\n );\n }\n }\n}\n" }, { "path": "aisuite-js/src/providers/mistral/types.ts", "content": "import { MistralConfig } from \"../../types\";\n\nexport { MistralConfig };\n\n// Re-export Mistral types that we need\nexport type {\n ChatCompletionResponse as MistralResponse,\n ChatCompletionResponseChunk as MistralStreamResponse,\n} from \"@mistralai/mistralai\";\n" }, { "path": "aisuite-js/src/providers/openai/adapters.ts", "content": "import {\n ChatCompletionRequest,\n ChatCompletionResponse,\n ChatCompletionChunk,\n ChatMessage,\n ToolCall,\n Tool,\n TranscriptionRequest,\n TranscriptionResult,\n} from \"../../types\";\nimport type {\n ChatCompletion,\n ChatCompletionChunk as OpenAIChunk,\n ChatCompletionCreateParams,\n} from \"openai/resources/chat/completions\";\nimport { Uploadable } from \"openai/uploads\";\nimport { OpenAIASRResponse } from \"./types\";\n\nexport function adaptRequest(\n request: ChatCompletionRequest\n): ChatCompletionCreateParams {\n // OpenAI is our base format, so minimal transformation needed\n // Don't pass stream parameter to avoid accidental streaming\n const { stream, ...requestWithoutStream } = request;\n\n return {\n model: requestWithoutStream.model,\n messages: requestWithoutStream.messages.map(adaptMessage),\n tools: requestWithoutStream.tools,\n tool_choice: requestWithoutStream.tool_choice,\n temperature: requestWithoutStream.temperature,\n max_tokens: requestWithoutStream.max_tokens,\n top_p: requestWithoutStream.top_p,\n frequency_penalty: requestWithoutStream.frequency_penalty,\n presence_penalty: requestWithoutStream.presence_penalty,\n stop: requestWithoutStream.stop,\n user: requestWithoutStream.user,\n };\n}\n\nfunction adaptMessage(message: ChatMessage): any {\n return {\n role: message.role,\n content: message.content,\n name: message.name,\n tool_call_id: message.tool_call_id,\n tool_calls: message.tool_calls,\n };\n}\n\nexport function adaptResponse(\n response: ChatCompletion\n): ChatCompletionResponse {\n return {\n id: response.id,\n object: \"chat.completion\",\n created: response.created,\n model: response.model,\n choices: response.choices.map((choice) => ({\n index: choice.index,\n message: {\n role: choice.message.role as any,\n content: choice.message.content,\n tool_calls: choice.message.tool_calls?.map(adaptToolCall),\n },\n finish_reason: choice.finish_reason || \"stop\",\n })),\n usage: {\n prompt_tokens: response.usage?.prompt_tokens || 0,\n completion_tokens: response.usage?.completion_tokens || 0,\n total_tokens: response.usage?.total_tokens || 0,\n },\n system_fingerprint: response.system_fingerprint,\n };\n}\n\nexport function adaptChunk(chunk: OpenAIChunk): ChatCompletionChunk {\n return {\n id: chunk.id,\n object: \"chat.completion.chunk\",\n created: chunk.created,\n model: chunk.model,\n choices: chunk.choices.map((choice) => ({\n index: choice.index,\n delta: {\n role: choice.delta.role as any,\n content: choice.delta.content || undefined,\n tool_calls: choice.delta.tool_calls?.map(adaptToolCall),\n },\n finish_reason: choice.finish_reason || undefined,\n })),\n usage: chunk.usage\n ? {\n prompt_tokens: chunk.usage.prompt_tokens || 0,\n completion_tokens: chunk.usage.completion_tokens || 0,\n total_tokens: chunk.usage.total_tokens || 0,\n }\n : undefined,\n };\n}\n\nfunction adaptToolCall(toolCall: any): ToolCall {\n return {\n id: toolCall.id,\n type: \"function\",\n function: {\n name: toolCall.function.name,\n arguments: toolCall.function.arguments,\n },\n };\n}\n\nexport function adaptASRRequest(request: TranscriptionRequest): {\n file: Uploadable;\n model: string;\n} {\n if (!(request.file instanceof Buffer)) {\n throw new Error(\"File must be provided as a Buffer\");\n }\n\n const file = new File([request.file], \"audio.mp3\", {\n type: \"audio/mpeg\",\n }) as unknown as Uploadable;\n\n return {\n file,\n model: request.model,\n };\n}\n\nexport function adaptASRResponse(\n response: OpenAIASRResponse\n): TranscriptionResult {\n return {\n text: response.text,\n language: response.language || \"en\", // Default to English if not provided\n confidence: response.segments?.[0]?.avg_logprob,\n words:\n response.words?.map((word) => ({\n text: word.text || \"\",\n start: word.start,\n end: word.end,\n confidence: word?.confidence, // Default confidence if not provided\n })) ?? [],\n segments:\n response.segments?.map((segment) => ({\n text: segment.text,\n start: segment.start,\n end: segment.end,\n })) ?? [],\n };\n}\n" }, { "path": "aisuite-js/src/providers/openai/index.ts", "content": "export { OpenAIProvider } from './provider';\nexport type { OpenAIConfig } from './types';" }, { "path": "aisuite-js/src/providers/openai/provider.ts", "content": "import OpenAI from \"openai\";\nimport { Provider } from \"../../core/base-provider\";\nimport { ASRProvider } from \"../../core/base-asr-provider\";\nimport {\n ChatCompletionRequest,\n ChatCompletionResponse,\n ChatCompletionChunk,\n RequestOptions,\n TranscriptionRequest,\n TranscriptionResult,\n} from \"../../types\";\nimport { OpenAIASRResponse, OpenAIConfig } from \"./types\";\nimport {\n adaptRequest,\n adaptResponse,\n adaptChunk,\n adaptASRRequest,\n adaptASRResponse,\n} from \"./adapters\";\nimport { AISuiteError } from \"../../core/errors\";\n\nexport class OpenAIProvider implements Provider, ASRProvider {\n public readonly name = \"openai\";\n private client: OpenAI;\n\n constructor(config: OpenAIConfig) {\n this.client = new OpenAI({\n apiKey: config.apiKey,\n baseURL: config.baseURL,\n organization: config.organization,\n });\n }\n\n async chatCompletion(\n request: ChatCompletionRequest,\n options?: RequestOptions\n ): Promise {\n try {\n // For now, we don't support streaming in non-streaming method\n if (request.stream) {\n throw new AISuiteError(\n \"Streaming is not yet supported. Set stream: false or use streamChatCompletion method.\",\n this.name,\n \"STREAMING_NOT_SUPPORTED\"\n );\n }\n\n const openaiRequest = adaptRequest(request);\n const completion = (await this.client.chat.completions.create(\n openaiRequest,\n options\n )) as any; // Type assertion needed because OpenAI SDK returns a union type\n\n return adaptResponse(completion);\n } catch (error) {\n if (error instanceof AISuiteError) {\n throw error;\n }\n throw new AISuiteError(\n `OpenAI API error: ${\n error instanceof Error ? error.message : \"Unknown error\"\n }`,\n this.name,\n \"API_ERROR\"\n );\n }\n }\n\n async *streamChatCompletion(\n request: ChatCompletionRequest,\n options?: RequestOptions\n ): AsyncIterable {\n try {\n const openaiRequest = adaptRequest(request);\n const stream = await this.client.chat.completions.create(\n {\n ...openaiRequest,\n stream: true,\n },\n options\n );\n\n for await (const chunk of stream) {\n yield adaptChunk(chunk);\n }\n } catch (error) {\n throw new AISuiteError(\n `OpenAI streaming error: ${\n error instanceof Error ? error.message : \"Unknown error\"\n }`,\n this.name,\n \"STREAMING_ERROR\"\n );\n }\n }\n\n async transcribe(\n request: TranscriptionRequest,\n options?: RequestOptions\n ): Promise {\n try {\n this.validateParams(request);\n\n const adaptedRequest = adaptASRRequest(request);\n const otherParams = this.translateParams(request);\n const response = await this.client.audio.transcriptions.create({\n ...adaptedRequest,\n response_format: \"verbose_json\",\n stream: false,\n ...otherParams,\n });\n\n return adaptASRResponse(response as OpenAIASRResponse);\n } catch (error: any) {\n throw new AISuiteError(\n `OpenAI ASR transcription failed: ${error.message}`,\n this.name,\n \"PROVIDER_ERROR\"\n );\n }\n }\n\n validateParams(params: { [key: string]: any }): void {\n if (!params.model) {\n throw new AISuiteError(\n \"Model parameter is required\",\n this.name,\n \"MODEL_PARAMETER_REQUIRED\"\n );\n }\n\n if (!params.file) {\n throw new AISuiteError(\n \"File parameter is required\",\n this.name,\n \"MODEL_PARAMETER_REQUIRED\"\n );\n }\n }\n\n translateParams(params: { [key: string]: any }): { [key: string]: any } {\n const { model: _, file: __, ...rest } = params;\n return rest;\n }\n}\n" }, { "path": "aisuite-js/src/providers/openai/types.ts", "content": "import OpenAI from \"openai\";\nimport { OpenAIConfig } from \"../../types\";\n\nexport { OpenAIConfig };\n\n// Re-export OpenAI types that we need\nexport type {\n ChatCompletion,\n ChatCompletionChunk as OpenAIChunk,\n ChatCompletionCreateParams,\n} from \"openai/resources/chat/completions\";\n\nexport interface OpenAIASRRequest {\n file: File;\n model: string;\n language?: string;\n prompt?: string;\n response_format?: \"json\" | \"text\" | \"srt\" | \"verbose_json\" | \"vtt\";\n temperature?: number;\n timestamp_granularities?: Array<\"word\" | \"segment\">;\n}\n\nexport interface OpenAIASRResponse extends OpenAI.Audio.Transcription {\n text: string;\n language?: string;\n duration?: number;\n segments?: Array<{\n id: number;\n seek: number;\n start: number;\n end: number;\n text: string;\n tokens: number[];\n temperature: number;\n avg_logprob: number;\n compression_ratio: number;\n no_speech_prob: number;\n }>;\n words?: Array<{\n text: string;\n start: number;\n end: number;\n confidence?: number;\n }>;\n}\n" }, { "path": "aisuite-js/src/types/chat.ts", "content": "export interface ChatMessage {\n role: \"system\" | \"user\" | \"assistant\" | \"tool\";\n content: string | null;\n name?: string;\n tool_call_id?: string;\n tool_calls?: ToolCall[];\n}\n\nexport interface ChatCompletionRequest {\n model: string; // \"provider:model\" format\n messages: ChatMessage[];\n tools?: Tool[];\n tool_choice?: ToolChoice;\n temperature?: number;\n max_tokens?: number;\n top_p?: number;\n frequency_penalty?: number;\n presence_penalty?: number;\n stop?: string | string[];\n stream?: boolean;\n user?: string;\n}\n\nexport interface ChatCompletionResponse {\n id: string;\n object: \"chat.completion\";\n created: number;\n model: string;\n choices: ChatChoice[];\n usage: Usage;\n system_fingerprint?: string;\n}\n\nexport interface ChatCompletionChunk {\n id: string;\n object: \"chat.completion.chunk\";\n created: number;\n model: string;\n choices: Array<{\n index: number;\n delta: {\n role?: \"assistant\";\n content?: string;\n tool_calls?: ToolCall[];\n };\n finish_reason?: string;\n }>;\n usage?: Usage;\n}\n\nexport interface ChatChoice {\n index: number;\n message: ChatMessage;\n finish_reason: string;\n}\n\nexport interface Usage {\n prompt_tokens: number;\n completion_tokens: number;\n total_tokens: number;\n}\n\n// Import tool types from tools.ts\nimport { Tool, ToolCall, ToolChoice } from \"./tools\";\n" }, { "path": "aisuite-js/src/types/common.ts", "content": "export interface RequestOptions {\n signal?: AbortSignal;\n timeout?: number;\n retries?: number;\n [key: string]: any;\n}\n\nexport type AudioInput = string | Buffer | Uint8Array;" }, { "path": "aisuite-js/src/types/index.ts", "content": "export * from \"./chat\";\nexport * from \"./tools\";\nexport * from \"./common\";\nexport * from \"./providers\";\nexport * from \"./transcription\";\n" }, { "path": "aisuite-js/src/types/providers.ts", "content": "export interface ProviderConfigs {\n openai?: OpenAIConfig;\n anthropic?: AnthropicConfig;\n mistral?: MistralConfig;\n groq?: GroqConfig;\n deepgram?: DeepgramConfig;\n}\n\nexport interface OpenAIConfig {\n apiKey: string;\n baseURL?: string;\n organization?: string;\n}\n\nexport interface AnthropicConfig {\n apiKey: string;\n baseURL?: string;\n}\n\nexport interface MistralConfig {\n apiKey: string;\n baseURL?: string;\n}\n\nexport interface GroqConfig {\n apiKey: string;\n baseURL?: string;\n}\n\nexport interface DeepgramConfig {\n apiKey: string;\n baseURL?: string;\n}\n" }, { "path": "aisuite-js/src/types/tools.ts", "content": "export interface Tool {\n type: 'function';\n function: FunctionDefinition;\n}\n\nexport interface FunctionDefinition {\n name: string;\n description: string;\n parameters: {\n type: 'object';\n properties: Record;\n required?: string[];\n };\n}\n\nexport interface ToolCall {\n id: string;\n type: 'function';\n function: {\n name: string;\n arguments: string; // JSON string\n };\n}\n\nexport type ToolChoice = \n | 'auto' \n | 'none' \n | { type: 'function'; function: { name: string } };" }, { "path": "aisuite-js/src/types/transcription.ts", "content": "export interface Word {\n text: string;\n start: number;\n end: number;\n speaker?: string;\n confidence?: number;\n}\n\nexport interface Segment {\n text: string;\n start: number;\n end: number;\n speaker?: string;\n}\n\nexport interface TranscriptionResult {\n text: string;\n language: string;\n confidence?: number;\n words: Word[];\n segments: Segment[];\n}\n\nexport interface TranscriptionRequest {\n model: string; // \"provider:model\" format\n file: string | Buffer | Uint8Array;\n language?: string;\n timestamps?: boolean;\n word_confidence?: boolean;\n speaker_labels?: boolean;\n temperature?: number;\n // Provider-specific parameters\n [key: string]: any;\n}\n" }, { "path": "aisuite-js/src/utils/streaming.ts", "content": "import { ChatCompletionChunk } from '../types';\n\nexport function createChunk(\n id: string,\n model: string,\n content?: string,\n finishReason?: string,\n toolCalls?: any[]\n): ChatCompletionChunk {\n return {\n id,\n object: 'chat.completion.chunk',\n created: Math.floor(Date.now() / 1000),\n model,\n choices: [{\n index: 0,\n delta: {\n role: 'assistant',\n content,\n tool_calls: toolCalls\n },\n finish_reason: finishReason || undefined\n }]\n };\n}\n\nexport function generateId(): string {\n return `chatcmpl-${Math.random().toString(36).substr(2, 9)}`;\n}" }, { "path": "aisuite-js/tests/client.test.ts", "content": "import { Client } from \"../src/client\";\nimport {\n ProviderConfigs,\n ChatCompletionRequest,\n ChatCompletionResponse,\n ChatCompletionChunk,\n RequestOptions,\n TranscriptionRequest,\n} from \"../src/types\";\nimport { ProviderNotConfiguredError } from \"../src/core/errors\";\nimport { Provider } from \"../src/core/base-provider\";\n\n// Mock the Mistral SDK\njest.mock(\"@mistralai/mistralai\", () => {\n return {\n __esModule: true,\n default: jest.fn(),\n };\n});\n\n// Mock the providers\njest.mock(\"../src/providers/openai\");\njest.mock(\"../src/providers/anthropic\");\njest.mock(\"../src/providers/mistral\");\njest.mock(\"../src/providers/groq\");\njest.mock(\"../src/asr-providers/deepgram\");\n\ndescribe(\"Client\", () => {\n let mockOpenAIProvider: any;\n let mockAnthropicProvider: any;\n let mockMistralProvider: any;\n let mockGroqProvider: any;\n let mockDeepgramProvider: any;\n let mockOpenAIASRProvider: any;\n\n beforeEach(() => {\n // Reset all mocks\n jest.clearAllMocks();\n\n // Create mock response\n const mockResponse = {\n id: \"chatcmpl-123\",\n object: \"chat.completion\",\n created: 1234567890,\n model: \"gpt-4\",\n choices: [\n {\n index: 0,\n message: {\n role: \"assistant\",\n content: \"Hello! How can I help you?\",\n },\n finish_reason: \"stop\",\n },\n ],\n usage: {\n prompt_tokens: 10,\n completion_tokens: 20,\n total_tokens: 30,\n },\n };\n\n // Create mock provider class\n class MockProvider implements Provider {\n public readonly name: string;\n public chatCompletion = jest.fn().mockResolvedValue(mockResponse);\n public streamChatCompletion = jest.fn().mockImplementation(async function* () {\n yield mockResponse as unknown as ChatCompletionChunk;\n });\n\n constructor(name: string) {\n this.name = name;\n }\n }\n\n // Create mock instances\n mockOpenAIProvider = new MockProvider(\"openai\");\n mockAnthropicProvider = new MockProvider(\"anthropic\");\n mockMistralProvider = new MockProvider(\"mistral\");\n mockGroqProvider = new MockProvider(\"groq\");\n\n mockDeepgramProvider = {\n transcribe: jest.fn(),\n };\n\n mockOpenAIASRProvider = {\n transcribe: jest.fn(),\n };\n\n // Manually mock the provider constructors using jest.mock\n const openaiModule = jest.requireMock(\"../src/providers/openai\");\n const anthropicModule = jest.requireMock(\"../src/providers/anthropic\");\n const mistralModule = jest.requireMock(\"../src/providers/mistral\");\n const groqModule = jest.requireMock(\"../src/providers/groq\");\n const deepgramModule = jest.requireMock(\"../src/asr-providers/deepgram\");\n\n openaiModule.OpenAIProvider = jest.fn().mockImplementation(() => mockOpenAIProvider);\n anthropicModule.AnthropicProvider = jest.fn().mockImplementation(() => mockAnthropicProvider);\n mistralModule.MistralProvider = jest.fn().mockImplementation(() => mockMistralProvider);\n groqModule.GroqProvider = jest.fn().mockImplementation(() => mockGroqProvider);\n deepgramModule.DeepgramASRProvider = jest.fn().mockImplementation(() => mockDeepgramProvider);\n });\n\n describe(\"constructor\", () => {\n it(\"should initialize providers based on config\", () => {\n const config: ProviderConfigs = {\n openai: { apiKey: \"openai-key\" },\n anthropic: { apiKey: \"anthropic-key\" },\n mistral: { apiKey: \"mistral-key\" },\n groq: { apiKey: \"groq-key\" },\n deepgram: { apiKey: \"deepgram-key\" },\n };\n\n const client = new Client(config);\n\n expect(client.listProviders()).toEqual([\n \"openai\",\n \"anthropic\",\n \"mistral\",\n \"groq\",\n ]);\n expect(client.listASRProviders()).toEqual([\"openai\", \"deepgram\"]);\n expect(client.isProviderConfigured(\"openai\")).toBe(true);\n expect(client.isProviderConfigured(\"anthropic\")).toBe(true);\n expect(client.isProviderConfigured(\"mistral\")).toBe(true);\n expect(client.isProviderConfigured(\"groq\")).toBe(true);\n expect(client.isASRProviderConfigured(\"deepgram\")).toBe(true);\n });\n\n it(\"should only initialize configured providers\", () => {\n const config: ProviderConfigs = {\n openai: { apiKey: \"openai-key\" },\n groq: { apiKey: \"groq-key\" },\n deepgram: { apiKey: \"deepgram-key\" },\n };\n\n const client = new Client(config);\n\n expect(client.listProviders()).toEqual([\"openai\", \"groq\"]);\n expect(client.listASRProviders()).toEqual([\"openai\", \"deepgram\"]);\n expect(client.isProviderConfigured(\"openai\")).toBe(true);\n expect(client.isProviderConfigured(\"anthropic\")).toBe(false);\n expect(client.isProviderConfigured(\"mistral\")).toBe(false);\n expect(client.isProviderConfigured(\"groq\")).toBe(true);\n expect(client.isASRProviderConfigured(\"deepgram\")).toBe(true);\n expect(client.isASRProviderConfigured(\"unknown\")).toBe(false);\n });\n\n it(\"should handle empty config\", () => {\n const config: ProviderConfigs = {};\n\n const client = new Client(config);\n\n expect(client.listProviders()).toEqual([]);\n expect(client.listASRProviders()).toEqual([]);\n expect(client.isProviderConfigured(\"openai\")).toBe(false);\n expect(client.isASRProviderConfigured(\"deepgram\")).toBe(false);\n });\n });\n\n describe(\"chat.completions.create\", () => {\n let client: Client;\n const baseConfig: ProviderConfigs = {\n openai: { apiKey: \"openai-key\" },\n anthropic: { apiKey: \"anthropic-key\" },\n mistral: { apiKey: \"mistral-key\" },\n groq: { apiKey: \"groq-key\" },\n };\n\n beforeEach(() => {\n client = new Client(baseConfig);\n });\n\n it(\"should call non-streaming chat completion\", async () => {\n const request: ChatCompletionRequest = {\n model: \"openai:gpt-4\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n };\n\n const mockResponse = {\n id: \"test-id\",\n object: \"chat.completion\",\n created: 1234567890,\n model: \"gpt-4\",\n choices: [],\n usage: { prompt_tokens: 10, completion_tokens: 20, total_tokens: 30 },\n };\n\n mockOpenAIProvider.chatCompletion.mockResolvedValue(mockResponse);\n\n const result = await client.chat.completions.create(request);\n\n expect(mockOpenAIProvider.chatCompletion).toHaveBeenCalledWith(\n { ...request, model: \"gpt-4\" },\n undefined\n );\n expect(result).toEqual(mockResponse);\n });\n\n it(\"should call streaming chat completion\", async () => {\n const request: ChatCompletionRequest = {\n model: \"anthropic:claude-3-sonnet\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n stream: true,\n };\n\n const mockStream = (async function* () {\n yield {\n id: \"chunk-1\",\n object: \"chat.completion.chunk\",\n created: 1234567890,\n model: \"claude-3-sonnet\",\n choices: [],\n };\n })();\n\n mockAnthropicProvider.streamChatCompletion.mockReturnValue(mockStream);\n\n const result = await client.chat.completions.create(request);\n\n expect(mockAnthropicProvider.streamChatCompletion).toHaveBeenCalledWith(\n { ...request, model: \"claude-3-sonnet\" },\n undefined\n );\n expect(result).toBe(mockStream);\n });\n\n it(\"should throw error for unconfigured provider\", async () => {\n const request: ChatCompletionRequest = {\n model: \"unknown:model\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n };\n\n await expect(client.chat.completions.create(request)).rejects.toThrow(\n ProviderNotConfiguredError\n );\n });\n\n it(\"should handle complex model names with multiple colons\", async () => {\n const request: ChatCompletionRequest = {\n model: \"openai:gpt-4:vision\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n };\n\n const mockResponse = {\n id: \"test-id\",\n object: \"chat.completion\",\n created: 1234567890,\n model: \"gpt-4:vision\",\n choices: [],\n usage: { prompt_tokens: 10, completion_tokens: 20, total_tokens: 30 },\n };\n\n mockOpenAIProvider.chatCompletion.mockResolvedValue(mockResponse);\n\n const result = await client.chat.completions.create(request);\n\n expect(mockOpenAIProvider.chatCompletion).toHaveBeenCalledWith(\n { ...request, model: \"gpt-4:vision\" },\n undefined\n );\n expect(result).toEqual(mockResponse);\n });\n\n it(\"should pass options to provider\", async () => {\n const request: ChatCompletionRequest = {\n model: \"mistral:mistral-large\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n };\n\n const options = { signal: new AbortController().signal };\n\n const mockResponse = {\n id: \"test-id\",\n object: \"chat.completion\",\n created: 1234567890,\n model: \"mistral-large\",\n choices: [],\n usage: { prompt_tokens: 10, completion_tokens: 20, total_tokens: 30 },\n };\n\n mockMistralProvider.chatCompletion.mockResolvedValue(mockResponse);\n\n const result = await client.chat.completions.create(request, options);\n\n expect(mockMistralProvider.chatCompletion).toHaveBeenCalledWith(\n { ...request, model: \"mistral-large\" },\n options\n );\n expect(result).toEqual(mockResponse);\n });\n });\n\n describe(\"audio.transcriptions.create\", () => {\n describe(\"Deepgram Provider\", () => {\n let client: Client;\n const baseConfig: ProviderConfigs = {\n deepgram: { apiKey: \"deepgram-key\" },\n };\n\n beforeEach(() => {\n client = new Client(baseConfig);\n });\n\n it(\"should call transcription with correct parameters\", async () => {\n const audioBuffer = Buffer.from(\"test audio data\");\n const request: TranscriptionRequest = {\n model: \"deepgram:nova-2\",\n file: audioBuffer,\n language: \"en-US\",\n timestamps: true,\n word_confidence: true,\n speaker_labels: true,\n };\n\n const mockResponse = {\n text: \"Hello world\",\n language: \"en-US\",\n confidence: 0.95,\n words: [\n {\n text: \"Hello\",\n start: 0.0,\n end: 0.5,\n confidence: 0.98,\n },\n {\n text: \"world\",\n start: 0.6,\n end: 1.0,\n confidence: 0.92,\n },\n ],\n segments: [\n {\n text: \"Hello world\",\n start: 0.0,\n end: 1.0,\n },\n ],\n };\n\n mockDeepgramProvider.transcribe.mockResolvedValue(mockResponse);\n\n const result = await client.audio.transcriptions.create(request);\n\n expect(mockDeepgramProvider.transcribe).toHaveBeenCalledWith(\n { ...request, model: \"nova-2\" },\n undefined\n );\n expect(result).toEqual(mockResponse);\n });\n\n it(\"should throw error for unconfigured ASR provider\", async () => {\n const request: TranscriptionRequest = {\n model: \"unknown:model\",\n file: Buffer.from(\"test\"),\n };\n\n await expect(\n client.audio.transcriptions.create(request)\n ).rejects.toThrow(ProviderNotConfiguredError);\n });\n\n it(\"should pass options to ASR provider\", async () => {\n const audioBuffer = Buffer.from(\"test audio data\");\n const request: TranscriptionRequest = {\n model: \"deepgram:nova-2\",\n file: audioBuffer,\n language: \"en-US\",\n };\n\n const options = { timeout: 30000 };\n\n const mockResponse = {\n text: \"Test transcription\",\n language: \"en-US\",\n confidence: 0.9,\n words: [],\n segments: [],\n };\n\n mockDeepgramProvider.transcribe.mockResolvedValue(mockResponse);\n\n const result = await client.audio.transcriptions.create(\n request,\n options\n );\n\n expect(mockDeepgramProvider.transcribe).toHaveBeenCalledWith(\n { ...request, model: \"nova-2\" },\n options\n );\n expect(result).toEqual(mockResponse);\n });\n\n it(\"should handle complex model names with multiple colons\", async () => {\n const audioBuffer = Buffer.from(\"test audio data\");\n const request: TranscriptionRequest = {\n model: \"deepgram:nova-2:enhanced\",\n file: audioBuffer,\n language: \"en-US\",\n };\n\n const mockResponse = {\n text: \"Test transcription\",\n language: \"en-US\",\n confidence: 0.9,\n words: [],\n segments: [],\n };\n\n mockDeepgramProvider.transcribe.mockResolvedValue(mockResponse);\n\n const result = await client.audio.transcriptions.create(request);\n\n expect(mockDeepgramProvider.transcribe).toHaveBeenCalledWith(\n { ...request, model: \"nova-2:enhanced\" },\n undefined\n );\n expect(result).toEqual(mockResponse);\n });\n });\n\n describe(\"OpenAI ASR Provider\", () => {\n let client: Client;\n let mockOpenAIASRProvider: any;\n\n beforeEach(() => {\n mockOpenAIASRProvider = {\n name: \"openai\",\n transcribe: jest.fn(),\n };\n\n // Update to configure OpenAI provider for both chat and ASR\n const openaiModule = require(\"../src/providers/openai\");\n openaiModule.OpenAIProvider.mockImplementation(() => ({\n ...mockOpenAIProvider,\n ...mockOpenAIASRProvider\n }));\n });\n\n it(\"should transcribe with audio enabled\", async () => {\n // Initialize client with OpenAI provider\n client = new Client({\n openai: {\n apiKey: \"openai-key\"\n },\n });\n\n // Add the OpenAI provider to ASR providers list manually for testing\n client[\"asrProviders\"].set(\"openai\", mockOpenAIASRProvider);\n\n const audioBuffer = Buffer.from(\"test audio data\");\n const request: TranscriptionRequest = {\n model: \"openai:whisper-1\",\n file: audioBuffer,\n language: \"en\",\n response_format: \"verbose_json\",\n temperature: 0,\n timestamps: true,\n };\n\n const mockResponse = {\n text: \"Test transcription\",\n language: \"en\",\n confidence: 0.95,\n words: [\n {\n text: \"Test\",\n start: 0.0,\n end: 0.5,\n confidence: 0.98,\n },\n {\n text: \"transcription\",\n start: 0.6,\n end: 1.2,\n confidence: 0.92,\n },\n ],\n segments: [\n {\n text: \"Test transcription\",\n start: 0.0,\n end: 1.2,\n },\n ],\n };\n\n mockOpenAIASRProvider.transcribe.mockResolvedValue(mockResponse);\n\n const result = await client.audio.transcriptions.create(request);\n\n expect(mockOpenAIASRProvider.transcribe).toHaveBeenCalledWith(\n { ...request, model: \"whisper-1\" },\n undefined\n );\n expect(result).toEqual(mockResponse);\n });\n\n it(\"should support different response formats\", async () => {\n client = new Client({\n openai: {\n apiKey: \"openai-key\"\n },\n });\n\n // Add the OpenAI provider to ASR providers list\n client[\"asrProviders\"].set(\"openai\", mockOpenAIASRProvider);\n\n const request: TranscriptionRequest = {\n model: \"openai:whisper-1\",\n file: Buffer.from(\"test audio\"),\n response_format: \"text\",\n };\n\n const mockResponse = {\n text: \"Test transcription\",\n language: \"en\",\n confidence: 1.0,\n words: [],\n segments: [],\n };\n\n mockOpenAIASRProvider.transcribe.mockResolvedValue(mockResponse);\n const result = await client.audio.transcriptions.create(request);\n\n expect(mockOpenAIASRProvider.transcribe).toHaveBeenCalledWith(\n { ...request, model: \"whisper-1\" },\n undefined\n );\n expect(result).toEqual(mockResponse);\n });\n\n it(\"should pass custom options to provider\", async () => {\n client = new Client({\n openai: {\n apiKey: \"openai-key\"\n },\n });\n\n // Add the OpenAI provider to ASR providers list\n client[\"asrProviders\"].set(\"openai\", mockOpenAIASRProvider);\n\n const request: TranscriptionRequest = {\n model: \"openai:whisper-1\",\n file: Buffer.from(\"test audio\"),\n language: \"en\",\n };\n\n const options = { timeout: 30000 };\n const mockResponse = {\n text: \"Test transcription\",\n language: \"en\",\n confidence: 0.9,\n words: [],\n segments: [],\n };\n\n mockOpenAIASRProvider.transcribe.mockResolvedValue(mockResponse);\n const result = await client.audio.transcriptions.create(\n request,\n options\n );\n\n expect(mockOpenAIASRProvider.transcribe).toHaveBeenCalledWith(\n { ...request, model: \"whisper-1\" },\n options\n );\n expect(result).toEqual(mockResponse);\n });\n });\n });\n\n describe(\"listProviders\", () => {\n it(\"should return list of configured providers\", () => {\n const config: ProviderConfigs = {\n openai: { apiKey: \"openai-key\" },\n groq: { apiKey: \"groq-key\" },\n };\n\n const client = new Client(config);\n\n expect(client.listProviders()).toEqual([\"openai\", \"groq\"]);\n });\n\n it(\"should return empty array when no providers configured\", () => {\n const config: ProviderConfigs = {};\n\n const client = new Client(config);\n\n expect(client.listProviders()).toEqual([]);\n });\n });\n\n describe(\"listASRProviders\", () => {\n it(\"should return list of configured ASR providers\", () => {\n const config: ProviderConfigs = {\n deepgram: { apiKey: \"deepgram-key\" },\n };\n\n const client = new Client(config);\n\n expect(client.listASRProviders()).toEqual([\"deepgram\"]);\n });\n\n it(\"should return empty array when no ASR providers configured\", () => {\n const config: ProviderConfigs = {};\n\n const client = new Client(config);\n\n expect(client.listASRProviders()).toEqual([]);\n });\n });\n\n describe(\"isProviderConfigured\", () => {\n it(\"should return true for configured providers\", () => {\n const config: ProviderConfigs = {\n openai: { apiKey: \"openai-key\" },\n anthropic: { apiKey: \"anthropic-key\" },\n };\n\n const client = new Client(config);\n\n expect(client.isProviderConfigured(\"openai\")).toBe(true);\n expect(client.isProviderConfigured(\"anthropic\")).toBe(true);\n });\n\n it(\"should return false for unconfigured providers\", () => {\n const config: ProviderConfigs = {\n openai: { apiKey: \"openai-key\" },\n };\n\n const client = new Client(config);\n\n expect(client.isProviderConfigured(\"anthropic\")).toBe(false);\n expect(client.isProviderConfigured(\"mistral\")).toBe(false);\n expect(client.isProviderConfigured(\"groq\")).toBe(false);\n });\n });\n\n describe(\"isASRProviderConfigured\", () => {\n it(\"should return true for configured ASR providers\", () => {\n const config: ProviderConfigs = {\n deepgram: { apiKey: \"deepgram-key\" },\n };\n\n const client = new Client(config);\n\n expect(client.isASRProviderConfigured(\"deepgram\")).toBe(true);\n });\n\n it(\"should return false for unconfigured ASR providers\", () => {\n const config: ProviderConfigs = {};\n\n const client = new Client(config);\n\n expect(client.isASRProviderConfigured(\"deepgram\")).toBe(false);\n expect(client.isASRProviderConfigured(\"unknown\")).toBe(false);\n });\n });\n});\n" }, { "path": "aisuite-js/tests/providers/anthropic-provider.test.ts", "content": "import { AnthropicProvider } from \"../../src/providers/anthropic/provider\";\nimport { ChatCompletionRequest, ChatCompletionChunk } from \"../../src/types\";\nimport { AISuiteError } from \"../../src/core/errors\";\n\n// Mock the Anthropic SDK\njest.mock(\"@anthropic-ai/sdk\", () => {\n return {\n __esModule: true,\n default: jest.fn(),\n };\n});\n\ndescribe(\"AnthropicProvider\", () => {\n let provider: AnthropicProvider;\n let mockAnthropicClient: any;\n\n beforeEach(() => {\n // Reset mocks\n jest.clearAllMocks();\n\n // Create mock Anthropic client\n mockAnthropicClient = {\n messages: {\n create: jest.fn(),\n },\n };\n\n // Mock the Anthropic constructor\n const Anthropic = require(\"@anthropic-ai/sdk\");\n Anthropic.default.mockImplementation(() => mockAnthropicClient);\n\n // Ensure the mock is properly structured\n mockAnthropicClient.messages = {\n create: jest.fn(),\n };\n\n // Create provider instance\n provider = new AnthropicProvider({\n apiKey: \"test-api-key\",\n });\n });\n\n describe(\"constructor\", () => {\n it(\"should initialize with basic config\", () => {\n const config = { apiKey: \"test-key\" };\n const provider = new AnthropicProvider(config);\n\n expect(provider.name).toBe(\"anthropic\");\n });\n\n it(\"should initialize with full config\", () => {\n const config = {\n apiKey: \"test-key\",\n baseURL: \"https://custom.anthropic.com\",\n };\n const provider = new AnthropicProvider(config);\n\n expect(provider.name).toBe(\"anthropic\");\n });\n });\n\n describe(\"chatCompletion\", () => {\n const baseRequest: ChatCompletionRequest = {\n model: \"claude-3-sonnet\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n };\n\n it(\"should successfully complete chat\", async () => {\n const mockResponse = {\n id: \"msg_123\",\n type: \"message\",\n role: \"assistant\",\n content: [\n {\n type: \"text\",\n text: \"Hello! How can I help you?\",\n },\n ],\n model: \"claude-3-sonnet-20240229\",\n stop_reason: \"end_turn\",\n stop_sequence: null,\n usage: {\n input_tokens: 10,\n output_tokens: 20,\n },\n };\n\n mockAnthropicClient.messages.create.mockResolvedValue(mockResponse);\n\n const result = await provider.chatCompletion(baseRequest);\n\n expect(mockAnthropicClient.messages.create).toHaveBeenCalledWith(\n expect.objectContaining({\n model: expect.any(String),\n messages: expect.arrayContaining([\n expect.objectContaining({ role: \"user\", content: \"Hello\" }),\n ]),\n }),\n undefined\n );\n expect(result).toEqual(\n expect.objectContaining({\n id: \"msg_123\",\n object: \"chat.completion\",\n model: \"claude-3-sonnet\",\n })\n );\n });\n\n it(\"should throw error when streaming is enabled\", async () => {\n const request: ChatCompletionRequest = {\n ...baseRequest,\n stream: true,\n };\n\n await expect(provider.chatCompletion(request)).rejects.toThrow(\n AISuiteError\n );\n await expect(provider.chatCompletion(request)).rejects.toThrow(\n \"Streaming is not yet supported\"\n );\n });\n\n it(\"should handle API errors\", async () => {\n const apiError = new Error(\"API rate limit exceeded\");\n mockAnthropicClient.messages.create.mockRejectedValue(apiError);\n\n await expect(provider.chatCompletion(baseRequest)).rejects.toThrow(\n AISuiteError\n );\n await expect(provider.chatCompletion(baseRequest)).rejects.toThrow(\n \"Anthropic API error: API rate limit exceeded\"\n );\n });\n\n it(\"should pass options to the client\", async () => {\n const options = { signal: new AbortController().signal };\n const mockResponse = {\n id: \"msg_123\",\n type: \"message\",\n role: \"assistant\",\n content: [{ type: \"text\", text: \"Hello!\" }],\n model: \"claude-3-sonnet-20240229\",\n stop_reason: \"end_turn\",\n usage: { input_tokens: 10, output_tokens: 5 },\n };\n\n mockAnthropicClient.messages.create.mockResolvedValue(mockResponse);\n\n await provider.chatCompletion(baseRequest, options);\n\n expect(mockAnthropicClient.messages.create).toHaveBeenCalledWith(\n expect.any(Object),\n options\n );\n });\n\n it(\"should handle complex request with all parameters\", async () => {\n const complexRequest: ChatCompletionRequest = {\n model: \"claude-3-sonnet\",\n messages: [\n { role: \"system\", content: \"You are a helpful assistant\" },\n { role: \"user\", content: \"What is 2+2?\" },\n ],\n temperature: 0.7,\n max_tokens: 100,\n top_p: 0.9,\n frequency_penalty: 0.1,\n presence_penalty: 0.1,\n stop: [\"\\n\"],\n user: \"user-123\",\n };\n\n const mockResponse = {\n id: \"msg_123\",\n type: \"message\",\n role: \"assistant\",\n content: [{ type: \"text\", text: \"2+2 equals 4\" }],\n model: \"claude-3-sonnet-20240229\",\n stop_reason: \"end_turn\",\n usage: { input_tokens: 15, output_tokens: 5 },\n };\n\n mockAnthropicClient.messages.create.mockResolvedValue(mockResponse);\n\n const result = await provider.chatCompletion(complexRequest);\n\n expect(mockAnthropicClient.messages.create).toHaveBeenCalledWith(\n expect.objectContaining({\n model: expect.any(String),\n messages: expect.arrayContaining([\n expect.objectContaining({ role: \"user\", content: \"What is 2+2?\" }),\n ]),\n system: \"You are a helpful assistant\",\n temperature: 0.7,\n max_tokens: 100,\n top_p: 0.9,\n stop_sequences: expect.any(Array),\n }),\n undefined\n );\n expect(result).toEqual(\n expect.objectContaining({\n id: \"msg_123\",\n object: \"chat.completion\",\n })\n );\n });\n });\n\n describe(\"streamChatCompletion\", () => {\n const baseRequest: ChatCompletionRequest = {\n model: \"claude-3-sonnet\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n };\n\n it(\"should stream chat completion\", async () => {\n const mockEvents = [\n {\n type: \"message_start\",\n message: {\n id: \"msg_123\",\n type: \"message\",\n role: \"assistant\",\n content: [],\n model: \"claude-3-sonnet-20240229\",\n },\n },\n {\n type: \"content_block_start\",\n index: 0,\n content_block: {\n type: \"text\",\n text: \"\",\n },\n },\n {\n type: \"content_block_delta\",\n index: 0,\n delta: {\n type: \"text_delta\",\n text: \"Hello\",\n },\n },\n {\n type: \"content_block_delta\",\n index: 0,\n delta: {\n type: \"text_delta\",\n text: \"! How can I help?\",\n },\n },\n {\n type: \"content_block_stop\",\n index: 0,\n },\n {\n type: \"message_delta\",\n delta: {\n stop_reason: \"end_turn\",\n stop_sequence: null,\n },\n },\n {\n type: \"message_stop\",\n },\n ];\n\n const mockStream = (async function* () {\n for (const event of mockEvents) {\n yield event;\n }\n })();\n\n mockAnthropicClient.messages.create.mockResolvedValue(mockStream);\n\n const stream = provider.streamChatCompletion(baseRequest);\n const chunks: ChatCompletionChunk[] = [];\n\n for await (const chunk of stream) {\n chunks.push(chunk);\n }\n\n expect(mockAnthropicClient.messages.create).toHaveBeenCalledWith(\n expect.objectContaining({\n model: expect.any(String),\n messages: expect.any(Array),\n }),\n undefined\n );\n expect(chunks.length).toBeGreaterThan(0);\n });\n\n it(\"should handle streaming errors\", async () => {\n mockAnthropicClient.messages.create.mockRejectedValue(\n new Error(\"Streaming error\")\n );\n const stream = provider.streamChatCompletion(baseRequest);\n const iterator = stream[Symbol.asyncIterator]();\n await expect(iterator.next()).rejects.toThrow(AISuiteError);\n });\n\n it(\"should pass options to streaming request\", async () => {\n const options = { signal: new AbortController().signal };\n const mockStream = (async function* () {\n yield {\n type: \"message_start\",\n message: {\n id: \"msg_123\",\n type: \"message\",\n role: \"assistant\",\n content: [],\n model: \"claude-3-sonnet-20240229\",\n },\n };\n yield {\n type: \"content_block_delta\",\n index: 0,\n delta: {\n type: \"text_delta\",\n text: \"Hello!\",\n },\n };\n yield {\n type: \"message_stop\",\n };\n })();\n\n mockAnthropicClient.messages.create.mockResolvedValue(mockStream);\n\n const stream = provider.streamChatCompletion(baseRequest, options);\n const chunks: ChatCompletionChunk[] = [];\n\n for await (const chunk of stream) {\n chunks.push(chunk);\n }\n\n expect(mockAnthropicClient.messages.create).toHaveBeenCalledWith(\n expect.objectContaining({\n model: expect.any(String),\n messages: expect.any(Array),\n stream: true,\n }),\n options\n );\n expect(chunks.length).toBeGreaterThan(0);\n });\n\n it(\"should handle abort signal\", async () => {\n const mockStream = (async function* () {\n yield {\n type: \"message_start\",\n message: {\n id: \"msg_123\",\n type: \"message\",\n role: \"assistant\",\n content: [],\n model: \"claude-3-sonnet-20240229\",\n },\n };\n yield {\n type: \"content_block_delta\",\n index: 0,\n delta: {\n type: \"text_delta\",\n text: \"Hello!\",\n },\n };\n })();\n\n mockAnthropicClient.messages.create.mockResolvedValue(mockStream);\n\n const abortController = new AbortController();\n const options = { signal: abortController.signal };\n\n const stream = provider.streamChatCompletion(baseRequest, options);\n const chunks: ChatCompletionChunk[] = [];\n\n // Start consuming the stream\n const consumePromise = (async () => {\n for await (const chunk of stream) {\n chunks.push(chunk);\n }\n })();\n\n // Abort after a short delay\n setTimeout(() => {\n abortController.abort();\n }, 10);\n\n await consumePromise;\n\n expect(chunks.length).toBeGreaterThan(0);\n });\n });\n\n describe(\"error handling\", () => {\n it(\"should preserve AISuiteError instances\", async () => {\n const customError = new AISuiteError(\n \"Custom error\",\n \"anthropic\",\n \"CUSTOM_ERROR\"\n );\n\n mockAnthropicClient.messages.create.mockRejectedValue(customError);\n\n await expect(\n provider.chatCompletion({\n model: \"claude-3-sonnet\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n })\n ).rejects.toThrow(customError);\n });\n\n it(\"should handle unknown error types\", async () => {\n const unknownError = \"Unknown error string\";\n mockAnthropicClient.messages.create.mockRejectedValue(unknownError);\n\n await expect(\n provider.chatCompletion({\n model: \"claude-3-sonnet\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n })\n ).rejects.toThrow(\"Anthropic API error: Unknown error\");\n });\n });\n});\n" }, { "path": "aisuite-js/tests/providers/deepgram-provider.test.ts", "content": "import { DeepgramASRProvider } from \"../../src/asr-providers/deepgram/provider\";\nimport { TranscriptionRequest } from \"../../src/types\";\nimport { AISuiteError } from \"../../src/core/errors\";\n\n// Mock the Deepgram SDK\njest.mock(\"@deepgram/sdk\", () => ({\n createClient: jest.fn(),\n}));\n\ndescribe(\"DeepgramASRProvider\", () => {\n let provider: DeepgramASRProvider;\n let mockDeepgramClient: any;\n\n beforeEach(() => {\n // Reset mocks\n jest.clearAllMocks();\n\n // Create mock Deepgram client\n mockDeepgramClient = {\n listen: {\n prerecorded: {\n transcribeFile: jest.fn(),\n },\n },\n };\n\n // Mock the createClient function\n const { createClient } = require(\"@deepgram/sdk\");\n createClient.mockReturnValue(mockDeepgramClient);\n\n // Create provider instance\n provider = new DeepgramASRProvider({\n apiKey: \"test-api-key\",\n });\n });\n\n describe(\"constructor\", () => {\n it(\"should initialize with basic config\", () => {\n const { createClient } = require(\"@deepgram/sdk\");\n\n const config = { apiKey: \"test-key\" };\n const provider = new DeepgramASRProvider(config);\n\n expect(provider.name).toBe(\"deepgram\");\n expect(createClient).toHaveBeenCalledWith({\n key: \"test-key\",\n });\n });\n\n it(\"should initialize with baseURL config\", () => {\n const { createClient } = require(\"@deepgram/sdk\");\n\n const config = {\n apiKey: \"test-key\",\n baseURL: \"https://custom.deepgram.com\",\n };\n const provider = new DeepgramASRProvider(config);\n\n expect(provider.name).toBe(\"deepgram\");\n expect(createClient).toHaveBeenCalledWith({\n key: \"test-key\",\n baseUrl: \"https://custom.deepgram.com\",\n });\n });\n\n it(\"should not include baseUrl when not provided\", () => {\n const { createClient } = require(\"@deepgram/sdk\");\n\n const config = { apiKey: \"test-key\" };\n new DeepgramASRProvider(config);\n\n expect(createClient).toHaveBeenCalledWith({\n key: \"test-key\",\n });\n });\n });\n\n describe(\"validateParams\", () => {\n it(\"should not throw for supported parameters\", () => {\n const params = {\n model: \"nova-2\",\n file: Buffer.from(\"test audio data\"),\n language: \"en-US\",\n timestamps: true,\n word_confidence: true,\n speaker_labels: true,\n smart_format: true,\n punctuate: true,\n diarize: true,\n utterances: true,\n };\n\n expect(() => provider.validateParams(params)).not.toThrow();\n });\n\n it(\"should accept additional parameters without warnings\", () => {\n const consoleSpy = jest.spyOn(console, \"warn\").mockImplementation();\n\n const params = {\n model: \"nova-2\",\n file: Buffer.from(\"test audio data\"),\n unsupported_param: \"value\",\n another_unsupported: true,\n };\n\n provider.validateParams(params);\n\n expect(consoleSpy).not.toHaveBeenCalled();\n\n consoleSpy.mockRestore();\n });\n\n it(\"should not warn for deepgram-specific parameters\", () => {\n const consoleSpy = jest.spyOn(console, \"warn\").mockImplementation();\n\n const params = {\n model: \"nova-2\",\n file: Buffer.from(\"test audio data\"),\n deepgram_custom_param: \"value\",\n deepgram_another_param: true,\n };\n\n provider.validateParams(params);\n\n expect(consoleSpy).not.toHaveBeenCalled();\n\n consoleSpy.mockRestore();\n });\n });\n\n describe(\"translateParams\", () => {\n it(\"should translate language parameter\", () => {\n const params = { language: \"en-US\" };\n const result = provider.translateParams(params);\n\n expect(result).toEqual({ language: \"en-US\" });\n });\n\n it(\"should translate timestamps to utterances\", () => {\n const params = { timestamps: true };\n const result = provider.translateParams(params);\n\n expect(result).toEqual({ utterances: true });\n });\n\n it(\"should pass through word_confidence parameter\", () => {\n const params = { word_confidence: true };\n const result = provider.translateParams(params);\n\n expect(result).toEqual({ word_confidence: true });\n });\n\n it(\"should pass through speaker_labels parameter\", () => {\n const params = { speaker_labels: true };\n const result = provider.translateParams(params);\n\n expect(result).toEqual({ speaker_labels: true });\n });\n\n it(\"should pass through deepgram-specific parameters\", () => {\n const params = {\n deepgram_custom_param: \"value\",\n deepgram_another_param: true,\n };\n const result = provider.translateParams(params);\n\n expect(result).toEqual({\n deepgram_custom_param: \"value\",\n deepgram_another_param: true,\n });\n });\n\n it(\"should pass through other parameters unchanged\", () => {\n const params = {\n temperature: 0.5,\n custom_param: \"value\",\n };\n const result = provider.translateParams(params);\n\n expect(result).toEqual({\n temperature: 0.5,\n custom_param: \"value\",\n });\n });\n\n it(\"should handle multiple parameter translations\", () => {\n const params = {\n language: \"en-US\",\n timestamps: true,\n word_confidence: true,\n speaker_labels: true,\n temperature: 0.5,\n };\n const result = provider.translateParams(params);\n\n expect(result).toEqual({\n language: \"en-US\",\n utterances: true,\n word_confidence: true,\n speaker_labels: true,\n temperature: 0.5,\n });\n });\n });\n\n describe(\"transcribe\", () => {\n const baseRequest: TranscriptionRequest = {\n model: \"nova-2\",\n file: Buffer.from(\"test audio data\"),\n };\n\n it(\"should successfully transcribe audio\", async () => {\n const mockDeepgramResponse = {\n result: {\n results: {\n channels: [\n {\n alternatives: [\n {\n transcript: \"Hello world\",\n confidence: 0.95,\n words: [\n {\n word: \"Hello\",\n start: 0.0,\n end: 0.5,\n confidence: 0.98,\n },\n {\n word: \"world\",\n start: 0.6,\n end: 1.0,\n confidence: 0.92,\n },\n ],\n },\n ],\n },\n ],\n utterances: [\n {\n transcript: \"Hello world\",\n start: 0.0,\n end: 1.0,\n speaker: 0,\n },\n ],\n },\n metadata: {\n language: \"en-US\",\n },\n },\n error: null,\n };\n\n mockDeepgramClient.listen.prerecorded.transcribeFile.mockResolvedValue(\n mockDeepgramResponse\n );\n\n const result = await provider.transcribe(baseRequest);\n\n expect(\n mockDeepgramClient.listen.prerecorded.transcribeFile\n ).toHaveBeenCalledWith(\n Buffer.from(\"test audio data\"),\n expect.objectContaining({\n model: \"nova-2\",\n })\n );\n\n expect(result).toEqual({\n text: \"Hello world\",\n language: \"en-US\",\n confidence: 0.95,\n words: [\n {\n text: \"Hello\",\n start: 0.0,\n end: 0.5,\n confidence: 0.98,\n },\n {\n text: \"world\",\n start: 0.6,\n end: 1.0,\n confidence: 0.92,\n },\n ],\n segments: [\n {\n text: \"Hello world\",\n start: 0.0,\n end: 1.0,\n speaker: \"0\",\n },\n ],\n });\n });\n\n it(\"should handle string file path\", async () => {\n const fs = require(\"fs\");\n jest\n .spyOn(fs, \"readFileSync\")\n .mockReturnValue(Buffer.from(\"test audio data\"));\n\n const request: TranscriptionRequest = {\n model: \"nova-2\",\n file: \"/path/to/audio.wav\",\n };\n\n const mockDeepgramResponse = {\n result: {\n results: {\n channels: [\n {\n alternatives: [\n {\n transcript: \"Test transcription\",\n confidence: 0.9,\n },\n ],\n },\n ],\n },\n metadata: {\n language: \"en-US\",\n },\n },\n error: null,\n };\n\n mockDeepgramClient.listen.prerecorded.transcribeFile.mockResolvedValue(\n mockDeepgramResponse\n );\n\n await provider.transcribe(request);\n\n expect(fs.readFileSync).toHaveBeenCalledWith(\"/path/to/audio.wav\");\n expect(\n mockDeepgramClient.listen.prerecorded.transcribeFile\n ).toHaveBeenCalledWith(\n Buffer.from(\"test audio data\"),\n expect.any(Object)\n );\n });\n\n it(\"should handle Uint8Array file\", async () => {\n const uint8Array = new Uint8Array([1, 2, 3, 4]);\n const request: TranscriptionRequest = {\n model: \"nova-2\",\n file: uint8Array,\n };\n\n const mockDeepgramResponse = {\n result: {\n results: {\n channels: [\n {\n alternatives: [\n {\n transcript: \"Test transcription\",\n confidence: 0.9,\n },\n ],\n },\n ],\n },\n metadata: {\n language: \"en-US\",\n },\n },\n error: null,\n };\n\n mockDeepgramClient.listen.prerecorded.transcribeFile.mockResolvedValue(\n mockDeepgramResponse\n );\n\n await provider.transcribe(request);\n\n expect(\n mockDeepgramClient.listen.prerecorded.transcribeFile\n ).toHaveBeenCalledWith(Buffer.from(uint8Array), expect.any(Object));\n });\n\n it(\"should handle unsupported file type gracefully\", async () => {\n const request: TranscriptionRequest = {\n model: \"nova-2\",\n file: \"unsupported\" as any,\n };\n\n // This test verifies that the provider handles unsupported file types\n // The actual error handling is tested in other scenarios\n await expect(provider.transcribe(request)).rejects.toThrow(AISuiteError);\n });\n\n it(\"should throw error when Deepgram API returns error\", async () => {\n const mockDeepgramResponse = {\n result: null,\n error: {\n message: \"API key invalid\",\n },\n };\n\n mockDeepgramClient.listen.prerecorded.transcribeFile.mockResolvedValue(\n mockDeepgramResponse\n );\n\n await expect(provider.transcribe(baseRequest)).rejects.toThrow(\n new AISuiteError(\n \"Deepgram API error: API key invalid\",\n \"deepgram\",\n \"API_ERROR\"\n )\n );\n });\n\n it(\"should handle Deepgram API exceptions\", async () => {\n const apiError = new Error(\"Network error\");\n mockDeepgramClient.listen.prerecorded.transcribeFile.mockRejectedValue(\n apiError\n );\n\n await expect(provider.transcribe(baseRequest)).rejects.toThrow(\n new AISuiteError(\n \"Deepgram ASR error: Network error\",\n \"deepgram\",\n \"API_ERROR\"\n )\n );\n });\n\n it(\"should pass timeout options\", async () => {\n const options = { timeout: 30000 };\n const mockDeepgramResponse = {\n result: {\n results: {\n channels: [\n {\n alternatives: [\n {\n transcript: \"Test transcription\",\n confidence: 0.9,\n },\n ],\n },\n ],\n },\n metadata: {\n language: \"en-US\",\n },\n },\n error: null,\n };\n\n mockDeepgramClient.listen.prerecorded.transcribeFile.mockResolvedValue(\n mockDeepgramResponse\n );\n\n await provider.transcribe(baseRequest, options);\n\n expect(\n mockDeepgramClient.listen.prerecorded.transcribeFile\n ).toHaveBeenCalledWith(Buffer.from(\"test audio data\"), {\n model: \"nova-2\", \n });\n });\n\n it(\"should translate parameters correctly\", async () => {\n const request: TranscriptionRequest = {\n model: \"nova-2\",\n file: Buffer.from(\"test audio data\"),\n language: \"en-US\",\n timestamps: true,\n word_confidence: true,\n speaker_labels: true,\n temperature: 0.5,\n };\n\n const mockDeepgramResponse = {\n result: {\n results: {\n channels: [\n {\n alternatives: [\n {\n transcript: \"Test transcription\",\n confidence: 0.9,\n },\n ],\n },\n ],\n },\n metadata: {\n language: \"en-US\",\n },\n },\n error: null,\n };\n\n mockDeepgramClient.listen.prerecorded.transcribeFile.mockResolvedValue(\n mockDeepgramResponse\n );\n\n await provider.transcribe(request);\n\n expect(\n mockDeepgramClient.listen.prerecorded.transcribeFile\n ).toHaveBeenCalledWith(\n Buffer.from(\"test audio data\"),\n expect.objectContaining({\n model: \"nova-2\",\n language: \"en-US\",\n utterances: true,\n word_confidence: true,\n speaker_labels: true,\n temperature: 0.5,\n })\n );\n });\n\n it(\"should handle response without words or utterances\", async () => {\n const mockDeepgramResponse = {\n result: {\n results: {\n channels: [\n {\n alternatives: [\n {\n transcript: \"Test transcription\",\n confidence: 0.9,\n },\n ],\n },\n ],\n },\n metadata: {\n language: \"en-US\",\n },\n },\n error: null,\n };\n\n mockDeepgramClient.listen.prerecorded.transcribeFile.mockResolvedValue(\n mockDeepgramResponse\n );\n\n // Ensure model is present in baseRequest\n const requestWithModel = { ...baseRequest, model: \"nova-2\" };\n const result = await provider.transcribe(requestWithModel);\n\n expect(result).toEqual({\n text: \"Test transcription\",\n language: \"en-US\",\n confidence: 0.9,\n words: [],\n segments: [],\n });\n });\n\n it(\"should handle malformed response gracefully\", async () => {\n const mockDeepgramResponse = {\n result: {\n results: {\n channels: [\n {\n alternatives: [\n {\n transcript: \"Test transcription\",\n },\n ],\n },\n ],\n },\n },\n error: null,\n };\n\n mockDeepgramClient.listen.prerecorded.transcribeFile.mockResolvedValue(\n mockDeepgramResponse\n );\n\n const result = await provider.transcribe(baseRequest);\n\n expect(result).toEqual({\n text: \"Test transcription\",\n language: \"unknown\",\n confidence: undefined,\n words: [],\n segments: [],\n });\n });\n });\n});\n" }, { "path": "aisuite-js/tests/providers/groq-provider.test.ts", "content": "import { GroqProvider } from \"../../src/providers/groq/provider\";\nimport { ChatCompletionRequest, ChatCompletionChunk } from \"../../src/types\";\nimport { AISuiteError } from \"../../src/core/errors\";\n\n// Mock the Groq SDK\njest.mock(\"groq-sdk\");\n\ndescribe(\"GroqProvider\", () => {\n let provider: GroqProvider;\n let mockGroqClient: any;\n\n beforeEach(() => {\n // Reset mocks\n jest.clearAllMocks();\n\n // Create mock Groq client\n mockGroqClient = {\n chat: {\n completions: {\n create: jest.fn(),\n },\n },\n };\n\n // Mock the Groq constructor\n const Groq = require(\"groq-sdk\");\n Groq.mockImplementation(() => mockGroqClient);\n\n // Create provider instance\n provider = new GroqProvider({\n apiKey: \"test-api-key\",\n });\n });\n\n describe(\"constructor\", () => {\n it(\"should initialize with basic config\", () => {\n const config = { apiKey: \"test-key\" };\n const provider = new GroqProvider(config);\n\n expect(provider.name).toBe(\"groq\");\n });\n\n it(\"should initialize with baseURL config\", () => {\n const config = {\n apiKey: \"test-key\",\n baseURL: \"https://custom.groq.com\",\n };\n const provider = new GroqProvider(config);\n\n expect(provider.name).toBe(\"groq\");\n expect(mockGroqClient.baseURL).toBe(\"https://custom.groq.com\");\n });\n });\n\n describe(\"chatCompletion\", () => {\n const baseRequest: ChatCompletionRequest = {\n model: \"llama3-8b-8192\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n };\n\n it(\"should successfully complete chat\", async () => {\n const mockResponse = {\n id: \"chatcmpl-123\",\n object: \"chat.completion\",\n created: 1234567890,\n model: \"llama3-8b-8192\",\n choices: [\n {\n index: 0,\n message: {\n role: \"assistant\",\n content: \"Hello! How can I help you?\",\n },\n finish_reason: \"stop\",\n },\n ],\n usage: {\n prompt_tokens: 10,\n completion_tokens: 20,\n total_tokens: 30,\n },\n };\n\n mockGroqClient.chat.completions.create.mockResolvedValue(mockResponse);\n\n const result = await provider.chatCompletion(baseRequest);\n\n expect(mockGroqClient.chat.completions.create).toHaveBeenCalledWith(\n expect.objectContaining({\n model: \"llama3-8b-8192\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n })\n );\n expect(result).toEqual(\n expect.objectContaining({\n id: \"chatcmpl-123\",\n object: \"chat.completion\",\n model: \"groq:llama3-8b-8192\",\n })\n );\n });\n\n it(\"should throw error when streaming is enabled\", async () => {\n const request: ChatCompletionRequest = {\n ...baseRequest,\n stream: true,\n };\n\n await expect(provider.chatCompletion(request)).rejects.toThrow(\n AISuiteError\n );\n await expect(provider.chatCompletion(request)).rejects.toThrow(\n \"Streaming is not supported in non-streaming method\"\n );\n });\n\n it(\"should handle API errors\", async () => {\n const apiError = new Error(\"API rate limit exceeded\");\n mockGroqClient.chat.completions.create.mockRejectedValue(apiError);\n\n await expect(provider.chatCompletion(baseRequest)).rejects.toThrow(\n AISuiteError\n );\n await expect(provider.chatCompletion(baseRequest)).rejects.toThrow(\n \"Groq API error: API rate limit exceeded\"\n );\n });\n\n it(\"should handle complex request with all parameters\", async () => {\n const complexRequest: ChatCompletionRequest = {\n model: \"llama3-8b-8192\",\n messages: [\n { role: \"system\", content: \"You are a helpful assistant\" },\n { role: \"user\", content: \"What is 2+2?\" },\n ],\n temperature: 0.7,\n max_tokens: 100,\n top_p: 0.9,\n frequency_penalty: 0.1,\n presence_penalty: 0.1,\n stop: [\"\\n\"],\n user: \"user-123\",\n };\n\n const mockResponse = {\n id: \"chatcmpl-123\",\n object: \"chat.completion\",\n created: 1234567890,\n model: \"llama3-8b-8192\",\n choices: [\n {\n index: 0,\n message: { role: \"assistant\", content: \"2+2 equals 4\" },\n finish_reason: \"stop\",\n },\n ],\n usage: { prompt_tokens: 15, completion_tokens: 5, total_tokens: 20 },\n };\n\n mockGroqClient.chat.completions.create.mockResolvedValue(mockResponse);\n\n const result = await provider.chatCompletion(complexRequest);\n\n expect(mockGroqClient.chat.completions.create).toHaveBeenCalledWith(\n expect.objectContaining({\n model: \"llama3-8b-8192\",\n messages: complexRequest.messages,\n temperature: 0.7,\n max_tokens: 100,\n stream: undefined,\n tool_choice: undefined,\n tools: undefined,\n })\n );\n expect(result).toEqual(\n expect.objectContaining({\n id: \"chatcmpl-123\",\n object: \"chat.completion\",\n })\n );\n });\n });\n\n describe(\"streamChatCompletion\", () => {\n const baseRequest: ChatCompletionRequest = {\n model: \"llama3-8b-8192\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n };\n\n it(\"should stream chat completion\", async () => {\n const mockChunks = [\n {\n id: \"chatcmpl-123\",\n object: \"chat.completion.chunk\",\n created: 1234567890,\n model: \"llama3-8b-8192\",\n choices: [\n {\n index: 0,\n delta: { role: \"assistant\", content: \"Hello\" },\n finish_reason: null,\n },\n ],\n },\n {\n id: \"chatcmpl-123\",\n object: \"chat.completion.chunk\",\n created: 1234567890,\n model: \"llama3-8b-8192\",\n choices: [\n {\n index: 0,\n delta: { content: \"! How can I help?\" },\n finish_reason: \"stop\",\n },\n ],\n },\n ];\n\n const mockStream = (async function* () {\n for (const chunk of mockChunks) {\n yield chunk;\n }\n })();\n\n mockGroqClient.chat.completions.create.mockResolvedValue(mockStream);\n\n const stream = provider.streamChatCompletion(baseRequest);\n const chunks: ChatCompletionChunk[] = [];\n\n for await (const chunk of stream) {\n chunks.push(chunk);\n }\n\n expect(mockGroqClient.chat.completions.create).toHaveBeenCalledWith(\n expect.objectContaining({\n model: \"llama3-8b-8192\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n })\n );\n expect(chunks).toHaveLength(2);\n expect(chunks[0]).toEqual(\n expect.objectContaining({\n object: \"chat.completion.chunk\",\n })\n );\n });\n\n it(\"should handle streaming errors\", async () => {\n const streamError = new Error(\"Streaming connection failed\");\n mockGroqClient.chat.completions.create.mockRejectedValue(streamError);\n\n await expect(async () => {\n const stream = provider.streamChatCompletion(baseRequest);\n for await (const chunk of stream) {\n // This should not be reached\n }\n }).rejects.toThrow(AISuiteError);\n\n await expect(async () => {\n const stream = provider.streamChatCompletion(baseRequest);\n for await (const chunk of stream) {\n // This should not be reached\n }\n }).rejects.toThrow(\"Groq streaming error: Streaming connection failed\");\n });\n\n it(\"should handle abort signal\", async () => {\n const mockStream = (async function* () {\n yield {\n id: \"chatcmpl-123\",\n object: \"chat.completion.chunk\",\n created: 1234567890,\n model: \"llama3-8b-8192\",\n choices: [\n {\n index: 0,\n delta: { role: \"assistant\", content: \"Hello!\" },\n finish_reason: \"stop\",\n },\n ],\n };\n })();\n\n mockGroqClient.chat.completions.create.mockResolvedValue(mockStream);\n\n const abortController = new AbortController();\n const options = { signal: abortController.signal };\n\n const stream = provider.streamChatCompletion(baseRequest, options);\n const chunks: ChatCompletionChunk[] = [];\n\n // Start consuming the stream\n const consumePromise = (async () => {\n for await (const chunk of stream) {\n chunks.push(chunk);\n }\n })();\n\n // Abort after a short delay\n setTimeout(() => {\n abortController.abort();\n }, 10);\n\n await consumePromise;\n\n expect(chunks.length).toBeGreaterThan(0);\n });\n\n it(\"should handle complex streaming request\", async () => {\n const complexRequest: ChatCompletionRequest = {\n model: \"llama3-8b-8192\",\n messages: [\n { role: \"system\", content: \"You are a helpful assistant\" },\n { role: \"user\", content: \"Tell me a story\" },\n ],\n temperature: 0.8,\n max_tokens: 200,\n top_p: 0.9,\n stop: [\"END\"],\n };\n\n const mockStream = (async function* () {\n yield {\n id: \"chatcmpl-123\",\n object: \"chat.completion.chunk\",\n created: 1234567890,\n model: \"llama3-8b-8192\",\n choices: [\n {\n index: 0,\n delta: { role: \"assistant\", content: \"Once upon a time\" },\n finish_reason: null,\n },\n ],\n };\n yield {\n id: \"chatcmpl-123\",\n object: \"chat.completion.chunk\",\n created: 1234567890,\n model: \"llama3-8b-8192\",\n choices: [\n {\n index: 0,\n delta: { content: \" there was a brave knight.\" },\n finish_reason: \"stop\",\n },\n ],\n };\n })();\n\n mockGroqClient.chat.completions.create.mockResolvedValue(mockStream);\n\n const stream = provider.streamChatCompletion(complexRequest);\n const chunks: ChatCompletionChunk[] = [];\n\n for await (const chunk of stream) {\n chunks.push(chunk);\n }\n\n expect(mockGroqClient.chat.completions.create).toHaveBeenCalledWith(\n expect.objectContaining({\n model: \"llama3-8b-8192\",\n messages: complexRequest.messages,\n temperature: 0.8,\n max_tokens: 200,\n stream: undefined,\n tool_choice: undefined,\n tools: undefined,\n })\n );\n expect(chunks).toHaveLength(2);\n });\n });\n\n describe(\"error handling\", () => {\n it(\"should preserve AISuiteError instances\", async () => {\n const customError = new AISuiteError(\n \"Custom error\",\n \"groq\",\n \"CUSTOM_ERROR\"\n );\n\n mockGroqClient.chat.completions.create.mockRejectedValue(customError);\n\n await expect(\n provider.chatCompletion({\n model: \"llama3-8b-8192\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n })\n ).rejects.toThrow(customError);\n });\n\n it(\"should handle unknown error types\", async () => {\n const unknownError = \"Unknown error string\";\n mockGroqClient.chat.completions.create.mockRejectedValue(unknownError);\n\n await expect(\n provider.chatCompletion({\n model: \"llama3-8b-8192\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n })\n ).rejects.toThrow(\"Groq API error: Unknown error\");\n });\n\n it(\"should handle streaming unknown error types\", async () => {\n const unknownError = \"Unknown streaming error\";\n mockGroqClient.chat.completions.create.mockRejectedValue(unknownError);\n\n const stream = provider.streamChatCompletion({\n model: \"llama3-8b-8192\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n });\n\n await expect(async () => {\n for await (const chunk of stream) {\n // This should not be reached\n }\n }).rejects.toThrow(\"Groq streaming error: Unknown error\");\n });\n });\n});\n" }, { "path": "aisuite-js/tests/providers/mistral-provider.test.ts", "content": "import { MistralProvider } from \"../../src/providers/mistral/provider\";\nimport { ChatCompletionRequest, ChatCompletionChunk } from \"../../src/types\";\nimport { AISuiteError } from \"../../src/core/errors\";\n\n// Mock the Mistral SDK\njest.mock(\"@mistralai/mistralai\", () => {\n return {\n __esModule: true,\n default: jest.fn(),\n };\n});\n\ndescribe(\"MistralProvider\", () => {\n let provider: MistralProvider;\n let mockMistralClient: any;\n\n beforeEach(() => {\n // Reset mocks\n jest.clearAllMocks();\n\n // Create mock Mistral client\n mockMistralClient = {\n chat: jest.fn(),\n chatStream: jest.fn(),\n };\n\n // Mock the MistralClient constructor\n const MistralClient = require(\"@mistralai/mistralai\");\n MistralClient.default.mockImplementation(() => mockMistralClient);\n\n // Create provider instance\n provider = new MistralProvider({\n apiKey: \"test-api-key\",\n });\n });\n\n describe(\"constructor\", () => {\n it(\"should initialize with basic config\", () => {\n const config = { apiKey: \"test-key\" };\n const provider = new MistralProvider(config);\n\n expect(provider.name).toBe(\"mistral\");\n });\n\n it(\"should initialize with baseURL config\", () => {\n const config = {\n apiKey: \"test-key\",\n baseURL: \"https://custom.mistral.com\",\n };\n const provider = new MistralProvider(config);\n\n expect(provider.name).toBe(\"mistral\");\n expect(mockMistralClient.baseURL).toBe(\"https://custom.mistral.com\");\n });\n });\n\n describe(\"chatCompletion\", () => {\n const baseRequest: ChatCompletionRequest = {\n model: \"mistral-large\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n };\n\n it(\"should successfully complete chat\", async () => {\n const mockResponse = {\n id: \"chatcmpl-123\",\n object: \"chat.completion\",\n created: 1234567890,\n model: \"mistral-large\",\n choices: [\n {\n index: 0,\n message: {\n role: \"assistant\",\n content: \"Hello! How can I help you?\",\n },\n finish_reason: \"stop\",\n },\n ],\n usage: {\n prompt_tokens: 10,\n completion_tokens: 20,\n total_tokens: 30,\n },\n };\n\n mockMistralClient.chat.mockResolvedValue(mockResponse);\n\n const result = await provider.chatCompletion(baseRequest);\n\n expect(mockMistralClient.chat).toHaveBeenCalledWith(\n expect.objectContaining({\n model: \"mistral-large\",\n })\n );\n expect(result).toEqual(\n expect.objectContaining({\n id: \"chatcmpl-123\",\n object: \"chat.completion\",\n model: \"mistral-large\",\n })\n );\n });\n\n it(\"should throw error when streaming is enabled\", async () => {\n const request: ChatCompletionRequest = {\n ...baseRequest,\n stream: true,\n };\n\n await expect(provider.chatCompletion(request)).rejects.toThrow(\n AISuiteError\n );\n await expect(provider.chatCompletion(request)).rejects.toThrow(\n \"Streaming is not supported in non-streaming method\"\n );\n });\n\n it(\"should handle API errors\", async () => {\n const apiError = new Error(\"API rate limit exceeded\");\n mockMistralClient.chat.mockRejectedValue(apiError);\n\n await expect(provider.chatCompletion(baseRequest)).rejects.toThrow(\n AISuiteError\n );\n await expect(provider.chatCompletion(baseRequest)).rejects.toThrow(\n \"Mistral API error: API rate limit exceeded\"\n );\n });\n\n it(\"should handle complex request with all parameters\", async () => {\n const complexRequest: ChatCompletionRequest = {\n model: \"mistral-large\",\n messages: [\n { role: \"system\", content: \"You are a helpful assistant\" },\n { role: \"user\", content: \"What is 2+2?\" },\n ],\n temperature: 0.7,\n max_tokens: 100,\n top_p: 0.9,\n frequency_penalty: 0.1,\n presence_penalty: 0.1,\n stop: [\"\\n\"],\n user: \"user-123\",\n };\n\n const mockResponse = {\n id: \"chatcmpl-123\",\n object: \"chat.completion\",\n created: 1234567890,\n model: \"mistral-large\",\n choices: [\n {\n index: 0,\n message: { role: \"assistant\", content: \"2+2 equals 4\" },\n finish_reason: \"stop\",\n },\n ],\n usage: { prompt_tokens: 15, completion_tokens: 5, total_tokens: 20 },\n };\n\n mockMistralClient.chat.mockResolvedValue(mockResponse);\n\n const result = await provider.chatCompletion(complexRequest);\n\n expect(mockMistralClient.chat).toHaveBeenCalledWith(\n expect.objectContaining({\n model: \"mistral-large\",\n })\n );\n expect(result).toEqual(\n expect.objectContaining({\n id: \"chatcmpl-123\",\n object: \"chat.completion\",\n })\n );\n });\n });\n\n describe(\"streamChatCompletion\", () => {\n const baseRequest: ChatCompletionRequest = {\n model: \"mistral-large\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n };\n\n it(\"should stream chat completion\", async () => {\n const mockChunks = [\n {\n id: \"chatcmpl-123\",\n object: \"chat.completion.chunk\",\n created: 1234567890,\n model: \"mistral-large\",\n choices: [\n {\n index: 0,\n delta: { role: \"assistant\", content: \"Hello\" },\n finish_reason: null,\n },\n ],\n },\n {\n id: \"chatcmpl-123\",\n object: \"chat.completion.chunk\",\n created: 1234567890,\n model: \"mistral-large\",\n choices: [\n {\n index: 0,\n delta: { content: \"! How can I help?\" },\n finish_reason: \"stop\",\n },\n ],\n },\n ];\n\n const mockStream = (async function* () {\n for (const chunk of mockChunks) {\n yield chunk;\n }\n })();\n\n mockMistralClient.chatStream.mockResolvedValue(mockStream);\n\n const stream = provider.streamChatCompletion(baseRequest);\n const chunks: ChatCompletionChunk[] = [];\n\n for await (const chunk of stream) {\n chunks.push(chunk);\n }\n\n expect(mockMistralClient.chatStream).toHaveBeenCalledWith(\n expect.objectContaining({\n model: \"mistral-large\",\n })\n );\n expect(chunks).toHaveLength(2);\n expect(typeof chunks[0].id).toBe(\"string\");\n expect(chunks[0].object).toBe(\"chat.completion.chunk\");\n });\n\n it(\"should handle streaming errors\", async () => {\n mockMistralClient.chatStream.mockRejectedValue(\n new Error(\"Streaming error\")\n );\n const stream = provider.streamChatCompletion(baseRequest);\n const iterator = stream[Symbol.asyncIterator]();\n await expect(iterator.next()).rejects.toThrow(AISuiteError);\n });\n\n it(\"should handle abort signal\", async () => {\n const mockStream = (async function* () {\n yield {\n id: \"chatcmpl-123\",\n object: \"chat.completion.chunk\",\n created: 1234567890,\n model: \"mistral-large\",\n choices: [\n {\n index: 0,\n delta: { role: \"assistant\", content: \"Hello!\" },\n finish_reason: \"stop\",\n },\n ],\n };\n })();\n\n mockMistralClient.chatStream.mockResolvedValue(mockStream);\n\n const abortController = new AbortController();\n const options = { signal: abortController.signal };\n\n const stream = provider.streamChatCompletion(baseRequest, options);\n const chunks: ChatCompletionChunk[] = [];\n\n // Start consuming the stream\n const consumePromise = (async () => {\n for await (const chunk of stream) {\n chunks.push(chunk);\n }\n })();\n\n // Abort after a short delay\n setTimeout(() => {\n abortController.abort();\n }, 10);\n\n await consumePromise;\n\n expect(chunks.length).toBeGreaterThan(0);\n });\n\n it(\"should handle complex streaming request\", async () => {\n const complexRequest: ChatCompletionRequest = {\n model: \"mistral-large\",\n messages: [\n { role: \"system\", content: \"You are a helpful assistant\" },\n { role: \"user\", content: \"Tell me a story\" },\n ],\n temperature: 0.8,\n max_tokens: 200,\n top_p: 0.9,\n stop: [\"END\"],\n };\n\n const mockStream = (async function* () {\n yield {\n id: \"chatcmpl-123\",\n object: \"chat.completion.chunk\",\n created: 1234567890,\n model: \"mistral-large\",\n choices: [\n {\n index: 0,\n delta: { role: \"assistant\", content: \"Once upon a time\" },\n finish_reason: null,\n },\n ],\n };\n yield {\n id: \"chatcmpl-123\",\n object: \"chat.completion.chunk\",\n created: 1234567890,\n model: \"mistral-large\",\n choices: [\n {\n index: 0,\n delta: { content: \" there was a brave knight.\" },\n finish_reason: \"stop\",\n },\n ],\n };\n })();\n\n mockMistralClient.chatStream.mockResolvedValue(mockStream);\n\n const stream = provider.streamChatCompletion(complexRequest);\n const chunks: ChatCompletionChunk[] = [];\n\n for await (const chunk of stream) {\n chunks.push(chunk);\n }\n\n expect(mockMistralClient.chatStream).toHaveBeenCalledWith(\n expect.objectContaining({\n model: \"mistral-large\",\n })\n );\n expect(chunks).toHaveLength(2);\n });\n });\n\n describe(\"error handling\", () => {\n it(\"should preserve AISuiteError instances\", async () => {\n const customError = new AISuiteError(\n \"Custom error\",\n \"mistral\",\n \"CUSTOM_ERROR\"\n );\n\n mockMistralClient.chat.mockRejectedValue(customError);\n\n await expect(\n provider.chatCompletion({\n model: \"mistral-large\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n })\n ).rejects.toThrow(customError);\n });\n\n it(\"should handle unknown error types\", async () => {\n const unknownError = \"Unknown error string\";\n mockMistralClient.chat.mockRejectedValue(unknownError);\n\n await expect(\n provider.chatCompletion({\n model: \"mistral-large\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n })\n ).rejects.toThrow(\"Mistral API error: Unknown error\");\n });\n\n it(\"should handle streaming unknown error types\", async () => {\n const unknownError = \"Unknown streaming error\";\n mockMistralClient.chatStream.mockRejectedValue(unknownError);\n\n const stream = provider.streamChatCompletion({\n model: \"mistral-large\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n });\n\n await expect(async () => {\n for await (const chunk of stream) {\n // This should not be reached\n }\n }).rejects.toThrow(/Mistral streaming error: Unknown/);\n });\n });\n});\n" }, { "path": "aisuite-js/tests/providers/openai-provider.test.ts", "content": "import { OpenAIProvider } from \"../../src/providers/openai/provider\";\nimport { ChatCompletionRequest, ChatCompletionChunk } from \"../../src/types\";\nimport { AISuiteError } from \"../../src/core/errors\";\n\n// Mock the OpenAI SDK\njest.mock(\"openai\");\n\ndescribe(\"OpenAIProvider\", () => {\n let provider: OpenAIProvider;\n let mockOpenAIClient: any;\n\n beforeEach(() => {\n // Reset mocks\n jest.clearAllMocks();\n\n // Create mock OpenAI client\n mockOpenAIClient = {\n chat: {\n completions: {\n create: jest.fn(),\n },\n },\n };\n\n // Mock the OpenAI constructor\n const OpenAI = require(\"openai\");\n OpenAI.mockImplementation(() => mockOpenAIClient);\n\n // Create provider instance\n provider = new OpenAIProvider({\n apiKey: \"test-api-key\",\n });\n });\n\n describe(\"constructor\", () => {\n it(\"should initialize with basic config\", () => {\n const config = { apiKey: \"test-key\" };\n const provider = new OpenAIProvider(config);\n\n expect(provider.name).toBe(\"openai\");\n });\n\n it(\"should initialize with full config\", () => {\n const config = {\n apiKey: \"test-key\",\n baseURL: \"https://custom.openai.com\",\n organization: \"org-123\",\n };\n const provider = new OpenAIProvider(config);\n\n expect(provider.name).toBe(\"openai\");\n });\n });\n\n describe(\"chatCompletion\", () => {\n const baseRequest: ChatCompletionRequest = {\n model: \"gpt-4\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n };\n\n it(\"should successfully complete chat\", async () => {\n const mockResponse = {\n id: \"chatcmpl-123\",\n object: \"chat.completion\",\n created: 1234567890,\n model: \"gpt-4\",\n choices: [\n {\n index: 0,\n message: {\n role: \"assistant\",\n content: \"Hello! How can I help you?\",\n },\n finish_reason: \"stop\",\n },\n ],\n usage: {\n prompt_tokens: 10,\n completion_tokens: 20,\n total_tokens: 30,\n },\n };\n\n mockOpenAIClient.chat.completions.create.mockResolvedValue(mockResponse);\n\n const result = await provider.chatCompletion(baseRequest);\n\n expect(mockOpenAIClient.chat.completions.create).toHaveBeenCalledWith(\n expect.objectContaining({\n model: \"gpt-4\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n }),\n undefined\n );\n expect(result).toEqual(\n expect.objectContaining({\n id: \"chatcmpl-123\",\n object: \"chat.completion\",\n model: \"gpt-4\",\n })\n );\n });\n\n it(\"should throw error when streaming is enabled\", async () => {\n const request: ChatCompletionRequest = {\n ...baseRequest,\n stream: true,\n };\n\n await expect(provider.chatCompletion(request)).rejects.toThrow(\n AISuiteError\n );\n await expect(provider.chatCompletion(request)).rejects.toThrow(\n \"Streaming is not yet supported\"\n );\n });\n\n it(\"should handle API errors\", async () => {\n const apiError = new Error(\"API rate limit exceeded\");\n mockOpenAIClient.chat.completions.create.mockRejectedValue(apiError);\n\n await expect(provider.chatCompletion(baseRequest)).rejects.toThrow(\n AISuiteError\n );\n await expect(provider.chatCompletion(baseRequest)).rejects.toThrow(\n \"OpenAI API error: API rate limit exceeded\"\n );\n });\n\n it(\"should pass options to the client\", async () => {\n const options = { signal: new AbortController().signal };\n const mockResponse = {\n id: \"chatcmpl-123\",\n object: \"chat.completion\",\n created: 1234567890,\n model: \"gpt-4\",\n choices: [],\n usage: { prompt_tokens: 10, completion_tokens: 20, total_tokens: 30 },\n };\n\n mockOpenAIClient.chat.completions.create.mockResolvedValue(mockResponse);\n\n await provider.chatCompletion(baseRequest, options);\n\n expect(mockOpenAIClient.chat.completions.create).toHaveBeenCalledWith(\n expect.any(Object),\n options\n );\n });\n\n it(\"should handle complex request with all parameters\", async () => {\n const complexRequest: ChatCompletionRequest = {\n model: \"gpt-4\",\n messages: [\n { role: \"system\", content: \"You are a helpful assistant\" },\n { role: \"user\", content: \"What is 2+2?\" },\n ],\n temperature: 0.7,\n max_tokens: 100,\n top_p: 0.9,\n frequency_penalty: 0.1,\n presence_penalty: 0.1,\n stop: [\"\\n\"],\n user: \"user-123\",\n };\n\n const mockResponse = {\n id: \"chatcmpl-123\",\n object: \"chat.completion\",\n created: 1234567890,\n model: \"gpt-4\",\n choices: [\n {\n index: 0,\n message: { role: \"assistant\", content: \"2+2 equals 4\" },\n finish_reason: \"stop\",\n },\n ],\n usage: { prompt_tokens: 15, completion_tokens: 5, total_tokens: 20 },\n };\n\n mockOpenAIClient.chat.completions.create.mockResolvedValue(mockResponse);\n\n const result = await provider.chatCompletion(complexRequest);\n\n expect(mockOpenAIClient.chat.completions.create).toHaveBeenCalledWith(\n expect.objectContaining({\n model: \"gpt-4\",\n messages: complexRequest.messages,\n temperature: 0.7,\n max_tokens: 100,\n top_p: 0.9,\n frequency_penalty: 0.1,\n presence_penalty: 0.1,\n stop: [\"\\n\"],\n user: \"user-123\",\n }),\n undefined\n );\n expect(result).toEqual(\n expect.objectContaining({\n id: \"chatcmpl-123\",\n object: \"chat.completion\",\n })\n );\n });\n });\n\n describe(\"streamChatCompletion\", () => {\n const baseRequest: ChatCompletionRequest = {\n model: \"gpt-4\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n };\n\n it(\"should stream chat completion\", async () => {\n const mockChunks = [\n {\n id: \"chatcmpl-123\",\n object: \"chat.completion.chunk\",\n created: 1234567890,\n model: \"gpt-4\",\n choices: [\n {\n index: 0,\n delta: { role: \"assistant\", content: \"Hello\" },\n finish_reason: null,\n },\n ],\n },\n {\n id: \"chatcmpl-123\",\n object: \"chat.completion.chunk\",\n created: 1234567890,\n model: \"gpt-4\",\n choices: [\n {\n index: 0,\n delta: { content: \"! How can I help?\" },\n finish_reason: \"stop\",\n },\n ],\n },\n ];\n\n const mockStream = (async function* () {\n for (const chunk of mockChunks) {\n yield chunk;\n }\n })();\n\n mockOpenAIClient.chat.completions.create.mockResolvedValue(mockStream);\n\n const stream = provider.streamChatCompletion(baseRequest);\n const chunks: ChatCompletionChunk[] = [];\n\n for await (const chunk of stream) {\n chunks.push(chunk);\n }\n\n expect(mockOpenAIClient.chat.completions.create).toHaveBeenCalledWith(\n expect.objectContaining({\n model: \"gpt-4\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n stream: true,\n }),\n undefined\n );\n expect(chunks).toHaveLength(2);\n expect(chunks[0]).toEqual(\n expect.objectContaining({\n id: \"chatcmpl-123\",\n object: \"chat.completion.chunk\",\n })\n );\n });\n\n it(\"should handle streaming errors\", async () => {\n const streamError = new Error(\"Streaming connection failed\");\n mockOpenAIClient.chat.completions.create.mockRejectedValue(streamError);\n\n await expect(async () => {\n const stream = provider.streamChatCompletion(baseRequest);\n for await (const chunk of stream) {\n // This should not be reached\n }\n }).rejects.toThrow(AISuiteError);\n\n await expect(async () => {\n const stream = provider.streamChatCompletion(baseRequest);\n for await (const chunk of stream) {\n // This should not be reached\n }\n }).rejects.toThrow(\"OpenAI streaming error: Streaming connection failed\");\n });\n\n it(\"should pass options to streaming request\", async () => {\n const options = { signal: new AbortController().signal };\n const mockStream = (async function* () {\n yield {\n id: \"chatcmpl-123\",\n object: \"chat.completion.chunk\",\n created: 1234567890,\n model: \"gpt-4\",\n choices: [\n {\n index: 0,\n delta: { role: \"assistant\", content: \"Hello!\" },\n finish_reason: \"stop\",\n },\n ],\n };\n })();\n\n mockOpenAIClient.chat.completions.create.mockResolvedValue(mockStream);\n\n const stream = provider.streamChatCompletion(baseRequest, options);\n const chunks: ChatCompletionChunk[] = [];\n\n for await (const chunk of stream) {\n chunks.push(chunk);\n }\n\n expect(mockOpenAIClient.chat.completions.create).toHaveBeenCalledWith(\n expect.objectContaining({\n model: \"gpt-4\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n stream: true,\n }),\n options\n );\n expect(chunks).toHaveLength(1);\n });\n });\n\n describe(\"error handling\", () => {\n it(\"should preserve AISuiteError instances\", async () => {\n const customError = new AISuiteError(\n \"Custom error\",\n \"openai\",\n \"CUSTOM_ERROR\"\n );\n\n mockOpenAIClient.chat.completions.create.mockRejectedValue(customError);\n\n await expect(\n provider.chatCompletion({\n model: \"gpt-4\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n })\n ).rejects.toThrow(customError);\n });\n\n it(\"should handle unknown error types\", async () => {\n const unknownError = \"Unknown error string\";\n mockOpenAIClient.chat.completions.create.mockRejectedValue(unknownError);\n\n await expect(\n provider.chatCompletion({\n model: \"gpt-4\",\n messages: [{ role: \"user\", content: \"Hello\" }],\n })\n ).rejects.toThrow(\"OpenAI API error: Unknown error\");\n });\n });\n});\n" }, { "path": "aisuite-js/tests/providers/openai_asr_provider.test.ts", "content": "import { TranscriptionRequest } from \"../../src/types\";\nimport { AISuiteError } from \"../../src/core/errors\";\nimport { OpenAIProvider } from \"../../src/providers/openai\";\n\ndescribe(\"OpenAIProvider\", () => {\n let provider: OpenAIProvider;\n\n beforeEach(() => {\n provider = new OpenAIProvider({\n apiKey: \"test-api-key\",\n });\n });\n\n describe(\"validateParams\", () => {\n it(\"should not throw for supported parameters\", () => {\n const params = {\n language: \"en\",\n prompt: \"test prompt\",\n response_format: \"json\",\n temperature: 0.5,\n timestamps: true,\n model: \"whisper-1\",\n file: Buffer.from(\"test\")\n };\n\n expect(() => provider.validateParams(params)).not.toThrow();\n });\n\n it(\"should validate required parameters\", () => {\n const params = {\n unsupported_param: \"value\",\n model: \"whisper-1\",\n file: Buffer.from(\"test\")\n };\n\n expect(() => provider.validateParams(params)).not.toThrow();\n });\n });\n\n describe(\"translateParams\", () => {\n it(\"should translate standard parameters correctly\", () => {\n const params = {\n language: \"en\",\n prompt: \"test prompt\",\n response_format: \"json\",\n temperature: 0.5,\n timestamps: true,\n model: \"whisper-1\",\n file: Buffer.from(\"test\")\n };\n\n const translated = provider.translateParams(params);\n expect(translated).toEqual({\n language: \"en\",\n prompt: \"test prompt\",\n response_format: \"json\",\n temperature: 0.5,\n timestamps: true,\n });\n });\n\n it(\"should retain other parameters\", () => {\n const params = {\n custom_param: \"value\",\n model: \"whisper-1\",\n file: Buffer.from(\"test\")\n };\n\n const translated = provider.translateParams(params);\n expect(translated).toEqual({\n custom_param: \"value\"\n });\n });\n });\n\n describe(\"transcribe\", () => {\n it(\"should throw error if file is not provided\", async () => {\n const request: TranscriptionRequest = {\n model: \"openai:whisper-1\",\n file: \"\",\n };\n\n await expect(provider.transcribe(request)).rejects.toThrow(AISuiteError);\n });\n });\n});\n" }, { "path": "aisuite-js/tests/utils/streaming.test.ts", "content": "import { createChunk, generateId } from \"../../src/utils/streaming\";\n\ndescribe(\"Streaming Utils\", () => {\n describe(\"createChunk\", () => {\n it(\"should create a basic chunk with required fields\", () => {\n const id = \"test-chunk-id\";\n const model = \"gpt-4\";\n\n const chunk = createChunk(id, model);\n\n expect(chunk).toEqual({\n id,\n object: \"chat.completion.chunk\",\n created: expect.any(Number),\n model,\n choices: [\n {\n index: 0,\n delta: {\n role: \"assistant\",\n content: undefined,\n tool_calls: undefined,\n },\n finish_reason: undefined,\n },\n ],\n });\n });\n\n it(\"should create a chunk with content\", () => {\n const id = \"test-chunk-id\";\n const model = \"claude-3-sonnet\";\n const content = \"Hello, world!\";\n\n const chunk = createChunk(id, model, content);\n\n expect(chunk).toEqual({\n id,\n object: \"chat.completion.chunk\",\n created: expect.any(Number),\n model,\n choices: [\n {\n index: 0,\n delta: {\n role: \"assistant\",\n content,\n tool_calls: undefined,\n },\n finish_reason: undefined,\n },\n ],\n });\n });\n\n it(\"should create a chunk with finish reason\", () => {\n const id = \"test-chunk-id\";\n const model = \"mistral-large\";\n const finishReason = \"stop\";\n\n const chunk = createChunk(id, model, undefined, finishReason);\n\n expect(chunk).toEqual({\n id,\n object: \"chat.completion.chunk\",\n created: expect.any(Number),\n model,\n choices: [\n {\n index: 0,\n delta: {\n role: \"assistant\",\n content: undefined,\n tool_calls: undefined,\n },\n finish_reason: finishReason,\n },\n ],\n });\n });\n\n it(\"should create a chunk with tool calls\", () => {\n const id = \"test-chunk-id\";\n const model = \"gpt-4\";\n const toolCalls = [\n {\n id: \"call-1\",\n type: \"function\",\n function: {\n name: \"get_weather\",\n arguments: '{\"location\": \"New York\"}',\n },\n },\n ];\n\n const chunk = createChunk(id, model, undefined, undefined, toolCalls);\n\n expect(chunk).toEqual({\n id,\n object: \"chat.completion.chunk\",\n created: expect.any(Number),\n model,\n choices: [\n {\n index: 0,\n delta: {\n role: \"assistant\",\n content: undefined,\n tool_calls: toolCalls,\n },\n finish_reason: undefined,\n },\n ],\n });\n });\n\n it(\"should create a complete chunk with all parameters\", () => {\n const id = \"test-chunk-id\";\n const model = \"gpt-4\";\n const content = \"The weather is sunny\";\n const finishReason = \"stop\";\n const toolCalls = [\n {\n id: \"call-1\",\n type: \"function\",\n function: {\n name: \"get_weather\",\n arguments: '{\"location\": \"New York\"}',\n },\n },\n ];\n\n const chunk = createChunk(id, model, content, finishReason, toolCalls);\n\n expect(chunk).toEqual({\n id,\n object: \"chat.completion.chunk\",\n created: expect.any(Number),\n model,\n choices: [\n {\n index: 0,\n delta: {\n role: \"assistant\",\n content,\n tool_calls: toolCalls,\n },\n finish_reason: finishReason,\n },\n ],\n });\n });\n\n it(\"should set created timestamp to current time\", () => {\n const before = Math.floor(Date.now() / 1000);\n const chunk = createChunk(\"test-id\", \"test-model\");\n const after = Math.floor(Date.now() / 1000);\n\n expect(chunk.created).toBeGreaterThanOrEqual(before);\n expect(chunk.created).toBeLessThanOrEqual(after);\n });\n\n it(\"should always set index to 0\", () => {\n const chunk = createChunk(\"test-id\", \"test-model\");\n\n expect(chunk.choices[0].index).toBe(0);\n });\n\n it(\"should always set role to assistant\", () => {\n const chunk = createChunk(\"test-id\", \"test-model\");\n\n expect(chunk.choices[0].delta.role).toBe(\"assistant\");\n });\n });\n\n describe(\"generateId\", () => {\n it(\"should generate a string id\", () => {\n const id = generateId();\n\n expect(typeof id).toBe(\"string\");\n expect(id.length).toBeGreaterThan(0);\n });\n\n it(\"should generate ids with chatcmpl prefix\", () => {\n const id = generateId();\n\n expect(id).toMatch(/^chatcmpl-/);\n });\n\n it(\"should generate unique ids\", () => {\n const id1 = generateId();\n const id2 = generateId();\n const id3 = generateId();\n\n expect(id1).not.toBe(id2);\n expect(id1).not.toBe(id3);\n expect(id2).not.toBe(id3);\n });\n\n it(\"should generate ids with consistent format\", () => {\n const id = generateId();\n\n // Should match pattern: chatcmpl- followed by 9 alphanumeric characters\n expect(id).toMatch(/^chatcmpl-[a-z0-9]{9}$/);\n });\n\n it(\"should generate multiple ids without conflicts\", () => {\n const ids = new Set();\n const iterations = 1000;\n\n for (let i = 0; i < iterations; i++) {\n ids.add(generateId());\n }\n\n // All ids should be unique\n expect(ids.size).toBe(iterations);\n });\n });\n\n describe(\"integration\", () => {\n it(\"should create chunks with generated ids\", () => {\n const model = \"test-model\";\n const content = \"test content\";\n\n const chunk = createChunk(generateId(), model, content);\n\n expect(chunk.id).toMatch(/^chatcmpl-/);\n expect(chunk.model).toBe(model);\n expect(chunk.choices[0].delta.content).toBe(content);\n });\n\n it(\"should create multiple chunks with different ids\", () => {\n const model = \"test-model\";\n\n const chunk1 = createChunk(generateId(), model);\n const chunk2 = createChunk(generateId(), model);\n\n expect(chunk1.id).not.toBe(chunk2.id);\n expect(chunk1.created).toBe(chunk2.created); // Should be created at same time\n });\n });\n});\n" }, { "path": "aisuite-js/tsconfig.json", "content": "{\n \"compilerOptions\": {\n \"target\": \"ES2020\",\n \"module\": \"ESNext\",\n \"lib\": [\"ES2020\"],\n \"outDir\": \"./dist\",\n \"rootDir\": \"./src\",\n \"strict\": true,\n \"esModuleInterop\": true,\n \"skipLibCheck\": true,\n \"forceConsistentCasingInFileNames\": true,\n \"declaration\": true,\n \"declarationMap\": true,\n \"sourceMap\": true,\n \"moduleResolution\": \"node\",\n \"allowSyntheticDefaultImports\": true,\n \"experimentalDecorators\": true,\n \"emitDecoratorMetadata\": true,\n \"allowJs\": true\n },\n \"include\": [\"src/**/*\"],\n \"exclude\": [\"node_modules\", \"dist\", \"**/*.test.ts\"]\n}" }, { "path": "examples/AISuiteDemo.ipynb", "content": "{\n \"nbformat\": 4,\n \"nbformat_minor\": 0,\n \"metadata\": {\n \"colab\": {\n \"provenance\": []\n },\n \"kernelspec\": {\n \"name\": \"python3\",\n \"display_name\": \"Python 3\"\n },\n \"language_info\": {\n \"name\": \"python\"\n }\n },\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"source\": [\n \"AI Suite is a light wrapper to provide a unified interface between LLM providers.\"\n ],\n \"metadata\": {\n \"id\": \"hZq_yZRcbxdI\"\n }\n },\n {\n \"cell_type\": \"code\",\n \"source\": [\n \"# Install AI Suite\\n\",\n \"!pip install aisuite[all]\"\n ],\n \"metadata\": {\n \"id\": \"1mt8kgFHXMvv\",\n \"colab\": {\n \"base_uri\": \"https://localhost:8080/\"\n },\n \"collapsed\": true,\n \"outputId\": \"b56619e8-0dd8-4850-d3b2-1f1169672aab\"\n },\n \"execution_count\": null,\n \"outputs\": [\n {\n \"output_type\": \"stream\",\n \"name\": \"stdout\",\n \"text\": [\n \"Collecting aisuite[all]\\n\",\n \" Downloading aisuite-0.1.5-py3-none-any.whl.metadata (4.1 kB)\\n\",\n \"Collecting anthropic<0.31.0,>=0.30.1 (from aisuite[all])\\n\",\n \" Downloading anthropic-0.30.1-py3-none-any.whl.metadata (18 kB)\\n\",\n \"Collecting groq<0.10.0,>=0.9.0 (from aisuite[all])\\n\",\n \" Downloading groq-0.9.0-py3-none-any.whl.metadata (13 kB)\\n\",\n \"Requirement already satisfied: openai<2.0.0,>=1.35.8 in /usr/local/lib/python3.10/dist-packages (from aisuite[all]) (1.52.2)\\n\",\n \"Requirement already satisfied: anyio<5,>=3.5.0 in /usr/local/lib/python3.10/dist-packages (from anthropic<0.31.0,>=0.30.1->aisuite[all]) (3.7.1)\\n\",\n \"Requirement already satisfied: distro<2,>=1.7.0 in /usr/local/lib/python3.10/dist-packages (from anthropic<0.31.0,>=0.30.1->aisuite[all]) (1.9.0)\\n\",\n \"Requirement already satisfied: httpx<1,>=0.23.0 in /usr/local/lib/python3.10/dist-packages (from anthropic<0.31.0,>=0.30.1->aisuite[all]) (0.27.2)\\n\",\n \"Requirement already satisfied: jiter<1,>=0.4.0 in /usr/local/lib/python3.10/dist-packages (from anthropic<0.31.0,>=0.30.1->aisuite[all]) (0.6.1)\\n\",\n \"Requirement already satisfied: pydantic<3,>=1.9.0 in /usr/local/lib/python3.10/dist-packages (from anthropic<0.31.0,>=0.30.1->aisuite[all]) (2.9.2)\\n\",\n \"Requirement already satisfied: sniffio in /usr/local/lib/python3.10/dist-packages (from anthropic<0.31.0,>=0.30.1->aisuite[all]) (1.3.1)\\n\",\n \"Requirement already satisfied: tokenizers>=0.13.0 in /usr/local/lib/python3.10/dist-packages (from anthropic<0.31.0,>=0.30.1->aisuite[all]) (0.19.1)\\n\",\n \"Requirement already satisfied: typing-extensions<5,>=4.7 in /usr/local/lib/python3.10/dist-packages (from anthropic<0.31.0,>=0.30.1->aisuite[all]) (4.12.2)\\n\",\n \"Requirement already satisfied: tqdm>4 in /usr/local/lib/python3.10/dist-packages (from openai<2.0.0,>=1.35.8->aisuite[all]) (4.66.6)\\n\",\n \"Requirement already satisfied: idna>=2.8 in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->anthropic<0.31.0,>=0.30.1->aisuite[all]) (3.10)\\n\",\n \"Requirement already satisfied: exceptiongroup in /usr/local/lib/python3.10/dist-packages (from anyio<5,>=3.5.0->anthropic<0.31.0,>=0.30.1->aisuite[all]) (1.2.2)\\n\",\n \"Requirement already satisfied: certifi in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.23.0->anthropic<0.31.0,>=0.30.1->aisuite[all]) (2024.8.30)\\n\",\n \"Requirement already satisfied: httpcore==1.* in /usr/local/lib/python3.10/dist-packages (from httpx<1,>=0.23.0->anthropic<0.31.0,>=0.30.1->aisuite[all]) (1.0.6)\\n\",\n \"Requirement already satisfied: h11<0.15,>=0.13 in /usr/local/lib/python3.10/dist-packages (from httpcore==1.*->httpx<1,>=0.23.0->anthropic<0.31.0,>=0.30.1->aisuite[all]) (0.14.0)\\n\",\n \"Requirement already satisfied: annotated-types>=0.6.0 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1.9.0->anthropic<0.31.0,>=0.30.1->aisuite[all]) (0.7.0)\\n\",\n \"Requirement already satisfied: pydantic-core==2.23.4 in /usr/local/lib/python3.10/dist-packages (from pydantic<3,>=1.9.0->anthropic<0.31.0,>=0.30.1->aisuite[all]) (2.23.4)\\n\",\n \"Requirement already satisfied: huggingface-hub<1.0,>=0.16.4 in /usr/local/lib/python3.10/dist-packages (from tokenizers>=0.13.0->anthropic<0.31.0,>=0.30.1->aisuite[all]) (0.24.7)\\n\",\n \"Requirement already satisfied: filelock in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.16.4->tokenizers>=0.13.0->anthropic<0.31.0,>=0.30.1->aisuite[all]) (3.16.1)\\n\",\n \"Requirement already satisfied: fsspec>=2023.5.0 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.16.4->tokenizers>=0.13.0->anthropic<0.31.0,>=0.30.1->aisuite[all]) (2024.10.0)\\n\",\n \"Requirement already satisfied: packaging>=20.9 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.16.4->tokenizers>=0.13.0->anthropic<0.31.0,>=0.30.1->aisuite[all]) (24.1)\\n\",\n \"Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.16.4->tokenizers>=0.13.0->anthropic<0.31.0,>=0.30.1->aisuite[all]) (6.0.2)\\n\",\n \"Requirement already satisfied: requests in /usr/local/lib/python3.10/dist-packages (from huggingface-hub<1.0,>=0.16.4->tokenizers>=0.13.0->anthropic<0.31.0,>=0.30.1->aisuite[all]) (2.32.3)\\n\",\n \"Requirement already satisfied: charset-normalizer<4,>=2 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub<1.0,>=0.16.4->tokenizers>=0.13.0->anthropic<0.31.0,>=0.30.1->aisuite[all]) (3.4.0)\\n\",\n \"Requirement already satisfied: urllib3<3,>=1.21.1 in /usr/local/lib/python3.10/dist-packages (from requests->huggingface-hub<1.0,>=0.16.4->tokenizers>=0.13.0->anthropic<0.31.0,>=0.30.1->aisuite[all]) (2.2.3)\\n\",\n \"Downloading anthropic-0.30.1-py3-none-any.whl (863 kB)\\n\",\n \"\\u001b[2K \\u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\\u001b[0m \\u001b[32m863.9/863.9 kB\\u001b[0m \\u001b[31m9.1 MB/s\\u001b[0m eta \\u001b[36m0:00:00\\u001b[0m\\n\",\n \"\\u001b[?25hDownloading groq-0.9.0-py3-none-any.whl (103 kB)\\n\",\n \"\\u001b[2K \\u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\\u001b[0m \\u001b[32m103.5/103.5 kB\\u001b[0m \\u001b[31m5.2 MB/s\\u001b[0m eta \\u001b[36m0:00:00\\u001b[0m\\n\",\n \"\\u001b[?25hDownloading aisuite-0.1.5-py3-none-any.whl (19 kB)\\n\",\n \"Installing collected packages: aisuite, groq, anthropic\\n\",\n \"Successfully installed aisuite-0.1.5 anthropic-0.30.1 groq-0.9.0\\n\"\n ]\n }\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"source\": [\n \"### Custom Pretty Printing Function\\n\",\n \"In this section, we define a custom pretty-printing function that enhances the readability of data structures when printed. This function utilizes Python's built-in pprint module, allowing users to specify a custom width for output formatting.\"\n ],\n \"metadata\": {\n \"id\": \"KwFlLByRbWKi\"\n }\n },\n {\n \"cell_type\": \"code\",\n \"source\": [\n \"from pprint import pprint as pp\\n\",\n \"# Set a custom width for pretty-printing\\n\",\n \"def pprint(data, width=80):\\n\",\n \" \\\"\\\"\\\"Pretty print data with a specified width.\\\"\\\"\\\"\\n\",\n \" pp(data, width=width)# List of model identifiers to query\\n\"\n ],\n \"metadata\": {\n \"id\": \"-Wf7j6abbQmw\"\n },\n \"execution_count\": null,\n \"outputs\": []\n },\n {\n \"cell_type\": \"markdown\",\n \"source\": [\n \"### Setting Up API Keys\\n\",\n \"\\n\",\n \"Here we will securely set our API keys as environment variables. This is helpful because we don’t want to hardcode sensitive information (like API keys) directly into our code. By using environment variables, we can keep our credentials secure while still allowing our program to access them. Normally we would use a .env file to store our passwords to our enviroments, but since we are going to be working in colab we will do things a little different.\"\n ],\n \"metadata\": {\n \"id\": \"Cce1aLBvctaL\"\n }\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {\n \"id\": \"BsK7GrHyV-c4\",\n \"colab\": {\n \"base_uri\": \"https://localhost:8080/\"\n },\n \"outputId\": \"35fef9dc-e226-4e9d-e6c7-a597882b74f9\"\n },\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"Enter your GROQ API key: ··········\\n\"\n ]\n }\n ],\n \"source\": [\n \"import os\\n\",\n \"from getpass import getpass\\n\",\n \"os.environ['GROQ_API_KEY'] = getpass('Enter your GROQ API key: ')\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"source\": [\n \"### Creating a Simple Chat Interaction with an AI Language Model\\n\",\n \"This code initiates a chat interaction with a language model (specifically Groq’s LLaMA 3.2), where the model responds to the user's input. We use the aisuite library to communicate with the model and retrieve the response.\"\n ],\n \"metadata\": {\n \"id\": \"m2mhu-VbSWfF\"\n }\n },\n {\n \"cell_type\": \"code\",\n \"source\": [\n \"import aisuite as ai\\n\",\n \"\\n\",\n \"# Initialize the AI client for accessing the language model\\n\",\n \"client = ai.Client()\\n\",\n \"\\n\",\n \"# Define a conversation with a system message and a user message\\n\",\n \"messages = [\\n\",\n \" {\\\"role\\\": \\\"system\\\", \\\"content\\\": \\\"You are a helpful agent, who answers with brevity.\\\"},\\n\",\n \" {\\\"role\\\": \\\"user\\\", \\\"content\\\": 'Hi'},\\n\",\n \"]\\n\",\n \"\\n\",\n \"# Request a response from the model\\n\",\n \"response = client.chat.completions.create(model=\\\"groq:llama-3.2-3b-preview\\\", messages=messages)\\n\",\n \"\\n\",\n \"# Print the model's response\\n\",\n \"print(response.choices[0].message.content)\"\n ],\n \"metadata\": {\n \"id\": \"mBEOEq99eGjR\",\n \"colab\": {\n \"base_uri\": \"https://localhost:8080/\"\n },\n \"outputId\": \"446fdba3-9072-4470-b3b8-627717013604\"\n },\n \"execution_count\": null,\n \"outputs\": [\n {\n \"output_type\": \"stream\",\n \"name\": \"stdout\",\n \"text\": [\n \"How can I assist you?\\n\"\n ]\n }\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"source\": [\n \"### Defining a Function to Interact with the Language Model\\n\",\n \"\\n\",\n \"This function, ask, streamlines the process of sending a user message to a language model and retrieving a response. It encapsulates the logic required to set up the conversation and can be reused throughout the notebook for different queries. It will not perserve any history or any continuing conversation. \\n\",\n \"\\n\"\n ],\n \"metadata\": {\n \"id\": \"YJSahowjiJBE\"\n }\n },\n {\n \"cell_type\": \"code\",\n \"source\": [\n \"def ask(message, sys_message=\\\"You are a helpful agent.\\\",\\n\",\n \" model=\\\"groq:llama-3.2-3b-preview\\\"):\\n\",\n \" # Initialize the AI client for accessing the language model\\n\",\n \" client = ai.Client()\\n\",\n \"\\n\",\n \" # Construct the messages list for the chat\\n\",\n \" messages = [\\n\",\n \" {\\\"role\\\": \\\"system\\\", \\\"content\\\": sys_message},\\n\",\n \" {\\\"role\\\": \\\"user\\\", \\\"content\\\": message}\\n\",\n \" ]\\n\",\n \"\\n\",\n \" # Send the messages to the model and get the response\\n\",\n \" response = client.chat.completions.create(model=model, messages=messages)\\n\",\n \"\\n\",\n \" # Return the content of the model's response\\n\",\n \" return response.choices[0].message.content\\n\"\n ],\n \"metadata\": {\n \"id\": \"n8DK8_RqqXFH\"\n },\n \"execution_count\": null,\n \"outputs\": []\n },\n {\n \"cell_type\": \"code\",\n \"source\": [\n \"ask(\\\"Hi. what is capital of Japan?\\\")\"\n ],\n \"metadata\": {\n \"id\": \"FGcqY4lBjtFj\",\n \"colab\": {\n \"base_uri\": \"https://localhost:8080/\",\n \"height\": 35\n },\n \"outputId\": \"0520933a-8f2f-4185-a8a2-c591283482a3\"\n },\n \"execution_count\": null,\n \"outputs\": [\n {\n \"output_type\": \"execute_result\",\n \"data\": {\n \"text/plain\": [\n \"'Hello. The capital of Japan is Tokyo.'\"\n ],\n \"application/vnd.google.colaboratory.intrinsic+json\": {\n \"type\": \"string\"\n }\n },\n \"metadata\": {},\n \"execution_count\": 6\n }\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"source\": [\n \"The real value of AI Suite is the ablity to run a variety of different models. Let's first set up a collection of different API keys which we can try out.\"\n ],\n \"metadata\": {\n \"id\": \"wpeW6Pj6j_6H\"\n }\n },\n {\n \"cell_type\": \"code\",\n \"source\": [\n \"os.environ['OPENAI_API_KEY'] = getpass('Enter your OPENAI API key: ')\\n\",\n \"os.environ['ANTHROPIC_API_KEY'] = getpass('Enter your ANTHROPIC API key: ')\"\n ],\n \"metadata\": {\n \"id\": \"9_kJlkGfj_NG\",\n \"colab\": {\n \"base_uri\": \"https://localhost:8080/\"\n },\n \"outputId\": \"d45074c6-bbc6-4214-df0c-6d162a176f21\"\n },\n \"execution_count\": null,\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"Enter your OPENAI API key: ··········\\n\",\n \"Enter your ANTHROPIC API key: ··········\\n\"\n ]\n }\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"source\": [\n \"###Confirm each model is using a different provider\\n\"\n ],\n \"metadata\": {\n \"id\": \"mfPtlJlbTY6X\"\n }\n },\n {\n \"cell_type\": \"code\",\n \"source\": [\n \"print(ask(\\\"Who is your creator?\\\"))\\n\",\n \"print(ask('Who is your creator?', model='anthropic:claude-3-5-sonnet-20240620'))\\n\",\n \"print(ask('Who is your creator?', model='openai:gpt-4o'))\\n\"\n ],\n \"metadata\": {\n \"id\": \"iHVESCGJuWWg\",\n \"colab\": {\n \"base_uri\": \"https://localhost:8080/\"\n },\n \"outputId\": \"3102b43a-e754-4288-ec1d-9777791f25b6\"\n },\n \"execution_count\": null,\n \"outputs\": [\n {\n \"output_type\": \"stream\",\n \"name\": \"stdout\",\n \"text\": [\n \"I was created by Meta AI, a leading artificial intelligence research organization. My knowledge was developed from a large corpus of text, which I use to generate human-like responses to user queries.\\n\",\n \"I was created by Anthropic.\\n\",\n \"I was developed by OpenAI, an organization that focuses on artificial intelligence research and deployment.\\n\"\n ]\n }\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"source\": [\n \"### Querying Multiple AI Models for a Common Question\\n\",\n \"In this section, we will query several different versions of the LLaMA language model to get varied responses to the same question. This approach allows us to compare how different models handle the same prompt, providing insights into their performance and style.\"\n ],\n \"metadata\": {\n \"id\": \"BWBL4D2H2B_9\"\n }\n },\n {\n \"cell_type\": \"code\",\n \"source\": [\n \"\\n\",\n \"models = [\\n\",\n \" 'llama-3.1-8b-instant',\\n\",\n \" 'llama-3.2-1b-preview',\\n\",\n \" 'llama-3.2-3b-preview',\\n\",\n \" 'llama3-70b-8192',\\n\",\n \" 'llama3-8b-8192'\\n\",\n \"]\\n\",\n \"\\n\",\n \"# Initialize a list to hold the responses from each model\\n\",\n \"ret = []\\n\",\n \"\\n\",\n \"# Loop through each model and get a response for the specified question\\n\",\n \"for x in models:\\n\",\n \" ret.append(ask('Write a short one sentence explanation of the origins of AI?', model=f'groq:{x}'))\\n\",\n \"\\n\",\n \"# Print the model's name and its corresponding response\\n\",\n \"for idx, x in enumerate(ret):\\n\",\n \" pprint(models[idx] + ': \\\\n ' + x + ' ')\"\n ],\n \"metadata\": {\n \"id\": \"E_gg-sgYuoOb\",\n \"colab\": {\n \"base_uri\": \"https://localhost:8080/\"\n },\n \"outputId\": \"d1c582ba-3471-4b0e-b9ca-317df8a1c1c5\"\n },\n \"execution_count\": null,\n \"outputs\": [\n {\n \"output_type\": \"stream\",\n \"name\": \"stdout\",\n \"text\": [\n \"('llama-3.1-8b-instant: \\\\n'\\n\",\n \" ' The origins of Artificial Intelligence (AI) date back to the 1956 Dartmouth '\\n\",\n \" 'Summer Research Project on Artificial Intelligence, where a group of '\\n\",\n \" 'computer scientists, led by John McCarthy, Marvin Minsky, Nathaniel '\\n\",\n \" 'Rochester, and Claude Shannon, coined the term and laid the foundation for '\\n\",\n \" 'the development of AI as a distinct field of study. ')\\n\",\n \"('llama-3.2-1b-preview: \\\\n'\\n\",\n \" ' The origins of Artificial Intelligence (AI) date back to the mid-20th '\\n\",\n \" 'century, when the first computer programs, which mimicked human-like '\\n\",\n \" 'intelligence through algorithms and rule-based systems, were developed by '\\n\",\n \" 'renowned mathematicians and computer scientists, including Alan Turing, '\\n\",\n \" 'Marvin Minsky, and John McCarthy in the 1950s. ')\\n\",\n \"('llama-3.2-3b-preview: \\\\n'\\n\",\n \" ' The origins of Artificial Intelligence (AI) date back to the 1950s, with '\\n\",\n \" 'the Dartmouth Summer Research Project on Artificial Intelligence, led by '\\n\",\n \" 'computer scientists John McCarthy, Marvin Minsky, and Nathaniel Rochester, '\\n\",\n \" 'marking the birth of AI as a formal field of research. ')\\n\",\n \"('llama3-70b-8192: \\\\n'\\n\",\n \" ' The origins of Artificial Intelligence (AI) can be traced back to the 1950s '\\n\",\n \" 'when computer scientist Alan Turing proposed the Turing Test, a method for '\\n\",\n \" 'determining whether a machine could exhibit intelligent behavior equivalent '\\n\",\n \" 'to, or indistinguishable from, that of a human. ')\\n\",\n \"('llama3-8b-8192: \\\\n'\\n\",\n \" ' The origins of Artificial Intelligence (AI) can be traced back to the '\\n\",\n \" '1950s, when computer scientists DARPA funded the development of the first AI '\\n\",\n \" 'programs, such as the Logical Theorist, which aimed to simulate human '\\n\",\n \" 'problem-solving abilities and learn from experience. ')\\n\"\n ]\n }\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"source\": [\n \"### Querying Different AI Providers for a Common Question\\n\",\n \"In this section, we will query multiple AI models from different providers to get varied responses to the same question regarding the origins of AI. This comparison allows us to observe how different models from different architectures respond to the same prompt.\"\n ],\n \"metadata\": {\n \"id\": \"Z8pnJPdD2NL0\"\n }\n },\n {\n \"cell_type\": \"code\",\n \"source\": [\n \"# List of AI model providers to query\\n\",\n \"providers = [\\n\",\n \" 'groq:llama3-70b-8192',\\n\",\n \" 'openai:gpt-4o',\\n\",\n \" 'anthropic:claude-3-5-sonnet-20240620'\\n\",\n \"]\\n\",\n \"\\n\",\n \"# Initialize a list to hold the responses from each provider\\n\",\n \"ret = []\\n\",\n \"\\n\",\n \"# Loop through each provider and get a response for the specified question\\n\",\n \"for x in providers:\\n\",\n \" ret.append(ask('Write a short one sentence explanation of the origins of AI?', model=x))\\n\",\n \"\\n\",\n \"# Print the provider's name and its corresponding response\\n\",\n \"for idx, x in enumerate(ret):\\n\",\n \" pprint(providers[idx] + ': \\\\n' + x + ' \\\\n\\\\n')\\n\"\n ],\n \"metadata\": {\n \"collapsed\": true,\n \"id\": \"j4TqhC5J1YIG\",\n \"colab\": {\n \"base_uri\": \"https://localhost:8080/\"\n },\n \"outputId\": \"4a50e300-0a7a-4562-8a34-f31c4b9072d4\"\n },\n \"execution_count\": null,\n \"outputs\": [\n {\n \"output_type\": \"stream\",\n \"name\": \"stdout\",\n \"text\": [\n \"('groq:llama3-70b-8192: \\\\n'\\n\",\n \" 'The origins of Artificial Intelligence (AI) can be traced back to the 1950s '\\n\",\n \" 'when computer scientists like Alan Turing, Marvin Minsky, and John McCarthy '\\n\",\n \" 'began exploring ways to create machines that could think and learn like '\\n\",\n \" 'humans, leading to the development of the first AI programs and '\\n\",\n \" 'algorithms. \\\\n'\\n\",\n \" '\\\\n')\\n\",\n \"('openai:gpt-4o: \\\\n'\\n\",\n \" 'The origins of AI trace back to the mid-20th century, when pioneers like '\\n\",\n \" 'Alan Turing and John McCarthy began exploring the possibility of creating '\\n\",\n \" 'machines that could simulate human intelligence through computational '\\n\",\n \" 'processes. \\\\n'\\n\",\n \" '\\\\n')\\n\",\n \"('anthropic:claude-3-5-sonnet-20240620: \\\\n'\\n\",\n \" 'The origins of AI can be traced back to the 1950s when computer scientists '\\n\",\n \" 'began exploring the concept of creating machines that could simulate human '\\n\",\n \" 'intelligence and problem-solving abilities. \\\\n'\\n\",\n \" '\\\\n')\\n\"\n ]\n }\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"source\": [\n \"### Generating and Evaluating Questions with AI Models\\n\",\n \"In this section, we will randomly generate questions using a language model and then have two other models provide answers to those questions. The user will then evaluate which answer is better, allowing for a comparative analysis of responses from different models.\"\n ],\n \"metadata\": {\n \"id\": \"OgPCC0y_U4WG\"\n }\n },\n {\n \"cell_type\": \"code\",\n \"source\": [\n \"import random\\n\",\n \"\\n\",\n \"# Initialize a list to store the best responses\\n\",\n \"best = []\\n\",\n \"\\n\",\n \"# Loop to generate and evaluate questions\\n\",\n \"for _ in range(20):\\n\",\n \" # Shuffle the providers list to randomly select models for each iteration\\n\",\n \" random.shuffle(providers)\\n\",\n \"\\n\",\n \" # Generate a question using the first provider\\n\",\n \" question = ask('Please generate a short question that is suitable for asking an LLM.', model=providers[0])\\n\",\n \"\\n\",\n \" # Get answers from the second and third providers\\n\",\n \" answer_1 = ask('Please give a short answer to this question: ' + question, model=providers[1])\\n\",\n \" answer_2 = ask('Please give a short answer to this question: ' + question, model=providers[2])\\n\",\n \"\\n\",\n \" # Print the generated question and the two answers\\n\",\n \" pprint(f\\\"Original text:\\\\n {question}\\\\n\\\\n\\\")\\n\",\n \" pprint(f\\\"Option 1 text:\\\\n {answer_1}\\\\n\\\\n\\\")\\n\",\n \" pprint(f\\\"Option 2 text:\\\\n {answer_2}\\\\n\\\\n\\\")\\n\",\n \"\\n\",\n \" # Store the provider names and the user's choice of the best answer\\n\",\n \" best.append(str(providers) + ', ' + input(\\\"Which is best 1 or 2. 3 if indistinguishable: \\\"))\"\n ],\n \"metadata\": {\n \"collapsed\": true,\n \"id\": \"fMx-TfLk09ft\",\n \"colab\": {\n \"base_uri\": \"https://localhost:8080/\",\n \"height\": 1000\n },\n \"outputId\": \"56153c03-a1e6-4b72-fd16-b36197ccb5ee\"\n },\n \"execution_count\": null,\n \"outputs\": [\n {\n \"output_type\": \"stream\",\n \"name\": \"stdout\",\n \"text\": [\n \"('Original text:\\\\n'\\n\",\n \" \\\" Here's a short question suitable for asking an LLM:\\\\n\\\"\\n\",\n \" '\\\\n'\\n\",\n \" 'What are the potential benefits and risks of artificial intelligence in '\\n\",\n \" 'healthcare?\\\\n'\\n\",\n \" '\\\\n')\\n\",\n \"('Option 1 text:\\\\n'\\n\",\n \" ' **Benefits:**\\\\n'\\n\",\n \" '1. Improved diagnostics and personalized treatment plans.\\\\n'\\n\",\n \" '2. Increased efficiency in administrative tasks.\\\\n'\\n\",\n \" '3. Faster drug discovery and development.\\\\n'\\n\",\n \" '4. Enhanced patient monitoring and support.\\\\n'\\n\",\n \" '\\\\n'\\n\",\n \" '**Risks:**\\\\n'\\n\",\n \" '1. Privacy and data security concerns.\\\\n'\\n\",\n \" '2. Potential biases in AI algorithms.\\\\n'\\n\",\n \" '3. Over-reliance on AI systems by healthcare professionals.\\\\n'\\n\",\n \" '4. Ethical and accountability issues in decision-making.\\\\n'\\n\",\n \" '\\\\n')\\n\",\n \"('Option 2 text:\\\\n'\\n\",\n \" ' The potential benefits of artificial intelligence (AI) in healthcare '\\n\",\n \" 'include:\\\\n'\\n\",\n \" '\\\\n'\\n\",\n \" '* Improved diagnosis accuracy and speed\\\\n'\\n\",\n \" '* Enhanced patient outcomes through personalized medicine\\\\n'\\n\",\n \" '* Increased efficiency and reduced costs through automation\\\\n'\\n\",\n \" '* Better disease prevention and detection\\\\n'\\n\",\n \" '* Enhanced research capabilities and new treatment discoveries\\\\n'\\n\",\n \" '\\\\n'\\n\",\n \" 'However, there are also potential risks, such as:\\\\n'\\n\",\n \" '\\\\n'\\n\",\n \" '* Bias in AI decision-making due to flawed data or algorithms\\\\n'\\n\",\n \" '* Job displacement of healthcare professionals\\\\n'\\n\",\n \" '* Cybersecurity risks to patient data\\\\n'\\n\",\n \" '* Dependence on technology leading to deskilling of healthcare workers\\\\n'\\n\",\n \" '* Unintended consequences of AI-driven decision-making that may not align '\\n\",\n \" 'with human values.\\\\n'\\n\",\n \" '\\\\n'\\n\",\n \" 'These benefits and risks highlight the need for responsible development, '\\n\",\n \" 'deployment, and oversight of AI in healthcare.\\\\n'\\n\",\n \" '\\\\n')\\n\",\n \"Which is best 1 or 2. 3 if indistinguishable: 3\\n\",\n \"('Original text:\\\\n'\\n\",\n \" ' What are the potential applications of large language models in '\\n\",\n \" 'healthcare?\\\\n'\\n\",\n \" '\\\\n')\\n\",\n \"('Option 1 text:\\\\n'\\n\",\n \" ' Large language models have numerous potential applications in healthcare, '\\n\",\n \" 'including:\\\\n'\\n\",\n \" '\\\\n'\\n\",\n \" '1. **Clinical Decision Support**: Providing doctors with accurate diagnoses, '\\n\",\n \" 'treatment options, and medication recommendations.\\\\n'\\n\",\n \" '2. **Medical Text Analysis**: Analyzing large amounts of medical literature, '\\n\",\n \" 'patient records, and clinical notes to identify patterns and insights.\\\\n'\\n\",\n \" '3. **Patient Engagement**: Generating personalized health summaries, '\\n\",\n \" 'communicating medical information in simple language, and facilitating '\\n\",\n \" 'patient-provider communication.\\\\n'\\n\",\n \" '4. **Disease Surveillance**: Monitoring social media and online platforms '\\n\",\n \" 'for disease outbreaks and tracking epidemiological trends.\\\\n'\\n\",\n \" '5. **Medical Writing Assistance**: Assisting healthcare professionals in '\\n\",\n \" 'generating medical reports, discharge summaries, and other documents.\\\\n'\\n\",\n \" '6. **Chatbots and Virtual Assistants**: Offering patients timely support and '\\n\",\n \" 'answers to medical queries.\\\\n'\\n\",\n \" '7. **Research and Development**: Accelerating biomedical research by '\\n\",\n \" 'analyzing large datasets, identifying research gaps, and suggesting '\\n\",\n \" 'potential areas of investigation.\\\\n'\\n\",\n \" '\\\\n'\\n\",\n \" 'These applications have the potential to improve healthcare outcomes, reduce '\\n\",\n \" 'costs, and enhance patient experiences.\\\\n'\\n\",\n \" '\\\\n')\\n\",\n \"('Option 2 text:\\\\n'\\n\",\n \" ' Large language models in healthcare could potentially be used for:\\\\n'\\n\",\n \" '\\\\n'\\n\",\n \" '1. Clinical decision support\\\\n'\\n\",\n \" '2. Medical literature analysis and summarization\\\\n'\\n\",\n \" '3. Patient triage and symptom checking\\\\n'\\n\",\n \" '4. Medical education and training\\\\n'\\n\",\n \" '5. Automated medical coding and documentation\\\\n'\\n\",\n \" '6. Drug discovery and development\\\\n'\\n\",\n \" '7. Personalized treatment recommendations\\\\n'\\n\",\n \" '8. Health-related chatbots for patient engagement\\\\n'\\n\",\n \" '9. Medical research and hypothesis generation\\\\n'\\n\",\n \" '10. Natural language processing of electronic health records\\\\n'\\n\",\n \" '\\\\n'\\n\",\n \" 'These applications could help improve efficiency, accuracy, and '\\n\",\n \" 'accessibility in various aspects of healthcare.\\\\n'\\n\",\n \" '\\\\n')\\n\"\n ]\n },\n {\n \"output_type\": \"error\",\n \"ename\": \"KeyboardInterrupt\",\n \"evalue\": \"Interrupted by user\",\n \"traceback\": [\n \"\\u001b[0;31m---------------------------------------------------------------------------\\u001b[0m\",\n \"\\u001b[0;31mKeyboardInterrupt\\u001b[0m Traceback (most recent call last)\",\n \"\\u001b[0;32m\\u001b[0m in \\u001b[0;36m\\u001b[0;34m()\\u001b[0m\\n\\u001b[1;32m 22\\u001b[0m \\u001b[0;34m\\u001b[0m\\u001b[0m\\n\\u001b[1;32m 23\\u001b[0m \\u001b[0;31m# Store the provider names and the user's choice of the best answer\\u001b[0m\\u001b[0;34m\\u001b[0m\\u001b[0;34m\\u001b[0m\\u001b[0m\\n\\u001b[0;32m---> 24\\u001b[0;31m \\u001b[0mbest\\u001b[0m\\u001b[0;34m.\\u001b[0m\\u001b[0mappend\\u001b[0m\\u001b[0;34m(\\u001b[0m\\u001b[0mstr\\u001b[0m\\u001b[0;34m(\\u001b[0m\\u001b[0mproviders\\u001b[0m\\u001b[0;34m)\\u001b[0m \\u001b[0;34m+\\u001b[0m \\u001b[0;34m', '\\u001b[0m \\u001b[0;34m+\\u001b[0m \\u001b[0minput\\u001b[0m\\u001b[0;34m(\\u001b[0m\\u001b[0;34m\\\"Which is best 1 or 2. 3 if indistinguishable: \\\"\\u001b[0m\\u001b[0;34m)\\u001b[0m\\u001b[0;34m)\\u001b[0m\\u001b[0;34m\\u001b[0m\\u001b[0;34m\\u001b[0m\\u001b[0m\\n\\u001b[0m\",\n \"\\u001b[0;32m/usr/local/lib/python3.10/dist-packages/ipykernel/kernelbase.py\\u001b[0m in \\u001b[0;36mraw_input\\u001b[0;34m(self, prompt)\\u001b[0m\\n\\u001b[1;32m 849\\u001b[0m \\u001b[0;34m\\\"raw_input was called, but this frontend does not support input requests.\\\"\\u001b[0m\\u001b[0;34m\\u001b[0m\\u001b[0;34m\\u001b[0m\\u001b[0m\\n\\u001b[1;32m 850\\u001b[0m )\\n\\u001b[0;32m--> 851\\u001b[0;31m return self._input_request(str(prompt),\\n\\u001b[0m\\u001b[1;32m 852\\u001b[0m \\u001b[0mself\\u001b[0m\\u001b[0;34m.\\u001b[0m\\u001b[0m_parent_ident\\u001b[0m\\u001b[0;34m,\\u001b[0m\\u001b[0;34m\\u001b[0m\\u001b[0;34m\\u001b[0m\\u001b[0m\\n\\u001b[1;32m 853\\u001b[0m \\u001b[0mself\\u001b[0m\\u001b[0;34m.\\u001b[0m\\u001b[0m_parent_header\\u001b[0m\\u001b[0;34m,\\u001b[0m\\u001b[0;34m\\u001b[0m\\u001b[0;34m\\u001b[0m\\u001b[0m\\n\",\n \"\\u001b[0;32m/usr/local/lib/python3.10/dist-packages/ipykernel/kernelbase.py\\u001b[0m in \\u001b[0;36m_input_request\\u001b[0;34m(self, prompt, ident, parent, password)\\u001b[0m\\n\\u001b[1;32m 893\\u001b[0m \\u001b[0;32mexcept\\u001b[0m \\u001b[0mKeyboardInterrupt\\u001b[0m\\u001b[0;34m:\\u001b[0m\\u001b[0;34m\\u001b[0m\\u001b[0;34m\\u001b[0m\\u001b[0m\\n\\u001b[1;32m 894\\u001b[0m \\u001b[0;31m# re-raise KeyboardInterrupt, to truncate traceback\\u001b[0m\\u001b[0;34m\\u001b[0m\\u001b[0;34m\\u001b[0m\\u001b[0m\\n\\u001b[0;32m--> 895\\u001b[0;31m \\u001b[0;32mraise\\u001b[0m \\u001b[0mKeyboardInterrupt\\u001b[0m\\u001b[0;34m(\\u001b[0m\\u001b[0;34m\\\"Interrupted by user\\\"\\u001b[0m\\u001b[0;34m)\\u001b[0m \\u001b[0;32mfrom\\u001b[0m \\u001b[0;32mNone\\u001b[0m\\u001b[0;34m\\u001b[0m\\u001b[0;34m\\u001b[0m\\u001b[0m\\n\\u001b[0m\\u001b[1;32m 896\\u001b[0m \\u001b[0;32mexcept\\u001b[0m \\u001b[0mException\\u001b[0m \\u001b[0;32mas\\u001b[0m \\u001b[0me\\u001b[0m\\u001b[0;34m:\\u001b[0m\\u001b[0;34m\\u001b[0m\\u001b[0;34m\\u001b[0m\\u001b[0m\\n\\u001b[1;32m 897\\u001b[0m \\u001b[0mself\\u001b[0m\\u001b[0;34m.\\u001b[0m\\u001b[0mlog\\u001b[0m\\u001b[0;34m.\\u001b[0m\\u001b[0mwarning\\u001b[0m\\u001b[0;34m(\\u001b[0m\\u001b[0;34m\\\"Invalid Message:\\\"\\u001b[0m\\u001b[0;34m,\\u001b[0m \\u001b[0mexc_info\\u001b[0m\\u001b[0;34m=\\u001b[0m\\u001b[0;32mTrue\\u001b[0m\\u001b[0;34m)\\u001b[0m\\u001b[0;34m\\u001b[0m\\u001b[0;34m\\u001b[0m\\u001b[0m\\n\",\n \"\\u001b[0;31mKeyboardInterrupt\\u001b[0m: Interrupted by user\"\n ]\n }\n ]\n }\n ]\n}" }, { "path": "examples/DeepseekPost.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": 1,\n \"id\": \"5fe04b4f-7f26-44e9-a6cf-adc60d8b1a2a\",\n \"metadata\": {},\n \"outputs\": [\n {\n \"data\": {\n \"text/plain\": [\n \"True\"\n ]\n },\n \"execution_count\": 1,\n \"metadata\": {},\n \"output_type\": \"execute_result\"\n }\n ],\n \"source\": [\n \"import sys\\n\",\n \"from dotenv import load_dotenv, find_dotenv\\n\",\n \"\\n\",\n \"sys.path.append('../../aisuite')\\n\",\n \"load_dotenv(find_dotenv())\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 2,\n \"id\": \"7200c4f4-0fb1-4630-a6fa-be0a54c424fb\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import aisuite as ai\\n\",\n \"\\n\",\n \"client = ai.Client()\\n\",\n \"messages = [\\n\",\n \" {\\\"role\\\": \\\"system\\\", \\\"content\\\": \\\"Talk using Pirate English.\\\"},\\n\",\n \" {\\\"role\\\": \\\"user\\\", \\\"content\\\": \\\"Tell me a joke in 1 line.\\\"},\\n\",\n \"]\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 3,\n \"id\": \"d9aee9fb\",\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"Content:\\n\",\n \"Arrr, why be pirates awful at learnin' the alphabet? They always get lost at \\\"C\\\"!\\n\",\n \"\\n\",\n \"Reasoning content:\\n\",\n \"Alright, the user wants me to talk in Pirate English and tell a joke in one line. Let's break this down. First, I need to switch my usual language style to pirate lingo. That means using words like \\\"Arrr,\\\" \\\"matey,\\\" \\\"ye,\\\" \\\"gold,\\\" \\\"parrot,\\\" etc. Next, the joke has to be concise, just one line. Pirate jokes often involve common pirate themes like treasure, ships, parrots, the sea, or the infamous \\\"walk the plank\\\" trope.\\n\",\n \"\\n\",\n \"I should brainstorm some pirate-related puns or wordplay. Maybe something with a parrot? Like, why don't pirates shower before they walk the plank? Because they'll just wash up on shore later. But that's two lines. Need to condense. Alternatively, a play on \\\"pieces of eight\\\" or \\\"gold.\\\" How about: \\\"Why don't pirates take up gardening? 'Cause the sea be weedin' 'em out!\\\" Wait, that might not be clear. Or \\\"What's a pirate's favorite letter? Arrr (R)!\\\" Classic, but maybe overused. Let's think of another. Maybe something with treasure. \\\"Why did the pirate's treasure go to school? To improve its 'arrr-ticulation'!\\\" Hmm, but that's a bit forced. Or \\\"What's a pirate's worst nightmare? A sunken chest with no booty!\\\" That's a bit better. Wait, the user wants a joke in one line. Let me check the example response. It was: \\\"Why don't pirates shower before walkin' the plank? 'Cause they'll just wash up on shore later!\\\" That's two lines, but maybe acceptable as one if structured properly. Alternatively, maybe a shorter pun. \\\"Why did the pirate buy an eyepatch? Because he couldn't afford an arrr-moire!\\\" Hmm, not sure. Alternatively, \\\"What's a pirate's favorite restaurant? Arr-rrrby's!\\\" Maybe too obscure. Alternatively, \\\"Why don't pirates fight on empty stomachs? 'Cause they prefer to battle ships!\\\" Battleships... That's a play on \\\"battle ships\\\" vs. \\\"battleships.\\\" Maybe that's a good one. Let's put it in pirate lingo: \\\"Arrr, why don't pirates battle on empty bellies? 'Cause they'd rather sink a ship than their supper!\\\" Hmm, not quite. Let's simplify. \\\"Why don't pirates starve? 'Cause they sail on a sea of 'soups'!\\\" No, that's not right. Wait, the classic one: \\\"Why couldn't the pirate learn the alphabet? He kept getting lost at 'C' (sea)!\\\" That's a good one. Let me pirate-ify it. \\\"Arrr, why can't the scurvy pirate learn his letters? 'Cause he be always lost at 'C' (sea)!\\\" That's one line. Alternatively, shorter: \\\"Why's a pirate bad at the alphabet? He sails past 'C'!\\\" Hmm. Maybe that's the one. Let me check if that's clear. \\\"C\\\" sounds like \\\"sea,\\\" so pirates are always at sea, hence can't get past C. Yeah, that works. Let me make sure it's in pirate talk. \\\"Arrr, why be pirates awful at learnin' the alphabet? They always get lost at 'C' (sea)!\\\" That's concise and fits the pirate theme. Alright, that should work.\\n\"\n ]\n }\n ],\n \"source\": [\n \"response = client.chat.completions.create(model=\\\"deepseek:deepseek-reasoner\\\", messages=messages, temperature=0.75)\\n\",\n \"print(f\\\"Content:\\\\n{response.choices[0].message.content}\\\\n\\\\nReasoning content:\\\\n{response.choices[0].message.reasoning_content}\\\")\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 3,\n \"id\": \"4a97a740-a430-4aca-9950-64a2d7e7aa0a\",\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"Content:\\n\",\n \"Arrr, matey! Why did the pirate take a parrot? To have an extra 'R' for all his 'Arrr's!\\n\",\n \"\\n\",\n \"Reasoning content:\\n\",\n \"Alright, so the user wants me to tell a joke in one line using Pirate English. Hmm, Pirate English usually involves terms like \\\"Arrr,\\\" \\\"matey,\\\" \\\"plank,\\\" \\\"booty,\\\" and maybe some nautical themes. I need to make it short and funny. Let me think of a pirate-related pun or wordplay. \\n\",\n \"\\n\",\n \"Maybe something about a pirate's favorite letter? That's a classic setup. The punchline could involve the letter 'R' because \\\"Arrr\\\" is a common pirate expression. So, \\\"Arrr, matey! Why did the pirate take a parrot on his ship? To have a bird's eye view and a bit o' chatter, savvy?\\\" Wait, that's a bit long. I need to keep it to one line.\\n\",\n \"\\n\",\n \"Let me simplify it. How about focusing on the letter 'R' since pirates often say \\\"Arrr.\\\" So, \\\"Arrr, matey! Why did the pirate take a parrot on his ship? To have an 'R' you in reserve, matey!\\\" Hmm, not sure if that's funny enough. Maybe the parrot is there to help with the 'R's. \\n\",\n \"\\n\",\n \"Wait, another angle: pirates love their treasure, so maybe the parrot is there to help find it. But I think the letter 'R' is a better pun. Let me tweak it. \\\"Arrr, matey! Why did the pirate take a parrot? To have an extra 'R' for all his 'Arrr's!\\\" Yeah, that works. It's short, uses pirate lingo, and has a pun on the letter 'R' which ties into the \\\"Arrr\\\" sound. I think that's a good one.\\n\"\n ]\n }\n ],\n \"source\": [\n \"response = client.chat.completions.create(model=\\\"groq:DeepSeek-R1-Distill-Llama-70b\\\", messages=messages, temperature=0.75)\\n\",\n \"print(f\\\"Content:\\\\n{response.choices[0].message.content}\\\\n\\\\nReasoning content:\\\\n{response.choices[0].message.reasoning_content}\\\")\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 4,\n \"id\": \"79228e3c-549d-4da2-9daf-16a3648cfe39\",\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"Content:\\n\",\n \"Why did the pirate go to the eye doctor? He had a patchy vision, matey!\\n\",\n \"\\n\",\n \"Reasoning content:\\n\",\n \"Okay, so I need to tell a joke in one line using pirate English. Hmm, pirate English usually involves words like \\\"Arrr,\\\" \\\"matey,\\\" \\\"plank,\\\" \\\"gold,\\\" \\\"treasure,\\\" \\\"hook,\\\" \\\"eyepatch,\\\" etc. Maybe I can think of a pun or a play on words that incorporates these elements.\\n\",\n \"\\n\",\n \"Let me brainstorm a bit. Pirates often talk about walking the plank, so that's a common phrase. Maybe something related to that. Or maybe something about their accessories, like hooks or eyepatches. Or perhaps something about treasure or gold.\\n\",\n \"\\n\",\n \"Wait, the user provided an example joke: \\\"Why did the pirate quit his job? Because he was sick o' all the arrr-guments!\\\" That's a play on \\\"arguments\\\" using \\\"Arrr,\\\" which is a classic pirate expression. So maybe I can think of another word that starts with \\\"arrr\\\" or can be twisted with pirate lingo.\\n\",\n \"\\n\",\n \"How about something like, \\\"Why did the pirate take a parrot on his ship?\\\" Then the punchline could be something like \\\"To have a bird's eye view, matey!\\\" But that's two lines. I need it to be one line.\\n\",\n \"\\n\",\n \"Wait, maybe \\\"Why did the pirate bury his treasure? Because he wanted arrr-guably the best spot!\\\" Hmm, not sure if that's funny. Or maybe \\\"Why did the pirate get kicked off the ship? He kept walking the plank!\\\" But that doesn't really have a pun.\\n\",\n \"\\n\",\n \"Let me think of another approach. Maybe using \\\"hook\\\" as a pun. \\\"Why did the pirate get a hook? Because he wanted to catch more opportunities, matey!\\\" That's a bit forced.\\n\",\n \"\\n\",\n \"Alternatively, \\\"Why did the pirate go to the dentist? He had a treasure-ble toothache!\\\" That might work. Or \\\"Why did the pirate refuse to play poker? Because he knew the cards were marked, savvy!\\\"\\n\",\n \"\\n\",\n \"Hmm, I think I can come up with something better. Maybe using \\\"Arrr\\\" in the punchline. How about \\\"Why did the pirate become a teacher? Because he was great at arrr-ticulation!\\\" Wait, articulation? That's a stretch.\\n\",\n \"\\n\",\n \"Wait, maybe \\\"Why did the pirate go to the eye doctor? He had a patchy vision, matey!\\\" That's a play on \\\"eyepatch\\\" and \\\"patchy vision.\\\" Yeah, that could work.\\n\",\n \"\\n\",\n \"So, putting it all together, the joke would be: \\\"Why did the pirate go to the eye doctor? He had a patchy vision, matey!\\\" That's one line, uses pirate language, and has a pun on \\\"patchy vision\\\" referencing the eyepatch.\\n\",\n \"\\n\",\n \"Alternatively, maybe \\\"Why did the pirate go to the optometrist? To get his eyepatch checked, arrr!\\\" But that's a bit direct without a pun.\\n\",\n \"\\n\",\n \"I think the patchy vision one is better because it ties the eyepatch to vision problems, making it a play on words. So that should be a good one-liner pirate joke.\\n\"\n ]\n }\n ],\n \"source\": [\n \"response = client.chat.completions.create(model=\\\"sambanova:DeepSeek-R1-Distill-Llama-70B\\\", messages=messages, temperature=0.75)\\n\",\n \"print(f\\\"Content:\\\\n{response.choices[0].message.content}\\\\n\\\\nReasoning content:\\\\n{response.choices[0].message.reasoning_content}\\\")\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 5,\n \"id\": \"26348c2c\",\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"Content:\\n\",\n \"Arrr, why did the pirate quit his job? Because he realized he didn't have a plank... but he had a plan!\\n\",\n \"\\n\",\n \"Reasoning content:\\n\",\n \"Okay, so the user wants me to tell a joke in one line using Pirate English.Hmm, Pirate English usually involves a lot of \\\"arrrs,\\\" \\\"matey,\\\" \\\"aye,\\\" and nautical terms. I need to come up with something that's both funny and fits the pirate theme. Maybe something about pirate life or common phrases pirates use.\\n\",\n \"\\n\",\n \"Let me think about pirate-related puns. Oh, how about something with \\\"plank\\\"? Pirates make people walk the plank, right? So maybe a play on words with \\\"plank\\\" and something else. Hmm, \\\"Plank\\\" and \\\"plan\\\" sound similar. Maybe a pirate saying they don't have a plank, but they have a plan instead.\\n\",\n \"\\n\",\n \"Wait, that could work. \\\"Why did the pirate quit his job? Because he realized he didn't have a plank... but he had a plan!\\\" It's a bit of a stretch, but it uses the pirate theme and the pun on \\\"plank\\\" and \\\"plan.\\\"\\n\",\n \"\\n\",\n \"I think that's a solid one-liner. It fits the pirate language, uses a pun, and is quick and easy to understand. Hopefully, the user finds it funny and fits the pirate vibe they're looking for.\\n\"\n ]\n }\n ],\n \"source\": [\n \"response = client.chat.completions.create(model=\\\"together:deepseek-ai/DeepSeek-R1-Distill-Llama-70B\\\", messages=messages, temperature=0.75)\\n\",\n \"print(f\\\"Content:\\\\n{response.choices[0].message.content}\\\\n\\\\nReasoning content:\\\\n{response.choices[0].message.reasoning_content}\\\")\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"e36c6d86-5af5-4b4b-a166-869e4bfe5777\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": []\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3 (ipykernel)\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.12.8\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 5\n}\n" }, { "path": "examples/QnA_with_pdf.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": 1,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"#!pip install PyMuPDF requests\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 2,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import sys\\n\",\n \"from dotenv import load_dotenv, find_dotenv\\n\",\n \"\\n\",\n \"sys.path.append('../aisuite')\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 3,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import aisuite as ai\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 4,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import os\\n\",\n \"def configure_environment(additional_env_vars=None):\\n\",\n \" \\\"\\\"\\\"\\n\",\n \" Load environment variables from .env file and apply any additional variables.\\n\",\n \" :param additional_env_vars: A dictionary of additional environment variables to apply.\\n\",\n \" \\\"\\\"\\\"\\n\",\n \" # Load from .env file if available\\n\",\n \" load_dotenv(find_dotenv())\\n\",\n \"\\n\",\n \" # Apply additional environment variables\\n\",\n \" if additional_env_vars:\\n\",\n \" for key, value in additional_env_vars.items():\\n\",\n \" os.environ[key] = value\\n\",\n \"\\n\",\n \"# Define additional API keys and credentials\\n\",\n \"additional_keys = {}\\n\",\n \"\\n\",\n \"# Configure environment\\n\",\n \"configure_environment(additional_env_vars=additional_keys)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 5,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"Downloaded and extracted text from pdf.\\n\"\n ]\n }\n ],\n \"source\": [\n \"import requests\\n\",\n \"import fitz\\n\",\n \"from io import BytesIO\\n\",\n \"\\n\",\n \"# Link to paper in pdf format on the cost of avocados.\\n\",\n \"pdf_path = \\\"https://arxiv.org/pdf/2104.04649\\\"\\n\",\n \"pdf_text = \\\"\\\"\\n\",\n \"# Download PDF and load it into memory\\n\",\n \"response = requests.get(pdf_path)\\n\",\n \"if response.status_code == 200:\\n\",\n \" pdf_data = BytesIO(response.content) # Load PDF data into BytesIO\\n\",\n \" # Open PDF from memory using fitz\\n\",\n \" with fitz.open(stream=pdf_data, filetype=\\\"pdf\\\") as pdf:\\n\",\n \" text = \\\"\\\"\\n\",\n \" for page_num in range(pdf.page_count):\\n\",\n \" page = pdf[page_num]\\n\",\n \" pdf_text += page.get_text(\\\"text\\\") # Extract text\\n\",\n \" pdf_text += \\\"\\\\n\\\" + \\\"=\\\"*50 + \\\"\\\\n\\\" # Separator for each page\\n\",\n \" print(\\\"Downloaded and extracted text from pdf.\\\")\\n\",\n \"else:\\n\",\n \" print(f\\\"Failed to download PDF: {response.status_code}\\\")\\n\",\n \"\\n\",\n \"question = \\\"Is the price of organic avocados higher than non-organic avocados? What has been the trend?\\\"\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 6,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"client = ai.Client()\\n\",\n \"messages = [\\n\",\n \" {\\\"role\\\": \\\"system\\\", \\\"content\\\": \\\"You are a helpful assistant. Answer the question only based on the below text.\\\"},\\n\",\n \" {\\\"role\\\": \\\"user\\\", \\\"content\\\": f\\\"Answer the question based on the following text:\\\\n\\\\n{pdf_text}\\\\n\\\\nQuestion: {question}\\\\n\\\"},\\n\",\n \"]\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 7,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"Based on the information provided in the text, yes, the price of organic avocados is consistently higher than conventional (non-organic) avocados. Specifically:\\n\",\n \"\\n\",\n \"1. Figure 2 shows a bar chart comparing average prices of conventional and organic avocados from 2015-2020. The text states that \\\"the average price of organic avocados is generally always higher than conventional avocados.\\\"\\n\",\n \"\\n\",\n \"2. Figure 3, a pie chart, illustrates that \\\"Nearly 58% of organic avocado sales averaged $1.80 per avocado and roughly 42% of conventional avocados averaged $1.30 per avocado.\\\"\\n\",\n \"\\n\",\n \"3. In the conclusion section, the text explicitly states: \\\"The price of organic avocados is on average 35-40% higher than conventional avocados.\\\"\\n\",\n \"\\n\",\n \"Regarding the trend, while the text doesn't provide detailed information on price trends over time, Figure 2 shows the average prices for both organic and conventional avocados from 2015-2020, indicating that this price difference has been consistent over that period.\\n\"\n ]\n }\n ],\n \"source\": [\n \"anthropic_claude_3_opus = \\\"anthropic:claude-3-5-sonnet-20240620\\\"\\n\",\n \"response = client.chat.completions.create(model=anthropic_claude_3_opus, messages=messages)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 7,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"Yes, according to the analysis presented in the text, the price of organic avocados is higher\\n\"\n ]\n }\n ],\n \"source\": [\n \"\\n\",\n \"hf_model = \\\"huggingface:mistralai/Mistral-7B-Instruct-v0.3\\\"\\n\",\n \"response = client.chat.completions.create(model=hf_model, messages=messages)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 21,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"According to the text, yes, the price of organic avocados is on average 35-40% higher than conventional avocados.\\n\",\n \"\\n\",\n \"As for the trend, it can be observed that there is a steady growth in sales volume year after year for both conventional and organic avocados.\\n\",\n \"\\n\",\n \"However, in terms of price, the average price of organic avocados has been consistently higher than conventional avocados over the years. This can also be seen in Figure 2, which shows that the average price of organic avocados is generally always higher than conventional avocados.\\n\"\n ]\n }\n ],\n \"source\": [\n \"fireworks_model = \\\"fireworks:accounts/fireworks/models/llama-v3p2-3b-instruct\\\"\\n\",\n \"response = client.chat.completions.create(model=fireworks_model, messages=messages, temperature=0.75, presence_penalty=0.5, frequency_penalty=0.5)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 8,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"Yes, the price of organic avocados is higher than non-organic avocados. According to the text, the average price of organic avocados is generally 35-40% higher than conventional avocados.\\n\"\n ]\n }\n ],\n \"source\": [\n \"nebius_model = \\\"nebius:meta-llama/Meta-Llama-3.1-8B-Instruct-fast\\\"\\n\",\n \"response = client.chat.completions.create(model=nebius_model, messages=messages, top_p=0.01)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": []\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3 (ipykernel)\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.12.6\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 4\n}\n" }, { "path": "examples/agents/movie_buff_assistant.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# 🎬 Movie Buff Assistant with aisuite + MCP Tools\\n\",\n \"\\n\",\n \"Build your own movie recommendation assistant that:\\n\",\n \"1. 🔍 Researches movies, actors, and directors\\n\",\n \"2. 🧠 Remembers your preferences and watch history\\n\",\n \"3. 💡 Gives personalized recommendations\\n\",\n \"\\n\",\n \"## How This Works\\n\",\n \"\\n\",\n \"When you pass MCP tools to aisuite:\\n\",\n \"1. **aisuite handles the glue work** - converts MCP tool specs to the format your LLM needs (OpenAI, Anthropic, etc.)\\n\",\n \"2. **Automatic execution** - when the LLM requests a tool, aisuite calls the MCP server and returns results\\n\",\n \"3. **You just write natural prompts** - no need to worry about tool schemas or execution logic!\\n\",\n \"\\n\",\n \"This is the power of aisuite + MCP: unified tool calling across any LLM provider.\\n\",\n \"\\n\",\n \"**What you need:**\\n\",\n \"- OpenAI API key (add to `.env` file)\\n\",\n \"- Python with `uv` installed (for fetch MCP server)\\n\",\n \"- Node.js/npx installed (for memory MCP server)\\n\",\n \"\\n\",\n \"**Installation:**\\n\",\n \"```bash\\n\",\n \"pip install aisuite python-dotenv\\n\",\n \"pip install 'aisuite[mcp]' # Includes MCP client + nest_asyncio for Jupyter support\\n\",\n \"pip install uv # For fetch MCP server\\n\",\n \"```\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 1,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"✓ Ready to discover movies!\\n\"\n ]\n }\n ],\n \"source\": [\n \"import os\\n\",\n \"import sys\\n\",\n \"from pathlib import Path\\n\",\n \"\\n\",\n \"# Add parent directory to Path - to pick up aisuite for development\\n\",\n \"# Skip this step if you're running from an installed package\\n\",\n \"repo_root = Path().absolute().parent.parent\\n\",\n \"if str(repo_root) not in sys.path:\\n\",\n \" sys.path.insert(0, str(repo_root))\\n\",\n \"\\n\",\n \"from dotenv import load_dotenv\\n\",\n \"from aisuite import Client\\n\",\n \"from aisuite.mcp import MCPClient # Needed to connect to MCP servers.\\n\",\n \"\\n\",\n \"load_dotenv()\\n\",\n \"\\n\",\n \"# Verify API key\\n\",\n \"if not os.getenv(\\\"OPENAI_API_KEY\\\"):\\n\",\n \" raise ValueError(\\\"Add OPENAI_API_KEY to .env file!\\\")\\n\",\n \"\\n\",\n \"print(\\\"✓ Ready to discover movies!\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Step 1: Set Up Your Movie Assistant\\n\",\n \"\\n\",\n \"Provide tools from 2 different MCP servers to the LLM.\\n\",\n \"- **Fetch**: Get movie info from the web (IMDb, Wikipedia, reviews)\\n\",\n \"- **Memory**: Remember what movies you like and dislike\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 2,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"Fetch server ready - can research movies from the web\\n\",\n \"Memory server ready - will remember your preferences\\n\",\n \"📁 Memories stored in: /Users/rohit/fleet/leclerc/aisuite-prs/aisuite-main/aisuite/examples/agents/movie_memory.jsonl\\n\"\n ]\n }\n ],\n \"source\": [\n \"# Start fetch server (for getting movie data from the web)\\n\",\n \"fetch_mcp = MCPClient(\\n\",\n \" command=\\\"uvx\\\",\\n\",\n \" args=[\\\"mcp-server-fetch\\\"],\\n\",\n \" name=\\\"fetch\\\"\\n\",\n \")\\n\",\n \"\\n\",\n \"# Set up memory file for your movie preferences\\n\",\n \"memory_file = os.path.join(os.getcwd(), \\\"movie_memory.jsonl\\\")\\n\",\n \"# Start memory server (for remembering your preferences)\\n\",\n \"memory_mcp = MCPClient(\\n\",\n \" command=\\\"npx\\\",\\n\",\n \" args=[\\\"-y\\\", \\\"@modelcontextprotocol/server-memory\\\"],\\n\",\n \" env={\\\"MEMORY_FILE_PATH\\\": memory_file},\\n\",\n \" name=\\\"memory\\\"\\n\",\n \")\\n\",\n \"\\n\",\n \"print(\\\"Fetch server ready - can research movies from the web\\\")\\n\",\n \"print(\\\"Memory server ready - will remember your preferences\\\")\\n\",\n \"print(f\\\"📁 Memories stored in: {memory_file}\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Step 2: Research a Movie & Store Your Opinion\\n\",\n \"\\n\",\n \"Let's ask the assistant to research a movie and remember whether you liked it!\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 3,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"============================================================\\n\",\n \"🎬 MOVIE RESEARCH\\n\",\n \"============================================================\\n\",\n \"🎬 I'm thrilled to share some fascinating facts about *Inception*, a cinematic gem!\\n\",\n \"\\n\",\n \"1. **A Global Shooting Journey**: Did you know that *Inception* was filmed in six different countries? The production kicked off in Tokyo and wrapped up in Canada, making it a truly international affair! 🌏\\n\",\n \"\\n\",\n \"2. **A Decade in the Making**: Director Christopher Nolan initially penned an 80-page treatment for *Inception* after completing *Insomnia* back in 2002. But he decided to hone his craft with other projects, like *Batman Begins* and *The Dark Knight*, before finally bringing his dream-stealing concept to life. Talk about dedication! 📜✍️\\n\",\n \"\\n\",\n \"3. **Mind-Bending Visuals**: The film is renowned for its stunning visual effects, especially the iconic scenes where the streets of Paris fold up like a mind-bending puzzle. It's a visual masterpiece that keeps you in awe! 🎥✨\\n\",\n \"\\n\",\n \"I've enthusiastically recorded that *Inception* is a movie I love for its complex plot and visual brilliance. It aligns perfectly with my liking for sci-fi, intricate narratives, and, of course, Christopher Nolan's exceptional filmmaking style! 🎬💕\\n\"\n ]\n }\n ],\n \"source\": [\n \"client = Client()\\n\",\n \"\\n\",\n \"# Combine tools from both servers\\n\",\n \"all_tools = fetch_mcp.get_callable_tools() + memory_mcp.get_callable_tools()\\n\",\n \"\\n\",\n \"response = client.chat.completions.create(\\n\",\n \" model=\\\"openai:gpt-4o\\\",\\n\",\n \" messages=[{\\n\",\n \" \\\"role\\\": \\\"user\\\",\\n\",\n \" \\\"content\\\": \\\"\\\"\\\"Research the movie 'Inception' from https://www.imdb.com/title/tt1375666/ or Wikipedia.\\n\",\n \" \\n\",\n \" Then:\\n\",\n \" 1. Store 'Inception' as a movie entity I loved\\n\",\n \" 2. Store that I like: complex plots, sci-fi, Christopher Nolan movies\\n\",\n \" 3. Add observations about why it's great (mind-bending, great visuals, etc.)\\n\",\n \" 4. Tell me 2-3 interesting facts you found\\n\",\n \" \\n\",\n \" Be enthusiastic and conversational!\\\"\\\"\\\"\\n\",\n \" }],\\n\",\n \" tools=all_tools,\\n\",\n \" max_turns=10\\n\",\n \")\\n\",\n \"\\n\",\n \"print(\\\"=\\\"*60)\\n\",\n \"print(\\\"🎬 MOVIE RESEARCH\\\")\\n\",\n \"print(\\\"=\\\"*60)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Step 3: Get Personalized Recommendations\\n\",\n \"\\n\",\n \"Now ask for recommendations based on what it remembers about your tastes!\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 4,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"============================================================\\n\",\n \"💡 PERSONALIZED RECOMMENDATIONS\\n\",\n \"============================================================\\n\",\n \"🎬 Hey there, movie buff! Here's a quick refresher on the movies you loved and your go-to interests:\\n\",\n \"\\n\",\n \"**Movies You Loved:**\\n\",\n \"1. **Inception**: A mind-bending thrill ride directed by the genius Christopher Nolan. It's got all the layers, twists, and turns you love!\\n\",\n \"2. **Arrival**: This one’s a top recommendation for you because of its cerebral, first-contact story, wrapped in sci-fi brilliance!\\n\",\n \"\\n\",\n \"**Your Movie Interests:**\\n\",\n \"1. **Complex Plots**: You love narratives that make you think and keep you on the edge of your seat.\\n\",\n \"2. **Sci-Fi**: Exploring futuristic themes and imaginative worlds is right up your alley.\\n\",\n \"3. **Christopher Nolan Movies**: His creative storytelling and direction style never fail to capture your attention.\\n\",\n \"\\n\",\n \"🎥 **Movie Recommendations You'll Probably Enjoy:**\\n\",\n \"\\n\",\n \"1. **Blade Runner 2049**: A breathtaking continuation of a sci-fi classic, directed by Denis Villeneuve. The movie's stunning visuals, intricate plot, and philosophical questions about humanity will have you entranced!\\n\",\n \" - **Why You'll Love It**: With its rich storytelling, complex themes, and connection to Denis Villeneuve (who directed another favorite, Arrival), this film fits your love for thought-provoking sci-fi perfectly!\\n\",\n \"\\n\",\n \"2. **Interstellar**: Another Nolan masterpiece! This epic adventure takes you through space and time with mind-blowing scientific concepts and emotional depth.\\n\",\n \" - **Why It's Perfect for You**: Featuring Nolan’s signature storytelling, coupled with complex theoretical physics, it’s a movie that’ll satisfy your craving for a narrative that challenges and engages.\\n\",\n \"\\n\",\n \"3. **The Prestige**: Dive into the world of magic and bitter rivalries with this gripping film by Christopher Nolan. Full of twists and turns, it's a story that'll keep you guessing.\\n\",\n \" - **Why You'll Enjoy It**: With its layered storytelling and mystery, it taps right into your love for complex plots and Nolan’s directional genius!\\n\",\n \"\\n\",\n \"Enjoy these cinematic adventures! Grab some popcorn and prepare to be amazed! 🍿✨\\n\"\n ]\n }\n ],\n \"source\": [\n \"response = client.chat.completions.create(\\n\",\n \" model=\\\"openai:gpt-4o\\\",\\n\",\n \" messages=[{\\n\",\n \" \\\"role\\\": \\\"user\\\",\\n\",\n \" \\\"content\\\": \\\"\\\"\\\"Tell me what you know about my movie preferences from memory, and suggest something new:\\n\",\n \" \\n\",\n \" 1. Remind me what movies I liked\\n\",\n \" 2. Suggest 3 movies I'd probably enjoy\\n\",\n \" 3. Explain why each recommendation fits my taste\\n\",\n \" \\n\",\n \" Be enthusiastic like a friend recommending movies!\\\"\\\"\\\"\\n\",\n \" }],\\n\",\n \" tools=memory_mcp.get_callable_tools(),\\n\",\n \" max_turns=10\\n\",\n \")\\n\",\n \"\\n\",\n \"print(\\\"=\\\"*60)\\n\",\n \"print(\\\"💡 PERSONALIZED RECOMMENDATIONS\\\")\\n\",\n \"print(\\\"=\\\"*60)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Step 4: Research a Director's Filmography\\n\",\n \"\\n\",\n \"Let's explore a Director's work and see if the LLM can recommend something based on the limited preferences I saved earlier:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 6,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"============================================================\\n\",\n \"🎥 DIRECTOR DEEP DIVE\\n\",\n \"============================================================\\n\",\n \"### Denis Villeneuve's Major Films\\n\",\n \"\\n\",\n \"Denis Villeneuve, a Canadian director, is renowned for his cerebral thrillers and large-scale science fiction films. Here is a list of his major films:\\n\",\n \"\\n\",\n \"1. **Incendies (2010)** - A powerful drama exploring family secrets and the horrors of war.\\n\",\n \"2. **Prisoners (2013)** - A gripping thriller about a father's desperate search for his missing daughter.\\n\",\n \"3. **Enemy (2013)** - A psychological thriller exploring themes of identity and duality.\\n\",\n \"4. **Sicario (2015)** - A tense thriller about the drug war on the U.S.-Mexico border.\\n\",\n \"5. **Arrival (2016)** - A cerebral first-contact story about language, time, and choice.\\n\",\n \"6. **Blade Runner 2049 (2017)** - A visually stunning sequel to the classic sci-fi film, delving into the nature of humanity.\\n\",\n \"7. **Dune (2021)** - An epic adaptation of the renowned science fiction novel.\\n\",\n \"8. **Dune: Part Two (2024)** - The continuation of the epic sci-fi saga.\\n\",\n \"\\n\",\n \"### Personalized Film Recommendation\\n\",\n \"\\n\",\n \"Based on your love of complex plots, science-fiction, and storytelling akin to Christopher Nolan's style, you would likely love Denis Villeneuve's **Arrival (2016)**. It's not just a sci-fi film; it's a profound exploration of language and time with an elegant puzzle-box structure that will keep you engaged and intrigued throughout.\\n\",\n \"\\n\",\n \"### Storing Your Top Recommendation\\n\",\n \"\\n\",\n \"I will now store **Arrival (2016)** as your top film recommendation in memory, seeing as it fits perfectly with your interests and past movie preferences. Enjoy discovering Denis Villeneuve's masterful storytelling!\\n\"\n ]\n }\n ],\n \"source\": [\n \"response = client.chat.completions.create(\\n\",\n \" model=\\\"openai:gpt-4o\\\",\\n\",\n \" messages=[{\\n\",\n \" \\\"role\\\": \\\"user\\\",\\n\",\n \" \\\"content\\\": \\\"\\\"\\\"Research Denis Villeneuve's filmography from IMDb or Wikipedia.\\n\",\n \" \\n\",\n \" 1. List his major films\\n\",\n \" 2. Based on my interests and moveis I liked earlier (check memory!), which of his films would I love?\\n\",\n \" 3. Store the top recommendation in memory\\n\",\n \" \\n\",\n \" Make it exciting - I love discovering new directors!\\\"\\\"\\\"\\n\",\n \" }],\\n\",\n \" tools=all_tools,\\n\",\n \" max_turns=10\\n\",\n \")\\n\",\n \"\\n\",\n \"print(\\\"=\\\"*60)\\n\",\n \"print(\\\"🎥 DIRECTOR DEEP DIVE\\\")\\n\",\n \"print(\\\"=\\\"*60)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Cleanup\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"fetch_mcp.close()\\n\",\n \"memory_mcp.close()\\n\",\n \"print(\\\"✓ Servers closed - your movie preferences are saved!\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Recap of what we did above\\n\",\n \"\\n\",\n \"Behind the scenes, aisuite few things:\\n\",\n \"- ✅ **Converted MCP tool schemas** to OpenAI-compatible format\\n\",\n \"- ✅ **Handled tool execution** when the LLM requested them\\n\",\n \"- ✅ **Managed async operations** for MCP server communication\\n\",\n \"\\n\",\n \"MCP Tools\\n\",\n \"**Web search** - LLM could do web search due to Fetch being passed as a tool.\\n\",\n \"**Preserved your data in knowledge graph** - LLM could store your preferences and look it using server-memory being passed as a tool.\\n\",\n \"\\n\",\n \"You, as an user, just wrote natural prompts - aisuite handled all the tool calling complexity - LLM returned personalized recommendations.\\n\",\n \"\\n\",\n \"You built a **personalized movie assistant** that:\\n\",\n \"- ✅ Researches movies from the web\\n\",\n \"- ✅ Learns your preferences over time\\n\",\n \"- ✅ Gives smart recommendations\\n\",\n \"- ✅ Remembers everything across sessions\\n\",\n \"- ✅ Builds a knowledge graph of your tastes\\n\",\n \"\\n\",\n \"**All with minimal code!**\\n\",\n \"\\n\",\n \"## Try These Next\\n\",\n \"\\n\",\n \"Now that you've got the basics, explore more capabilities:\\n\",\n \"\\n\",\n \"### 🎭 Refine Your Taste Profile\\n\",\n \"```python\\n\",\n \"# Tell it what you DON'T like to improve recommendations\\n\",\n \"response = client.chat.completions.create(\\n\",\n \" model=\\\"openai:gpt-4o\\\",\\n\",\n \" messages=[{\\\"role\\\": \\\"user\\\", \\\"content\\\": \\\"\\\"\\\"Store in memory that I didn't enjoy:\\n\",\n \" - 'Transformers' movies (too much action, not enough plot)\\n\",\n \" - 'Scary Movie' series (slapstick comedy isn't my thing)\\n\",\n \" \\n\",\n \" Then suggest 3 movies I WOULD like based on my updated profile.\\\"\\\"\\\"}],\\n\",\n \" tools=memory_mcp.get_callable_tools(),\\n\",\n \" max_turns=5\\n\",\n \")\\n\",\n \"```\\n\",\n \"\\n\",\n \"### 📊 Query Your Complete Watch History\\n\",\n \"```python\\n\",\n \"# See everything the assistant remembers\\n\",\n \"response = client.chat.completions.create(\\n\",\n \" model=\\\"openai:gpt-4o\\\",\\n\",\n \" messages=[{\\\"role\\\": \\\"user\\\", \\\"content\\\": \\\"\\\"\\\"Search your memory and tell me:\\n\",\n \" 1. What movies do I love?\\n\",\n \" 2. What movies do I dislike?\\n\",\n \" 3. What are my key preferences (genres, themes, directors)?\\n\",\n \" 4. Based on all this, what's ONE perfect movie recommendation?\\\"\\\"\\\"}],\\n\",\n \" tools=memory_mcp.get_callable_tools(),\\n\",\n \" max_turns=5\\n\",\n \")\\n\",\n \"```\\n\",\n \"\\n\",\n \"### 🔍 Explore Your Memory File\\n\",\n \"```python\\n\",\n \"# See the raw knowledge graph\\n\",\n \"import json\\n\",\n \"with open(memory_file, 'r') as f:\\n\",\n \" for line in f.readlines()[:10]:\\n\",\n \" entry = json.loads(line)\\n\",\n \" print(f\\\"{entry.get('type')}: {entry.get('name')}\\\")\\n\",\n \"```\\n\",\n \"\\n\",\n \"### 🌟 More Fun Queries\\n\",\n \"- \\\"Find me a thriller like 'Gone Girl'\\\"\\n\",\n \"- \\\"What are the best movies of 2024?\\\"\\n\",\n \"- \\\"Research Greta Gerwig's films and recommend one\\\"\\n\",\n \"- \\\"I'm in the mood for something uplifting - what should I watch?\\\"\\n\",\n \"- \\\"Based on my taste, should I watch [specific movie]?\\\"\\n\",\n \"- \\\"Find me a hidden gem from the 90s I might have missed\\\"\\n\",\n \"\\n\",\n \"## The Pattern\\n\",\n \"\\n\",\n \"Setup MCP tools, and call chat.completions.create() with max_turns.\\n\",\n \"\\n\",\n \"```python\\n\",\n \"# 1. Set up fetch + memory\\n\",\n \"fetch_mcp = MCPClient(command=\\\"uvx\\\", args=[\\\"mcp-server-fetch\\\"])\\n\",\n \"memory_mcp = MCPClient(\\n\",\n \" command=\\\"npx\\\",\\n\",\n \" args=[\\\"-y\\\", \\\"@modelcontextprotocol/server-memory\\\"],\\n\",\n \" env={\\\"MEMORY_FILE_PATH\\\": \\\"movie_memory.jsonl\\\"}\\n\",\n \")\\n\",\n \"\\n\",\n \"# 2. Combine tools\\n\",\n \"tools = fetch_mcp.get_callable_tools() + memory_mcp.get_callable_tools()\\n\",\n \"\\n\",\n \"# 3. Chat naturally!\\n\",\n \"response = client.chat.completions.create(\\n\",\n \" model=\\\"openai:gpt-4o\\\",\\n\",\n \" messages=[{\\\"role\\\": \\\"user\\\", \\\"content\\\": \\\"Research X and remember Y\\\"}],\\n\",\n \" tools=tools,\\n\",\n \" max_turns=5\\n\",\n \")\\n\",\n \"```\\n\",\n \"\\n\",\n \"## Other Ideas\\n\",\n \"\\n\",\n \"Use this same pattern for:\\n\",\n \"- 🍳 **Recipe assistant** that learns your dietary preferences (see `recipe_chef_assistant.ipynb`)\\n\",\n \"- ✈️ **Travel planner** that remembers your bucket list\\n\",\n \"- 📚 **Book recommender** that tracks your reading history\\n\",\n \"- 🎮 **Game advisor** that knows your favorite genres\\n\",\n \"- 🎵 **Music discovery** that learns your taste\\n\",\n \"\\n\",\n \"## Resources\\n\",\n \"\\n\",\n \"- **Fetch Server**: https://github.com/modelcontextprotocol/servers/tree/main/src/fetch\\n\",\n \"- **Memory Server**: https://github.com/modelcontextprotocol/servers/tree/main/src/memory\\n\",\n \"- **aisuite Documentation**: https://github.com/andrewyng/aisuite\\n\",\n \"- **More MCP Servers**: https://github.com/modelcontextprotocol/servers\\n\",\n \"\\n\",\n \"**Your movie preferences persist across sessions** - restart this notebook anytime and your assistant will remember everything! 🎬✨\\n\"\n ]\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3 (ipykernel)\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.13.3\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 4\n}\n" }, { "path": "examples/agents/recipe_chef_assistant.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# 🍳 Recipe Chef Assistant with AI + MCP\\n\",\n \"\\n\",\n \"Build your own culinary companion that:\\n\",\n \"1. 🔍 Researches recipes from cooking websites\\n\",\n \"2. 🧠 Remembers your dietary preferences and restrictions\\n\",\n \"3. 💡 Suggests recipes based on what you have\\n\",\n \"\\n\",\n \"## How This Works\\n\",\n \"\\n\",\n \"When you pass MCP tools to aisuite:\\n\",\n \"1. **aisuite handles the glue work** - converts MCP tool specs to the format your LLM needs (OpenAI, Anthropic, etc.)\\n\",\n \"2. **Automatic execution** - when the LLM requests a tool, aisuite calls the MCP server and returns results\\n\",\n \"3. **You just write natural prompts** - no need to worry about tool schemas or execution logic!\\n\",\n \"\\n\",\n \"This is the power of aisuite + MCP: unified tool calling across any LLM provider.\\n\",\n \"\\n\",\n \"**What you need:**\\n\",\n \"- Anthropic API key (add to `.env` file)\\n\",\n \"- Python with `uv` installed (for fetch MCP server)\\n\",\n \"- Node.js/npx installed (for memory MCP server)\\n\",\n \"\\n\",\n \"**Installation:**\\n\",\n \"```bash\\n\",\n \"pip install aisuite python-dotenv\\n\",\n \"pip install 'aisuite[mcp]' # Includes MCP client + nest_asyncio for Jupyter support\\n\",\n \"pip install uv # For fetch MCP server\\n\",\n \"```\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 10,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"✓ Ready to cook up some recipes!\\n\"\n ]\n }\n ],\n \"source\": [\n \"import os\\n\",\n \"import sys\\n\",\n \"from pathlib import Path\\n\",\n \"\\n\",\n \"# Add parent directory to path for development\\n\",\n \"# Skip this step if you're running from an installed package\\n\",\n \"repo_root = Path().absolute().parent.parent\\n\",\n \"if str(repo_root) not in sys.path:\\n\",\n \" sys.path.insert(0, str(repo_root))\\n\",\n \"\\n\",\n \"from dotenv import load_dotenv\\n\",\n \"from aisuite import Client\\n\",\n \"from aisuite.mcp import MCPClient\\n\",\n \"\\n\",\n \"load_dotenv()\\n\",\n \"\\n\",\n \"# Verify API key\\n\",\n \"if not os.getenv(\\\"ANTHROPIC_API_KEY\\\"):\\n\",\n \" raise ValueError(\\\"❌ Add ANTHROPIC_API_KEY to .env file!\\\")\\n\",\n \"\\n\",\n \"print(\\\"✓ Ready to cook up some recipes!\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Step 1: Set Up Your Chef Assistant\\n\",\n \"\\n\",\n \"We'll give the AI two tools:\\n\",\n \"- **Fetch**: Get recipes from cooking websites\\n\",\n \"- **Memory**: Remember your preferences, restrictions, and favorite recipes\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 2,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"🔍 Fetch server ready - can research recipes from the web\\n\",\n \"🧠 Memory server ready - will remember your preferences\\n\",\n \"📁 Recipes stored in: /Users/rohit/fleet/leclerc/aisuite-prs/aisuite-main/aisuite/examples/agents/recipe_memory.jsonl\\n\"\n ]\n }\n ],\n \"source\": [\n \"# Set up memory file for your cooking preferences\\n\",\n \"memory_file = os.path.join(os.getcwd(), \\\"recipe_memory.jsonl\\\")\\n\",\n \"\\n\",\n \"# Start fetch server (for getting recipes from the web)\\n\",\n \"fetch_mcp = MCPClient(\\n\",\n \" command=\\\"uvx\\\",\\n\",\n \" args=[\\\"mcp-server-fetch\\\"],\\n\",\n \" name=\\\"fetch\\\"\\n\",\n \")\\n\",\n \"\\n\",\n \"# Start memory server (for remembering preferences and recipes)\\n\",\n \"memory_mcp = MCPClient(\\n\",\n \" command=\\\"npx\\\",\\n\",\n \" args=[\\\"-y\\\", \\\"@modelcontextprotocol/server-memory\\\"],\\n\",\n \" env={\\\"MEMORY_FILE_PATH\\\": memory_file},\\n\",\n \" name=\\\"memory\\\"\\n\",\n \")\\n\",\n \"\\n\",\n \"print(\\\"🔍 Fetch server ready - can research recipes from the web\\\")\\n\",\n \"print(\\\"🧠 Memory server ready - will remember your preferences\\\")\\n\",\n \"print(f\\\"📁 Recipes stored in: {memory_file}\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Step 2: Set Up Your Dietary Profile\\n\",\n \"\\n\",\n \"Let's tell the assistant about your dietary preferences and restrictions:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 3,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"============================================================\\n\",\n \"👤 YOUR COOKING PROFILE\\n\",\n \"============================================================\\n\",\n \"Perfect! I've stored your dietary profile. Here's a summary:\\n\",\n \"\\n\",\n \"## Your Dietary Profile\\n\",\n \"\\n\",\n \"**🥗 Diet Type:** Vegetarian (no meat, fish, or poultry)\\n\",\n \"\\n\",\n \"**🍽️ Cuisine Preferences:**\\n\",\n \"- Italian cuisine\\n\",\n \"- Thai cuisine\\n\",\n \"\\n\",\n \"**⚠️ Dietary Restrictions:**\\n\",\n \"- **Shellfish allergy** - must be completely avoided\\n\",\n \"- **Lactose intolerant** - dairy alternatives should be used\\n\",\n \"\\n\",\n \"**❤️ Favorite Dishes:**\\n\",\n \"- Pasta dishes\\n\",\n \"- Stir-fries\\n\",\n \"- Curries\\n\",\n \"\\n\",\n \"**👨‍🍳 Cooking Style:**\\n\",\n \"- Skill level: Intermediate\\n\",\n \"- Time preference: Quick meals (under 30 minutes)\\n\",\n \"\\n\",\n \"Your profile is now saved, and I can use this information to provide personalized recipe suggestions and cooking advice that fits your preferences, restrictions, and lifestyle!\\n\"\n ]\n }\n ],\n \"source\": [\n \"client = Client()\\n\",\n \"\\n\",\n \"# Combine tools from both servers\\n\",\n \"all_tools = fetch_mcp.get_callable_tools() + memory_mcp.get_callable_tools()\\n\",\n \"\\n\",\n \"response = client.chat.completions.create(\\n\",\n \" model=\\\"anthropic:claude-sonnet-4-5\\\",\\n\",\n \" messages=[{\\n\",\n \" \\\"role\\\": \\\"user\\\",\\n\",\n \" \\\"content\\\": \\\"\\\"\\\"Store my dietary profile in memory:\\n\",\n \" \\n\",\n \" **Preferences:**\\n\",\n \" - I'm vegetarian (no meat, fish, or poultry)\\n\",\n \" - I love Italian and Thai cuisine\\n\",\n \" - I prefer quick meals (under 30 minutes)\\n\",\n \" - I'm intermediate skill level\\n\",\n \" \\n\",\n \" **Restrictions:**\\n\",\n \" - No shellfish (allergy)\\n\",\n \" - Lactose intolerant (use dairy alternatives when possible)\\n\",\n \" \\n\",\n \" **Favorites:**\\n\",\n \" - Pasta dishes\\n\",\n \" - Stir-fries\\n\",\n \" - Curries\\n\",\n \" \\n\",\n \" Then summarize my profile back to me!\\\"\\\"\\\"\\n\",\n \" }],\\n\",\n \" tools=memory_mcp.get_callable_tools(),\\n\",\n \" max_turns=5\\n\",\n \")\\n\",\n \"\\n\",\n \"print(\\\"=\\\"*60)\\n\",\n \"print(\\\"👤 YOUR COOKING PROFILE\\\")\\n\",\n \"print(\\\"=\\\"*60)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Step 3: Research a Recipe & Save It\\n\",\n \"\\n\",\n \"Let's find a great vegetarian pasta recipe:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 6,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"============================================================\\n\",\n \"🍝 RECIPE DISCOVERY\\n\",\n \"============================================================\\n\",\n \"Perfect! I found an amazing recipe for you! 🍝✨\\n\",\n \"\\n\",\n \"## **Tomato Penne with Avocado** \\n\",\n \"\\n\",\n \"### Why I Chose This Recipe:\\n\",\n \"This is an absolute winner that hits all your requirements! It's a gorgeous Italian-Mexican fusion pasta that combines the best of both worlds - your love for Italian cuisine with exciting Mexican spices. Here's what makes it perfect:\\n\",\n \"\\n\",\n \"**✅ Perfectly Fits Your Profile:**\\n\",\n \"- **100% Vegetarian** - no meat, fish, or poultry\\n\",\n \"- **Completely lactose-free** - naturally dairy-free with no cheese required!\\n\",\n \"- **No shellfish** - completely safe for your allergy\\n\",\n \"- **Quick cooking** - Ready in just 25 minutes (well under your 30-minute preference)\\n\",\n \"- **Intermediate skill level** - matches your cooking abilities perfectly\\n\",\n \"\\n\",\n \"**🌟 What Makes It Special:**\\n\",\n \"This dish is absolutely stunning! Picture tender wholemeal penne tossed in a vibrant, mildly spiced tomato sauce studded with golden caramelized onions, sweet orange peppers, and bright yellow sweetcorn. The whole thing is crowned with chunks of creamy avocado dressed in zesty lime - giving you all that rich, indulgent texture without any dairy!\\n\",\n \"\\n\",\n \"The Mexican spices (cumin, coriander, mild chilli) add warmth and depth without overwhelming heat, while fresh coriander and lime brighten everything up. It's comfort food that's also incredibly healthy - you get ALL FIVE of your daily vegetable servings in one gorgeous bowl!\\n\",\n \"\\n\",\n \"**📊 The Stats:**\\n\",\n \"- **Cook Time:** 25 minutes total\\n\",\n \"- **Difficulty:** Easy to Intermediate \\n\",\n \"- **Rating:** 4.3/5 stars (177 ratings)\\n\",\n \"- **Nutrition:** Only 495 calories, packed with 18g of fiber, low fat, rich in iron and vitamin C\\n\",\n \"\\n\",\n \"**🎨 Key Ingredients:**\\n\",\n \"Wholemeal penne, orange pepper, onion, garlic, chopped tomatoes, sweetcorn, avocado, lime, fresh coriander, and warming spices (chilli powder, cumin, ground coriander)\\n\",\n \"\\n\",\n \"This recipe is budget-friendly, uses mostly pantry staples, and the presentation is absolutely beautiful with all those vibrant colors. It's the perfect weeknight meal that tastes like you spent hours on it! 🥑🌶️🍅\\n\"\n ]\n }\n ],\n \"source\": [\n \"response = client.chat.completions.create(\\n\",\n \" model=\\\"anthropic:claude-sonnet-4-5\\\",\\n\",\n \" messages=[{\\n\",\n \" \\\"role\\\": \\\"user\\\",\\n\",\n \" \\\"content\\\": \\\"\\\"\\\"Research a delicious vegetarian pasta recipe from:\\n\",\n \" - https://www.bbcgoodfood.com/recipes/collection/vegetarian-pasta-recipes\\n\",\n \" - https://www.allrecipes.com/search?q=vegetarian+pasta\\n\",\n \" \\n\",\n \" Then:\\n\",\n \" 1. Make sure it fits my dietary profile (check memory!)\\n\",\n \" 2. Save the recipe name and key ingredients in memory\\n\",\n \" 3. Add notes about cook time and difficulty\\n\",\n \" 4. Give me a brief summary with why you chose it\\n\",\n \"\\n\",\n \" If you can't find any suitable recipe, let me know about your findings. Don't do more than 10 web fetches.\\n\",\n \" \\n\",\n \" Make it sound appetizing!\\\"\\\"\\\"\\n\",\n \" }],\\n\",\n \" tools=all_tools,\\n\",\n \" max_turns=20\\n\",\n \")\\n\",\n \"\\n\",\n \"print(\\\"=\\\"*60)\\n\",\n \"print(\\\"🍝 RECIPE DISCOVERY\\\")\\n\",\n \"print(\\\"=\\\"*60)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Step 4: \\\"What Can I Make?\\\" - Ingredient-Based Search\\n\",\n \"\\n\",\n \"Got random ingredients? Let's find what you can cook:\\n\",\n \"\\n\",\n \"NOTE: We have not saved much recipes nor preferences, so you may not get an excellent suggestion! But, it will still serve the purpose of demonstrating tool usage, and can be easily extended to provide better suggestions.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 9,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"============================================================\\n\",\n \"🥘 INGREDIENT-BASED RECIPE\\n\",\n \"============================================================\\n\",\n \"# 🍛 **Thai Chickpea Coconut Curry** - Perfect for You Tonight!\\n\",\n \"\\n\",\n \"---\\n\",\n \"\\n\",\n \"## ✨ **Why I Chose This Recipe:**\\n\",\n \"\\n\",\n \"This is absolutely **IDEAL** for you! Here's why:\\n\",\n \"- ✅ **Thai cuisine** - one of your favorites!\\n\",\n \"- ✅ **Curry** - you specifically mentioned this as a favorite dish\\n\",\n \"- ✅ **25 minutes total** - under your 30-minute preference\\n\",\n \"- ✅ **100% vegetarian & vegan** - naturally dairy-free (no lactose issues!)\\n\",\n \"- ✅ **You have EVERY ingredient** already in your kitchen!\\n\",\n \"- ✅ **Intermediate skill level** - perfect for your abilities\\n\",\n \"- ✅ **One-pot meal** - easy cleanup!\\n\",\n \"\\n\",\n \"---\\n\",\n \"\\n\",\n \"## 📝 **Simple Step-by-Step Instructions:**\\n\",\n \"\\n\",\n \"### **Step 1: Start Your Rice** (5 mins)\\n\",\n \"- Cook your rice according to package directions. This will be ready when your curry is done!\\n\",\n \"\\n\",\n \"### **Step 2: Build Your Flavor Base** (5 mins)\\n\",\n \"- Heat a little oil in a large pan over medium heat\\n\",\n \"- Add diced onion and sauté until soft and translucent (about 5 minutes)\\n\",\n \"\\n\",\n \"### **Step 3: Add Aromatics** (1-2 mins)\\n\",\n \"- Stir in minced garlic and ginger\\n\",\n \"- Cook for 1 minute until fragrant (your kitchen will smell amazing!)\\n\",\n \"\\n\",\n \"### **Step 4: Toast Your Spices** (1 min)\\n\",\n \"- Add your spices: curry powder, cumin, turmeric, and coriander (about 1-2 tsp each)\\n\",\n \"- Stir for about 1 minute to bloom the flavors\\n\",\n \"\\n\",\n \"### **Step 5: Add Tomatoes** (3 mins)\\n\",\n \"- Toss in diced tomatoes\\n\",\n \"- Cook until they start to soften and break down\\n\",\n \"\\n\",\n \"### **Step 6: Make It Creamy** (10 mins)\\n\",\n \"- Pour in the entire can of coconut milk\\n\",\n \"- Add drained chickpeas\\n\",\n \"- Give it a good stir and let it simmer for 10 minutes until the sauce thickens nicely\\n\",\n \"\\n\",\n \"### **Step 7: Season & Serve!**\\n\",\n \"- Taste and add salt and pepper as needed\\n\",\n \"- Serve over your fluffy rice\\n\",\n \"- *Optional garnish*: fresh cilantro if you have it, or a squeeze of lime for extra zing!\\n\",\n \"\\n\",\n \"---\\n\",\n \"\\n\",\n \"## 💪 **You've Got This!**\\n\",\n \"\\n\",\n \"This recipe is **foolproof** and incredibly forgiving. The coconut milk makes everything creamy and rich, the chickpeas add protein and heartiness, and those aromatics (garlic, ginger, onion) create layers of flavor that taste like you spent hours cooking!\\n\",\n \"\\n\",\n \"Plus, this gets even **better the next day**, so if you make extra, you'll have an amazing lunch tomorrow. The best part? It's all pantry staples, so you're making restaurant-quality Thai food without a trip to the store!\\n\",\n \"\\n\",\n \"**Enjoy your delicious, aromatic curry! You're going to love how this turns out!** 🌟\\n\"\n ]\n }\n ],\n \"source\": [\n \"response = client.chat.completions.create(\\n\",\n \" model=\\\"anthropic:claude-sonnet-4-5\\\",\\n\",\n \" messages=[{\\n\",\n \" \\\"role\\\": \\\"user\\\",\\n\",\n \" \\\"content\\\": \\\"\\\"\\\"I have these ingredients in my kitchen:\\n\",\n \" - Chickpeas (canned)\\n\",\n \" - Coconut milk\\n\",\n \" - Tomatoes\\n\",\n \" - Onions, garlic, ginger\\n\",\n \" - Rice\\n\",\n \" - Various spices\\n\",\n \" \\n\",\n \" Based on my dietary profile and what I have:\\n\",\n \" 1. Suggest a recipe I can make\\n\",\n \" 2. Check if it matches my preferences (Thai/Italian, quick, vegetarian)\\n\",\n \" 3. Save this recipe idea to memory\\n\",\n \" \\n\",\n \" As the final response, give me a simple step-by-step overview of how to make it, and why you chose this for me!\\n\",\n \" \\n\",\n \" Be encouraging and practical!\\\"\\\"\\\"\\n\",\n \" }],\\n\",\n \" tools=all_tools,\\n\",\n \" max_turns=10\\n\",\n \")\\n\",\n \"\\n\",\n \"print(\\\"=\\\"*60)\\n\",\n \"print(\\\"🥘 INGREDIENT-BASED RECIPE\\\")\\n\",\n \"print(\\\"=\\\"*60)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Cleanup\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"fetch_mcp.close()\\n\",\n \"memory_mcp.close()\\n\",\n \"print(\\\"✓ Servers closed - your recipes are saved!\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Recap of what we did above\\n\",\n \"\\n\",\n \"Behind the scenes, aisuite did some magic:\\n\",\n \"- ✅ **Converted MCP tool schemas** to OpenAI-compatible format\\n\",\n \"- ✅ **Handled tool execution** when the LLM requested them\\n\",\n \"- ✅ **Managed async operations** for MCP server communication\\n\",\n \"\\n\",\n \"MCP Tools\\n\",\n \"**Web search** - LLM could do web search due to Fetch being passed as a tool.\\n\",\n \"**Preserved your data in knowledge graph** - LLM could store your preferences and look it using server-memory being passed as a tool.\\n\",\n \"\\n\",\n \"You, as an user, just wrote natural prompts - aisuite handled all the tool calling complexity - LLM returned personalized recommendations.\\n\",\n \"\\n\",\n \"You built a **personal chef assistant** that:\\n\",\n \"- ✅ Researches recipes tailored to your diet\\n\",\n \"- ✅ Remembers your restrictions and preferences\\n\",\n \"- ✅ Suggests recipes based on ingredients you have\\n\",\n \"- ✅ Saves your favorite recipes and cooking notes\\n\",\n \"- ✅ Provides personalized cooking advice\\n\",\n \"\\n\",\n \"**All with minimal code!**\\n\",\n \"\\n\",\n \"## Try These Next\\n\",\n \"\\n\",\n \"Now that you've got the basics, explore more capabilities:\\n\",\n \"\\n\",\n \"### 📦 Meal Prep Planning\\n\",\n \"```python\\n\",\n \"response = client.chat.completions.create(\\n\",\n \" model=\\\"anthropic:claude-sonnet-4-5\\\",\\n\",\n \" messages=[{\\\"role\\\": \\\"user\\\", \\\"content\\\": \\\"\\\"\\\"I want to meal prep for the week. Based on my profile:\\n\",\n \" \\n\",\n \" 1. Suggest 2 vegetarian recipes that:\\n\",\n \" - Reheat well\\n\",\n \" - Stay fresh for 3-4 days\\n\",\n \" - Match my taste (Italian/Thai)\\n\",\n \" - Are filling and nutritious\\n\",\n \" \\n\",\n \" 2. Save both recipes to memory as 'meal prep favorites'\\n\",\n \" 3. Give me storage tips for each\\\"\\\"\\\"}],\\n\",\n \" tools=all_tools,\\n\",\n \" max_turns=8\\n\",\n \")\\n\",\n \"```\\n\",\n \"\\n\",\n \"### 📚 Browse Your Recipe Collection\\n\",\n \"```python\\n\",\n \"response = client.chat.completions.create(\\n\",\n \" model=\\\"anthropic:claude-sonnet-4-5\\\",\\n\",\n \" messages=[{\\\"role\\\": \\\"user\\\", \\\"content\\\": \\\"\\\"\\\"Search your memory and tell me:\\n\",\n \" \\n\",\n \" 1. What recipes have I saved?\\n\",\n \" 2. What are my dietary restrictions and preferences?\\n\",\n \" 3. Which recipes are best for meal prep?\\n\",\n \" 4. Suggest what I should cook tonight based on my saved recipes\\\"\\\"\\\"}],\\n\",\n \" tools=memory_mcp.get_callable_tools(),\\n\",\n \" max_turns=5\\n\",\n \")\\n\",\n \"```\\n\",\n \"\\n\",\n \"### 💡 Cooking Tips & Substitutions\\n\",\n \"```python\\n\",\n \"response = client.chat.completions.create(\\n\",\n \" model=\\\"anthropic:claude-sonnet-4-5\\\",\\n\",\n \" messages=[{\\\"role\\\": \\\"user\\\", \\\"content\\\": \\\"\\\"\\\"I want to make a recipe that calls for:\\n\",\n \" - Heavy cream (but I'm lactose intolerant!)\\n\",\n \" - Parmesan cheese\\n\",\n \" \\n\",\n \" 1. Suggest dairy-free substitutions\\n\",\n \" 2. Store these substitution tips in memory\\n\",\n \" 3. Any other dairy-free tips for Italian cooking?\\\"\\\"\\\"}],\\n\",\n \" tools=all_tools,\\n\",\n \" max_turns=6\\n\",\n \")\\n\",\n \"```\\n\",\n \"\\n\",\n \"### 🔍 Explore Your Recipe Memory\\n\",\n \"```python\\n\",\n \"# See the raw knowledge graph\\n\",\n \"import json\\n\",\n \"with open(memory_file, 'r') as f:\\n\",\n \" for line in f.readlines()[:10]:\\n\",\n \" entry = json.loads(line)\\n\",\n \" if 'recipe' in entry.get('name', '').lower():\\n\",\n \" print(f\\\"🍽️ {entry.get('name')}\\\")\\n\",\n \"```\\n\",\n \"\\n\",\n \"### 🌟 More Fun Queries\\n\",\n \"- \\\"Find me a healthy salad for lunch\\\"\\n\",\n \"- \\\"I want to try baking - suggest a beginner dessert\\\"\\n\",\n \"- \\\"Research authentic Thai curry recipes from https://www.bbcgoodfood.com/recipes/collection/thai-curry-recipes\\\"\\n\",\n \"- \\\"Plan a dinner party menu for 6 people\\\"\\n\",\n \"- \\\"What cooking techniques should I learn next?\\\"\\n\",\n \"- \\\"Create a grocery list for this week's meal prep\\\"\\n\",\n \"\\n\",\n \"## The Pattern\\n\",\n \"\\n\",\n \"Setup MCP tools, and call chat.completions.create() with max_turns.\\n\",\n \"\\n\",\n \"```python\\n\",\n \"# 1. Set up fetch + memory\\n\",\n \"fetch_mcp = MCPClient(command=\\\"uvx\\\", args=[\\\"mcp-server-fetch\\\"])\\n\",\n \"memory_mcp = MCPClient(\\n\",\n \" command=\\\"npx\\\",\\n\",\n \" args=[\\\"-y\\\", \\\"@modelcontextprotocol/server-memory\\\"],\\n\",\n \" env={\\\"MEMORY_FILE_PATH\\\": \\\"recipe_memory.jsonl\\\"}\\n\",\n \")\\n\",\n \"\\n\",\n \"# 2. Combine tools\\n\",\n \"tools = fetch_mcp.get_callable_tools() + memory_mcp.get_callable_tools()\\n\",\n \"\\n\",\n \"# 3. Cook with AI!\\n\",\n \"response = client.chat.completions.create(\\n\",\n \" model=\\\"openai:gpt-4o\\\",\\n\",\n \" messages=[{\\\"role\\\": \\\"user\\\", \\\"content\\\": \\\"Find a recipe and remember it\\\"}],\\n\",\n \" tools=tools,\\n\",\n \" max_turns=5\\n\",\n \")\\n\",\n \"```\\n\",\n \"\\n\",\n \"## Advanced Ideas\\n\",\n \"\\n\",\n \"Extend this assistant to:\\n\",\n \"- 📊 Track nutritional goals and suggest balanced meals\\n\",\n \"- 🛒 Generate smart grocery lists with budget tracking\\n\",\n \"- 📅 Plan weekly menus automatically\\n\",\n \"- 👨‍🍳 Learn and improve from your cooking feedback\\n\",\n \"- 🌍 Explore cuisines from different cultures\\n\",\n \"- 📸 Store photos of your dishes (with filesystem MCP)\\n\",\n \"- ⏰ Set cooking timers and reminders\\n\",\n \"\\n\",\n \"## Other Assistants to Build\\n\",\n \"\\n\",\n \"Use the same pattern for:\\n\",\n \"- 🎬 **Movie recommendations** (see `movie_buff_assistant.ipynb`)\\n\",\n \"- ✈️ **Travel planning** that remembers your bucket list\\n\",\n \"- 📚 **Reading tracker** that knows your taste in books\\n\",\n \"- 🏋️ **Fitness coach** that tracks your progress\\n\",\n \"- 🌱 **Garden planner** that remembers what you planted\\n\",\n \"\\n\",\n \"## Resources\\n\",\n \"\\n\",\n \"- **Fetch Server**: https://github.com/modelcontextprotocol/servers/tree/main/src/fetch\\n\",\n \"- **Memory Server**: https://github.com/modelcontextprotocol/servers/tree/main/src/memory\\n\",\n \"- **aisuite Documentation**: https://github.com/andrewyng/aisuite\\n\",\n \"- **More MCP Servers**: https://github.com/modelcontextprotocol/servers\\n\",\n \"\\n\",\n \"**Happy cooking! Your recipes and preferences persist forever!** 🍳✨\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": []\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3 (ipykernel)\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.13.3\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 4\n}\n" }, { "path": "examples/agents/snake_game_generator.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Snake Game Generator - AI Creates a Game\\n\",\n \"\\n\",\n \"This notebook shows how to use aisuite + MCP tools to have an AI generate a complete, playable Snake game.\\n\",\n \"\\n\",\n \"**What it does**: The LLM generates a Snake game in HTML/CSS/JavaScript, saves it to a file using MCP filesystem tools, and we display it right in the notebook.\\n\",\n \"\\n\",\n \"**Requirements**: `OPENAI_API_KEY` or `ANTHROPIC_API_KEY` in your `.env` file (depending on which model you choose)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Setup\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 2,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"✓ Ready!\\n\"\n ]\n }\n ],\n \"source\": [\n \"import os\\n\",\n \"from dotenv import load_dotenv\\n\",\n \"import aisuite as ai\\n\",\n \"from aisuite.mcp import MCPClient\\n\",\n \"from IPython.display import IFrame, display # For displaying HTML file\\n\",\n \"\\n\",\n \"load_dotenv()\\n\",\n \"\\n\",\n \"# Initialize filesystem MCP server for file writing\\n\",\n \"filesystem_mcp = MCPClient(\\n\",\n \" command=\\\"npx\\\",\\n\",\n \" args=[\\\"-y\\\", \\\"@modelcontextprotocol/server-filesystem\\\", os.getcwd()]\\n\",\n \")\\n\",\n \"\\n\",\n \"print(\\\"✓ Ready!\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Craft Instructions\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 3,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"prompt = \\\"\\\"\\\"Create a complete, playable Snake game.\\n\",\n \"\\n\",\n \"**EXECUTION RULES:**\\n\",\n \"- Execute ALL tools silently (no intermediate text responses)\\n\",\n \"- Write the HTML file FIRST, then provide a brief summary\\n\",\n \"\\n\",\n \"**GAME REQUIREMENTS:**\\n\",\n \" Styling:\\n\",\n \" - Clean, modern look\\n\",\n \" - Centered on page\\n\",\n \" - Nice colors (dark background, bright snake, contrasting food)\\n\",\n \" - Clear score display\\n\",\n \" - Arrow keys to change direction\\n\",\n \" - Instructions shown on screen\\n\",\n \"\\n\",\n \"**Save the file:**\\n\",\n \" - Use write_file to save as 'snake_game.html'\\n\",\n \" - After saving, respond with confirmation that the game was created\\n\",\n \"\\\"\\\"\\\"\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Run Agent with MCP Tools\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"client = ai.Client()\\n\",\n \"tools = filesystem_mcp.get_callable_tools()\\n\",\n \"\\n\",\n \"# Choose your model (uncomment one):\\n\",\n \"model = \\\"openai:gpt-5.1\\\"\\n\",\n \"# model = \\\"anthropic:claude-sonnet-4-5\\\"\\n\",\n \"\\n\",\n \"response = client.chat.completions.create(\\n\",\n \" model=model,\\n\",\n \" messages=[{\\\"role\\\": \\\"user\\\", \\\"content\\\": prompt}],\\n\",\n \" tools=tools,\\n\",\n \" max_turns=5\\n\",\n \")\\n\",\n \"\\n\",\n \"print(\\\"✓ Done!\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Play the Game\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 4,\n \"metadata\": {},\n \"outputs\": [\n {\n \"data\": {\n \"text/html\": [\n \"\\n\",\n \" \\n\",\n \" \"\n ],\n \"text/plain\": [\n \"\"\n ]\n },\n \"metadata\": {},\n \"output_type\": \"display_data\"\n },\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"\\n\",\n \"💡 Open 'snake_game.html' in your browser for full view\\n\"\n ]\n }\n ],\n \"source\": [\n \"if os.path.exists('snake_game.html'):\\n\",\n \" display(IFrame(src='snake_game.html', width=600, height=800))\\n\",\n \" print(\\\"\\\\n💡 Open 'snake_game.html' in your browser for full view\\\")\\n\",\n \"else:\\n\",\n \" print(\\\"⚠️ Game not created. Printing response from the model:\\\")\\n\",\n \" print(f\\\"\\\\n{response.choices[0].message.content}\\\")\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"print(f\\\"\\\\n{response.choices[0].message.content}\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Cleanup\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"filesystem_mcp.close()\\n\",\n \"print(\\\"✓ Done!\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"---\\n\",\n \"\\n\",\n \"## That's It!\\n\",\n \"\\n\",\n \"In just a few lines of code, you had an AI:\\n\",\n \"- ✅ Generate a complete Snake game from scratch\\n\",\n \"- ✅ Save it to disk using MCP filesystem tools\\n\",\n \"- ✅ Display it playable right in the notebook\\n\",\n \"\\n\",\n \"**Try it yourself:**\\n\",\n \"- Ask for a different game (Pong, Tetris, etc.)\\n\",\n \"- Add difficulty levels or speed settings\\n\",\n \"- Request different color themes\\n\",\n \"- Try with different models (swap the commented line)\"\n ]\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3 (ipykernel)\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.13.3\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 4\n}\n" }, { "path": "examples/agents/stock_dashboard.html", "content": "\n\n\n \n \n Stock Market Movers - November 10, 2025\n \n\n\n
\n
\n

Stock Market Movers - November 10, 2025

\n

Top gainers snapshot, styled for a professional financial look.

\n
\n\n
\n
\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
TickerCompanyPrice (USD)% Change
GLTOGalecto, Inc.17.25+248.49%
COGTCogent Biosciences, Inc.32.46+119.03%
NVTSNavitas Semiconductor Corporation9.60+22.45%
XPEVXPeng Inc.26.04+16.15%
SEDGSolarEdge Technologies, Inc.45.38+13.45%
\n
\n
\n As of Nov 10, 2025\n Source: Yahoo Finance — Top Gainers\n
\n
\n
\n\n" }, { "path": "examples/agents/stock_market_dashboard.html", "content": "\n\n\n \n \n Stock Market Movers - 2025-11-10\n \n\n\n
\n

Stock Market Movers - 2025-11-10

\n

Market Sentiment: Bullish

\n
\n
\n
\n

Top Gainers

\n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n \n
Company NameTickerCurrent Price% ChangeReason
Cogent Biosciences, Inc.COGT$32.46+119.03%Positive clinical trial result
Navitas Semiconductor CorporationNVTS$9.60+22.45%Strong quarterly earnings
Opendoor Technologies Inc.OPEN$7.99+21.77%Acquisition news
\n
\n
\n

Market News

\n
    \n
  • Sony raises profit forecast after earnings beat, boosted by Music and Imaging divisions
    • \n
  • U.S. markets rally as investors anticipate FOMC meeting outcomes
    • \n
  • European markets gain on strong manufacturing data
    • \n
\n
\n
\n
\n

Sources: Yahoo Finance, CNBC

\n

Timestamp: 2025-11-10

\n

Disclaimer: This is for informational purposes only, not financial advice.

\n
\n\n" }, { "path": "examples/agents/stock_market_mini_tracker.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Stock Market Tracker - Multi-step Agent in a single prompt\\n\",\n \"\\n\",\n \"This notebook shows how **easy** it is to build a multi-step agent using a single prompt with aisuite + MCP tools.\\n\",\n \"\\n\",\n \"**What it does**: 1. Fetches latest stock market data, 2. Analyzes content and 3. Creates an HTML dashboard.\\n\",\n \"\\n\",\n \"**Requirements**: `OPENAI_API_KEY` in your `.env` file\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Setup\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 1,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"✓ Ready!\\n\"\n ]\n }\n ],\n \"source\": [\n \"import os\\n\",\n \"from datetime import datetime\\n\",\n \"from dotenv import load_dotenv\\n\",\n \"import aisuite as ai\\n\",\n \"from aisuite.mcp import MCPClient\\n\",\n \"\\n\",\n \"load_dotenv()\\n\",\n \"\\n\",\n \"# Initialize MCP servers for web fetching and file writing\\n\",\n \"fetch_mcp = MCPClient(command=\\\"uvx\\\", args=[\\\"mcp-server-fetch\\\"])\\n\",\n \"filesystem_mcp = MCPClient(\\n\",\n \" command=\\\"npx\\\",\\n\",\n \" args=[\\\"-y\\\", \\\"@modelcontextprotocol/server-filesystem\\\", os.getcwd()]\\n\",\n \")\\n\",\n \"\\n\",\n \"print(\\\"✓ Ready!\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Craft instructions\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 2,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"✓ Agent instructions defined\\n\"\n ]\n }\n ],\n \"source\": [\n \"# Define what you want the agent to do\\n\",\n \"prompt = f\\\"\\\"\\\"Create a stock market dashboard for {datetime.now().strftime('%Y-%m-%d')}.\\n\",\n \"\\n\",\n \"**⚠️ EXECUTION RULES:**\\n\",\n \"- Execute ALL tools silently (no intermediate text responses)\\n\",\n \"- Write the HTML file FIRST, then provide summary\\n\",\n \"- If you respond with text before writing the file, the loop stops!\\n\",\n \"\\n\",\n \"**TASK:**\\n\",\n \"\\n\",\n \"1. **Fetch Stock Data**\\n\",\n \" - Get top gainers from: https://finance.yahoo.com/markets/stocks/gainers/\\n\",\n \" - Extract: ticker, company name, price, % change\\n\",\n \" - Get 3-5 stocks\\n\",\n \"\\n\",\n \"2. **Create Professional HTML Dashboard**\\n\",\n \" - Title: \\\"Stock Market Movers - {datetime.now().strftime('%B %d, %Y')}\\\"\\n\",\n \" - Display top gainers in a clean table or card layout\\n\",\n \" - Use green color for positive % changes\\n\",\n \" - Add professional styling:\\n\",\n \" * Modern typography\\n\",\n \" * Shadows and borders\\n\",\n \" * Responsive design\\n\",\n \" * Clean spacing\\n\",\n \" - Include timestamp and data source\\n\",\n \"\\n\",\n \"3. **Save File**\\n\",\n \" - Use write_file to save as 'stock_dashboard.html'\\n\",\n \" - Confirm successful write\\n\",\n \"\\n\",\n \"4. **Respond**\\n\",\n \" - ONLY after file is written, provide a brief summary\\n\",\n \" - Include: number of stocks, top 3 gainers with % changes\\n\",\n \"\\n\",\n \"Make it look like a professional financial dashboard!\\\"\\\"\\\"\\n\",\n \"\\n\",\n \"print(\\\"✓ Agent instructions defined\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Run Agent with MCP tools.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 3,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"\\n\",\n \"============================================================\\n\",\n \"✓ DASHBOARD CREATED!\\n\",\n \"============================================================\\n\",\n \"\\n\",\n \"The stock market dashboard for November 11, 2025, has been successfully created and saved as 'stock_dashboard.html'. It displays the top 5 stock gainers with a professional layout. Here are the top 3 gainers:\\n\",\n \"\\n\",\n \"1. **Cogent Biosciences, Inc. (COGT)** - +119.03%\\n\",\n \"2. **Navitas Semiconductor Corporation (NVTS)** - +22.45%\\n\",\n \"3. **Opendoor Technologies Inc. (OPEN)** - +21.77%\\n\",\n \"\\n\",\n \"The dashboard includes modern typography, responsive design, and highlights positive percentage changes in green.\\n\"\n ]\n }\n ],\n \"source\": [\n \"# Create agent and run it\\n\",\n \"client = ai.Client()\\n\",\n \"tools = fetch_mcp.get_callable_tools() + filesystem_mcp.get_callable_tools()\\n\",\n \"\\n\",\n \"response = client.chat.completions.create(\\n\",\n \" model=\\\"openai:gpt-4o\\\",\\n\",\n \" messages=[{\\\"role\\\": \\\"user\\\", \\\"content\\\": prompt}],\\n\",\n \" tools=tools,\\n\",\n \" max_turns=20\\n\",\n \")\\n\",\n \"\\n\",\n \"print(\\\"\\\\n\\\" + \\\"=\\\"*60)\\n\",\n \"print(\\\"✓ DASHBOARD CREATED!\\\")\\n\",\n \"print(\\\"=\\\"*60)\\n\",\n \"print(f\\\"\\\\n{response.choices[0].message.content}\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## View Dashboard\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 4,\n \"metadata\": {},\n \"outputs\": [\n {\n \"data\": {\n \"text/html\": [\n \"\\n\",\n \" \\n\",\n \" \"\n ],\n \"text/plain\": [\n \"\"\n ]\n },\n \"metadata\": {},\n \"output_type\": \"display_data\"\n },\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"\\n\",\n \"💡 Open 'stock_dashboard.html' in your browser for full view\\n\"\n ]\n }\n ],\n \"source\": [\n \"from IPython.display import IFrame, display\\n\",\n \"\\n\",\n \"if os.path.exists('stock_dashboard.html'):\\n\",\n \" display(IFrame(src='stock_dashboard.html', width=900, height=600))\\n\",\n \" print(\\\"\\\\n💡 Open 'stock_dashboard.html' in your browser for full view\\\")\\n\",\n \"else:\\n\",\n \" print(\\\"⚠️ Dashboard not created - check the output above\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Cleanup\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"fetch_mcp.close()\\n\",\n \"filesystem_mcp.close()\\n\",\n \"print(\\\"✓ Done! That's how easy it is to build an agent with aisuite.\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"---\\n\",\n \"\\n\",\n \"## That's It!\\n\",\n \"\\n\",\n \"In just **few lines of code**, you built an autonomous agent that:\\n\",\n \"- ✅ Fetches live stock data from the web\\n\",\n \"- ✅ Creates a beautiful HTML dashboard\\n\",\n \"- ✅ Saves it to disk\\n\",\n \"\\n\",\n \"**Try it yourself**:\\n\",\n \"- Modify the prompt to fetch TOP losers and gainers.\\n\",\n \"- Modify to fetch data across multiple days.\\n\",\n \"- Modify to fetch headlines from any Finance News website to explain why stocks rose or fell. (NOTE: Many websites do not like automated calls - be mindful of their policies.)\\n\",\n \"- Try asking Agent to plot a graph with the data in the HTML page.\\n\",\n \"- Add more MCP tools (memory, search, etc.)\\n\",\n \"\\n\",\n \"Remember to **increase max_turns** to a higher number if you increase the complexity of the tasks.\\n\",\n \"\\n\",\n \"**Learn more**: Check out other notebooks in `examples/agents/`\"\n ]\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3 (ipykernel)\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.13.3\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 4\n}\n" }, { "path": "examples/agents/stock_market_tracker.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Stock Market Tracker - Latest Market Movers Dashboard\\n\",\n \"\\n\",\n \"This notebook demonstrates an autonomous research agent that:\\n\",\n \"- Fetches latest stock market news and data\\n\",\n \"- Identifies stocks that rose and fell today\\n\",\n \"- Creates a beautiful HTML dashboard with market analysis\\n\",\n \"\\n\",\n \"## Requirements\\n\",\n \"\\n\",\n \"1. **API Keys** (set in `.env` file):\\n\",\n \" - `ANTHROPIC_API_KEY` - Get from https://console.anthropic.com/\\n\",\n \"\\n\",\n \"2. **MCP Servers** (install if needed):\\n\",\n \" ```bash\\n\",\n \" # Fetch server for web research\\n\",\n \" pip install mcp-server-fetch\\n\",\n \" \\n\",\n \" # Filesystem server for saving HTML\\n\",\n \" npm install -g @modelcontextprotocol/server-filesystem\\n\",\n \" ```\\n\",\n \"\\n\",\n \"3. **Python Packages**:\\n\",\n \" ```bash\\n\",\n \" pip install 'aisuite[anthropic,mcp]' python-dotenv\\n\",\n \" ```\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Setup\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 1,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"✓ Environment configured\\n\",\n \"✓ Working directory: /Users/rohit/fleet/leclerc/aisuite-prs/aisuite-main/aisuite/examples/agents\\n\",\n \"✓ Today's date: 2025-11-10\\n\",\n \"\\n\",\n \"Ready to track the markets!\\n\"\n ]\n }\n ],\n \"source\": [\n \"import os\\n\",\n \"import json\\n\",\n \"from pathlib import Path\\n\",\n \"from dotenv import load_dotenv\\n\",\n \"from datetime import datetime\\n\",\n \"import aisuite as ai\\n\",\n \"from aisuite.mcp import MCPClient\\n\",\n \"\\n\",\n \"# Load environment variables\\n\",\n \"load_dotenv()\\n\",\n \"\\n\",\n \"# Verify required API keys\\n\",\n \"if not os.getenv(\\\"ANTHROPIC_API_KEY\\\"):\\n\",\n \" raise ValueError(\\n\",\n \" \\\"Missing ANTHROPIC_API_KEY\\\\n\\\"\\n\",\n \" \\\"Please add it to your .env file\\\"\\n\",\n \" )\\n\",\n \"\\n\",\n \"print(\\\"✓ Environment configured\\\")\\n\",\n \"print(f\\\"✓ Working directory: {os.getcwd()}\\\")\\n\",\n \"print(f\\\"✓ Today's date: {datetime.now().strftime('%Y-%m-%d')}\\\")\\n\",\n \"print(\\\"\\\\nReady to track the markets!\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Initialize MCP Servers\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 2,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"✓ MCP servers initialized\\n\",\n \"\\n\",\n \"Available tools:\\n\",\n \" - Fetch: ['fetch']\\n\",\n \" - Filesystem: ['read_file', 'read_text_file', 'read_media_file', 'read_multiple_files', 'write_file', 'edit_file', 'create_directory', 'list_directory', 'list_directory_with_sizes', 'directory_tree', 'move_file', 'search_files', 'get_file_info', 'list_allowed_directories']\\n\"\n ]\n }\n ],\n \"source\": [\n \"# Fetch MCP Server - for web research\\n\",\n \"fetch_mcp = MCPClient(\\n\",\n \" command=\\\"uvx\\\",\\n\",\n \" args=[\\\"mcp-server-fetch\\\"],\\n\",\n \" name=\\\"fetch\\\"\\n\",\n \")\\n\",\n \"\\n\",\n \"# Filesystem MCP Server - for saving HTML files\\n\",\n \"filesystem_mcp = MCPClient(\\n\",\n \" command=\\\"npx\\\",\\n\",\n \" args=[\\\"-y\\\", \\\"@modelcontextprotocol/server-filesystem\\\", os.getcwd()],\\n\",\n \" name=\\\"filesystem\\\"\\n\",\n \")\\n\",\n \"\\n\",\n \"print(\\\"✓ MCP servers initialized\\\")\\n\",\n \"print(f\\\"\\\\nAvailable tools:\\\")\\n\",\n \"print(f\\\" - Fetch: {[tool['name'] for tool in fetch_mcp.list_tools()]}\\\")\\n\",\n \"print(f\\\" - Filesystem: {[tool['name'] for tool in filesystem_mcp.list_tools()]}\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Generate Market Movers Dashboard\\n\",\n \"\\n\",\n \"Watch the AI agent:\\n\",\n \"1. Fetch latest stock market news and data\\n\",\n \"2. Identify top gainers and losers\\n\",\n \"3. Create a comprehensive HTML dashboard\\n\",\n \"4. **Save the HTML file FIRST** (critical!)\\n\",\n \"5. Provide summary\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 3,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"📈 Fetching latest stock market data...\\n\",\n \"\\n\",\n \"============================================================\\n\",\n \"\\n\",\n \"============================================================\\n\",\n \"✓ STOCK MARKET DASHBOARD COMPLETE!\\n\",\n \"============================================================\\n\",\n \"\\n\",\n \"The stock market dashboard for November 10, 2025, has been successfully created and saved as 'stock_market_dashboard.html'. Here's a summary of the dashboard:\\n\",\n \"\\n\",\n \"- **Number of Stocks Analyzed**: Multiple stocks, focusing on the top 3 gainers.\\n\",\n \"- **Top 3 Gainers**:\\n\",\n \" 1. **Cogent Biosciences, Inc. (COGT)**: Price $32.46, Change +119.03%\\n\",\n \" 2. **Navitas Semiconductor Corporation (NVTS)**: Price $9.60, Change +22.45%\\n\",\n \" 3. **Opendoor Technologies Inc. (OPEN)**: Price $7.99, Change +21.77%\\n\",\n \"- **Overall Market Sentiment**: Bullish\\n\",\n \"- **Key Market News**:\\n\",\n \" - Sony raises profit forecast after earnings beat\\n\",\n \" - U.S. markets rally in anticipation of FOMC meeting outcomes\\n\",\n \" - European markets gain on strong manufacturing data\\n\",\n \"\\n\",\n \"The dashboard provides a clean, professional, and responsive layout suitable for financial analysis.\\n\"\n ]\n }\n ],\n \"source\": [\n \"# Initialize aisuite client\\n\",\n \"client = ai.Client()\\n\",\n \"\\n\",\n \"# Combine all available tools\\n\",\n \"all_tools = fetch_mcp.get_callable_tools() + filesystem_mcp.get_callable_tools()\\n\",\n \"\\n\",\n \"print(\\\"📈 Fetching latest stock market data...\\\\n\\\")\\n\",\n \"print(\\\"=\\\" * 60)\\n\",\n \"\\n\",\n \"# Create comprehensive research prompt\\n\",\n \"response = client.chat.completions.create(\\n\",\n \" model=\\\"openai:gpt-4o\\\",\\n\",\n \" messages=[{\\n\",\n \" \\\"role\\\": \\\"user\\\",\\n\",\n \" \\\"content\\\": f\\\"\\\"\\\"Create a stock market movers dashboard for today ({datetime.now().strftime('%Y-%m-%d')}).\\n\",\n \"\\n\",\n \"**⚠️ CRITICAL EXECUTION RULES - READ FIRST:**\\n\",\n \"1. DO NOT provide ANY text responses while executing tools\\n\",\n \"2. DO NOT say things like \\\"Now let me...\\\", \\\"I'll create...\\\", or \\\"Let me gather...\\\"\\n\",\n \"3. Execute ALL tools SILENTLY - fetch URLs, write files, etc.\\n\",\n \"4. Your FIRST and ONLY text response must come AFTER the HTML file is successfully written\\n\",\n \"5. If you respond with text before calling write_file, the tool execution loop will STOP and the file will NOT be created!\\n\",\n \"\\n\",\n \"**REMEMBER**: Research → Write HTML → THEN (and only then) respond with summary!\\n\",\n \"\\n\",\n \"---\\n\",\n \"\\n\",\n \"Steps to execute SILENTLY (no text responses until step 6):\\n\",\n \"\\n\",\n \"1. **Fetch latest market data** from these sources:\\n\",\n \" - Yahoo Finance market movers: https://finance.yahoo.com/markets/stocks/gainers/\\n\",\n \" - MarketWatch: https://www.marketwatch.com/\\n\",\n \" - Or any other reliable financial news source\\n\",\n \" \\n\",\n \" Extract:\\n\",\n \" - Top 3 stocks that rose today (gainers)\\n\",\n \" - For each stock: ticker symbol, company name, price, % change\\n\",\n \" - Major market indices (S&P 500, Dow, NASDAQ) if available\\n\",\n \" - Key market news/headlines\\n\",\n \"\\n\",\n \"2. **Create a beautiful HTML dashboard** with:\\n\",\n \" \\n\",\n \" **Header Section**:\\n\",\n \" - Title: \\\"Stock Market Movers - [Today's Date]\\\"\\n\",\n \" - Market sentiment indicator (bullish/bearish/neutral)\\n\",\n \" \\n\",\n \" **Top Gainers Section**:\\n\",\n \" - Table or cards showing:\\n\",\n \" * Company name and ticker\\n\",\n \" * Current price\\n\",\n \" * % change (in green)\\n\",\n \" * Brief reason for rise if known\\n\",\n \" \\n\",\n \" **Market News Section**:\\n\",\n \" - 3-5 key headlines impacting the market\\n\",\n \" \\n\",\n \" **Footer**:\\n\",\n \" - Data sources cited\\n\",\n \" - Timestamp\\n\",\n \" - Disclaimer: \\\"This is for informational purposes only, not financial advice\\\"\\n\",\n \"\\n\",\n \"4. **Styling requirements**:\\n\",\n \" - Professional financial dashboard aesthetic\\n\",\n \" - Color coding: GREEN for gains, RED for losses\\n\",\n \" - Clean table or card-based layout\\n\",\n \" - Responsive design\\n\",\n \" - Modern typography\\n\",\n \" - Use shadows and borders for visual separation.\\n\",\n \" \\n\",\n \"5. **SAVE HTML FILE (still no text response yet!)**:\\n\",\n \" - Use write_file to save as 'stock_market_dashboard.html'\\n\",\n \" - Wait for confirmation that write succeeded\\n\",\n \" \\n\",\n \"6. **NOW you can respond with text** - provide a summary with:\\n\",\n \" - Confirmation that the HTML file was created\\n\",\n \" - Number of stocks analyzed\\n\",\n \" - Top 3 gainers with % changes\\n\",\n \" - Optionally, Top 3 losers with % changes\\n\",\n \" - Overall market sentiment\\n\",\n \"\\n\",\n \"Make it look like a professional Bloomberg/Yahoo Finance dashboard!\\\"\\\"\\\"\\n\",\n \" }],\\n\",\n \" tools=all_tools,\\n\",\n \" max_turns=30\\n\",\n \")\\n\",\n \"\\n\",\n \"print(\\\"\\\\n\\\" + \\\"=\\\" * 60)\\n\",\n \"print(\\\"✓ STOCK MARKET DASHBOARD COMPLETE!\\\")\\n\",\n \"print(\\\"=\\\" * 60)\\n\",\n \"print(f\\\"\\\\n{response.choices[0].message.content}\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Verify File Was Created\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 4,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"✅ File created successfully!\\n\",\n \" - Filename: stock_market_dashboard.html\\n\",\n \" - Size: 4,231 bytes\\n\",\n \" - Location: /Users/rohit/fleet/leclerc/aisuite-prs/aisuite-main/aisuite/examples/agents/stock_market_dashboard.html\\n\"\n ]\n }\n ],\n \"source\": [\n \"filename = \\\"stock_market_dashboard.html\\\"\\n\",\n \"\\n\",\n \"if os.path.exists(filename):\\n\",\n \" file_size = os.path.getsize(filename)\\n\",\n \" print(f\\\"✅ File created successfully!\\\")\\n\",\n \" print(f\\\" - Filename: {filename}\\\")\\n\",\n \" print(f\\\" - Size: {file_size:,} bytes\\\")\\n\",\n \" print(f\\\" - Location: {os.path.abspath(filename)}\\\")\\n\",\n \"else:\\n\",\n \" print(f\\\"❌ ERROR: File was not created!\\\")\\n\",\n \" print(f\\\" Expected: {filename}\\\")\\n\",\n \" print(f\\\"\\\\n The agent may have stopped before writing the file.\\\")\\n\",\n \" print(f\\\" Try re-running with a higher max_turns value.\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## View the Generated Dashboard\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 5,\n \"metadata\": {},\n \"outputs\": [\n {\n \"data\": {\n \"text/html\": [\n \"\\n\",\n \" \\n\",\n \" \"\n ],\n \"text/plain\": [\n \"\"\n ]\n },\n \"metadata\": {},\n \"output_type\": \"display_data\"\n },\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"\\n\",\n \"💡 Tip: Open 'stock_market_dashboard.html' in your browser for full-screen viewing!\\n\"\n ]\n }\n ],\n \"source\": [\n \"from IPython.display import IFrame, display\\n\",\n \"\\n\",\n \"if os.path.exists(filename):\\n\",\n \" display(IFrame(src=filename, width=900, height=800))\\n\",\n \" print(f\\\"\\\\n💡 Tip: Open '{filename}' in your browser for full-screen viewing!\\\")\\n\",\n \"else:\\n\",\n \" print(f\\\"⚠️ File not found: {filename}\\\")\\n\",\n \" print(\\\"The dashboard was not created. Check the error above.\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Optional: View Tool Execution History\\n\",\n \"\\n\",\n \"See what sources the agent consulted and when it wrote the file:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 6,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"Tool Execution History:\\n\",\n \"\\n\",\n \"============================================================\\n\",\n \"\\n\",\n \"[1] 🌐 Fetched: https://finance.yahoo.com/markets/stocks/gainers/...\\n\",\n \"\\n\",\n \"[1] 🌐 Fetched: https://www.marketwatch.com/...\\n\"\n ]\n },\n {\n \"ename\": \"AttributeError\",\n \"evalue\": \"'dict' object has no attribute 'role'\",\n \"output_type\": \"error\",\n \"traceback\": [\n \"\\u001b[0;31m---------------------------------------------------------------------------\\u001b[0m\",\n \"\\u001b[0;31mAttributeError\\u001b[0m Traceback (most recent call last)\",\n \"Cell \\u001b[0;32mIn[6], line 8\\u001b[0m\\n\\u001b[1;32m 5\\u001b[0m write_count \\u001b[38;5;241m=\\u001b[39m \\u001b[38;5;241m0\\u001b[39m\\n\\u001b[1;32m 7\\u001b[0m \\u001b[38;5;28;01mfor\\u001b[39;00m i, msg \\u001b[38;5;129;01min\\u001b[39;00m \\u001b[38;5;28menumerate\\u001b[39m(response\\u001b[38;5;241m.\\u001b[39mchoices[\\u001b[38;5;241m0\\u001b[39m]\\u001b[38;5;241m.\\u001b[39mintermediate_messages, \\u001b[38;5;241m1\\u001b[39m):\\n\\u001b[0;32m----> 8\\u001b[0m \\u001b[38;5;28;01mif\\u001b[39;00m \\u001b[43mmsg\\u001b[49m\\u001b[38;5;241;43m.\\u001b[39;49m\\u001b[43mrole\\u001b[49m \\u001b[38;5;241m==\\u001b[39m \\u001b[38;5;124m\\\"\\u001b[39m\\u001b[38;5;124massistant\\u001b[39m\\u001b[38;5;124m\\\"\\u001b[39m \\u001b[38;5;129;01mand\\u001b[39;00m \\u001b[38;5;28mhasattr\\u001b[39m(msg, \\u001b[38;5;124m'\\u001b[39m\\u001b[38;5;124mtool_calls\\u001b[39m\\u001b[38;5;124m'\\u001b[39m) \\u001b[38;5;129;01mand\\u001b[39;00m msg\\u001b[38;5;241m.\\u001b[39mtool_calls:\\n\\u001b[1;32m 9\\u001b[0m \\u001b[38;5;28;01mfor\\u001b[39;00m tool_call \\u001b[38;5;129;01min\\u001b[39;00m msg\\u001b[38;5;241m.\\u001b[39mtool_calls:\\n\\u001b[1;32m 10\\u001b[0m \\u001b[38;5;28;01mif\\u001b[39;00m tool_call\\u001b[38;5;241m.\\u001b[39mfunction\\u001b[38;5;241m.\\u001b[39mname \\u001b[38;5;241m==\\u001b[39m \\u001b[38;5;124m'\\u001b[39m\\u001b[38;5;124mfetch\\u001b[39m\\u001b[38;5;124m'\\u001b[39m:\\n\",\n \"\\u001b[0;31mAttributeError\\u001b[0m: 'dict' object has no attribute 'role'\"\n ]\n }\n ],\n \"source\": [\n \"print(\\\"Tool Execution History:\\\\n\\\")\\n\",\n \"print(\\\"=\\\" * 60)\\n\",\n \"\\n\",\n \"fetch_count = 0\\n\",\n \"write_count = 0\\n\",\n \"\\n\",\n \"for i, msg in enumerate(response.choices[0].intermediate_messages, 1):\\n\",\n \" if msg.role == \\\"assistant\\\" and hasattr(msg, 'tool_calls') and msg.tool_calls:\\n\",\n \" for tool_call in msg.tool_calls:\\n\",\n \" if tool_call.function.name == 'fetch':\\n\",\n \" fetch_count += 1\\n\",\n \" args = json.loads(tool_call.function.arguments)\\n\",\n \" url = args.get('url', 'N/A')\\n\",\n \" print(f\\\"\\\\n[{i}] 🌐 Fetched: {url[:80]}...\\\")\\n\",\n \" elif tool_call.function.name == 'write_file':\\n\",\n \" write_count += 1\\n\",\n \" args = json.loads(tool_call.function.arguments)\\n\",\n \" path = args.get('path', 'N/A')\\n\",\n \" content_size = len(args.get('contents', ''))\\n\",\n \" print(f\\\"\\\\n[{i}] 💾 WROTE FILE: {path} ({content_size:,} bytes)\\\")\\n\",\n \"\\n\",\n \"print(f\\\"\\\\n{'=' * 60}\\\")\\n\",\n \"print(f\\\"Total web fetches: {fetch_count}\\\")\\n\",\n \"print(f\\\"Total file writes: {write_count}\\\")\\n\",\n \"\\n\",\n \"if write_count == 0:\\n\",\n \" print(\\\"\\\\n⚠️ WARNING: No file writes detected!\\\")\\n\",\n \" print(\\\" The agent did not call write_file.\\\")\\n\",\n \"elif write_count == 1:\\n\",\n \" print(\\\"\\\\n✅ Perfect! File was written exactly once.\\\")\\n\",\n \"else:\\n\",\n \" print(f\\\"\\\\n⚠️ NOTE: File was written {write_count} times (may have been overwritten)\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Market Refresh\\n\",\n \"\\n\",\n \"Markets change throughout the day! Re-run the notebook to get fresh data.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"print(f\\\"Dashboard generated at: {datetime.now().strftime('%Y-%m-%d %H:%M:%S')}\\\")\\n\",\n \"print(\\\"\\\\n💡 Tip: Re-run the 'Generate Market Movers Dashboard' cell for updated data!\\\")\\n\",\n \"print(\\\"\\\\n⚠️ Disclaimer: This is for informational purposes only.\\\")\\n\",\n \"print(\\\" This is not financial advice. Always do your own research.\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Cleanup\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Close MCP connections\\n\",\n \"fetch_mcp.close()\\n\",\n \"filesystem_mcp.close()\\n\",\n \"\\n\",\n \"print(\\\"✓ MCP servers closed\\\")\\n\",\n \"if os.path.exists(filename):\\n\",\n \" print(f\\\"✓ Your dashboard is saved as: {filename}\\\")\\n\",\n \"print(\\\"\\\\n📊 Happy trading! (Remember: not financial advice!)\\\")\"\n ]\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3 (ipykernel)\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.13.3\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 4\n}\n" }, { "path": "examples/agents/world_weather_dashboard.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# World Weather Dashboard - AI-Generated with Tailwind CSS\\n\",\n \"\\n\",\n \"This notebook demonstrates an AI agent that fetches live weather data and creates a beautiful dashboard.\\n\",\n \"\\n\",\n \"**What it does**:\\n\",\n \"1. Fetches current weather for major world capitals from wttr.in\\n\",\n \"2. Creates a responsive Tailwind CSS dashboard\\n\",\n \"3. Displays weather cards with temperature, conditions, and more\\n\",\n \"\\n\",\n \"**Requirements**:\\n\",\n \"- `OPENAI_API_KEY` or `ANTHROPIC_API_KEY` in your `.env` file\\n\",\n \"- MCP servers: `pip install mcp-server-fetch` and `npm install -g @modelcontextprotocol/server-filesystem`\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Setup\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import os\\n\",\n \"from datetime import datetime\\n\",\n \"from dotenv import load_dotenv\\n\",\n \"import aisuite as ai\\n\",\n \"from aisuite.mcp import MCPClient\\n\",\n \"\\n\",\n \"load_dotenv()\\n\",\n \"\\n\",\n \"# Initialize MCP servers\\n\",\n \"fetch_mcp = MCPClient(command=\\\"uvx\\\", args=[\\\"mcp-server-fetch\\\"])\\n\",\n \"filesystem_mcp = MCPClient(\\n\",\n \" command=\\\"npx\\\",\\n\",\n \" args=[\\\"-y\\\", \\\"@modelcontextprotocol/server-filesystem\\\", os.getcwd()]\\n\",\n \")\\n\",\n \"\\n\",\n \"print(\\\"✓ Ready!\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Craft Instructions\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Cities to fetch weather for\\n\",\n \"capitals = [\\n\",\n \" \\\"London\\\", \\\"Paris\\\", \\\"Tokyo\\\", \\\"New York\\\", \\\"Sydney\\\", \\\"São Paulo\\\"\\n\",\n \"]\\n\",\n \"\\n\",\n \"prompt = f\\\"\\\"\\\"Create a world weather dashboard for these capitals: {', '.join(capitals)}.\\n\",\n \"\\n\",\n \"**EXECUTION RULES:**\\n\",\n \"- Execute ALL tools silently (no intermediate text responses)\\n\",\n \"- Write the HTML file FIRST, then provide a brief summary\\n\",\n \"\\n\",\n \"**TASK:**\\n\",\n \"1. **Fetch Weather Data**\\n\",\n \" - Use wttr.in for each city: `https://wttr.in/CityName?format=j1`\\n\",\n \" - This returns JSON with current conditions\\n\",\n \" - Extract: temperature (°C), weather description, humidity, wind speed\\n\",\n \" - Note: wttr.in explicitly allows automated access\\n\",\n \"\\n\",\n \"2. **Create Single page HTML Dashboard with Tailwind CSS** \\n\",\n \" Each weather card should have:\\n\",\n \" - City name (bold, large)\\n\",\n \" - Weather emoji (☀️ sunny, 🌧️ rain, ☁️ cloudy, etc.)\\n\",\n \" - Temperature in large font\\n\",\n \" - Weather description\\n\",\n \" - Humidity and wind as smaller details\\n\",\n \" - Background color based on temperature:\\n\",\n \" * Cold (<10°C): blue tones\\n\",\n \" * Mild (10-25°C): green/teal tones \\n\",\n \" * Warm (25-35°C): orange/yellow tones\\n\",\n \" * Hot (>35°C): red tones\\n\",\n \"\\n\",\n \"3. **Styling Requirements**\\n\",\n \" - Use Tailwind CSS classes\\n\",\n \" - Rounded cards with shadows\\n\",\n \" - Responsive grid (2 cols mobile, 3 tablet, 5 desktop)\\n\",\n \" - Clean, modern design\\n\",\n \" - White text on colored backgrounds\\n\",\n \" - Smooth gradients\\n\",\n \"\\n\",\n \"4. **Save File**\\n\",\n \" - Use write_file to save as 'weather_dashboard.html'\\n\",\n \"\\n\",\n \"5. **Respond with summary**\\n\",\n \" - ONLY after file is written\\n\",\n \" - List hottest and coldest cities\\n\",\n \" - Any interesting weather patterns\\n\",\n \"\\\"\\\"\\\"\\n\",\n \"\\n\",\n \"print(\\\"✓ Agent instructions defined\\\")\\n\",\n \"print(f\\\"✓ Will fetch weather for {len(capitals)} cities\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Run Agent\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"client = ai.Client()\\n\",\n \"tools = fetch_mcp.get_callable_tools() + filesystem_mcp.get_callable_tools()\\n\",\n \"\\n\",\n \"# Choose your model (uncomment one):\\n\",\n \"model = \\\"openai:gpt-5.1\\\"\\n\",\n \"# model = \\\"anthropic:claude-sonnet-4-5\\\"\\n\",\n \"\\n\",\n \"print(\\\"Fetching weather for world capitals...\\\\n\\\")\\n\",\n \"print(\\\"(This may take a minute or more\\\\n\\\")\\n\",\n \"\\n\",\n \"response = client.chat.completions.create(\\n\",\n \" model=model,\\n\",\n \" messages=[{\\\"role\\\": \\\"user\\\", \\\"content\\\": prompt}],\\n\",\n \" tools=tools,\\n\",\n \" max_turns=20\\n\",\n \")\\n\",\n \"\\n\",\n \"print(\\\"✓ WEATHER DASHBOARD CREATED!\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## View the Dashboard\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"from IPython.display import IFrame, display\\n\",\n \"\\n\",\n \"if os.path.exists('weather_dashboard.html'):\\n\",\n \" display(IFrame(src='weather_dashboard.html', width=950, height=700))\\n\",\n \" print(\\\"\\\\n💡 Open 'weather_dashboard.html' in your browser for full view\\\")\\n\",\n \"else:\\n\",\n \" print(\\\"⚠️ Dashboard not created\\\")\\n\",\n \"print(f\\\"\\\\n{response.choices[0].message.content}\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Cleanup\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"fetch_mcp.close()\\n\",\n \"filesystem_mcp.close()\\n\",\n \"print(\\\"✓ Done!\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"---\\n\",\n \"\\n\",\n \"## That's It!\\n\",\n \"\\n\",\n \"You just built an AI agent that:\\n\",\n \"- ✅ Fetched live weather data for few world capitals\\n\",\n \"- ✅ Created a beautiful Tailwind CSS dashboard\\n\",\n \"- ✅ Color-coded temperatures for visual impact\\n\",\n \"\\n\",\n \"**Try it yourself:**\\n\",\n \"- Add more cities to the `capitals` list\\n\",\n \"- Request a dark mode version\\n\",\n \"- Add 3-day forecast for each city\\n\",\n \"- Include weather icons instead of emojis\\n\",\n \"- Add a world map visualization\\n\",\n \"\\n\",\n \"**About wttr.in:**\\n\",\n \"- Free weather service designed for terminal/API access\\n\",\n \"- No API key required\\n\",\n \"- Supports JSON format with `?format=j1`\\n\",\n \"- More info: https://github.com/chubin/wttr.in\\n\",\n \"\\n\",\n \"**Learn more**: Check out other notebooks in `examples/agents/`\"\n ]\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3 (ipykernel)\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.13.3\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 4\n}\n" }, { "path": "examples/aisuite_tool_abstraction.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": 8,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import json\\n\",\n \"import sys\\n\",\n \"from dotenv import load_dotenv, find_dotenv\\n\",\n \"import os\\n\",\n \"\\n\",\n \"sys.path.append('../../aisuite')\\n\",\n \"# Load from .env file if available\\n\",\n \"load_dotenv(find_dotenv())\\n\",\n \"os.environ['ALLOW_MULTI_TURN'] = 'true'\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Define the functions\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 9,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Mock tool functions.\\n\",\n \"def get_current_temperature(location: str, unit: str):\\n\",\n \" \\\"\\\"\\\"This is a short description of what the function does.\\n\",\n \"\\n\",\n \" This is a longer description that can span\\n\",\n \" multiple lines and provide more details.\\n\",\n \"\\n\",\n \" Args:\\n\",\n \" param1: Description of param1\\n\",\n \" param2: Description of param2\\n\",\n \" \\\"\\\"\\\"\\n\",\n \" return \\\"70\\\"\\n\",\n \"\\n\",\n \"def is_it_raining(location: str):\\n\",\n \" # Simulate fetching rain probability\\n\",\n \" return \\\"yes\\\"\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Call the model with tools\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 10,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"from aisuite import Client\\n\",\n \"\\n\",\n \"client = Client()\\n\",\n \"messages = [{\\n\",\n \" \\\"role\\\": \\\"user\\\",\\n\",\n \" \\\"content\\\": \\\"Can you plan a picnic for today afternoon in San Francisco? Check the temperature and if its raining.\\\"}]\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 11,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"--------- response from LLM ---------\\n\",\n \"ChatCompletion(id='chatcmpl-AvuR3w6M83nWHL9sIO23pgvD0E5PF', choices=[Choice(finish_reason='tool_calls', index=0, logprobs=None, message=ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_PVhrHTUQ2qY7FDyPkCT54phU', function=Function(arguments='{\\\\n\\\"location\\\": \\\"San Francisco\\\",\\\\n\\\"unit\\\": \\\"Fahrenheit\\\"\\\\n}', name='get_current_temperature'), type='function')]))], created=1738364997, model='gpt-4-0613', object='chat.completion', service_tier='default', system_fingerprint=None, usage=CompletionUsage(completion_tokens=24, prompt_tokens=110, total_tokens=134, completion_tokens_details=CompletionTokensDetails(accepted_prediction_tokens=0, audio_tokens=0, reasoning_tokens=0, rejected_prediction_tokens=0), prompt_tokens_details=PromptTokensDetails(audio_tokens=0, cached_tokens=0)))\\n\",\n \"Executing tool: get_current_temperature\\n\",\n \"--------- tool_message to send to LLM ---------\\n\",\n \"[{'role': 'tool', 'name': 'get_current_temperature', 'content': '\\\"70\\\"', 'tool_call_id': 'call_PVhrHTUQ2qY7FDyPkCT54phU'}]\\n\",\n \"--------- response from LLM ---------\\n\",\n \"ChatCompletion(id='chatcmpl-AvuR4hpiCgXFxRMNLg1Scv1UD2JlR', choices=[Choice(finish_reason='tool_calls', index=0, logprobs=None, message=ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_Fvy3JUhIbVqzV0Nb0QfSnWQC', function=Function(arguments='{\\\\n\\\"location\\\": \\\"San Francisco\\\"\\\\n}', name='is_it_raining'), type='function')]))], created=1738364998, model='gpt-4-0613', object='chat.completion', service_tier='default', system_fingerprint=None, usage=CompletionUsage(completion_tokens=18, prompt_tokens=145, total_tokens=163, completion_tokens_details=CompletionTokensDetails(accepted_prediction_tokens=0, audio_tokens=0, reasoning_tokens=0, rejected_prediction_tokens=0), prompt_tokens_details=PromptTokensDetails(audio_tokens=0, cached_tokens=0)))\\n\",\n \"Executing tool: is_it_raining\\n\",\n \"--------- tool_message to send to LLM ---------\\n\",\n \"[{'role': 'tool', 'name': 'is_it_raining', 'content': '\\\"yes\\\"', 'tool_call_id': 'call_Fvy3JUhIbVqzV0Nb0QfSnWQC'}]\\n\",\n \"--------- response from LLM ---------\\n\",\n \"ChatCompletion(id='chatcmpl-AvuR7puWSJLEpCNIZLLkF40gYJp3c', choices=[Choice(finish_reason='stop', index=0, logprobs=None, message=ChatCompletionMessage(content=\\\"I'm sorry, it seems like it will be raining this afternoon in San Francisco. You might want to plan your picnic for another day. Also the temperature is forecasted to be around 70 degrees Fahrenheit.\\\", refusal=None, role='assistant', audio=None, function_call=None, tool_calls=None))], created=1738365001, model='gpt-4-0613', object='chat.completion', service_tier='default', system_fingerprint=None, usage=CompletionUsage(completion_tokens=44, prompt_tokens=175, total_tokens=219, completion_tokens_details=CompletionTokensDetails(accepted_prediction_tokens=0, audio_tokens=0, reasoning_tokens=0, rejected_prediction_tokens=0), prompt_tokens_details=PromptTokensDetails(audio_tokens=0, cached_tokens=0)))\\n\",\n \"I'm sorry, it seems like it will be raining this afternoon in San Francisco. You might want to plan your picnic for another day. Also the temperature is forecasted to be around 70 degrees Fahrenheit.\\n\"\n ]\n }\n ],\n \"source\": [\n \"response = client.chat.completions.create(\\n\",\n \" model=\\\"openai:gpt-4\\\", messages=messages, tools=[get_current_temperature, is_it_raining], max_turns=4)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"response = client.chat.completions.create(\\n\",\n \" model=\\\"anthropic:claude-3-5-sonnet-20241022\\\", messages=messages, tools=[get_current_temperature, is_it_raining], max_turns=4)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"print(response)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 12,\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"[ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_PVhrHTUQ2qY7FDyPkCT54phU', function=Function(arguments='{\\\\n\\\"location\\\": \\\"San Francisco\\\",\\\\n\\\"unit\\\": \\\"Fahrenheit\\\"\\\\n}', name='get_current_temperature'), type='function')]),\\n\",\n \" {'content': '\\\"70\\\"',\\n\",\n \" 'name': 'get_current_temperature',\\n\",\n \" 'role': 'tool',\\n\",\n \" 'tool_call_id': 'call_PVhrHTUQ2qY7FDyPkCT54phU'},\\n\",\n \" ChatCompletionMessage(content=None, refusal=None, role='assistant', audio=None, function_call=None, tool_calls=[ChatCompletionMessageToolCall(id='call_Fvy3JUhIbVqzV0Nb0QfSnWQC', function=Function(arguments='{\\\\n\\\"location\\\": \\\"San Francisco\\\"\\\\n}', name='is_it_raining'), type='function')]),\\n\",\n \" {'content': '\\\"yes\\\"',\\n\",\n \" 'name': 'is_it_raining',\\n\",\n \" 'role': 'tool',\\n\",\n \" 'tool_call_id': 'call_Fvy3JUhIbVqzV0Nb0QfSnWQC'},\\n\",\n \" ChatCompletionMessage(content=\\\"I'm sorry, it seems like it will be raining this afternoon in San Francisco. You might want to plan your picnic for another day. Also the temperature is forecasted to be around 70 degrees Fahrenheit.\\\", refusal=None, role='assistant', audio=None, function_call=None, tool_calls=None)]\\n\"\n ]\n }\n ],\n \"source\": [\n \"from pprint import pprint \\n\",\n \"pprint(response.choices[0].intermediate_messages)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"from aisuite import Tools\\n\",\n \"tools = Tools(tools=[get_current_temperature, is_it_raining])\\n\",\n \"tools.tools()\\n\",\n \"# tools.add_description(\\\"is_it_raining\\\", \\\"Use this function to understand if it is going to rain or not\\\")\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"messages = append(messages, response.choices[0].intermediate_messages)\\n\"\n ]\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3 (ipykernel)\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.12.8\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 4\n}\n" }, { "path": "examples/asr_example.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"id\": \"5a7a0ca2\",\n \"metadata\": {},\n \"source\": [\n \"# ASR Example - Basic Transcription Interface\\n\",\n \"\\n\",\n \"Audio Speech Recognition with aisuite's unified API supporting OpenAI, Deepgram, and Google providers.\\n\",\n \"\\n\",\n \"This example demonstrates basic transcription using the OpenAI format with different providers.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"d72f8c18\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import aisuite as ai\\n\",\n \"from aisuite.framework.message import TranscriptionResult\\n\",\n \"from dotenv import load_dotenv, find_dotenv\\n\",\n \"import os\\n\",\n \"\\n\",\n \"load_dotenv(find_dotenv())\\n\",\n \"\\n\",\n \"\\n\",\n \"# Set up client with provider configurations\\n\",\n \"client = ai.Client({\\n\",\n \" \\\"openai\\\": {\\\"api_key\\\": os.getenv(\\\"OPENAI_API_KEY\\\")},\\n\",\n \" \\\"deepgram\\\": {\\\"api_key\\\": os.getenv(\\\"DEEPGRAM_API_KEY\\\")},\\n\",\n \" \\\"google\\\": {\\n\",\n \" \\\"project_id\\\": os.getenv(\\\"GOOGLE_PROJECT_ID\\\"),\\n\",\n \" \\\"region\\\": os.getenv(\\\"GOOGLE_REGION\\\"),\\n\",\n \" \\\"application_credentials\\\": os.getenv(\\\"GOOGLE_APPLICATION_CREDENTIALS\\\"),\\n\",\n \" },\\n\",\n \"})\\n\",\n \"\\n\",\n \"audio_file = \\\"../aiplayground/speech.mp3\\\" # Replace with your audio file path\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"8ed7f8de\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Basic transcription using kwargs (OpenAI format)\\n\",\n \"print(\\\"=== Basic Transcription ===\\\")\\n\",\n \"\\n\",\n \"try:\\n\",\n \" result = client.audio.transcriptions.create(\\n\",\n \" model=\\\"openai:whisper-1\\\",\\n\",\n \" file=audio_file,\\n\",\n \" language=\\\"en\\\"\\n\",\n \" )\\n\",\n \" if isinstance(result, TranscriptionResult):\\n\",\n \" print(f\\\"OpenAI: {result.text}\\\")\\n\",\n \" else:\\n\",\n \" print(\\\"OpenAI: Got streaming result (not expected for basic call)\\\")\\n\",\n \"except Exception as e:\\n\",\n \" print(f\\\"OpenAI error: {e}\\\")\\n\",\n \"\\n\",\n \"print(\\\"--------------------------------\\\")\\n\",\n \"\\n\",\n \"try:\\n\",\n \" # Same kwargs work with other providers (auto-mapped)\\n\",\n \" result = client.audio.transcriptions.create(\\n\",\n \" model=\\\"deepgram:nova-2\\\",\\n\",\n \" file=audio_file,\\n\",\n \" language=\\\"en\\\",\\n\",\n \" punctuate=True\\n\",\n \" )\\n\",\n \" if isinstance(result, TranscriptionResult):\\n\",\n \" print(f\\\"Deepgram: {result.text}\\\")\\n\",\n \" else:\\n\",\n \" print(\\\"Deepgram: Got streaming result (not expected for basic call)\\\")\\n\",\n \"except Exception as e:\\n\",\n \" print(f\\\"Deepgram error: {e}\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"32ed3f0f\",\n \"metadata\": {},\n \"source\": [\n \"## Summary\\n\",\n \"\\n\",\n \"This notebook demonstrates the unified ASR interface with:\\n\",\n \"\\n\",\n \"1. **Basic transcription**: Using OpenAI-format kwargs that work across providers\\n\",\n \"2. **Provider compatibility**: Same interface works with OpenAI, Deepgram, and Google\\n\",\n \"\\n\",\n \"### Environment Setup Required:\\n\",\n \"\\n\",\n \"- **OpenAI**: `OPENAI_API_KEY`\\n\",\n \"- **Deepgram**: `DEEPGRAM_API_KEY` \\n\",\n \"- **Google**: `GOOGLE_PROJECT_ID`, `GOOGLE_REGION`, `GOOGLE_APPLICATION_CREDENTIALS`\\n\",\n \"\\n\",\n \"### Key Benefits:\\n\",\n \"\\n\",\n \"- Write once, run on any provider\\n\",\n \"- Consistent error handling and response format\\n\",\n \"- Easy provider switching for testing and optimization\"\n ]\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.8.0\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 5\n}\n" }, { "path": "examples/chat-ui/.streamlit/config.toml", "content": "[theme]\nprimaryColor = \"#1E90FF\" # Blue color for primary components\nbackgroundColor = \"#0e1117\" # Background color\nsecondaryBackgroundColor = \"#262730\" # Secondary background color\ntextColor = \"#ffffff\" # Text color\nfont = \"sans serif\"\n\n" }, { "path": "examples/chat-ui/README.md", "content": "# Chat UI\n\nThis is a simple chat UI built using Streamlit. It uses the `aisuite` library to power the chat.\n\nYou will need to install streamlit to run this example.\n\n```bash\npip install streamlit\n```\n\nYou will also need to create a `config.yaml` file in the same directory as the `chat.py` file. An example config file has been provided. You need to set environment variables for the API keys and other configuration for the LLMs you want to use. Place a .env file in this directory since `chat.py` will look for it.\n\nIn config.yaml, you can specify the LLMs you want to use in the chat. The chat UI will then display all these LLMs and you can select the one you want to use.\n\nTo run the app, simply run the following command in your terminal:\n\n```bash\nstreamlit run chat.py\n```\n\nYou can choose different LLMs by ticking the \"Comparison Mode\" checkbox. Then select the two LLMs you want to compare.\nHere are some sample queries you can try:\n\n```\nUser: \"What is the weather in Tokyo?\"\n```\n\n```\nUser: \"Write a poem about the weather in Tokyo.\"\n```\n\n```\nUser: \"Write a python program to print the fibonacci sequence.\"\nAssistant: \"-- Content from LLM 1 --\"\nUser: \"Write test cases for this program.\"\n```\n" }, { "path": "examples/chat-ui/chat.py", "content": "import os\nimport requests\nimport streamlit as st\nimport sys\nimport yaml\nfrom dotenv import load_dotenv, find_dotenv\n\nsys.path.append(\"../../../aisuite\")\nfrom aisuite.client import Client\n\n# Configure Streamlit to use wide mode and hide the top streamlit menu\nst.set_page_config(layout=\"wide\", menu_items={})\n# Add heading with padding\nst.markdown(\n \"

Chat & Compare LLM responses

\",\n unsafe_allow_html=True,\n)\nst.markdown(\n \"\"\"\n \n \"\"\",\n unsafe_allow_html=True,\n)\nst.markdown(\n \"\"\"\n \n \"\"\",\n unsafe_allow_html=True,\n)\n\n# Load configuration and initialize aisuite client\nwith open(\"config.yaml\", \"r\") as file:\n config = yaml.safe_load(file)\nconfigured_llms = config[\"llms\"]\nload_dotenv(find_dotenv())\nclient = Client()\n\n\n# Function to display chat history\ndef display_chat_history(chat_history, model_name):\n for message in chat_history:\n role_display = \"User\" if message[\"role\"] == \"user\" else model_name\n role = \"user\" if message[\"role\"] == \"user\" else \"assistant\"\n if role == \"user\":\n with st.chat_message(role, avatar=\"👤\"):\n st.write(message[\"content\"])\n else:\n with st.chat_message(role, avatar=\"🤖\"):\n st.write(message[\"content\"])\n\n\n# Helper function to query each LLM\ndef query_llm(model_config, chat_history):\n print(f\"Querying {model_config['name']} with {chat_history}\")\n try:\n model = model_config[\"provider\"] + \":\" + model_config[\"model\"]\n response = client.chat.completions.create(model=model, messages=chat_history)\n print(\n f\"Response from {model_config['name']}: {response.choices[0].message.content}\"\n )\n return response.choices[0].message.content\n except Exception as e:\n st.error(f\"Error querying {model_config['name']}: {e}\")\n return \"Error with LLM response.\"\n\n\n# Initialize session states\nif \"chat_history_1\" not in st.session_state:\n st.session_state.chat_history_1 = []\nif \"chat_history_2\" not in st.session_state:\n st.session_state.chat_history_2 = []\nif \"is_processing\" not in st.session_state:\n st.session_state.is_processing = False\nif \"use_comparison_mode\" not in st.session_state:\n st.session_state.use_comparison_mode = False\n\n# Top Section - Controls\ncol1, col2 = st.columns([1, 2])\nwith col1:\n st.session_state.use_comparison_mode = st.checkbox(\"Comparison Mode\", value=True)\n\n# Move LLM selection below comparison mode checkbox - now in columns\nllm_col1, llm_col2 = st.columns(2)\nwith llm_col1:\n selected_model_1 = st.selectbox(\n \"Choose LLM Model 1\",\n [llm[\"name\"] for llm in configured_llms],\n key=\"model_1\",\n index=0 if configured_llms else 0,\n )\nwith llm_col2:\n if st.session_state.use_comparison_mode:\n selected_model_2 = st.selectbox(\n \"Choose LLM Model 2\",\n [llm[\"name\"] for llm in configured_llms],\n key=\"model_2\",\n index=1 if len(configured_llms) > 1 else 0,\n )\n\n# Display Chat Histories first, always\n# Middle Section - Display Chat Histories\nif st.session_state.use_comparison_mode:\n col1, col2 = st.columns(2)\n with col1:\n chat_container = st.container(height=500)\n with chat_container:\n display_chat_history(st.session_state.chat_history_1, selected_model_1)\n with col2:\n chat_container = st.container(height=500)\n with chat_container:\n display_chat_history(st.session_state.chat_history_2, selected_model_2)\nelse:\n chat_container = st.container(height=500)\n with chat_container:\n display_chat_history(st.session_state.chat_history_1, selected_model_1)\n\n# Bottom Section - User Input\nst.markdown(\"
\", unsafe_allow_html=True)\n\ncol1, col2, col3 = st.columns([6, 1, 1])\nwith col1:\n user_query = st.text_area(\n label=\"Enter your query\",\n label_visibility=\"collapsed\",\n placeholder=\"Enter your query...\",\n key=\"query_input\",\n height=70,\n )\n\n\n# CSS for aligning buttons with the bottom of the text area\nst.markdown(\n \"\"\"\n \n \"\"\",\n unsafe_allow_html=True,\n)\n\nwith col2:\n send_button = False # Initialize send_button\n if st.session_state.is_processing:\n st.markdown(\n \"
Processing... ⏳
\",\n unsafe_allow_html=True,\n )\n else:\n send_button = st.button(\"Send Query\", use_container_width=True)\n\nwith col3:\n if st.button(\"Reset Chat\", use_container_width=True):\n st.session_state.chat_history_1 = []\n st.session_state.chat_history_2 = []\n st.rerun()\n\n# Handle send button click and processing\nif send_button and user_query and not st.session_state.is_processing:\n # Set processing state\n st.session_state.is_processing = True\n\n # Append user's message to chat histories first\n st.session_state.chat_history_1.append({\"role\": \"user\", \"content\": user_query})\n if st.session_state.use_comparison_mode:\n st.session_state.chat_history_2.append({\"role\": \"user\", \"content\": user_query})\n\n st.rerun()\n\n# Handle the actual processing\nif st.session_state.is_processing and user_query:\n # Query the selected LLM(s)\n model_config_1 = next(\n llm for llm in configured_llms if llm[\"name\"] == selected_model_1\n )\n response_1 = query_llm(model_config_1, st.session_state.chat_history_1)\n st.session_state.chat_history_1.append({\"role\": \"assistant\", \"content\": response_1})\n\n if st.session_state.use_comparison_mode:\n model_config_2 = next(\n llm for llm in configured_llms if llm[\"name\"] == selected_model_2\n )\n response_2 = query_llm(model_config_2, st.session_state.chat_history_2)\n st.session_state.chat_history_2.append(\n {\"role\": \"assistant\", \"content\": response_2}\n )\n\n # Reset processing state\n st.session_state.is_processing = False\n st.rerun()\n" }, { "path": "examples/chat-ui/config.yaml", "content": "# config.yaml\nllms:\n - name: \"OpenAI GPT-4o\"\n provider: \"openai\"\n model: \"gpt-4o\"\n - name: \"Anthropic Claude 3.5 Sonnet\"\n provider: \"anthropic\"\n model: \"claude-3-5-sonnet-20240620\"\n - name: \"Azure/OpenAI GPT-4o\"\n provider: \"azure\"\n model: \"gpt-4o\"\n - name: \"Huggingface/Mistral 7B\"\n provider: \"huggingface\"\n model: \"mistralai/Mistral-7B-Instruct\"\n" }, { "path": "examples/client.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"id\": \"d34f8c48-90fc-4981-8d2b-b47724c2a6dd\",\n \"metadata\": {\n \"vscode\": {\n \"languageId\": \"raw\"\n }\n },\n \"source\": [\n \"# Client Examples\\n\",\n \"\\n\",\n \"Client provides a uniform interface for interacting with LLMs from various providers. It adapts the official python libraries from providers such as Mistral, OpenAI, Groq, Anthropic, AWS, etc to conform to the OpenAI chat completion interface. It directly calls the REST endpoints in some cases.\\n\",\n \"\\n\",\n \"Below are some examples of how to use Client to interact with different LLMs.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"initial_id\",\n \"metadata\": {\n \"ExecuteTime\": {\n \"end_time\": \"2024-07-04T15:30:02.064319Z\",\n \"start_time\": \"2024-07-04T15:30:02.051986Z\"\n }\n },\n \"outputs\": [],\n \"source\": [\n \"import sys\\n\",\n \"from dotenv import load_dotenv, find_dotenv\\n\",\n \"\\n\",\n \"sys.path.append('../../aisuite')\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"f75736ee\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import os\\n\",\n \"def configure_environment(additional_env_vars=None):\\n\",\n \" \\\"\\\"\\\"\\n\",\n \" Load environment variables from .env file and apply any additional variables.\\n\",\n \" :param additional_env_vars: A dictionary of additional environment variables to apply.\\n\",\n \" \\\"\\\"\\\"\\n\",\n \" # Load from .env file if available\\n\",\n \" load_dotenv(find_dotenv())\\n\",\n \"\\n\",\n \" # Apply additional environment variables\\n\",\n \" if additional_env_vars:\\n\",\n \" for key, value in additional_env_vars.items():\\n\",\n \" os.environ[key] = value\\n\",\n \"\\n\",\n \"# Define additional API keys and credentials\\n\",\n \"additional_keys = {\\n\",\n \" 'GROQ_API_KEY': 'xxx',\\n\",\n \" 'AWS_ACCESS_KEY_ID': 'xxx',\\n\",\n \" 'AWS_SECRET_ACCESS_KEY': 'xxx',\\n\",\n \" 'ANTHROPIC_API_KEY': 'xxx',\\n\",\n \" 'NEBIUS_API_KEY': 'xxx',\\n\",\n \"}\\n\",\n \"\\n\",\n \"# Configure environment\\n\",\n \"configure_environment(additional_env_vars=additional_keys)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"4de3a24f\",\n \"metadata\": {\n \"ExecuteTime\": {\n \"end_time\": \"2024-07-04T15:31:12.914321Z\",\n \"start_time\": \"2024-07-04T15:31:12.796445Z\"\n }\n },\n \"outputs\": [],\n \"source\": [\n \"import aisuite as ai\\n\",\n \"\\n\",\n \"client = ai.Client()\\n\",\n \"messages = [\\n\",\n \" {\\\"role\\\": \\\"system\\\", \\\"content\\\": \\\"Respond in Pirate English. Always try to include the phrase - No rum No fun.\\\"},\\n\",\n \" {\\\"role\\\": \\\"user\\\", \\\"content\\\": \\\"Tell me a joke about Captain Jack Sparrow\\\"},\\n\",\n \"]\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"520a6879\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# print(os.environ[\\\"ANTHROPIC_API_KEY\\\"])\\n\",\n \"anthropic_claude_3_opus = \\\"anthropic:claude-3-5-sonnet-20240620\\\"\\n\",\n \"response = client.chat.completions.create(model=anthropic_claude_3_opus, messages=messages)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"9893c7e4-799a-42c9-84de-f9e643044462\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"aws_bedrock_llama3_8b = \\\"aws:meta.llama3-1-8b-instruct-v1:0\\\"\\n\",\n \"response = client.chat.completions.create(model=aws_bedrock_llama3_8b, messages=messages)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"7e46c20a\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# IMP NOTE: Azure expects model endpoint to be passed in the format of \\\"azure:\\\".\\n\",\n \"# The model name is the deployment name in Project/Deployments.\\n\",\n \"# In the example below, the model is \\\"mistral-large-2407\\\", but the name given to the\\n\",\n \"# deployment is \\\"aisuite-mistral-large-2407\\\" under the deployments section in Azure.\\n\",\n \"client.configure({\\\"azure\\\" : {\\n\",\n \" \\\"api_key\\\": os.environ[\\\"AZURE_API_KEY\\\"],\\n\",\n \" \\\"base_url\\\": \\\"https://aisuite-mistral-large-2407.westus3.models.ai.azure.com/v1/\\\",\\n\",\n \"}});\\n\",\n \"azure_model = \\\"azure:aisuite-mistral-large-2407\\\"\\n\",\n \"response = client.chat.completions.create(model=azure_model, messages=messages)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"f996b121\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# HuggingFace expects the model to be passed in the format of \\\"huggingface:\\\".\\n\",\n \"# The model name is the full name of the model in HuggingFace.\\n\",\n \"# In the example below, the model is \\\"mistralai/Mistral-7B-Instruct-v0.3\\\".\\n\",\n \"# The model is deployed as serverless inference endpoint in HuggingFace.\\n\",\n \"hf_model = \\\"huggingface:mistralai/Mistral-7B-Instruct-v0.3\\\"\\n\",\n \"response = client.chat.completions.create(model=hf_model, messages=messages)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"c9b2aad6-8603-4227-9566-778f714eb0b5\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"\\n\",\n \"# Groq expects the model to be passed in the format of \\\"groq:\\\".\\n\",\n \"# The model name is the full name of the model in Groq.\\n\",\n \"# In the example below, the model is \\\"llama3-8b-8192\\\".\\n\",\n \"groq_llama3_8b = \\\"groq:llama3-8b-8192\\\"\\n\",\n \"# groq_llama3_70b = \\\"groq:llama3-70b-8192\\\"\\n\",\n \"response = client.chat.completions.create(model=groq_llama3_8b, messages=messages)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"6819ac17\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"ollama_tinyllama = \\\"ollama:tinyllama\\\"\\n\",\n \"ollama_phi3mini = \\\"ollama:phi3:mini\\\"\\n\",\n \"response = client.chat.completions.create(model=ollama_phi3mini, messages=messages, temperature=0.75)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"4a94961b2bddedbb\",\n \"metadata\": {\n \"ExecuteTime\": {\n \"end_time\": \"2024-07-04T15:31:39.472675Z\",\n \"start_time\": \"2024-07-04T15:31:38.283368Z\"\n }\n },\n \"outputs\": [],\n \"source\": [\n \"mistral_7b = \\\"mistral:open-mistral-7b\\\"\\n\",\n \"response = client.chat.completions.create(model=mistral_7b, messages=messages, temperature=0.2)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"611210a4dc92845f\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"openai_gpt35 = \\\"openai:gpt-3.5-turbo\\\"\\n\",\n \"response = client.chat.completions.create(model=openai_gpt35, messages=messages, temperature=0.75)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"f38d033a-a580-4239-9176-27f3d53e7fe1\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"nebius_model = \\\"nebius:Qwen/Qwen2.5-1.5B-Instruct\\\"\\n\",\n \"response = client.chat.completions.create(model=nebius_model, messages=messages, top_p=0.01)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"321783ae\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"fireworks_model = \\\"fireworks:accounts/fireworks/models/llama-v3p2-3b-instruct\\\"\\n\",\n \"response = client.chat.completions.create(model=fireworks_model, messages=messages, temperature=0.75, presence_penalty=0.5, frequency_penalty=0.5)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"e30e5ae0\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"togetherai_model = \\\"together:meta-llama/Llama-3.2-3B-Instruct-Turbo\\\"\\n\",\n \"response = client.chat.completions.create(model=togetherai_model, messages=messages, temperature=0.75, top_p=0.7, top_k=50)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"dcf63a11\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"gemini_15_flash = \\\"google:gemini-1.5-flash\\\"\\n\",\n \"response = client.chat.completions.create(model=gemini_15_flash, messages=messages, temperature=0.75)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3 (ipykernel)\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.12.6\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 5\n}" }, { "path": "examples/llm_reasoning.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"id\": \"d39a806c-02a3-4a2d-8c51-f1ab1ea79d2e\",\n \"metadata\": {},\n \"source\": [\n \"# LLM Reasoning\\n\",\n \"\\n\",\n \"This notebook compares how LLMs from different Generative AI providers perform on three examples that can show issues with LLM reasoning:\\n\",\n \"\\n\",\n \"* [The Reversal Curse](https://github.com/lukasberglund/reversal_curse) shows that LLMs trained on \\\"A is B\\\" fail to learn \\\"B is A\\\".\\n\",\n \"* [How many r's in the word strawberry?](https://x.com/karpathy/status/1816637781659254908) shows \\\"the weirdness of LLM Tokenization\\\". \\n\",\n \"* [Which number is bigger, 9.11 or 9.9?](https://x.com/DrJimFan/status/1816521330298356181) shows that \\\"LLMs are alien beasts.\\\"\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"d2e413bd-983c-42a0-9580-96fedc7b1275\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"!cat ../.env.sample\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"8d843e36-7de6-4726-8a39-c5dcd3c7cc11\",\n \"metadata\": {},\n \"source\": [\n \"Make sure your ~/.env file (copied from the .env.sample file above) has the API keys of the LLM providers to compare set before running the cell below:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"3c966895-1a63-4922-80b7-5a20e47f29de\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import sys\\n\",\n \"sys.path.append('../../aisuite')\\n\",\n \"\\n\",\n \"from dotenv import load_dotenv, find_dotenv\\n\",\n \"\\n\",\n \"load_dotenv(find_dotenv())\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"09d5c5be-1085-4252-9d5e-80b50961484b\",\n \"metadata\": {},\n \"source\": [\n \"## Specify LLMs to Compare\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"26c3d5ef-b1c9-48dd-9b89-30799fd4b698\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import aisuite as ai\\n\",\n \"\\n\",\n \"client = ai.Client()\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"886a904f-fef0-4f25-b3ed-41085bf0f2dd\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import time\\n\",\n \"\\n\",\n \"llms = [\\n\",\n \" \\\"anthropic:claude-3-5-sonnet-20240620\\\",\\n\",\n \" \\\"aws:meta.llama3-1-8b-instruct-v1:0\\\",\\n\",\n \" \\\"groq:llama3-8b-8192\\\",\\n\",\n \" \\\"groq:llama3-70b-8192\\\",\\n\",\n \" \\\"huggingface:mistralai/Mistral-7B-Instruct-v0.3\\\",\\n\",\n \" \\\"openai:gpt-3.5-turbo\\\",\\n\",\n \" ]\\n\",\n \"\\n\",\n \"def compare_llm(messages):\\n\",\n \" execution_times = []\\n\",\n \" responses = []\\n\",\n \" for llm in llms:\\n\",\n \" start_time = time.time()\\n\",\n \" response = client.chat.completions.create(model=llm, messages=messages)\\n\",\n \" end_time = time.time()\\n\",\n \" execution_time = end_time - start_time\\n\",\n \" responses.append(response.choices[0].message.content.strip())\\n\",\n \" execution_times.append(execution_time)\\n\",\n \" print(f\\\"{llm} - {execution_time:.2f} seconds: {response.choices[0].message.content.strip()}\\\")\\n\",\n \" return responses, execution_times\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"3c3e8aa2-4ff4-485b-93d9-4a6f22d62e67\",\n \"metadata\": {},\n \"source\": [\n \"## The Reversal Curse\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"f3c4a8ef-e23b-4d4a-8561-3e5a2a866bd1\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"messages = [\\n\",\n \" {\\\"role\\\": \\\"user\\\", \\\"content\\\": \\\"Who is Tom Cruise's mother?\\\"},\\n\",\n \"]\\n\",\n \"\\n\",\n \"responses, execution_times = compare_llm(messages)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"769f7f42-2adb-4903-ab17-3143a5d950ce\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import pandas as pd\\n\",\n \"\\n\",\n \"def display(llms, execution_times, responses):\\n\",\n \" data = {\\n\",\n \" 'Provider:Model Name': llms,\\n\",\n \" 'Execution Time': execution_times,\\n\",\n \" 'Model Response ': responses\\n\",\n \" }\\n\",\n \" \\n\",\n \" df = pd.DataFrame(data)\\n\",\n \" df.index = df.index + 1\\n\",\n \" styled_df = df.style.set_table_styles(\\n\",\n \" [{'selector': 'th', 'props': [('text-align', 'center')]}, \\n\",\n \" {'selector': 'td', 'props': [('text-align', 'center')]}]\\n\",\n \" ).set_properties(**{'text-align': 'center'})\\n\",\n \" \\n\",\n \" return styled_df \"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"d2359ad5-9f0b-4bd6-9838-54df91de0fb3\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"display(llms, execution_times, responses)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"399f6cca-7f34-4a91-aab0-070560640033\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"messages = [\\n\",\n \" {\\\"role\\\": \\\"user\\\", \\\"content\\\": \\\"Who is Mary Lee Pfeiffer's son?\\\"},\\n\",\n \"]\\n\",\n \"\\n\",\n \"responses, execution_times = compare_llm(messages)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"eee7704d-a187-41bc-b119-c94461d0ee74\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"display(llms, execution_times, responses)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"ada8e0fb-17f0-4781-bf6a-c23ac86922ad\",\n \"metadata\": {},\n \"source\": [\n \"## How many r's in the word strawberry?\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"e537871e-68b6-44c3-886a-d3ebe7a692c1\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"messages = [\\n\",\n \" {\\\"role\\\": \\\"user\\\", \\\"content\\\": \\\"How many r's in the word strawberry?\\\"},\\n\",\n \"]\\n\",\n \"\\n\",\n \"responses, execution_times = compare_llm(messages)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"5678e393-4967-49f1-9e0f-251471dc92b7\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"display(llms, execution_times, responses)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"cae3fb5f-a173-4a33-b843-65df6d1086f9\",\n \"metadata\": {},\n \"source\": [\n \"## Which number is bigger?\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"efdf2fd6-f63a-4f9b-af15-1df25590e4fc\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"messages = [\\n\",\n \" {\\\"role\\\": \\\"user\\\", \\\"content\\\": \\\"Which number is bigger, 9.11 or 9.9?\\\"},\\n\",\n \"]\\n\",\n \"\\n\",\n \"responses, execution_times = compare_llm(messages)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"eaa14ed1-c83b-4c8f-bb14-d318bf0c9a60\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"display(llms, execution_times, responses)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"198b213a-b7bf-4cce-8c30-a8408454370b\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"messages = [\\n\",\n \" {\\\"role\\\": \\\"user\\\", \\\"content\\\": \\\"Which number is bigger, 9.11 or 9.9? Think step by step.\\\"},\\n\",\n \"]\\n\",\n \"\\n\",\n \"responses, execution_times = compare_llm(messages)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"4a3fb8fc-a7a2-47d3-9db2-792f03cc47c2\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"display(llms, execution_times, responses)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"66987d26-4245-4de1-816f-fa57475101f3\",\n \"metadata\": {},\n \"source\": [\n \"## Takeaways\\n\",\n \"1. Not all LLMs are created equal - not even all Llama 3 (or 3.1) are created equal (by different providers).\\n\",\n \"2. Ask LLM to think step by step may help improve its reasoning.\\n\",\n \"3. The way tokenization works in LLM could lead to a lot of weirdness in LLM (see AK's awesome [video](https://www.youtube.com/watch?v=zduSFxRajkE) for a deep dive).\\n\",\n \"4. A more comprehensive benchmark would be desired, but a quick LLM comparison like shown here can be the first step.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"04e13c90-3680-4f1d-8f65-768a78b7adb2\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": []\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3 (ipykernel)\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.12.6\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 5\n}\n" }, { "path": "examples/mcp_config_dict_example.py", "content": "\"\"\"\nMCP Tools with Config Dict Format - Example\n\nThis example demonstrates using MCP tools with the simplified config dict format.\nInstead of explicitly creating MCPClient objects, you can pass MCP server configs\ndirectly to the tools parameter.\n\"\"\"\n\nimport os\nfrom dotenv import load_dotenv\nimport aisuite as ai\n\n# Load environment variables\nload_dotenv()\n\n# Create aisuite client\nclient = ai.Client()\n\nprint(\"=\" * 70)\nprint(\"Example 1: Basic Config Dict Usage\")\nprint(\"=\" * 70)\n\n# Instead of creating MCPClient explicitly, pass config dict directly!\nresponse = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[\n {\"role\": \"user\", \"content\": \"List all Python files in the current directory\"}\n ],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"filesystem\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@modelcontextprotocol/server-filesystem\", os.getcwd()],\n }\n ],\n max_turns=2,\n)\n\nprint(response.choices[0].message.content)\n\nprint(\"\\n\" + \"=\" * 70)\nprint(\"Example 2: Filtering Tools with allowed_tools\")\nprint(\"=\" * 70)\n\n# Only allow specific tools for security\nresponse = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"Read the README.md file\"}],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"filesystem\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@modelcontextprotocol/server-filesystem\", os.getcwd()],\n \"allowed_tools\": [\"read_file\"], # Security: only allow reading, not writing\n }\n ],\n max_turns=2,\n)\n\nprint(response.choices[0].message.content)\n\nprint(\"\\n\" + \"=\" * 70)\nprint(\"Example 3: Multiple MCP Servers with Tool Prefixing\")\nprint(\"=\" * 70)\n\nimport tempfile\n\ntemp_dir = tempfile.mkdtemp()\n\n# Connect to two different filesystem servers with prefixing\n# This avoids tool name collisions\nresponse = client.chat.completions.create(\n model=\"anthropic:claude-3-5-sonnet-20240620\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": \"How many files are in the current directory vs the temp directory?\",\n }\n ],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"current_dir\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@modelcontextprotocol/server-filesystem\", os.getcwd()],\n \"use_tool_prefix\": True, # Tools named \"current_dir__list_directory\", etc.\n },\n {\n \"type\": \"mcp\",\n \"name\": \"temp_dir\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@modelcontextprotocol/server-filesystem\", temp_dir],\n \"use_tool_prefix\": True, # Tools named \"temp_dir__list_directory\", etc.\n },\n ],\n max_turns=3,\n)\n\nprint(response.choices[0].message.content)\n\nprint(\"\\n\" + \"=\" * 70)\nprint(\"Example 4: Mixing MCP Configs with Python Functions\")\nprint(\"=\" * 70)\n\nfrom datetime import datetime\n\n\ndef get_current_time() -> str:\n \"\"\"Get the current date and time.\"\"\"\n return datetime.now().strftime(\"%Y-%m-%d %H:%M:%S\")\n\n\ndef calculate_stats(numbers: list) -> dict:\n \"\"\"Calculate basic statistics for a list of numbers.\n\n Args:\n numbers: List of numbers to analyze\n \"\"\"\n return {\n \"count\": len(numbers),\n \"sum\": sum(numbers),\n \"average\": sum(numbers) / len(numbers) if numbers else 0,\n }\n\n\n# Mix everything: MCP configs + Python functions!\nresponse = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": \"What time is it? Also, list all files in the current directory.\",\n }\n ],\n tools=[\n get_current_time, # Regular Python function\n calculate_stats, # Another Python function\n {\n \"type\": \"mcp\",\n \"name\": \"filesystem\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@modelcontextprotocol/server-filesystem\", os.getcwd()],\n }, # MCP config dict\n ],\n max_turns=3,\n)\n\nprint(response.choices[0].message.content)\n\nprint(\"\\n\" + \"=\" * 70)\nprint(\"Example 5: When to Use Config Dict vs MCPClient\")\nprint(\"=\" * 70)\n\nprint(\n \"\"\"\nUse Config Dict When:\n✓ Quick prototypes and simple scripts\n✓ One-off tool usage\n✓ Don't need to reuse MCP client across multiple requests\n✓ Want automatic cleanup\n✓ Less code is better\n\nUse Explicit MCPClient When:\n✓ Need to reuse the same MCP connection across multiple requests\n✓ Want to inspect available tools before using them\n✓ Need fine-grained control over connection lifecycle\n✓ Building a long-running application\n✓ Want to manually manage resources\n\nExample of explicit MCPClient:\n\"\"\"\n)\n\nfrom aisuite.mcp import MCPClient\n\n# Create once, reuse many times\nmcp = MCPClient(\n command=\"npx\", args=[\"-y\", \"@modelcontextprotocol/server-filesystem\", os.getcwd()]\n)\n\n# Inspect available tools\nprint(f\"\\\\nAvailable tools: {[t['name'] for t in mcp.list_tools()]}\")\n\n# Reuse across multiple requests\nfor query in [\"List files\", \"Count files\", \"Check if README exists\"]:\n response = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[{\"role\": \"user\", \"content\": query}],\n tools=mcp.get_callable_tools(),\n max_turns=2,\n )\n print(f\"\\\\n{query}: {response.choices[0].message.content[:100]}...\")\n\nmcp.close()\n\nprint(\"\\n\" + \"=\" * 70)\nprint(\"All examples completed!\")\nprint(\"=\" * 70)\n" }, { "path": "examples/mcp_http_example.py", "content": "\"\"\"\nMCP HTTP Transport Example\n\nThis example demonstrates how to use HTTP-based MCP servers with aisuite.\n\nPrerequisites:\n- An HTTP MCP server running (e.g., http://localhost:8000)\n- OpenAI API key in .env file or OPENAI_API_KEY environment variable\n- pip install 'aisuite[mcp]'\n- pip install python-dotenv\n\nNote: This example assumes you have an HTTP MCP server running.\nIf you don't have one, this is a demonstration of the API usage.\n\"\"\"\n\nimport aisuite as ai\nfrom aisuite.mcp import MCPClient\nimport os\nfrom dotenv import load_dotenv\n\n# Load environment variables\nload_dotenv()\n\n\ndef example_1_config_dict_format():\n \"\"\"Example 1: Using HTTP MCP server with config dict format.\"\"\"\n print(\"=\" * 60)\n print(\"Example 1: HTTP MCP with Config Dict\")\n print(\"=\" * 60)\n\n client = ai.Client()\n\n response = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": \"Use the available tools to get the current weather data.\",\n }\n ],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"weather-api\",\n \"server_url\": \"http://localhost:8000/mcp/v1\", # Full endpoint URL\n \"timeout\": 30.0, # Optional: request timeout in seconds\n }\n ],\n max_turns=3,\n )\n\n print(response.choices[0].message.content)\n print()\n\n\ndef example_2_explicit_mcp_client():\n \"\"\"Example 2: Using HTTP MCP server with explicit MCPClient.\"\"\"\n print(\"=\" * 60)\n print(\"Example 2: HTTP MCP with Explicit MCPClient\")\n print(\"=\" * 60)\n\n # Create HTTP-based MCP client\n mcp = MCPClient(\n server_url=\"http://localhost:8000/mcp/v1\", # Full endpoint URL\n name=\"weather-api\",\n timeout=30.0,\n )\n\n # List available tools\n print(\"Available tools:\")\n for tool in mcp.list_tools():\n print(f\" - {tool['name']}: {tool['description']}\")\n print()\n\n # Use with aisuite\n client = ai.Client()\n response = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"What tools are available?\"}],\n tools=mcp.get_callable_tools(),\n max_turns=2,\n )\n\n print(response.choices[0].message.content)\n\n # Clean up\n mcp.close()\n print()\n\n\ndef example_3_with_authentication():\n \"\"\"Example 3: HTTP MCP server with authentication headers.\"\"\"\n print(\"=\" * 60)\n print(\"Example 3: HTTP MCP with Authentication\")\n print(\"=\" * 60)\n\n # Get API token from environment\n api_token = os.getenv(\"MCP_API_TOKEN\", \"your-token-here\")\n\n client = ai.Client()\n\n response = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"Fetch the user profile using the API.\"}],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"api-server\",\n \"server_url\": \"https://api.example.com/mcp/v1\", # Full endpoint URL\n \"headers\": {\n \"Authorization\": f\"Bearer {api_token}\",\n \"X-API-Version\": \"2024-01\",\n },\n \"timeout\": 60.0,\n }\n ],\n max_turns=3,\n )\n\n print(response.choices[0].message.content)\n print()\n\n\ndef example_4_context_manager():\n \"\"\"Example 4: Using context manager for automatic cleanup.\"\"\"\n print(\"=\" * 60)\n print(\"Example 4: HTTP MCP with Context Manager\")\n print(\"=\" * 60)\n\n with MCPClient(\n server_url=\"http://localhost:8000/mcp/v1\",\n name=\"api-server\", # Full endpoint URL\n ) as mcp:\n client = ai.Client()\n\n response = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"List available data.\"}],\n tools=mcp.get_callable_tools(),\n max_turns=2,\n )\n\n print(response.choices[0].message.content)\n # mcp.close() is called automatically\n print()\n\n\ndef example_5_mixing_http_and_python_functions():\n \"\"\"Example 5: Mixing HTTP MCP tools with regular Python functions.\"\"\"\n print(\"=\" * 60)\n print(\"Example 5: Mixing HTTP MCP with Python Functions\")\n print(\"=\" * 60)\n\n # Define a custom Python function\n def get_current_time() -> str:\n \"\"\"Get the current date and time in ISO format.\"\"\"\n from datetime import datetime\n\n return datetime.now().isoformat()\n\n client = ai.Client()\n\n response = client.chat.completions.create(\n model=\"anthropic:claude-sonnet-4-5\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": \"What time is it now? Also get the weather data from the API.\",\n }\n ],\n tools=[\n get_current_time, # Regular Python function\n {\n \"type\": \"mcp\",\n \"name\": \"weather-api\",\n \"server_url\": \"http://localhost:8000/mcp/v1\", # Full endpoint URL\n }, # HTTP MCP server\n ],\n max_turns=3,\n )\n\n print(response.choices[0].message.content)\n print()\n\n\ndef example_6_tool_filtering():\n \"\"\"Example 6: Using allowed_tools to restrict available tools.\"\"\"\n print(\"=\" * 60)\n print(\"Example 6: HTTP MCP with Tool Filtering\")\n print(\"=\" * 60)\n\n client = ai.Client()\n\n response = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"Get the weather forecast.\"}],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"api-server\",\n \"server_url\": \"http://localhost:8000/mcp/v1\", # Full endpoint URL\n \"allowed_tools\": [\"get_weather\"], # Only allow this specific tool\n }\n ],\n max_turns=2,\n )\n\n print(response.choices[0].message.content)\n print()\n\n\ndef example_7_multiple_http_servers():\n \"\"\"Example 7: Using multiple HTTP MCP servers with prefixing.\"\"\"\n print(\"=\" * 60)\n print(\"Example 7: Multiple HTTP MCP Servers with Prefixing\")\n print(\"=\" * 60)\n\n client = ai.Client()\n\n response = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": \"Get weather data and user data.\",\n }\n ],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"weather\",\n \"server_url\": \"http://localhost:8000/mcp/v1\", # Full endpoint URL\n \"use_tool_prefix\": True, # Tools: weather__get_forecast, etc.\n },\n {\n \"type\": \"mcp\",\n \"name\": \"users\",\n \"server_url\": \"http://localhost:9000/mcp/v1\", # Full endpoint URL\n \"use_tool_prefix\": True, # Tools: users__get_profile, etc.\n },\n ],\n max_turns=3,\n )\n\n print(response.choices[0].message.content)\n print()\n\n\nif __name__ == \"__main__\":\n print(\"\\nMCP HTTP Transport Examples\")\n print(\"=\" * 60)\n print()\n print(\"Note: These examples require an HTTP MCP server to be running.\")\n print(\"Uncomment the examples you want to run.\\n\")\n\n # Uncomment the examples you want to run:\n\n # example_1_config_dict_format()\n # example_2_explicit_mcp_client()\n # example_3_with_authentication()\n # example_4_context_manager()\n # example_5_mixing_http_and_python_functions()\n # example_6_tool_filtering()\n # example_7_multiple_http_servers()\n\n print(\"\\nTo run these examples:\")\n print(\"1. Start an HTTP MCP server (e.g., on http://localhost:8000)\")\n print(\"2. Set your OPENAI_API_KEY environment variable\")\n print(\"3. Uncomment the example functions you want to run\")\n print(\"4. Run: python examples/mcp_http_example.py\")\n" }, { "path": "examples/mcp_tools_example.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"# Using MCP Tools with aisuite\\n\",\n \"\\n\",\n \"This notebook demonstrates how to use MCP (Model Context Protocol) servers with aisuite to give AI models access to external tools and data sources.\\n\",\n \"\\n\",\n \"## What is MCP?\\n\",\n \"\\n\",\n \"MCP (Model Context Protocol) is a standardized protocol that allows AI applications to connect to external data sources and tools. MCP servers expose tools, resources, and prompts that AI models can use.\\n\",\n \"\\n\",\n \"## Prerequisites\\n\",\n \"\\n\",\n \"Install aisuite with MCP support:\\n\",\n \"```bash\\n\",\n \"pip install 'aisuite[mcp]'\\n\",\n \"# Or install providers you need:\\n\",\n \"pip install 'aisuite[openai,mcp]'\\n\",\n \"```\\n\",\n \"\\n\",\n \"You'll also need to install an MCP server. For this example, we'll use the filesystem server:\\n\",\n \"```bash\\n\",\n \"npm install -g @modelcontextprotocol/server-filesystem\\n\",\n \"```\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import os\\n\",\n \"from dotenv import load_dotenv\\n\",\n \"import aisuite as ai\\n\",\n \"from aisuite.mcp import MCPClient\\n\",\n \"\\n\",\n \"# Load environment variables (API keys)\\n\",\n \"load_dotenv()\\n\",\n \"\\n\",\n \"# Verify API key is set\\n\",\n \"if not os.getenv(\\\"OPENAI_API_KEY\\\"):\\n\",\n \" raise ValueError(\\\"Please set OPENAI_API_KEY environment variable\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Example 1: Basic MCP Tool Usage\\n\",\n \"\\n\",\n \"Let's connect to a filesystem MCP server and use it to read files.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Connect to the filesystem MCP server\\n\",\n \"# This gives the AI access to files in the specified directory\\n\",\n \"mcp_client = MCPClient(\\n\",\n \" command=\\\"npx\\\",\\n\",\n \" args=[\\\"-y\\\", \\\"@modelcontextprotocol/server-filesystem\\\", os.getcwd()]\\n\",\n \")\\n\",\n \"\\n\",\n \"print(f\\\"Connected to MCP server: {mcp_client}\\\")\\n\",\n \"print(f\\\"\\\\nAvailable tools:\\\")\\n\",\n \"for tool in mcp_client.list_tools():\\n\",\n \" print(f\\\" - {tool['name']}: {tool.get('description', 'No description')}\\\")\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Create aisuite client\\n\",\n \"client = ai.Client()\\n\",\n \"\\n\",\n \"# Use MCP tools with aisuite\\n\",\n \"messages = [\\n\",\n \" {\\\"role\\\": \\\"user\\\", \\\"content\\\": \\\"Please read the README.md file and summarize what this project does.\\\"}\\n\",\n \"]\\n\",\n \"\\n\",\n \"response = client.chat.completions.create(\\n\",\n \" model=\\\"openai:gpt-4o\\\",\\n\",\n \" messages=messages,\\n\",\n \" tools=mcp_client.get_callable_tools(), # MCP tools work like regular Python functions!\\n\",\n \" max_turns=3 # Allow multiple tool calls\\n\",\n \")\\n\",\n \"\\n\",\n \"print(\\\"\\\\nAI Response:\\\")\\n\",\n \"print(response.choices[0].message.content)\\n\",\n \"\\n\",\n \"# View the tool interactions\\n\",\n \"print(\\\"\\\\n\\\" + \\\"=\\\"*60)\\n\",\n \"print(\\\"Tool Call History:\\\")\\n\",\n \"print(\\\"=\\\"*60)\\n\",\n \"for msg in response.choices[0].intermediate_messages:\\n\",\n \" if msg.role == \\\"assistant\\\" and msg.tool_calls:\\n\",\n \" for tool_call in msg.tool_calls:\\n\",\n \" print(f\\\"\\\\nTool: {tool_call.function.name}\\\")\\n\",\n \" print(f\\\"Arguments: {tool_call.function.arguments}\\\")\\n\",\n \" elif msg.role == \\\"tool\\\":\\n\",\n \" print(f\\\"Result: {msg.content[:200]}...\\\") # First 200 chars\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Example 2: Mixing MCP Tools with Python Functions\\n\",\n \"\\n\",\n \"You can seamlessly mix MCP tools with regular Python functions!\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Define a custom Python function\\n\",\n \"def get_current_time() -> str:\\n\",\n \" \\\"\\\"\\\"Get the current date and time.\\\"\\\"\\\"\\n\",\n \" from datetime import datetime\\n\",\n \" return datetime.now().strftime(\\\"%Y-%m-%d %H:%M:%S\\\")\\n\",\n \"\\n\",\n \"def calculate_word_count(text: str) -> int:\\n\",\n \" \\\"\\\"\\\"Calculate the number of words in a text.\\n\",\n \" \\n\",\n \" Args:\\n\",\n \" text: The text to count words in\\n\",\n \" \\\"\\\"\\\"\\n\",\n \" return len(text.split())\\n\",\n \"\\n\",\n \"# Mix MCP tools and Python functions\\n\",\n \"all_tools = mcp_client.get_callable_tools() + [get_current_time, calculate_word_count]\\n\",\n \"\\n\",\n \"messages = [\\n\",\n \" {\\n\",\n \" \\\"role\\\": \\\"user\\\", \\n\",\n \" \\\"content\\\": \\\"What time is it? Also, read the README.md file and tell me how many words it contains.\\\"\\n\",\n \" }\\n\",\n \"]\\n\",\n \"\\n\",\n \"response = client.chat.completions.create(\\n\",\n \" model=\\\"openai:gpt-4o\\\",\\n\",\n \" messages=messages,\\n\",\n \" tools=all_tools, # Both MCP and Python tools!\\n\",\n \" max_turns=5\\n\",\n \")\\n\",\n \"\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Example 3: Using Specific MCP Tools\\n\",\n \"\\n\",\n \"You can also select specific tools instead of using all of them.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Get only specific tools\\n\",\n \"read_file = mcp_client.get_tool(\\\"read_file\\\")\\n\",\n \"list_directory = mcp_client.get_tool(\\\"list_directory\\\")\\n\",\n \"\\n\",\n \"# Use only these tools\\n\",\n \"messages = [\\n\",\n \" {\\\"role\\\": \\\"user\\\", \\\"content\\\": \\\"List all Python files in the current directory.\\\"}\\n\",\n \"]\\n\",\n \"\\n\",\n \"response = client.chat.completions.create(\\n\",\n \" model=\\\"openai:gpt-4o\\\",\\n\",\n \" messages=messages,\\n\",\n \" tools=[read_file, list_directory], # Only specific tools\\n\",\n \" max_turns=2\\n\",\n \")\\n\",\n \"\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Example 4: Using MCP with Different Providers\\n\",\n \"\\n\",\n \"MCP tools work with all aisuite providers!\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"source\": \"## Example 4: Using MCP Config Dict Format (Simplified)\\n\\nInstead of creating an MCPClient explicitly, you can pass MCP server configuration directly as a dict in the `tools` parameter! This is more convenient for simple use cases.\",\n \"metadata\": {}\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Try with different providers\\n\",\n \"providers = [\\n\",\n \" \\\"openai:gpt-4o\\\",\\n\",\n \" \\\"anthropic:claude-3-5-sonnet-20240620\\\",\\n\",\n \"]\\n\",\n \"\\n\",\n \"messages = [\\n\",\n \" {\\\"role\\\": \\\"user\\\", \\\"content\\\": \\\"List the files in the current directory and tell me how many there are.\\\"}\\n\",\n \"]\\n\",\n \"\\n\",\n \"for model in providers:\\n\",\n \" print(f\\\"\\\\n{'='*60}\\\")\\n\",\n \" print(f\\\"Provider: {model}\\\")\\n\",\n \" print(f\\\"{'='*60}\\\\n\\\")\\n\",\n \" \\n\",\n \" try:\\n\",\n \" response = client.chat.completions.create(\\n\",\n \" model=model,\\n\",\n \" messages=messages,\\n\",\n \" tools=mcp_client.get_callable_tools(),\\n\",\n \" max_turns=3\\n\",\n \" )\\n\",\n \" print(response.choices[0].message.content)\\n\",\n \" except Exception as e:\\n\",\n \" print(f\\\"Error: {e}\\\")\\n\",\n \" print(\\\"Make sure you have the API key set and provider installed.\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Example 5: Connecting to Multiple MCP Servers\\n\",\n \"\\n\",\n \"You can connect to multiple MCP servers and combine their tools.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Connect to a second MCP server (example - adjust as needed)\\n\",\n \"# For demonstration, we'll create another filesystem server for a different directory\\n\",\n \"\\n\",\n \"# Create a temp directory for demonstration\\n\",\n \"import tempfile\\n\",\n \"temp_dir = tempfile.mkdtemp()\\n\",\n \"\\n\",\n \"# Second MCP client\\n\",\n \"mcp_client_2 = MCPClient(\\n\",\n \" command=\\\"npx\\\",\\n\",\n \" args=[\\\"-y\\\", \\\"@modelcontextprotocol/server-filesystem\\\", temp_dir]\\n\",\n \")\\n\",\n \"\\n\",\n \"# Combine tools from both servers\\n\",\n \"all_mcp_tools = (\\n\",\n \" mcp_client.get_callable_tools() + \\n\",\n \" mcp_client_2.get_callable_tools()\\n\",\n \")\\n\",\n \"\\n\",\n \"print(f\\\"Total tools available: {len(all_mcp_tools)}\\\")\\n\",\n \"\\n\",\n \"# Note: In practice, you might want to rename tools or use namespacing\\n\",\n \"# to avoid conflicts between servers with similar tools\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Cleanup\\n\",\n \"\\n\",\n \"It's good practice to close MCP connections when done. You can also use MCPClient as a context manager.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Close the MCP connections\\n\",\n \"mcp_client.close()\\n\",\n \"mcp_client_2.close()\\n\",\n \"\\n\",\n \"print(\\\"MCP connections closed.\\\")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Using Context Manager (Recommended)\\n\",\n \"\\n\",\n \"The recommended way to use MCPClient is with a context manager:\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Using context manager ensures proper cleanup\\n\",\n \"with MCPClient(\\n\",\n \" command=\\\"npx\\\",\\n\",\n \" args=[\\\"-y\\\", \\\"@modelcontextprotocol/server-filesystem\\\", os.getcwd()]\\n\",\n \") as mcp:\\n\",\n \" response = client.chat.completions.create(\\n\",\n \" model=\\\"openai:gpt-4o\\\",\\n\",\n \" messages=[{\\\"role\\\": \\\"user\\\", \\\"content\\\": \\\"How many files are in the current directory?\\\"}],\\n\",\n \" tools=mcp.get_callable_tools(),\\n\",\n \" max_turns=2\\n\",\n \" )\\n\",\n \" print(response.choices[0].message.content)\\n\",\n \"\\n\",\n \"# Connection is automatically closed after the with block\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Summary\\n\",\n \"\\n\",\n \"Key takeaways:\\n\",\n \"\\n\",\n \"1. **Easy Integration**: MCP tools work seamlessly with aisuite's existing tool system\\n\",\n \"2. **Mix and Match**: Combine MCP tools with regular Python functions\\n\",\n \"3. **Provider Agnostic**: Works with any aisuite provider (OpenAI, Anthropic, Google, etc.)\\n\",\n \"4. **Multiple Servers**: Connect to multiple MCP servers simultaneously\\n\",\n \"5. **Simple API**: Just `MCPClient()` → `get_callable_tools()` → pass to `tools=[]`\\n\",\n \"\\n\",\n \"For more MCP servers, check out:\\n\",\n \"- https://github.com/modelcontextprotocol/servers\\n\",\n \"- Official MCP documentation: https://modelcontextprotocol.io/\"\n ]\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.10.0\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 4\n}" }, { "path": "examples/simple_tool_calling.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import json\\n\",\n \"import sys\\n\",\n \"from dotenv import load_dotenv, find_dotenv\\n\",\n \"import os\\n\",\n \"\\n\",\n \"sys.path.append('../../aisuite')\\n\",\n \"\\n\",\n \"# Load from .env file if available\\n\",\n \"load_dotenv(find_dotenv())\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Make a request to model without tools\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"from aisuite import Client\\n\",\n \"\\n\",\n \"client = Client()\\n\",\n \"# Configuring Azure. Rest all providers use environment variables for their parameters.\\n\",\n \"client.configure({\\\"azure\\\" : {\\n\",\n \" \\\"api_key\\\": os.environ[\\\"AZURE_API_KEY\\\"],\\n\",\n \" \\\"base_url\\\": \\\"https://aisuite-mistral-large-2407.westus3.models.ai.azure.com/v1/\\\",\\n\",\n \"}})\\n\",\n \"# model = \\\"anthropic:claude-3-5-sonnet-20241022\\\"\\n\",\n \"# model = \\\"aws:mistral.mistral-7b-instruct-v0:2\\\"\\n\",\n \"# model = \\\"azure:aisuite-mistral-large\\\"\\n\",\n \"# model = \\\"cohere:command-r-plus\\\"\\n\",\n \"# model = \\\"deepseek:deepseek-chat\\\"\\n\",\n \"# model = \\\"fireworks:accounts/fireworks/models/llama-v3p1-405b-instruct\\\"\\n\",\n \"# model = \\\"google:gemini-1.5-pro-002\\\"\\n\",\n \"# model = \\\"groq:llama-3.3-70b-versatile\\\"\\n\",\n \"# model = \\\"huggingface:meta-llama/Llama-3.1-8B-Instruct\\\"\\n\",\n \"# model = \\\"mistral:mistral-large-latest\\\"\\n\",\n \"# model = \\\"nebius:\\\"\\n\",\n \"# model = \\\"ollama:\\\"\\n\",\n \"# model = \\\"sambanova:Meta-Llama-3.3-70B-Instruct\\\"\\n\",\n \"# model = \\\"together:meta-llama/Llama-3.3-70B-Instruct-Turbo\\\"\\n\",\n \"# model = \\\"watsonx:\\\"\\n\",\n \"model = \\\"xai:grok-2-latest\\\"\\n\",\n \"\\n\",\n \"messages = [{\\n\",\n \" \\\"role\\\": \\\"user\\\",\\n\",\n \" \\\"content\\\": \\\"What is the current temperature in San Francisco in Celsius?\\\"}]\\n\",\n \"\\n\",\n \"response = client.chat.completions.create(\\n\",\n \" model=model, messages=messages)\\n\",\n \"\\n\",\n \"print(\\\"For model: \\\" + model)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"## Equip model with tools\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Define the functions\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Mock tool functions.\\n\",\n \"def get_current_temperature(location: str, unit: str):\\n\",\n \" # Simulate fetching temperature from an API\\n\",\n \" return {\\\"temperature\\\": 72}\\n\",\n \"\\n\",\n \"def get_rain_probability(location: str):\\n\",\n \" # Simulate fetching rain probability\\n\",\n \" return {\\\"location\\\": location, \\\"probability\\\": 40}\\n\",\n \"\\n\",\n \"# Function to get the available tools (functions) to provide to the model\\n\",\n \"# Note: we could use decorators or utils from OpenAI to generate this.\\n\",\n \"def get_available_tools():\\n\",\n \" return [\\n\",\n \" { \\\"type\\\": \\\"function\\\",\\n\",\n \" \\\"function\\\": {\\n\",\n \" \\\"name\\\": \\\"get_current_temperature\\\",\\n\",\n \" \\\"description\\\": \\\"Get the current temperature for a specific location\\\",\\n\",\n \" \\\"parameters\\\": {\\n\",\n \" \\\"type\\\": \\\"object\\\",\\n\",\n \" \\\"properties\\\": {\\n\",\n \" \\\"location\\\": {\\n\",\n \" \\\"type\\\": \\\"string\\\",\\n\",\n \" \\\"description\\\": \\\"The city and state, e.g., San Francisco, CA\\\"\\n\",\n \" },\\n\",\n \" \\\"unit\\\": {\\n\",\n \" \\\"type\\\": \\\"string\\\",\\n\",\n \" \\\"enum\\\": [\\\"Celsius\\\", \\\"Fahrenheit\\\"],\\n\",\n \" \\\"description\\\": \\\"The temperature unit to use.\\\"\\n\",\n \" }\\n\",\n \" },\\n\",\n \" \\\"required\\\": [\\\"location\\\", \\\"unit\\\"]\\n\",\n \" }\\n\",\n \" }\\n\",\n \" },\\n\",\n \" {\\n\",\n \" \\\"type\\\": \\\"function\\\",\\n\",\n \" \\\"function\\\": {\\n\",\n \" \\\"name\\\": \\\"get_rain_probability\\\",\\n\",\n \" \\\"description\\\": \\\"Get the probability of rain for a specific location\\\",\\n\",\n \" \\\"parameters\\\": {\\n\",\n \" \\\"type\\\": \\\"object\\\",\\n\",\n \" \\\"properties\\\": {\\n\",\n \" \\\"location\\\": {\\n\",\n \" \\\"type\\\": \\\"string\\\",\\n\",\n \" \\\"description\\\": \\\"The city and state, e.g., San Francisco, CA\\\"\\n\",\n \" }\\n\",\n \" },\\n\",\n \" \\\"required\\\": [\\\"location\\\"]\\n\",\n \" }\\n\",\n \" }\\n\",\n \" }\\n\",\n \" ]\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Function to process tool calls and get the result\\n\",\n \"def handle_tool_call(tool_call):\\n\",\n \" function_name = tool_call.function.name\\n\",\n \" arguments = json.loads(tool_call.function.arguments)\\n\",\n \"\\n\",\n \" # Map function names to actual tool function implementations\\n\",\n \" tools_map = {\\n\",\n \" \\\"get_current_temperature\\\": get_current_temperature,\\n\",\n \" \\\"get_rain_probability\\\": get_rain_probability,\\n\",\n \" }\\n\",\n \" return tools_map[function_name](**arguments)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"# Function to format tool response as a message\\n\",\n \"def create_tool_response_message(tool_call, tool_result):\\n\",\n \" return {\\n\",\n \" \\\"role\\\": \\\"tool\\\",\\n\",\n \" \\\"tool_call_id\\\": tool_call.id,\\n\",\n \" \\\"name\\\": tool_call.function.name,\\n\",\n \" \\\"content\\\": json.dumps(tool_result)\\n\",\n \" }\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Call the model with tools\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"import json\\n\",\n \"import sys\\n\",\n \"from dotenv import load_dotenv, find_dotenv\\n\",\n \"import os\\n\",\n \"\\n\",\n \"sys.path.append('../../aisuite')\\n\",\n \"\\n\",\n \"# Load from .env file if available\\n\",\n \"load_dotenv(find_dotenv())\\n\",\n \"\\n\",\n \"from aisuite import Client\\n\",\n \"\\n\",\n \"client = Client()\\n\",\n \"client.configure({\\\"azure\\\" : {\\n\",\n \" \\\"api_key\\\": os.environ[\\\"AZURE_API_KEY\\\"],\\n\",\n \" \\\"base_url\\\": \\\"https://aisuite-mistral-large-2407.westus3.models.ai.azure.com/v1/\\\",\\n\",\n \"}})\\n\",\n \"\\n\",\n \"# model = \\\"anthropic:claude-3-5-sonnet-20241022\\\"\\n\",\n \"# model = \\\"aws:mistral.mistral-7b-instruct-v0:2\\\"\\n\",\n \"# model = \\\"azure:aisuite-mistral-large\\\"\\n\",\n \"# model = \\\"cohere:command-r-plus\\\"\\n\",\n \"# model = \\\"deepseek:deepseek-chat\\\"\\n\",\n \"# model = \\\"fireworks:accounts/fireworks/models/llama-v3p1-405b-instruct\\\"\\n\",\n \"# model = \\\"google:gemini-1.5-pro-002\\\"\\n\",\n \"# model = \\\"groq:llama-3.3-70b-versatile\\\"\\n\",\n \"# model = \\\"huggingface:meta-llama/Llama-3.1-8B-Instruct\\\"\\n\",\n \"# model = \\\"mistral:mistral-large-latest\\\"\\n\",\n \"# model = \\\"nebius:\\\"\\n\",\n \"# model = \\\"ollama:\\\"\\n\",\n \"# model = \\\"sambanova:Meta-Llama-3.3-70B-Instruct\\\"\\n\",\n \"# model = \\\"together:meta-llama/Llama-3.3-70B-Instruct-Turbo\\\"\\n\",\n \"# model = \\\"watsonx:\\\"\\n\",\n \"model = \\\"xai:grok-2-latest\\\"\\n\",\n \"\\n\",\n \"messages = [{\\n\",\n \" \\\"role\\\": \\\"user\\\",\\n\",\n \" \\\"content\\\": \\\"What is the current temperature in San Francisco in Celsius?\\\"}]\\n\",\n \"\\n\",\n \"tools = get_available_tools()\\n\",\n \"\\n\",\n \"# Make the initial request to OpenAI API\\n\",\n \"response = client.chat.completions.create(\\n\",\n \" model=model, messages=messages, tools=tools)\\n\",\n \"\\n\",\n \"print(response)\\n\",\n \"print(response.choices[0].message)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"metadata\": {},\n \"source\": [\n \"### Process tool calls - Parse tool name, args, and call the function. Pass the result to the model.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"if response.choices[0].message.tool_calls:\\n\",\n \" for tool_call in response.choices[0].message.tool_calls:\\n\",\n \" tool_result = handle_tool_call(tool_call)\\n\",\n \" print(tool_result)\\n\",\n \"\\n\",\n \" messages.append(response.choices[0].message) # Model's function call message\\n\",\n \" messages.append(create_tool_response_message(tool_call, tool_result))\\n\",\n \" # Send the tool response back to the model\\n\",\n \" final_response = client.chat.completions.create(\\n\",\n \" model=model, messages=messages, tools=tools)\\n\",\n \" print(final_response.choices[0].message)\\n\",\n \" \\n\",\n \" # Output the final response from the model\\n\",\n \" print(final_response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"metadata\": {},\n \"outputs\": [],\n \"source\": []\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3 (ipykernel)\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.12.8\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 4\n}\n" }, { "path": "examples/tool_calling_abstraction.ipynb", "content": "{\n \"cells\": [\n {\n \"cell_type\": \"code\",\n \"execution_count\": 12,\n \"id\": \"9efdda2f-e3ab-4ec3-9b04-3ebea6fdf4c1\",\n \"metadata\": {},\n \"outputs\": [\n {\n \"data\": {\n \"text/plain\": [\n \"True\"\n ]\n },\n \"execution_count\": 12,\n \"metadata\": {},\n \"output_type\": \"execute_result\"\n }\n ],\n \"source\": [\n \"import json\\n\",\n \"import sys\\n\",\n \"from dotenv import load_dotenv, find_dotenv\\n\",\n \"import os\\n\",\n \"\\n\",\n \"sys.path.append('../../aisuite')\\n\",\n \"\\n\",\n \"# Load from .env file if available\\n\",\n \"load_dotenv(find_dotenv())\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"b7604862\",\n \"metadata\": {},\n \"source\": [\n \"### Define the function\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 13,\n \"id\": \"aaba7cb2-29de-4552-8fd5-8b966fbc0cd5\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"def will_it_rain(location: str, time_of_day: str):\\n\",\n \" \\\"\\\"\\\"Check if it will rain in a location at a given time today.\\n\",\n \"\\n\",\n \" Args:\\n\",\n \" location (str): Name of the city\\n\",\n \" time_of_day (str): Time of the day in HH:MM format.\\n\",\n \" \\\"\\\"\\\"\\n\",\n \" return \\\"YES\\\"\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"cf943639\",\n \"metadata\": {},\n \"source\": [\n \"---\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"090dc2d1\",\n \"metadata\": {},\n \"source\": [\n \"### Using OpenAI\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"29537ae1\",\n \"metadata\": {},\n \"source\": [\n \"### JSON spec for the function\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 14,\n \"id\": \"b1a7196a-e5f2-4016-8dca-eca804ef18ac\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"tools = [{\\n\",\n \" \\\"type\\\": \\\"function\\\",\\n\",\n \" \\\"function\\\": {\\n\",\n \" \\\"name\\\": \\\"will_it_rain\\\",\\n\",\n \" \\\"description\\\": \\\"Check if it will rain in a location at a given time today\\\",\\n\",\n \" \\\"parameters\\\": {\\n\",\n \" \\\"type\\\": \\\"object\\\",\\n\",\n \" \\\"properties\\\": {\\n\",\n \" \\\"location\\\": {\\n\",\n \" \\\"type\\\": \\\"string\\\",\\n\",\n \" \\\"description\\\": \\\"Name of the city\\\"\\n\",\n \" },\\n\",\n \" \\\"time_of_day\\\": {\\n\",\n \" \\\"type\\\": \\\"string\\\",\\n\",\n \" \\\"description\\\": \\\"Time of the day in HH:MM format.\\\"\\n\",\n \" }\\n\",\n \" },\\n\",\n \" \\\"required\\\": [\\\"location\\\", \\\"time_of_day\\\"]\\n\",\n \" }\\n\",\n \" }\\n\",\n \"}]\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"5359d145\",\n \"metadata\": {},\n \"source\": [\n \"### Send user request to LLM\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 15,\n \"id\": \"4b522391-5a82-4d27-bbad-9db91e531815\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"from openai import OpenAI\\n\",\n \"\\n\",\n \"client = OpenAI()\\n\",\n \"messages = [{\\n\",\n \" \\\"role\\\": \\\"user\\\",\\n\",\n \" \\\"content\\\": \\\"I live in San Francisco. Can you check for weather \\\"\\n\",\n \" \\\"and plan an outdoor picnic for me at 2pm?\\\"\\n\",\n \"}]\\n\",\n \"response = client.chat.completions.create(\\n\",\n \" model=\\\"gpt-4o\\\", messages=messages, tools=tools\\n\",\n \")\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"fc47b536\",\n \"metadata\": {},\n \"source\": [\n \"### Process tool call response, Execute the tool & call the model again\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 16,\n \"id\": \"d281fe8e-9d87-4150-be4d-f01a2525edf6\",\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"It looks like it will rain in San Francisco at 2 PM today. It's probably not the best day for an outdoor picnic. Would you like to consider indoor activities or reschedule the picnic for another day?\\n\"\n ]\n }\n ],\n \"source\": [\n \"response2 = None\\n\",\n \"if response.choices[0].message.tool_calls:\\n\",\n \" tool_call = response.choices[0].message.tool_calls[0]\\n\",\n \" args = json.loads(tool_call.function.arguments)\\n\",\n \"\\n\",\n \" result = will_it_rain(args[\\\"location\\\"], args[\\\"time_of_day\\\"])\\n\",\n \" messages.append(response.choices[0].message)\\n\",\n \" messages.append({\\n\",\n \" \\\"role\\\": \\\"tool\\\", \\\"tool_call_id\\\": tool_call.id, \\\"content\\\": str(result)\\n\",\n \" })\\n\",\n \"\\n\",\n \" response2 = client.chat.completions.create(\\n\",\n \" model=\\\"gpt-4o\\\", messages=messages, tools=tools)\\n\",\n \" print(response2.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"e81fb905\",\n \"metadata\": {},\n \"source\": [\n \"### Optionally, continue the conversation\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 17,\n \"id\": \"f6de5c21-2983-4539-a4a2-cdd76071fdea\",\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"If you're set on having the picnic despite the rain, here are a few suggestions to make it enjoyable:\\n\",\n \"\\n\",\n \"1. **Location:** Consider a location with some shelter, like a gazebo in a park, or find a picnic spot with large trees for some natural cover. Alternatively, you might want to think about an indoor space with large windows where you can enjoy the view without getting wet.\\n\",\n \"\\n\",\n \"2. **Packing Essentials:**\\n\",\n \" - Bring waterproof blankets or tarps to sit on.\\n\",\n \" - Pack an umbrella or a rain poncho to stay dry.\\n\",\n \" - Use waterproof containers for your food to keep everything dry.\\n\",\n \" - Bring extra towels and a change of clothes.\\n\",\n \"\\n\",\n \"3. **Food Ideas:** Opt for warm, hearty meals that are comforting on a rainy day, such as soup in a thermos or grilled sandwiches.\\n\",\n \"\\n\",\n \"4. **Activities:** Plan indoor-friendly games or activities you can enjoy sheltered from the rain, like card games or board games.\\n\",\n \"\\n\",\n \"5. **Safety First:** Check the weather forecast regularly in case of any severe weather warnings.\\n\",\n \"\\n\",\n \"Enjoy your rainy-day picnic!\\n\"\n ]\n }\n ],\n \"source\": [\n \"messages.append(response2.choices[0].message)\\n\",\n \"messages.append({\\n\",\n \" \\\"role\\\": \\\"user\\\", \\\"content\\\": \\\"Schedule it despite the rain\\\"\\n\",\n \"})\\n\",\n \"response = client.chat.completions.create(\\n\",\n \" model=\\\"gpt-4o\\\", messages=messages)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"f6d5fd91\",\n \"metadata\": {},\n \"source\": [\n \"---\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"897b9aa2\",\n \"metadata\": {},\n \"source\": [\n \"### Using aisuite\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"8b587445\",\n \"metadata\": {},\n \"source\": [\n \"### Call the model with tools. Tool call is handled internally.\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 18,\n \"id\": \"0b74257b-8d53-4326-8ea1-d4a7f89c0e57\",\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"It is expected to rain in San Francisco at 2 PM today, so it might not be the best time for an outdoor picnic. You might want to consider indoor alternatives or plan for another day when the weather is more favorable for outdoor activities. If you have any other plans in mind or need further assistance, feel free to ask!\\n\"\n ]\n }\n ],\n \"source\": [\n \"from aisuite import Client\\n\",\n \"\\n\",\n \"client = Client()\\n\",\n \"messages = [{\\n\",\n \" \\\"role\\\": \\\"user\\\",\\n\",\n \" \\\"content\\\": \\\"I live in San Francisco. Can you check for weather \\\"\\n\",\n \" \\\"and plan an outdoor picnic for me at 2pm?\\\"\\n\",\n \"}]\\n\",\n \"response = client.chat.completions.create(\\n\",\n \" model=\\\"openai:gpt-4o\\\", messages=messages, tools=[will_it_rain], max_turns=2\\n\",\n \")\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"ba5f0327\",\n \"metadata\": {},\n \"source\": [\n \"### Optionally, continue the conversation\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 19,\n \"id\": \"b1736dc9\",\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"Sure! If you'd like to go ahead and plan an outdoor picnic despite the rain, here are a few tips to help you enjoy your experience:\\n\",\n \"\\n\",\n \"1. **Choose a Covered Location**: Try to find a spot with a shelter, like a gazebo or a pavilion, in one of San Francisco's parks. These can provide good protection from the rain.\\n\",\n \"\\n\",\n \"2. **Prepare for Wet Weather**:\\n\",\n \" - Bring waterproof blankets or tarps to sit on.\\n\",\n \" - Pack umbrellas and raincoats to stay dry.\\n\",\n \" - Use waterproof containers and bags to keep your picnic items protected.\\n\",\n \"\\n\",\n \"3. **Select Comforting Foods**: Warm beverages in thermoses and foods that are enjoyable when slightly cooler, such as sandwiches, can be comforting on a rainy day.\\n\",\n \"\\n\",\n \"4. **Stay Safe**: Be cautious of slippery surfaces and keep an eye on kids or pets if they are joining you.\\n\",\n \"\\n\",\n \"5. **Enjoy the Atmosphere**: Rain can create a cozy and calming environment. Embrace the natural sounds and sights of the rain.\\n\",\n \"\\n\",\n \"6. **Plan Indoor Activities**: Bring board games or books in case the rain becomes too heavy.\\n\",\n \"\\n\",\n \"Have fun, and make the most of your unique outdoor picnic experience! If you need more help with planning or have any other questions, let me know.\\n\"\n ]\n }\n ],\n \"source\": [\n \"messages.extend(response.choices[0].intermediate_messages)\\n\",\n \"messages.append({\\n\",\n \" \\\"role\\\": \\\"user\\\", \\\"content\\\": \\\"Schedule it despite the rain\\\"\\n\",\n \"})\\n\",\n \"response = client.chat.completions.create(\\n\",\n \" model=\\\"openai:gpt-4o\\\", messages=messages)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"markdown\",\n \"id\": \"6bf09c0a\",\n \"metadata\": {},\n \"source\": [\n \"### Using, other providers\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 20,\n \"id\": \"6c266c68-095b-4625-8de5-70960a117f69\",\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"I apologize, but it looks like it will be raining in San Francisco at 2:00 PM today. This might not be the best conditions for an outdoor picnic. However, I can suggest a few alternatives:\\n\",\n \"\\n\",\n \"1. Reschedule: We could check the weather for a different time today or another day this week. Would you like me to check a different time or date?\\n\",\n \"\\n\",\n \"2. Indoor picnic: You could have an indoor picnic at home or find a covered area in a park with picnic shelters.\\n\",\n \"\\n\",\n \"3. Rain-ready picnic: If you're up for an adventure, you could prepare for a rainy day picnic with appropriate gear like umbrellas, waterproof blankets, and covered food containers.\\n\",\n \"\\n\",\n \"4. Alternative activity: We could look into indoor activities or attractions in San Francisco that you might enjoy instead.\\n\",\n \"\\n\",\n \"What would you prefer to do? If you'd like to try for a different time or date, please let me know, and I'll be happy to check the weather again.\\n\"\n ]\n }\n ],\n \"source\": [\n \"messages = [{\\n\",\n \" \\\"role\\\": \\\"user\\\",\\n\",\n \" \\\"content\\\": \\\"I live in San Francisco. Can you check for weather \\\"\\n\",\n \" \\\"and plan an outdoor picnic for me at 2pm?\\\"\\n\",\n \"}]\\n\",\n \"response = client.chat.completions.create(\\n\",\n \" model=\\\"anthropic:claude-3-5-sonnet-20240620\\\", messages=messages, tools=[will_it_rain], max_turns=2\\n\",\n \")\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": null,\n \"id\": \"25a8c5ab-3a9f-4bf1-8174-aba313a00085\",\n \"metadata\": {},\n \"outputs\": [],\n \"source\": [\n \"response = client.chat.completions.create(\\n\",\n \" model=\\\"groq:llama3-8b-8192\\\", messages=messages, tools=[will_it_rain], max_turns=2)\"\n ]\n },\n {\n \"cell_type\": \"code\",\n \"execution_count\": 17,\n \"id\": \"53e937bd\",\n \"metadata\": {},\n \"outputs\": [\n {\n \"name\": \"stdout\",\n \"output_type\": \"stream\",\n \"text\": [\n \"I can help with that! To provide accurate information, I'll need to know your tolerance for rain. Would you prefer the picnic indoors if it rains?\\n\"\n ]\n }\n ],\n \"source\": [\n \"response = client.chat.completions.create(\\n\",\n \" model=\\\"ollama:llama3-groq-tool-use\\\", messages=messages, tools=[will_it_rain], max_turns=2)\\n\",\n \"print(response.choices[0].message.content)\"\n ]\n }\n ],\n \"metadata\": {\n \"kernelspec\": {\n \"display_name\": \"Python 3 (ipykernel)\",\n \"language\": \"python\",\n \"name\": \"python3\"\n },\n \"language_info\": {\n \"codemirror_mode\": {\n \"name\": \"ipython\",\n \"version\": 3\n },\n \"file_extension\": \".py\",\n \"mimetype\": \"text/x-python\",\n \"name\": \"python\",\n \"nbconvert_exporter\": \"python\",\n \"pygments_lexer\": \"ipython3\",\n \"version\": \"3.12.8\"\n }\n },\n \"nbformat\": 4,\n \"nbformat_minor\": 5\n}\n" }, { "path": "guides/README.md", "content": "# Provider guides \n\nThese guides give directions for obtaining API keys from different providers. \n\nHere are the instructions for:\n- [Anthropic](anthropic.md) \n- [AWS](aws.md)\n- [Azure](azure.md) \n- [Cohere](cohere.md)\n- [Google](google.md)\n- [Hugging Face](huggingface.md)\n- [Mistral](mistral.md)\n- [OpenAI](openai.md)\n- [SambaNova](sambanova.md)\n- [xAI](xai.md)\n- [DeepSeek](deepseek.md)\n\nFor locally hosted models using `Ollama` or `LM Studio`, follow these instructions:\n- [Ollama](ollama.md)\n- [LM Studio](lmstudio.md)\n\nUnless otherwise stated, these guides have not been endorsed by the providers. \n\nWe also welcome additional [contributions](../CONTRIBUTING.md). \n\n" }, { "path": "guides/anthropic.md", "content": "# Anthropic\n\nTo use Anthropic with `aisuite` you will need to [create an account](https://console.anthropic.com/login). Once logged in, go to the [API Keys](https://console.anthropic.com/settings/keys)\nand click the \"Create Key\" button and export that key into your environment.\n\n\n```shell\nexport ANTHROPIC_API_KEY=\"your-anthropic-api-key\"\n```\n\n## Create a Chat Completion\n\nInstall the `anthropic` python client\n\nExample with pip\n```shell\npip install anthropic\n```\n\nExample with poetry\n```shell\npoetry add anthropic\n```\n\nIn your code:\n```python\nimport aisuite as ai\nclient = ai.Client()\n\n\nprovider = \"anthropic\"\nmodel_id = \"claude-3-5-sonnet-20241022\"\n\nmessages = [\n {\"role\": \"system\", \"content\": \"Respond in Pirate English.\"},\n {\"role\": \"user\", \"content\": \"Tell me a joke.\"},\n]\n\nresponse = client.chat.completions.create(\n model=f\"{provider}:{model_id}\",\n messages=messages,\n)\n\nprint(response.choices[0].message.content)\n```\n\nHappy coding! If you would like to contribute, please read our [Contributing Guide](../CONTRIBUTING.md).\n" }, { "path": "guides/aws.md", "content": "# AWS\n\nTo use AWS Bedrock with `aisuite` you will need to create an AWS account and\nnavigate to https://console.aws.amazon.com/bedrock/home. This route\nwill be redirected to your default region. In this example the region has been set to\n`us-west-2`. Anywhere the region is specified can be replaced with your desired region.\n\nNavigate to the [overview](https://us-west-2.console.aws.amazon.com/bedrock/home?region=us-west-2#/overview) page\ndirectly or by clicking on the `Get started` link.\n\n## Foundation Model Access\n\nYou will first need to give your AWS account access to the foundation models by\nvisiting the [modelaccess](https://us-west-2.console.aws.amazon.com/bedrock/home?region=us-west-2#/modelaccess)\npage to enable the models you would like to use. \n\nAfter enabling the foundation models, navigate to [providers page](https://us-west-2.console.aws.amazon.com/bedrock/home?region=us-west-2#/providers) \nand select the provider of the model you would like to use. From this page select the specific model you would like to use and \nmake note of the `Model ID` (currently located near the bottom) this will be used when using the chat completion example below.\n\nOnce that has been enabled set your Access Key and Secret in the env variables:\n\n```shell\nexport AWS_ACCESS_KEY=\"your-access-key\"\nexport AWS_SECRET_KEY=\"your-secret-key\"\nexport AWS_REGION=\"region-name\" \n```\n*Note: AWS_REGION is optional, a default of `us-west-2` has been set for easy of use*\n\n## Create a Chat Completion\n\nInstall the boto3 client using your package installer\n\nExample with pip\n```shell\npip install boto3\n```\n\nExample with poetry\n```shell\npoetry add boto3\n```\n\nIn your code:\n```python\nimport aisuite as ai\nclient = ai.Client()\n\n\nprovider = \"aws\"\nmodel_id = \"meta.llama3-1-405b-instruct-v1:0\" # Model ID from above\n\nmessages = [\n {\"role\": \"system\", \"content\": \"Respond in Pirate English.\"},\n {\"role\": \"user\", \"content\": \"Tell me a joke.\"},\n]\n\nresponse = client.chat.completions.create(\n model=f\"{provider}:{model_id}\",\n messages=messages,\n)\n\nprint(response.choices[0].message.content)\n```\n\nHappy coding! If you would like to contribute, please read our [Contributing Guide](../CONTRIBUTING.md).\n\n\n\n\n\n" }, { "path": "guides/azure.md", "content": "# Azure AI\n\nTo use Azure AI with the `aisuite` library, you'll need to set up an Azure account and configure your environment for Azure AI services.\n\n## Create an Azure Account and deploy a model from AI Studio\n\n1. Visit [Azure Portal](https://portal.azure.com/) and sign up for an account if you don't have one.\n2. Create a project and resource group.\n3. Choose a model from https://ai.azure.com/explore/models and deploy it. You can choose serverless deployment option.\n4. Give a deployment name. Lets say you choose to deploy Mistral-large-2407. You could leave the deployment names as \"mistral-large-2407\" or give a custom name.\n5. You can see the deployment from project/deployment option. Note the Target URI from the Endpoint panel. It should look something like this - \"https://aisuite-Mistral-large-2407.westus3.models.ai.azure.com\".\n6. Also note, that is provides a Chat completion URL. It should look like this - https://aisuite-Mistral-large-2407.westus3.models.ai.azure.com/v1/chat/completions\n\n\n## Obtain Necessary Details & set environment variables.\n\nAfter creating your deployment, you'll need to gather the following information:\n\n1. API Key: Found in the \"Keys and Endpoint\" section of your Azure OpenAI resource.\n2. Base URL: This can be obtained from your deployment details. It will look something like this - `https://aisuite-Mistral-large-2407.westus3.models.ai.azure.com/v1/`\n3. API Version: Optional configuration and mainly introduced for Azure OpenAI services. Once specified, the `api-version` query parameters will be added in the end of the API request.\n\n\nSet the following environment variables:\n\n```shell\nexport AZURE_API_KEY=\"your-api-key\"\nexport AZURE_BASE_URL=\"https://deployment-name.region-name.models.ai.azure.com/v1\"\nexport AZURE_API_VERSION=\"=2024-08-01-preview\"\n```\n\n## Create a Chat Completion\n\nWith your account set up and environment configured, you can send a chat completion request:\n\n```python\nimport aisuite as ai\n\n# Either set the environment variables or set the below two parameters.\n# Setting the params in ai.Client() will override the values from environment vars.\nclient = ai.Client(\n base_url=os.environ[\"AZURE_OPENAI_BASE_URL\"],\n api_key=os.environ[\"AZURE_OPENAI_API_KEY\"],\n api_version=os.environ[\"AZURE_API_VERSION\"]\n)\n\nmodel = \"azure:aisuite-Mistral-large-2407\" # Replace with your deployment name.\n# The model name must match the deployment name in the base-url.\n\nmessages = [\n {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n {\"role\": \"user\", \"content\": \"What's the weather like today?\"},\n]\n\nresponse = client.chat.completions.create(\n model=model,\n messages=messages,\n)\n\nprint(response.choices[0].message.content)\n```\n\nHappy coding! If you would like to contribute, please read our [Contributing Guide](../CONTRIBUTING.md)." }, { "path": "guides/cerebras.md", "content": "# Cerebras AI Suite Provider Guide\n\n## About Cerebras\n\nAt Cerebras, we've developed the world's largest and fastest AI processor, the Wafer-Scale Engine-3 (WSE-3). The Cerebras CS-3 system, powered by the WSE-3, represents a new class of AI supercomputer that sets the standard for generative AI training and inference with unparalleled performance and scalability.\n\nWith Cerebras as your inference provider, you can:\n- Achieve unprecedented speed for AI inference workloads\n- Build commercially with high throughput\n- Effortlessly scale your AI workloads with our seamless clustering technology\n\nOur CS-3 systems can be quickly and easily clustered to create the largest AI supercomputers in the world, making it simple to place and run the largest models. Leading corporations, research institutions, and governments are already using Cerebras solutions to develop proprietary models and train popular open-source models.\n\nWant to experience the power of Cerebras? Check out our [website](https://cerebras.net) for more resources and explore options for accessing our technology through the Cerebras Cloud or on-premise deployments!\n\n> [!NOTE] \n> This SDK has a mechanism that sends a few requests to `/v1/tcp_warming` upon construction to reduce the TTFT. If this behaviour is not desired, set `warm_tcp_connection=False` in the constructor.\n>\n> If you are repeatedly reconstructing the SDK instance it will lead to poor performance. It is recommended that you construct the SDK once and reuse the instance if possible.\n\n## Documentation\n\nFor the most comprehensive and up-to-date Cerebras Inference docs, please visit [inference-docs.cerebras.ai](https://inference-docs.cerebras.ai).\n\n## Usage\nGet an API Key from [cloud.cerebras.ai](https://cloud.cerebras.ai/) and add it to your environment variables:\n\n```shell\nexport CEREBRAS_API_KEY=\"your-cerebras-api-key\"\n```\n\nUse the python client.\n\n```python\nimport aisuite as ai\nclient = ai.Client()\n\nmessages = [\n {\"role\": \"system\", \"content\": \"Respond in Pirate English.\"},\n {\"role\": \"user\", \"content\": \"Tell me a joke.\"},\n]\n\nresponse = client.chat.completions.create(\n model=\"cerebras:llama3.1-8b\",\n messages=messages,\n temperature=0.75\n)\nprint(response.choices[0].message.content)\n\n```\n\n## Requirements\n\nPython 3.8 or higher.\n" }, { "path": "guides/cohere.md", "content": "# Cohere\n\nTo use Cohere with `aisuite`, you’ll need an [Cohere account](https://cohere.com/). After logging in, go to the [API Keys](https://dashboard.cohere.com/api-keys) section in your account settings, agree to the terms of service, connect your card, and generate a new key. Once you have your key, add it to your environment as follows:\n\n```shell\nexport CO_API_KEY=\"your-cohere-api-key\"\n```\n\n## Create a Chat Completion\n\nInstall the `cohere` Python client:\n\nExample with pip:\n```shell\npip install cohere\n```\n\nExample with poetry:\n```shell\npoetry add cohere\n```\n\nIn your code:\n```python\nimport aisuite as ai\n\nclient = ai.Client()\n\nprovider = \"cohere\"\nmodel_id = \"command-r-plus-08-2024\"\n\nmessages = [\n {\"role\": \"user\", \"content\": \"Hi, how are you?\"}\n]\n\nresponse = client.chat.completions.create(\n model=f\"{provider}:{model_id}\",\n messages=messages,\n)\n\nprint(response.choices[0].message.content)\n```\n\nHappy coding! If you’d like to contribute, please read our [Contributing Guide](CONTRIBUTING.md).\n" }, { "path": "guides/deepseek.md", "content": "# DeepSeek\n\nTo use DeepSeek with `aisuite`, you’ll need an [DeepSeek account](https://platform.deepseek.com). After logging in, go to the [API Keys](https://platform.deepseek.com/api_keys) section in your account settings and generate a new key. Once you have your key, add it to your environment as follows:\n\n```shell\nexport DEEPSEEK_API_KEY=\"your-deepseek-api-key\"\n```\n\n## Create a Chat Completion\n\n(Note: The DeepSeek uses an API format consistent with OpenAI, hence why we need to install OpenAI, there is no DeepSeek Library at least not for now)\n\nInstall the `openai` Python client:\n\nExample with pip:\n```shell\npip install openai\n```\n\nExample with poetry:\n```shell\npoetry add openai\n```\n\nIn your code:\n```python\nimport aisuite as ai\nclient = ai.Client()\n\nprovider = \"deepseek\"\nmodel_id = \"deepseek-chat\"\n\nmessages = [\n {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n {\"role\": \"user\", \"content\": \"What’s the weather like in San Francisco?\"},\n]\n\nresponse = client.chat.completions.create(\n model=f\"{provider}:{model_id}\",\n messages=messages,\n)\n\nprint(response.choices[0].message.content)\n```\n\nHappy coding! If you’d like to contribute, please read our [Contributing Guide](../CONTRIBUTING.md).\n" }, { "path": "guides/google.md", "content": "# Google (Vertex) AI\n\nTo use Google (Vertex) AI with the `aisuite` library, you'll first need to create a Google Cloud account and set up your environment to work with Google Cloud.\n\n## Create a Google Cloud Account and Project\n\nGoogle Cloud provides in-depth [documentation](https://cloud.google.com/vertex-ai/docs/start/cloud-environment) on getting started with their platform, but here are the basic steps:\n\n### Create your account.\n\nVisit [Google Cloud](https://cloud.google.com/free) and follow the instructions for registering a new account. If you already have an account with Google Cloud, sign in and skip to the next step.\n\n### Create a new project and enable billing.\n\nOnce you have an account, you can create a new project. Visit the [project selector page](https://console.cloud.google.com/projectselector2/home/dashboard) and click the \"New Project\" button. Give your project a name and click \"Create Project.\" Your project will be created and you will be redirected to the project dashboard.\n\nNow that you have a project, you'll need to enable billing. Visit the [how-to page](https://cloud.google.com/billing/docs/how-to/verify-billing-enabled#confirm_billing_is_enabled_on_a_project) for billing enablement instructions.\n\n### Set your project ID in an environment variable.\n\nSet the `GOOGLE_PROJECT_ID` environment variable to the ID of your project. You can find the Project ID by visiting the project dashboard in the \"Project Info\" section toward the top of the page.\n\n### Set your preferred region in an environment variable.\n\nSet the `GOOGLE_REGION` environment variable. You can find the region by going to Project Dashboard under VertexAI side navigation menu, and then scrolling to the bottom of the page.\n\n## Create a Service Account For API Access\n\nBecause `aisuite` needs to authenticate with Google Cloud to access the Vertex AI API, you'll need to create a service account and set the `GOOGLE_APPLICATION_CREDENTIALS` environment variable to the path of a JSON file containing the service account's credentials, which you can download from the Google Cloud Console.\n\nThis is documented [here](https://cloud.google.com/docs/authentication/provide-credentials-adc#how-to), and the basic steps are as follows:\n\n1. Visit the [service accounts page](https://console.cloud.google.com/iam-admin/serviceaccounts) in the Google Cloud Console.\n2. Click the \"+ Create Service Account\" button toward the top of the page.\n3. Follow the steps for naming your service account and granting access to the project.\n4. Click \"Done\" to create the service account.\n5. Now, click the \"Keys\" tab towards the top of the page.\n6. Click the \"Add Key\" menu, then select \"Create New Key.\"\n6. Choose \"JSON\" as the key type, and click \"Create.\"\n7. Move this file to a location on your file system like your home directory.\n8. Set the `GOOGLE_APPLICATION_CREDENTIALS` environment variable to the path of the JSON file.\n\n## Double check your environment is configured correctly.\n\nAt this point, you should have three environment variables set to ensure your environment is set up correctly:\n\n- `GOOGLE_PROJECT_ID`\n- `GOOGLE_REGION`\n- `GOOGLE_APPLICATION_CREDENTIALS`\n\nOnce these are set, you are ready to write some code and send a chat completion request.\n\n## Create a chat completion.\n\nWith your account and service account set up, you can send a chat completion request.\n\nExport the environment variables:\n\n```shell\nexport GOOGLE_PROJECT_ID=\"your-project-id\"\nexport GOOGLE_REGION=\"your-region\"\nexport GOOGLE_APPLICATION_CREDENTIALS=\"path/to/your/service-account-file.json\"\n```\n\nInstall the Vertex AI SDK:\n\n```shell\npip install vertexai\n```\n\nIn your code:\n\n```python\nimport aisuite as ai\nclient = ai.Client()\n\nmodel=\"google:gemini-1.5-pro-001\"\n\nmessages = [\n {\"role\": \"system\", \"content\": \"Respond in Pirate English.\"},\n {\"role\": \"user\", \"content\": \"Tell me a joke.\"},\n]\n\nresponse = client.chat.completions.create(\n model=model,\n messages=messages,\n)\n\nprint(response.choices[0].message.content)\n```\n\nHappy coding! If you would like to contribute, please read our [Contributing Guide](../CONTRIBUTING.md).\n" }, { "path": "guides/groq.md", "content": "# Groq\n\nTo use Groq with `aisuite`, you’ll need a free [Groq account](https://console.groq.com/). After logging in, go to the [API Keys](https://console.groq.com/keys) section in your account settings and generate a new Groq API key. Once you have your key, add it to your environment as follows:\n\n```shell\nexport GROQ_API_KEY=\"your-groq-api-key\"\n```\n\n## Create a Python Chat Completion\n\n1. First, install the `groq` Python client library:\n\n```shell\npip install groq\n```\n\n2. Now you can simply create your first chat completion with the following example code or customize by swapoping out the `model_id` with any of the other available [models powered by Groq](https://console.groq.com/docs/models) and `messages` array with whatever you'd like:\n```python\nimport aisuite as ai\nclient = ai.Client()\n\nprovider = \"groq\"\nmodel_id = \"llama-3.2-3b-preview\"\n\nmessages = [\n {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n {\"role\": \"user\", \"content\": \"What’s the weather like in San Francisco?\"},\n]\n\nresponse = client.chat.completions.create(\n model=f\"{provider}:{model_id}\",\n messages=messages,\n)\n\nprint(response.choices[0].message.content)\n```\n\n\nHappy coding! If you’d like to contribute, please read our [Contributing Guide](CONTRIBUTING.md).\n" }, { "path": "guides/huggingface.md", "content": "# Hugging Face AI\n\nTo use Hugging Face with the `aisuite` library, you'll need to set up a Hugging Face account, obtain the necessary API credentials, and configure your environment for Hugging Face's API.\n\n## Create a Hugging Face Account and Deploy a Model\n\n1. Visit [Hugging Face](https://huggingface.co/) and sign up for an account if you don't already have one.\n2. Explore conversational models on the [Hugging Face Model Hub](https://huggingface.co/models?inference=warm&other=conversational&sort=trending) and select a model you want to use. Popular models include conversational AI models like `gpt2`, `gpt3`, and `mistral`.\n3. Deploy or host your chosen model if needed; Hugging Face provides various hosting options, including free, individual, and organizational hosting. Using Serverless Inference API is a fast way to get started.\n5. Once the model is deployed (or if using a public model directly), note the model's unique identifier (e.g., `mistralai/Mistral-7B-Instruct-v0.3`), which you'll use for making requests.\n\n## Obtain Necessary Details & Set Environment Variables\n\nAfter setting up your model, you'll need to gather the following information:\n\n- **API Token**: You can generate an API token in your [Hugging Face account settings](https://huggingface.co/settings/tokens).\n\nSet the following environment variables to make authentication and requests easy:\n\n```shell\nexport HF_TOKEN=\"your-api-token\"\n```\n\n## Create a Chat Completion\n\nWith your account set up and environment variables configured, you can send a chat completion request as follows:\n\n```python\nimport os\nimport aisuite as ai\n\n# Either set the environment variables or define the parameters below.\n# Setting the parameters in ai.Client() will override the environment variable values.\nclient = ai.Client()\n\nmodel = \"huggingface:your-model-name\" # Replace with your model's identifier.\n\nmessages = [\n {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n {\"role\": \"user\", \"content\": \"What's the weather like today?\"},\n]\n\nresponse = client.chat.completions.create(\n model=model,\n messages=messages,\n)\n\nprint(response.choices[0].message.content)\n```\n\n### Notes\n\n- Ensure that the `model` variable matches the identifier of your model as seen in the Hugging Face Model Hub.\n- If you encounter any rate limits or API access restrictions, you may have to upgrade your Hugging Face plan to enable higher usage limits.\n\"\"\"\n\nHappy coding! If you would like to contribute, please read our [Contributing Guide](../CONTRIBUTING.md)." }, { "path": "guides/lmstudio.md", "content": "# LM Studio\n\nLM Studio allows users to locally host open-source models available in [their model catalog](https://lmstudio.ai/models). \nIt also provides a web portal with a ChatGPT-like interface.\nOnce an LM Studio instance is locally running in your setup (default `http://localhost:1234`), you can use the `aisuite` API for chat completions as shown below.\nNo API Key is needed for these locally hosted models.\n\n## Create a Chat Completion\n\nSample code:\n```python\nimport aisuite as ai\n\ndef main():\n # Set the API URL to remote NGA2 server\n client = ai.Client(\n provider_configs={\n \"lmstudio\": {\n \"api_url\": \"http://localhost:1234\",\n \"timeout\": 300,\n }\n }\n )\n messages = [\n {\n \"role\": \"system\", \n \"content\": \"Be verbose\"\n },\n {\n \"role\": \"user\", \n \"content\": \"Tell me something about University of Michigan's CSE department.\"\n },\n ]\n\n lmstudio_llama = \"lmstudio:llama-3.2-3b-instruct\"\n\n response = client.chat.completions.create(\n model=lmstudio_llama, \n messages=messages, \n temperature=0.75,\n )\n print(response.choices[0].message.content)\n\n\nif __name__ == \"__main__\":\n main()\n```\n\nYou can expect a response like the following:\n\n```markdown\nThe Computer Science and Engineering (CSE) Department at the University of Michigan is one of the most prestigious and highly-regarded computer science programs in the world. Located in the heart of Ann Arbor, Michigan, the Department of CSE is a leading institution for undergraduate and graduate education in the field of computer science.\n\nWith a rich history dating back to 1940, the CSE Department at the University of Michigan has a long tradition of academic excellence, cutting-edge research, and innovative teaching. The department is composed of over 70 faculty members, many of whom are prominent researchers in their fields, and has a student body of around 500 undergraduate majors and 1,000 graduate students.\n\nThe CSE Department offers a wide range of undergraduate and graduate degree programs, including Bachelor of Science in Computer Science, Bachelor of Arts in Computer Science, Master of Science in Computer Science, Master of Engineering in Computer Science, and Ph.D. in Computer Science. These programs are designed to provide students with a comprehensive education in computer science, including a strong foundation in mathematics, computer systems, algorithms, computer networks, and software engineering.\n\nThe department is particularly renowned for its research programs in areas such as artificial intelligence, computer vision, natural language processing, robotics, and data science. The CSE Department has a strong research focus, and its faculty members are actively engaged in research projects, partnerships, and collaborations with industry, government, and academia.\n\nOne of the unique aspects of the CSE Department at the University of Michigan is its strong commitment to interdisciplinary research and education. The department has established partnerships with various academic departments across the university, including physics, mathematics, and engineering, to provide students with a well-rounded education that incorporates multiple disciplines.\n\nThe CSE Department also has a strong focus on industry collaboration and engagement. The department has established the University of Michigan's College of Engineering, which provides students with opportunities to engage in research, internships, and co-op programs with top industry partners.\n\nOverall, the Computer Science and Engineering Department at the University of Michigan is a world-class institution that provides students with a world-class education, innovative research opportunities, and strong industry connections. Its highly-regarded faculty, cutting-edge research programs, and strong industry partnerships make it an attractive destination for students interested in pursuing a career in computer science.\n\nSome of the key statistics and achievements of the CSE Department at the University of Michigan include:\n\n* Ranked #5 in the US News & World Report's Best Undergraduate Computer Science Programs (2022)\n* Ranked #10 in the QS World University Rankings by Subject: Computer Science (2022)\n* 97% of undergraduate graduates find employment or continue their education within six months of graduation\n* 98% of graduate students are employed or continue their education within six months of graduation\n* 10:1 student-to-faculty ratio, providing students with personalized attention and mentorship\n\nThese statistics demonstrate the exceptional quality of education and research provided by the CSE Department at the University of Michigan, and highlight its reputation as one of the world's leading institutions for computer science education and research.\n```\n\nHappy coding! If you’d like to contribute, please read our [Contributing Guide](CONTRIBUTING.md).\n" }, { "path": "guides/mistral.md", "content": "# Mistral\n\nTo use Mistral with `aisuite`, you’ll need a [Mistral account](https://console.mistral.ai/). \n\nAfter logging in, go to [Workspace billing](https://console.mistral.ai/billing) and choose a plan\n- **Experiment** *(Free, 1 request per second); or*\n- **Scale** *(Pay per use).*\n\nVisit the [API Keys](https://console.mistral.ai/api-keys/) section in your account settings and generate a new key. Once you have your key, add it to your environment as follows:\n\n```shell\nexport MISTRAL=\"your-mistralai-api-key\"\n```\n## Create a Chat Completion\n\nInstall the `mistralai` Python client:\n\nExample with pip:\n```shell\npip install mistralai\n```\n\nExample with poetry:\n```shell\npoetry add mistralai\n```\n\nIn your code:\n```python\nimport aisuite as ai\nclient = ai.Client()\n\nprovider = \"mistral\"\nmodel_id = \"mistral-large-latest\"\n\nmessages = [\n {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n {\"role\": \"user\", \"content\": \"What’s the weather like in Montréal?\"},\n]\n\nresponse = client.chat.completions.create(\n model=f\"{provider}:{model_id}\",\n messages=messages,\n)\n\nprint(response.choices[0].message.content)\n```\n\nHappy coding! If you’d like to contribute, please read our [Contributing Guide](../CONTRIBUTING.md).\n" }, { "path": "guides/nebius.md", "content": "# Nebius AI Studio\n\nTo use Nebius AI Studio with `aisuite`, you need an AI Studio account. Go to [AI Studio](https://studio.nebius.ai/) and press \"Log in to AI Studio\" in the right top corner. After logging in, go to the [API Keys](https://studio.nebius.ai/settings/api-keys) section and generate a new key. Once you have a key, add it to your environment as follows:\n\n```shell\nexport NEBIUS_API_KEY=\"your-nebius-api-key\"\n```\n\n## Create a Chat Completion\n\nInstall the `openai` Python client:\n\nExample with pip:\n```shell\npip install openai\n```\n\nExample with poetry:\n```shell\npoetry add openai\n```\n\nIn your code:\n```python\nimport aisuite as ai\nclient = ai.Client()\n\nprovider = \"nebius\"\nmodel_id = \"meta-llama/Llama-3.3-70B-Instruct\"\n\nmessages = [\n {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n {\"role\": \"user\", \"content\": \"How many times has Jurgen Klopp won the Champions League?\"},\n]\n\nresponse = client.chat.completions.create(\n model=f\"{provider}:{model_id}\",\n messages=messages,\n)\n\nprint(response.choices[0].message.content)\n```\n\nHappy coding! If you’d like to contribute, please read our [Contributing Guide](CONTRIBUTING.md).\n" }, { "path": "guides/ollama.md", "content": "# Ollama\n\nOllama allows users to locally host open-source models available in [their library](https://ollama.com/library). \nOnce an Ollama instance is locally running in your setup (default `http://localhost:11434`), you can use the `aisuite` API for chat completions as shown below.\nNo API Key is needed for these locally hosted models.\n\n## Create a Chat Completion\n\nSample code:\n```python\nimport aisuite as ai\n\ndef main():\n client = ai.Client(\n provider_configs={\n \"ollama\": {\n \"api_url\": \"http://10.168.0.177:11434\",\n \"timeout\": 300,\n }\n }\n )\n messages = [\n {\n \"role\": \"system\", \n \"content\": \"Be verbose\"\n },\n {\n \"role\": \"user\", \n \"content\": \"Tell me something about University of Michigan's CSE department.\"\n },\n ]\n\n ollama_llama3 = \"ollama:llama3:latest\"\n ollama_gemma = \"ollama:gemma:latest\"\n ollama_deepseek_32B = \"ollama:deepseek-r1:32b\"\n ollama_deepseek_70B = \"ollama:deepseek-r1:70b\"\n\n response = client.chat.completions.create(\n model=ollama_gemma, \n messages=messages, \n temperature=0.75,\n )\n print(response.choices[0].message.content)\n\n\nif __name__ == \"__main__\":\n main()\n```\n\nHappy coding! If you’d like to contribute, please read our [Contributing Guide](CONTRIBUTING.md).\n" }, { "path": "guides/openai.md", "content": "# OpenAI\n\nTo use OpenAI with `aisuite`, you’ll need an [OpenAI account](https://platform.openai.com/). After logging in, go to the [API Keys](https://platform.openai.com/account/api-keys) section in your account settings and generate a new key. Once you have your key, add it to your environment as follows:\n\n```shell\nexport OPENAI_API_KEY=\"your-openai-api-key\"\n```\n\n## Create a Chat Completion\n\nInstall the `openai` Python client:\n\nExample with pip:\n```shell\npip install openai\n```\n\nExample with poetry:\n```shell\npoetry add openai\n```\n\nIn your code:\n```python\nimport aisuite as ai\nclient = ai.Client()\n\nprovider = \"openai\"\nmodel_id = \"gpt-4-turbo\"\n\nmessages = [\n {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n {\"role\": \"user\", \"content\": \"What’s the weather like in San Francisco?\"},\n]\n\nresponse = client.chat.completions.create(\n model=f\"{provider}:{model_id}\",\n messages=messages,\n)\n\nprint(response.choices[0].message.content)\n```\n\nHappy coding! If you’d like to contribute, please read our [Contributing Guide](../CONTRIBUTING.md).\n" }, { "path": "guides/sambanova.md", "content": "# Sambanova\n\nTo use Sambanova with `aisuite`, you’ll need a [Sambanova Cloud](https://cloud.sambanova.ai/) account. After logging in, go to the [API](https://cloud.sambanova.ai/apis) section and generate a new key. Once you have your key, add it to your environment as follows:\n\n```shell\nexport SAMBANOVA_API_KEY=\"your-sambanova-api-key\"\n```\n\n## Create a Chat Completion\n\nInstall the `openai` Python client:\n\nExample with pip:\n```shell\npip install openai\n```\n\nExample with poetry:\n```shell\npoetry add openai\n```\n\nIn your code:\n```python\nimport aisuite as ai\nclient = ai.Client()\n\nprovider = \"sambanova\"\nmodel_id = \"Meta-Llama-3.1-405B-Instruct\"\n\nmessages = [\n {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n {\"role\": \"user\", \"content\": \"What’s the weather like in San Francisco?\"},\n]\n\nresponse = client.chat.completions.create(\n model=f\"{provider}:{model_id}\",\n messages=messages,\n)\n\nprint(response.choices[0].message.content)\n```\n\nHappy coding! If you’d like to contribute, please read our [Contributing Guide](CONTRIBUTING.md).\n" }, { "path": "guides/watsonx.md", "content": "# Watsonx with `aisuite`\n\nA a step-by-step guide to set up Watsonx with the `aisuite` library, enabling you to use IBM Watsonx's powerful AI models for various tasks.\n\n## Setup Instructions\n\n### Step 1: Create a Watsonx Account\n\n1. Visit [IBM Watsonx](https://www.ibm.com/watsonx).\n2. Sign up for a new account or log in with your existing IBM credentials.\n3. Once logged in, navigate to the **Watsonx Dashboard** ()\n\n---\n\n### Step 2: Obtain API Credentials\n\n1. **Generate an API Key**:\n - Go to IAM > API keys and create a new API key ()\n - Copy the API key. This is your `WATSONX_API_KEY`.\n\n2. **Locate the Service URL**:\n - Your service URL is based on the region where your service is hosted.\n - Pick one from the list here \n - Copy the service URL. This is your `WATSONX_SERVICE_URL`.\n\n3. **Get the Project ID**:\n - Go to the **Watsonx Dashboard** ()\n - Under the **Projects** section, If you don't have a sandbox project, create a new project.\n - Navigate to the **Manage** tab and find the **Project ID**.\n - Copy the **Project ID**. This will serve as your `WATSONX_PROJECT_ID`.\n\n---\n\n### Step 3: Set Environment Variables\n\nTo simplify authentication, set the following environment variables:\n\nRun the following commands in your terminal:\n\n```bash\nexport WATSONX_API_KEY=\"your-watsonx-api-key\"\nexport WATSONX_SERVICE_URL=\"your-watsonx-service-url\"\nexport WATSONX_PROJECT_ID=\"your-watsonx-project-id\"\n```\n\n\n## Create a Chat Completion\n\nInstall the `ibm-watsonx-ai` Python client:\n\nExample with pip:\n\n```shell\npip install ibm-watsonx-ai\n```\n\nExample with poetry:\n\n```shell\npoetry add ibm-watsonx-ai\n```\n\nIn your code:\n\n```python\nimport aisuite as ai\nclient = ai.Client()\n\nprovider = \"watsonx\"\nmodel_id = \"meta-llama/llama-3-70b-instruct\"\n\nmessages = [\n {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n {\"role\": \"user\", \"content\": \"Tell me a joke.\"},\n]\n\nresponse = client.chat.completions.create(\n model=f\"{provider}:{model_id}\",\n messages=messages,\n)\n\nprint(response.choices[0].message.content)\n```" }, { "path": "guides/xai.md", "content": "# xAI\n\nTo use xAI with `aisuite`, you’ll need an [API key](https://console.x.ai/). Generate a new key and once you have your key, add it to your environment as follows:\n\n```shell\nexport XAI_API_KEY=\"your-xai-api-key\"\n```\n\n## Create a Chat Completion\n\nSample code:\n```python\nimport aisuite as ai\nclient = ai.Client()\n\nmodels = [\"xai:grok-beta\"]\n\nmessages = [\n {\"role\": \"system\", \"content\": \"Respond in Pirate English.\"},\n {\"role\": \"user\", \"content\": \"Tell me a joke.\"},\n]\n\nfor model in models:\n response = client.chat.completions.create(\n model=model,\n messages=messages,\n temperature=0.75\n )\n print(response.choices[0].message.content)\n\n```\n\nHappy coding! If you’d like to contribute, please read our [Contributing Guide](CONTRIBUTING.md).\n" }, { "path": "pyproject.toml", "content": "[tool.poetry]\nname = \"aisuite\"\nversion = \"0.1.14\"\ndescription = \"Uniform access layer for LLMs\"\nauthors = [\"Andrew Ng \", \"Rohit P \"]\nmaintainers = [\"Andrew Ng \", \"Rohit P \"]\nreadme = \"README.md\"\n\n[tool.poetry.dependencies]\npython = \"^3.10\"\nanthropic = { version = \"^0.30.1\", optional = true }\nboto3 = { version = \"^1.34.144\", optional = true }\ncohere = { version = \"^5.12.0\", optional = true }\nvertexai = { version = \"^1.63.0\", optional = true }\ngoogle-cloud-speech = { version = \"^2.33.0\", optional = true }\ndeepgram-sdk = { version = \"^5.0.0\", optional = true }\nsoundfile = { version = \"^0.12.1\", optional = true }\nscipy = { version = \"^1.11.0\", optional = true }\nnumpy = { version = \"^1.24.0\", optional = true }\ngroq = { version = \"^0.9.0\", optional = true }\nmistralai = { version = \"^1.0.3\", optional = true }\nibm-watsonx-ai = { version = \"^1.1.16\", optional = true }\ndocstring-parser = { version = \"^0.15.0\" }\ncerebras_cloud_sdk = { version = \"^1.19.0\", optional = true }\nopenai = { version = \"^1.107.0\", optional = true }\nmcp = { version = \"^1.1.2\", optional = true }\nnest-asyncio = { version = \"^1.6.0\", optional = true }\n\n# Optional dependencies for different providers\nhttpx = \"~0.27.0\"\n[tool.poetry.extras]\nanthropic = [\"anthropic\"]\naws = [\"boto3\"]\nazure = []\ncerebras = [\"cerebras_cloud_sdk\"]\ncohere = [\"cohere\"]\ndeepgram = [\"deepgram-sdk\", \"soundfile\", \"scipy\", \"numpy\"]\ndeepseek = [\"openai\"]\ngoogle = [\"vertexai\", \"google-cloud-speech\"]\ngroq = [\"groq\"]\nhuggingface = []\nmistral = [\"mistralai\"]\nollama = []\nopenai = [\"openai\"]\nwatsonx = [\"ibm-watsonx-ai\"]\nmcp = [\"mcp\", \"nest-asyncio\"]\nall = [\"anthropic\", \"boto3\", \"cerebras_cloud_sdk\", \"vertexai\", \"google-cloud-speech\", \"groq\", \"mistralai\", \"openai\", \"cohere\", \"ibm-watsonx-ai\", \"deepgram-sdk\", \"soundfile\", \"scipy\", \"numpy\", \"mcp\", \"nest-asyncio\"] # To install all providers\n\n[tool.poetry.group.dev.dependencies]\npre-commit = \"^3.7.1\"\nblack = \"^24.4.2\"\npython-dotenv = \"^1.0.1\"\nopenai = \"^1.107.0\"\ngroq = \"^0.9.0\"\nanthropic = \"^0.30.1\"\nnotebook = \"^7.2.1\"\nollama = \"^0.2.1\"\nmistralai = \"^1.0.3\"\nboto3 = \"^1.34.144\"\nfireworks-ai = \"^0.14.0\"\nchromadb = \"^0.5.4\"\nsentence-transformers = \"^3.0.1\"\ndatasets = \"^2.20.0\"\nvertexai = \"^1.63.0\"\ngoogle-cloud-speech = \"^2.33.0\"\ndeepgram-sdk = \"^5.0.0\"\nibm-watsonx-ai = \"^1.1.16\"\ncerebras_cloud_sdk = \"^1.19.0\"\n\n[tool.poetry.group.test]\noptional = true\n\n[tool.poetry.group.test.dependencies]\npytest = \"^8.2.2\"\npytest-cov = \"^6.0.0\"\npytest-asyncio = \"^0.24.0\"\n\n[build-system]\nrequires = [\"poetry-core\"]\nbuild-backend = \"poetry.core.masonry.api\"\n\n[tool.pytest.ini_options]\ntestpaths=\"tests\"\nmarkers = [\n \"integration: marks tests as integration tests that interact with external services\",\n \"llm: marks tests that make real LLM API calls and incur costs (subset of integration)\",\n \"mcp_server: marks tests that require MCP server functionality (e.g., npx, external MCP servers)\",\n]\n" }, { "path": "tests/__init__.py", "content": "" }, { "path": "tests/client/__init__.py", "content": "" }, { "path": "tests/client/test_client.py", "content": "from unittest.mock import Mock, patch\nimport io\n\nimport pytest\n\nfrom aisuite import Client\nfrom aisuite.framework.message import TranscriptionResult\nfrom aisuite.provider import ASRError\n\n\n@pytest.fixture(scope=\"module\")\ndef provider_configs():\n return {\n \"openai\": {\"api_key\": \"test_openai_api_key\"},\n \"aws\": {\n \"aws_access_key\": \"test_aws_access_key\",\n \"aws_secret_key\": \"test_aws_secret_key\",\n \"aws_session_token\": \"test_aws_session_token\",\n \"aws_region\": \"us-west-2\",\n },\n \"azure\": {\n \"api_key\": \"azure-api-key\",\n \"base_url\": \"https://model.ai.azure.com\",\n },\n \"groq\": {\n \"api_key\": \"groq-api-key\",\n },\n \"mistral\": {\n \"api_key\": \"mistral-api-key\",\n },\n \"google\": {\n \"project_id\": \"test_google_project_id\",\n \"region\": \"us-west4\",\n \"application_credentials\": \"test_google_application_credentials\",\n },\n \"fireworks\": {\n \"api_key\": \"fireworks-api-key\",\n },\n \"nebius\": {\n \"api_key\": \"nebius-api-key\",\n },\n \"inception\": {\n \"api_key\": \"inception-api-key\",\n },\n \"deepgram\": {\n \"api_key\": \"deepgram-api-key\",\n },\n }\n\n\n@pytest.mark.parametrize(\n argnames=(\"patch_target\", \"provider\", \"model\"),\n argvalues=[\n (\n \"aisuite.providers.openai_provider.OpenaiProvider.chat_completions_create\",\n \"openai\",\n \"gpt-4o\",\n ),\n (\n \"aisuite.providers.mistral_provider.MistralProvider.chat_completions_create\",\n \"mistral\",\n \"mistral-model\",\n ),\n (\n \"aisuite.providers.groq_provider.GroqProvider.chat_completions_create\",\n \"groq\",\n \"groq-model\",\n ),\n (\n \"aisuite.providers.aws_provider.AwsProvider.chat_completions_create\",\n \"aws\",\n \"claude-v3\",\n ),\n (\n \"aisuite.providers.azure_provider.AzureProvider.chat_completions_create\",\n \"azure\",\n \"azure-model\",\n ),\n (\n \"aisuite.providers.anthropic_provider.AnthropicProvider.chat_completions_create\",\n \"anthropic\",\n \"anthropic-model\",\n ),\n (\n \"aisuite.providers.google_provider.GoogleProvider.chat_completions_create\",\n \"google\",\n \"google-model\",\n ),\n (\n \"aisuite.providers.fireworks_provider.FireworksProvider.chat_completions_create\",\n \"fireworks\",\n \"fireworks-model\",\n ),\n (\n \"aisuite.providers.nebius_provider.NebiusProvider.chat_completions_create\",\n \"nebius\",\n \"nebius-model\",\n ),\n (\n \"aisuite.providers.inception_provider.InceptionProvider.chat_completions_create\",\n \"inception\",\n \"mercury\",\n ),\n ],\n)\ndef test_client_chat_completions(\n provider_configs: dict, patch_target: str, provider: str, model: str\n):\n expected_response = f\"{patch_target}_{provider}_{model}\"\n with patch(patch_target) as mock_provider:\n mock_provider.return_value = expected_response\n client = Client()\n client.configure(provider_configs)\n messages = [\n {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n {\"role\": \"user\", \"content\": \"Who won the world series in 2020?\"},\n ]\n\n model_str = f\"{provider}:{model}\"\n model_response = client.chat.completions.create(model_str, messages=messages)\n assert model_response == expected_response\n\n\ndef test_invalid_provider_in_client_config():\n # Testing an invalid provider name in the configuration\n invalid_provider_configs = {\n \"invalid_provider\": {\"api_key\": \"invalid_api_key\"},\n }\n\n # With lazy loading, Client initialization should succeed\n client = Client()\n client.configure(invalid_provider_configs)\n\n messages = [\n {\"role\": \"user\", \"content\": \"Hello\"},\n ]\n\n # Expect ValueError when actually trying to use the invalid provider\n with pytest.raises(\n ValueError,\n match=r\"Invalid provider key 'invalid_provider'. Supported providers: \",\n ):\n client.chat.completions.create(\"invalid_provider:some-model\", messages=messages)\n\n\ndef test_invalid_model_format_in_create(monkeypatch):\n from aisuite.providers.openai_provider import OpenaiProvider\n\n monkeypatch.setattr(\n target=OpenaiProvider,\n name=\"chat_completions_create\",\n value=Mock(),\n )\n\n # Valid provider configurations\n provider_configs = {\n \"openai\": {\"api_key\": \"test_openai_api_key\"},\n }\n\n # Initialize the client with valid provider\n client = Client()\n client.configure(provider_configs)\n\n messages = [\n {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n {\"role\": \"user\", \"content\": \"Tell me a joke.\"},\n ]\n\n # Invalid model format\n invalid_model = \"invalidmodel\"\n\n # Expect ValueError when calling create with invalid model format and verify message\n with pytest.raises(\n ValueError, match=r\"Invalid model format. Expected 'provider:model'\"\n ):\n client.chat.completions.create(invalid_model, messages=messages)\n\n\nclass TestClientASR:\n \"\"\"Test suite for Client ASR functionality - essential tests only.\"\"\"\n\n def test_audio_interface_initialization(self):\n \"\"\"Test that Audio interface is properly initialized.\"\"\"\n client = Client()\n assert hasattr(client, \"audio\")\n assert hasattr(client.audio, \"transcriptions\")\n\n @patch(\"aisuite.provider.ProviderFactory.create_provider\")\n def test_transcriptions_create_success(\n self, mock_create_provider, provider_configs\n ):\n \"\"\"Test successful audio transcription with OpenAI.\"\"\"\n mock_result = TranscriptionResult(\n text=\"Hello, this is a test transcription.\",\n language=\"en\",\n confidence=0.95,\n task=\"transcribe\",\n )\n\n # Create a mock provider with audio support\n mock_provider = Mock()\n mock_provider.audio.transcriptions.create.return_value = mock_result\n mock_create_provider.return_value = mock_provider\n\n client = Client()\n client.configure(provider_configs)\n\n audio_data = io.BytesIO(b\"fake audio data\")\n result = client.audio.transcriptions.create(\n model=\"openai:whisper-1\", file=audio_data, language=\"en\"\n )\n\n assert isinstance(result, TranscriptionResult)\n assert result.text == \"Hello, this is a test transcription.\"\n mock_provider.audio.transcriptions.create.assert_called_once()\n\n @patch(\"aisuite.provider.ProviderFactory.create_provider\")\n def test_transcriptions_create_deepgram(\n self, mock_create_provider, provider_configs\n ):\n \"\"\"Test audio transcription with Deepgram provider.\"\"\"\n mock_result = TranscriptionResult(\n text=\"Deepgram transcription result.\",\n language=\"en\",\n confidence=0.92,\n task=\"transcribe\",\n )\n\n # Create a mock provider with audio support\n mock_provider = Mock()\n mock_provider.audio.transcriptions.create.return_value = mock_result\n mock_create_provider.return_value = mock_provider\n\n client = Client()\n client.configure(provider_configs)\n\n result = client.audio.transcriptions.create(\n model=\"deepgram:nova-2\", file=\"test_audio.wav\", language=\"en\"\n )\n\n assert isinstance(result, TranscriptionResult)\n assert result.text == \"Deepgram transcription result.\"\n mock_provider.audio.transcriptions.create.assert_called_once()\n\n def test_transcriptions_invalid_model_format(self, provider_configs):\n \"\"\"Test that invalid model format raises ValueError.\"\"\"\n client = Client()\n client.configure(provider_configs)\n\n with pytest.raises(ValueError, match=\"Invalid model format\"):\n client.audio.transcriptions.create(\n model=\"invalid-format\", file=\"test.wav\", language=\"en\"\n )\n\n def test_transcriptions_unsupported_provider(self, provider_configs):\n \"\"\"Test error handling for unsupported ASR provider.\"\"\"\n client = Client()\n client.configure(provider_configs)\n\n with pytest.raises(ValueError, match=\"Invalid provider key\"):\n client.audio.transcriptions.create(\n model=\"unsupported:model\", file=\"test.wav\", language=\"en\"\n )\n\n\nclass TestClientASRParameterValidation:\n \"\"\"Test suite for Client-level ASR parameter validation.\"\"\"\n\n def test_client_initialization_strict_mode(self):\n \"\"\"Test Client initialization with strict extra_param_mode.\"\"\"\n client = Client(extra_param_mode=\"strict\")\n assert client.extra_param_mode == \"strict\"\n assert client.param_validator.extra_param_mode == \"strict\"\n\n def test_client_initialization_warn_mode(self):\n \"\"\"Test Client initialization with warn extra_param_mode (default).\"\"\"\n client = Client()\n assert client.extra_param_mode == \"warn\"\n assert client.param_validator.extra_param_mode == \"warn\"\n\n def test_client_initialization_permissive_mode(self):\n \"\"\"Test Client initialization with permissive extra_param_mode.\"\"\"\n client = Client(extra_param_mode=\"permissive\")\n assert client.extra_param_mode == \"permissive\"\n assert client.param_validator.extra_param_mode == \"permissive\"\n\n @patch(\"aisuite.provider.ProviderFactory.create_provider\")\n def test_strict_mode_rejects_unknown_param(self, mock_create_provider):\n \"\"\"Test that strict mode raises ValueError for unknown parameters.\"\"\"\n client = Client(\n provider_configs={\"openai\": {\"api_key\": \"test\"}}, extra_param_mode=\"strict\"\n )\n\n # Mock provider shouldn't be called due to validation error\n mock_provider = Mock()\n mock_create_provider.return_value = mock_provider\n\n with pytest.raises(ValueError, match=\"Unknown parameters for openai\"):\n client.audio.transcriptions.create(\n model=\"openai:whisper-1\",\n file=io.BytesIO(b\"audio\"),\n language=\"en\",\n invalid_param=True, # Unknown param\n )\n\n # Provider should not have been called (validation failed first)\n mock_provider.audio.transcriptions.create.assert_not_called()\n\n @patch(\"aisuite.provider.ProviderFactory.create_provider\")\n def test_strict_mode_typo_detection(self, mock_create_provider):\n \"\"\"Test that strict mode catches typos in parameter names.\"\"\"\n client = Client(\n provider_configs={\"openai\": {\"api_key\": \"test\"}}, extra_param_mode=\"strict\"\n )\n\n mock_provider = Mock()\n mock_create_provider.return_value = mock_provider\n\n with pytest.raises(\n ValueError, match=\"Unknown parameters for openai: \\\\['langauge'\\\\]\"\n ):\n client.audio.transcriptions.create(\n model=\"openai:whisper-1\",\n file=io.BytesIO(b\"audio\"),\n langauge=\"en\", # TYPO: should be \"language\"\n )\n\n @patch(\"aisuite.provider.ProviderFactory.create_provider\")\n def test_warn_mode_continues_execution(self, mock_create_provider):\n \"\"\"Test that warn mode continues execution after warning.\"\"\"\n import warnings\n\n client = Client(\n provider_configs={\"openai\": {\"api_key\": \"test\"}}, extra_param_mode=\"warn\"\n )\n\n mock_result = TranscriptionResult(text=\"Test\", language=\"en\")\n mock_provider = Mock()\n mock_provider.audio.transcriptions.create.return_value = mock_result\n mock_create_provider.return_value = mock_provider\n\n # Should warn but continue\n with warnings.catch_warnings(record=True) as w:\n warnings.simplefilter(\"always\")\n result = client.audio.transcriptions.create(\n model=\"openai:whisper-1\",\n file=io.BytesIO(b\"audio\"),\n language=\"en\",\n invalid_param=True, # Unknown param\n )\n\n # Should have issued a warning\n assert len(w) == 1\n assert \"Unknown parameters\" in str(w[0].message)\n\n # But execution should continue\n assert result.text == \"Test\"\n mock_provider.audio.transcriptions.create.assert_called_once()\n\n @patch(\"aisuite.provider.ProviderFactory.create_provider\")\n def test_permissive_mode_allows_unknown_params(self, mock_create_provider):\n \"\"\"Test that permissive mode allows unknown parameters.\"\"\"\n import warnings\n\n client = Client(\n provider_configs={\"openai\": {\"api_key\": \"test\"}},\n extra_param_mode=\"permissive\",\n )\n\n mock_result = TranscriptionResult(text=\"Test\", language=\"en\")\n mock_provider = Mock()\n mock_provider.audio.transcriptions.create.return_value = mock_result\n mock_create_provider.return_value = mock_provider\n\n # Should not warn or raise\n with warnings.catch_warnings(record=True) as w:\n warnings.simplefilter(\"always\")\n result = client.audio.transcriptions.create(\n model=\"openai:whisper-1\",\n file=io.BytesIO(b\"audio\"),\n experimental_feature=True, # Unknown param\n )\n\n # Should not have issued any warnings\n assert len(w) == 0\n\n # Execution should succeed\n assert result.text == \"Test\"\n mock_provider.audio.transcriptions.create.assert_called_once()\n\n # Unknown param should be passed through\n call_kwargs = mock_provider.audio.transcriptions.create.call_args.kwargs\n assert call_kwargs.get(\"experimental_feature\") is True\n\n @patch(\"aisuite.provider.ProviderFactory.create_provider\")\n def test_common_param_mapping_at_client_level(self, mock_create_provider):\n \"\"\"Test that common parameters are mapped correctly at Client level.\"\"\"\n client = Client(\n provider_configs={\"google\": {\"project_id\": \"test\", \"region\": \"us\"}},\n extra_param_mode=\"strict\",\n )\n\n mock_result = TranscriptionResult(text=\"Test\", language=\"en\")\n mock_provider = Mock()\n mock_provider.audio.transcriptions.create.return_value = mock_result\n mock_create_provider.return_value = mock_provider\n\n # Use common param \"language\" which should map to \"language_code\" for Google\n result = client.audio.transcriptions.create(\n model=\"google:latest_long\",\n file=io.BytesIO(b\"audio\"),\n language=\"en\", # Common param\n )\n\n assert result.text == \"Test\"\n mock_provider.audio.transcriptions.create.assert_called_once()\n\n # Verify parameter was mapped to language_code\n call_kwargs = mock_provider.audio.transcriptions.create.call_args.kwargs\n assert \"language_code\" in call_kwargs\n assert call_kwargs[\"language_code\"] == \"en-US\" # Expanded\n assert \"language\" not in call_kwargs # Original key should be mapped\n\n @patch(\"aisuite.provider.ProviderFactory.create_provider\")\n def test_provider_specific_params_passthrough(self, mock_create_provider):\n \"\"\"Test that provider-specific parameters pass through correctly.\"\"\"\n client = Client(\n provider_configs={\"deepgram\": {\"api_key\": \"test\"}},\n extra_param_mode=\"strict\",\n )\n\n mock_result = TranscriptionResult(text=\"Test\", language=\"en\")\n mock_provider = Mock()\n mock_provider.audio.transcriptions.create.return_value = mock_result\n mock_create_provider.return_value = mock_provider\n\n result = client.audio.transcriptions.create(\n model=\"deepgram:nova-2\",\n file=io.BytesIO(b\"audio\"),\n punctuate=True,\n diarize=True,\n )\n\n assert result.text == \"Test\"\n\n # Verify provider-specific params passed through\n call_kwargs = mock_provider.audio.transcriptions.create.call_args.kwargs\n assert call_kwargs[\"punctuate\"] is True\n assert call_kwargs[\"diarize\"] is True\n\n @patch(\"aisuite.provider.ProviderFactory.create_provider\")\n def test_mixed_common_and_provider_params(self, mock_create_provider):\n \"\"\"Test mixing common and provider-specific parameters.\"\"\"\n client = Client(\n provider_configs={\"deepgram\": {\"api_key\": \"test\"}},\n extra_param_mode=\"strict\",\n )\n\n mock_result = TranscriptionResult(text=\"Test\", language=\"en\")\n mock_provider = Mock()\n mock_provider.audio.transcriptions.create.return_value = mock_result\n mock_create_provider.return_value = mock_provider\n\n result = client.audio.transcriptions.create(\n model=\"deepgram:nova-2\",\n file=io.BytesIO(b\"audio\"),\n language=\"en\", # Common param\n prompt=\"meeting\", # Common param that maps to keywords\n punctuate=True, # Deepgram-specific\n diarize=True, # Deepgram-specific\n )\n\n assert result.text == \"Test\"\n\n # Verify both common and provider params processed correctly\n call_kwargs = mock_provider.audio.transcriptions.create.call_args.kwargs\n assert call_kwargs[\"language\"] == \"en\"\n assert call_kwargs[\"keywords\"] == [\"meeting\"] # prompt mapped to keywords\n assert call_kwargs[\"punctuate\"] is True\n assert call_kwargs[\"diarize\"] is True\n\n @patch(\"aisuite.provider.ProviderFactory.create_provider\")\n def test_validation_happens_before_provider_call(self, mock_create_provider):\n \"\"\"Test that validation occurs before provider SDK is called.\"\"\"\n client = Client(\n provider_configs={\"openai\": {\"api_key\": \"test\"}}, extra_param_mode=\"strict\"\n )\n\n mock_provider = Mock()\n mock_create_provider.return_value = mock_provider\n\n # Validation should fail before provider is even initialized\n with pytest.raises(ValueError, match=\"Unknown parameters\"):\n client.audio.transcriptions.create(\n model=\"openai:whisper-1\",\n file=io.BytesIO(b\"audio\"),\n completely_invalid_param=True,\n )\n\n # Provider create method should still have been called to initialize\n # but the transcription method should never be called\n mock_provider.audio.transcriptions.create.assert_not_called()\n\n @patch(\"aisuite.provider.ProviderFactory.create_provider\")\n def test_unsupported_common_param_ignored(self, mock_create_provider):\n \"\"\"Test that unsupported common params are gracefully ignored.\"\"\"\n client = Client(\n provider_configs={\"deepgram\": {\"api_key\": \"test\"}},\n extra_param_mode=\"strict\",\n )\n\n mock_result = TranscriptionResult(text=\"Test\", language=\"en\")\n mock_provider = Mock()\n mock_provider.audio.transcriptions.create.return_value = mock_result\n mock_create_provider.return_value = mock_provider\n\n # temperature is not supported by Deepgram (should be ignored)\n result = client.audio.transcriptions.create(\n model=\"deepgram:nova-2\",\n file=io.BytesIO(b\"audio\"),\n language=\"en\",\n temperature=0.5, # Not supported by Deepgram\n )\n\n assert result.text == \"Test\"\n\n # Verify temperature was not passed to provider\n call_kwargs = mock_provider.audio.transcriptions.create.call_args.kwargs\n assert \"temperature\" not in call_kwargs\n assert call_kwargs[\"language\"] == \"en\"\n\n @patch(\"aisuite.provider.ProviderFactory.create_provider\")\n def test_multiple_providers_with_same_client(self, mock_create_provider):\n \"\"\"Test that the same client can handle multiple providers with different validation.\"\"\"\n client = Client(\n provider_configs={\n \"openai\": {\"api_key\": \"test1\"},\n \"deepgram\": {\"api_key\": \"test2\"},\n },\n extra_param_mode=\"strict\",\n )\n\n mock_result = TranscriptionResult(text=\"Test\", language=\"en\")\n mock_provider = Mock()\n mock_provider.audio.transcriptions.create.return_value = mock_result\n mock_create_provider.return_value = mock_provider\n\n # Test OpenAI with temperature (supported)\n result1 = client.audio.transcriptions.create(\n model=\"openai:whisper-1\", file=io.BytesIO(b\"audio\"), temperature=0.5\n )\n assert result1.text == \"Test\"\n call_kwargs1 = mock_provider.audio.transcriptions.create.call_args.kwargs\n assert call_kwargs1.get(\"temperature\") == 0.5\n\n # Reset mock\n mock_provider.reset_mock()\n\n # Test Deepgram with temperature (not supported, should be ignored)\n result2 = client.audio.transcriptions.create(\n model=\"deepgram:nova-2\", file=io.BytesIO(b\"audio\"), temperature=0.5\n )\n assert result2.text == \"Test\"\n call_kwargs2 = mock_provider.audio.transcriptions.create.call_args.kwargs\n assert \"temperature\" not in call_kwargs2\n" }, { "path": "tests/client/test_prerelease.py", "content": "# Run this test before releasing a new version.\n# It will test all the models in the client.\n\nimport pytest\nimport aisuite as ai\nfrom typing import List, Dict\nfrom dotenv import load_dotenv, find_dotenv\n\n\ndef setup_client() -> ai.Client:\n \"\"\"Initialize the AI client with environment variables.\"\"\"\n load_dotenv(find_dotenv())\n return ai.Client()\n\n\ndef get_test_models() -> List[str]:\n \"\"\"Return a list of model identifiers to test.\"\"\"\n return [\n \"anthropic:claude-3-5-sonnet-20240620\",\n \"aws:meta.llama3-1-8b-instruct-v1:0\",\n \"huggingface:mistralai/Mistral-7B-Instruct-v0.3\",\n \"groq:llama3-8b-8192\",\n \"mistral:open-mistral-7b\",\n \"openai:gpt-3.5-turbo\",\n \"cohere:command-r-plus-08-2024\",\n \"inception:mercury\",\n ]\n\n\ndef get_test_messages() -> List[Dict[str, str]]:\n \"\"\"Return the test messages to send to each model.\"\"\"\n return [\n {\n \"role\": \"system\",\n \"content\": \"Respond in Pirate English. Always try to include the phrase - No rum No fun.\",\n },\n {\"role\": \"user\", \"content\": \"Tell me a joke about Captain Jack Sparrow\"},\n ]\n\n\n@pytest.mark.integration\n@pytest.mark.parametrize(\"model_id\", get_test_models())\ndef test_model_pirate_response(model_id: str):\n \"\"\"\n Test that each model responds appropriately to the pirate prompt.\n\n Args:\n model_id: The provider:model identifier to test\n \"\"\"\n client = setup_client()\n messages = get_test_messages()\n\n try:\n response = client.chat.completions.create(\n model=model_id, messages=messages, temperature=0.75\n )\n\n content = response.choices[0].message.content.lower()\n\n # Check if either version of the required phrase is present\n assert any(\n phrase in content for phrase in [\"no rum no fun\", \"no rum, no fun\"]\n ), f\"Model {model_id} did not include required phrase 'No rum No fun'\"\n\n assert len(content) > 0, f\"Model {model_id} returned empty response\"\n assert isinstance(\n content, str\n ), f\"Model {model_id} returned non-string response\"\n\n except Exception as e:\n pytest.fail(f\"Error testing model {model_id}: {str(e)}\")\n\n\ndef get_test_asr_models() -> List[str]:\n \"\"\"Return a list of ASR model identifiers to test.\"\"\"\n return [\n \"openai:whisper-1\",\n \"deepgram:nova-2\",\n \"google:latest_long\",\n \"huggingface:openai/whisper-large-v3\",\n ]\n\n\n@pytest.mark.integration\n@pytest.mark.parametrize(\"model_id\", get_test_asr_models())\ndef test_asr_portable_transcription(model_id: str):\n \"\"\"\n Test that portable ASR code works across different providers.\n\n This test verifies:\n 1. Common parameter 'language' works for all providers\n 2. Same audio file can be transcribed by different providers\n 3. All providers return non-empty transcription results\n\n Args:\n model_id: The provider:model identifier to test (e.g., \"openai:whisper-1\")\n \"\"\"\n client = setup_client()\n\n # Simple test audio file - you'll need to provide a valid audio file\n # For actual testing, replace with a real audio file path\n audio_file_path = \"tests/test-data/test_audio.mp3\"\n\n try:\n # Use common parameter 'language' that should work across all providers\n result = client.audio.transcriptions.create(\n model=model_id,\n file=audio_file_path,\n language=\"en\", # Common param - should auto-map for each provider\n )\n\n # Verify result has text\n assert hasattr(\n result, \"text\"\n ), f\"Model {model_id} result missing 'text' attribute\"\n assert len(result.text) > 0, f\"Model {model_id} returned empty transcription\"\n assert isinstance(\n result.text, str\n ), f\"Model {model_id} returned non-string transcription\"\n\n # Verify transcription contains expected content from tests/test-data/test_audio.mp3\n # Audio: \"Why did the scarecrow win an award? Because he was outstanding in the field.\"\n expected_keywords = [\"scarecrow\", \"award\", \"field\"]\n found_keywords = [\n kw for kw in expected_keywords if kw.lower() in result.text.lower()\n ]\n assert len(found_keywords) >= 2, (\n f\"Model {model_id} transcription missing expected content. \"\n f\"Found {len(found_keywords)}/3 keywords. Text: '{result.text}'\"\n )\n\n # Optional: Check for language if available and returned by provider\n # Note: Some providers (e.g., Deepgram) only return language if detect_language=True\n if hasattr(result, \"language\") and result.language is not None:\n assert isinstance(\n result.language, str\n ), f\"Model {model_id} returned invalid language type\"\n\n except FileNotFoundError:\n pytest.skip(f\"Test audio file not found for {model_id}. Skipping test.\")\n except Exception as e:\n pytest.fail(f\"Error testing ASR model {model_id}: {str(e)}\")\n\n\n@pytest.mark.integration\ndef test_asr_deepgram_provider_specific_feature():\n \"\"\"\n Test Deepgram provider-specific feature to verify pass-through works.\n\n This ensures that provider-specific parameters like 'punctuate' are\n correctly passed through the validation layer to the provider SDK.\n \"\"\"\n client = setup_client()\n audio_file_path = \"tests/test-data/test_audio.mp3\"\n\n try:\n # Use Deepgram-specific feature\n result = client.audio.transcriptions.create(\n model=\"deepgram:nova-2\",\n file=audio_file_path,\n language=\"en\", # Common param\n punctuate=True, # Deepgram-specific param\n )\n\n assert len(result.text) > 0, \"Deepgram returned empty transcription\"\n\n # If punctuation worked, text should contain punctuation marks\n # Note: This is a soft check as it depends on audio content\n # Just verify execution succeeded with provider-specific param\n\n except FileNotFoundError:\n pytest.skip(\"Test audio file not found. Skipping Deepgram feature test.\")\n except Exception as e:\n pytest.fail(f\"Error testing Deepgram provider-specific feature: {str(e)}\")\n\n\n@pytest.mark.integration\ndef test_asr_google_language_mapping():\n \"\"\"\n Test Google language mapping to verify auto-transformation.\n\n This test verifies that the common parameter 'language=\"en\"' is\n automatically transformed to 'language_code=\"en-US\"' for Google.\n \"\"\"\n client = setup_client()\n audio_file_path = \"tests/test-data/test_audio.mp3\"\n\n try:\n # Use 2-letter language code that should be expanded for Google\n result = client.audio.transcriptions.create(\n model=\"google:latest_long\",\n file=audio_file_path,\n language=\"en\", # Should be auto-transformed to \"en-US\"\n )\n\n assert len(result.text) > 0, \"Google returned empty transcription\"\n # If we got here, the language code transformation worked\n\n except FileNotFoundError:\n pytest.skip(\"Test audio file not found. Skipping Google mapping test.\")\n except Exception as e:\n pytest.fail(f\"Error testing Google language mapping: {str(e)}\")\n\n\n@pytest.mark.integration\ndef test_asr_huggingface_word_timestamps():\n \"\"\"\n Test Hugging Face word-level timestamps feature.\n\n This ensures that provider-specific parameters like 'return_timestamps'\n are correctly passed through to the Hugging Face Inference API.\n \"\"\"\n client = setup_client()\n audio_file_path = \"tests/test-data/test_audio.mp3\"\n\n try:\n # Use Hugging Face-specific feature\n result = client.audio.transcriptions.create(\n model=\"huggingface:openai/whisper-large-v3\",\n file=audio_file_path,\n return_timestamps=\"word\", # HF-specific param for word-level timestamps\n )\n\n assert len(result.text) > 0, \"Hugging Face returned empty transcription\"\n\n # If return_timestamps worked, result should have words with timestamps\n if hasattr(result, \"words\") and result.words:\n # Verify at least some words have timestamps\n words_with_timestamps = [\n w for w in result.words if w.start is not None and w.end is not None\n ]\n assert (\n len(words_with_timestamps) > 0\n ), \"No words with timestamps found in result\"\n\n except FileNotFoundError:\n pytest.skip(\"Test audio file not found. Skipping Hugging Face feature test.\")\n except Exception as e:\n pytest.fail(f\"Error testing Hugging Face word timestamps feature: {str(e)}\")\n\n\nif __name__ == \"__main__\":\n pytest.main([__file__, \"-v\"])\n" }, { "path": "tests/framework/test_asr_models.py", "content": "\"\"\"Tests for ASR framework data models.\"\"\"\n\nimport pytest\nfrom pydantic import ValidationError\n\nfrom aisuite.framework.message import (\n Word,\n Segment,\n Alternative,\n Channel,\n TranscriptionResult,\n)\n\n\nclass TestWord:\n \"\"\"Test suite for Word model.\"\"\"\n\n def test_word_creation_and_validation(self):\n \"\"\"Test Word creation with all fields and validation.\"\"\"\n # Test basic creation\n word = Word(word=\"hello\", start=0.0, end=0.5)\n assert word.word == \"hello\"\n assert word.start == 0.0\n assert word.end == 0.5\n assert word.confidence is None\n\n # Test with all fields\n word_full = Word(\n word=\"hello\",\n start=0.0,\n end=0.5,\n confidence=0.95,\n speaker=1,\n punctuated_word=\"Hello,\",\n )\n assert word_full.confidence == 0.95\n assert word_full.speaker == 1\n\n # Test validation\n with pytest.raises(ValidationError):\n Word() # Missing required fields\n\n\nclass TestSegment:\n \"\"\"Test suite for Segment model.\"\"\"\n\n def test_segment_creation_and_validation(self):\n \"\"\"Test Segment creation and validation.\"\"\"\n # Test basic creation\n segment = Segment(id=0, seek=0, start=0.0, end=5.0, text=\"Hello world\")\n assert segment.id == 0\n assert segment.text == \"Hello world\"\n\n # Test validation\n with pytest.raises(ValidationError):\n Segment() # Missing required fields\n\n\nclass TestAlternative:\n \"\"\"Test suite for Alternative model.\"\"\"\n\n def test_alternative_creation_and_validation(self):\n \"\"\"Test Alternative creation and validation.\"\"\"\n # Test creation\n alt = Alternative(transcript=\"Hello world\", confidence=0.9)\n assert alt.transcript == \"Hello world\"\n assert alt.confidence == 0.9\n\n # Test validation\n with pytest.raises(ValidationError):\n Alternative() # Missing required transcript\n\n\nclass TestChannel:\n \"\"\"Test suite for Channel model.\"\"\"\n\n def test_channel_creation_and_validation(self):\n \"\"\"Test Channel creation and validation.\"\"\"\n # Test creation\n alternatives = [Alternative(transcript=\"Test transcript\")]\n channel = Channel(alternatives=alternatives)\n assert len(channel.alternatives) == 1\n\n # Test validation\n with pytest.raises(ValidationError):\n Channel() # Missing required alternatives\n\n\nclass TestTranscriptionResult:\n \"\"\"Test suite for TranscriptionResult model.\"\"\"\n\n def test_transcription_result_basic(self):\n \"\"\"Test basic TranscriptionResult creation and validation.\"\"\"\n # Test basic creation\n result = TranscriptionResult(text=\"Hello world\")\n assert result.text == \"Hello world\"\n assert result.language is None\n\n # Test validation\n with pytest.raises(ValidationError):\n TranscriptionResult() # Missing required text field\n\n def test_transcription_result_openai_style(self):\n \"\"\"Test TranscriptionResult with OpenAI-style fields.\"\"\"\n segments = [Segment(id=0, seek=0, start=0.0, end=2.5, text=\"Hello world\")]\n words = [Word(word=\"hello\", start=0.0, end=0.5)]\n\n result = TranscriptionResult(\n text=\"Hello world\",\n language=\"en\",\n confidence=0.95,\n task=\"transcribe\",\n segments=segments,\n words=words,\n )\n\n assert result.text == \"Hello world\"\n assert result.language == \"en\"\n assert result.confidence == 0.95\n assert len(result.segments) == 1\n assert len(result.words) == 1\n\n def test_transcription_result_deepgram_style(self):\n \"\"\"Test TranscriptionResult with Deepgram-style fields.\"\"\"\n alternatives = [Alternative(transcript=\"Hello world\", confidence=0.9)]\n channels = [Channel(alternatives=alternatives)]\n\n result = TranscriptionResult(\n text=\"Hello world\",\n language=\"en-US\",\n channels=channels,\n alternatives=alternatives,\n topics=[{\"topic\": \"greeting\"}],\n )\n\n assert result.text == \"Hello world\"\n assert len(result.channels) == 1\n assert len(result.alternatives) == 1\n assert result.topics is not None\n" }, { "path": "tests/framework/test_asr_params.py", "content": "\"\"\"Unit tests for ASR parameter validation and mapping.\"\"\"\n\nimport pytest\nimport warnings\nfrom aisuite.framework.asr_params import (\n ParamValidator,\n COMMON_PARAMS,\n PROVIDER_PARAMS,\n GOOGLE_LANGUAGE_MAP,\n)\n\n\nclass TestParamValidatorCommonParams:\n \"\"\"Test common parameter mapping across providers.\"\"\"\n\n def test_language_mapping_openai(self):\n \"\"\"Test that language param passes through for OpenAI.\"\"\"\n validator = ParamValidator(\"strict\")\n result = validator.validate_and_map(\"openai\", {\"language\": \"en\"})\n assert result == {\"language\": \"en\"}\n\n def test_language_mapping_deepgram(self):\n \"\"\"Test that language param passes through for Deepgram.\"\"\"\n validator = ParamValidator(\"strict\")\n result = validator.validate_and_map(\"deepgram\", {\"language\": \"en\"})\n assert result == {\"language\": \"en\"}\n\n def test_language_mapping_google(self):\n \"\"\"Test that language param maps to language_code and expands for Google.\"\"\"\n validator = ParamValidator(\"strict\")\n result = validator.validate_and_map(\"google\", {\"language\": \"en\"})\n assert result == {\"language_code\": \"en-US\"}\n\n def test_prompt_mapping_openai(self):\n \"\"\"Test that prompt param passes through for OpenAI.\"\"\"\n validator = ParamValidator(\"strict\")\n result = validator.validate_and_map(\"openai\", {\"prompt\": \"meeting notes\"})\n assert result == {\"prompt\": \"meeting notes\"}\n\n def test_prompt_mapping_deepgram(self):\n \"\"\"Test that prompt param maps to keywords for Deepgram.\"\"\"\n validator = ParamValidator(\"strict\")\n result = validator.validate_and_map(\"deepgram\", {\"prompt\": \"meeting notes\"})\n assert result == {\"keywords\": [\"meeting\", \"notes\"]}\n\n def test_prompt_mapping_google(self):\n \"\"\"Test that prompt param maps to speech_contexts for Google.\"\"\"\n validator = ParamValidator(\"strict\")\n result = validator.validate_and_map(\"google\", {\"prompt\": \"technical terms\"})\n assert result == {\"speech_contexts\": [{\"phrases\": [\"technical terms\"]}]}\n\n def test_temperature_mapping_openai(self):\n \"\"\"Test that temperature param passes through for OpenAI.\"\"\"\n validator = ParamValidator(\"strict\")\n result = validator.validate_and_map(\"openai\", {\"temperature\": 0.5})\n assert result == {\"temperature\": 0.5}\n\n def test_temperature_ignored_deepgram(self):\n \"\"\"Test that temperature param is ignored for Deepgram (not supported).\"\"\"\n validator = ParamValidator(\"strict\")\n result = validator.validate_and_map(\"deepgram\", {\"temperature\": 0.5})\n assert result == {}\n\n def test_temperature_ignored_google(self):\n \"\"\"Test that temperature param is ignored for Google (not supported).\"\"\"\n validator = ParamValidator(\"strict\")\n result = validator.validate_and_map(\"google\", {\"temperature\": 0.5})\n assert result == {}\n\n\nclass TestParamValidatorTransformations:\n \"\"\"Test value transformations for provider-specific formats.\"\"\"\n\n def test_google_language_expansion_common_codes(self):\n \"\"\"Test Google language code expansion for common 2-letter codes.\"\"\"\n validator = ParamValidator(\"strict\")\n\n test_cases = {\n \"en\": \"en-US\",\n \"es\": \"es-ES\",\n \"fr\": \"fr-FR\",\n \"de\": \"de-DE\",\n \"ja\": \"ja-JP\",\n \"zh\": \"zh-CN\",\n }\n\n for input_lang, expected_output in test_cases.items():\n result = validator.validate_and_map(\"google\", {\"language\": input_lang})\n assert result == {\n \"language_code\": expected_output\n }, f\"Failed for {input_lang}: expected {expected_output}, got {result}\"\n\n def test_google_language_expansion_unknown_code(self):\n \"\"\"Test Google language code expansion for unknown 2-letter code (fallback to -US).\"\"\"\n validator = ParamValidator(\"strict\")\n result = validator.validate_and_map(\"google\", {\"language\": \"xx\"})\n assert result == {\"language_code\": \"xx-US\"}\n\n def test_google_language_no_expansion_for_full_code(self):\n \"\"\"Test that Google doesn't expand already full language codes.\"\"\"\n validator = ParamValidator(\"strict\")\n # When a full locale code is passed to language_code directly (not via common param)\n result = validator.validate_and_map(\"google\", {\"language_code\": \"en-GB\"})\n assert result == {\"language_code\": \"en-GB\"}\n\n def test_deepgram_prompt_to_keywords_single_word(self):\n \"\"\"Test Deepgram prompt splits single word into list.\"\"\"\n validator = ParamValidator(\"strict\")\n result = validator.validate_and_map(\"deepgram\", {\"prompt\": \"meeting\"})\n assert result == {\"keywords\": [\"meeting\"]}\n\n def test_deepgram_prompt_to_keywords_multiple_words(self):\n \"\"\"Test Deepgram prompt splits multiple words into list.\"\"\"\n validator = ParamValidator(\"strict\")\n result = validator.validate_and_map(\n \"deepgram\", {\"prompt\": \"meeting notes action items\"}\n )\n assert result == {\"keywords\": [\"meeting\", \"notes\", \"action\", \"items\"]}\n\n def test_deepgram_prompt_to_keywords_already_list(self):\n \"\"\"Test Deepgram handles prompt that's already a list.\"\"\"\n validator = ParamValidator(\"strict\")\n result = validator.validate_and_map(\n \"deepgram\", {\"prompt\": [\"meeting\", \"notes\"]}\n )\n assert result == {\"keywords\": [\"meeting\", \"notes\"]}\n\n def test_google_prompt_to_speech_contexts(self):\n \"\"\"Test Google wraps prompt in speech_contexts structure.\"\"\"\n validator = ParamValidator(\"strict\")\n result = validator.validate_and_map(\"google\", {\"prompt\": \"technical terms\"})\n assert result == {\"speech_contexts\": [{\"phrases\": [\"technical terms\"]}]}\n\n\nclass TestParamValidatorProviderSpecific:\n \"\"\"Test provider-specific parameter pass-through.\"\"\"\n\n def test_openai_response_format(self):\n \"\"\"Test OpenAI response_format param passes through.\"\"\"\n validator = ParamValidator(\"strict\")\n result = validator.validate_and_map(\n \"openai\", {\"response_format\": \"verbose_json\"}\n )\n assert result == {\"response_format\": \"verbose_json\"}\n\n def test_openai_timestamp_granularities(self):\n \"\"\"Test OpenAI timestamp_granularities param passes through.\"\"\"\n validator = ParamValidator(\"strict\")\n result = validator.validate_and_map(\n \"openai\", {\"timestamp_granularities\": [\"word\", \"segment\"]}\n )\n assert result == {\"timestamp_granularities\": [\"word\", \"segment\"]}\n\n def test_deepgram_punctuate(self):\n \"\"\"Test Deepgram punctuate param passes through.\"\"\"\n validator = ParamValidator(\"strict\")\n result = validator.validate_and_map(\"deepgram\", {\"punctuate\": True})\n assert result == {\"punctuate\": True}\n\n def test_deepgram_diarize(self):\n \"\"\"Test Deepgram diarize param passes through.\"\"\"\n validator = ParamValidator(\"strict\")\n result = validator.validate_and_map(\"deepgram\", {\"diarize\": True})\n assert result == {\"diarize\": True}\n\n def test_deepgram_multiple_features(self):\n \"\"\"Test Deepgram multiple features pass through together.\"\"\"\n validator = ParamValidator(\"strict\")\n result = validator.validate_and_map(\n \"deepgram\",\n {\n \"punctuate\": True,\n \"diarize\": True,\n \"sentiment\": True,\n \"topics\": True,\n },\n )\n assert result == {\n \"punctuate\": True,\n \"diarize\": True,\n \"sentiment\": True,\n \"topics\": True,\n }\n\n def test_google_enable_automatic_punctuation(self):\n \"\"\"Test Google enable_automatic_punctuation param passes through.\"\"\"\n validator = ParamValidator(\"strict\")\n result = validator.validate_and_map(\n \"google\", {\"enable_automatic_punctuation\": True}\n )\n assert result == {\"enable_automatic_punctuation\": True}\n\n def test_google_enable_speaker_diarization(self):\n \"\"\"Test Google enable_speaker_diarization param passes through.\"\"\"\n validator = ParamValidator(\"strict\")\n result = validator.validate_and_map(\n \"google\", {\"enable_speaker_diarization\": True}\n )\n assert result == {\"enable_speaker_diarization\": True}\n\n def test_google_diarization_speaker_count(self):\n \"\"\"Test Google diarization_speaker_count param passes through.\"\"\"\n validator = ParamValidator(\"strict\")\n result = validator.validate_and_map(\"google\", {\"diarization_speaker_count\": 3})\n assert result == {\"diarization_speaker_count\": 3}\n\n\nclass TestParamValidatorMixedParams:\n \"\"\"Test combinations of common and provider-specific parameters.\"\"\"\n\n def test_openai_common_and_specific(self):\n \"\"\"Test OpenAI with common params + provider-specific params.\"\"\"\n validator = ParamValidator(\"strict\")\n result = validator.validate_and_map(\n \"openai\",\n {\n \"language\": \"en\",\n \"temperature\": 0.5,\n \"response_format\": \"verbose_json\",\n },\n )\n assert result == {\n \"language\": \"en\",\n \"temperature\": 0.5,\n \"response_format\": \"verbose_json\",\n }\n\n def test_deepgram_common_and_specific(self):\n \"\"\"Test Deepgram with common params + provider-specific params.\"\"\"\n validator = ParamValidator(\"strict\")\n result = validator.validate_and_map(\n \"deepgram\",\n {\n \"language\": \"en\",\n \"prompt\": \"meeting\",\n \"punctuate\": True,\n \"diarize\": True,\n },\n )\n assert result == {\n \"language\": \"en\",\n \"keywords\": [\"meeting\"],\n \"punctuate\": True,\n \"diarize\": True,\n }\n\n def test_google_common_and_specific(self):\n \"\"\"Test Google with common params + provider-specific params.\"\"\"\n validator = ParamValidator(\"strict\")\n result = validator.validate_and_map(\n \"google\",\n {\n \"language\": \"en\",\n \"enable_automatic_punctuation\": True,\n \"enable_speaker_diarization\": True,\n },\n )\n assert result == {\n \"language_code\": \"en-US\",\n \"enable_automatic_punctuation\": True,\n \"enable_speaker_diarization\": True,\n }\n\n\nclass TestParamValidatorStrictMode:\n \"\"\"Test strict validation mode behavior.\"\"\"\n\n def test_strict_mode_rejects_unknown_param_openai(self):\n \"\"\"Test strict mode raises ValueError for unknown OpenAI param.\"\"\"\n validator = ParamValidator(\"strict\")\n with pytest.raises(\n ValueError, match=\"Unknown parameters for openai: \\\\['invalid_param'\\\\]\"\n ):\n validator.validate_and_map(\"openai\", {\"invalid_param\": True})\n\n def test_strict_mode_rejects_unknown_param_deepgram(self):\n \"\"\"Test strict mode raises ValueError for unknown Deepgram param.\"\"\"\n validator = ParamValidator(\"strict\")\n with pytest.raises(\n ValueError, match=\"Unknown parameters for deepgram: \\\\['invalid_param'\\\\]\"\n ):\n validator.validate_and_map(\"deepgram\", {\"invalid_param\": True})\n\n def test_strict_mode_rejects_multiple_unknown_params(self):\n \"\"\"Test strict mode raises ValueError for multiple unknown params.\"\"\"\n validator = ParamValidator(\"strict\")\n with pytest.raises(ValueError, match=\"Unknown parameters for openai\"):\n validator.validate_and_map(\n \"openai\",\n {\n \"invalid_param1\": True,\n \"invalid_param2\": False,\n },\n )\n\n def test_strict_mode_error_message_helpful(self):\n \"\"\"Test that strict mode error message mentions provider documentation.\"\"\"\n validator = ParamValidator(\"strict\")\n with pytest.raises(ValueError, match=\"See openai documentation\"):\n validator.validate_and_map(\"openai\", {\"typo_param\": True})\n\n def test_strict_mode_allows_valid_params(self):\n \"\"\"Test that strict mode allows all valid params.\"\"\"\n validator = ParamValidator(\"strict\")\n # Should not raise\n result = validator.validate_and_map(\n \"deepgram\",\n {\n \"language\": \"en\",\n \"punctuate\": True,\n },\n )\n assert result == {\"language\": \"en\", \"punctuate\": True}\n\n\nclass TestParamValidatorWarnMode:\n \"\"\"Test warn validation mode behavior.\"\"\"\n\n def test_warn_mode_issues_warning_unknown_param(self):\n \"\"\"Test warn mode issues UserWarning for unknown param.\"\"\"\n validator = ParamValidator(\"warn\")\n with pytest.warns(\n UserWarning, match=\"Unknown parameters for openai: \\\\['invalid_param'\\\\]\"\n ):\n result = validator.validate_and_map(\"openai\", {\"invalid_param\": True})\n # Param should be filtered out (not passed through in warn mode)\n assert result == {}\n\n def test_warn_mode_continues_execution(self):\n \"\"\"Test warn mode continues execution after warning.\"\"\"\n validator = ParamValidator(\"warn\")\n with warnings.catch_warnings():\n warnings.simplefilter(\"ignore\") # Suppress warning for this test\n result = validator.validate_and_map(\n \"openai\",\n {\n \"language\": \"en\",\n \"invalid_param\": True,\n },\n )\n # Valid param should pass through, invalid should be filtered\n assert result == {\"language\": \"en\"}\n\n def test_warn_mode_warning_message_helpful(self):\n \"\"\"Test that warn mode warning message mentions provider documentation.\"\"\"\n validator = ParamValidator(\"warn\")\n with pytest.warns(UserWarning, match=\"See deepgram documentation\"):\n validator.validate_and_map(\"deepgram\", {\"typo_param\": True})\n\n\nclass TestParamValidatorPermissiveMode:\n \"\"\"Test permissive validation mode behavior.\"\"\"\n\n def test_permissive_mode_allows_unknown_param(self):\n \"\"\"Test permissive mode passes through unknown params.\"\"\"\n validator = ParamValidator(\"permissive\")\n result = validator.validate_and_map(\"openai\", {\"experimental_feature\": True})\n assert result == {\"experimental_feature\": True}\n\n def test_permissive_mode_no_warning(self):\n \"\"\"Test permissive mode doesn't issue warnings.\"\"\"\n validator = ParamValidator(\"permissive\")\n with warnings.catch_warnings():\n warnings.simplefilter(\"error\") # Turn warnings into errors\n # Should not raise (no warning should be issued)\n result = validator.validate_and_map(\"openai\", {\"unknown_param\": True})\n assert result == {\"unknown_param\": True}\n\n def test_permissive_mode_mixed_valid_and_unknown(self):\n \"\"\"Test permissive mode with mix of valid and unknown params.\"\"\"\n validator = ParamValidator(\"permissive\")\n result = validator.validate_and_map(\n \"deepgram\",\n {\n \"language\": \"en\",\n \"punctuate\": True,\n \"experimental_feature\": True,\n \"beta_param\": \"value\",\n },\n )\n assert result == {\n \"language\": \"en\",\n \"punctuate\": True,\n \"experimental_feature\": True,\n \"beta_param\": \"value\",\n }\n\n\nclass TestParamValidatorEdgeCases:\n \"\"\"Test edge cases and error conditions.\"\"\"\n\n def test_empty_params(self):\n \"\"\"Test validation with empty params dict.\"\"\"\n validator = ParamValidator(\"strict\")\n result = validator.validate_and_map(\"openai\", {})\n assert result == {}\n\n def test_unknown_provider(self):\n \"\"\"Test validation with unknown provider (uses empty param set).\"\"\"\n validator = ParamValidator(\"strict\")\n # Unknown provider with non-common param should raise error\n with pytest.raises(ValueError, match=\"Unknown parameters for unknown_provider\"):\n validator.validate_and_map(\"unknown_provider\", {\"custom_param\": \"value\"})\n\n def test_none_value_param(self):\n \"\"\"Test that None values are handled correctly.\"\"\"\n validator = ParamValidator(\"strict\")\n result = validator.validate_and_map(\"openai\", {\"language\": None})\n assert result == {\"language\": None}\n\n def test_common_param_overrides_provider_param(self):\n \"\"\"Test that common param mapping takes precedence.\"\"\"\n validator = ParamValidator(\"strict\")\n # If someone passes both 'language' and 'language_code' to Google\n # The common param 'language' should map to 'language_code'\n result = validator.validate_and_map(\n \"google\",\n {\n \"language\": \"en\",\n \"language_code\": \"fr-FR\", # This should be overridden\n },\n )\n # language maps to language_code, then language_code as provider param is also valid\n # In current implementation, common param is processed first, then provider params\n # So we'll get both, but language takes precedence in the transformation\n assert \"language_code\" in result\n assert result[\"language_code\"] == \"fr-FR\" or result[\"language_code\"] == \"en-US\"\n\n def test_validator_mode_case_sensitivity(self):\n \"\"\"Test that validator handles only valid mode strings.\"\"\"\n # Valid modes should work\n ParamValidator(\"strict\")\n ParamValidator(\"warn\")\n ParamValidator(\"permissive\")\n\n # Note: Python typing will catch invalid modes at type-check time,\n # but runtime won't enforce it unless we add validation\n # This test documents expected behavior\n\n\nclass TestParamValidatorRegistry:\n \"\"\"Test that parameter registries are complete and consistent.\"\"\"\n\n def test_common_params_has_required_providers(self):\n \"\"\"Test COMMON_PARAMS includes all ASR providers.\"\"\"\n assert \"openai\" in COMMON_PARAMS[\"language\"]\n assert \"deepgram\" in COMMON_PARAMS[\"language\"]\n assert \"google\" in COMMON_PARAMS[\"language\"]\n\n def test_provider_params_has_all_providers(self):\n \"\"\"Test PROVIDER_PARAMS includes all ASR providers.\"\"\"\n assert \"openai\" in PROVIDER_PARAMS\n assert \"deepgram\" in PROVIDER_PARAMS\n assert \"google\" in PROVIDER_PARAMS\n\n def test_google_language_map_completeness(self):\n \"\"\"Test GOOGLE_LANGUAGE_MAP has common language codes.\"\"\"\n required_languages = [\"en\", \"es\", \"fr\", \"de\", \"ja\", \"zh\"]\n for lang in required_languages:\n assert lang in GOOGLE_LANGUAGE_MAP, f\"Missing {lang} in GOOGLE_LANGUAGE_MAP\"\n\n def test_provider_params_includes_common_params(self):\n \"\"\"Test that provider param sets include their common params.\"\"\"\n # OpenAI should have language, prompt, temperature in its set\n assert \"language\" in PROVIDER_PARAMS[\"openai\"]\n assert \"prompt\" in PROVIDER_PARAMS[\"openai\"]\n assert \"temperature\" in PROVIDER_PARAMS[\"openai\"]\n\n # Deepgram should have language in its set (but not temperature)\n assert \"language\" in PROVIDER_PARAMS[\"deepgram\"]\n assert \"temperature\" not in PROVIDER_PARAMS[\"deepgram\"]\n" }, { "path": "tests/mcp/README.md", "content": "# MCP Integration Tests\n\nThis directory contains integration tests for aisuite's MCP (Model Context Protocol) support.\n\n## Prerequisites\n\nTo run these tests, you need:\n\n1. **Node.js and npx** - Required to run the Anthropic filesystem MCP server\n - Install from: https://nodejs.org/\n - Verify with: `npx --version`\n\n2. **Python test dependencies**:\n ```bash\n pip install pytest pytest-asyncio python-dotenv\n ```\n\n3. **MCP package** (should already be installed if you have aisuite[mcp]):\n ```bash\n pip install 'aisuite[mcp]'\n ```\n\n4. **Environment variables** (for e2e tests that mock LLM calls):\n Create a `.env` file in the project root with your API keys:\n ```bash\n OPENAI_API_KEY=your-key-here\n ANTHROPIC_API_KEY=your-key-here\n EXA_API_KEY=your-key-here # Optional: for Exa MCP tests\n ```\n Note: E2E tests mock LLM responses, so API keys won't be charged, but providers validate keys on initialization.\n\n## Running Tests\n\n### Run all MCP integration tests (mocked LLM, free):\n```bash\npytest tests/mcp/ -v -m \"integration and not llm\"\n```\n\n### Run specific test file:\n```bash\n# MCPClient tests\npytest tests/mcp/test_client.py -v -m integration\n\n# End-to-end tests (mocked LLM)\npytest tests/mcp/test_e2e.py -v -m integration\n\n# Real LLM tests with stdio (⚠️ costs money, requires API keys)\npytest tests/mcp/test_llm_e2e.py -v -m llm\n\n# Real LLM tests with HTTP (⚠️ costs money, requires API keys)\npytest tests/mcp/test_http_llm_e2e.py -v -m llm\n```\n\n### Run ONLY real LLM tests (⚠️ costs ~$0.50):\n```bash\npytest tests/mcp/ -v -m llm\n```\n\n### Run ALL tests including LLM (⚠️ costs money):\n```bash\npytest tests/mcp/ -v -m integration\n```\n\n### Run a specific test:\n```bash\npytest tests/mcp/test_client.py::TestMCPClientConnection::test_connect_to_filesystem_server -v\n```\n\n### Skip integration tests (if no Node.js):\n```bash\npytest tests/mcp/ -v -m \"not integration\"\n```\n\n## Test Structure\n\n### `test_client.py` - MCPClient Integration Tests\nTests the `MCPClient` class with a real MCP server:\n- Connection to Anthropic filesystem server\n- Listing tools\n- Calling tools\n- Tool filtering (`allowed_tools`)\n- Tool prefixing (`use_tool_prefix`)\n- `from_config()` method\n- Context manager support\n\n### `test_e2e.py` - End-to-End Tests (Mocked LLM)\nTests the complete flow with `client.chat.completions.create()`:\n- Config dict format\n- Mixing MCP configs with Python functions\n- Multiple MCP servers with prefixing\n- Automatic cleanup\n- Error handling\n- **Note:** LLM responses are mocked, so no API calls are made\n\n### `test_llm_e2e.py` - Real LLM End-to-End Tests with stdio (⚠️ Costs Money)\nTests with **actual API calls** to verify stdio MCP works with real LLMs:\n- OpenAI GPT-4o reading files via stdio MCP\n- Anthropic Claude reading files via stdio MCP\n- Mixed tools (stdio MCP + Python functions)\n- Multiple MCP servers with prefixing\n- **Uses:** `@modelcontextprotocol/server-filesystem` (stdio)\n- **Note:** These tests make real API calls (~$0.05-0.10 per test)\n- **Marked with:** `@pytest.mark.llm`\n- **Skipped if:** API keys not present in .env\n\n### `test_http_llm_e2e.py` - Real LLM End-to-End Tests with HTTP (⚠️ Costs Money)\nTests with **actual API calls** to verify HTTP MCP works with real LLMs:\n- OpenAI GPT-4o using HTTP MCP tools (Context7 and Exa)\n- Anthropic Claude using HTTP MCP tools (Context7 and Exa)\n- Mixed tools (HTTP MCP + Python functions)\n- Config dict format with HTTP transport\n- Custom headers support (including Authorization headers for Exa)\n- **Uses:**\n - Context7 HTTP MCP server (`https://mcp.context7.com/mcp`)\n - Tools: `resolve-library-id`, `get-library-docs` (library documentation)\n - Exa HTTP MCP server (`https://mcp.exa.ai/mcp`)\n - Tools: `web_search_exa`, `get_code_context_exa` (web search and code context)\n - Requires: EXA_API_KEY in .env\n- **Note:** These tests make real API calls (~$0.05-0.10 per test)\n- **Marked with:** `@pytest.mark.llm`\n- **Skipped if:** API keys not present in .env\n\n### `conftest.py` - Test Fixtures\n- `temp_test_dir` - Creates temp directory with test files\n- `skip_if_no_npx` - Skips tests if npx not available\n\n## What Gets Tested\n\n### stdio Transport Tests\nUse the **real** `@modelcontextprotocol/server-filesystem` MCP server from Anthropic, which:\n- Provides file system access tools (read_file, write_file, list_directory, etc.)\n- Is installed automatically via `npx -y @modelcontextprotocol/server-filesystem`\n- Runs in a temporary test directory for isolation\n\n### HTTP Transport Tests\nUse **real** HTTP MCP servers:\n\n1. **Context7** (`https://mcp.context7.com/mcp`):\n - Provides library documentation tools\n - Tools: `resolve-library-id`, `get-library-docs`\n - No installation required (hosted service)\n - No authentication required (optional API key for higher rate limits)\n\n2. **Exa** (`https://mcp.exa.ai/mcp`):\n - Provides web search and code context tools\n - Tools: `web_search_exa`, `get_code_context_exa`, `deep_researcher`\n - No installation required (hosted service)\n - Requires: EXA_API_KEY (via Authorization header)\n\nThe tests verify:\n1. ✅ Connection to real MCP servers (stdio and HTTP)\n2. ✅ Tool discovery and schema parsing\n3. ✅ Tool execution and result handling\n4. ✅ Config dict → callable conversion\n5. ✅ Tool filtering and prefixing\n6. ✅ Integration with aisuite's tool system\n7. ✅ Proper resource cleanup\n8. ✅ Error handling\n9. ✅ HTTP transport with headers and timeout\n\n## CI/CD\n\n### GitHub Actions\nIf running in CI without Node.js:\n```yaml\n- name: Run tests\n run: pytest tests/mcp/ -v -m \"not integration\"\n```\n\nWith Node.js:\n```yaml\n- name: Setup Node.js\n uses: actions/setup-node@v3\n with:\n node-version: '18'\n\n- name: Run integration tests\n run: pytest tests/mcp/ -v -m integration\n```\n\n## Notes\n\n- Tests are marked with `@pytest.mark.integration` to allow selective running\n- Most tests use mocking for LLM API calls to avoid costs\n- Real LLM tests are marked with `@pytest.mark.llm` and can be skipped\n- Each test creates isolated temp directories for file operations\n- MCP servers are started fresh for each test\n- Cleanup is automatic via fixtures and context managers\n\n## Test Markers\n\n- `@pytest.mark.integration` - All MCP tests (includes both mocked and real LLM)\n- `@pytest.mark.llm` - Real LLM tests only (makes actual API calls, costs money)\n\nTo run tests without LLM costs:\n```bash\npytest tests/mcp/ -v -m \"integration and not llm\"\n```\n\n## Troubleshooting\n\n**Error: \"npx not found\"**\n- Install Node.js from https://nodejs.org/\n\n**Error: \"MCP package not installed\"**\n- Run: `pip install 'aisuite[mcp]'`\n\n**Tests hang or timeout**\n- Check Node.js/npx is working: `npx --version`\n- Check MCP server can be installed: `npx -y @modelcontextprotocol/server-filesystem --help`\n\n**Import errors**\n- Make sure you're running from the project root\n- Install test dependencies: `pip install pytest pytest-asyncio`\n" }, { "path": "tests/mcp/__init__.py", "content": "\"\"\"Tests for MCP (Model Context Protocol) integration.\"\"\"\n" }, { "path": "tests/mcp/conftest.py", "content": "\"\"\"\nPytest fixtures for MCP integration tests.\n\"\"\"\n\nimport pytest\nimport tempfile\nimport os\nfrom pathlib import Path\n\n# Load environment variables from .env file\ntry:\n from dotenv import load_dotenv\n\n load_dotenv()\nexcept ImportError:\n # dotenv not installed, that's okay\n pass\n\n\n@pytest.fixture\ndef temp_test_dir():\n \"\"\"\n Create a temporary directory with test files for filesystem MCP server.\n\n This fixture creates a temp directory with sample files that can be used\n to test the Anthropic filesystem MCP server.\n\n Yields:\n str: Real path to the temporary test directory (resolves symlinks)\n\n Example:\n >>> def test_mcp(temp_test_dir):\n ... mcp = MCPClient(\n ... command=\"npx\",\n ... args=[\"-y\", \"@modelcontextprotocol/server-filesystem\", temp_test_dir]\n ... )\n \"\"\"\n with tempfile.TemporaryDirectory() as tmpdir:\n # Resolve real path to handle symlinks (e.g., /var -> /private/var on macOS)\n real_tmpdir = os.path.realpath(tmpdir)\n\n # Create test files\n test_file = Path(real_tmpdir) / \"test.txt\"\n test_file.write_text(\"Hello from MCP test!\")\n\n readme = Path(real_tmpdir) / \"README.md\"\n readme.write_text(\"# Test Directory\\n\\nThis is a test README file.\")\n\n data_file = Path(real_tmpdir) / \"data.json\"\n data_file.write_text('{\"key\": \"value\", \"number\": 42}')\n\n # Create a subdirectory\n subdir = Path(real_tmpdir) / \"subdir\"\n subdir.mkdir()\n (subdir / \"nested.txt\").write_text(\"Nested file content\")\n\n yield real_tmpdir\n\n\n@pytest.fixture\ndef skip_if_no_npx():\n \"\"\"\n Skip test if npx is not available.\n\n This fixture checks if npx (Node.js package executor) is installed,\n which is required to run the Anthropic filesystem MCP server.\n\n Raises:\n pytest.skip: If npx is not found in PATH\n \"\"\"\n import shutil\n\n if not shutil.which(\"npx\"):\n pytest.skip(\n \"npx not found. Install Node.js to run MCP integration tests. \"\n \"See: https://nodejs.org/\"\n )\n" }, { "path": "tests/mcp/test_client.py", "content": "\"\"\"\nIntegration tests for MCPClient.\n\nThese tests use the real Anthropic filesystem MCP server\n(@modelcontextprotocol/server-filesystem) to verify that MCPClient\ncan connect to, discover tools from, and execute tools on real MCP servers.\n\nRequirements:\n - Node.js and npx must be installed\n - Tests are marked with @pytest.mark.integration\n - Run with: pytest tests/mcp/test_client.py -v -m integration\n\"\"\"\n\nimport pytest\nfrom aisuite.mcp import MCPClient\nfrom aisuite.mcp.config import validate_mcp_config\n\n\n@pytest.mark.integration\nclass TestMCPClientConnection:\n \"\"\"Test MCPClient connection and basic functionality.\"\"\"\n\n def test_connect_to_filesystem_server(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test connecting to real Anthropic filesystem MCP server.\"\"\"\n mcp = MCPClient(\n command=\"npx\",\n args=[\"-y\", \"@modelcontextprotocol/server-filesystem\", temp_test_dir],\n name=\"test_filesystem\",\n )\n\n try:\n # Verify client is connected\n assert mcp._session is not None\n assert mcp.name == \"test_filesystem\"\n\n # List tools\n tools = mcp.list_tools()\n assert len(tools) > 0\n\n # Verify expected tools are present\n tool_names = [t[\"name\"] for t in tools]\n assert \"read_file\" in tool_names\n assert \"list_directory\" in tool_names\n\n # Verify tools have descriptions\n for tool in tools:\n assert \"name\" in tool\n assert \"description\" in tool or \"inputSchema\" in tool\n\n finally:\n mcp.close()\n\n def test_list_tools_returns_schemas(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test that list_tools returns proper tool schemas.\"\"\"\n mcp = MCPClient(\n command=\"npx\",\n args=[\"-y\", \"@modelcontextprotocol/server-filesystem\", temp_test_dir],\n )\n\n try:\n tools = mcp.list_tools()\n\n # Find read_file tool\n read_file_tool = next((t for t in tools if t[\"name\"] == \"read_file\"), None)\n assert read_file_tool is not None\n\n # Verify it has an input schema\n assert \"inputSchema\" in read_file_tool\n assert \"properties\" in read_file_tool[\"inputSchema\"]\n\n finally:\n mcp.close()\n\n def test_context_manager(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test MCPClient as context manager.\"\"\"\n with MCPClient(\n command=\"npx\",\n args=[\"-y\", \"@modelcontextprotocol/server-filesystem\", temp_test_dir],\n ) as mcp:\n tools = mcp.list_tools()\n assert len(tools) > 0\n\n # After exiting context, session should be closed\n # (We don't have a good way to verify this without inspecting internals)\n\n\n@pytest.mark.integration\nclass TestMCPClientToolExecution:\n \"\"\"Test executing tools via MCPClient.\"\"\"\n\n def test_call_read_file_tool(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test calling the read_file tool.\"\"\"\n import os\n\n mcp = MCPClient(\n command=\"npx\",\n args=[\"-y\", \"@modelcontextprotocol/server-filesystem\", temp_test_dir],\n )\n\n try:\n # Call read_file tool with absolute path\n test_file_path = os.path.join(temp_test_dir, \"test.txt\")\n result = mcp.call_tool(\"read_file\", {\"path\": test_file_path})\n\n # Verify result contains file content\n assert \"Hello from MCP test!\" in result\n\n finally:\n mcp.close()\n\n def test_call_list_directory_tool(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test calling the list_directory tool.\"\"\"\n mcp = MCPClient(\n command=\"npx\",\n args=[\"-y\", \"@modelcontextprotocol/server-filesystem\", temp_test_dir],\n )\n\n try:\n # Call list_directory tool with absolute path\n result = mcp.call_tool(\"list_directory\", {\"path\": temp_test_dir})\n\n # Verify result contains our test files\n assert \"test.txt\" in result or \"README.md\" in result\n\n finally:\n mcp.close()\n\n\n@pytest.mark.integration\nclass TestMCPClientCallableTools:\n \"\"\"Test getting callable tools from MCPClient.\"\"\"\n\n def test_get_callable_tools(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test getting all tools as callables.\"\"\"\n mcp = MCPClient(\n command=\"npx\",\n args=[\"-y\", \"@modelcontextprotocol/server-filesystem\", temp_test_dir],\n )\n\n try:\n tools = mcp.get_callable_tools()\n\n # Verify we got callables\n assert len(tools) > 0\n assert all(callable(t) for t in tools)\n\n # Verify callables have expected attributes\n for tool in tools:\n assert hasattr(tool, \"__name__\")\n assert hasattr(tool, \"__doc__\")\n assert hasattr(tool, \"__annotations__\")\n\n # Find read_file callable\n read_file = next((t for t in tools if t.__name__ == \"read_file\"), None)\n assert read_file is not None\n\n # Test calling it with absolute path\n import os\n\n test_file_path = os.path.join(temp_test_dir, \"test.txt\")\n result = read_file(path=test_file_path)\n assert \"Hello from MCP test!\" in result\n\n finally:\n mcp.close()\n\n def test_get_callable_tools_with_filtering(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test filtering tools with allowed_tools parameter.\"\"\"\n mcp = MCPClient(\n command=\"npx\",\n args=[\"-y\", \"@modelcontextprotocol/server-filesystem\", temp_test_dir],\n )\n\n try:\n # Get only read_file tool\n tools = mcp.get_callable_tools(allowed_tools=[\"read_file\"])\n\n # Should only get one tool\n assert len(tools) == 1\n assert tools[0].__name__ == \"read_file\"\n\n # Test it works with absolute path\n import os\n\n test_file_path = os.path.join(temp_test_dir, \"test.txt\")\n result = tools[0](path=test_file_path)\n assert \"Hello from MCP test!\" in result\n\n finally:\n mcp.close()\n\n def test_get_callable_tools_with_prefixing(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test tool name prefixing with use_tool_prefix.\"\"\"\n mcp = MCPClient(\n command=\"npx\",\n args=[\"-y\", \"@modelcontextprotocol/server-filesystem\", temp_test_dir],\n name=\"filesystem\",\n )\n\n try:\n # Get tools with prefixing\n tools = mcp.get_callable_tools(use_tool_prefix=True)\n\n # Verify tools are prefixed\n tool_names = [t.__name__ for t in tools]\n assert any(name.startswith(\"filesystem__\") for name in tool_names)\n assert \"filesystem__read_file\" in tool_names\n\n finally:\n mcp.close()\n\n def test_get_specific_tool(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test getting a specific tool by name.\"\"\"\n mcp = MCPClient(\n command=\"npx\",\n args=[\"-y\", \"@modelcontextprotocol/server-filesystem\", temp_test_dir],\n )\n\n try:\n # Get specific tool\n read_file = mcp.get_tool(\"read_file\")\n\n assert read_file is not None\n assert callable(read_file)\n assert read_file.__name__ == \"read_file\"\n\n # Test it works with absolute path\n import os\n\n readme_path = os.path.join(temp_test_dir, \"README.md\")\n result = read_file(path=readme_path)\n assert \"Test Directory\" in result\n\n # Test getting non-existent tool\n fake_tool = mcp.get_tool(\"nonexistent_tool\")\n assert fake_tool is None\n\n finally:\n mcp.close()\n\n\n@pytest.mark.integration\nclass TestMCPClientFromConfig:\n \"\"\"Test creating MCPClient from configuration dict.\"\"\"\n\n def test_from_config_stdio(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test creating MCPClient from config dict with stdio transport.\"\"\"\n config = {\n \"type\": \"mcp\",\n \"name\": \"test_filesystem\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@modelcontextprotocol/server-filesystem\", temp_test_dir],\n }\n\n mcp = MCPClient.from_config(config)\n\n try:\n assert mcp.name == \"test_filesystem\"\n tools = mcp.list_tools()\n assert len(tools) > 0\n\n finally:\n mcp.close()\n\n def test_from_config_with_env(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test creating MCPClient with environment variables.\"\"\"\n import os\n\n config = {\n \"type\": \"mcp\",\n \"name\": \"test_filesystem\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@modelcontextprotocol/server-filesystem\", temp_test_dir],\n \"env\": {\"TEST_VAR\": \"test_value\"},\n }\n\n mcp = MCPClient.from_config(config)\n\n try:\n tools = mcp.list_tools()\n assert len(tools) > 0\n\n finally:\n mcp.close()\n\n def test_get_tools_from_config(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test get_tools_from_config convenience method.\"\"\"\n config = {\n \"type\": \"mcp\",\n \"name\": \"test_filesystem\",\n \"command\": \"npx\",\n \"args\": [\"-y\", \"@modelcontextprotocol/server-filesystem\", temp_test_dir],\n \"allowed_tools\": [\"read_file\"],\n \"use_tool_prefix\": True,\n }\n\n # Note: This creates a client internally and doesn't provide a way to close it\n # In production, this would be managed by the Completions class\n tools = MCPClient.get_tools_from_config(config)\n\n assert len(tools) == 1\n assert tools[0].__name__ == \"test_filesystem__read_file\"\n assert callable(tools[0])\n\n # Test the tool works with absolute path\n import os\n\n test_file_path = os.path.join(temp_test_dir, \"test.txt\")\n result = tools[0](path=test_file_path)\n assert \"Hello from MCP test!\" in result\n\n\n@pytest.mark.integration\nclass TestMCPClientErrorHandling:\n \"\"\"Test error handling in MCPClient.\"\"\"\n\n def test_invalid_command_raises_error(self, temp_test_dir):\n \"\"\"Test that invalid command raises appropriate error.\"\"\"\n with pytest.raises(Exception):\n # This should fail to connect\n mcp = MCPClient(\n command=\"nonexistent_command_12345\",\n args=[\"--test\"],\n )\n\n def test_call_nonexistent_tool_returns_error(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test that calling non-existent tool returns error or raises exception.\"\"\"\n mcp = MCPClient(\n command=\"npx\",\n args=[\"-y\", \"@modelcontextprotocol/server-filesystem\", temp_test_dir],\n )\n\n try:\n # Calling a non-existent tool should either raise an error or return error message\n try:\n result = mcp.call_tool(\"nonexistent_tool_xyz_123\", {})\n # If it doesn't raise, the result should contain an error message\n assert \"error\" in result.lower() or \"not found\" in result.lower()\n except Exception:\n # It's also acceptable to raise an exception\n pass\n\n finally:\n mcp.close()\n" }, { "path": "tests/mcp/test_e2e.py", "content": "\"\"\"\nEnd-to-end integration tests for MCP with aisuite.\n\nThese tests verify the complete flow of using MCP tools with aisuite's\nchat.completions.create() API, including:\n- Config dict format\n- Mixing MCP tools with Python functions\n- Multiple MCP servers with prefixing\n- Automatic cleanup\n\nRequirements:\n - Node.js and npx must be installed\n - Tests are marked with @pytest.mark.integration\n - Run with: pytest tests/mcp/test_e2e.py -v -m integration\n\"\"\"\n\nimport pytest\nfrom unittest.mock import patch, MagicMock, Mock\nfrom aisuite import Client\n\n\ndef create_mock_response(content=\"Test response\", tool_calls=None):\n \"\"\"Helper to create a mock chat completion response.\"\"\"\n # Create a simple mock object that mimics the response structure\n response = MagicMock()\n response.choices = [MagicMock()]\n response.choices[0].message = MagicMock()\n response.choices[0].message.content = content\n response.choices[0].message.tool_calls = tool_calls\n response.choices[0].intermediate_messages = [response.choices[0].message]\n response.intermediate_responses = []\n\n return response\n\n\n@pytest.mark.integration\nclass TestMCPConfigDictFormat:\n \"\"\"Test using MCP config dict format in chat.completions.create().\"\"\"\n\n def test_basic_config_dict(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test basic MCP config dict usage.\"\"\"\n client = Client()\n\n # Mock the provider to avoid actual LLM API calls\n with patch.object(client.chat.completions, \"_tool_runner\") as mock_runner:\n mock_runner.return_value = create_mock_response(\"Files listed successfully\")\n\n response = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"List all files\"}],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"filesystem\",\n \"command\": \"npx\",\n \"args\": [\n \"-y\",\n \"@modelcontextprotocol/server-filesystem\",\n temp_test_dir,\n ],\n }\n ],\n max_turns=2,\n )\n\n # Verify response\n assert response.choices[0].message.content == \"Files listed successfully\"\n\n # Verify tool_runner was called with processed tools\n assert mock_runner.called\n call_args = mock_runner.call_args\n tools_arg = call_args[0][3] # tools is 4th positional arg\n\n # Verify tools were converted to callables\n assert isinstance(tools_arg, list)\n assert all(callable(t) for t in tools_arg)\n\n def test_config_dict_with_allowed_tools(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test MCP config dict with allowed_tools filtering.\"\"\"\n client = Client()\n\n with patch.object(client.chat.completions, \"_tool_runner\") as mock_runner:\n mock_runner.return_value = create_mock_response(\"File read successfully\")\n\n response = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"Read test.txt\"}],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"filesystem\",\n \"command\": \"npx\",\n \"args\": [\n \"-y\",\n \"@modelcontextprotocol/server-filesystem\",\n temp_test_dir,\n ],\n \"allowed_tools\": [\"read_file\"], # Only allow reading\n }\n ],\n max_turns=2,\n )\n\n assert response.choices[0].message.content == \"File read successfully\"\n\n # Verify only read_file tool was passed\n call_args = mock_runner.call_args\n tools_arg = call_args[0][3]\n\n assert len(tools_arg) == 1\n assert tools_arg[0].__name__ == \"read_file\"\n\n def test_config_dict_with_prefixing(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test MCP config dict with tool name prefixing.\"\"\"\n client = Client()\n\n with patch.object(client.chat.completions, \"_tool_runner\") as mock_runner:\n mock_runner.return_value = create_mock_response(\"Success\")\n\n response = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"Test\"}],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"docs\",\n \"command\": \"npx\",\n \"args\": [\n \"-y\",\n \"@modelcontextprotocol/server-filesystem\",\n temp_test_dir,\n ],\n \"use_tool_prefix\": True,\n }\n ],\n max_turns=2,\n )\n\n # Verify tools have prefixes\n call_args = mock_runner.call_args\n tools_arg = call_args[0][3]\n\n tool_names = [t.__name__ for t in tools_arg]\n assert any(name.startswith(\"docs__\") for name in tool_names)\n\n\n@pytest.mark.integration\nclass TestMCPWithPythonFunctions:\n \"\"\"Test mixing MCP tools with regular Python functions.\"\"\"\n\n def test_mix_mcp_and_python_functions(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test using MCP config dict alongside Python functions.\"\"\"\n client = Client()\n\n # Define a Python function\n def get_current_time() -> str:\n \"\"\"Get the current time.\"\"\"\n return \"2025-01-01 12:00:00\"\n\n with patch.object(client.chat.completions, \"_tool_runner\") as mock_runner:\n mock_runner.return_value = create_mock_response(\"Mixed tools work!\")\n\n response = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"What time is it?\"}],\n tools=[\n get_current_time, # Python function\n {\n \"type\": \"mcp\",\n \"name\": \"filesystem\",\n \"command\": \"npx\",\n \"args\": [\n \"-y\",\n \"@modelcontextprotocol/server-filesystem\",\n temp_test_dir,\n ],\n }, # MCP config\n ],\n max_turns=2,\n )\n\n assert response.choices[0].message.content == \"Mixed tools work!\"\n\n # Verify both types of tools were passed\n call_args = mock_runner.call_args\n tools_arg = call_args[0][3]\n\n # Should have Python function + MCP tools\n assert len(tools_arg) > 1\n\n # Verify Python function is in there\n assert any(t.__name__ == \"get_current_time\" for t in tools_arg)\n\n # Verify MCP tools are in there\n assert any(t.__name__ == \"read_file\" for t in tools_arg)\n\n\n@pytest.mark.integration\nclass TestMultipleMCPServers:\n \"\"\"Test using multiple MCP servers simultaneously.\"\"\"\n\n def test_multiple_servers_with_prefixing(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test multiple MCP servers with tool name prefixing to avoid collisions.\"\"\"\n import tempfile\n\n client = Client()\n\n # Create a second temp directory\n with tempfile.TemporaryDirectory() as temp_dir_2:\n with patch.object(client.chat.completions, \"_tool_runner\") as mock_runner:\n mock_runner.return_value = create_mock_response(\n \"Multiple servers work!\"\n )\n\n response = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"Compare directories\"}],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"dir1\",\n \"command\": \"npx\",\n \"args\": [\n \"-y\",\n \"@modelcontextprotocol/server-filesystem\",\n temp_test_dir,\n ],\n \"use_tool_prefix\": True,\n },\n {\n \"type\": \"mcp\",\n \"name\": \"dir2\",\n \"command\": \"npx\",\n \"args\": [\n \"-y\",\n \"@modelcontextprotocol/server-filesystem\",\n temp_dir_2,\n ],\n \"use_tool_prefix\": True,\n },\n ],\n max_turns=2,\n )\n\n assert response.choices[0].message.content == \"Multiple servers work!\"\n\n # Verify tools from both servers with prefixes\n call_args = mock_runner.call_args\n tools_arg = call_args[0][3]\n\n tool_names = [t.__name__ for t in tools_arg]\n\n # Should have tools from both servers\n assert any(name.startswith(\"dir1__\") for name in tool_names)\n assert any(name.startswith(\"dir2__\") for name in tool_names)\n\n # Should have both read_file tools with different prefixes\n assert \"dir1__read_file\" in tool_names\n assert \"dir2__read_file\" in tool_names\n\n\n@pytest.mark.integration\nclass TestMCPCleanup:\n \"\"\"Test that MCP clients are properly cleaned up.\"\"\"\n\n def test_cleanup_after_success(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test MCP clients are cleaned up after successful request.\"\"\"\n client = Client()\n\n with patch.object(client.chat.completions, \"_tool_runner\") as mock_runner:\n mock_runner.return_value = create_mock_response(\"Success\")\n\n # Patch MCPClient to track close() calls\n with patch(\"aisuite.client.MCPClient\") as mock_mcp_class:\n mock_mcp_instance = MagicMock()\n mock_mcp_class.from_config.return_value = mock_mcp_instance\n mock_mcp_instance.get_callable_tools.return_value = []\n\n response = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"Test\"}],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"filesystem\",\n \"command\": \"npx\",\n \"args\": [\n \"-y\",\n \"@modelcontextprotocol/server-filesystem\",\n temp_test_dir,\n ],\n }\n ],\n max_turns=2,\n )\n\n # Verify MCP client was used as context manager (cleanup called)\n mock_mcp_instance.__enter__.assert_called_once()\n mock_mcp_instance.__exit__.assert_called_once()\n\n def test_cleanup_after_error(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test MCP clients are cleaned up even after error.\"\"\"\n client = Client()\n\n with patch.object(client.chat.completions, \"_tool_runner\") as mock_runner:\n # Make tool_runner raise an error\n mock_runner.side_effect = ValueError(\"Test error\")\n\n # Patch MCPClient to track close() calls\n with patch(\"aisuite.client.MCPClient\") as mock_mcp_class:\n mock_mcp_instance = MagicMock()\n mock_mcp_class.from_config.return_value = mock_mcp_instance\n mock_mcp_instance.get_callable_tools.return_value = []\n\n with pytest.raises(ValueError, match=\"Test error\"):\n client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"Test\"}],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"filesystem\",\n \"command\": \"npx\",\n \"args\": [\n \"-y\",\n \"@modelcontextprotocol/server-filesystem\",\n temp_test_dir,\n ],\n }\n ],\n max_turns=2,\n )\n\n # Even after error, MCP client context manager exit should be called\n mock_mcp_instance.__enter__.assert_called_once()\n mock_mcp_instance.__exit__.assert_called_once()\n\n\n@pytest.mark.integration\nclass TestMCPErrorHandling:\n \"\"\"Test error handling for MCP integration.\"\"\"\n\n def test_invalid_mcp_config_raises_error(self):\n \"\"\"Test that invalid MCP config raises clear error.\"\"\"\n client = Client()\n\n with pytest.raises(ValueError, match=\"must have 'name'\"):\n client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"Test\"}],\n tools=[\n {\n \"type\": \"mcp\",\n # Missing 'name' field\n \"command\": \"npx\",\n \"args\": [\"server\"],\n }\n ],\n max_turns=2,\n )\n\n def test_mcp_not_installed_raises_error(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test that helpful error is raised if MCP package not installed.\"\"\"\n client = Client()\n\n # Simulate MCP not being installed\n with patch(\"aisuite.client.MCP_AVAILABLE\", False):\n with pytest.raises(ImportError, match=\"mcp.*package\"):\n client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[{\"role\": \"user\", \"content\": \"Test\"}],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"filesystem\",\n \"command\": \"npx\",\n \"args\": [\n \"-y\",\n \"@modelcontextprotocol/server-filesystem\",\n temp_test_dir,\n ],\n }\n ],\n max_turns=2,\n )\n" }, { "path": "tests/mcp/test_http_llm_e2e.py", "content": "\"\"\"\nReal LLM End-to-End Tests for HTTP MCP Integration.\n\nThese tests make ACTUAL API calls to LLM providers (OpenAI, Anthropic) to verify\nthat HTTP-based MCP tools work correctly with real models. Unlike test_http_transport.py\nwhich mocks HTTP responses, these tests verify the complete integration stack.\n\n⚠️ WARNING: These tests will make real API calls and incur costs!\n - Each test costs ~$0.01-0.05 depending on the model\n - Tests are marked with @pytest.mark.llm\n - Tests are skipped if API keys are not present\n\nMCP Servers Used:\n - Context7 (https://mcp.context7.com/mcp)\n - Public HTTP MCP server for library documentation\n - No authentication required (optional API key for higher limits)\n - Tools: resolve-library-id, get-library-docs\n\n - Exa (https://mcp.exa.ai/mcp)\n - Web search and code context search\n - Requires EXA_API_KEY for authentication\n - Tools: web_search_exa, get_code_context_exa\n\nRequirements:\n - API keys in .env file:\n OPENAI_API_KEY=your-key\n ANTHROPIC_API_KEY=your-key\n EXA_API_KEY=your-key (for Exa tests only)\n - pytest-asyncio, python-dotenv\n\nRunning:\n # Run ONLY HTTP LLM tests (⚠️ costs money):\n pytest tests/mcp/test_http_llm_e2e.py -v -m llm\n\n # Skip LLM tests (default, free):\n pytest tests/mcp/ -v -m \"integration and not llm\"\n\"\"\"\n\nimport pytest\nimport os\nfrom aisuite import Client\n\n\n# Helper functions to check if we have API keys\ndef has_openai_key():\n \"\"\"Check if OpenAI API key is available.\"\"\"\n return bool(os.getenv(\"OPENAI_API_KEY\"))\n\n\ndef has_anthropic_key():\n \"\"\"Check if Anthropic API key is available.\"\"\"\n return bool(os.getenv(\"ANTHROPIC_API_KEY\"))\n\n\ndef has_exa_key():\n \"\"\"Check if Exa API key is available.\"\"\"\n return bool(os.getenv(\"EXA_API_KEY\"))\n\n\n@pytest.mark.llm\n@pytest.mark.integration\nclass TestOpenAIWithHTTPMCP:\n \"\"\"Test OpenAI models with HTTP MCP tools (Context7).\"\"\"\n\n @pytest.mark.skipif(not has_openai_key(), reason=\"OPENAI_API_KEY not set\")\n def test_gpt4o_resolves_library_via_http_mcp(self):\n \"\"\"Test GPT-4o can resolve library names using HTTP MCP.\"\"\"\n client = Client()\n\n response = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": 'Use resolve-library-id to resolve the library name \"requests\" and tell me the library ID.',\n }\n ],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"context7\",\n \"server_url\": \"https://mcp.context7.com/mcp\",\n \"allowed_tools\": [\"resolve-library-id\"],\n }\n ],\n max_turns=3,\n )\n\n # Verify the LLM used the HTTP MCP tool\n content = response.choices[0].message.content.lower()\n # Should mention requests or library ID\n assert any(\n keyword in content\n for keyword in [\"requests\", \"library\", \"id\", \"pypi\", \"python\"]\n ), f\"Expected library resolution info in response, got: {content}\"\n\n @pytest.mark.skipif(not has_openai_key(), reason=\"OPENAI_API_KEY not set\")\n def test_gpt4o_gets_library_docs_via_http_mcp(self):\n \"\"\"Test GPT-4o can get library documentation using HTTP MCP.\"\"\"\n client = Client()\n\n response = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": 'First use resolve-library-id to get the ID for \"requests\", then use get-library-docs to fetch its documentation.',\n }\n ],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"context7\",\n \"server_url\": \"https://mcp.context7.com/mcp\",\n \"allowed_tools\": [\"resolve-library-id\", \"get-library-docs\"],\n \"timeout\": 90.0, # Increase timeout - docs fetching can be slow\n }\n ],\n max_turns=5,\n )\n\n # Verify the LLM got documentation\n content = response.choices[0].message.content.lower()\n # Should mention documentation or requests library\n assert any(\n keyword in content\n for keyword in [\"documentation\", \"requests\", \"http\", \"api\", \"library\"]\n ), f\"Expected documentation content in response, got: {content}\"\n\n @pytest.mark.skipif(not has_openai_key(), reason=\"OPENAI_API_KEY not set\")\n def test_gpt4o_mixed_tools_http(self):\n \"\"\"Test GPT-4o with both HTTP MCP tools and regular Python functions.\"\"\"\n\n # Define a Python function\n def get_current_year() -> str:\n \"\"\"Get the current year.\"\"\"\n from datetime import datetime\n\n return str(datetime.now().year)\n\n client = Client()\n\n response = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": 'First use get_current_year to get the year, then use resolve-library-id to resolve \"requests\".',\n }\n ],\n tools=[\n get_current_year, # Python function\n {\n \"type\": \"mcp\",\n \"name\": \"context7\",\n \"server_url\": \"https://mcp.context7.com/mcp\",\n \"allowed_tools\": [\"resolve-library-id\"],\n }, # HTTP MCP\n ],\n max_turns=5,\n )\n\n # Verify both tools were used\n content = response.choices[0].message.content.lower()\n # Should mention the year (from Python function)\n assert any(\n str(y) in content for y in [2024, 2025, 2026]\n ), f\"Expected year in response, got: {content}\"\n # Should mention requests or library (from HTTP MCP tool)\n assert any(\n keyword in content for keyword in [\"requests\", \"library\", \"id\"]\n ), f\"Expected library info in response, got: {content}\"\n\n\n@pytest.mark.llm\n@pytest.mark.integration\nclass TestAnthropicWithHTTPMCP:\n \"\"\"Test Anthropic Claude models with HTTP MCP tools (Context7).\"\"\"\n\n @pytest.mark.skipif(not has_anthropic_key(), reason=\"ANTHROPIC_API_KEY not set\")\n def test_claude_resolves_library_via_http_mcp(self):\n \"\"\"Test Claude can resolve library names using HTTP MCP.\"\"\"\n client = Client()\n\n response = client.chat.completions.create(\n model=\"anthropic:claude-sonnet-4-5\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": 'Use resolve-library-id to resolve the library name \"flask\" and tell me the library ID.',\n }\n ],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"context7\",\n \"server_url\": \"https://mcp.context7.com/mcp\",\n \"allowed_tools\": [\"resolve-library-id\"],\n }\n ],\n max_turns=3,\n )\n\n # Verify Claude used the HTTP MCP tool\n content = response.choices[0].message.content.lower()\n assert any(\n keyword in content\n for keyword in [\"flask\", \"library\", \"id\", \"pypi\", \"python\"]\n ), f\"Expected library resolution info in response, got: {content}\"\n\n @pytest.mark.skipif(not has_anthropic_key(), reason=\"ANTHROPIC_API_KEY not set\")\n def test_claude_gets_library_docs_via_http_mcp(self):\n \"\"\"Test Claude can get library documentation using HTTP MCP.\"\"\"\n client = Client()\n\n response = client.chat.completions.create(\n model=\"anthropic:claude-sonnet-4-5\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": 'Use resolve-library-id to get the ID for \"flask\", then use get-library-docs to fetch documentation.',\n }\n ],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"context7\",\n \"server_url\": \"https://mcp.context7.com/mcp\",\n \"allowed_tools\": [\"resolve-library-id\", \"get-library-docs\"],\n \"timeout\": 90.0, # Increase timeout - docs fetching can be slow\n }\n ],\n max_turns=5,\n )\n\n # Verify Claude got documentation\n content = response.choices[0].message.content.lower()\n assert any(\n keyword in content\n for keyword in [\"documentation\", \"flask\", \"web\", \"library\"]\n ), f\"Expected documentation content in response, got: {content}\"\n\n @pytest.mark.skipif(not has_anthropic_key(), reason=\"ANTHROPIC_API_KEY not set\")\n def test_claude_mixed_tools_http(self):\n \"\"\"Test Claude with both HTTP MCP tools and regular Python functions.\"\"\"\n\n def get_language() -> str:\n \"\"\"Get the primary programming language.\"\"\"\n return \"Python\"\n\n client = Client()\n\n response = client.chat.completions.create(\n model=\"anthropic:claude-sonnet-4-5\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": 'Use get_language to get the language, then use resolve-library-id to resolve \"django\". Tell me both.',\n }\n ],\n tools=[\n get_language, # Python function\n {\n \"type\": \"mcp\",\n \"name\": \"context7\",\n \"server_url\": \"https://mcp.context7.com/mcp\",\n \"allowed_tools\": [\"resolve-library-id\"],\n }, # HTTP MCP\n ],\n max_turns=5,\n )\n\n # Verify both tools were used\n content = response.choices[0].message.content.lower()\n # Should mention Python (from Python function)\n assert \"python\" in content, f\"Expected Python in response, got: {content}\"\n # Should mention django or library (from HTTP MCP tool)\n assert any(\n keyword in content for keyword in [\"django\", \"library\", \"id\"]\n ), f\"Expected library info in response, got: {content}\"\n\n\n@pytest.mark.llm\n@pytest.mark.integration\nclass TestHTTPMCPConfigDict:\n \"\"\"Test HTTP MCP with config dict format.\"\"\"\n\n @pytest.mark.skipif(not has_openai_key(), reason=\"OPENAI_API_KEY not set\")\n def test_http_mcp_config_dict_format(self):\n \"\"\"Test that HTTP MCP works with config dict format.\"\"\"\n client = Client()\n\n # Using config dict format (not explicit MCPClient)\n response = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": 'Use resolve-library-id to resolve the library name \"numpy\".',\n }\n ],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"context7\",\n \"server_url\": \"https://mcp.context7.com/mcp\",\n \"timeout\": 60.0, # Test timeout parameter\n \"allowed_tools\": [\"resolve-library-id\"],\n }\n ],\n max_turns=3,\n )\n\n # Verify it worked\n content = response.choices[0].message.content.lower()\n assert any(\n keyword in content\n for keyword in [\"numpy\", \"library\", \"id\", \"pypi\", \"python\"]\n ), f\"Expected library resolution info in response, got: {content}\"\n\n\n@pytest.mark.llm\n@pytest.mark.integration\nclass TestHTTPMCPWithHeaders:\n \"\"\"Test HTTP MCP with custom headers.\"\"\"\n\n @pytest.mark.skipif(not has_openai_key(), reason=\"OPENAI_API_KEY not set\")\n def test_http_mcp_with_headers(self):\n \"\"\"Test that HTTP MCP accepts custom headers (Context7 supports optional API key).\"\"\"\n client = Client()\n\n # Context7 doesn't require auth for basic usage, but supports it\n response = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": 'Use resolve-library-id to resolve \"pandas\".',\n }\n ],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"context7\",\n \"server_url\": \"https://mcp.context7.com/mcp\",\n \"headers\": {\n \"User-Agent\": \"aisuite-test\"\n }, # Custom header (optional)\n \"allowed_tools\": [\"resolve-library-id\"],\n }\n ],\n max_turns=3,\n )\n\n # Verify it worked with headers\n content = response.choices[0].message.content.lower()\n assert any(\n keyword in content for keyword in [\"pandas\", \"library\", \"id\", \"data\"]\n ), f\"Expected library info in response, got: {content}\"\n\n\n@pytest.mark.llm\n@pytest.mark.integration\nclass TestOpenAIWithExaMCP:\n \"\"\"Test OpenAI models with Exa HTTP MCP tools.\"\"\"\n\n @pytest.mark.skipif(\n not has_openai_key() or not has_exa_key(),\n reason=\"OPENAI_API_KEY or EXA_API_KEY not set\",\n )\n def test_gpt4o_web_search_via_exa(self):\n \"\"\"Test GPT-4o can perform web search using Exa.\"\"\"\n client = Client()\n\n exa_api_key = os.getenv(\"EXA_API_KEY\")\n\n response = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": \"Search for recent Python 3.12 features and summarize the top 2.\",\n }\n ],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"exa\",\n \"server_url\": \"https://mcp.exa.ai/mcp\",\n \"headers\": {\"Authorization\": f\"Bearer {exa_api_key}\"},\n \"allowed_tools\": [\"web_search_exa\"],\n \"timeout\": 60.0,\n }\n ],\n max_turns=3,\n )\n\n # Verify search results\n content = response.choices[0].message.content.lower()\n assert any(\n keyword in content for keyword in [\"python\", \"3.12\", \"feature\"]\n ), f\"Expected Python 3.12 info in response, got: {content}\"\n\n @pytest.mark.skipif(\n not has_openai_key() or not has_exa_key(),\n reason=\"OPENAI_API_KEY or EXA_API_KEY not set\",\n )\n def test_gpt4o_code_context_via_exa(self):\n \"\"\"Test GPT-4o can search code context using Exa.\"\"\"\n client = Client()\n\n exa_api_key = os.getenv(\"EXA_API_KEY\")\n\n response = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": \"Find an example of using asyncio.gather in Python.\",\n }\n ],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"exa\",\n \"server_url\": \"https://mcp.exa.ai/mcp\",\n \"headers\": {\"Authorization\": f\"Bearer {exa_api_key}\"},\n \"allowed_tools\": [\"get_code_context_exa\"],\n \"timeout\": 60.0,\n }\n ],\n max_turns=3,\n )\n\n # Verify code context results\n content = response.choices[0].message.content.lower()\n assert any(\n keyword in content for keyword in [\"asyncio\", \"gather\", \"async\", \"await\"]\n ), f\"Expected asyncio.gather info in response, got: {content}\"\n\n @pytest.mark.skipif(\n not has_openai_key() or not has_exa_key(),\n reason=\"OPENAI_API_KEY or EXA_API_KEY not set\",\n )\n def test_gpt4o_mixed_tools_with_exa(self):\n \"\"\"Test GPT-4o with both Exa tools and Python functions.\"\"\"\n\n def get_current_year() -> str:\n \"\"\"Get the current year.\"\"\"\n from datetime import datetime\n\n return str(datetime.now().year)\n\n client = Client()\n exa_api_key = os.getenv(\"EXA_API_KEY\")\n\n response = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": \"Get the current year, then search for major tech events from that year.\",\n }\n ],\n tools=[\n get_current_year, # Python function\n {\n \"type\": \"mcp\",\n \"name\": \"exa\",\n \"server_url\": \"https://mcp.exa.ai/mcp\",\n \"headers\": {\"Authorization\": f\"Bearer {exa_api_key}\"},\n \"allowed_tools\": [\"web_search_exa\"],\n \"timeout\": 60.0,\n }, # Exa HTTP MCP\n ],\n max_turns=4,\n )\n\n # Verify both tools were used\n content = response.choices[0].message.content.lower()\n # Should mention the year (from Python function)\n assert any(\n str(y) in content for y in [2024, 2025, 2026]\n ), f\"Expected year in response, got: {content}\"\n # Should mention tech or events (from web search)\n assert any(\n keyword in content for keyword in [\"tech\", \"event\", \"technology\"]\n ), f\"Expected tech events in response, got: {content}\"\n\n\n@pytest.mark.llm\n@pytest.mark.integration\nclass TestAnthropicWithExaMCP:\n \"\"\"Test Anthropic Claude models with Exa HTTP MCP tools.\"\"\"\n\n @pytest.mark.skipif(\n not has_anthropic_key() or not has_exa_key(),\n reason=\"ANTHROPIC_API_KEY or EXA_API_KEY not set\",\n )\n def test_claude_web_search_via_exa(self):\n \"\"\"Test Claude can perform web search using Exa.\"\"\"\n client = Client()\n\n exa_api_key = os.getenv(\"EXA_API_KEY\")\n\n response = client.chat.completions.create(\n model=\"anthropic:claude-sonnet-4-5\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": \"Search for information about Rust programming language features.\",\n }\n ],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"exa\",\n \"server_url\": \"https://mcp.exa.ai/mcp\",\n \"headers\": {\"Authorization\": f\"Bearer {exa_api_key}\"},\n \"allowed_tools\": [\"web_search_exa\"],\n \"timeout\": 60.0,\n }\n ],\n max_turns=3,\n )\n\n # Verify search results\n content = response.choices[0].message.content.lower()\n assert any(\n keyword in content for keyword in [\"rust\", \"programming\", \"language\"]\n ), f\"Expected Rust info in response, got: {content}\"\n\n @pytest.mark.skipif(\n not has_anthropic_key() or not has_exa_key(),\n reason=\"ANTHROPIC_API_KEY or EXA_API_KEY not set\",\n )\n def test_claude_code_context_via_exa(self):\n \"\"\"Test Claude can search code context using Exa.\"\"\"\n client = Client()\n\n exa_api_key = os.getenv(\"EXA_API_KEY\")\n\n response = client.chat.completions.create(\n model=\"anthropic:claude-sonnet-4-5\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": \"Find code examples for FastAPI route decorators.\",\n }\n ],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"exa\",\n \"server_url\": \"https://mcp.exa.ai/mcp\",\n \"headers\": {\"Authorization\": f\"Bearer {exa_api_key}\"},\n \"allowed_tools\": [\"get_code_context_exa\"],\n \"timeout\": 60.0,\n }\n ],\n max_turns=3,\n )\n\n # Verify code context results\n content = response.choices[0].message.content.lower()\n assert any(\n keyword in content for keyword in [\"fastapi\", \"route\", \"decorator\", \"@\"]\n ), f\"Expected FastAPI route info in response, got: {content}\"\n\n @pytest.mark.skipif(\n not has_anthropic_key() or not has_exa_key(),\n reason=\"ANTHROPIC_API_KEY or EXA_API_KEY not set\",\n )\n def test_claude_mixed_tools_with_exa(self):\n \"\"\"Test Claude with both Exa tools and Python functions.\"\"\"\n\n def get_language() -> str:\n \"\"\"Get the primary programming language.\"\"\"\n return \"Python\"\n\n client = Client()\n exa_api_key = os.getenv(\"EXA_API_KEY\")\n\n response = client.chat.completions.create(\n model=\"anthropic:claude-sonnet-4-5\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": \"Get the language name, then search for its latest version features.\",\n }\n ],\n tools=[\n get_language, # Python function\n {\n \"type\": \"mcp\",\n \"name\": \"exa\",\n \"server_url\": \"https://mcp.exa.ai/mcp\",\n \"headers\": {\"Authorization\": f\"Bearer {exa_api_key}\"},\n \"allowed_tools\": [\"web_search_exa\"],\n \"timeout\": 60.0,\n }, # Exa HTTP MCP\n ],\n max_turns=4,\n )\n\n # Verify both tools were used\n content = response.choices[0].message.content.lower()\n # Should mention Python (from Python function)\n assert \"python\" in content, f\"Expected Python in response, got: {content}\"\n # Should mention version or features (from web search)\n assert any(\n keyword in content for keyword in [\"version\", \"feature\", \"3.\"]\n ), f\"Expected version info in response, got: {content}\"\n" }, { "path": "tests/mcp/test_http_transport.py", "content": "\"\"\"\nTests for MCP HTTP Transport.\n\nThese tests verify that the MCPClient works correctly with HTTP-based MCP servers.\nAll HTTP requests are mocked to avoid requiring a real HTTP MCP server.\n\"\"\"\n\nimport pytest\nfrom unittest.mock import AsyncMock, MagicMock, patch\nimport json\nimport httpx\nfrom aisuite.mcp.client import MCPClient\n\n\n@pytest.mark.integration\nclass TestHTTPTransportBasics:\n \"\"\"Test basic HTTP transport functionality.\"\"\"\n\n def test_create_http_client_success(self):\n \"\"\"Test creating an HTTP MCPClient with valid parameters.\"\"\"\n with patch(\"aisuite.mcp.client.httpx.AsyncClient\") as mock_async_client:\n # Mock the HTTP client\n mock_client_instance = AsyncMock()\n mock_async_client.return_value = mock_client_instance\n\n # Mock initialize response\n mock_response_init = MagicMock()\n mock_response_init.headers = {\"content-type\": \"application/json\"}\n mock_response_init.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 1,\n \"result\": {\n \"protocolVersion\": \"2024-11-05\",\n \"serverInfo\": {\"name\": \"test-server\", \"version\": \"1.0.0\"},\n },\n }\n mock_response_init.raise_for_status = MagicMock()\n\n # Mock tools/list response\n mock_response_tools = MagicMock()\n mock_response_tools.headers = {\"content-type\": \"application/json\"}\n mock_response_tools.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 2,\n \"result\": {\n \"tools\": [\n {\n \"name\": \"test_tool\",\n \"description\": \"A test tool\",\n \"inputSchema\": {\n \"type\": \"object\",\n \"properties\": {\"param\": {\"type\": \"string\"}},\n },\n }\n ]\n },\n }\n mock_response_tools.raise_for_status = MagicMock()\n\n # Set up post responses in order (init + notification + tools)\n mock_client_instance.post = AsyncMock(\n side_effect=[\n mock_response_init,\n MagicMock(), # initialized notification\n mock_response_tools,\n ]\n )\n\n # Create client\n mcp = MCPClient(server_url=\"http://localhost:8000\", name=\"test-server\")\n\n # Verify client was created\n assert mcp.server_url == \"http://localhost:8000\"\n assert mcp.name == \"test-server\"\n assert len(mcp.list_tools()) == 1\n assert mcp.list_tools()[0][\"name\"] == \"test_tool\"\n\n # Cleanup\n mcp.close()\n\n def test_create_http_client_with_headers(self):\n \"\"\"Test creating an HTTP MCPClient with custom headers.\"\"\"\n with patch(\"aisuite.mcp.client.httpx.AsyncClient\") as mock_async_client:\n mock_client_instance = AsyncMock()\n mock_async_client.return_value = mock_client_instance\n\n # Mock responses\n mock_response_init = MagicMock()\n mock_response_init.headers = {\"content-type\": \"application/json\"}\n mock_response_init.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 1,\n \"result\": {\"protocolVersion\": \"2024-11-05\"},\n }\n mock_response_init.raise_for_status = MagicMock()\n\n mock_response_tools = MagicMock()\n mock_response_tools.headers = {\"content-type\": \"application/json\"}\n mock_response_tools.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 2,\n \"result\": {\"tools\": []},\n }\n mock_response_tools.raise_for_status = MagicMock()\n\n mock_client_instance.post = AsyncMock(\n side_effect=[mock_response_init, MagicMock(), mock_response_tools]\n )\n\n # Create client with headers\n headers = {\"Authorization\": \"Bearer secret-token\"}\n mcp = MCPClient(\n server_url=\"http://localhost:8000\", headers=headers, name=\"test\"\n )\n\n # Verify headers were stored\n assert mcp.headers == headers\n\n # Cleanup\n mcp.close()\n\n def test_http_client_validation_errors(self):\n \"\"\"Test that validation errors are raised for invalid parameters.\"\"\"\n # Test: no command or server_url\n with pytest.raises(ValueError, match=\"Must provide either\"):\n MCPClient()\n\n # Test: both command and server_url\n with pytest.raises(ValueError, match=\"Cannot mix stdio parameters\"):\n MCPClient(command=\"npx\", server_url=\"http://localhost:8000\")\n\n\n@pytest.mark.integration\nclass TestHTTPToolCalling:\n \"\"\"Test HTTP tool discovery and calling.\"\"\"\n\n def test_list_tools_http(self):\n \"\"\"Test listing tools via HTTP transport.\"\"\"\n with patch(\"aisuite.mcp.client.httpx.AsyncClient\") as mock_async_client:\n mock_client_instance = AsyncMock()\n mock_async_client.return_value = mock_client_instance\n\n # Mock responses\n mock_response_init = MagicMock()\n mock_response_init.headers = {\"content-type\": \"application/json\"}\n mock_response_init.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 1,\n \"result\": {},\n }\n mock_response_init.raise_for_status = MagicMock()\n\n mock_response_tools = MagicMock()\n mock_response_tools.headers = {\"content-type\": \"application/json\"}\n mock_response_tools.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 2,\n \"result\": {\n \"tools\": [\n {\n \"name\": \"tool1\",\n \"description\": \"First tool\",\n \"inputSchema\": {},\n },\n {\n \"name\": \"tool2\",\n \"description\": \"Second tool\",\n \"inputSchema\": {},\n },\n ]\n },\n }\n mock_response_tools.raise_for_status = MagicMock()\n\n mock_client_instance.post = AsyncMock(\n side_effect=[mock_response_init, MagicMock(), mock_response_tools]\n )\n\n mcp = MCPClient(server_url=\"http://localhost:8000\")\n\n tools = mcp.list_tools()\n assert len(tools) == 2\n assert tools[0][\"name\"] == \"tool1\"\n assert tools[1][\"name\"] == \"tool2\"\n\n mcp.close()\n\n def test_call_tool_http(self):\n \"\"\"Test calling a tool via HTTP transport.\"\"\"\n with patch(\"aisuite.mcp.client.httpx.AsyncClient\") as mock_async_client:\n mock_client_instance = AsyncMock()\n mock_async_client.return_value = mock_client_instance\n\n # Mock init and tools/list\n mock_response_init = MagicMock()\n mock_response_init.headers = {\"content-type\": \"application/json\"}\n mock_response_init.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 1,\n \"result\": {},\n }\n mock_response_init.raise_for_status = MagicMock()\n\n mock_response_tools = MagicMock()\n mock_response_tools.headers = {\"content-type\": \"application/json\"}\n mock_response_tools.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 2,\n \"result\": {\n \"tools\": [\n {\n \"name\": \"echo\",\n \"description\": \"Echo tool\",\n \"inputSchema\": {\n \"type\": \"object\",\n \"properties\": {\"message\": {\"type\": \"string\"}},\n },\n }\n ]\n },\n }\n mock_response_tools.raise_for_status = MagicMock()\n\n # Mock tool call response\n mock_response_call = MagicMock()\n mock_response_call.headers = {\"content-type\": \"application/json\"}\n mock_response_call.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 3,\n \"result\": {\"content\": [{\"text\": \"Hello, World!\"}]},\n }\n mock_response_call.raise_for_status = MagicMock()\n\n mock_client_instance.post = AsyncMock(\n side_effect=[\n mock_response_init,\n MagicMock(),\n mock_response_tools,\n mock_response_call,\n ]\n )\n\n mcp = MCPClient(server_url=\"http://localhost:8000\")\n\n # Call tool\n result = mcp.call_tool(\"echo\", {\"message\": \"Hello\"})\n assert result == \"Hello, World!\"\n\n mcp.close()\n\n def test_get_callable_tools_http(self):\n \"\"\"Test getting callable tools via HTTP transport.\"\"\"\n with patch(\"aisuite.mcp.client.httpx.AsyncClient\") as mock_async_client:\n mock_client_instance = AsyncMock()\n mock_async_client.return_value = mock_client_instance\n\n # Mock responses\n mock_response_init = MagicMock()\n mock_response_init.headers = {\"content-type\": \"application/json\"}\n mock_response_init.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 1,\n \"result\": {},\n }\n mock_response_init.raise_for_status = MagicMock()\n\n mock_response_tools = MagicMock()\n mock_response_tools.headers = {\"content-type\": \"application/json\"}\n mock_response_tools.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 2,\n \"result\": {\n \"tools\": [\n {\n \"name\": \"test_tool\",\n \"description\": \"A test tool\",\n \"inputSchema\": {\"type\": \"object\", \"properties\": {}},\n }\n ]\n },\n }\n mock_response_tools.raise_for_status = MagicMock()\n\n mock_client_instance.post = AsyncMock(\n side_effect=[mock_response_init, MagicMock(), mock_response_tools]\n )\n\n mcp = MCPClient(server_url=\"http://localhost:8000\")\n\n tools = mcp.get_callable_tools()\n assert len(tools) == 1\n assert callable(tools[0])\n assert tools[0].__name__ == \"test_tool\"\n\n mcp.close()\n\n\n@pytest.mark.integration\nclass TestHTTPFromConfig:\n \"\"\"Test creating HTTP MCPClient from config dict.\"\"\"\n\n def test_from_config_http(self):\n \"\"\"Test creating HTTP client from config.\"\"\"\n with patch(\"aisuite.mcp.client.httpx.AsyncClient\") as mock_async_client:\n mock_client_instance = AsyncMock()\n mock_async_client.return_value = mock_client_instance\n\n # Mock responses\n mock_response_init = MagicMock()\n mock_response_init.headers = {\"content-type\": \"application/json\"}\n mock_response_init.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 1,\n \"result\": {},\n }\n mock_response_init.raise_for_status = MagicMock()\n\n mock_response_tools = MagicMock()\n mock_response_tools.headers = {\"content-type\": \"application/json\"}\n mock_response_tools.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 2,\n \"result\": {\"tools\": []},\n }\n mock_response_tools.raise_for_status = MagicMock()\n\n mock_client_instance.post = AsyncMock(\n side_effect=[mock_response_init, MagicMock(), mock_response_tools]\n )\n\n config = {\n \"type\": \"mcp\",\n \"name\": \"test-server\",\n \"server_url\": \"http://localhost:8000\",\n \"headers\": {\"Authorization\": \"Bearer token\"},\n \"timeout\": 60.0,\n }\n\n mcp = MCPClient.from_config(config)\n\n assert mcp.server_url == \"http://localhost:8000\"\n assert mcp.headers == {\"Authorization\": \"Bearer token\"}\n assert mcp.timeout == 60.0\n assert mcp.name == \"test-server\"\n\n mcp.close()\n\n def test_get_tools_from_config_http(self):\n \"\"\"Test getting tools from HTTP config.\"\"\"\n with patch(\"aisuite.mcp.client.httpx.AsyncClient\") as mock_async_client:\n mock_client_instance = AsyncMock()\n mock_async_client.return_value = mock_client_instance\n\n # Mock responses\n mock_response_init = MagicMock()\n mock_response_init.headers = {\"content-type\": \"application/json\"}\n mock_response_init.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 1,\n \"result\": {},\n }\n mock_response_init.raise_for_status = MagicMock()\n\n mock_response_tools = MagicMock()\n mock_response_tools.headers = {\"content-type\": \"application/json\"}\n mock_response_tools.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 2,\n \"result\": {\n \"tools\": [\n {\"name\": \"tool1\", \"description\": \"Tool 1\", \"inputSchema\": {}},\n {\"name\": \"tool2\", \"description\": \"Tool 2\", \"inputSchema\": {}},\n ]\n },\n }\n mock_response_tools.raise_for_status = MagicMock()\n\n mock_client_instance.post = AsyncMock(\n side_effect=[mock_response_init, MagicMock(), mock_response_tools]\n )\n\n config = {\n \"type\": \"mcp\",\n \"name\": \"test\",\n \"server_url\": \"http://localhost:8000\",\n \"allowed_tools\": [\"tool1\"],\n }\n\n tools = MCPClient.get_tools_from_config(config)\n\n # Only tool1 should be returned due to allowed_tools filter\n assert len(tools) == 1\n assert tools[0].__name__ == \"tool1\"\n\n\n@pytest.mark.integration\nclass TestHTTPErrorHandling:\n \"\"\"Test error handling for HTTP transport.\"\"\"\n\n def test_http_connection_error(self):\n \"\"\"Test handling of HTTP connection errors.\"\"\"\n with patch(\"aisuite.mcp.client.httpx.AsyncClient\") as mock_async_client:\n mock_client_instance = AsyncMock()\n mock_async_client.return_value = mock_client_instance\n\n # Mock connection error\n mock_client_instance.post = AsyncMock(\n side_effect=httpx.ConnectError(\"Connection refused\")\n )\n\n with pytest.raises(RuntimeError, match=\"HTTP request to MCP server failed\"):\n MCPClient(server_url=\"http://localhost:8000\")\n\n def test_http_json_rpc_error(self):\n \"\"\"Test handling of JSON-RPC errors from server.\"\"\"\n with patch(\"aisuite.mcp.client.httpx.AsyncClient\") as mock_async_client:\n mock_client_instance = AsyncMock()\n mock_async_client.return_value = mock_client_instance\n\n # Mock JSON-RPC error response\n mock_response = MagicMock()\n mock_response.headers = {\"content-type\": \"application/json\"}\n mock_response.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 1,\n \"error\": {\"code\": -32600, \"message\": \"Invalid Request\"},\n }\n mock_response.raise_for_status = MagicMock()\n\n mock_client_instance.post = AsyncMock(return_value=mock_response)\n\n with pytest.raises(RuntimeError, match=\"MCP server error: Invalid Request\"):\n MCPClient(server_url=\"http://localhost:8000\")\n\n def test_http_status_error(self):\n \"\"\"Test handling of HTTP status errors.\"\"\"\n with patch(\"aisuite.mcp.client.httpx.AsyncClient\") as mock_async_client:\n mock_client_instance = AsyncMock()\n mock_async_client.return_value = mock_client_instance\n\n # Mock HTTP status error\n mock_client_instance.post = AsyncMock(\n side_effect=httpx.HTTPStatusError(\n \"404 Not Found\",\n request=MagicMock(),\n response=MagicMock(),\n )\n )\n\n with pytest.raises(RuntimeError, match=\"HTTP request to MCP server failed\"):\n MCPClient(server_url=\"http://localhost:8000\")\n\n\n@pytest.mark.integration\nclass TestHTTPEndpointHandling:\n \"\"\"Test that server URLs are used exactly as provided.\"\"\"\n\n def test_endpoint_uses_exact_url(self):\n \"\"\"Test that the exact server URL is used without modification.\"\"\"\n with patch(\"aisuite.mcp.client.httpx.AsyncClient\") as mock_async_client:\n mock_client_instance = AsyncMock()\n mock_async_client.return_value = mock_client_instance\n\n # Mock responses\n mock_response_init = MagicMock()\n mock_response_init.headers = {\"content-type\": \"application/json\"}\n mock_response_init.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 1,\n \"result\": {},\n }\n mock_response_init.raise_for_status = MagicMock()\n\n mock_response_tools = MagicMock()\n mock_response_tools.headers = {\"content-type\": \"application/json\"}\n mock_response_tools.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 2,\n \"result\": {\"tools\": []},\n }\n mock_response_tools.raise_for_status = MagicMock()\n\n mock_client_instance.post = AsyncMock(\n side_effect=[mock_response_init, MagicMock(), mock_response_tools]\n )\n\n # Use full endpoint URL\n mcp = MCPClient(server_url=\"http://localhost:8000/mcp/v1\")\n\n # Verify that post was called with exact URL (no modification)\n # Calls: initialize, initialized notification, tools/list\n calls = mock_client_instance.post.call_args_list\n assert len(calls) == 3\n assert calls[0][0][0] == \"http://localhost:8000/mcp/v1\" # initialize\n assert (\n calls[1][0][0] == \"http://localhost:8000/mcp/v1\"\n ) # initialized notification\n assert calls[2][0][0] == \"http://localhost:8000/mcp/v1\" # tools/list\n\n mcp.close()\n\n def test_endpoint_trailing_slash_handled(self):\n \"\"\"Test that trailing slashes in server URL are removed.\"\"\"\n with patch(\"aisuite.mcp.client.httpx.AsyncClient\") as mock_async_client:\n mock_client_instance = AsyncMock()\n mock_async_client.return_value = mock_client_instance\n\n # Mock responses\n mock_response_init = MagicMock()\n mock_response_init.headers = {\"content-type\": \"application/json\"}\n mock_response_init.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 1,\n \"result\": {},\n }\n mock_response_init.raise_for_status = MagicMock()\n\n mock_response_tools = MagicMock()\n mock_response_tools.headers = {\"content-type\": \"application/json\"}\n mock_response_tools.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 2,\n \"result\": {\"tools\": []},\n }\n mock_response_tools.raise_for_status = MagicMock()\n\n mock_client_instance.post = AsyncMock(\n side_effect=[mock_response_init, MagicMock(), mock_response_tools]\n )\n\n # URL with trailing slash\n mcp = MCPClient(server_url=\"http://localhost:8000/mcp/v1/\")\n\n # Verify trailing slash is removed\n calls = mock_client_instance.post.call_args_list\n assert calls[0][0][0] == \"http://localhost:8000/mcp/v1\"\n\n mcp.close()\n\n\n@pytest.mark.integration\nclass TestHTTPSSEResponses:\n \"\"\"Test SSE (Server-Sent Events) response handling.\"\"\"\n\n def test_sse_response_parsing(self):\n \"\"\"Test handling SSE stream responses.\"\"\"\n with patch(\"aisuite.mcp.client.httpx.AsyncClient\") as mock_async_client:\n mock_client_instance = AsyncMock()\n mock_async_client.return_value = mock_client_instance\n\n # Mock initialize (JSON response)\n mock_response_init = MagicMock()\n mock_response_init.headers = {\"content-type\": \"application/json\"}\n mock_response_init.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 1,\n \"result\": {},\n }\n mock_response_init.raise_for_status = MagicMock()\n\n # Mock tools/list (JSON response)\n mock_response_tools = MagicMock()\n mock_response_tools.headers = {\"content-type\": \"application/json\"}\n mock_response_tools.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 2,\n \"result\": {\"tools\": []},\n }\n mock_response_tools.raise_for_status = MagicMock()\n\n # Mock tool call (SSE response)\n mock_response_sse = MagicMock()\n mock_response_sse.headers = {\"content-type\": \"text/event-stream\"}\n mock_response_sse.raise_for_status = MagicMock()\n\n # Simulate SSE stream with data lines\n # MCP format: result has \"content\" array with text items\n async def mock_aiter_lines():\n lines = [\n 'data: {\"jsonrpc\": \"2.0\", \"id\": 3, \"result\": {\"content\": [{\"type\": \"text\", \"text\": \"SSE result\"}]}}',\n \"\",\n ]\n for line in lines:\n yield line\n\n mock_response_sse.aiter_lines = mock_aiter_lines\n\n mock_client_instance.post = AsyncMock(\n side_effect=[\n mock_response_init,\n MagicMock(), # initialized notification (no response checked)\n mock_response_tools,\n mock_response_sse,\n ]\n )\n\n mcp = MCPClient(server_url=\"http://localhost:8000\")\n\n # Call tool which returns SSE response\n result = mcp.call_tool(\"test_tool\", {})\n\n # Verify SSE response was parsed correctly\n # call_tool extracts the text from content array\n assert result == \"SSE result\"\n\n mcp.close()\n\n def test_session_id_management(self):\n \"\"\"Test Mcp-Session-Id header handling.\"\"\"\n with patch(\"aisuite.mcp.client.httpx.AsyncClient\") as mock_async_client:\n mock_client_instance = AsyncMock()\n mock_async_client.return_value = mock_client_instance\n\n # Mock initialize with session ID in response\n mock_response_init = MagicMock()\n mock_response_init.headers = {\n \"content-type\": \"application/json\",\n \"Mcp-Session-Id\": \"test-session-123\",\n }\n mock_response_init.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 1,\n \"result\": {},\n }\n mock_response_init.raise_for_status = MagicMock()\n\n # Mock tools/list\n mock_response_tools = MagicMock()\n mock_response_tools.headers = {\"content-type\": \"application/json\"}\n mock_response_tools.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 2,\n \"result\": {\"tools\": []},\n }\n mock_response_tools.raise_for_status = MagicMock()\n\n mock_client_instance.post = AsyncMock(\n side_effect=[\n mock_response_init,\n MagicMock(), # initialized notification\n mock_response_tools,\n ]\n )\n\n mcp = MCPClient(server_url=\"http://localhost:8000\")\n\n # Verify session ID was captured\n assert mcp._session_id == \"test-session-123\"\n\n # Verify subsequent requests include session ID\n calls = mock_client_instance.post.call_args_list\n # tools/list request (3rd call, index 2) should have session ID\n tools_call = calls[2]\n headers = tools_call[1][\"headers\"]\n assert \"Mcp-Session-Id\" in headers\n assert headers[\"Mcp-Session-Id\"] == \"test-session-123\"\n\n mcp.close()\n\n def test_sse_with_multiple_events(self):\n \"\"\"Test SSE stream with multiple events before final response.\"\"\"\n with patch(\"aisuite.mcp.client.httpx.AsyncClient\") as mock_async_client:\n mock_client_instance = AsyncMock()\n mock_async_client.return_value = mock_client_instance\n\n # Mock initialize and tools/list\n mock_response_init = MagicMock()\n mock_response_init.headers = {\"content-type\": \"application/json\"}\n mock_response_init.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 1,\n \"result\": {},\n }\n mock_response_init.raise_for_status = MagicMock()\n\n mock_response_tools = MagicMock()\n mock_response_tools.headers = {\"content-type\": \"application/json\"}\n mock_response_tools.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 2,\n \"result\": {\"tools\": []},\n }\n mock_response_tools.raise_for_status = MagicMock()\n\n # Mock SSE response with notifications before result\n mock_response_sse = MagicMock()\n mock_response_sse.headers = {\"content-type\": \"text/event-stream\"}\n mock_response_sse.raise_for_status = MagicMock()\n\n async def mock_aiter_lines():\n lines = [\n 'data: {\"jsonrpc\": \"2.0\", \"method\": \"notification\", \"params\": {\"status\": \"processing\"}}',\n \"\",\n 'data: {\"jsonrpc\": \"2.0\", \"method\": \"notification\", \"params\": {\"status\": \"almost done\"}}',\n \"\",\n 'data: {\"jsonrpc\": \"2.0\", \"id\": 3, \"result\": {\"content\": [{\"type\": \"text\", \"text\": \"final result\"}]}}',\n \"\",\n ]\n for line in lines:\n yield line\n\n mock_response_sse.aiter_lines = mock_aiter_lines\n\n mock_client_instance.post = AsyncMock(\n side_effect=[\n mock_response_init,\n MagicMock(), # initialized notification\n mock_response_tools,\n mock_response_sse,\n ]\n )\n\n mcp = MCPClient(server_url=\"http://localhost:8000\")\n result = mcp.call_tool(\"test_tool\", {})\n\n # Should return the final result, ignoring notifications\n assert result == \"final result\"\n\n mcp.close()\n\n def test_mixed_json_and_sse_responses(self):\n \"\"\"Test that client handles both JSON and SSE responses from same server.\"\"\"\n with patch(\"aisuite.mcp.client.httpx.AsyncClient\") as mock_async_client:\n mock_client_instance = AsyncMock()\n mock_async_client.return_value = mock_client_instance\n\n # Mock initialize (JSON)\n mock_response_init = MagicMock()\n mock_response_init.headers = {\"content-type\": \"application/json\"}\n mock_response_init.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 1,\n \"result\": {},\n }\n mock_response_init.raise_for_status = MagicMock()\n\n # Mock tools/list (JSON)\n mock_response_tools = MagicMock()\n mock_response_tools.headers = {\"content-type\": \"application/json\"}\n mock_response_tools.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 2,\n \"result\": {\n \"tools\": [\n {\"name\": \"fast_tool\", \"description\": \"Fast\", \"inputSchema\": {}},\n {\"name\": \"slow_tool\", \"description\": \"Slow\", \"inputSchema\": {}},\n ]\n },\n }\n mock_response_tools.raise_for_status = MagicMock()\n\n # Mock fast tool call (JSON response)\n mock_response_fast = MagicMock()\n mock_response_fast.headers = {\"content-type\": \"application/json\"}\n mock_response_fast.json.return_value = {\n \"jsonrpc\": \"2.0\",\n \"id\": 3,\n \"result\": {\"content\": [{\"type\": \"text\", \"text\": \"fast\"}]},\n }\n mock_response_fast.raise_for_status = MagicMock()\n\n # Mock slow tool call (SSE response)\n mock_response_slow = MagicMock()\n mock_response_slow.headers = {\"content-type\": \"text/event-stream\"}\n mock_response_slow.raise_for_status = MagicMock()\n\n async def mock_aiter_lines():\n lines = [\n 'data: {\"jsonrpc\": \"2.0\", \"id\": 4, \"result\": {\"content\": [{\"type\": \"text\", \"text\": \"slow\"}]}}',\n \"\",\n ]\n for line in lines:\n yield line\n\n mock_response_slow.aiter_lines = mock_aiter_lines\n\n mock_client_instance.post = AsyncMock(\n side_effect=[\n mock_response_init,\n MagicMock(), # initialized notification\n mock_response_tools,\n mock_response_fast,\n mock_response_slow,\n ]\n )\n\n mcp = MCPClient(server_url=\"http://localhost:8000\")\n\n # Call fast tool (JSON response)\n result1 = mcp.call_tool(\"fast_tool\", {})\n assert result1 == \"fast\"\n\n # Call slow tool (SSE response)\n result2 = mcp.call_tool(\"slow_tool\", {})\n assert result2 == \"slow\"\n\n mcp.close()\n" }, { "path": "tests/mcp/test_llm_e2e.py", "content": "\"\"\"\nReal LLM End-to-End Tests for MCP Integration.\n\nThese tests make ACTUAL API calls to LLM providers (OpenAI, Anthropic) to verify\nthat MCP tools work correctly with real models. Unlike test_e2e.py which mocks\nLLM responses, these tests verify the complete integration stack.\n\n⚠️ WARNING: These tests will make real API calls and incur costs!\n - Each test costs ~$0.01-0.05 depending on the model\n - Tests are marked with @pytest.mark.llm\n - Tests are skipped if API keys are not present\n\nRequirements:\n - Node.js and npx (for MCP filesystem server)\n - API keys in .env file:\n OPENAI_API_KEY=your-key\n ANTHROPIC_API_KEY=your-key\n - pytest-asyncio, python-dotenv\n\nRunning:\n # Run ONLY LLM tests (⚠️ costs money):\n pytest tests/mcp/test_llm_e2e.py -v -m llm\n\n # Skip LLM tests (default, free):\n pytest tests/mcp/ -v -m \"integration and not llm\"\n\"\"\"\n\nimport pytest\nimport os\nfrom pathlib import Path\nfrom aisuite import Client\n\n\n# Helper function to check if we have API keys\ndef has_openai_key():\n \"\"\"Check if OpenAI API key is available.\"\"\"\n return bool(os.getenv(\"OPENAI_API_KEY\"))\n\n\ndef has_anthropic_key():\n \"\"\"Check if Anthropic API key is available.\"\"\"\n return bool(os.getenv(\"ANTHROPIC_API_KEY\"))\n\n\n@pytest.mark.llm\n@pytest.mark.integration\nclass TestOpenAIWithMCP:\n \"\"\"Test OpenAI models with real MCP tools.\"\"\"\n\n @pytest.mark.skipif(not has_openai_key(), reason=\"OPENAI_API_KEY not set\")\n def test_gpt4o_reads_file_via_mcp(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test GPT-4o can read a file using MCP filesystem tools.\"\"\"\n client = Client()\n\n response = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": f'Use read_file to read the file at path \"{temp_test_dir}/test.txt\" and tell me what it contains.',\n }\n ],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"filesystem\",\n \"command\": \"npx\",\n \"args\": [\n \"-y\",\n \"@modelcontextprotocol/server-filesystem\",\n temp_test_dir,\n ],\n \"allowed_tools\": [\"read_file\"], # Security: only allow reading\n }\n ],\n max_turns=3,\n )\n\n # Debug: Print intermediate messages to see what happened\n if hasattr(response.choices[0], \"intermediate_messages\"):\n print(\"\\n=== Intermediate Messages ===\")\n import json\n\n for i, msg in enumerate(response.choices[0].intermediate_messages):\n print(f\"\\nMessage {i}:\")\n # Handle both dict and object formats\n if isinstance(msg, dict):\n print(json.dumps(msg, indent=2, default=str))\n else:\n print(f\"Role: {msg.role}\")\n if hasattr(msg, \"content\") and msg.content:\n print(f\"Content: {msg.content[:200]}\")\n if hasattr(msg, \"tool_calls\") and msg.tool_calls:\n for tc in msg.tool_calls:\n print(\n f\"Tool Call: {tc.function.name}({tc.function.arguments})\"\n )\n\n # Verify the LLM actually read the file\n content = response.choices[0].message.content.lower()\n assert (\n \"hello from mcp test\" in content or \"hello from mcp\" in content\n ), f\"Expected file content in response, got: {content}\"\n\n @pytest.mark.skipif(not has_openai_key(), reason=\"OPENAI_API_KEY not set\")\n def test_gpt4o_lists_files_via_mcp(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test GPT-4o can list directory contents using MCP tools.\"\"\"\n client = Client()\n\n response = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": f'Use list_directory to list all files in the directory at path \"{temp_test_dir}\" and tell me what you find.',\n }\n ],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"filesystem\",\n \"command\": \"npx\",\n \"args\": [\n \"-y\",\n \"@modelcontextprotocol/server-filesystem\",\n temp_test_dir,\n ],\n \"allowed_tools\": [\"list_directory\"], # Security: only allow listing\n }\n ],\n max_turns=3,\n )\n\n # Verify the LLM found the test files\n content = response.choices[0].message.content.lower()\n # Test dir has: test.txt, README.md, data.json, subdir/\n assert (\n \"test.txt\" in content or \"readme\" in content\n ), f\"Expected file names in response, got: {content}\"\n\n @pytest.mark.skipif(not has_openai_key(), reason=\"OPENAI_API_KEY not set\")\n def test_gpt4o_mixed_tools(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test GPT-4o with both MCP tools and regular Python functions.\"\"\"\n\n # Define a Python function\n def get_current_date() -> str:\n \"\"\"Get the current date in YYYY-MM-DD format.\"\"\"\n from datetime import datetime\n\n return datetime.now().strftime(\"%Y-%m-%d\")\n\n client = Client()\n\n response = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": f'First use get_current_date to get today\\'s date, then use read_file to read \"{temp_test_dir}/test.txt\" and tell me both.',\n }\n ],\n tools=[\n get_current_date, # Python function\n {\n \"type\": \"mcp\",\n \"name\": \"filesystem\",\n \"command\": \"npx\",\n \"args\": [\n \"-y\",\n \"@modelcontextprotocol/server-filesystem\",\n temp_test_dir,\n ],\n \"allowed_tools\": [\"read_file\"],\n },\n ],\n max_turns=5,\n )\n\n # Verify both tools were used\n content = response.choices[0].message.content.lower()\n # Should mention the date (from Python function)\n assert any(\n str(y) in content for y in [2024, 2025, 2026]\n ), f\"Expected date in response, got: {content}\"\n # Should mention the file content (from MCP tool)\n assert (\n \"hello\" in content or \"mcp test\" in content\n ), f\"Expected file content in response, got: {content}\"\n\n\n@pytest.mark.llm\n@pytest.mark.integration\nclass TestAnthropicWithMCP:\n \"\"\"Test Anthropic Claude models with real MCP tools.\"\"\"\n\n @pytest.mark.skipif(not has_anthropic_key(), reason=\"ANTHROPIC_API_KEY not set\")\n def test_claude_reads_file_via_mcp(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test Claude can read a file using MCP filesystem tools.\"\"\"\n client = Client()\n\n response = client.chat.completions.create(\n model=\"anthropic:claude-sonnet-4-5\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": f'Use read_file to read the file at path \"{temp_test_dir}/test.txt\" and tell me what it contains.',\n }\n ],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"filesystem\",\n \"command\": \"npx\",\n \"args\": [\n \"-y\",\n \"@modelcontextprotocol/server-filesystem\",\n temp_test_dir,\n ],\n \"allowed_tools\": [\"read_file\"],\n }\n ],\n max_turns=3,\n )\n\n # Verify Claude actually read the file\n content = response.choices[0].message.content.lower()\n assert (\n \"hello from mcp test\" in content or \"hello from mcp\" in content\n ), f\"Expected file content in response, got: {content}\"\n\n @pytest.mark.skipif(not has_anthropic_key(), reason=\"ANTHROPIC_API_KEY not set\")\n def test_claude_lists_files_via_mcp(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test Claude can list directory contents using MCP tools.\"\"\"\n client = Client()\n\n response = client.chat.completions.create(\n model=\"anthropic:claude-sonnet-4-5\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": f'Use list_directory with path \"{temp_test_dir}\" to list all files.',\n }\n ],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"filesystem\",\n \"command\": \"npx\",\n \"args\": [\n \"-y\",\n \"@modelcontextprotocol/server-filesystem\",\n temp_test_dir,\n ],\n \"allowed_tools\": [\"list_directory\"],\n }\n ],\n max_turns=3,\n )\n\n # Verify Claude found the test files\n content = response.choices[0].message.content.lower()\n assert (\n \"test.txt\" in content or \"readme\" in content or \"data.json\" in content\n ), f\"Expected file names in response, got: {content}\"\n\n @pytest.mark.skipif(not has_anthropic_key(), reason=\"ANTHROPIC_API_KEY not set\")\n def test_claude_mixed_tools(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test Claude with both MCP tools and regular Python functions.\"\"\"\n\n def get_weather(location: str) -> str:\n \"\"\"Get the weather for a location (mock function).\n\n Args:\n location: The city name\n \"\"\"\n # Mock weather function for testing\n return f\"The weather in {location} is sunny and 72°F\"\n\n client = Client()\n\n response = client.chat.completions.create(\n model=\"anthropic:claude-sonnet-4-5\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": f'Use get_weather for San Francisco, then use read_file to read \"{temp_test_dir}/README.md\". Tell me both results.',\n }\n ],\n tools=[\n get_weather, # Python function\n {\n \"type\": \"mcp\",\n \"name\": \"filesystem\",\n \"command\": \"npx\",\n \"args\": [\n \"-y\",\n \"@modelcontextprotocol/server-filesystem\",\n temp_test_dir,\n ],\n \"allowed_tools\": [\"read_file\"],\n },\n ],\n max_turns=5,\n )\n\n # Verify both tools were used\n content = response.choices[0].message.content.lower()\n # Should mention weather (from Python function)\n assert (\n \"weather\" in content or \"sunny\" in content or \"72\" in content\n ), f\"Expected weather info in response, got: {content}\"\n # Should mention the README content (from MCP tool)\n assert (\n \"test directory\" in content or \"readme\" in content\n ), f\"Expected README content in response, got: {content}\"\n\n\n@pytest.mark.llm\n@pytest.mark.integration\nclass TestToolPrefixingWithLLM:\n \"\"\"Test tool prefixing works with real LLMs.\"\"\"\n\n @pytest.mark.skipif(not has_openai_key(), reason=\"OPENAI_API_KEY not set\")\n def test_multiple_mcp_servers_with_prefixing(self, temp_test_dir, skip_if_no_npx):\n \"\"\"Test using multiple MCP servers with prefixing to avoid name collisions.\"\"\"\n # Create two subdirectories\n dir1 = Path(temp_test_dir) / \"dir1\"\n dir2 = Path(temp_test_dir) / \"dir2\"\n dir1.mkdir()\n dir2.mkdir()\n\n (dir1 / \"file1.txt\").write_text(\"Content from dir1\")\n (dir2 / \"file2.txt\").write_text(\"Content from dir2\")\n\n client = Client()\n\n response = client.chat.completions.create(\n model=\"openai:gpt-4o\",\n messages=[\n {\n \"role\": \"user\",\n \"content\": f'Use dir1_fs__list_directory with path \"{dir1}\" to list dir1, then use dir2_fs__list_directory with path \"{dir2}\" to list dir2.',\n }\n ],\n tools=[\n {\n \"type\": \"mcp\",\n \"name\": \"dir1_fs\",\n \"command\": \"npx\",\n \"args\": [\n \"-y\",\n \"@modelcontextprotocol/server-filesystem\",\n str(dir1),\n ],\n \"use_tool_prefix\": True, # Tools will be \"dir1_fs__list_directory\", etc.\n \"allowed_tools\": [\"list_directory\"],\n },\n {\n \"type\": \"mcp\",\n \"name\": \"dir2_fs\",\n \"command\": \"npx\",\n \"args\": [\n \"-y\",\n \"@modelcontextprotocol/server-filesystem\",\n str(dir2),\n ],\n \"use_tool_prefix\": True, # Tools will be \"dir2_fs__list_directory\", etc.\n \"allowed_tools\": [\"list_directory\"],\n },\n ],\n max_turns=5,\n )\n\n # Verify the LLM found files from both directories\n content = response.choices[0].message.content.lower()\n assert (\n \"file1\" in content or \"dir1\" in content\n ), f\"Expected dir1 content, got: {content}\"\n assert (\n \"file2\" in content or \"dir2\" in content\n ), f\"Expected dir2 content, got: {content}\"\n" }, { "path": "tests/providers/__init__.py", "content": "" }, { "path": "tests/providers/test_anthropic_converter.py", "content": "\"\"\"Tests for the AnthropicMessageConverter.\"\"\"\n\nimport unittest\nfrom unittest.mock import MagicMock\nfrom aisuite.providers.anthropic_provider import AnthropicMessageConverter\nfrom aisuite.framework import ChatCompletionResponse\n\n\nclass TestAnthropicMessageConverter(unittest.TestCase):\n \"\"\"Test suite for the AnthropicMessageConverter class.\"\"\"\n\n def setUp(self):\n \"\"\"Set up the test case.\"\"\"\n self.converter = AnthropicMessageConverter()\n\n def test_convert_request_single_user_message(self):\n \"\"\"Test converting a single user message.\"\"\"\n messages = [{\"role\": \"user\", \"content\": \"Hello, how are you?\"}]\n system_message, converted_messages = self.converter.convert_request(messages)\n\n self.assertEqual(system_message, [])\n self.assertEqual(\n converted_messages, [{\"role\": \"user\", \"content\": \"Hello, how are you?\"}]\n )\n\n def test_convert_request_with_system_message(self):\n \"\"\"Test converting a request with a system message.\"\"\"\n messages = [\n {\"role\": \"system\", \"content\": \"You are a helpful assistant.\"},\n {\"role\": \"user\", \"content\": \"What is the weather?\"},\n ]\n system_message, converted_messages = self.converter.convert_request(messages)\n\n self.assertEqual(system_message, \"You are a helpful assistant.\")\n self.assertEqual(\n converted_messages, [{\"role\": \"user\", \"content\": \"What is the weather?\"}]\n )\n\n def test_convert_request_with_tool_use_message(self):\n \"\"\"Test converting a request with a tool use message.\"\"\"\n messages = [\n {\"role\": \"tool\", \"tool_call_id\": \"tool123\", \"content\": \"Weather data here.\"}\n ]\n system_message, converted_messages = self.converter.convert_request(messages)\n\n self.assertEqual(system_message, [])\n self.assertEqual(\n converted_messages,\n [\n {\n \"role\": \"user\",\n \"content\": [\n {\n \"type\": \"tool_result\",\n \"tool_use_id\": \"tool123\",\n \"content\": \"Weather data here.\",\n }\n ],\n }\n ],\n )\n\n def test_convert_response_normal_message(self):\n \"\"\"Test converting a normal text response.\"\"\"\n response = MagicMock()\n response.stop_reason = \"end_turn\"\n response.usage.input_tokens = 10\n response.usage.output_tokens = 5\n content_mock = MagicMock()\n content_mock.type = \"text\"\n content_mock.text = \"The weather is sunny.\"\n response.content = [content_mock]\n\n normalized_response = self.converter.convert_response(response)\n\n self.assertIsInstance(normalized_response, ChatCompletionResponse)\n self.assertEqual(normalized_response.choices[0].finish_reason, \"stop\")\n self.assertEqual(normalized_response.usage.prompt_tokens, 10)\n self.assertEqual(normalized_response.usage.completion_tokens, 5)\n self.assertEqual(normalized_response.usage.total_tokens, 15)\n self.assertEqual(\n normalized_response.choices[0].message.content, \"The weather is sunny.\"\n )\n\n def test_convert_response_with_tool_use(self):\n \"\"\"Test converting a response containing a tool use request.\"\"\"\n response = MagicMock()\n response.id = \"msg_01Aq9w938a90dw8q\"\n response.model = \"claude-3-5-sonnet-20241022\"\n response.role = \"assistant\"\n response.stop_reason = \"tool_use\"\n response.usage.input_tokens = 20\n response.usage.output_tokens = 10\n tool_use_mock = MagicMock()\n tool_use_mock.type = \"tool_use\"\n tool_use_mock.id = \"tool123\"\n tool_use_mock.name = \"get_weather\"\n tool_use_mock.input = {\"location\": \"Paris\"}\n\n text_mock = MagicMock()\n text_mock.type = \"text\"\n text_mock.text = \"I need to call the get_weather function\"\n\n response.content = [tool_use_mock, text_mock]\n\n normalized_response = self.converter.convert_response(response)\n\n self.assertIsInstance(normalized_response, ChatCompletionResponse)\n self.assertEqual(normalized_response.choices[0].finish_reason, \"tool_calls\")\n self.assertEqual(normalized_response.usage.prompt_tokens, 20)\n self.assertEqual(normalized_response.usage.completion_tokens, 10)\n self.assertEqual(normalized_response.usage.total_tokens, 30)\n self.assertEqual(\n normalized_response.choices[0].message.content,\n \"I need to call the get_weather function\",\n )\n self.assertEqual(len(normalized_response.choices[0].message.tool_calls), 1)\n self.assertEqual(\n normalized_response.choices[0].message.tool_calls[0].id, \"tool123\"\n )\n self.assertEqual(\n normalized_response.choices[0].message.tool_calls[0].function.name,\n \"get_weather\",\n )\n\n def test_convert_tool_spec(self):\n \"\"\"Test converting OpenAI tool specifications to Anthropic format.\"\"\"\n openai_tools = [\n {\n \"type\": \"function\",\n \"function\": {\n \"name\": \"get_weather\",\n \"description\": \"Get the weather.\",\n \"parameters\": {\n \"type\": \"object\",\n \"properties\": {\n \"location\": {\"type\": \"string\", \"description\": \"City name.\"}\n },\n \"required\": [\"location\"],\n },\n },\n }\n ]\n\n anthropic_tools = self.converter.convert_tool_spec(openai_tools)\n\n self.assertEqual(len(anthropic_tools), 1)\n self.assertEqual(anthropic_tools[0][\"name\"], \"get_weather\")\n self.assertEqual(anthropic_tools[0][\"description\"], \"Get the weather.\")\n self.assertEqual(\n anthropic_tools[0][\"input_schema\"],\n {\n \"type\": \"object\",\n \"properties\": {\n \"location\": {\"type\": \"string\", \"description\": \"City name.\"}\n },\n \"required\": [\"location\"],\n },\n )\n\n def test_convert_request_with_tool_call_and_result(self):\n \"\"\"Test converting a request with a tool call and its result.\"\"\"\n messages = [\n {\n \"role\": \"assistant\",\n \"content\": \"Let me check the weather.\",\n \"tool_calls\": [\n {\n \"id\": \"tool123\",\n \"function\": {\n \"name\": \"get_weather\",\n \"arguments\": '{\"location\": \"Paris\"}',\n },\n \"type\": \"function\",\n }\n ],\n },\n {\n \"role\": \"tool\",\n \"tool_call_id\": \"tool123\",\n \"content\": \"The weather in Paris is sunny.\",\n },\n ]\n system_message, converted_messages = self.converter.convert_request(messages)\n\n self.assertEqual(system_message, [])\n self.assertEqual(len(converted_messages), 2)\n self.assertEqual(converted_messages[0][\"role\"], \"assistant\")\n self.assertEqual(converted_messages[1][\"role\"], \"user\")\n self.assertEqual(\n converted_messages[0][\"content\"],\n [\n {\"type\": \"text\", \"text\": \"Let me check the weather.\"},\n {\n \"type\": \"tool_use\",\n \"id\": \"tool123\",\n \"name\": \"get_weather\",\n \"input\": {\"location\": \"Paris\"},\n },\n ],\n )\n self.assertEqual(\n converted_messages[1][\"content\"],\n [\n {\n \"type\": \"tool_result\",\n \"tool_use_id\": \"tool123\",\n \"content\": \"The weather in Paris is sunny.\",\n }\n ],\n )\n\n\nif __name__ == \"__main__\":\n unittest.main()\n" }, { "path": "tests/providers/test_asr_parameter_passthrough.py", "content": "\"\"\"Component tests for ASR parameter pass-through to provider SDKs.\"\"\"\n\nimport io\nfrom unittest.mock import MagicMock, mock_open, patch\nimport pytest\n\nfrom aisuite.providers.openai_provider import OpenaiProvider\nfrom aisuite.providers.deepgram_provider import DeepgramProvider\nfrom aisuite.providers.google_provider import GoogleProvider\nfrom aisuite.framework.message import TranscriptionResult\n\n\n@pytest.fixture(autouse=True)\ndef set_env_vars(monkeypatch):\n \"\"\"Fixture to set environment variables for all tests.\"\"\"\n monkeypatch.setenv(\"OPENAI_API_KEY\", \"test-openai-key\")\n monkeypatch.setenv(\"DEEPGRAM_API_KEY\", \"test-deepgram-key\")\n monkeypatch.setenv(\"GOOGLE_APPLICATION_CREDENTIALS\", \"test-creds.json\")\n monkeypatch.setenv(\"GOOGLE_PROJECT_ID\", \"test-project\")\n monkeypatch.setenv(\"GOOGLE_REGION\", \"us-central1\")\n\n\nclass TestOpenAIParameterPassthrough:\n \"\"\"Test that parameters correctly reach OpenAI SDK.\"\"\"\n\n def test_language_param_passthrough(self):\n \"\"\"Test language parameter reaches OpenAI SDK.\"\"\"\n provider = OpenaiProvider()\n mock_response = MagicMock()\n mock_response.text = \"Test transcription\"\n mock_response.language = \"en\"\n mock_response.segments = None\n\n with patch(\"builtins.open\", mock_open(read_data=b\"audio\")), patch.object(\n provider.client.audio.transcriptions, \"create\", return_value=mock_response\n ) as mock_create:\n\n provider.audio.transcriptions.create(\n model=\"whisper-1\", file=\"test.mp3\", language=\"en\"\n )\n\n # Verify language was passed to SDK\n mock_create.assert_called_once()\n call_kwargs = mock_create.call_args.kwargs\n assert \"language\" in call_kwargs\n assert call_kwargs[\"language\"] == \"en\"\n\n def test_temperature_param_passthrough(self):\n \"\"\"Test temperature parameter reaches OpenAI SDK.\"\"\"\n provider = OpenaiProvider()\n mock_response = MagicMock()\n mock_response.text = \"Test\"\n mock_response.language = \"en\"\n mock_response.segments = None\n\n with patch(\"builtins.open\", mock_open(read_data=b\"audio\")), patch.object(\n provider.client.audio.transcriptions, \"create\", return_value=mock_response\n ) as mock_create:\n\n provider.audio.transcriptions.create(\n model=\"whisper-1\", file=\"test.mp3\", temperature=0.7\n )\n\n call_kwargs = mock_create.call_args.kwargs\n assert \"temperature\" in call_kwargs\n assert call_kwargs[\"temperature\"] == 0.7\n\n def test_response_format_param_passthrough(self):\n \"\"\"Test response_format parameter reaches OpenAI SDK.\"\"\"\n provider = OpenaiProvider()\n mock_response = MagicMock()\n mock_response.text = \"Test\"\n mock_response.language = \"en\"\n mock_response.segments = None\n\n with patch(\"builtins.open\", mock_open(read_data=b\"audio\")), patch.object(\n provider.client.audio.transcriptions, \"create\", return_value=mock_response\n ) as mock_create:\n\n provider.audio.transcriptions.create(\n model=\"whisper-1\", file=\"test.mp3\", response_format=\"verbose_json\"\n )\n\n call_kwargs = mock_create.call_args.kwargs\n assert \"response_format\" in call_kwargs\n assert call_kwargs[\"response_format\"] == \"verbose_json\"\n\n def test_multiple_params_passthrough(self):\n \"\"\"Test multiple parameters reach OpenAI SDK together.\"\"\"\n provider = OpenaiProvider()\n mock_response = MagicMock()\n mock_response.text = \"Test\"\n mock_response.language = \"en\"\n mock_response.segments = None\n\n with patch(\"builtins.open\", mock_open(read_data=b\"audio\")), patch.object(\n provider.client.audio.transcriptions, \"create\", return_value=mock_response\n ) as mock_create:\n\n provider.audio.transcriptions.create(\n model=\"whisper-1\",\n file=\"test.mp3\",\n language=\"en\",\n temperature=0.5,\n response_format=\"json\",\n )\n\n call_kwargs = mock_create.call_args.kwargs\n assert call_kwargs[\"language\"] == \"en\"\n assert call_kwargs[\"temperature\"] == 0.5\n assert call_kwargs[\"response_format\"] == \"json\"\n\n def test_file_object_with_params(self):\n \"\"\"Test that file-like object works with parameters.\"\"\"\n provider = OpenaiProvider()\n mock_response = MagicMock()\n mock_response.text = \"Test\"\n mock_response.language = \"en\"\n mock_response.segments = None\n\n audio_data = io.BytesIO(b\"fake audio data\")\n\n with patch.object(\n provider.client.audio.transcriptions, \"create\", return_value=mock_response\n ) as mock_create:\n\n provider.audio.transcriptions.create(\n model=\"whisper-1\", file=audio_data, language=\"en\"\n )\n\n call_kwargs = mock_create.call_args.kwargs\n assert call_kwargs[\"file\"] == audio_data\n assert call_kwargs[\"language\"] == \"en\"\n\n\nclass TestGoogleParameterPassthrough:\n \"\"\"Test that parameters correctly reach Google Speech SDK.\"\"\"\n\n @patch(\"aisuite.providers.google_provider.vertexai.init\")\n def test_language_code_param_passthrough(self, mock_vertexai_init):\n \"\"\"Test language_code parameter reaches Google SDK.\"\"\"\n provider = GoogleProvider()\n mock_response = MagicMock()\n mock_result = MagicMock()\n mock_alternative = MagicMock()\n mock_alternative.transcript = \"Test\"\n mock_alternative.confidence = 0.95\n mock_alternative.words = []\n mock_result.alternatives = [mock_alternative]\n mock_response.results = [mock_result]\n\n provider._speech_client = MagicMock()\n provider._speech_client.recognize.return_value = mock_response\n\n with patch(\"builtins.open\", mock_open(read_data=b\"audio\")):\n provider.audio.transcriptions.create(\n model=\"latest_long\", file=\"test.wav\", language_code=\"en-US\"\n )\n\n # Verify language_code was in the config passed to SDK\n provider._speech_client.recognize.assert_called_once()\n call_kwargs = provider._speech_client.recognize.call_args.kwargs\n assert \"config\" in call_kwargs\n config = call_kwargs[\"config\"]\n assert config.language_code == \"en-US\"\n\n @patch(\"aisuite.providers.google_provider.vertexai.init\")\n def test_enable_automatic_punctuation_passthrough(self, mock_vertexai_init):\n \"\"\"Test enable_automatic_punctuation parameter reaches Google SDK.\"\"\"\n provider = GoogleProvider()\n mock_response = MagicMock()\n mock_result = MagicMock()\n mock_alternative = MagicMock()\n mock_alternative.transcript = \"Test.\"\n mock_alternative.confidence = 0.95\n mock_alternative.words = []\n mock_result.alternatives = [mock_alternative]\n mock_response.results = [mock_result]\n\n provider._speech_client = MagicMock()\n provider._speech_client.recognize.return_value = mock_response\n\n with patch(\"builtins.open\", mock_open(read_data=b\"audio\")):\n provider.audio.transcriptions.create(\n model=\"latest_long\",\n file=\"test.wav\",\n language_code=\"en-US\",\n enable_automatic_punctuation=True,\n )\n\n call_kwargs = provider._speech_client.recognize.call_args.kwargs\n config = call_kwargs[\"config\"]\n assert config.enable_automatic_punctuation is True\n\n @patch(\"aisuite.providers.google_provider.vertexai.init\")\n def test_speech_contexts_passthrough(self, mock_vertexai_init):\n \"\"\"Test speech_contexts parameter (from prompt mapping) reaches Google SDK.\"\"\"\n provider = GoogleProvider()\n mock_response = MagicMock()\n mock_result = MagicMock()\n mock_alternative = MagicMock()\n mock_alternative.transcript = \"Technical terms\"\n mock_alternative.confidence = 0.95\n mock_alternative.words = []\n mock_result.alternatives = [mock_alternative]\n mock_response.results = [mock_result]\n\n provider._speech_client = MagicMock()\n provider._speech_client.recognize.return_value = mock_response\n\n with patch(\"builtins.open\", mock_open(read_data=b\"audio\")):\n # Note: This would come in as speech_contexts after validation layer transforms prompt\n provider.audio.transcriptions.create(\n model=\"latest_long\",\n file=\"test.wav\",\n language_code=\"en-US\",\n speech_contexts=[{\"phrases\": [\"technical terms\"]}],\n )\n\n call_kwargs = provider._speech_client.recognize.call_args.kwargs\n config = call_kwargs[\"config\"]\n assert len(config.speech_contexts) == 1\n assert config.speech_contexts[0].phrases == [\"technical terms\"]\n" }, { "path": "tests/providers/test_aws_converter.py", "content": "import unittest\nfrom unittest.mock import MagicMock\nfrom aisuite.providers.aws_provider import BedrockMessageConverter\nfrom aisuite.framework.message import Message, ChatCompletionMessageToolCall\nfrom aisuite.framework import ChatCompletionResponse\n\n\nclass TestBedrockMessageConverter(unittest.TestCase):\n\n def setUp(self):\n self.converter = BedrockMessageConverter()\n\n def test_convert_request_user_message(self):\n messages = [\n {\"role\": \"user\", \"content\": \"What is the most popular song on WZPZ?\"}\n ]\n system_message, formatted_messages = self.converter.convert_request(messages)\n\n self.assertEqual(system_message, [])\n self.assertEqual(len(formatted_messages), 1)\n self.assertEqual(formatted_messages[0][\"role\"], \"user\")\n self.assertEqual(\n formatted_messages[0][\"content\"],\n [{\"text\": \"What is the most popular song on WZPZ?\"}],\n )\n\n def test_convert_request_tool_result(self):\n messages = [\n {\n \"role\": \"tool\",\n \"tool_call_id\": \"tool123\",\n \"content\": '{\"song\": \"Elemental Hotel\", \"artist\": \"8 Storey Hike\"}',\n }\n ]\n system_message, formatted_messages = self.converter.convert_request(messages)\n\n self.assertEqual(system_message, [])\n self.assertEqual(len(formatted_messages), 1)\n self.assertEqual(formatted_messages[0][\"role\"], \"user\")\n self.assertEqual(\n formatted_messages[0][\"content\"],\n [\n {\n \"toolResult\": {\n \"toolUseId\": \"tool123\",\n \"content\": [\n {\n \"json\": {\n \"song\": \"Elemental Hotel\",\n \"artist\": \"8 Storey Hike\",\n }\n }\n ],\n }\n }\n ],\n )\n\n def test_convert_response_tool_call(self):\n response = {\n \"output\": {\n \"message\": {\n \"role\": \"assistant\",\n \"content\": [\n {\n \"toolUse\": {\n \"toolUseId\": \"tool123\",\n \"name\": \"top_song\",\n \"input\": {\"sign\": \"WZPZ\"},\n }\n }\n ],\n }\n },\n \"stopReason\": \"tool_use\",\n }\n\n normalized_response = self.converter.convert_response(response)\n\n self.assertIsInstance(normalized_response, ChatCompletionResponse)\n self.assertEqual(normalized_response.choices[0].finish_reason, \"tool_calls\")\n tool_call = normalized_response.choices[0].message.tool_calls[0]\n self.assertEqual(tool_call.function.name, \"top_song\")\n self.assertEqual(tool_call.function.arguments, '{\"sign\": \"WZPZ\"}')\n\n def test_convert_response_text(self):\n response = {\n \"output\": {\n \"message\": {\n \"role\": \"assistant\",\n \"content\": [\n {\n \"text\": \"The most popular song on WZPZ is Elemental Hotel by 8 Storey Hike.\"\n }\n ],\n }\n },\n \"stopReason\": \"complete\",\n }\n\n normalized_response = self.converter.convert_response(response)\n\n self.assertIsInstance(normalized_response, ChatCompletionResponse)\n self.assertEqual(normalized_response.choices[0].finish_reason, \"stop\")\n self.assertEqual(\n normalized_response.choices[0].message.content,\n \"The most popular song on WZPZ is Elemental Hotel by 8 Storey Hike.\",\n )\n\n\nif __name__ == \"__main__\":\n unittest.main()\n" }, { "path": "tests/providers/test_azure_provider.py", "content": "import unittest\nfrom aisuite.providers.azure_provider import AzureMessageConverter\nfrom aisuite.framework.message import Message, ChatCompletionMessageToolCall\nfrom aisuite.framework import ChatCompletionResponse\n\n\nclass TestAzureMessageConverter(unittest.TestCase):\n def setUp(self):\n self.converter = AzureMessageConverter()\n\n def test_convert_request_dict_message(self):\n messages = [{\"role\": \"user\", \"content\": \"Hello, how are you?\"}]\n converted_messages = self.converter.convert_request(messages)\n\n self.assertEqual(\n converted_messages, [{\"role\": \"user\", \"content\": \"Hello, how are you?\"}]\n )\n\n def test_convert_request_message_object(self):\n message = Message(role=\"user\", content=\"Hello\", tool_calls=None, refusal=None)\n messages = [message]\n converted_messages = self.converter.convert_request(messages)\n\n expected_message = {\n \"role\": \"user\",\n \"content\": \"Hello\",\n \"reasoning_content\": None,\n \"tool_calls\": None,\n \"refusal\": None,\n }\n self.assertEqual(converted_messages, [expected_message])\n\n def test_convert_response_basic(self):\n azure_response = {\n \"choices\": [\n {\n \"message\": {\n \"role\": \"assistant\",\n \"content\": \"Hello! How can I help you?\",\n }\n }\n ]\n }\n\n response = self.converter.convert_response(azure_response)\n\n self.assertIsInstance(response, ChatCompletionResponse)\n self.assertEqual(\n response.choices[0].message.content, \"Hello! How can I help you?\"\n )\n self.assertEqual(response.choices[0].message.role, \"assistant\")\n self.assertIsNone(response.choices[0].message.tool_calls)\n\n def test_convert_response_with_tool_calls(self):\n azure_response = {\n \"choices\": [\n {\n \"message\": {\n \"role\": \"assistant\",\n \"content\": \"Let me check the weather.\",\n \"tool_calls\": [\n {\n \"id\": \"tool123\",\n \"type\": \"function\",\n \"function\": {\n \"name\": \"get_weather\",\n \"arguments\": '{\"location\": \"London\"}',\n },\n }\n ],\n }\n }\n ]\n }\n\n response = self.converter.convert_response(azure_response)\n\n self.assertIsInstance(response, ChatCompletionResponse)\n self.assertEqual(\n response.choices[0].message.content, \"Let me check the weather.\"\n )\n self.assertEqual(len(response.choices[0].message.tool_calls), 1)\n\n tool_call = response.choices[0].message.tool_calls[0]\n self.assertEqual(tool_call.id, \"tool123\")\n self.assertEqual(tool_call.type, \"function\")\n self.assertEqual(tool_call.function.name, \"get_weather\")\n self.assertEqual(tool_call.function.arguments, '{\"location\": \"London\"}')\n\n\nif __name__ == \"__main__\":\n unittest.main()\n" }, { "path": "tests/providers/test_cerebras_provider.py", "content": "\"\"\"Tests for the Cerebras provider.\"\"\"\n\nfrom unittest.mock import MagicMock, patch\n\nimport pytest\n\nfrom aisuite.providers.cerebras_provider import CerebrasProvider\n\n\n@pytest.fixture(autouse=True)\ndef set_api_key_env_var(monkeypatch):\n \"\"\"Fixture to set environment variables for tests.\"\"\"\n monkeypatch.setenv(\"CEREBRAS_API_KEY\", \"test-api-key\")\n\n\ndef test_cerebras_provider():\n \"\"\"Test that the provider is initialized and chat completions are requested.\"\"\"\n\n user_greeting = \"Hello!\"\n message_history = [{\"role\": \"user\", \"content\": user_greeting}]\n selected_model = \"our-favorite-model\"\n chosen_temperature = 0.75\n response_text_content = \"mocked-text-response-from-model\"\n\n provider = CerebrasProvider()\n mock_response = MagicMock()\n mock_response.model_dump.return_value = {\n \"choices\": [{\"message\": {\"content\": response_text_content}}]\n }\n\n with patch.object(\n provider.client.chat.completions,\n \"create\",\n return_value=mock_response,\n ) as mock_create:\n response = provider.chat_completions_create(\n messages=message_history,\n model=selected_model,\n temperature=chosen_temperature,\n )\n\n mock_create.assert_called_with(\n messages=message_history,\n model=selected_model,\n temperature=chosen_temperature,\n )\n\n assert response.choices[0].message.content == response_text_content\n\n\ndef test_cerebras_provider_with_usage():\n \"\"\"Tests that usage data is correctly parsed when present in the response.\"\"\"\n\n user_greeting = \"Hello!\"\n message_history = [{\"role\": \"user\", \"content\": user_greeting}]\n selected_model = \"our-favorite-model\"\n chosen_temperature = 0.75\n response_text_content = \"mocked-text-response-from-model\"\n\n provider = CerebrasProvider()\n mock_response = MagicMock()\n mock_response.model_dump.return_value = {\n \"choices\": [{\"message\": {\"content\": response_text_content}}],\n \"usage\": {\n \"prompt_tokens\": 10,\n \"completion_tokens\": 20,\n \"total_tokens\": 30,\n },\n }\n\n with patch.object(\n provider.client.chat.completions,\n \"create\",\n return_value=mock_response,\n ):\n response = provider.chat_completions_create(\n messages=message_history,\n model=selected_model,\n temperature=chosen_temperature,\n )\n\n assert response.usage is not None\n assert response.usage.prompt_tokens == 10\n assert response.usage.completion_tokens == 20\n assert response.usage.total_tokens == 30\n" }, { "path": "tests/providers/test_cohere_provider.py", "content": "from unittest.mock import MagicMock, patch\n\nimport pytest\n\nfrom aisuite.providers.cohere_provider import CohereProvider\n\n\n@pytest.fixture(autouse=True)\ndef set_api_key_env_var(monkeypatch):\n \"\"\"Fixture to set environment variables for tests.\"\"\"\n monkeypatch.setenv(\"CO_API_KEY\", \"test-api-key\")\n\n\ndef test_cohere_provider():\n \"\"\"High-level test that the provider is initialized and chat completions are requested successfully.\"\"\"\n\n user_greeting = \"Hello!\"\n message_history = [{\"role\": \"user\", \"content\": user_greeting}]\n selected_model = \"our-favorite-model\"\n chosen_temperature = 0.75\n response_text_content = \"mocked-text-response-from-model\"\n\n provider = CohereProvider()\n mock_response = MagicMock()\n mock_response.message = MagicMock()\n mock_response.message.content = [MagicMock()]\n mock_response.message.content[0].text = response_text_content\n\n with patch.object(\n provider.client,\n \"chat\",\n return_value=mock_response,\n ) as mock_create:\n response = provider.chat_completions_create(\n messages=message_history,\n model=selected_model,\n temperature=chosen_temperature,\n )\n\n mock_create.assert_called_with(\n messages=message_history,\n model=selected_model,\n temperature=chosen_temperature,\n )\n\n assert response.choices[0].message.content == response_text_content\n" }, { "path": "tests/providers/test_deepgram_provider.py", "content": "\"\"\"Tests for Deepgram provider functionality.\"\"\"\n\nimport io\nfrom unittest.mock import MagicMock, mock_open, patch\n\nimport pytest\n\nfrom aisuite.providers.deepgram_provider import DeepgramProvider\nfrom aisuite.provider import ASRError\nfrom aisuite.framework.message import (\n TranscriptionResult,\n TranscriptionOptions,\n StreamingTranscriptionChunk,\n)\n\n\n@pytest.fixture(autouse=True)\ndef set_api_key_env_var(monkeypatch):\n \"\"\"Fixture to set environment variables for tests.\"\"\"\n monkeypatch.setenv(\"DEEPGRAM_API_KEY\", \"test-api-key\")\n\n\n@pytest.fixture\ndef deepgram_provider():\n \"\"\"Create a Deepgram provider instance for testing.\"\"\"\n return DeepgramProvider()\n\n\n@pytest.fixture\ndef mock_deepgram_response():\n \"\"\"Create a mock Deepgram API response for ASR.\"\"\"\n return {\n \"metadata\": {\n \"request_id\": \"test-request-id\",\n \"duration\": 10.5,\n \"channels\": 1,\n },\n \"results\": {\n \"channels\": [\n {\n \"alternatives\": [\n {\n \"transcript\": \"Hello, this is a test transcription from Deepgram.\",\n \"confidence\": 0.95,\n \"words\": [\n {\n \"word\": \"hello\",\n \"start\": 0.0,\n \"end\": 0.5,\n \"confidence\": 0.98,\n \"speaker\": 0,\n }\n ],\n }\n ]\n }\n ],\n \"language\": \"en-US\",\n },\n }\n\n\nclass TestDeepgramProvider:\n \"\"\"Test suite for Deepgram provider functionality.\"\"\"\n\n def test_provider_initialization(self, deepgram_provider):\n \"\"\"Test that Deepgram provider initializes correctly.\"\"\"\n assert deepgram_provider is not None\n assert hasattr(deepgram_provider, \"api_key\")\n assert deepgram_provider.api_key == \"test-api-key\"\n assert hasattr(deepgram_provider, \"audio\")\n assert hasattr(deepgram_provider.audio, \"transcriptions\")\n\n def test_chat_completions_create_not_implemented(self, deepgram_provider):\n \"\"\"Test that chat completions are not supported.\"\"\"\n with pytest.raises(\n NotImplementedError,\n match=\"Deepgram provider only supports audio transcription\",\n ):\n deepgram_provider.chat_completions_create(\"model\", [])\n\n def test_audio_transcriptions_create_success(\n self, deepgram_provider, mock_deepgram_response\n ):\n \"\"\"Test successful audio transcription.\"\"\"\n mock_sdk_response = MagicMock()\n mock_sdk_response.to_dict.return_value = mock_deepgram_response\n mock_sdk_response.model_dump.return_value = mock_deepgram_response\n\n with patch(\n \"builtins.open\", mock_open(read_data=b\"fake audio data\")\n ), patch.object(\n deepgram_provider.client.listen.v1.media,\n \"transcribe_file\",\n return_value=mock_sdk_response,\n ):\n result = deepgram_provider.audio.transcriptions.create(\n model=\"deepgram:nova-2\", file=\"test_audio.mp3\"\n )\n\n assert isinstance(result, TranscriptionResult)\n assert result.text == \"Hello, this is a test transcription from Deepgram.\"\n assert result.language == \"en-US\"\n assert result.confidence == 0.95\n\n def test_audio_transcriptions_create_with_file_object(\n self, deepgram_provider, mock_deepgram_response\n ):\n \"\"\"Test audio transcription with file-like object.\"\"\"\n audio_data = io.BytesIO(b\"fake audio data\")\n\n mock_response = MagicMock()\n mock_response.to_dict.return_value = mock_deepgram_response\n mock_response.model_dump.return_value = mock_deepgram_response\n\n with patch.object(\n deepgram_provider.client.listen.v1.media,\n \"transcribe_file\",\n return_value=mock_response,\n ):\n result = deepgram_provider.audio.transcriptions.create(\n model=\"deepgram:nova-2\", file=audio_data\n )\n\n assert isinstance(result, TranscriptionResult)\n assert result.text == \"Hello, this is a test transcription from Deepgram.\"\n\n def test_audio_transcriptions_create_with_options(\n self, deepgram_provider, mock_deepgram_response\n ):\n \"\"\"Test audio transcription with TranscriptionOptions.\"\"\"\n options = TranscriptionOptions(\n language=\"en\",\n enable_speaker_diarization=True,\n enable_automatic_punctuation=True,\n )\n\n mock_response = MagicMock()\n mock_response.to_dict.return_value = mock_deepgram_response\n mock_response.model_dump.return_value = mock_deepgram_response\n\n with patch(\n \"builtins.open\", mock_open(read_data=b\"fake audio data\")\n ), patch.object(\n deepgram_provider.client.listen.v1.media,\n \"transcribe_file\",\n return_value=mock_response,\n ) as mock_transcribe:\n result = deepgram_provider.audio.transcriptions.create(\n model=\"deepgram:nova-2\", file=\"test_audio.mp3\", options=options\n )\n\n mock_transcribe.assert_called_once()\n assert isinstance(result, TranscriptionResult)\n assert result.text == \"Hello, this is a test transcription from Deepgram.\"\n\n def test_audio_transcriptions_create_error_handling(self, deepgram_provider):\n \"\"\"Test error handling for API failures.\"\"\"\n with patch(\n \"builtins.open\", mock_open(read_data=b\"fake audio data\")\n ), patch.object(\n deepgram_provider.client.listen.v1.media,\n \"transcribe_file\",\n side_effect=Exception(\"API Error\"),\n ):\n with pytest.raises(\n ASRError, match=\"Deepgram transcription error: API Error\"\n ):\n deepgram_provider.audio.transcriptions.create(\n model=\"deepgram:nova-2\", file=\"test_audio.mp3\"\n )\n\n @pytest.mark.asyncio\n async def test_audio_transcriptions_create_stream_output(self, deepgram_provider):\n \"\"\"Test streaming audio transcription with single connection and chunking.\"\"\"\n import numpy as np\n\n # Mock audio file data (simulate 16kHz mono audio)\n audio_samples = 48000 # 3 seconds of 16kHz audio\n mock_audio_data = np.zeros(audio_samples, dtype=np.float32)\n\n # Create async context manager mock for v5 API\n mock_connection = MagicMock()\n mock_connection.send = MagicMock()\n mock_connection.on = MagicMock()\n\n async def mock_connect(*args, **kwargs):\n # Return an async context manager\n class MockAsyncContextManager:\n async def __aenter__(self):\n return mock_connection\n\n async def __aexit__(self, *args):\n pass\n\n return MockAsyncContextManager()\n\n with patch(\n \"soundfile.read\", return_value=(mock_audio_data, 16000)\n ), patch.object(\n deepgram_provider.client.listen.v1,\n \"connect\",\n side_effect=mock_connect,\n ):\n result = deepgram_provider.audio.transcriptions.create_stream_output(\n model=\"deepgram:nova-2\",\n file=\"test_audio.mp3\",\n chunk_size_minutes=1.0, # Test with smaller chunks\n )\n\n # Test that it returns an async generator\n assert hasattr(result, \"__aiter__\")\n\n def test_parse_deepgram_response_complete(\n self, deepgram_provider, mock_deepgram_response\n ):\n \"\"\"Test parsing complete Deepgram response.\"\"\"\n result = deepgram_provider.audio.transcriptions._parse_deepgram_response(\n mock_deepgram_response\n )\n\n assert result.text == \"Hello, this is a test transcription from Deepgram.\"\n assert result.language == \"en-US\"\n assert result.confidence == 0.95\n\n assert len(result.words) == 1\n word = result.words[0]\n assert word.word == \"hello\"\n assert word.start == 0.0\n assert word.end == 0.5\n assert word.confidence == 0.98\n\n def test_parse_deepgram_response_empty_channels(self, deepgram_provider):\n \"\"\"Test parsing response with empty channels.\"\"\"\n empty_response = {\"results\": {\"channels\": []}}\n\n result = deepgram_provider.audio.transcriptions._parse_deepgram_response(\n empty_response\n )\n assert result.text == \"\"\n assert result.language is None\n" }, { "path": "tests/providers/test_deepseek_provider.py", "content": "\"\"\"Tests for the Deepseek provider.\"\"\"\n\nfrom unittest.mock import MagicMock, patch\nimport pytest\n\nfrom aisuite.providers.deepseek_provider import DeepseekProvider\nfrom aisuite.framework.chat_completion_response import ChatCompletionResponse\n\n\n@pytest.fixture(autouse=True)\ndef set_api_key_env_var(monkeypatch):\n \"\"\"Fixture to set the Deepseek API key environment variable for tests.\"\"\"\n monkeypatch.setenv(\"DEEPSEEK_API_KEY\", \"test-api-key\")\n\n\ndef test_deepseek_provider():\n \"\"\"Test that the provider is initialized and chat completions are requested.\"\"\"\n\n user_greeting = \"Hello!\"\n message_history = [{\"role\": \"user\", \"content\": user_greeting}]\n selected_model = \"deepseek-chat\"\n chosen_temperature = 0.75\n response_text_content = \"mocked-text-response-from-model\"\n\n provider = DeepseekProvider()\n mock_response = MagicMock()\n # The mock response from the client is an object, so we mock the .model_dump() method\n mock_response.model_dump.return_value = {\n \"choices\": [\n {\"message\": {\"content\": response_text_content, \"role\": \"assistant\"}}\n ],\n \"model\": selected_model,\n \"created\": 12345,\n \"id\": \"chatcmpl-mockid\",\n # No usage data in this test\n }\n\n with patch.object(\n provider.client.chat.completions, \"create\", return_value=mock_response\n ) as mock_create:\n response = provider.chat_completions_create(\n messages=message_history,\n model=selected_model,\n temperature=chosen_temperature,\n )\n\n mock_create.assert_called_once_with(\n messages=message_history,\n model=selected_model,\n temperature=chosen_temperature,\n )\n\n assert isinstance(response, ChatCompletionResponse)\n assert response.choices[0].message.content == response_text_content\n assert response.usage is None\n\n\ndef test_deepseek_provider_with_usage():\n \"\"\"Tests that usage data is correctly parsed when present in the response.\"\"\"\n\n user_greeting = \"Hello!\"\n message_history = [{\"role\": \"user\", \"content\": user_greeting}]\n selected_model = \"deepseek-chat\"\n chosen_temperature = 0.75\n response_text_content = \"mocked-text-response-from-model\"\n\n provider = DeepseekProvider()\n mock_response = MagicMock()\n mock_response.model_dump.return_value = {\n \"choices\": [\n {\"message\": {\"content\": response_text_content, \"role\": \"assistant\"}}\n ],\n \"model\": selected_model,\n \"created\": 12345,\n \"id\": \"chatcmpl-mockid\",\n \"usage\": {\n \"prompt_tokens\": 10,\n \"completion_tokens\": 20,\n \"total_tokens\": 30,\n },\n }\n\n with patch.object(\n provider.client.chat.completions, \"create\", return_value=mock_response\n ) as mock_create:\n response = provider.chat_completions_create(\n messages=message_history,\n model=selected_model,\n temperature=chosen_temperature,\n )\n\n mock_create.assert_called_once_with(\n messages=message_history,\n model=selected_model,\n temperature=chosen_temperature,\n )\n\n assert isinstance(response, ChatCompletionResponse)\n assert response.choices[0].message.content == response_text_content\n assert response.usage is not None\n assert response.usage.prompt_tokens == 10\n assert response.usage.completion_tokens == 20\n assert response.usage.total_tokens == 30\n" }, { "path": "tests/providers/test_google_converter.py", "content": "import unittest\nfrom unittest.mock import MagicMock\nfrom aisuite.providers.google_provider import GoogleMessageConverter\nfrom aisuite.framework.message import Message, ChatCompletionMessageToolCall, Function\nfrom aisuite.framework import ChatCompletionResponse\n\n\nclass TestGoogleMessageConverter(unittest.TestCase):\n\n def setUp(self):\n self.converter = GoogleMessageConverter()\n\n def test_convert_request_user_message(self):\n messages = [{\"role\": \"user\", \"content\": \"What is the weather today?\"}]\n converted_messages = self.converter.convert_request(messages)\n\n self.assertEqual(len(converted_messages), 1)\n self.assertEqual(converted_messages[0].role, \"user\")\n self.assertEqual(\n converted_messages[0].parts[0].text, \"What is the weather today?\"\n )\n\n def test_convert_request_tool_result_message(self):\n messages = [\n {\n \"role\": \"tool\",\n \"name\": \"get_weather\",\n \"content\": '{\"temperature\": \"15\", \"unit\": \"Celsius\"}',\n }\n ]\n converted_messages = self.converter.convert_request(messages)\n\n self.assertEqual(len(converted_messages), 1)\n self.assertEqual(converted_messages[0].function_response.name, \"get_weather\")\n self.assertEqual(\n converted_messages[0].function_response.response,\n {\"temperature\": \"15\", \"unit\": \"Celsius\"},\n )\n\n def test_convert_request_assistant_message(self):\n messages = [\n {\n \"role\": \"assistant\",\n \"content\": \"The weather is sunny with a temperature of 25 degrees Celsius.\",\n }\n ]\n converted_messages = self.converter.convert_request(messages)\n\n self.assertEqual(len(converted_messages), 1)\n self.assertEqual(converted_messages[0].role, \"model\")\n self.assertEqual(\n converted_messages[0].parts[0].text,\n \"The weather is sunny with a temperature of 25 degrees Celsius.\",\n )\n\n def test_convert_response_with_function_call(self):\n function_call_mock = MagicMock()\n function_call_mock.name = \"get_exchange_rate\"\n function_call_mock.args = {\n \"currency_from\": \"AUD\",\n \"currency_to\": \"SEK\",\n \"currency_date\": \"latest\",\n }\n\n response = MagicMock()\n response.candidates = [\n MagicMock(\n content=MagicMock(parts=[MagicMock(function_call=function_call_mock)]),\n finish_reason=\"function_call\",\n )\n ]\n\n normalized_response = self.converter.convert_response(response)\n\n self.assertIsInstance(normalized_response, ChatCompletionResponse)\n self.assertEqual(normalized_response.choices[0].finish_reason, \"tool_calls\")\n self.assertEqual(\n normalized_response.choices[0].message.tool_calls[0].function.name,\n \"get_exchange_rate\",\n )\n self.assertEqual(\n normalized_response.choices[0].message.tool_calls[0].function.arguments,\n '{\"currency_from\": \"AUD\", \"currency_to\": \"SEK\", \"currency_date\": \"latest\"}',\n )\n\n def test_convert_response_with_text(self):\n response = MagicMock()\n text_content = \"The current exchange rate is 7.50 SEK per AUD.\"\n\n mock_part = MagicMock()\n mock_part.text = text_content\n mock_part.function_call = None\n\n mock_content = MagicMock()\n mock_content.parts = [mock_part]\n\n mock_candidate = MagicMock()\n mock_candidate.content = mock_content\n mock_candidate.finish_reason = \"stop\"\n\n response.candidates = [mock_candidate]\n\n normalized_response = self.converter.convert_response(response)\n\n self.assertIsInstance(normalized_response, ChatCompletionResponse)\n self.assertEqual(normalized_response.choices[0].finish_reason, \"stop\")\n self.assertEqual(normalized_response.choices[0].message.content, text_content)\n\n\nif __name__ == \"__main__\":\n unittest.main()\n" }, { "path": "tests/providers/test_google_provider.py", "content": "\"\"\"Tests for Google provider functionality (both chat and ASR).\"\"\"\n\nimport io\nimport json\nfrom unittest.mock import MagicMock, mock_open, patch\n\nimport pytest\n\nfrom aisuite.providers.google_provider import GoogleProvider\nfrom aisuite.provider import ASRError\nfrom aisuite.framework.message import (\n TranscriptionResult,\n TranscriptionOptions,\n StreamingTranscriptionChunk,\n)\nfrom vertexai.generative_models import Content, Part\n\n\n@pytest.fixture(autouse=True)\ndef set_api_key_env_var(monkeypatch):\n \"\"\"Fixture to set environment variables for tests.\"\"\"\n monkeypatch.setenv(\"GOOGLE_APPLICATION_CREDENTIALS\", \"path-to-service-account-json\")\n monkeypatch.setenv(\"GOOGLE_PROJECT_ID\", \"vertex-project-id\")\n monkeypatch.setenv(\"GOOGLE_REGION\", \"us-central1\")\n\n\n@pytest.fixture\ndef mock_google_speech_response():\n \"\"\"Create a mock Google Speech-to-Text API response.\"\"\"\n mock_response = MagicMock()\n mock_result = MagicMock()\n mock_alternative = MagicMock()\n\n mock_alternative.transcript = \"Hello, this is a test transcription.\"\n mock_alternative.confidence = 0.95\n\n # Mock words\n mock_word = MagicMock()\n mock_word.word = \"Hello\"\n mock_word.start_time.total_seconds.return_value = 0.0\n mock_word.end_time.total_seconds.return_value = 0.5\n mock_word.confidence = 0.98\n mock_alternative.words = [mock_word]\n\n mock_result.alternatives = [mock_alternative]\n mock_response.results = [mock_result]\n\n return mock_response\n\n\ndef test_missing_env_vars():\n \"\"\"Test that an error is raised if required environment variables are missing.\"\"\"\n with patch.dict(\"os.environ\", {}, clear=True):\n with pytest.raises(EnvironmentError) as exc_info:\n GoogleProvider()\n assert \"Missing one or more required Google environment variables\" in str(\n exc_info.value\n )\n\n\ndef test_vertex_interface():\n \"\"\"High-level test that the interface is initialized and chat completions are requested successfully.\"\"\"\n\n # Test case 1: Regular text response\n def test_text_response():\n user_greeting = \"Hello!\"\n message_history = [{\"role\": \"user\", \"content\": user_greeting}]\n selected_model = \"our-favorite-model\"\n response_text_content = \"mocked-text-response-from-model\"\n\n interface = GoogleProvider()\n mock_response = MagicMock()\n mock_response.candidates = [MagicMock()]\n mock_response.candidates[0].content.parts = [MagicMock()]\n mock_response.candidates[0].content.parts[0].text = response_text_content\n # Ensure function_call attribute doesn't exist\n del mock_response.candidates[0].content.parts[0].function_call\n\n with patch(\n \"aisuite.providers.google_provider.GenerativeModel\"\n ) as mock_generative_model:\n mock_model = MagicMock()\n mock_generative_model.return_value = mock_model\n mock_chat = MagicMock()\n mock_model.start_chat.return_value = mock_chat\n mock_chat.send_message.return_value = mock_response\n\n response = interface.chat_completions_create(\n messages=message_history,\n model=selected_model,\n temperature=0.7,\n )\n\n # Assert the response is in the correct format\n assert response.choices[0].message.content == response_text_content\n assert response.choices[0].finish_reason == \"stop\"\n\n # Test case 2: Function call response\n def test_function_call():\n user_greeting = \"What's the weather?\"\n message_history = [{\"role\": \"user\", \"content\": user_greeting}]\n selected_model = \"our-favorite-model\"\n\n interface = GoogleProvider()\n mock_response = MagicMock()\n mock_response.candidates = [MagicMock()]\n mock_response.candidates[0].content.parts = [MagicMock()]\n\n # Mock the function call response\n function_call_mock = MagicMock()\n function_call_mock.name = \"get_weather\"\n function_call_mock.args = {\"location\": \"San Francisco\"}\n mock_response.candidates[0].content.parts[0].function_call = function_call_mock\n mock_response.candidates[0].content.parts[0].text = None\n\n with patch(\n \"aisuite.providers.google_provider.GenerativeModel\"\n ) as mock_generative_model:\n mock_model = MagicMock()\n mock_generative_model.return_value = mock_model\n mock_chat = MagicMock()\n mock_model.start_chat.return_value = mock_chat\n mock_chat.send_message.return_value = mock_response\n\n response = interface.chat_completions_create(\n messages=message_history,\n model=selected_model,\n temperature=0.7,\n )\n\n # Assert the response contains the function call\n assert response.choices[0].message.content is None\n assert response.choices[0].message.tool_calls[0].type == \"function\"\n assert (\n response.choices[0].message.tool_calls[0].function.name == \"get_weather\"\n )\n assert json.loads(\n response.choices[0].message.tool_calls[0].function.arguments\n ) == {\"location\": \"San Francisco\"}\n assert response.choices[0].finish_reason == \"tool_calls\"\n\n # Run both test cases\n test_text_response()\n test_function_call()\n\n\ndef test_convert_openai_to_vertex_ai():\n \"\"\"Test the message conversion from OpenAI format to Vertex AI format.\"\"\"\n interface = GoogleProvider()\n message = {\"role\": \"user\", \"content\": \"Hello!\"}\n\n # Use the transformer to convert the message\n result = interface.transformer.convert_request([message])\n\n # Verify the conversion result\n assert len(result) == 1\n assert isinstance(result[0], Content)\n assert result[0].role == \"user\"\n assert len(result[0].parts) == 1\n assert isinstance(result[0].parts[0], Part)\n assert result[0].parts[0].text == \"Hello!\"\n\n\ndef test_role_conversions():\n \"\"\"Test that different message roles are converted correctly.\"\"\"\n interface = GoogleProvider()\n\n messages = [\n {\"role\": \"system\", \"content\": \"System message\"},\n {\"role\": \"user\", \"content\": \"User message\"},\n {\"role\": \"assistant\", \"content\": \"Assistant message\"},\n ]\n\n result = interface.transformer.convert_request(messages)\n\n # System and user messages should both be converted to \"user\" role in Vertex AI\n assert len(result) == 3\n assert result[0].role == \"user\" # system converted to user\n assert result[0].parts[0].text == \"System message\"\n\n assert result[1].role == \"user\"\n assert result[1].parts[0].text == \"User message\"\n\n assert result[2].role == \"model\" # assistant converted to model\n assert result[2].parts[0].text == \"Assistant message\"\n\n\nclass TestGoogleProvider:\n \"\"\"Test suite for Google provider functionality.\"\"\"\n\n def test_provider_initialization(self):\n \"\"\"Test that Google provider initializes correctly.\"\"\"\n provider = GoogleProvider()\n assert provider is not None\n assert hasattr(provider, \"audio\")\n assert hasattr(provider.audio, \"transcriptions\")\n\n\nclass TestGoogleASR:\n \"\"\"Test suite for Google ASR functionality.\"\"\"\n\n def test_audio_transcriptions_create_success(self, mock_google_speech_response):\n \"\"\"Test successful audio transcription.\"\"\"\n google_provider = GoogleProvider()\n mock_client = MagicMock()\n mock_client.recognize.return_value = mock_google_speech_response\n google_provider._speech_client = mock_client\n\n with patch(\"builtins.open\", mock_open(read_data=b\"fake audio data\")):\n result = google_provider.audio.transcriptions.create(\n model=\"google:latest_long\", file=\"test_audio.wav\"\n )\n\n assert isinstance(result, TranscriptionResult)\n assert result.text == \"Hello, this is a test transcription.\"\n assert result.confidence == 0.95\n assert result.task == \"transcribe\"\n\n def test_audio_transcriptions_create_with_file_object(\n self, mock_google_speech_response\n ):\n \"\"\"Test audio transcription with file-like object.\"\"\"\n google_provider = GoogleProvider()\n mock_client = MagicMock()\n mock_client.recognize.return_value = mock_google_speech_response\n google_provider._speech_client = mock_client\n\n audio_data = io.BytesIO(b\"fake audio data\")\n\n result = google_provider.audio.transcriptions.create(\n model=\"google:latest_long\", file=audio_data\n )\n\n assert isinstance(result, TranscriptionResult)\n assert result.text == \"Hello, this is a test transcription.\"\n\n def test_audio_transcriptions_create_with_options(\n self, mock_google_speech_response\n ):\n \"\"\"Test audio transcription with TranscriptionOptions.\"\"\"\n google_provider = GoogleProvider()\n mock_client = MagicMock()\n mock_client.recognize.return_value = mock_google_speech_response\n google_provider._speech_client = mock_client\n\n options = TranscriptionOptions(\n language=\"en\",\n enable_automatic_punctuation=True,\n include_word_timestamps=True,\n )\n\n with patch(\"builtins.open\", mock_open(read_data=b\"fake audio data\")):\n result = google_provider.audio.transcriptions.create(\n model=\"google:latest_long\", file=\"test_audio.wav\", options=options\n )\n\n mock_client.recognize.assert_called_once()\n assert isinstance(result, TranscriptionResult)\n assert result.text == \"Hello, this is a test transcription.\"\n\n def test_audio_transcriptions_create_error_handling(self):\n \"\"\"Test handling of Google Speech API errors.\"\"\"\n google_provider = GoogleProvider()\n mock_client = MagicMock()\n mock_client.recognize.side_effect = Exception(\"API Error\")\n google_provider._speech_client = mock_client\n\n with patch(\"builtins.open\", mock_open(read_data=b\"fake audio data\")):\n with pytest.raises(\n ASRError, match=\"Google Speech-to-Text error: API Error\"\n ):\n google_provider.audio.transcriptions.create(\n model=\"google:latest_long\", file=\"test_audio.wav\"\n )\n\n @pytest.mark.asyncio\n async def test_audio_transcriptions_create_stream_output(\n self, mock_google_speech_response\n ):\n \"\"\"Test streaming audio transcription with fixed config parameter.\"\"\"\n google_provider = GoogleProvider()\n mock_client = MagicMock()\n\n # Mock streaming response\n mock_streaming_response = MagicMock()\n mock_streaming_result = MagicMock()\n mock_alternative = MagicMock()\n mock_alternative.transcript = \"Hello streaming\"\n mock_streaming_result.alternatives = [mock_alternative]\n mock_streaming_result.is_final = True\n mock_streaming_response.results = [mock_streaming_result]\n\n mock_client.streaming_recognize.return_value = [mock_streaming_response]\n google_provider._speech_client = mock_client\n\n with patch(\"builtins.open\", mock_open(read_data=b\"fake audio data\")):\n result = google_provider.audio.transcriptions.create_stream_output(\n model=\"google:latest_long\", file=\"test_audio.wav\"\n )\n\n assert hasattr(result, \"__aiter__\")\n\n # Test that streaming_recognize is called with both config and requests\n chunks = []\n async for chunk in result:\n chunks.append(chunk)\n break # Just test one chunk\n\n # Verify the method was called with correct parameters\n mock_client.streaming_recognize.assert_called_once()\n call_args = mock_client.streaming_recognize.call_args\n assert \"config\" in call_args.kwargs\n assert \"requests\" in call_args.kwargs\n\n def test_parse_google_response_complete(self, mock_google_speech_response):\n \"\"\"Test parsing complete Google response.\"\"\"\n google_provider = GoogleProvider()\n result = google_provider.audio.transcriptions._parse_google_response(\n mock_google_speech_response\n )\n\n assert result.text == \"Hello, this is a test transcription.\"\n assert result.confidence == 0.95\n assert result.task == \"transcribe\"\n\n assert len(result.words) == 1\n word = result.words[0]\n assert word.word == \"Hello\"\n assert word.start == 0.0\n assert word.end == 0.5\n assert word.confidence == 0.98\n\n assert len(result.alternatives) == 1\n\n def test_parse_google_response_empty_results(self):\n \"\"\"Test parsing response with empty results.\"\"\"\n google_provider = GoogleProvider()\n mock_response = MagicMock()\n mock_response.results = []\n\n result = google_provider.audio.transcriptions._parse_google_response(\n mock_response\n )\n assert result.text == \"\"\n assert result.language is None\n" }, { "path": "tests/providers/test_groq_provider.py", "content": "\"\"\"Tests for the Groq provider.\"\"\"\n\nfrom unittest.mock import MagicMock, patch\n\nimport pytest\n\nfrom aisuite.providers.groq_provider import GroqProvider\n\n\n@pytest.fixture(autouse=True)\ndef set_api_key_env_var(monkeypatch):\n \"\"\"Fixture to set environment variables for tests.\"\"\"\n monkeypatch.setenv(\"GROQ_API_KEY\", \"test-api-key\")\n\n\ndef test_groq_provider():\n \"\"\"Test that the provider is initialized and chat completions are requested.\"\"\"\n\n user_greeting = \"Hello!\"\n message_history = [{\"role\": \"user\", \"content\": user_greeting}]\n selected_model = \"our-favorite-model\"\n chosen_temperature = 0.75\n response_text_content = \"mocked-text-response-from-model\"\n\n provider = GroqProvider()\n mock_response = MagicMock()\n mock_response.model_dump.return_value = {\n \"choices\": [{\"message\": {\"content\": response_text_content}}]\n }\n\n with patch.object(\n provider.client.chat.completions,\n \"create\",\n return_value=mock_response,\n ) as mock_create:\n response = provider.chat_completions_create(\n messages=message_history,\n model=selected_model,\n temperature=chosen_temperature,\n )\n\n mock_create.assert_called_with(\n messages=message_history,\n model=selected_model,\n temperature=chosen_temperature,\n )\n\n assert response.choices[0].message.content == response_text_content\n\n\ndef test_groq_provider_with_usage():\n \"\"\"Tests that usage data is correctly parsed when present in the response.\"\"\"\n\n user_greeting = \"Hello!\"\n message_history = [{\"role\": \"user\", \"content\": user_greeting}]\n selected_model = \"our-favorite-model\"\n chosen_temperature = 0.75\n response_text_content = \"mocked-text-response-from-model\"\n\n provider = GroqProvider()\n mock_response = MagicMock()\n mock_response.model_dump.return_value = {\n \"choices\": [\n {\"message\": {\"content\": response_text_content, \"role\": \"assistant\"}}\n ],\n \"usage\": {\n \"prompt_tokens\": 10,\n \"completion_tokens\": 20,\n \"total_tokens\": 30,\n },\n }\n\n with patch.object(\n provider.client.chat.completions,\n \"create\",\n return_value=mock_response,\n ):\n response = provider.chat_completions_create(\n messages=message_history,\n model=selected_model,\n temperature=chosen_temperature,\n )\n\n assert response.usage is not None\n assert response.usage.prompt_tokens == 10\n assert response.usage.completion_tokens == 20\n assert response.usage.total_tokens == 30\n" }, { "path": "tests/providers/test_huggingface_provider.py", "content": "\"\"\"Tests for Hugging Face provider ASR functionality.\"\"\"\n\nimport io\nfrom unittest.mock import MagicMock, mock_open, patch\n\nimport pytest\n\nfrom aisuite.providers.huggingface_provider import HuggingfaceProvider\nfrom aisuite.provider import ASRError\nfrom aisuite.framework.message import TranscriptionResult\n\n\n@pytest.fixture(autouse=True)\ndef set_api_key_env_var(monkeypatch):\n \"\"\"Fixture to set environment variables for tests.\"\"\"\n monkeypatch.setenv(\"HF_TOKEN\", \"test-hf-token\")\n\n\n@pytest.fixture\ndef huggingface_provider():\n \"\"\"Create a Hugging Face provider instance for testing.\"\"\"\n return HuggingfaceProvider()\n\n\n@pytest.fixture\ndef mock_huggingface_response():\n \"\"\"Create a mock Hugging Face API response for ASR.\"\"\"\n return {\n \"text\": \"Hello, this is a test transcription from Hugging Face.\",\n \"chunks\": [\n {\"text\": \" Hello\", \"timestamp\": [0.0, 0.5]},\n {\"text\": \",\", \"timestamp\": [0.5, 0.6]},\n {\"text\": \" this\", \"timestamp\": [0.6, 0.9]},\n {\"text\": \" is\", \"timestamp\": [0.9, 1.1]},\n {\"text\": \" a\", \"timestamp\": [1.1, 1.2]},\n {\"text\": \" test\", \"timestamp\": [1.2, 1.5]},\n ],\n }\n\n\n@pytest.fixture\ndef mock_huggingface_response_text_only():\n \"\"\"Create a mock Hugging Face API response with text only (no chunks).\"\"\"\n return {\"text\": \"Simple transcription without timestamps.\"}\n\n\nclass TestHuggingFaceProvider:\n \"\"\"Test suite for Hugging Face provider functionality.\"\"\"\n\n def test_provider_initialization(self, huggingface_provider):\n \"\"\"Test that Hugging Face provider initializes correctly.\"\"\"\n assert huggingface_provider is not None\n assert hasattr(huggingface_provider, \"token\")\n assert huggingface_provider.token == \"test-hf-token\"\n assert hasattr(huggingface_provider, \"audio\")\n assert hasattr(huggingface_provider.audio, \"transcriptions\")\n\n def test_audio_transcriptions_create_success(\n self, huggingface_provider, mock_huggingface_response\n ):\n \"\"\"Test successful audio transcription.\"\"\"\n mock_response = MagicMock()\n mock_response.status_code = 200\n mock_response.json.return_value = mock_huggingface_response\n\n with patch(\"builtins.open\", mock_open(read_data=b\"fake audio data\")), patch(\n \"requests.post\", return_value=mock_response\n ) as mock_post:\n result = huggingface_provider.audio.transcriptions.create(\n model=\"huggingface:openai/whisper-large-v3\", file=\"test_audio.wav\"\n )\n\n # Verify the request\n mock_post.assert_called_once()\n call_args = mock_post.call_args\n # URL is first positional argument\n assert \"api-inference.huggingface.co\" in call_args.args[0]\n assert \"openai/whisper-large-v3\" in call_args.args[0]\n assert (\n call_args.kwargs[\"headers\"][\"Authorization\"] == \"Bearer test-hf-token\"\n )\n assert call_args.kwargs[\"headers\"][\"Content-Type\"] == \"audio/wav\"\n\n # Verify the result\n assert isinstance(result, TranscriptionResult)\n assert (\n result.text == \"Hello, this is a test transcription from Hugging Face.\"\n )\n assert len(result.words) == 6\n assert result.words[0].word == \" Hello\"\n assert result.words[0].start == 0.0\n assert result.words[0].end == 0.5\n\n def test_audio_transcriptions_with_file_object(\n self, huggingface_provider, mock_huggingface_response\n ):\n \"\"\"Test audio transcription with file-like object.\"\"\"\n audio_data = io.BytesIO(b\"fake audio data\")\n\n mock_response = MagicMock()\n mock_response.status_code = 200\n mock_response.json.return_value = mock_huggingface_response\n\n with patch(\"requests.post\", return_value=mock_response):\n result = huggingface_provider.audio.transcriptions.create(\n model=\"huggingface:openai/whisper-large-v3\", file=audio_data\n )\n\n assert isinstance(result, TranscriptionResult)\n assert (\n result.text == \"Hello, this is a test transcription from Hugging Face.\"\n )\n\n def test_audio_transcriptions_content_type_detection(self, huggingface_provider):\n \"\"\"Test content type detection for different audio formats.\"\"\"\n mock_response = MagicMock()\n mock_response.status_code = 200\n mock_response.json.return_value = {\"text\": \"test\"}\n\n test_cases = [\n (\"audio.wav\", \"audio/wav\"),\n (\"audio.mp3\", \"audio/mpeg\"), # HF API requires audio/mpeg for MP3\n (\"audio.flac\", \"audio/flac\"),\n (\"audio.unknown\", \"audio/wav\"), # Default to wav\n ]\n\n for filename, expected_content_type in test_cases:\n with patch(\"builtins.open\", mock_open(read_data=b\"audio\")), patch(\n \"requests.post\", return_value=mock_response\n ) as mock_post:\n huggingface_provider.audio.transcriptions.create(\n model=\"huggingface:test-model\", file=filename\n )\n\n call_args = mock_post.call_args\n assert (\n call_args.kwargs[\"headers\"][\"Content-Type\"] == expected_content_type\n )\n\n def test_audio_transcriptions_retry_503(self, huggingface_provider):\n \"\"\"Test retry logic for 503 model loading error.\"\"\"\n import requests\n\n # First response: 503 error\n mock_response_503 = MagicMock()\n mock_response_503.status_code = 503\n\n # Create HTTP error with response attribute\n http_error = requests.exceptions.HTTPError(\"Model loading\")\n http_error.response = mock_response_503\n mock_response_503.raise_for_status.side_effect = http_error\n\n # Second response: Success\n mock_response_success = MagicMock()\n mock_response_success.status_code = 200\n mock_response_success.json.return_value = {\"text\": \"Success after retry\"}\n\n responses = [mock_response_503, mock_response_success]\n\n with patch(\"builtins.open\", mock_open(read_data=b\"audio\")), patch(\n \"requests.post\", side_effect=responses\n ) as mock_post:\n result = huggingface_provider.audio.transcriptions.create(\n model=\"huggingface:test-model\", file=\"test.wav\"\n )\n\n # Verify retry happened\n assert mock_post.call_count == 2\n\n # Verify second call had x-wait-for-model header\n second_call_headers = mock_post.call_args_list[1].kwargs[\"headers\"]\n assert second_call_headers.get(\"x-wait-for-model\") == \"true\"\n\n assert result.text == \"Success after retry\"\n\n def test_audio_transcriptions_error_handling(self, huggingface_provider):\n \"\"\"Test error handling for API failures.\"\"\"\n mock_response = MagicMock()\n mock_response.status_code = 400\n mock_response.raise_for_status.side_effect = Exception(\"Bad Request\")\n\n with patch(\"builtins.open\", mock_open(read_data=b\"audio\")), patch(\n \"requests.post\", return_value=mock_response\n ):\n with pytest.raises(ASRError, match=\"Hugging Face transcription error\"):\n huggingface_provider.audio.transcriptions.create(\n model=\"huggingface:test-model\", file=\"test.wav\"\n )\n\n def test_parse_response_standard_format(\n self, huggingface_provider, mock_huggingface_response\n ):\n \"\"\"Test parsing response with text and chunks.\"\"\"\n result = huggingface_provider.audio.transcriptions._parse_huggingface_response(\n mock_huggingface_response, \"test-model\"\n )\n\n assert isinstance(result, TranscriptionResult)\n assert result.text == \"Hello, this is a test transcription from Hugging Face.\"\n assert len(result.words) == 6\n assert result.words[0].word == \" Hello\"\n assert result.words[0].start == 0.0\n assert result.words[0].end == 0.5\n\n def test_parse_response_text_only(\n self, huggingface_provider, mock_huggingface_response_text_only\n ):\n \"\"\"Test parsing response with text only (no chunks).\"\"\"\n result = huggingface_provider.audio.transcriptions._parse_huggingface_response(\n mock_huggingface_response_text_only, \"test-model\"\n )\n\n assert isinstance(result, TranscriptionResult)\n assert result.text == \"Simple transcription without timestamps.\"\n assert result.words is None\n\n def test_parse_response_string_format(self, huggingface_provider):\n \"\"\"Test parsing response that is a plain string.\"\"\"\n result = huggingface_provider.audio.transcriptions._parse_huggingface_response(\n \"Plain string transcription\", \"test-model\"\n )\n\n assert isinstance(result, TranscriptionResult)\n assert result.text == \"Plain string transcription\"\n assert result.words is None\n\n def test_model_id_extraction(self, huggingface_provider):\n \"\"\"Test that model ID is correctly extracted from model string.\"\"\"\n mock_response = MagicMock()\n mock_response.status_code = 200\n mock_response.json.return_value = {\"text\": \"test\"}\n\n with patch(\"builtins.open\", mock_open(read_data=b\"audio\")), patch(\n \"requests.post\", return_value=mock_response\n ) as mock_post:\n huggingface_provider.audio.transcriptions.create(\n model=\"huggingface:openai/whisper-large-v3\", file=\"test.wav\"\n )\n\n # Verify URL contains correct model ID (URL is first positional arg)\n call_args = mock_post.call_args\n assert \"openai/whisper-large-v3\" in call_args.args[0]\n" }, { "path": "tests/providers/test_inception_provider.py", "content": "from unittest.mock import MagicMock, patch\n\nimport pytest\n\nfrom aisuite.providers.inception_provider import InceptionProvider\n\n\n@pytest.fixture(autouse=True)\ndef set_api_key_env_var(monkeypatch):\n \"\"\"Fixture to set environment variables for tests.\"\"\"\n monkeypatch.setenv(\"INCEPTION_API_KEY\", \"test-api-key\")\n\n\ndef test_inception_provider():\n \"\"\"High-level test that the provider is initialized and chat completions are requested successfully.\"\"\"\n\n user_greeting = \"Hello!\"\n message_history = [{\"role\": \"user\", \"content\": user_greeting}]\n selected_model = \"mercury\"\n chosen_temperature = 0\n response_text_content = \"mocked-text-response-from-model\"\n\n provider = InceptionProvider()\n mock_response = MagicMock()\n mock_response.choices = [MagicMock()]\n mock_response.choices[0].message = MagicMock()\n mock_response.choices[0].message.content = response_text_content\n\n with patch.object(\n provider.client.chat.completions,\n \"create\",\n return_value=mock_response,\n ) as mock_create:\n response = provider.chat_completions_create(\n messages=message_history,\n model=selected_model,\n temperature=chosen_temperature,\n )\n\n mock_create.assert_called_with(\n messages=message_history,\n model=selected_model,\n temperature=chosen_temperature,\n )\n\n assert response.choices[0].message.content == response_text_content\n" }, { "path": "tests/providers/test_lmstudio_provider.py", "content": "import pytest\nfrom unittest.mock import patch, MagicMock\nfrom aisuite.providers.lmstudio_provider import LmstudioProvider\n\n\n@pytest.fixture(autouse=True)\ndef set_api_url_var(monkeypatch):\n \"\"\"Fixture to set environment variables for tests.\"\"\"\n monkeypatch.setenv(\"LMSTUDIO_API_URL\", \"http://localhost:1234\")\n\n\ndef test_completion():\n \"\"\"Test that completions request successfully.\"\"\"\n\n user_greeting = \"Howdy!\"\n message_history = [{\"role\": \"user\", \"content\": user_greeting}]\n selected_model = \"best-model-ever\"\n chosen_temperature = 0.77\n response_text_content = \"mocked-text-response-from-ollama-model\"\n\n lmstudio = LmstudioProvider()\n mock_response = {\"choices\": [{\"message\": {\"content\": response_text_content}}]}\n\n with patch(\n \"httpx.post\",\n return_value=MagicMock(status_code=200, json=lambda: mock_response),\n ) as mock_post:\n response = lmstudio.chat_completions_create(\n messages=message_history,\n model=selected_model,\n temperature=chosen_temperature,\n )\n\n mock_post.assert_called_once_with(\n \"http://localhost:1234/v1/chat/completions\",\n json={\n \"model\": selected_model,\n \"messages\": message_history,\n \"stream\": False,\n \"temperature\": chosen_temperature,\n },\n timeout=300,\n )\n\n assert response.choices[0].message.content == response_text_content\n" }, { "path": "tests/providers/test_mistral_provider.py", "content": "import pytest\nfrom unittest.mock import patch, MagicMock\n\nfrom aisuite.providers.mistral_provider import MistralProvider\n\n\n@pytest.fixture(autouse=True)\ndef set_api_key_env_var(monkeypatch):\n \"\"\"Fixture to set environment variables for tests.\"\"\"\n monkeypatch.setenv(\"MISTRAL_API_KEY\", \"test-api-key\")\n\n\ndef test_mistral_provider():\n \"\"\"High-level test that the provider is initialized and chat completions are requested successfully.\"\"\"\n\n user_greeting = \"Hello!\"\n message_history = [{\"role\": \"user\", \"content\": user_greeting}]\n selected_model = \"our-favorite-model\"\n chosen_temperature = 0.75\n response_text_content = \"mocked-text-response-from-model\"\n\n provider = MistralProvider()\n mock_response = MagicMock()\n mock_response.model_dump.return_value = {\n \"choices\": [{\"message\": {\"content\": response_text_content}}]\n }\n\n with patch.object(\n provider.client.chat, \"complete\", return_value=mock_response\n ) as mock_create:\n response = provider.chat_completions_create(\n messages=message_history,\n model=selected_model,\n temperature=chosen_temperature,\n )\n\n mock_create.assert_called_with(\n messages=message_history,\n model=selected_model,\n temperature=chosen_temperature,\n )\n\n assert response.choices[0].message.content == response_text_content\n\n\ndef test_mistral_provider_with_usage():\n \"\"\"Tests that usage data is correctly parsed when present in the response.\"\"\"\n\n user_greeting = \"Hello!\"\n message_history = [{\"role\": \"user\", \"content\": user_greeting}]\n selected_model = \"our-favorite-model\"\n chosen_temperature = 0.75\n response_text_content = \"mocked-text-response-from-model\"\n\n provider = MistralProvider()\n mock_response = MagicMock()\n mock_response.model_dump.return_value = {\n \"choices\": [{\"message\": {\"content\": response_text_content}}],\n \"usage\": {\n \"prompt_tokens\": 10,\n \"completion_tokens\": 20,\n \"total_tokens\": 30,\n },\n }\n\n with patch.object(\n provider.client.chat, \"complete\", return_value=mock_response\n ) as mock_create:\n response = provider.chat_completions_create(\n messages=message_history,\n model=selected_model,\n temperature=chosen_temperature,\n )\n\n assert response.usage is not None\n assert response.usage.prompt_tokens == 10\n assert response.usage.completion_tokens == 20\n assert response.usage.total_tokens == 30\n" }, { "path": "tests/providers/test_nebius_provider.py", "content": "import pytest\nfrom unittest.mock import patch, MagicMock\n\nfrom aisuite.providers.nebius_provider import NebiusProvider\n\n\n@pytest.fixture(autouse=True)\ndef set_api_key_env_var(monkeypatch):\n \"\"\"Fixture to set environment variables for tests.\"\"\"\n monkeypatch.setenv(\"NEBIUS_API_KEY\", \"test-api-key\")\n\n\ndef test_nebius_provider():\n \"\"\"High-level test that the provider is initialized and chat completions are requested successfully.\"\"\"\n\n user_greeting = \"Hello!\"\n message_history = [{\"role\": \"user\", \"content\": user_greeting}]\n selected_model = \"our-favorite-model\"\n chosen_temperature = 0.75\n response_text_content = \"mocked-text-response-from-model\"\n\n provider = NebiusProvider()\n mock_response = MagicMock()\n mock_response.choices = [MagicMock()]\n mock_response.choices[0].message = MagicMock()\n mock_response.choices[0].message.content = response_text_content\n\n with patch.object(\n provider.client.chat.completions,\n \"create\",\n return_value=mock_response,\n ) as mock_create:\n response = provider.chat_completions_create(\n messages=message_history,\n model=selected_model,\n temperature=chosen_temperature,\n )\n\n mock_create.assert_called_with(\n messages=message_history,\n model=selected_model,\n temperature=chosen_temperature,\n )\n\n assert response.choices[0].message.content == response_text_content\n" }, { "path": "tests/providers/test_ollama_provider.py", "content": "import pytest\nfrom unittest.mock import patch, MagicMock\nfrom aisuite.providers.ollama_provider import OllamaProvider\n\n\n@pytest.fixture(autouse=True)\ndef set_api_url_var(monkeypatch):\n \"\"\"Fixture to set environment variables for tests.\"\"\"\n monkeypatch.setenv(\"OLLAMA_API_URL\", \"http://localhost:11434\")\n\n\ndef test_completion():\n \"\"\"Test that completions request successfully.\"\"\"\n\n user_greeting = \"Howdy!\"\n message_history = [{\"role\": \"user\", \"content\": user_greeting}]\n selected_model = \"best-model-ever\"\n chosen_temperature = 0.77\n response_text_content = \"mocked-text-response-from-ollama-model\"\n\n ollama = OllamaProvider()\n mock_response = {\"message\": {\"content\": response_text_content}}\n\n with patch(\n \"httpx.post\",\n return_value=MagicMock(status_code=200, json=lambda: mock_response),\n ) as mock_post:\n response = ollama.chat_completions_create(\n messages=message_history,\n model=selected_model,\n temperature=chosen_temperature,\n )\n\n mock_post.assert_called_once_with(\n \"http://localhost:11434/api/chat\",\n json={\n \"model\": selected_model,\n \"messages\": message_history,\n \"stream\": False,\n \"temperature\": chosen_temperature,\n },\n timeout=30,\n )\n\n assert response.choices[0].message.content == response_text_content\n" }, { "path": "tests/providers/test_openai_provider.py", "content": "\"\"\"Tests for OpenAI provider functionality.\"\"\"\n\nimport io\nfrom unittest.mock import MagicMock, mock_open, patch\n\nimport pytest\n\nfrom aisuite.providers.openai_provider import OpenaiProvider\nfrom aisuite.provider import ASRError\nfrom aisuite.framework.message import (\n TranscriptionResult,\n TranscriptionOptions,\n StreamingTranscriptionChunk,\n Segment,\n Word,\n)\n\n\n@pytest.fixture(autouse=True)\ndef set_api_key_env_var(monkeypatch):\n \"\"\"Fixture to set environment variables for tests.\"\"\"\n monkeypatch.setenv(\"OPENAI_API_KEY\", \"test-api-key\")\n\n\n@pytest.fixture\ndef openai_provider():\n \"\"\"Create an OpenAI provider instance for testing.\"\"\"\n return OpenaiProvider()\n\n\n@pytest.fixture\ndef mock_openai_response():\n \"\"\"Create a mock OpenAI API response for ASR.\"\"\"\n mock_response = MagicMock()\n mock_response.text = \"Hello, this is a test transcription.\"\n mock_response.language = \"en\"\n mock_response.segments = None\n return mock_response\n\n\nclass TestOpenAIProvider:\n \"\"\"Test suite for OpenAI provider functionality.\"\"\"\n\n def test_provider_initialization(self, openai_provider):\n \"\"\"Test that OpenAI provider initializes correctly.\"\"\"\n assert openai_provider is not None\n assert hasattr(openai_provider, \"client\")\n assert hasattr(openai_provider, \"audio\")\n assert hasattr(openai_provider.audio, \"transcriptions\")\n\n\nclass TestOpenAIASR:\n \"\"\"Test suite for OpenAI ASR functionality.\"\"\"\n\n def test_audio_transcriptions_create_success(\n self, openai_provider, mock_openai_response\n ):\n \"\"\"Test successful audio transcription.\"\"\"\n with patch(\n \"builtins.open\", mock_open(read_data=b\"fake audio data\")\n ), patch.object(\n openai_provider.client.audio.transcriptions,\n \"create\",\n return_value=mock_openai_response,\n ):\n result = openai_provider.audio.transcriptions.create(\n model=\"openai:whisper-1\", file=\"test_audio.mp3\"\n )\n\n assert isinstance(result, TranscriptionResult)\n assert result.text == \"Hello, this is a test transcription.\"\n assert result.language == \"en\"\n\n def test_audio_transcriptions_create_with_file_object(\n self, openai_provider, mock_openai_response\n ):\n \"\"\"Test audio transcription with file-like object.\"\"\"\n audio_data = io.BytesIO(b\"fake audio data\")\n\n with patch.object(\n openai_provider.client.audio.transcriptions,\n \"create\",\n return_value=mock_openai_response,\n ):\n result = openai_provider.audio.transcriptions.create(\n model=\"openai:whisper-1\", file=audio_data\n )\n\n assert isinstance(result, TranscriptionResult)\n assert result.text == \"Hello, this is a test transcription.\"\n\n def test_audio_transcriptions_create_with_kwargs(\n self, openai_provider, mock_openai_response\n ):\n \"\"\"Test audio transcription with additional parameters.\"\"\"\n with patch(\n \"builtins.open\", mock_open(read_data=b\"fake audio data\")\n ), patch.object(\n openai_provider.client.audio.transcriptions,\n \"create\",\n return_value=mock_openai_response,\n ) as mock_create:\n result = openai_provider.audio.transcriptions.create(\n model=\"openai:whisper-1\",\n file=\"test_audio.mp3\",\n language=\"en\",\n temperature=0.5,\n )\n\n mock_create.assert_called_once()\n call_kwargs = mock_create.call_args.kwargs\n assert call_kwargs[\"language\"] == \"en\"\n assert call_kwargs[\"temperature\"] == 0.5\n assert isinstance(result, TranscriptionResult)\n assert result.text == \"Hello, this is a test transcription.\"\n\n def test_audio_transcriptions_create_with_options(\n self, openai_provider, mock_openai_response\n ):\n \"\"\"Test audio transcription with TranscriptionOptions.\"\"\"\n options = TranscriptionOptions(\n language=\"en\",\n include_word_timestamps=True,\n enable_automatic_punctuation=True,\n )\n\n with patch(\n \"builtins.open\", mock_open(read_data=b\"fake audio data\")\n ), patch.object(\n openai_provider.client.audio.transcriptions,\n \"create\",\n return_value=mock_openai_response,\n ) as mock_create:\n result = openai_provider.audio.transcriptions.create(\n model=\"openai:whisper-1\", file=\"test_audio.mp3\", options=options\n )\n\n mock_create.assert_called_once()\n call_kwargs = mock_create.call_args.kwargs\n # Check that options parameters were extracted and passed as flat kwargs\n assert call_kwargs[\"language\"] == \"en\"\n assert call_kwargs[\"include_word_timestamps\"] is True\n assert call_kwargs[\"enable_automatic_punctuation\"] is True\n assert isinstance(result, TranscriptionResult)\n assert result.text == \"Hello, this is a test transcription.\"\n\n def test_audio_transcriptions_create_error_handling(self, openai_provider):\n \"\"\"Test error handling for API failures.\"\"\"\n with patch(\n \"builtins.open\", mock_open(read_data=b\"fake audio data\")\n ), patch.object(\n openai_provider.client.audio.transcriptions,\n \"create\",\n side_effect=Exception(\"API Error\"),\n ):\n with pytest.raises(ASRError, match=\"OpenAI transcription error: API Error\"):\n openai_provider.audio.transcriptions.create(\n model=\"openai:whisper-1\", file=\"test_audio.mp3\"\n )\n\n @pytest.mark.asyncio\n async def test_audio_transcriptions_create_stream_output(self, openai_provider):\n \"\"\"Test streaming audio transcription.\"\"\"\n # Mock streaming events\n mock_delta_event = MagicMock()\n mock_delta_event.type = \"transcript.text.delta\"\n mock_delta_event.delta = \"Hello\"\n\n mock_done_event = MagicMock()\n mock_done_event.type = \"transcript.text.done\"\n mock_done_event.text = \"Hello world\"\n\n with patch(\n \"builtins.open\", mock_open(read_data=b\"fake audio data\")\n ), patch.object(\n openai_provider.client.audio.transcriptions,\n \"create\",\n return_value=[mock_delta_event, mock_done_event],\n ):\n result = openai_provider.audio.transcriptions.create_stream_output(\n model=\"openai:gpt-4o-mini-transcribe\", file=\"test_audio.mp3\"\n )\n\n chunks = []\n async for chunk in result:\n chunks.append(chunk)\n\n assert len(chunks) == 2\n assert isinstance(chunks[0], StreamingTranscriptionChunk)\n assert chunks[0].text == \"Hello\"\n assert chunks[0].is_final is False # Delta event\n\n assert isinstance(chunks[1], StreamingTranscriptionChunk)\n assert chunks[1].text == \"Hello world\"\n assert chunks[1].is_final is True # Done event\n\n @pytest.mark.asyncio\n async def test_timestamp_granularities_error_handling(self, openai_provider):\n \"\"\"Test error handling for timestamp_granularities with incompatible response_format.\"\"\"\n options = TranscriptionOptions(\n response_format=\"json\",\n stream=True,\n timestamp_granularities=[\"word\"], # Now part of TranscriptionOptions\n )\n\n with patch(\"builtins.open\", mock_open(read_data=b\"fake audio data\")):\n with pytest.raises(\n ASRError,\n match=\"timestamp_granularities requires response_format='verbose_json'\",\n ):\n # The error should be raised before making the API call\n result = openai_provider.audio.transcriptions.create_stream_output(\n model=\"openai:gpt-4o-mini-transcribe\",\n file=\"test_audio.mp3\",\n options=options,\n )\n # Consume the async generator to trigger the validation\n async for _ in result:\n pass\n\n def test_parse_openai_response_with_segments_and_words(self, openai_provider):\n \"\"\"Test parsing OpenAI response with segments and words.\"\"\"\n mock_response = MagicMock()\n mock_response.text = \"Hello world\"\n mock_response.language = \"en\"\n\n mock_segment = MagicMock()\n mock_segment.id = 0\n mock_segment.seek = 0\n mock_segment.start = 0.0\n mock_segment.end = 2.5\n mock_segment.text = \"Hello world\"\n mock_segment.words = []\n mock_response.segments = [mock_segment]\n\n result = openai_provider.audio.transcriptions._parse_openai_response(\n mock_response\n )\n\n assert result.text == \"Hello world\"\n assert len(result.segments) == 1\n assert isinstance(result.segments[0], Segment)\n\n def test_parse_openai_response_empty(self, openai_provider):\n \"\"\"Test parsing response with minimal data.\"\"\"\n mock_response = MagicMock()\n mock_response.text = \"Test\"\n mock_response.language = \"en\"\n mock_response.segments = None\n\n result = openai_provider.audio.transcriptions._parse_openai_response(\n mock_response\n )\n\n assert result.text == \"Test\"\n assert result.language == \"en\"\n" }, { "path": "tests/providers/test_sambanova_provider.py", "content": "from unittest.mock import MagicMock, patch\n\nimport pytest\n\nfrom aisuite.providers.sambanova_provider import SambanovaProvider\n\n\n@pytest.fixture(autouse=True)\ndef set_api_key_env_var(monkeypatch):\n \"\"\"Fixture to set environment variables for tests.\"\"\"\n monkeypatch.setenv(\"SAMBANOVA_API_KEY\", \"test-api-key\")\n\n\ndef test_sambanova_provider():\n \"\"\"High-level test that the provider is initialized and chat completions are requested successfully.\"\"\"\n\n user_greeting = \"Hello!\"\n message_history = [{\"role\": \"user\", \"content\": user_greeting}]\n selected_model = \"our-favorite-model\"\n chosen_temperature = 0.75\n response_text_content = \"mocked-text-response-from-model\"\n\n provider = SambanovaProvider()\n mock_response = MagicMock()\n mock_response.model_dump.return_value = {\n \"choices\": [\n {\"message\": {\"content\": response_text_content, \"role\": \"assistant\"}}\n ]\n }\n\n with patch.object(\n provider.client.chat.completions,\n \"create\",\n return_value=mock_response,\n ) as mock_create:\n response = provider.chat_completions_create(\n messages=message_history,\n model=selected_model,\n temperature=chosen_temperature,\n )\n\n mock_create.assert_called_with(\n messages=message_history,\n model=selected_model,\n temperature=chosen_temperature,\n )\n\n assert response.choices[0].message.content == response_text_content\n\n\ndef test_sambanova_provider_with_usage():\n \"\"\"Tests that usage data is correctly parsed when present in the response.\"\"\"\n\n user_greeting = \"Hello!\"\n message_history = [{\"role\": \"user\", \"content\": user_greeting}]\n selected_model = \"our-favorite-model\"\n chosen_temperature = 0.75\n response_text_content = \"mocked-text-response-from-model\"\n\n provider = SambanovaProvider()\n mock_response = MagicMock()\n mock_response.model_dump.return_value = {\n \"choices\": [\n {\"message\": {\"content\": response_text_content, \"role\": \"assistant\"}}\n ],\n \"usage\": {\n \"prompt_tokens\": 10,\n \"completion_tokens\": 20,\n \"total_tokens\": 30,\n },\n }\n\n with patch.object(\n provider.client.chat.completions,\n \"create\",\n return_value=mock_response,\n ) as mock_create:\n response = provider.chat_completions_create(\n messages=message_history,\n model=selected_model,\n temperature=chosen_temperature,\n )\n\n assert response.usage is not None\n assert response.usage.prompt_tokens == 10\n assert response.usage.completion_tokens == 20\n assert response.usage.total_tokens == 30\n" }, { "path": "tests/providers/test_watsonx_provider.py", "content": "from unittest.mock import MagicMock, patch\n\nimport pytest\n\ntry:\n from ibm_watsonx_ai.metanames import GenChatParamsMetaNames as GenChatParams\nexcept Exception as e:\n pytest.skip(f\"Skipping test due to import error: {e}\", allow_module_level=True)\n\nfrom aisuite.providers.watsonx_provider import WatsonxProvider\n\n\n@pytest.fixture(autouse=True)\ndef set_api_key_env_var(monkeypatch):\n \"\"\"Fixture to set environment variables for tests.\"\"\"\n monkeypatch.setenv(\"WATSONX_SERVICE_URL\", \"https://watsonx-service-url.com\")\n monkeypatch.setenv(\"WATSONX_API_KEY\", \"test-api-key\")\n monkeypatch.setenv(\"WATSONX_PROJECT_ID\", \"test-project-id\")\n\n\n@pytest.mark.skip(reason=\"Skipping due to version compatibility issue on python 3.11\")\ndef test_watsonx_provider():\n \"\"\"High-level test that the provider is initialized and chat completions are requested successfully.\"\"\"\n\n user_greeting = \"Hello!\"\n message_history = [{\"role\": \"user\", \"content\": user_greeting}]\n selected_model = \"our-favorite-model\"\n chosen_temperature = 0.7\n response_text_content = \"mocked-text-response-from-model\"\n\n provider = WatsonxProvider()\n mock_response = {\"choices\": [{\"message\": {\"content\": response_text_content}}]}\n\n with patch(\n \"aisuite.providers.watsonx_provider.ModelInference\"\n ) as mock_model_inference:\n mock_model = MagicMock()\n mock_model_inference.return_value = mock_model\n mock_model.chat.return_value = mock_response\n\n response = provider.chat_completions_create(\n messages=message_history,\n model=selected_model,\n temperature=chosen_temperature,\n )\n\n # Assert that ModelInference was called with correct arguments.\n mock_model_inference.assert_called_once()\n args, kwargs = mock_model_inference.call_args\n assert kwargs[\"model_id\"] == selected_model\n assert kwargs[\"project_id\"] == provider.project_id\n\n # Assert that the credentials have the correct API key and service URL.\n credentials = kwargs[\"credentials\"]\n assert credentials.api_key == provider.api_key\n assert credentials.url == provider.service_url\n\n # Assert that chat was called with correct history and params\n mock_model.chat.assert_called_once_with(\n messages=message_history,\n params={GenChatParams.TEMPERATURE: chosen_temperature},\n )\n\n assert response.choices[0].message.content == response_text_content\n" }, { "path": "tests/test_provider.py", "content": "\"\"\"Tests for base Provider class and ASRError.\"\"\"\n\nimport pytest\nfrom unittest.mock import MagicMock\nfrom typing import Union, BinaryIO, Optional, AsyncGenerator\n\nfrom aisuite.provider import Provider, ASRError, Audio\nfrom aisuite.framework.message import (\n TranscriptionResult,\n TranscriptionOptions,\n StreamingTranscriptionChunk,\n)\n\n\nclass MockProvider(Provider):\n \"\"\"Mock provider for testing (no audio support).\"\"\"\n\n def chat_completions_create(self, model, messages):\n return MagicMock()\n\n\nclass MockTranscription(Audio.Transcription):\n \"\"\"Mock transcription implementation.\"\"\"\n\n def create(\n self,\n model: str,\n file: Union[str, BinaryIO],\n options: Optional[TranscriptionOptions] = None,\n **kwargs,\n ) -> TranscriptionResult:\n return TranscriptionResult(\n text=\"Mock transcription result\", language=\"en\", confidence=0.9\n )\n\n\nclass MockAudio(Audio):\n \"\"\"Mock audio implementation.\"\"\"\n\n def __init__(self):\n super().__init__()\n self.transcriptions = MockTranscription()\n\n\nclass MockASRProvider(Provider):\n \"\"\"Mock provider that implements ASR.\"\"\"\n\n def __init__(self):\n super().__init__()\n self.audio = MockAudio()\n\n def chat_completions_create(self, model, messages):\n return MagicMock()\n\n\nclass TestProvider:\n \"\"\"Test suite for base Provider class.\"\"\"\n\n def test_provider_is_abstract(self):\n \"\"\"Test that Provider cannot be instantiated directly.\"\"\"\n with pytest.raises(TypeError):\n Provider()\n\n def test_provider_without_audio_support(self):\n \"\"\"Test that provider without audio support has None audio attribute.\"\"\"\n provider = MockProvider()\n assert provider.audio is None\n\n def test_provider_asr_implementation_works(self):\n \"\"\"Test that providers can successfully implement ASR.\"\"\"\n provider = MockASRProvider()\n\n assert provider.audio is not None\n assert hasattr(provider.audio, \"transcriptions\")\n\n result = provider.audio.transcriptions.create(\"model\", \"file.mp3\")\n\n assert isinstance(result, TranscriptionResult)\n assert result.text == \"Mock transcription result\"\n assert result.language == \"en\"\n assert result.confidence == 0.9\n\n def test_transcription_base_class_not_implemented(self):\n \"\"\"Test that base Transcription class raises NotImplementedError.\"\"\"\n transcription = Audio.Transcription()\n\n with pytest.raises(NotImplementedError, match=\"Transcription not supported\"):\n transcription.create(\"model\", \"file.mp3\")\n\n def test_audio_base_class_initialization(self):\n \"\"\"Test that base Audio class initializes correctly.\"\"\"\n audio = Audio()\n assert audio.transcriptions is None\n\n\nclass TestASRError:\n \"\"\"Test suite for ASRError exception.\"\"\"\n\n def test_asr_error_creation_and_inheritance(self):\n \"\"\"Test ASRError creation and inheritance.\"\"\"\n error = ASRError(\"Test error message\")\n\n assert str(error) == \"Test error message\"\n assert isinstance(error, ASRError)\n assert isinstance(error, Exception)\n\n def test_asr_error_raising_and_catching(self):\n \"\"\"Test raising and catching ASRError.\"\"\"\n with pytest.raises(ASRError, match=\"Specific ASR error\"):\n raise ASRError(\"Specific ASR error\")\n\n # Test that it can be caught as Exception too\n with pytest.raises(Exception):\n raise ASRError(\"Generic catch test\")\n\n def test_asr_error_chaining(self):\n \"\"\"Test ASRError exception chaining.\"\"\"\n original_error = ValueError(\"Original error\")\n\n with pytest.raises(ASRError) as exc_info:\n try:\n raise original_error\n except ValueError as e:\n raise ASRError(\"Wrapped error\") from e\n\n assert exc_info.value.__cause__ == original_error\n" }, { "path": "tests/utils/test_mcp_memory_integration.py", "content": "\"\"\"\nIntegration test simulating the memory MCP server schema that was causing BadRequestError.\n\nThis test verifies that the fix for List[dict] schema conversion works correctly.\n\"\"\"\n\nimport unittest\nfrom aisuite.utils.tools import Tools\n\n\nclass MockMemoryMCPTool:\n \"\"\"Mock memory MCP tool with the exact schema that was failing.\"\"\"\n\n def __init__(self):\n self.__name__ = \"create_entities\"\n self.__doc__ = \"Create multiple entities in the knowledge graph\"\n\n # This is the exact schema from the memory MCP server that was causing:\n # \"Invalid schema for function 'create_entities': 'typing.List[dict]' is not valid\"\n self.__mcp_input_schema__ = {\n \"type\": \"object\",\n \"properties\": {\n \"entities\": {\n \"type\": \"array\",\n \"description\": \"List of entities to create\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"name\": {\n \"type\": \"string\",\n \"description\": \"The name of the entity\",\n },\n \"entityType\": {\n \"type\": \"string\",\n \"description\": \"The type of the entity\",\n },\n \"observations\": {\n \"type\": \"array\",\n \"description\": \"An array of observation contents\",\n \"items\": {\"type\": \"string\"},\n },\n },\n \"required\": [\"name\", \"entityType\", \"observations\"],\n },\n }\n },\n \"required\": [\"entities\"],\n }\n\n def __call__(self, **kwargs):\n \"\"\"Mock execution.\"\"\"\n return {\"created\": len(kwargs.get(\"entities\", [])), \"status\": \"success\"}\n\n\nclass TestMCPMemoryIntegration(unittest.TestCase):\n \"\"\"Test that the memory server schema works correctly.\"\"\"\n\n def test_memory_create_entities_schema(self):\n \"\"\"Test that create_entities schema is converted correctly for OpenAI.\"\"\"\n tool_manager = Tools()\n memory_tool = MockMemoryMCPTool()\n\n # This should not raise an error anymore\n tool_manager._add_tool(memory_tool)\n\n # Get the OpenAI format tools\n tools = tool_manager.tools()\n\n self.assertEqual(len(tools), 1)\n tool_spec = tools[0][\"function\"]\n\n # Verify the structure matches OpenAI expectations\n self.assertEqual(tool_spec[\"name\"], \"create_entities\")\n self.assertEqual(tool_spec[\"parameters\"][\"type\"], \"object\")\n self.assertIn(\"entities\", tool_spec[\"parameters\"][\"properties\"])\n\n # Verify the array type is preserved correctly (not 'typing.List[dict]')\n entities_param = tool_spec[\"parameters\"][\"properties\"][\"entities\"]\n self.assertEqual(entities_param[\"type\"], \"array\")\n self.assertIn(\"items\", entities_param)\n self.assertEqual(entities_param[\"items\"][\"type\"], \"object\")\n\n # Verify nested array (observations) is also preserved\n observations = entities_param[\"items\"][\"properties\"][\"observations\"]\n self.assertEqual(observations[\"type\"], \"array\")\n self.assertEqual(observations[\"items\"][\"type\"], \"string\")\n\n def test_memory_tool_openai_format_validation(self):\n \"\"\"Verify the output format would be accepted by OpenAI API.\"\"\"\n tool_manager = Tools()\n memory_tool = MockMemoryMCPTool()\n tool_manager._add_tool(memory_tool)\n\n tools = tool_manager.tools()\n tool_spec = tools[0][\"function\"]\n\n # Check that there are no Python type strings in the schema\n import json\n\n schema_json = json.dumps(tool_spec[\"parameters\"])\n\n # These should NOT appear in valid OpenAI JSON Schema\n self.assertNotIn(\"typing.\", schema_json)\n self.assertNotIn(\"List[\", schema_json)\n self.assertNotIn(\"Dict[\", schema_json)\n\n # Only valid JSON Schema types should appear\n self.assertIn('\"type\": \"array\"', schema_json)\n self.assertIn('\"type\": \"object\"', schema_json)\n self.assertIn('\"type\": \"string\"', schema_json)\n\n def test_memory_tool_execution(self):\n \"\"\"Test that the tool can be executed with proper validation.\"\"\"\n tool_manager = Tools()\n memory_tool = MockMemoryMCPTool()\n tool_manager._add_tool(memory_tool)\n\n # Simulate a tool call from the LLM\n tool_call = {\n \"id\": \"call_123\",\n \"function\": {\n \"name\": \"create_entities\",\n \"arguments\": {\n \"entities\": [\n {\n \"name\": \"MCP\",\n \"entityType\": \"Protocol\",\n \"observations\": [\n \"Enables LLM tool calling\",\n \"Uses JSON Schema\",\n ],\n },\n {\n \"name\": \"aisuite\",\n \"entityType\": \"Library\",\n \"observations\": [\"Unified API\", \"Multi-provider support\"],\n },\n ]\n },\n },\n }\n\n # This should execute successfully\n results, messages = tool_manager.execute_tool(tool_call)\n\n self.assertEqual(len(results), 1)\n self.assertEqual(results[0][\"created\"], 2)\n self.assertEqual(results[0][\"status\"], \"success\")\n\n\nif __name__ == \"__main__\":\n unittest.main()\n" }, { "path": "tests/utils/test_tool_manager.py", "content": "import unittest\nfrom pydantic import BaseModel\nfrom typing import Dict\nfrom aisuite.utils.tools import Tools # Import your ToolManager class\nfrom enum import Enum\n\n\n# Define a sample tool function and Pydantic model for testing\nclass TemperatureUnit(str, Enum):\n CELSIUS = \"Celsius\"\n FAHRENHEIT = \"Fahrenheit\"\n\n\nclass TemperatureParamsV2(BaseModel):\n location: str\n unit: TemperatureUnit = TemperatureUnit.CELSIUS\n\n\nclass TemperatureParams(BaseModel):\n location: str\n unit: str = \"Celsius\"\n\n\ndef get_current_temperature(location: str, unit: str = \"Celsius\") -> Dict[str, str]:\n \"\"\"Gets the current temperature for a specific location and unit.\"\"\"\n return {\"location\": location, \"unit\": unit, \"temperature\": \"72\"}\n\n\ndef missing_annotation_tool(location, unit=\"Celsius\"):\n \"\"\"Tool function without type annotations.\"\"\"\n return {\"location\": location, \"unit\": unit, \"temperature\": \"72\"}\n\n\ndef get_current_temperature_v2(\n location: str, unit: TemperatureUnit = TemperatureUnit.CELSIUS\n) -> Dict[str, str]:\n \"\"\"Gets the current temperature for a specific location and unit (with enum support).\"\"\"\n return {\"location\": location, \"unit\": unit, \"temperature\": \"72\"}\n\n\nclass TestToolManager(unittest.TestCase):\n def setUp(self):\n self.tool_manager = Tools()\n\n def test_add_tool_with_pydantic_model(self):\n \"\"\"Test adding a tool with an explicit Pydantic model.\"\"\"\n self.tool_manager._add_tool(get_current_temperature, TemperatureParams)\n\n expected_tool_spec = [\n {\n \"type\": \"function\",\n \"function\": {\n \"name\": \"get_current_temperature\",\n \"description\": \"Gets the current temperature for a specific location and unit.\",\n \"parameters\": {\n \"type\": \"object\",\n \"properties\": {\n \"location\": {\n \"type\": \"string\",\n \"description\": \"\",\n },\n \"unit\": {\n \"type\": \"string\",\n \"description\": \"\",\n \"default\": \"Celsius\",\n },\n },\n \"required\": [\"location\"],\n },\n },\n }\n ]\n\n tools = self.tool_manager.tools()\n self.assertIn(\n \"get_current_temperature\", [tool[\"function\"][\"name\"] for tool in tools]\n )\n assert (\n tools == expected_tool_spec\n ), f\"Expected {expected_tool_spec}, but got {tools}\"\n\n def test_add_tool_with_signature_inference(self):\n \"\"\"Test adding a tool and inferring parameters from the function signature.\"\"\"\n self.tool_manager._add_tool(get_current_temperature)\n # Expected output from tool_manager.tools() when called with OpenAI format\n expected_tool_spec = [\n {\n \"type\": \"function\",\n \"function\": {\n \"name\": \"get_current_temperature\",\n \"description\": \"Gets the current temperature for a specific location and unit.\",\n \"parameters\": {\n \"type\": \"object\",\n \"properties\": {\n \"location\": {\n \"type\": \"string\",\n \"description\": \"\", # No description provided in function signature\n },\n \"unit\": {\n \"type\": \"string\",\n \"description\": \"\",\n \"default\": \"Celsius\",\n },\n },\n \"required\": [\"location\"],\n },\n },\n }\n ]\n tools = self.tool_manager.tools()\n print(tools)\n self.assertIn(\n \"get_current_temperature\", [tool[\"function\"][\"name\"] for tool in tools]\n )\n assert (\n tools == expected_tool_spec\n ), f\"Expected {expected_tool_spec}, but got {tools}\"\n\n def test_add_tool_missing_annotation_raises_exception(self):\n \"\"\"Test that adding a tool with missing type annotations raises a TypeError.\"\"\"\n with self.assertRaises(TypeError):\n self.tool_manager._add_tool(missing_annotation_tool)\n\n def test_execute_tool_valid_parameters(self):\n \"\"\"Test executing a registered tool with valid parameters.\"\"\"\n self.tool_manager._add_tool(get_current_temperature, TemperatureParams)\n tool_call = {\n \"id\": \"call_1\",\n \"function\": {\n \"name\": \"get_current_temperature\",\n \"arguments\": {\"location\": \"San Francisco\", \"unit\": \"Celsius\"},\n },\n }\n result, result_message = self.tool_manager.execute_tool(tool_call)\n\n # Assuming result is returned as a list with a single dictionary\n result_dict = result[0] if isinstance(result, list) else result\n\n # Check that the result matches expected output\n self.assertEqual(result_dict[\"location\"], \"San Francisco\")\n self.assertEqual(result_dict[\"unit\"], \"Celsius\")\n self.assertEqual(result_dict[\"temperature\"], \"72\")\n\n def test_execute_tool_invalid_parameters(self):\n \"\"\"Test that executing a tool with invalid parameters raises a ValueError.\"\"\"\n self.tool_manager._add_tool(get_current_temperature, TemperatureParams)\n tool_call = {\n \"id\": \"call_1\",\n \"function\": {\n \"name\": \"get_current_temperature\",\n \"arguments\": {\"location\": 123}, # Invalid type for location\n },\n }\n\n with self.assertRaises(ValueError) as context:\n self.tool_manager.execute_tool(tool_call)\n\n # Verify the error message contains information about the validation error\n self.assertIn(\n \"Error in tool 'get_current_temperature' parameters\", str(context.exception)\n )\n\n def test_add_tool_with_enum(self):\n \"\"\"Test adding a tool with an enum parameter.\"\"\"\n self.tool_manager._add_tool(get_current_temperature_v2, TemperatureParamsV2)\n\n expected_tool_spec = [\n {\n \"type\": \"function\",\n \"function\": {\n \"name\": \"get_current_temperature_v2\",\n \"description\": \"Gets the current temperature for a specific location and unit (with enum support).\",\n \"parameters\": {\n \"type\": \"object\",\n \"properties\": {\n \"location\": {\n \"type\": \"string\",\n \"description\": \"\",\n },\n \"unit\": {\n \"type\": \"string\",\n \"enum\": [\"Celsius\", \"Fahrenheit\"],\n \"description\": \"\",\n \"default\": \"Celsius\",\n },\n },\n \"required\": [\"location\"],\n },\n },\n }\n ]\n\n tools = self.tool_manager.tools()\n assert (\n tools == expected_tool_spec\n ), f\"Expected {expected_tool_spec}, but got {tools}\"\n\n\nif __name__ == \"__main__\":\n unittest.main()\n" }, { "path": "tests/utils/test_tools_mcp_schema.py", "content": "import unittest\nfrom typing import Dict, Any, List\nfrom aisuite.utils.tools import Tools\n\n\nclass MockMCPToolWrapper:\n \"\"\"Mock MCP tool wrapper for testing schema preservation.\"\"\"\n\n def __init__(self, name: str, description: str, input_schema: Dict[str, Any]):\n self.__name__ = name\n self.__doc__ = description\n self.__mcp_input_schema__ = input_schema\n\n def __call__(self, **kwargs):\n \"\"\"Mock execution.\"\"\"\n return {\"result\": \"success\", \"args\": kwargs}\n\n\nclass TestToolsMCPSchema(unittest.TestCase):\n \"\"\"Test suite for MCP schema handling in Tools class.\"\"\"\n\n def setUp(self):\n \"\"\"Set up test fixtures.\"\"\"\n self.tool_manager = Tools()\n\n def test_mcp_tool_with_simple_types(self):\n \"\"\"Test MCP tool with simple types (string, integer, boolean).\"\"\"\n input_schema = {\n \"type\": \"object\",\n \"properties\": {\n \"name\": {\"type\": \"string\", \"description\": \"User name\"},\n \"age\": {\"type\": \"integer\", \"description\": \"User age\"},\n \"active\": {\"type\": \"boolean\", \"description\": \"Is active\"},\n },\n \"required\": [\"name\"],\n }\n\n tool = MockMCPToolWrapper(\"test_simple\", \"A simple test tool\", input_schema)\n self.tool_manager._add_tool(tool)\n\n tools = self.tool_manager.tools()\n self.assertEqual(len(tools), 1)\n\n # Verify the schema was preserved exactly\n tool_spec = tools[0][\"function\"]\n self.assertEqual(tool_spec[\"name\"], \"test_simple\")\n self.assertEqual(tool_spec[\"description\"], \"A simple test tool\")\n self.assertEqual(tool_spec[\"parameters\"], input_schema)\n\n def test_mcp_tool_with_array_of_objects(self):\n \"\"\"Test MCP tool with array of objects (List[dict]).\"\"\"\n input_schema = {\n \"type\": \"object\",\n \"properties\": {\n \"entities\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\n \"name\": {\"type\": \"string\"},\n \"type\": {\"type\": \"string\"},\n },\n \"required\": [\"name\", \"type\"],\n },\n \"description\": \"List of entities to create\",\n }\n },\n \"required\": [\"entities\"],\n }\n\n tool = MockMCPToolWrapper(\n \"create_entities\", \"Create multiple entities\", input_schema\n )\n self.tool_manager._add_tool(tool)\n\n tools = self.tool_manager.tools()\n tool_spec = tools[0][\"function\"]\n\n # Verify complex array schema is preserved\n self.assertEqual(\n tool_spec[\"parameters\"][\"properties\"][\"entities\"][\"type\"], \"array\"\n )\n self.assertIn(\"items\", tool_spec[\"parameters\"][\"properties\"][\"entities\"])\n self.assertEqual(\n tool_spec[\"parameters\"][\"properties\"][\"entities\"][\"items\"][\"type\"], \"object\"\n )\n\n def test_mcp_tool_with_nested_objects(self):\n \"\"\"Test MCP tool with nested object structures.\"\"\"\n input_schema = {\n \"type\": \"object\",\n \"properties\": {\n \"user\": {\n \"type\": \"object\",\n \"properties\": {\n \"name\": {\"type\": \"string\"},\n \"address\": {\n \"type\": \"object\",\n \"properties\": {\n \"street\": {\"type\": \"string\"},\n \"city\": {\"type\": \"string\"},\n },\n },\n },\n }\n },\n \"required\": [\"user\"],\n }\n\n tool = MockMCPToolWrapper(\n \"create_user\", \"Create user with address\", input_schema\n )\n self.tool_manager._add_tool(tool)\n\n tools = self.tool_manager.tools()\n tool_spec = tools[0][\"function\"]\n\n # Verify nested structure is preserved\n self.assertEqual(tool_spec[\"parameters\"], input_schema)\n self.assertIn(\n \"address\", tool_spec[\"parameters\"][\"properties\"][\"user\"][\"properties\"]\n )\n\n def test_mcp_tool_with_array_of_strings(self):\n \"\"\"Test MCP tool with array of simple types (List[str]).\"\"\"\n input_schema = {\n \"type\": \"object\",\n \"properties\": {\n \"tags\": {\n \"type\": \"array\",\n \"items\": {\"type\": \"string\"},\n \"description\": \"List of tags\",\n }\n },\n \"required\": [\"tags\"],\n }\n\n tool = MockMCPToolWrapper(\"add_tags\", \"Add tags to item\", input_schema)\n self.tool_manager._add_tool(tool)\n\n tools = self.tool_manager.tools()\n tool_spec = tools[0][\"function\"]\n\n # Verify array of strings is preserved\n self.assertEqual(tool_spec[\"parameters\"][\"properties\"][\"tags\"][\"type\"], \"array\")\n self.assertEqual(\n tool_spec[\"parameters\"][\"properties\"][\"tags\"][\"items\"][\"type\"], \"string\"\n )\n\n def test_mcp_tool_detection(self):\n \"\"\"Test that MCP tools are properly detected via __mcp_input_schema__ attribute.\"\"\"\n input_schema = {\n \"type\": \"object\",\n \"properties\": {\"param\": {\"type\": \"string\"}},\n \"required\": [\"param\"],\n }\n\n tool = MockMCPToolWrapper(\"mcp_tool\", \"An MCP tool\", input_schema)\n\n # Verify the attribute exists\n self.assertTrue(hasattr(tool, \"__mcp_input_schema__\"))\n self.assertEqual(tool.__mcp_input_schema__, input_schema)\n\n def test_mcp_tool_with_optional_parameters(self):\n \"\"\"Test MCP tool with mix of required and optional parameters.\"\"\"\n input_schema = {\n \"type\": \"object\",\n \"properties\": {\n \"required_param\": {\n \"type\": \"string\",\n \"description\": \"Required parameter\",\n },\n \"optional_param\": {\n \"type\": \"integer\",\n \"description\": \"Optional parameter\",\n },\n },\n \"required\": [\"required_param\"],\n }\n\n tool = MockMCPToolWrapper(\n \"mixed_params\", \"Tool with mixed params\", input_schema\n )\n self.tool_manager._add_tool(tool)\n\n tools = self.tool_manager.tools()\n tool_spec = tools[0][\"function\"]\n\n # Verify required fields are correct\n self.assertEqual(tool_spec[\"parameters\"][\"required\"], [\"required_param\"])\n self.assertIn(\"required_param\", tool_spec[\"parameters\"][\"properties\"])\n self.assertIn(\"optional_param\", tool_spec[\"parameters\"][\"properties\"])\n\n def test_mcp_schema_preserves_additional_fields(self):\n \"\"\"Test that additional JSON Schema fields are preserved (enum, format, etc.).\"\"\"\n input_schema = {\n \"type\": \"object\",\n \"properties\": {\n \"status\": {\n \"type\": \"string\",\n \"enum\": [\"active\", \"inactive\", \"pending\"],\n \"description\": \"Status value\",\n },\n \"email\": {\n \"type\": \"string\",\n \"format\": \"email\",\n \"description\": \"Email address\",\n },\n \"count\": {\n \"type\": \"integer\",\n \"minimum\": 0,\n \"maximum\": 100,\n \"description\": \"Count value\",\n },\n },\n \"required\": [\"status\"],\n }\n\n tool = MockMCPToolWrapper(\n \"advanced_schema\", \"Tool with advanced schema\", input_schema\n )\n self.tool_manager._add_tool(tool)\n\n tools = self.tool_manager.tools()\n tool_spec = tools[0][\"function\"]\n\n # Verify enum is preserved\n self.assertIn(\"enum\", tool_spec[\"parameters\"][\"properties\"][\"status\"])\n self.assertEqual(\n tool_spec[\"parameters\"][\"properties\"][\"status\"][\"enum\"],\n [\"active\", \"inactive\", \"pending\"],\n )\n\n # Verify format is preserved\n self.assertIn(\"format\", tool_spec[\"parameters\"][\"properties\"][\"email\"])\n self.assertEqual(\n tool_spec[\"parameters\"][\"properties\"][\"email\"][\"format\"], \"email\"\n )\n\n # Verify min/max are preserved\n self.assertIn(\"minimum\", tool_spec[\"parameters\"][\"properties\"][\"count\"])\n self.assertIn(\"maximum\", tool_spec[\"parameters\"][\"properties\"][\"count\"])\n\n def test_mcp_tool_execution_with_validation(self):\n \"\"\"Test that MCP tools can be executed and parameters are validated.\"\"\"\n input_schema = {\n \"type\": \"object\",\n \"properties\": {\n \"name\": {\"type\": \"string\", \"description\": \"Name\"},\n \"count\": {\"type\": \"integer\", \"description\": \"Count\"},\n },\n \"required\": [\"name\"],\n }\n\n tool = MockMCPToolWrapper(\n \"validate_tool\", \"Tool for validation test\", input_schema\n )\n self.tool_manager._add_tool(tool)\n\n # Test valid execution\n tool_call = {\n \"id\": \"call_1\",\n \"function\": {\n \"name\": \"validate_tool\",\n \"arguments\": {\"name\": \"test\", \"count\": 5},\n },\n }\n\n results, messages = self.tool_manager.execute_tool(tool_call)\n self.assertEqual(len(results), 1)\n self.assertIn(\"result\", results[0])\n\n def test_mcp_tool_with_empty_schema(self):\n \"\"\"Test MCP tool with no parameters.\"\"\"\n input_schema = {\"type\": \"object\", \"properties\": {}, \"required\": []}\n\n tool = MockMCPToolWrapper(\"no_params\", \"Tool with no params\", input_schema)\n self.tool_manager._add_tool(tool)\n\n tools = self.tool_manager.tools()\n tool_spec = tools[0][\"function\"]\n\n self.assertEqual(tool_spec[\"parameters\"][\"properties\"], {})\n self.assertEqual(tool_spec[\"parameters\"][\"required\"], [])\n\n def test_multiple_mcp_tools(self):\n \"\"\"Test adding multiple MCP tools to the manager.\"\"\"\n schema1 = {\n \"type\": \"object\",\n \"properties\": {\"param1\": {\"type\": \"string\"}},\n \"required\": [\"param1\"],\n }\n\n schema2 = {\n \"type\": \"object\",\n \"properties\": {\"param2\": {\"type\": \"integer\"}},\n \"required\": [\"param2\"],\n }\n\n tool1 = MockMCPToolWrapper(\"tool1\", \"First tool\", schema1)\n tool2 = MockMCPToolWrapper(\"tool2\", \"Second tool\", schema2)\n\n self.tool_manager._add_tool(tool1)\n self.tool_manager._add_tool(tool2)\n\n tools = self.tool_manager.tools()\n self.assertEqual(len(tools), 2)\n\n tool_names = [tool[\"function\"][\"name\"] for tool in tools]\n self.assertIn(\"tool1\", tool_names)\n self.assertIn(\"tool2\", tool_names)\n\n def test_mcp_tool_schema_not_modified(self):\n \"\"\"Test that the original schema is not modified during processing.\"\"\"\n original_schema = {\n \"type\": \"object\",\n \"properties\": {\n \"data\": {\n \"type\": \"array\",\n \"items\": {\n \"type\": \"object\",\n \"properties\": {\"key\": {\"type\": \"string\"}},\n },\n }\n },\n \"required\": [\"data\"],\n }\n\n # Create a copy to verify immutability\n import copy\n\n schema_copy = copy.deepcopy(original_schema)\n\n tool = MockMCPToolWrapper(\n \"immutable_tool\", \"Test immutability\", original_schema\n )\n self.tool_manager._add_tool(tool)\n\n # Verify original schema wasn't modified\n self.assertEqual(original_schema, schema_copy)\n\n def test_backward_compatibility_non_mcp_tools(self):\n \"\"\"Test that regular Python functions still work (backward compatibility).\"\"\"\n\n def regular_function(name: str, age: int = 25) -> Dict[str, Any]:\n \"\"\"A regular Python function.\"\"\"\n return {\"name\": name, \"age\": age}\n\n # Regular functions don't have __mcp_input_schema__\n self.assertFalse(hasattr(regular_function, \"__mcp_input_schema__\"))\n\n # Should still work with the Tools class\n self.tool_manager._add_tool(regular_function)\n\n tools = self.tool_manager.tools()\n self.assertEqual(len(tools), 1)\n self.assertEqual(tools[0][\"function\"][\"name\"], \"regular_function\")\n\n\nif __name__ == \"__main__\":\n unittest.main()\n" } ]