# How to integrate Bench MCP with Pydantic AI ```json { "title": "How to integrate Bench MCP with Pydantic AI", "toolkit": "Bench", "toolkit_slug": "bench", "framework": "Pydantic AI", "framework_slug": "pydantic-ai", "url": "https://composio.dev/toolkits/bench/framework/pydantic-ai", "markdown_url": "https://composio.dev/toolkits/bench/framework/pydantic-ai.md", "updated_at": "2026-05-12T10:02:44.495Z" } ``` ## Introduction This guide walks you through connecting Bench to Pydantic AI using the Composio tool router. By the end, you'll have a working Bench agent that can pause workflow for 10 seconds, simulate delay during test runs, wait before running next benchmark through natural language commands. This guide will help you understand how to give your Pydantic AI agent real control over a Bench account through Composio's Bench MCP server. Before we dive in, let's take a quick look at the key ideas and tools involved. ## Also integrate Bench with - [OpenAI Agents SDK](https://composio.dev/toolkits/bench/framework/open-ai-agents-sdk) - [Claude Agent SDK](https://composio.dev/toolkits/bench/framework/claude-agents-sdk) - [Claude Code](https://composio.dev/toolkits/bench/framework/claude-code) - [Claude Cowork](https://composio.dev/toolkits/bench/framework/claude-cowork) - [Codex](https://composio.dev/toolkits/bench/framework/codex) - [OpenClaw](https://composio.dev/toolkits/bench/framework/openclaw) - [Hermes](https://composio.dev/toolkits/bench/framework/hermes-agent) - [CLI](https://composio.dev/toolkits/bench/framework/cli) - [Google ADK](https://composio.dev/toolkits/bench/framework/google-adk) - [LangChain](https://composio.dev/toolkits/bench/framework/langchain) - [Vercel AI SDK](https://composio.dev/toolkits/bench/framework/ai-sdk) - [Mastra AI](https://composio.dev/toolkits/bench/framework/mastra-ai) - [LlamaIndex](https://composio.dev/toolkits/bench/framework/llama-index) - [CrewAI](https://composio.dev/toolkits/bench/framework/crew-ai) ## TL;DR Here's what you'll learn: - How to set up your Composio API key and User ID - How to create a Composio Tool Router session for Bench - How to attach an MCP Server to a Pydantic AI agent - How to stream responses and maintain chat history - How to build a simple REPL-style chat interface to test your Bench workflows ## What is Pydantic AI? Pydantic AI is a Python framework for building AI agents with strong typing and validation. It leverages Pydantic's data validation capabilities to create robust, type-safe AI applications. Key features include: - Type Safety: Built on Pydantic for automatic data validation - MCP Support: Native support for Model Context Protocol servers - Streaming: Built-in support for streaming responses - Async First: Designed for async/await patterns ## What is the Bench MCP server, and what's possible with it? The Bench MCP server is an implementation of the Model Context Protocol that connects your AI agent and assistants like Claude, Cursor, etc directly to your Bench environment. It provides structured and secure access to benchmarking operations, so your agent can perform actions like simulating wait times, testing response delays, and orchestrating timed pauses within automated workflows. - Simulate processing delays: Instruct your agent to initiate a sleep or wait state, mimicking real-world processing or benchmarking scenarios. - Test agent response times: Have the agent pause for a set duration to measure how your applications handle delays or timeouts. - Orchestrate workflow timing: Use controlled sleep intervals to sequence multi-step tasks or coordinate between services during automation runs. - Benchmark system latency: Trigger sleep commands to evaluate how your systems and integrations perform under different timing constraints. ## Supported Tools | Tool slug | Name | Description | |---|---|---| | `BENCH_SLEEP` | Sleep | Sleep | ## Supported Triggers None listed. ## Creating MCP Server - Stand-alone vs Composio SDK The Bench MCP server is an implementation of the Model Context Protocol that connects your AI agent to Bench. It provides structured and secure access so your agent can perform Bench operations on your behalf through a secure, permission-based interface. With Composio's managed implementation, you don't have to create your own developer app. For production, if you're building an end product, we recommend using your own credentials. The managed server helps you prototype fast and go from 0-1 faster. ## Step-by-step Guide ### 1. Prerequisites Before starting, make sure you have: - Python 3.9 or higher - A Composio account with an active API key - Basic familiarity with Python and async programming ### 1. Getting API Keys for OpenAI and Composio OpenAI API Key - Go to the [OpenAI dashboard](https://platform.openai.com/settings/organization/api-keys) and create an API key. You'll need credits to use the models, or you can connect to another model provider. - Keep the API key safe. Composio API Key - Log in to the [Composio dashboard](https://dashboard.composio.dev?utm_source=toolkits&utm_medium=framework_docs). - Navigate to your API settings and generate a new API key. - Store this key securely as you'll need it for authentication. ### 2. Install dependencies Install the required libraries. What's happening: - composio connects your agent to external SaaS tools like Bench - pydantic-ai lets you create structured AI agents with tool support - python-dotenv loads your environment variables securely from a .env file ```bash pip install composio pydantic-ai python-dotenv ``` ### 3. Set up environment variables Create a .env file in your project root. What's happening: - COMPOSIO_API_KEY authenticates your agent to Composio's API - USER_ID associates your session with your account for secure tool access - OPENAI_API_KEY to access OpenAI LLMs ```bash COMPOSIO_API_KEY=your_composio_api_key_here USER_ID=your_user_id_here OPENAI_API_KEY=your_openai_api_key ``` ### 4. Import dependencies What's happening: - We load environment variables and import required modules - Composio manages connections to Bench - MCPServerStreamableHTTP connects to the Bench MCP server endpoint - Agent from Pydantic AI lets you define and run the AI assistant ```python import asyncio import os from dotenv import load_dotenv from composio import Composio from pydantic_ai import Agent from pydantic_ai.mcp import MCPServerStreamableHTTP load_dotenv() ``` ### 5. Create a Tool Router Session What's happening: - We're creating a Tool Router session that gives your agent access to Bench tools - The create method takes the user ID and specifies which toolkits should be available - The returned session.mcp.url is the MCP server URL that your agent will use ```python async def main(): api_key = os.getenv("COMPOSIO_API_KEY") user_id = os.getenv("USER_ID") if not api_key or not user_id: raise RuntimeError("Set COMPOSIO_API_KEY and USER_ID in your environment") # Create a Composio Tool Router session for Bench composio = Composio(api_key=api_key) session = composio.create( user_id=user_id, toolkits=["bench"], ) url = session.mcp.url if not url: raise ValueError("Composio session did not return an MCP URL") ``` ### 6. Initialize the Pydantic AI Agent What's happening: - The MCP client connects to the Bench endpoint - The agent uses GPT-5 to interpret user commands and perform Bench operations - The instructions field defines the agent's role and behavior ```python # Attach the MCP server to a Pydantic AI Agent bench_mcp = MCPServerStreamableHTTP(url, headers={"x-api-key": COMPOSIO_API_KEY}) agent = Agent( "openai:gpt-5", toolsets=[bench_mcp], instructions=( "You are a Bench assistant. Use Bench tools to help users " "with their requests. Ask clarifying questions when needed." ), ) ``` ### 7. Build the chat interface What's happening: - The agent reads input from the terminal and streams its response - Bench API calls happen automatically under the hood - The model keeps conversation history to maintain context across turns ```python # Simple REPL with message history history = [] print("Chat started! Type 'exit' or 'quit' to end.\n") print("Try asking the agent to help you with Bench.\n") while True: user_input = input("You: ").strip() if user_input.lower() in {"exit", "quit", "bye"}: print("\nGoodbye!") break if not user_input: continue print("\nAgent is thinking...\n", flush=True) async with agent.run_stream(user_input, message_history=history) as stream_result: collected_text = "" async for chunk in stream_result.stream_output(): text_piece = None if isinstance(chunk, str): text_piece = chunk elif hasattr(chunk, "delta") and isinstance(chunk.delta, str): text_piece = chunk.delta elif hasattr(chunk, "text"): text_piece = chunk.text if text_piece: collected_text += text_piece result = stream_result print(f"Agent: {collected_text}\n") history = result.all_messages() ``` ### 8. Run the application What's happening: - The asyncio loop launches the agent and keeps it running until you exit ```python if __name__ == "__main__": asyncio.run(main()) ``` ## Complete Code ```python import asyncio import os from dotenv import load_dotenv from composio import Composio from pydantic_ai import Agent from pydantic_ai.mcp import MCPServerStreamableHTTP load_dotenv() async def main(): api_key = os.getenv("COMPOSIO_API_KEY") user_id = os.getenv("USER_ID") if not api_key or not user_id: raise RuntimeError("Set COMPOSIO_API_KEY and USER_ID in your environment") # Create a Composio Tool Router session for Bench composio = Composio(api_key=api_key) session = composio.create( user_id=user_id, toolkits=["bench"], ) url = session.mcp.url if not url: raise ValueError("Composio session did not return an MCP URL") # Attach the MCP server to a Pydantic AI Agent bench_mcp = MCPServerStreamableHTTP(url, headers={"x-api-key": COMPOSIO_API_KEY}) agent = Agent( "openai:gpt-5", toolsets=[bench_mcp], instructions=( "You are a Bench assistant. Use Bench tools to help users " "with their requests. Ask clarifying questions when needed." ), ) # Simple REPL with message history history = [] print("Chat started! Type 'exit' or 'quit' to end.\n") print("Try asking the agent to help you with Bench.\n") while True: user_input = input("You: ").strip() if user_input.lower() in {"exit", "quit", "bye"}: print("\nGoodbye!") break if not user_input: continue print("\nAgent is thinking...\n", flush=True) async with agent.run_stream(user_input, message_history=history) as stream_result: collected_text = "" async for chunk in stream_result.stream_output(): text_piece = None if isinstance(chunk, str): text_piece = chunk elif hasattr(chunk, "delta") and isinstance(chunk.delta, str): text_piece = chunk.delta elif hasattr(chunk, "text"): text_piece = chunk.text if text_piece: collected_text += text_piece result = stream_result print(f"Agent: {collected_text}\n") history = result.all_messages() if __name__ == "__main__": asyncio.run(main()) ``` ## Conclusion You've built a Pydantic AI agent that can interact with Bench through Composio's Tool Router. With this setup, your agent can perform real Bench actions through natural language. You can extend this further by: - Adding other toolkits like Gmail, HubSpot, or Salesforce - Building a web-based chat interface around this agent - Using multiple MCP endpoints to enable cross-app workflows (for example, Gmail + Bench for workflow automation) This architecture makes your AI agent "agent-native", able to securely use APIs in a unified, composable way without custom integrations. ## How to build Bench MCP Agent with another framework - [OpenAI Agents SDK](https://composio.dev/toolkits/bench/framework/open-ai-agents-sdk) - [Claude Agent SDK](https://composio.dev/toolkits/bench/framework/claude-agents-sdk) - [Claude Code](https://composio.dev/toolkits/bench/framework/claude-code) - [Claude Cowork](https://composio.dev/toolkits/bench/framework/claude-cowork) - [Codex](https://composio.dev/toolkits/bench/framework/codex) - [OpenClaw](https://composio.dev/toolkits/bench/framework/openclaw) - [Hermes](https://composio.dev/toolkits/bench/framework/hermes-agent) - [CLI](https://composio.dev/toolkits/bench/framework/cli) - [Google ADK](https://composio.dev/toolkits/bench/framework/google-adk) - [LangChain](https://composio.dev/toolkits/bench/framework/langchain) - [Vercel AI SDK](https://composio.dev/toolkits/bench/framework/ai-sdk) - [Mastra AI](https://composio.dev/toolkits/bench/framework/mastra-ai) - [LlamaIndex](https://composio.dev/toolkits/bench/framework/llama-index) - [CrewAI](https://composio.dev/toolkits/bench/framework/crew-ai) ## Related Toolkits - [Supabase](https://composio.dev/toolkits/supabase) - Supabase is an open-source backend platform offering scalable Postgres databases, authentication, storage, and real-time APIs. It lets developers build modern apps without managing infrastructure. - [Codeinterpreter](https://composio.dev/toolkits/codeinterpreter) - Codeinterpreter is a Python-based coding environment with built-in data analysis and visualization. It lets you instantly run scripts, plot results, and prototype solutions inside supported platforms. - [GitHub](https://composio.dev/toolkits/github) - GitHub is a code hosting platform for version control and collaborative software development. It streamlines project management, code review, and team workflows in one place. - [Ably](https://composio.dev/toolkits/ably) - Ably is a real-time messaging platform for live chat and data sync in modern apps. It offers global scale and rock-solid reliability for seamless, instant experiences. - [Abuselpdb](https://composio.dev/toolkits/abuselpdb) - Abuselpdb is a central database for reporting and checking IPs linked to malicious online activity. Use it to quickly identify and report suspicious or abusive IP addresses. - [Alchemy](https://composio.dev/toolkits/alchemy) - Alchemy is a blockchain development platform offering APIs and tools for Ethereum apps. It simplifies building and scaling Web3 projects with robust infrastructure. - [Algolia](https://composio.dev/toolkits/algolia) - Algolia is a hosted search API that powers lightning-fast, relevant search experiences for web and mobile apps. It helps developers deliver instant, typo-tolerant, and scalable search without complex infrastructure. - [Anchor browser](https://composio.dev/toolkits/anchor_browser) - Anchor browser is a developer platform for AI-powered web automation. It transforms complex browser actions into easy API endpoints for streamlined web interaction. - [Apiflash](https://composio.dev/toolkits/apiflash) - Apiflash is a website screenshot API for programmatically capturing web pages. It delivers high-quality screenshots on demand for automation, monitoring, or reporting. - [Apiverve](https://composio.dev/toolkits/apiverve) - Apiverve delivers a suite of powerful APIs that simplify integration for developers. It's designed for reliability and scalability so you can build faster, smarter applications without the integration headache. - [Appcircle](https://composio.dev/toolkits/appcircle) - Appcircle is an enterprise-grade mobile CI/CD platform for building, testing, and publishing mobile apps. It streamlines mobile DevOps so teams ship faster and with more confidence. - [Appdrag](https://composio.dev/toolkits/appdrag) - Appdrag is a cloud platform for building websites, APIs, and databases with drag-and-drop tools and code editing. It accelerates development and iteration by combining hosting, database management, and low-code features in one place. - [Appveyor](https://composio.dev/toolkits/appveyor) - AppVeyor is a cloud-based continuous integration service for building, testing, and deploying applications. It helps developers automate and streamline their software delivery pipelines. - [Backendless](https://composio.dev/toolkits/backendless) - Backendless is a backend-as-a-service platform for mobile and web apps, offering database, file storage, user authentication, and APIs. It helps developers ship scalable applications faster without managing server infrastructure. - [Baserow](https://composio.dev/toolkits/baserow) - Baserow is an open-source no-code database platform for building collaborative data apps. It makes it easy for teams to organize data and automate workflows without writing code. - [Better stack](https://composio.dev/toolkits/better_stack) - Better Stack is a monitoring, logging, and incident management solution for apps and services. It helps teams ensure application reliability and performance with real-time insights. - [Bitbucket](https://composio.dev/toolkits/bitbucket) - Bitbucket is a Git-based code hosting and collaboration platform for teams. It enables secure repository management and streamlined code reviews. - [Blazemeter](https://composio.dev/toolkits/blazemeter) - Blazemeter is a continuous testing platform for web and mobile app performance. It empowers teams to automate and analyze large-scale tests with ease. - [Blocknative](https://composio.dev/toolkits/blocknative) - Blocknative delivers real-time mempool monitoring and transaction management for public blockchains. Instantly track pending transactions and optimize blockchain interactions with live data. - [Bolt iot](https://composio.dev/toolkits/bolt_iot) - Bolt IoT is a platform for building and managing IoT projects with cloud-based device control and monitoring. It makes connecting sensors and actuators to the internet seamless for automation and data insights. ## Frequently Asked Questions ### What are the differences in Tool Router MCP and Bench MCP? With a standalone Bench MCP server, the agents and LLMs can only access a fixed set of Bench tools tied to that server. However, with the Composio Tool Router, agents can dynamically load tools from Bench and many other apps based on the task at hand, all through a single MCP endpoint. ### Can I use Tool Router MCP with Pydantic AI? Yes, you can. Pydantic AI fully supports MCP integration. You get structured tool calling, message history handling, and model orchestration while Tool Router takes care of discovering and serving the right Bench tools. ### Can I manage the permissions and scopes for Bench while using Tool Router? Yes, absolutely. You can configure which Bench scopes and actions are allowed when connecting your account to Composio. You can also bring your own OAuth credentials or API configuration so you keep full control over what the agent can do. ### How safe is my data with Composio Tool Router? All sensitive data such as tokens, keys, and configuration is fully encrypted at rest and in transit. Composio is SOC 2 Type 2 compliant and follows strict security practices so your Bench data and credentials are handled as safely as possible. --- [See all toolkits](https://composio.dev/toolkits) ยท [Composio docs](https://docs.composio.dev/llms.txt)