# How to integrate Browseai MCP with Autogen ```json { "title": "How to integrate Browseai MCP with Autogen", "toolkit": "Browseai", "toolkit_slug": "browseai", "framework": "AutoGen", "framework_slug": "autogen", "url": "https://composio.dev/toolkits/browseai/framework/autogen", "markdown_url": "https://composio.dev/toolkits/browseai/framework/autogen.md", "updated_at": "2026-05-12T10:04:12.979Z" } ``` ## Introduction This guide walks you through connecting Browseai to AutoGen using the Composio tool router. By the end, you'll have a working Browseai agent that can extract product prices from competitor websites, monitor job postings for new remote roles, get detailed status of last robot run through natural language commands. This guide will help you understand how to give your AutoGen agent real control over a Browseai account through Composio's Browseai MCP server. Before we dive in, let's take a quick look at the key ideas and tools involved. ## Also integrate Browseai with - [OpenAI Agents SDK](https://composio.dev/toolkits/browseai/framework/open-ai-agents-sdk) - [Claude Agent SDK](https://composio.dev/toolkits/browseai/framework/claude-agents-sdk) - [Claude Code](https://composio.dev/toolkits/browseai/framework/claude-code) - [Claude Cowork](https://composio.dev/toolkits/browseai/framework/claude-cowork) - [Codex](https://composio.dev/toolkits/browseai/framework/codex) - [OpenClaw](https://composio.dev/toolkits/browseai/framework/openclaw) - [Hermes](https://composio.dev/toolkits/browseai/framework/hermes-agent) - [CLI](https://composio.dev/toolkits/browseai/framework/cli) - [Google ADK](https://composio.dev/toolkits/browseai/framework/google-adk) - [LangChain](https://composio.dev/toolkits/browseai/framework/langchain) - [Vercel AI SDK](https://composio.dev/toolkits/browseai/framework/ai-sdk) - [Mastra AI](https://composio.dev/toolkits/browseai/framework/mastra-ai) - [LlamaIndex](https://composio.dev/toolkits/browseai/framework/llama-index) - [CrewAI](https://composio.dev/toolkits/browseai/framework/crew-ai) ## TL;DR Here's what you'll learn: - Get and set up your OpenAI and Composio API keys - Install the required dependencies for Autogen and Composio - Initialize Composio and create a Tool Router session for Browseai - Wire that MCP URL into Autogen using McpWorkbench and StreamableHttpServerParams - Configure an Autogen AssistantAgent that can call Browseai tools - Run a live chat loop where you ask the agent to perform Browseai operations ## What is AutoGen? Autogen is a framework for building multi-agent conversational AI systems from Microsoft. It enables you to create agents that can collaborate, use tools, and maintain complex workflows. Key features include: - Multi-Agent Systems: Build collaborative agent workflows - MCP Workbench: Native support for Model Context Protocol tools - Streaming HTTP: Connect to external services through streamable HTTP - AssistantAgent: Pre-built agent class for tool-using assistants ## What is the Browseai MCP server, and what's possible with it? The Browseai MCP server is an implementation of the Model Context Protocol that connects your AI agent and assistants like Claude, Cursor, etc directly to your Browseai account. It provides structured and secure access to your robots, tasks, and monitoring workflows, so your agent can perform actions like running robots, extracting website data, monitoring for changes, and managing web automation tasks on your behalf. - On-demand web data extraction: Trigger your Browseai robots to collect data from any website instantly, using custom input parameters for targeted web scraping. - Automated website monitoring: Set up and manage recurring monitors that watch websites for changes, so your agent can alert you to updates or new information as soon as they happen. - Bulk task execution: Launch thousands of data extraction tasks at once for large-scale or repeated scraping jobs, streamlining high-volume automation for your workflows. - Task and robot management: Retrieve lists of your robots, fetch detailed results for specific tasks, check execution history, and delete obsolete tasks or monitors to keep your automation setup organized. - Real-time notifications and workflow integration: Configure webhooks so your agent receives instant updates when tasks complete, fail, or when website changes are detected, enabling seamless integration with your broader systems. ## Supported Tools | Tool slug | Name | Description | |---|---|---| | `BROWSEAI_BULK_RUN_TASKS` | Bulk Run Tasks | This action allows users to bulk run up to 1,000 tasks per API call using a specified robot. For larger datasets, submit multiple bulk runs sequentially (up to 500,000 tasks total). It provides a POST endpoint at /v2/robots/{robotId}/bulk-runs and supports parameters such as robot_id (required), title (required), and input_parameters (required). This bulk operation is essential for large-scale data extraction. | | `BROWSEAI_CREATE_MONITOR` | Create Monitor | Creates a new monitor for a Browse AI robot to automatically track website changes over time. A monitor runs a robot on a recurring schedule and can notify you when the captured data (screenshots or text) changes. This is useful for tracking price changes, content updates, availability status, or any other dynamic website information. You must first have a robot created (via the Browse AI dashboard) before you can monitor it. Use the GET_ROBOTS action to find available robot IDs. Key features: - Configure recurring schedules (hourly, daily, weekly) using RRULE format - Get email notifications when screenshots or text content changes - Customize notification sensitivity with threshold settings - Override robot parameters for each monitor | | `BROWSEAI_CREATE_WEBHOOK` | Create Webhook | This tool creates a new webhook for a Browse AI robot. Webhooks are HTTP callbacks that Browse AI sends to your server immediately when specific events occur, eliminating the need for polling. It is useful for: - Setting up automated notifications for task completion - Receiving real-time updates when changes are detected - Integrating Browse AI with your own systems - Automating workflows based on robot task results The webhook can be configured to trigger on different event types: - taskFinished: Triggers on any task completion (success or failure) - taskFinishedSuccessfully: Triggers only on successful task completions - taskFinishedWithError: Triggers only on failed task completions - taskCapturedDataChanged: Triggers when data changes are detected during monitoring - tableExportFinishedSuccessfully: Triggers when table export completes (Beta feature) Note: Browse AI only supports one event type per webhook. To monitor multiple event types, create separate webhooks for each event type. | | `BROWSEAI_DELETE_MONITOR` | Delete a specific monitor | This tool allows users to delete a specific monitor from their Browse AI account. It uses the DELETE method and requires a valid monitor_id. | | `BROWSEAI_DELETE_TASK` | Delete a specific task | This tool allows you to delete a specific task in BrowseAI by its task ID. It is used for cleaning up completed or failed tasks, managing resources, and maintaining your task list. | | `BROWSEAI_GET_ROBOTS` | Get Robots List | Retrieves a list of all robots (automated web tasks) under your Browse AI account. A robot is an automated browser task that can be trained to perform web operations such as: - Opening webpages and navigating through sites - Logging into websites - Clicking buttons and filling forms - Extracting structured data from web pages - Monitoring websites for changes This action returns comprehensive information about each robot including: - Robot ID (required for running tasks via other API endpoints) - Robot name and timestamps (creation/update) - Input parameters the robot accepts (parameter names, types, and whether they're required) Use this to: - Discover available robots and their IDs for task execution - Check robot configuration and required input parameters - Monitor robot creation and update activity - Get an overview of all automated tasks in your account No parameters required - simply call this action to retrieve all robots. | | `BROWSEAI_GET_ROBOT_TASKS` | Get Robot Tasks | Retrieves a paginated list of all tasks (execution runs) for a specific Browse AI robot. Each task represents one execution of the robot, which includes its current status, execution timestamps, input parameters, captured text data, screenshots, and extracted lists. This tool returns tasks in reverse chronological order (newest first) by default. Use the limit and offset parameters to paginate through large result sets. Use this tool to: - Check the status of recent robot executions (pending, running, successful, failed) - Access data captured by robot tasks (text, screenshots, lists) - Monitor robot performance and execution history - Retrieve task IDs for use with other task-related actions - Debug failed robot executions Note: You must have at least one robot configured in your Browse AI account to use this action. Use the Get Robots List action first to find available robot IDs. | | `BROWSEAI_GET_SYSTEM_STATUS` | Get System Status | Tool to check the operational status of Browse AI infrastructure, including the tasks queue condition. Use when you need to verify if Browse AI services are operational before running robots or tasks. | | `BROWSEAI_GET_TASK_DETAILS` | Get Task Details | Retrieves detailed information about a specific Browse AI task by robot ID and task ID. This tool returns comprehensive task details including execution status, captured data, screenshots, input parameters, timestamps (created, started, completed), and error information if the task failed. It provides an in-depth view of individual task execution for monitoring, debugging, and data retrieval purposes. Use this tool when you need to: - Check the current status of a task (running, successful, failed) - Retrieve data captured during task execution - Access screenshots taken during the task - Review input parameters used for the task - Debug task failures by examining error details - Monitor task execution timestamps | | `BROWSEAI_LIST_ROBOT_WEBHOOKS` | List Robot Webhooks | Tool to retrieve all webhooks configured for a specific Browse AI robot. Use when you need to view, audit, or manage webhook configurations for a robot. Returns webhook details including URL, event type, active status, and creation timestamp. | | `BROWSEAI_RUN_ROBOT` | Run Robot | Triggers the execution of a Browse.ai robot to extract data from websites. This action starts a new robot task that will scrape/extract data according to the robot's pre-configured settings. The robot runs asynchronously and returns a task ID immediately. To retrieve the extracted data, use BROWSEAI_GET_TASK_DETAILS with the returned task ID. Prerequisites: - You must have at least one robot configured in your Browse.ai account - Obtain robot_id using BROWSEAI_GET_ROBOTS action first - Know what input parameters your robot expects (configured during robot creation) Common use cases: - Scraping product data from e-commerce sites - Monitoring website changes for specific content - Extracting structured data from multiple web pages - Automating repetitive web data collection tasks Note: The robot executes asynchronously. Check task status using BROWSEAI_GET_TASK_DETAILS or BROWSEAI_GET_ROBOT_TASKS to retrieve results when the task completes. | ## Supported Triggers None listed. ## Creating MCP Server - Stand-alone vs Composio SDK The Browseai MCP server is an implementation of the Model Context Protocol that connects your AI agents and assistants directly to Browseai. Instead of manually wiring Browseai APIs, OAuth, and scopes yourself, you get a structured, tool-based interface that an LLM can call safely. With Composio's managed implementation, you don't have to create your own developer app. For production, if you're building an end product, we recommend using your own credentials. The managed server helps you prototype fast and go from 0-1 faster. ## Step-by-step Guide ### 1. Prerequisites You will need: - A Composio API key - An OpenAI API key (used by Autogen's OpenAIChatCompletionClient) - A Browseai account you can connect to Composio - Some basic familiarity with Autogen and Python async ### 1. Getting API Keys for OpenAI and Composio OpenAI API Key - Go to the [OpenAI dashboard](https://platform.openai.com/settings/organization/api-keys) and create an API key. You'll need credits to use the models, or you can connect to another model provider. - Keep the API key safe. Composio API Key - Log in to the [Composio dashboard](https://dashboard.composio.dev?utm_source=toolkits&utm_medium=framework_docs). - Navigate to your API settings and generate a new API key. - Store this key securely as you'll need it for authentication. ### 2. Install dependencies Install Composio, Autogen extensions, and dotenv. What's happening: - composio connects your agent to Browseai via MCP - autogen-agentchat provides the AssistantAgent class - autogen-ext-openai provides the OpenAI model client - autogen-ext-tools provides MCP workbench support ```bash pip install composio python-dotenv pip install autogen-agentchat autogen-ext-openai autogen-ext-tools ``` ### 3. Set up environment variables Create a .env file in your project folder. What's happening: - COMPOSIO_API_KEY is required to talk to Composio - OPENAI_API_KEY is used by Autogen's OpenAI client - USER_ID is how Composio identifies which user's Browseai connections to use ```bash COMPOSIO_API_KEY=your-composio-api-key OPENAI_API_KEY=your-openai-api-key USER_ID=your-user-identifier@example.com ``` ### 4. Import dependencies and create Tool Router session What's happening: - load_dotenv() reads your .env file - Composio(api_key=...) initializes the SDK - create(...) creates a Tool Router session that exposes Browseai tools - session.mcp.url is the MCP endpoint that Autogen will connect to ```python import asyncio import os from dotenv import load_dotenv from composio import Composio from autogen_agentchat.agents import AssistantAgent from autogen_ext.models.openai import OpenAIChatCompletionClient from autogen_ext.tools.mcp import McpWorkbench, StreamableHttpServerParams load_dotenv() async def main(): # Initialize Composio and create a Browseai session composio = Composio(api_key=os.getenv("COMPOSIO_API_KEY")) session = composio.create( user_id=os.getenv("USER_ID"), toolkits=["browseai"] ) url = session.mcp.url ``` ### 5. Configure MCP parameters for Autogen Autogen expects parameters describing how to talk to the MCP server. That is what StreamableHttpServerParams is for. What's happening: - url points to the Tool Router MCP endpoint from Composio - timeout is the HTTP timeout for requests - sse_read_timeout controls how long to wait when streaming responses - terminate_on_close=True cleans up the MCP server process when the workbench is closed ```python # Configure MCP server parameters for Streamable HTTP server_params = StreamableHttpServerParams( url=url, timeout=30.0, sse_read_timeout=300.0, terminate_on_close=True, headers={"x-api-key": os.getenv("COMPOSIO_API_KEY")} ) ``` ### 6. Create the model client and agent What's happening: - OpenAIChatCompletionClient wraps the OpenAI model for Autogen - McpWorkbench connects the agent to the MCP tools - AssistantAgent is configured with the Browseai tools from the workbench ```python # Create model client model_client = OpenAIChatCompletionClient( model="gpt-5", api_key=os.getenv("OPENAI_API_KEY") ) # Use McpWorkbench as context manager async with McpWorkbench(server_params) as workbench: # Create Browseai assistant agent with MCP tools agent = AssistantAgent( name="browseai_assistant", description="An AI assistant that helps with Browseai operations.", model_client=model_client, workbench=workbench, model_client_stream=True, max_tool_iterations=10 ) ``` ### 7. Run the interactive chat loop What's happening: - The script prompts you in a loop with You: - Autogen passes your input to the model, which decides which Browseai tools to call via MCP - agent.run_stream(...) yields streaming messages as the agent thinks and calls tools - Typing exit, quit, or bye ends the loop ```python print("Chat started! Type 'exit' or 'quit' to end the conversation.\n") print("Ask any Browseai related question or task to the agent.\n") # Conversation loop while True: user_input = input("You: ").strip() if user_input.lower() in ["exit", "quit", "bye"]: print("\nGoodbye!") break if not user_input: continue print("\nAgent is thinking...\n") # Run the agent with streaming try: response_text = "" async for message in agent.run_stream(task=user_input): if hasattr(message, "content") and message.content: response_text = message.content # Print the final response if response_text: print(f"Agent: {response_text}\n") else: print("Agent: I encountered an issue processing your request.\n") except Exception as e: print(f"Agent: Sorry, I encountered an error: {str(e)}\n") ``` ## Complete Code ```python import asyncio import os from dotenv import load_dotenv from composio import Composio from autogen_agentchat.agents import AssistantAgent from autogen_ext.models.openai import OpenAIChatCompletionClient from autogen_ext.tools.mcp import McpWorkbench, StreamableHttpServerParams load_dotenv() async def main(): # Initialize Composio and create a Browseai session composio = Composio(api_key=os.getenv("COMPOSIO_API_KEY")) session = composio.create( user_id=os.getenv("USER_ID"), toolkits=["browseai"] ) url = session.mcp.url # Configure MCP server parameters for Streamable HTTP server_params = StreamableHttpServerParams( url=url, timeout=30.0, sse_read_timeout=300.0, terminate_on_close=True, headers={"x-api-key": os.getenv("COMPOSIO_API_KEY")} ) # Create model client model_client = OpenAIChatCompletionClient( model="gpt-5", api_key=os.getenv("OPENAI_API_KEY") ) # Use McpWorkbench as context manager async with McpWorkbench(server_params) as workbench: # Create Browseai assistant agent with MCP tools agent = AssistantAgent( name="browseai_assistant", description="An AI assistant that helps with Browseai operations.", model_client=model_client, workbench=workbench, model_client_stream=True, max_tool_iterations=10 ) print("Chat started! Type 'exit' or 'quit' to end the conversation.\n") print("Ask any Browseai related question or task to the agent.\n") # Conversation loop while True: user_input = input("You: ").strip() if user_input.lower() in ['exit', 'quit', 'bye']: print("\nGoodbye!") break if not user_input: continue print("\nAgent is thinking...\n") # Run the agent with streaming try: response_text = "" async for message in agent.run_stream(task=user_input): if hasattr(message, 'content') and message.content: response_text = message.content # Print the final response if response_text: print(f"Agent: {response_text}\n") else: print("Agent: I encountered an issue processing your request.\n") except Exception as e: print(f"Agent: Sorry, I encountered an error: {str(e)}\n") if __name__ == "__main__": asyncio.run(main()) ``` ## Conclusion You now have an Autogen assistant wired into Browseai through Composio's Tool Router and MCP. From here you can: - Add more toolkits to the toolkits list, for example notion or hubspot - Refine the agent description to point it at specific workflows - Wrap this script behind a UI, Slack bot, or internal tool Once the pattern is clear for Browseai, you can reuse the same structure for other MCP-enabled apps with minimal code changes. ## How to build Browseai MCP Agent with another framework - [OpenAI Agents SDK](https://composio.dev/toolkits/browseai/framework/open-ai-agents-sdk) - [Claude Agent SDK](https://composio.dev/toolkits/browseai/framework/claude-agents-sdk) - [Claude Code](https://composio.dev/toolkits/browseai/framework/claude-code) - [Claude Cowork](https://composio.dev/toolkits/browseai/framework/claude-cowork) - [Codex](https://composio.dev/toolkits/browseai/framework/codex) - [OpenClaw](https://composio.dev/toolkits/browseai/framework/openclaw) - [Hermes](https://composio.dev/toolkits/browseai/framework/hermes-agent) - [CLI](https://composio.dev/toolkits/browseai/framework/cli) - [Google ADK](https://composio.dev/toolkits/browseai/framework/google-adk) - [LangChain](https://composio.dev/toolkits/browseai/framework/langchain) - [Vercel AI SDK](https://composio.dev/toolkits/browseai/framework/ai-sdk) - [Mastra AI](https://composio.dev/toolkits/browseai/framework/mastra-ai) - [LlamaIndex](https://composio.dev/toolkits/browseai/framework/llama-index) - [CrewAI](https://composio.dev/toolkits/browseai/framework/crew-ai) ## Related Toolkits - [Firecrawl](https://composio.dev/toolkits/firecrawl) - Firecrawl automates large-scale web crawling and data extraction. It helps organizations efficiently gather, index, and analyze content from online sources. - [Tavily](https://composio.dev/toolkits/tavily) - Tavily offers powerful search and data retrieval from documents, databases, and the web. It helps teams locate and filter information instantly, saving hours on research. - [Exa](https://composio.dev/toolkits/exa) - Exa is a data extraction and search platform for gathering and analyzing information from websites, APIs, or databases. It helps teams quickly surface insights and automate data-driven workflows. - [Serpapi](https://composio.dev/toolkits/serpapi) - SerpApi is a real-time API for structured search engine results. It lets you automate SERP data collection, parsing, and analysis for SEO and research. - [Peopledatalabs](https://composio.dev/toolkits/peopledatalabs) - Peopledatalabs delivers B2B data enrichment and identity resolution APIs. Supercharge your apps with accurate, up-to-date business and contact data. - [Snowflake](https://composio.dev/toolkits/snowflake) - Snowflake is a cloud data warehouse built for elastic scaling, secure data sharing, and fast SQL analytics across major clouds. - [Posthog](https://composio.dev/toolkits/posthog) - PostHog is an open-source analytics platform for tracking user interactions and product metrics. It helps teams refine features, analyze funnels, and reduce churn with actionable insights. - [Amplitude](https://composio.dev/toolkits/amplitude) - Amplitude is a digital analytics platform for product and behavioral data insights. It helps teams analyze user journeys and make data-driven decisions quickly. - [Bright Data MCP](https://composio.dev/toolkits/brightdata_mcp) - Bright Data MCP is an AI-powered web scraping and data collection platform. Instantly access public web data in real time with advanced scraping tools. - [ClickHouse](https://composio.dev/toolkits/clickhouse) - ClickHouse is an open-source, column-oriented database for real-time analytics and big data processing using SQL. Its lightning-fast query performance makes it ideal for handling large datasets and delivering instant insights. - [Coinmarketcal](https://composio.dev/toolkits/coinmarketcal) - CoinMarketCal is a community-powered crypto calendar for upcoming events, announcements, and releases. It helps traders track market-moving developments and stay ahead in the crypto space. - [Control d](https://composio.dev/toolkits/control_d) - Control d is a customizable DNS filtering and traffic redirection platform. It helps you manage internet access, enforce policies, and monitor usage across devices and networks. - [Databox](https://composio.dev/toolkits/databox) - Databox is a business analytics platform that connects your data from any tool and device. It helps you track KPIs, build dashboards, and discover actionable insights. - [Databricks](https://composio.dev/toolkits/databricks) - Databricks is a unified analytics platform for big data and AI on the lakehouse architecture. It empowers data teams to collaborate, analyze, and build scalable solutions efficiently. - [Datagma](https://composio.dev/toolkits/datagma) - Datagma delivers data intelligence and analytics for business growth and market discovery. Get actionable market insights and track competitors to inform your strategy. - [Delighted](https://composio.dev/toolkits/delighted) - Delighted is a customer feedback platform based on the Net Promoter System®. It helps you quickly gather, track, and act on customer sentiment. - [Dovetail](https://composio.dev/toolkits/dovetail) - Dovetail is a research analysis platform for transcript review and insight generation. It helps teams code interviews, analyze feedback, and create actionable research summaries. - [Dub](https://composio.dev/toolkits/dub) - Dub is a short link management platform with analytics and API access. Use it to easily create, manage, and track branded short links for your business. - [Elasticsearch](https://composio.dev/toolkits/elasticsearch) - Elasticsearch is a distributed, RESTful search and analytics engine for all types of data. It delivers fast, scalable search and powerful analytics across massive datasets. - [Fireflies](https://composio.dev/toolkits/fireflies) - Fireflies.ai is an AI-powered meeting assistant that records, transcribes, and analyzes voice conversations. It helps teams capture call notes automatically and search or summarize meetings effortlessly. ## Frequently Asked Questions ### What are the differences in Tool Router MCP and Browseai MCP? With a standalone Browseai MCP server, the agents and LLMs can only access a fixed set of Browseai tools tied to that server. However, with the Composio Tool Router, agents can dynamically load tools from Browseai and many other apps based on the task at hand, all through a single MCP endpoint. ### Can I use Tool Router MCP with Autogen? Yes, you can. Autogen fully supports MCP integration. You get structured tool calling, message history handling, and model orchestration while Tool Router takes care of discovering and serving the right Browseai tools. ### Can I manage the permissions and scopes for Browseai while using Tool Router? Yes, absolutely. You can configure which Browseai scopes and actions are allowed when connecting your account to Composio. You can also bring your own OAuth credentials or API configuration so you keep full control over what the agent can do. ### How safe is my data with Composio Tool Router? All sensitive data such as tokens, keys, and configuration is fully encrypted at rest and in transit. Composio is SOC 2 Type 2 compliant and follows strict security practices so your Browseai data and credentials are handled as safely as possible. --- [See all toolkits](https://composio.dev/toolkits) · [Composio docs](https://docs.composio.dev/llms.txt)