Brightdata Integration for AI Agents

Securely connect your AI agents and chatbots (Claude, ChatGPT, Cursor, etc) with Brightdata MCP or direct API to scrape websites, extract SERP results, bypass anti-bot protection, and automate web data collection through natural language.
BOOK A CALLGET STARTED FOR FREE
Brightdata Logo
Gradient Top
Gradient Middle
Gradient Bottom
divider

Supported Tools

Tools
Trigger Site CrawlTool to trigger a site crawl job to extract content across multiple pages or entire domains.
Browse Available ScrapersTool to list all available pre-made scrapers (datasets) from bright data's marketplace.
Filter DatasetTool to apply custom filter criteria to a marketplace dataset (beta).
Get Available CitiesTool to get available static network cities for a given country.
Get Available CountriesTool to list available countries and their iso 3166-1 alpha-2 codes.
Download Scraped DataTool to retrieve the scraped data from a completed crawl job by snapshot id.
Check Crawl StatusTool to check the processing status of a crawl job using snapshot id.
List Unlocker ZonesTool to list your configured web unlocker zones and proxy endpoints.
SERP SearchTool to perform serp (search engine results page) searches across different search engines.
Web UnlockerTool to bypass bot detection, captcha, and other anti-scraping measures to extract content from websites.

Connect Brightdata MCP Tool with your Agent

Python
TypeScript

Install Composio

python
pip install composio claude-agent-sdk
Install the Composio SDK and Claude Agent SDK

Create Tool Router Session

python
from composio import Composio
from claude_agent_sdk import ClaudeSDKClient, ClaudeAgentOptions

composio = Composio(api_key='your-composio-api-key')
session = composio.create(user_id='your-user-id')
url = session.mcp.url
Initialize the Composio client and create a Tool Router session

Connect to AI Agent

python
import asyncio

options = ClaudeAgentOptions(
    permission_mode='bypassPermissions',
    mcp_servers={
        'tool_router': {
            'type': 'http',
            'url': url,
            'headers': {
                'x-api-key': 'your-composio-api-key'
            }
        }
    },
    system_prompt='You are a helpful assistant with access to Brightdata tools.',
    max_turns=10
)

async def main():
    async with ClaudeSDKClient(options=options) as client:
        await client.query('Trigger a site crawl for https://example.com and download the scraped data')
        async for message in client.receive_response():
            if hasattr(message, 'content'):
                for block in message.content:
                    if hasattr(block, 'text'):
                        print(block.text)

asyncio.run(main())
Use the MCP server with your AI agent

Connect Brightdata API Tool with your Agent

Python
TypeScript

Install Composio

python
pip install composio_openai
Install the Composio SDK

Initialize Composio and Create Tool Router Session

python
from openai import OpenAI
from composio import Composio
from composio_openai import OpenAIResponsesProvider

composio = Composio(provider=OpenAIResponsesProvider())
openai = OpenAI()
session = composio.create(user_id='your-user-id')
Import and initialize Composio client, then create a Tool Router session

Execute Brightdata Tools via Tool Router with Your Agent

python
tools = session.tools
response = openai.responses.create(
  model='gpt-4.1',
  tools=tools,
  input=[{
    'role': 'user',
    'content': 'Trigger a crawl to extract all product listings from example.com'
  }]
)
result = composio.provider.handle_tool_calls(
  response=response,
  user_id='your-user-id'
)
print(result)
Get tools from Tool Router session and execute Brightdata actions with your Agent

Why Use Composio?

AI Native Brightdata Integration

  • Supports both Brightdata MCP and direct API based integrations
  • Structured, LLM-friendly schemas for reliable tool execution
  • Rich coverage for reading, writing, and querying your Brightdata data

Managed Auth

  • Built-in API key management and automatic rotation
  • Central place to manage, scope, and revoke Brightdata access
  • Per user and per environment credentials instead of hard-coded keys

Agent Optimized Design

  • Tools are tuned using real error and success rates to improve reliability over time
  • Comprehensive execution logs so you always know what ran, when, and on whose behalf

Enterprise Grade Security

  • Fine-grained RBAC so you control which agents and users can access Brightdata
  • Scoped, least privilege access to Brightdata resources
  • Full audit trail of agent actions to support review and compliance

Use Brightdata with any AI Agent Framework

Choose a Framework you want to connect Brightdata with

Frequently Asked Questions

Do I need my own developer credentials to use Brightdata with Composio?

Yes, Brightdata requires you to configure your own API key credentials. Once set up, Composio handles secure credential storage and API request handling for you.

Can I use multiple toolkits together?

Yes! Composio's Tool Router enables agents to use multiple toolkits. Learn more.

Is Composio secure?

Composio is SOC 2 and ISO 27001 compliant with all data encrypted in transit and at rest. Learn more.

What if the API changes?

Composio maintains and updates all toolkit integrations automatically, so your agents always work with the latest API versions.

Used by agents from

Context
ASU
Letta
glean
HubSpot
Agent.ai
Altera
DataStax
Entelligence
Rolai
Context
ASU
Letta
glean
HubSpot
Agent.ai
Altera
DataStax
Entelligence
Rolai
Context
ASU
Letta
glean
HubSpot
Agent.ai
Altera
DataStax
Entelligence
Rolai

Never worry about agent reliability

We handle tool reliability, observability, and security so you never have to second-guess an agent action.

Book a call↗Get started for free