# How to connect Crawlbase to Claude

```json
{
  "title": "How to connect Crawlbase to Claude",
  "toolkit": "Crawlbase",
  "toolkit_slug": "crawlbase",
  "framework": "Claude Cowork",
  "framework_slug": "claude-cowork",
  "url": "https://composio.dev/toolkits/crawlbase/framework/claude-cowork",
  "markdown_url": "https://composio.dev/toolkits/crawlbase/framework/claude-cowork.md",
  "updated_at": "2026-11-06T09:05:03.903Z"
}
```

## Introduction

Claude is Anthropic's AI assistant, available on the web, in the desktop app, and on mobile. Cowork is its agent for knowledge work, and it runs on all of those surfaces too. Connected to your apps, Claude can work with your files and services to accomplish complex tasks on your behalf.
This guide walks you through the easiest and most secure way to connect your Crawlbase account to Claude via Composio Connect, enabling it to crawl product pages for pricing changes, extract headlines from competitor blogs, check crawl queue status today, and more such actions on your behalf without compromising your account security.
Setup is the same on Claude Web, Desktop, and Cowork, and you only need to do it once. The connector is tied to your account, so it's available in all three.

## Also integrate Crawlbase with

- [ChatGPT](https://composio.dev/toolkits/crawlbase/framework/chatgpt)
- [Grok Bot](https://composio.dev/toolkits/crawlbase/framework/grok-bot)
- [Hermes](https://composio.dev/toolkits/crawlbase/framework/hermes-agent)
- [Atomic Agent](https://composio.dev/toolkits/crawlbase/framework/atomic-agent)

## Connect Crawlbase to Claude Cowork

### Connecting Crawlbase to Claude Web, Desktop, and Cowork
Two ways to add the Composio connector. Pick whichever you prefer.
### 1. Add with one click
Click the button. Claude opens with the Composio connector filled in. Click Add, then go to step 2.
[Add to Claude](https://claude.ai/customize/connectors?modal=add-custom-connector&connectorName=Composio&connectorUrl=https%3A%2F%2Fconnect.composio.dev%2Fmcp)
### Add manually
In Claude Desktop, click Customize in the left sidebar, then select Connectors and click the + icon at the top.
Click Add custom connector and paste in the Composio MCP server URL:

```bash
https://connect.composio.dev/mcp
```

## What is the Crawlbase MCP server, and what's possible with it?

The Crawlbase MCP server is an implementation of the Model Context Protocol that connects your AI agent and assistants like Claude, Cursor, etc directly to your Crawlbase account. It provides structured and secure access to web crawling, cloud storage, account usage, and user agent generation, so your agent can crawl public pages, retrieve stored content, delete saved pages, monitor usage, and generate user agents on your behalf.
- Public web page crawling: Have your agent fetch content from any public HTTP or HTTPS page and review whether the crawl succeeded.
- Stored page retrieval: Let the agent find and retrieve individual or grouped pages saved in Crawlbase Cloud Storage.
- Stored content cleanup: Direct your agent to permanently delete selected pages from cloud storage and confirm each deletion outcome.
- Usage and storage monitoring: Instruct your agent to check current crawling usage, review previous month statistics, and count stored pages.
- User agent generation: Have your agent generate up to ten realistic randomized user agent strings for a selected device class.

## Supported Tools

| Tool slug | Name | Description |
|---|---|---|
| `CRAWLBASE_CRAWL_URL` | Crawl URL | Fetch a public HTTP or HTTPS URL through Crawlbase using the connected Normal token and return its content with separate target and Crawlbase statuses. |
| `CRAWLBASE_DELETE_STORED_PAGES` | Delete Stored Pages | Irreversibly delete explicitly selected pages from Crawlbase Cloud Storage by request ID and return each page's deletion outcome. |
| `CRAWLBASE_GENERATE_USER_AGENTS` | Generate User Agents | Generate one to ten realistic randomized User-Agent strings for a selected device class. |
| `CRAWLBASE_GET_CRAWLING_USAGE` | Get Crawling Usage | Return current Crawling API usage and optionally previous-month statistics for the connected Crawlbase account. |
| `CRAWLBASE_GET_STORAGE_COUNT` | Get Storage Count | Return the number of pages currently held in Crawlbase Cloud Storage. |
| `CRAWLBASE_GET_STORED_PAGE` | Get Stored Page | Retrieve the latest Crawlbase Cloud Storage page by exactly one request ID or original URL. |
| `CRAWLBASE_GET_STORED_PAGES` | Get Stored Pages | Retrieve up to 100 Crawlbase Cloud Storage pages by request ID without deleting them. Returns decoded page content and identifies requested IDs that were not found. |
| `CRAWLBASE_LIST_STORED_PAGE_IDS` | List Stored Page IDs | Return one page of Crawlbase Cloud Storage request IDs and a short-lived continuation cursor. |

## Supported Triggers

None listed.

## Troubleshooting

### 1. Why does Claude not use the Composio connector for Crawlbase?

The connector can show as connected, but Claude must use it in each Crawlbase conversation.
- Click + at the lower left of the chat, or type /.
- Hover Connectors.
- Turn on Composio.
- Ask Claude to connect to Crawlbase again.

### 2. Why does Claude show an MCP server error before Crawlbase connects?

Claude can show Couldn't reach the MCP server or Authorization with the MCP server failed when you add the connector for Crawlbase.
- Check that the connector URL is https://connect.composio.dev/mcp.
- Connectors run from Anthropic's cloud, so your device network is usually not the problem.
- Add the connector again from Customize > Connectors, then ask Claude to connect Crawlbase.
- If it fails again, send the error and the ofid_ reference id to support@composio.dev.

### 3. Why does the Crawlbase authorization link not work?

The Composio connector is one server. Your Crawlbase account connects separately when Claude asks for it.
- If the browser does not open, or the link expires, ask Claude to retry the Crawlbase connection.
- Composio creates a fresh Crawlbase authorization link.
- Open that link in your browser and finish the Crawlbase sign-in.

### 4. Why do Crawlbase actions fail after they worked before?

The Crawlbase connection can expire or lose permission.
- Ask Claude to inspect the Crawlbase connection.
- If the connection is unhealthy, disconnect it when Claude prompts you.
- Reconnect the Crawlbase account and retry the action.
- If Claude shows a permission prompt, approve it before Claude changes data in Crawlbase.
- If the action still fails with a permission error, check that the connected account has that permission in Crawlbase itself.

### 5. How do I switch to a different Crawlbase account?

Claude can manage the Crawlbase connection for you.
- Ask Claude to disconnect or delete the current Crawlbase account.
- Ask Claude to connect Crawlbase again.
- Open the new Composio authorization link for Crawlbase.
- Sign in with the other Crawlbase account in the browser.
- After the account connects, ask Claude to retry the Crawlbase task.

### 6. Why is my Crawlbase connection missing in Claude Desktop or on a Team plan?

The connector belongs to your Claude account, so your Crawlbase connection works in Claude Web, Desktop, and Cowork.
- In Claude Desktop, open Customize > Connectors.
- Next to Composio, click ⋮ and click Clear cache.
- If Crawlbase tools still do not appear, click ⋮, disconnect, remove the connector, and add it again.
- On Team or Enterprise, an Owner or Primary Owner must enable the connector before you can connect Crawlbase.

## Creating MCP Server - Stand-alone vs Composio SDK

The Crawlbase MCP server connects Claude Web, Desktop, and Cowork to your Crawlbase account through Composio. Once connected, Claude can use the available Crawlbase tools and triggers to complete tasks on your behalf.

## Complete Code

None listed.

## How to build Crawlbase MCP Agent with another framework

- [ChatGPT](https://composio.dev/toolkits/crawlbase/framework/chatgpt)
- [Grok Bot](https://composio.dev/toolkits/crawlbase/framework/grok-bot)
- [Hermes](https://composio.dev/toolkits/crawlbase/framework/hermes-agent)
- [Atomic Agent](https://composio.dev/toolkits/crawlbase/framework/atomic-agent)

## Related Toolkits

- [Firecrawl](https://composio.dev/toolkits/firecrawl) - Firecrawl automates large-scale web crawling and data extraction. It helps organizations efficiently gather, index, and analyze content from online sources.
- [Tavily](https://composio.dev/toolkits/tavily) - Tavily offers powerful search and data retrieval from documents, databases, and the web. It helps teams locate and filter information instantly, saving hours on research.
- [Exa](https://composio.dev/toolkits/exa) - Exa is a data extraction and search platform for gathering and analyzing information from websites, APIs, or databases. It helps teams quickly surface insights and automate data-driven workflows.
- [Serpapi](https://composio.dev/toolkits/serpapi) - SerpApi is a real-time API for structured search engine results. It lets you automate SERP data collection, parsing, and analysis for SEO and research.
- [Excel](https://composio.dev/toolkits/excel) - Microsoft Excel is a robust spreadsheet application for organizing, analyzing, and visualizing data. It's the go-to tool for calculations, reporting, and flexible data management.
- [Peopledatalabs](https://composio.dev/toolkits/peopledatalabs) - Peopledatalabs delivers B2B data enrichment and identity resolution APIs. Supercharge your apps with accurate, up-to-date business and contact data.
- [Snowflake](https://composio.dev/toolkits/snowflake) - Snowflake is a cloud data warehouse built for elastic scaling, secure data sharing, and fast SQL analytics across major clouds.
- [Posthog](https://composio.dev/toolkits/posthog) - PostHog is an open-source analytics platform for tracking user interactions and product metrics. It helps teams refine features, analyze funnels, and reduce churn with actionable insights.
- [21risk](https://composio.dev/toolkits/_21risk) - 21RISK is a web app built for easy checklist, audit, and compliance management. It streamlines risk processes so teams can focus on what matters.
- [Abstract](https://composio.dev/toolkits/abstract) - Abstract provides a suite of APIs for automating data validation and enrichment tasks. It helps developers streamline workflows and ensure data quality with minimal effort.
- [AccuWeather](https://composio.dev/toolkits/accu_weather) - AccuWeather is a global weather service providing current conditions, forecasts, MinuteCast precipitation, alerts, and lightning data. Use it for fast, localized forecasts and real-time weather alerts.
- [Adafruit IO](https://composio.dev/toolkits/adafruit_io) - Adafruit IO is Adafruit's cloud platform for connected device and IoT project data. It helps you store, visualize, and act on sensor feeds without running your own backend.
- [Addressfinder](https://composio.dev/toolkits/addressfinder) - Addressfinder is a data quality platform for verifying addresses, emails, and phone numbers. It helps you ensure accurate customer and contact data every time.
- [Agentql](https://composio.dev/toolkits/agentql) - Agentql is a toolkit that connects AI agents to the web using a specialized query language. It enables structured web interaction and data extraction for smarter automations.
- [Agenty](https://composio.dev/toolkits/agenty) - Agenty is a web scraping and automation platform for extracting data and automating browser tasks—no coding needed. It streamlines data collection, monitoring, and repetitive online actions.
- [Ahrefs MCP](https://composio.dev/toolkits/ahrefs_mcp) - Ahrefs MCP is Ahrefs' hosted MCP server for SEO data and insights. Use it to access backlinks, organic metrics, keyword research, and competitor analysis.
- [Akta](https://composio.dev/toolkits/akta) - Akta is a company intelligence platform providing enrichment, news, and alternative business signals. It helps teams build targeted company lists and enrich data-driven workflows.
- [Ambee](https://composio.dev/toolkits/ambee) - Ambee is an environmental data platform providing real-time, hyperlocal APIs for air quality, weather, and pollen. Get precise environmental insights to power smarter decisions in your apps and workflows.
- [Ambient weather](https://composio.dev/toolkits/ambient_weather) - Ambient Weather is a platform for personal weather stations with a robust API for accessing local, real-time, and historical weather data. Get detailed environmental insights directly from your own sensors for smarter apps and automations.
- [Amplitude](https://composio.dev/toolkits/amplitude) - Amplitude is a digital analytics platform for product and behavioral data insights. It helps teams analyze user journeys and make data-driven decisions quickly.

## Frequently Asked Questions

### What are the differences in Tool Router MCP and Crawlbase MCP?

With a standalone Crawlbase MCP server, the agents and LLMs can only access a fixed set of Crawlbase tools tied to that server. However, with the Composio Tool Router, agents can dynamically load tools from Crawlbase and many other apps based on the task at hand, all through a single MCP endpoint.

### Can I use Tool Router MCP with Claude Cowork?

Yes, you can. Claude Cowork fully supports MCP integration. You get structured tool calling, message history handling, and model orchestration while Tool Router takes care of discovering and serving the right Crawlbase tools.

### Can I manage the permissions and scopes for Crawlbase while using Tool Router?

Yes, absolutely. You can configure which Crawlbase scopes and actions are allowed when connecting your account to Composio. You can also bring your own OAuth credentials or API configuration so you keep full control over what the agent can do.

### How safe is my data with Composio Tool Router?

All sensitive data such as tokens, keys, and configuration is fully encrypted at rest and in transit. Composio is SOC 2 Type 2 compliant and follows strict security practices so your Crawlbase data and credentials are handled as safely as possible.

---
[See all toolkits](https://composio.dev/toolkits) · [Composio docs](https://docs.composio.dev/llms.txt)
