How to connect Spider to Claude

Connect Claude Web, Desktop and Cowork to Spider MCP. Crawl a product site and summarize pricing, search the web for competitor product updates, and more using natural language, with authentication handled for you.

Spider logoSpider
Api Key

Spider is a web data platform for crawling, scraping, search, screenshots, and AI extraction. It helps teams collect and transform web content at high concurrency.

15 Tools

Introduction

Claude is Anthropic's AI assistant, available on the web, in the desktop app, and on mobile. Cowork is its agent for knowledge work, and it runs on all of those surfaces too. Connected to your apps, Claude can work with your files and services to accomplish complex tasks on your behalf.

This guide walks you through the easiest and most secure way to connect your Spider account to Claude via Composio Connect, enabling it to crawl a product site and summarize pricing, search the web for competitor product updates, extract contact details from company websites, and more such actions on your behalf without compromising your account security.

Setup is the same on Claude Web, Desktop, and Cowork, and you only need to do it once. The connector is tied to your account, so it's available in all three.

For exact tool slugs and parameter tables, see the tool catalog in the Composio docs.

Also integrate Spider with

Connecting Spider to Claude Web, Desktop, and Cowork

Two ways to add the Composio connector. Pick whichever you prefer.

1. Add with one click

Click the button. Claude opens with the Composio connector filled in. Click Add, then go to step 2.

Add to Claude

Add manually

In Claude Desktop, click Customize in the left sidebar, then select Connectors and click the + icon at the top.

Claude Desktop connectors screen with Add custom connector selected

Click Add custom connector and paste in the Composio MCP server URL:

bash
https://connect.composio.dev/mcp
Add custom connector dialog with Composio MCP server URL

2. Authorize in your browser

Click Connect. You'll be redirected to a browser window where you can authorize Composio to continue.

Composio authorization screen for Claude Web, Desktop, and Cowork

3. Connect your Spider account

Back in Claude, ask it to connect to Spider or give it any Spider-related task.

For example, ask Claude to:

  • "Crawl a product site and summarize pricing"
  • "Search the web for competitor product updates"
  • "Extract contact details from company websites"

It will prompt you to authenticate and authorize access.

That's it. Composio's tools are now available in Claude, and your Spider account is ready to use.

What is the Spider Claude Connector, and what's possible with it?

The Spider MCP server is an implementation of the Model Context Protocol that connects your AI agent and assistants like Claude, Cursor, etc directly to your Spider account. It provides structured and secure access to web crawling, scraping, search, screenshots, content transformation, and AI extraction tools, so your agent can crawl sites, scrape pages, search the web, extract structured data, and transform or capture web content on your behalf.

  • AI-powered website extraction: Have your agent crawl websites or scrape specific pages, then extract structured information from them using natural-language instructions.
  • Web search and discovery: Direct your agent to search the web, fetch relevant result pages, or discover links that match a specific description across a site.
  • Browser-based research: Let your agent use the AI browser to navigate websites from a natural-language prompt and return the requested results.
  • Page screenshots: Instruct your agent to capture a rendered webpage as a PNG, JPEG, or WebP image.
  • Content transformation: Have your agent convert supplied HTML into markdown, text, or cleaned output without fetching a URL.

Troubleshooting

1. Why does Claude not use the Composio connector for Spider?

The connector can show as connected, but Claude must use it in each Spider conversation.

  • Click + at the lower left of the chat, or type /.
  • Hover Connectors.
  • Turn on Composio.
  • Ask Claude to connect to Spider again.

2. Why does Claude show an MCP server error before Spider connects?

Claude can show Couldn't reach the MCP server or Authorization with the MCP server failed when you add the connector for Spider.

  • Check that the connector URL is https://connect.composio.dev/mcp.
  • Connectors run from Anthropic's cloud, so your device network is usually not the problem.
  • Add the connector again from Customize > Connectors, then ask Claude to connect Spider.
  • If it fails again, send the error and the ofid_ reference id to support@composio.dev.

3. Why does the Spider authorization link not work?

The Composio connector is one server. Your Spider account connects separately when Claude asks for it.

  • If the browser does not open, or the link expires, ask Claude to retry the Spider connection.
  • Composio creates a fresh Spider authorization link.
  • Open that link in your browser and finish the Spider sign-in.

4. Why do Spider actions fail after they worked before?

The Spider connection can expire or lose permission.

  • Ask Claude to inspect the Spider connection.
  • If the connection is unhealthy, disconnect it when Claude prompts you.
  • Reconnect the Spider account and retry the action.
  • If Claude shows a permission prompt, approve it before Claude changes data in Spider.
  • If the action still fails with a permission error, check that the connected account has that permission in Spider itself.

5. How do I switch to a different Spider account?

Claude can manage the Spider connection for you.

  • Ask Claude to disconnect or delete the current Spider account.
  • Ask Claude to connect Spider again.
  • Open the new Composio authorization link for Spider.
  • Sign in with the other Spider account in the browser.
  • After the account connects, ask Claude to retry the Spider task.

6. Why is my Spider connection missing in Claude Desktop or on a Team plan?

The connector belongs to your Claude account, so your Spider connection works in Claude Web, Desktop, and Cowork.

  • In Claude Desktop, open Customize > Connectors.
  • Next to Composio, click ⋮ and click Clear cache.
  • If Spider tools still do not appear, click ⋮, disconnect, remove the connector, and add it again.
  • On Team or Enterprise, an Owner or Primary Owner must enable the connector before you can connect Spider.
TOOLS

Supported Tools

Every Spider action and event your agent gets out of the box.

AI Browser

Drive a browser from a natural-language prompt and extract results.

AI Crawl Website

Crawl pages and use an AI prompt to extract structured data.

AI Discover Links

Collect links matching a natural-language prompt and return structured context.

AI Scrape Page

Fetch exact pages and use an AI prompt to extract structured data.

AI Search Web

Run prompt-driven web search with optional page fetching and structured extraction.

Capture Screenshot

Render one page and capture a PNG, JPEG, or WebP screenshot with a provider-side response-content cap.

Crawl Website

Crawl pages from one or more starting URLs.

Discover Links

Discover URLs from a site without returning full page content.

Fetch Configured Page

Fetch a domain and path with Spider's saved scraper configuration, discovering and storing one on first use.

Get Credit Balance

Get the connected Spider account's remaining credit balance and documented record metadata.

List Crawl Logs

Return the connected account's crawl and scrape activity from the last 24 hours.

List Scraper Directory

Browse public curated scraper configurations with domain, category, text, and page-number filters.

Scrape Pages

Fetch and extract exact pages without following links.

Search Web

Search the web and optionally fetch result pages.

Transform Content

Convert supplied HTML to markdown, text, or cleaned output without fetching a URL; the total serialized JSON request must not exceed 10 MB.

FAQ

Frequently asked questions

Start with Spider.It takes 30 seconds.

Managed auth, hosted MCP servers, and every Spider tool your agent needs.Free to start.

Start building