X (Twitter)LinkedInGitHubDiscordYouTube

How to connect Scrape do to Claude

Connect Claude Web, Desktop and Cowork to Scrape do MCP. Scrape product prices from a dynamic website, extract news headlines with javascript rendering, and more using natural language, with authentication handled for you.

Get started for freeGet a demo
Scrape do logoScrape do
Api Key

Scrape.do is a web scraping API offering rotating proxies, headless browser support, and session management to bypass anti-bot protections. Get reliable, scalable extraction of structured web data in JSON or HTML formats.

16 Tools

Introduction

Claude is Anthropic's AI assistant, available on the web, in the desktop app, and on mobile. Cowork is its agent for knowledge work, and it runs on all of those surfaces too. Connected to your apps, Claude can work with your files and services to accomplish complex tasks on your behalf.

This guide walks you through the easiest and most secure way to connect your Scrape do account to Claude via Composio Connect, enabling it to scrape product prices from a dynamic website, extract news headlines with JavaScript rendering, bypass Cloudflare to get full page HTML, and more such actions on your behalf without compromising your account security.

Setup is the same on Claude Web, Desktop, and Cowork, and you only need to do it once. The connector is tied to your account, so it's available in all three.

Also integrate Scrape do with

OpenAI Agents SDKClaude Agent SDKClaude CodeCodexOpenClawHermesAtomic AgentGoogle ADKLangChainVercel AI SDKMastra AILlamaIndexCrewAIPydantic AIAutoGen

Connecting Scrape do to Claude Web, Desktop, and Cowork

Two ways to add the Composio connector. Pick whichever you prefer.

1. Add with one click

Click the button. Claude opens with the Composio connector filled in. Click Add, then go to step 2.

Add to Claude

Add manually

In Claude Desktop, click Customize in the left sidebar, then select Connectors and click the + icon at the top.

Claude Desktop connectors screen with Add custom connector selected

Click Add custom connector and paste in the Composio MCP server URL:

bash
https://connect.composio.dev/mcp
Add custom connector dialog with Composio MCP server URL

2. Authorize in your browser

Click Connect. You'll be redirected to a browser window where you can authorize Composio to continue.

Composio authorization screen for Claude Web, Desktop, and Cowork

3. Connect your Scrape do account

Back in Claude, ask it to connect to Scrape do or give it any Scrape do-related task.

For example, ask Claude to:

  • "Scrape product prices from a dynamic website"
  • "Extract news headlines with JavaScript rendering"
  • "Bypass Cloudflare to get full page HTML"

It will prompt you to authenticate and authorize access.

That's it. Composio's tools are now available in Claude, and your Scrape do account is ready to use.

What is the Scrape do Claude Connector, and what's possible with it?

The Scrape do MCP server is an implementation of the Model Context Protocol that connects your AI agent and assistants like Claude, Cursor, etc directly to your Scrape do account. It provides structured and secure access to robust web scraping tools, so your agent can perform actions like scraping dynamic pages, managing sessions, setting custom headers or proxies, and extracting structured data from any website on your behalf.

  • Dynamic page scraping with headless browsers: Retrieve fully rendered HTML content from JavaScript-heavy or protected websites by leveraging advanced browser emulation and proxy rotation.
  • Custom scraping session management: Set device type, cookies, wait times, and custom headers to imitate different users, maintain sessions, or access device-specific content for tailored data extraction.
  • Proxy and anti-bot bypass control: Enable super or proxy modes to utilize residential, mobile, or datacenter proxies, helping your agent bypass strict anti-bot systems and geo-restrictions seamlessly.
  • Targeted resource filtering: Block specific URLs like ads or analytics scripts during scraping to increase speed, avoid distractions, and improve privacy.
  • Account usage and statistics retrieval: Access real-time usage stats, subscription status, and remaining request limits so your agent can monitor scraping quotas and avoid interruptions.

Troubleshooting

1. Why does Claude not use the Composio connector for Scrape do?

The connector can show as connected, but Claude must use it in each Scrape do conversation.

  • Click + at the lower left of the chat, or type /.
  • Hover Connectors.
  • Turn on Composio.
  • Ask Claude to connect to Scrape do again.

2. Why does Claude show an MCP server error before Scrape do connects?

Claude can show Couldn't reach the MCP server or Authorization with the MCP server failed when you add the connector for Scrape do.

  • Check that the connector URL is https://connect.composio.dev/mcp.
  • Connectors run from Anthropic's cloud, so your device network is usually not the problem.
  • Add the connector again from Customize > Connectors, then ask Claude to connect Scrape do.
  • If it fails again, send the error and the ofid_ reference id to support@composio.dev.

3. Why does the Scrape do authorization link not work?

The Composio connector is one server. Your Scrape do account connects separately when Claude asks for it.

  • If the browser does not open, or the link expires, ask Claude to retry the Scrape do connection.
  • Composio creates a fresh Scrape do authorization link.
  • Open that link in your browser and finish the Scrape do sign-in.

4. Why do Scrape do actions fail after they worked before?

The Scrape do connection can expire or lose permission.

  • Ask Claude to inspect the Scrape do connection.
  • If the connection is unhealthy, disconnect it when Claude prompts you.
  • Reconnect the Scrape do account and retry the action.
  • If Claude shows a permission prompt, approve it before Claude changes data in Scrape do.
  • If the action still fails with a permission error, check that the connected account has that permission in Scrape do itself.

5. How do I switch to a different Scrape do account?

Claude can manage the Scrape do connection for you.

  • Ask Claude to disconnect or delete the current Scrape do account.
  • Ask Claude to connect Scrape do again.
  • Open the new Composio authorization link for Scrape do.
  • Sign in with the other Scrape do account in the browser.
  • After the account connects, ask Claude to retry the Scrape do task.

6. Why is my Scrape do connection missing in Claude Desktop or on a Team plan?

The connector belongs to your Claude account, so your Scrape do connection works in Claude Web, Desktop, and Cowork.

  • In Claude Desktop, open Customize > Connectors.
  • Next to Composio, click ⋮ and click Clear cache.
  • If Scrape do tools still do not appear, click ⋮, disconnect, remove the connector, and add it again.
  • On Team or Enterprise, an Owner or Primary Owner must enable the connector before you can connect Scrape do.
TOOLS

Supported Tools

Every Scrape do action and event your agent gets out of the box.

Cancel Async Job

Tool to cancel an asynchronous scraping job.

Create Async Scraping Job

Tool to create an asynchronous scraping job with specified targets and options.

Get Account Information

Retrieves account information and usage statistics from Scrape.

Get Amazon Product Offers

Get all seller offers for any Amazon product.

Get Amazon product details

Extract structured product data from Amazon product detail pages (PDP).

Get Amazon raw HTML

Tool to get raw HTML from any Amazon page with ZIP code geo-targeting.

Get Async API Account Information

Tool to get account information for the Async API including concurrency limits and usage statistics.

Get Async Job Details

Tool to retrieve details and status of a specific asynchronous scraping job.

Get Async Task Result

Tool to retrieve the result of a specific task within an asynchronous job.

Scrape webpage using scrape.do

A tool to scrape web pages using scrape.

List Asynchronous Scraping Jobs

Tool to list all asynchronous scraping jobs.

Use Scrape.do Proxy Mode

This tool implements the Proxy Mode functionality of scrape.

Scrape URL using POST method

Tool to scrape web pages using POST method via scrape.

Search Amazon products

Tool to search Amazon and scrape product listings with structured results.

Block specific URLs during scraping

This tool allows users to block specific URLs during the scraping process.

Set Regional Geolocation for Scraping

This tool allows users to set a broader geographical targeting by specifying a region code instead of a specific country code.

FRAMEWORKS

How to build Scrape do MCP Agent with another framework

OpenAI Agents SDK logo

OpenAI Agents SDK

Use Scrape do MCP with OpenAI Agents SDK

Claude Agent SDK logo

Claude Agent SDK

Use Scrape do MCP with Claude Agent SDK

Claude Code logo

Claude Code

Use Scrape do MCP with Claude Code

Codex logo

Codex

Use Scrape do MCP with Codex

OpenClaw logo

OpenClaw

Use Scrape do MCP with OpenClaw

Hermes logo

Hermes

Use Scrape do MCP with Hermes

Atomic Agent logo

Atomic Agent

Use Scrape do MCP with Atomic Agent

Google ADK logo

Google ADK

Use Scrape do MCP with Google ADK

LangChain logo

LangChain

Use Scrape do MCP with LangChain

Vercel AI SDK logo

Vercel AI SDK

Use Scrape do MCP with Vercel AI SDK

Mastra AI logo

Mastra AI

Use Scrape do MCP with Mastra AI

LlamaIndex logo

LlamaIndex

Use Scrape do MCP with LlamaIndex

CrewAI logo

CrewAI

Use Scrape do MCP with CrewAI

Pydantic AI logo

Pydantic AI

Use Scrape do MCP with Pydantic AI

AutoGen logo

AutoGen

Use Scrape do MCP with AutoGen

FAQ

Frequently asked questions

With a standalone Scrape do MCP server, the agents and LLMs can only access a fixed set of Scrape do tools tied to that server. However, with the Composio Tool Router, agents can dynamically load tools from Scrape do and many other apps based on the task at hand, all through a single MCP endpoint.

Yes, you can. Claude Cowork fully supports MCP integration. You get structured tool calling, message history handling, and model orchestration while Tool Router takes care of discovering and serving the right Scrape do tools.

Yes, absolutely. You can configure which Scrape do scopes and actions are allowed when connecting your account to Composio. You can also bring your own OAuth credentials or API configuration so you keep full control over what the agent can do.

All sensitive data such as tokens, keys, and configuration is fully encrypted at rest and in transit. Composio is SOC 2 Type 2 compliant and follows strict security practices so your Scrape do data and credentials are handled as safely as possible.

Start with Scrape do.It takes 30 seconds.

Managed auth, hosted MCP servers, and every Scrape do tool your agent needs.Free to start.

Start building