# MrScraper

```json
{
  "name": "MrScraper",
  "slug": "mrscraper",
  "url": "https://composio.dev/toolkits/mrscraper",
  "markdown_url": "https://composio.dev/toolkits/mrscraper.md",
  "logo_url": "https://logos.composio.dev/api/mrscraper",
  "categories": [
    "data & analytics"
  ],
  "is_composio_managed": false,
  "updated_at": "2026-10-04T15:02:31.858Z"
}
```

![MrScraper logo](https://logos.composio.dev/api/mrscraper)

## Description

Securely connect your AI agents and chatbots (Claude, ChatGPT, Cursor, etc) with MrScraper MCP or direct API to scrape websites, extract structured data, fetch search results, and collect marketplace or travel data through natural language.

## Summary

MrScraper is a web scraping API platform for structured extraction, search results, marketplace data, travel data, and web unblocking.
Use it to turn messy web pages into clean, usable data without managing scraper infrastructure.

## Categories

- data & analytics

## Toolkit Details

- Tools: 11

## Images

- Logo: https://logos.composio.dev/api/mrscraper

## Authentication

- **Api Key**
  - Type: `api_key`
  - Description: Api Key authentication for MrScraper.
  - Setup:
    - Configure Api Key credentials for MrScraper.
    - Use the credentials when creating an auth config in Composio.

## Suggested Prompts

- Extract product prices from competitor pages
- Scrape Google results for brand mentions
- Collect hotel rates from travel sites

## Supported Tools

| Tool slug | Name | Description |
|---|---|---|
| `MRSCRAPER_BULK_RERUN_AI_SCRAPER` | Bulk Rerun AI Scraper | Run an existing General AI scraper configuration against multiple URLs in one request. Returns the accepted bulk result ID for Get Scrape Result. |
| `MRSCRAPER_BULK_RERUN_MANUAL_SCRAPER` | Bulk Rerun Manual Scraper | Run an existing saved manual scraper workflow against multiple URLs in one request. Returns the accepted bulk result ID for Get Scrape Result. |
| `MRSCRAPER_CREATE_AI_SCRAPER` | Create AI Scraper | Create and run a general, listing, or map AI scraper against a website. |
| `MRSCRAPER_GET_ACCOUNT_USAGE` | Get Account Usage | Return the connected MrScraper account's plan, token usage, concurrency allowance, and rate limits without exposing user identity or token details. |
| `MRSCRAPER_GET_ANALYTIC_STATUSES` | Get Scrape Status Analytics | Return aggregate scrape counts, success rate, request rate, latency, and status totals for a domain and action over a time range. |
| `MRSCRAPER_GET_ANALYTIC_TIMELINE` | Get Scrape Analytics Timeline | Return time-series scrape activity and success metrics for a domain and action over a time range. |
| `MRSCRAPER_GET_RESULT` | Get Scrape Result | Retrieve the current status and output for one accessible scrape result by ID. |
| `MRSCRAPER_HEAL_MANUAL_SCRAPER` | Heal Manual Scraper | Ask MrScraper to repair an existing manual scraper workflow that no longer matches its target site. |
| `MRSCRAPER_LIST_RESULTS` | List Scrape Results | List scrape results for the connected account with filtering, sorting, and agent-controlled pagination. |
| `MRSCRAPER_RERUN_AI_SCRAPER` | Rerun AI Scraper | Run an existing AI scraper configuration against a new URL, with optional map crawl controls. |
| `MRSCRAPER_RERUN_MANUAL_SCRAPER` | Rerun Manual Scraper | Run an existing saved manual scraper workflow against a new URL. |

## Supported Triggers

None listed.

## Installation and MCP Setup

### Path 1: SDK Installation

#### Path 1, Step 1: Install Composio

Install the Composio SDK, the OpenAI provider, and the OpenAI SDK
```python
pip install composio composio_openai openai
```

```typescript
npm install @composio/core @composio/openai openai
```

#### Path 1, Step 2: Create a Composio session

Initialize Composio with the OpenAI Responses provider and create a session scoped to MrScraper
```python
import json
from openai import OpenAI
from composio import Composio
from composio_openai import OpenAIResponsesProvider

composio = Composio(provider=OpenAIResponsesProvider())
client = OpenAI()

session = composio.create(user_id="your-user-id", toolkits=["mrscraper"])
tools = session.tools()
```

```typescript
import OpenAI from "openai";
import { Composio } from "@composio/core";
import { OpenAIResponsesProvider } from "@composio/openai";

const composio = new Composio({ provider: new OpenAIResponsesProvider() });
const client = new OpenAI();

const session = await composio.create("your-user-id", { toolkits: ["mrscraper"] });
const tools = await session.tools();
```

#### Path 1, Step 3: Run MrScraper tools with your agent

Send a request, execute the MrScraper tool calls through the session, and print the final answer
```python
response = client.responses.create(
    model="gpt-5.6-sol",
    tools=tools,
    input=[{"role": "user", "content": "Scrape structured product details and prices from https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html"}],
)

while True:
    tool_calls = [o for o in response.output if o.type == "function_call"]
    if not tool_calls:
        break
    results = composio.provider.handle_tool_calls(response=response, session=session)
    response = client.responses.create(
        model="gpt-5.6-sol",
        tools=tools,
        previous_response_id=response.id,
        input=[
            {"type": "function_call_output", "call_id": call.call_id, "output": json.dumps(results[i])}
            for i, call in enumerate(tool_calls)
        ],
    )

print(response.output_text)
```

```typescript
let response = await client.responses.create({
  model: "gpt-5.6-sol",
  tools,
  input: [{ role: "user", content: "Scrape structured product details and prices from https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html" }],
});

while (response.output.some((o) => o.type === "function_call")) {
  const outputs = await composio.provider.handleToolCalls(session, response.output);
  response = await client.responses.create({
    model: "gpt-5.6-sol",
    tools,
    previous_response_id: response.id,
    input: outputs,
  });
}

console.log(response.output_text);
```

### Path 2: MCP Server Setup

#### Path 2, Step 1: Install Composio

Install the Composio SDK and your agent framework
```python
pip install composio claude-agent-sdk
```

```typescript
npm install @composio/core ai @ai-sdk/mcp @ai-sdk/openai
```

#### Path 2, Step 2: Create a session with MCP enabled

Create a session scoped to MrScraper and read its MCP URL and headers
```python
from composio import Composio

composio = Composio()
session = composio.create(user_id="your-user-id", toolkits=["mrscraper"], mcp=True)
```

```typescript
import { Composio } from "@composio/core";

const composio = new Composio();
const { mcp } = await composio.create("your-user-id", {
  toolkits: ["mrscraper"],
  mcp: true,
});
```

#### Path 2, Step 3: Connect your agent to the MCP server

Pass the session's MCP URL and headers to your agent and run a MrScraper request
```python
import asyncio
from claude_agent_sdk import ClaudeSDKClient, ClaudeAgentOptions, AssistantMessage, TextBlock

options = ClaudeAgentOptions(
    mcp_servers={
        "composio": {
            "type": "http",
            "url": session.mcp.url,
            "headers": session.mcp.headers,
        }
    },
    system_prompt="You are a helpful assistant with access to MrScraper tools.",
    tools=[],
    allowed_tools=["mcp__composio__*"],
)

async def main():
    async with ClaudeSDKClient(options=options) as client:
        await client.query("Scrape structured product details and prices from https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html")
        async for message in client.receive_response():
            if isinstance(message, AssistantMessage):
                for block in message.content:
                    if isinstance(block, TextBlock):
                        print(block.text)

asyncio.run(main())
```

```typescript
import { createMCPClient } from "@ai-sdk/mcp";
import { openai } from "@ai-sdk/openai";
import { generateText, stepCountIs } from "ai";

const client = await createMCPClient({
  transport: { type: "http", url: mcp.url, headers: mcp.headers },
});

const { text } = await generateText({
  model: openai("gpt-5.6-sol"),
  tools: await client.tools(),
  prompt: "Scrape structured product details and prices from https://books.toscrape.com/catalogue/a-light-in-the-attic_1000/index.html",
  stopWhen: stepCountIs(10),
});

console.log(text);
await client.close();
```

## Why Use Composio?

### 1. AI Native MrScraper Integration

- Supports both MrScraper MCP and direct API based integrations
- Structured, LLM-friendly schemas for reliable scraping and extraction tasks
- Rich coverage for website scraping, search result collection, marketplace data, travel data, and web unblocking

### 2. Managed Auth

- Secure API key handling so agents can use MrScraper without hard-coded secrets
- Create API-key auth configs with auth_configs.create() and link user credentials with connected_accounts.link()
- Per user and per environment credentials, which keeps local tests and production scrapers cleanly separated

### 3. Agent Optimized Design

- Tools are tuned using real error and success rates to improve scraping reliability over time
- Comprehensive execution logs so you always know which URL was scraped, when it ran, and on whose behalf
- Clear schemas help agents return structured page data instead of messy raw HTML

### 4. Enterprise Grade Security

- Fine-grained RBAC so you control which agents and users can access MrScraper
- Scoped, least privilege access for scraping, search, marketplace, travel, and unblocking workflows
- Full audit trail of agent scraping actions to support review and compliance

## Use MrScraper with any AI Agent Framework

Choose a framework you want to connect MrScraper with:

None listed.

## Related Toolkits

- [Excel](https://composio.dev/toolkits/excel) - Microsoft Excel is a robust spreadsheet application for organizing, analyzing, and visualizing data. It's the go-to tool for calculations, reporting, and flexible data management.
- [21risk](https://composio.dev/toolkits/_21risk) - 21RISK is a web app built for easy checklist, audit, and compliance management. It streamlines risk processes so teams can focus on what matters.
- [Abstract](https://composio.dev/toolkits/abstract) - Abstract provides a suite of APIs for automating data validation and enrichment tasks. It helps developers streamline workflows and ensure data quality with minimal effort.
- [AccuWeather](https://composio.dev/toolkits/accu_weather) - AccuWeather is a global weather service providing current conditions, forecasts, MinuteCast precipitation, alerts, and lightning data. Use it for fast, localized forecasts and real-time weather alerts.
- [Adafruit IO](https://composio.dev/toolkits/adafruit_io) - Adafruit IO is Adafruit's cloud platform for connected device and IoT project data. It helps you store, visualize, and act on sensor feeds without running your own backend.
- [Addressfinder](https://composio.dev/toolkits/addressfinder) - Addressfinder is a data quality platform for verifying addresses, emails, and phone numbers. It helps you ensure accurate customer and contact data every time.
- [Agentql](https://composio.dev/toolkits/agentql) - Agentql is a toolkit that connects AI agents to the web using a specialized query language. It enables structured web interaction and data extraction for smarter automations.
- [Agenty](https://composio.dev/toolkits/agenty) - Agenty is a web scraping and automation platform for extracting data and automating browser tasks—no coding needed. It streamlines data collection, monitoring, and repetitive online actions.
- [Ambee](https://composio.dev/toolkits/ambee) - Ambee is an environmental data platform providing real-time, hyperlocal APIs for air quality, weather, and pollen. Get precise environmental insights to power smarter decisions in your apps and workflows.
- [Ambient weather](https://composio.dev/toolkits/ambient_weather) - Ambient Weather is a platform for personal weather stations with a robust API for accessing local, real-time, and historical weather data. Get detailed environmental insights directly from your own sensors for smarter apps and automations.
- [Anonyflow](https://composio.dev/toolkits/anonyflow) - Anonyflow is a service for encryption-based data anonymization and secure data sharing. It helps organizations meet GDPR, CCPA, and HIPAA data privacy compliance requirements.
- [ApexVerify](https://composio.dev/toolkits/apexverify) - ApexVerify is a contact-data validation service for emails and phone numbers. Use it to normalize, verify, and clean leads before they enter your systems.
- [Api ninjas](https://composio.dev/toolkits/api_ninjas) - Api ninjas offers 120+ public APIs spanning categories like weather, finance, sports, and more. Developers use it to supercharge apps with real-time data and actionable endpoints.
- [Api sports](https://composio.dev/toolkits/api_sports) - Api sports is a comprehensive sports data platform covering 2,000+ competitions with live scores and 15+ years of stats. Instantly access up-to-date sports information for analysis, apps, or chatbots.
- [Apify](https://composio.dev/toolkits/apify) - Apify is a cloud platform for building, deploying, and managing web scraping and automation tools called Actors. It lets you automate data extraction and workflow tasks at scale—no infrastructure headaches.
- [Autom](https://composio.dev/toolkits/autom) - Autom is a lightning-fast search engine results data platform for Google, Bing, and Brave. Developers use it to access fresh, low-latency SERP data on demand.
- [Beach Day API](https://composio.dev/toolkits/beach_day_api) - Beach Day API is a beach intelligence service for search, conditions, scores, rules, amenities, tides, and country coverage. Use it to power trip planning, surf reports, safety checks, and location-aware beach discovery.
- [Beaconchain](https://composio.dev/toolkits/beaconchain) - Beaconchain is a real-time analytics platform for Ethereum 2.0's Beacon Chain. It provides detailed insights into validators, blocks, and overall network performance.
- [Big data cloud](https://composio.dev/toolkits/big_data_cloud) - BigDataCloud provides APIs for geolocation, reverse geocoding, and address validation. Instantly access reliable location intelligence to enhance your applications and workflows.
- [Bigpicture io](https://composio.dev/toolkits/bigpicture_io) - BigPicture.io offers APIs for accessing detailed company and profile data. Instantly enrich your applications with up-to-date insights on 20M+ businesses.

## Frequently Asked Questions

### Do I need my own developer credentials to use MrScraper with Composio?

Yes, MrScraper requires you to configure your own API key. Once set up, Composio handles secure credential storage and API request handling for you.

### Can I use multiple toolkits together?

Yes! Composio's Tool Router enables agents to use multiple toolkits. [Learn more](https://docs.composio.dev/tool-router/overview).

### Is Composio secure?

Composio is SOC 2 and ISO 27001 compliant with all data encrypted in transit and at rest. [Learn more](https://trust.composio.dev).

### What if the API changes?

Composio maintains and updates all toolkit integrations automatically, so your agents always work with the latest API versions.

---
[See all toolkits](https://composio.dev/toolkits) · [Composio docs](https://docs.composio.dev/llms.txt)