# How to integrate Scrapegraph ai MCP with Atomic Agent ```json { "title": "How to integrate Scrapegraph ai MCP with Atomic Agent", "toolkit": "Scrapegraph ai", "toolkit_slug": "scrapegraph_ai", "framework": "Atomic Agent", "framework_slug": "atomic-agent", "url": "https://composio.dev/toolkits/scrapegraph_ai/framework/atomic-agent", "markdown_url": "https://composio.dev/toolkits/scrapegraph_ai/framework/atomic-agent.md", "updated_at": "2026-08-11T13:14:17.788Z" } ``` ## Introduction Atomic Agent is an open-source AI agent that runs on your machine and supports local models through llama.cpp. Connect it to 1,500+ apps through Composio, including Scrapegraph ai. This guide uses Atomic Agent's built-in Composio integration. You need Atomic Agent v0.5.6 or later, your own Composio API key, and a Scrapegraph ai account. See the [Atomic Agent documentation](https://atomicagent.io/docs/?cta_placement=atomic-agent-docs) for installation instructions. ## Also integrate Scrapegraph ai with - [OpenAI Agents SDK](https://composio.dev/toolkits/scrapegraph_ai/framework/open-ai-agents-sdk) - [Claude Agent SDK](https://composio.dev/toolkits/scrapegraph_ai/framework/claude-agents-sdk) - [Claude Code](https://composio.dev/toolkits/scrapegraph_ai/framework/claude-code) - [Claude Cowork](https://composio.dev/toolkits/scrapegraph_ai/framework/claude-cowork) - [Codex](https://composio.dev/toolkits/scrapegraph_ai/framework/codex) - [OpenClaw](https://composio.dev/toolkits/scrapegraph_ai/framework/openclaw) - [Hermes](https://composio.dev/toolkits/scrapegraph_ai/framework/hermes-agent) - [Google ADK](https://composio.dev/toolkits/scrapegraph_ai/framework/google-adk) - [LangChain](https://composio.dev/toolkits/scrapegraph_ai/framework/langchain) - [Vercel AI SDK](https://composio.dev/toolkits/scrapegraph_ai/framework/ai-sdk) - [Mastra AI](https://composio.dev/toolkits/scrapegraph_ai/framework/mastra-ai) - [LlamaIndex](https://composio.dev/toolkits/scrapegraph_ai/framework/llama-index) - [CrewAI](https://composio.dev/toolkits/scrapegraph_ai/framework/crew-ai) - [Pydantic AI](https://composio.dev/toolkits/scrapegraph_ai/framework/pydantic-ai) - [AutoGen](https://composio.dev/toolkits/scrapegraph_ai/framework/autogen) ## TL;DR ### How Atomic Agent connects to Scrapegraph ai Atomic Agent connects to Composio's Tool Router over Streamable HTTP MCP. It discovers the tools needed for your request, and Composio handles account authentication. Execution and account-connection tools follow Atomic Agent's configured approval policy. Review any approval prompt before allowing an action on your connected account. ## Connect Scrapegraph ai to Atomic Agent ### Connect Scrapegraph ai to Atomic Agent - Sign up or sign in to the [Composio dashboard](https://dashboard.composio.dev/?cta_placement=atomic-agent-api-key). Copy your API key. - In Atomic Agent, open the Integrations tab and select Composio. Select API key, press e, paste the key, and press Enter. The tools become available immediately, without a restart. - Ask Atomic Agent to do something in Scrapegraph ai. It searches Composio for the right tool. - On first use, open the sign-in link returned in the chat. Connect your Scrapegraph ai account, then return to Atomic Agent to continue. ### Headless setup Add the following line to ~/.atomic-agent/.env, replacing YOUR_COMPOSIO_API_KEY with your key. Start Atomic Agent after saving the file. ```bash COMPOSIO_API_KEY=YOUR_COMPOSIO_API_KEY ``` ## What is the Scrapegraph ai MCP server, and what's possible with it? The Scrapegraph ai MCP server is an implementation of the Model Context Protocol that connects your AI agent and assistants like Claude, Cursor, etc directly to your Scrapegraph ai account. It provides structured and secure access to powerful web scraping and data extraction tools, so your agent can perform actions like running AI-powered scrapers, converting webpages to markdown, monitoring job statuses, and managing your account usage with ease. - AI-powered web scraping and search: Instruct your agent to extract structured data from any website or perform detailed web searches with parsed, organized results. - Webpage to markdown conversion: Let your agent instantly convert any webpage into clean, readable markdown for easy documentation or analysis. - Automated job status tracking: Check on the progress and results of ongoing scraping, crawling, or conversion jobs to stay updated without manual effort. - Smart multi-page crawling: Direct the agent to launch intelligent crawlers that gather data across multiple linked pages in a single workflow. - Account usage monitoring and feedback: Retrieve your remaining credits, track API usage, and submit feedback on completed tasks—all through your AI agent. ## Supported Tools | Tool slug | Name | Description | |---|---|---| | `SCRAPEGRAPH_AI_CONVERT_WEBPAGE_TO_MARKDOWN_V2` | Convert Webpage to Markdown (V2) | Tool to convert any webpage into clean, well-formatted Markdown with full parameter control. Use when you need advanced options like stealth mode, custom headers, or webhook notifications. Supports all Markdownify API parameters. | | `SCRAPEGRAPH_AI_GENERATE_SCHEMA` | Generate Schema | Generate or modify a JSON schema based on a search query for structured data extraction. Use when you need a schema template for scraping specific data fields. | | `SCRAPEGRAPH_AI_GET_AGENTIC_SCRAPER_HISTORY` | Get Agentic Scraper History | Retrieve paginated history of agentic scraper jobs. Use to view past scraping requests, their status, and results. | | `SCRAPEGRAPH_AI_GET_CRAWLER_HISTORY` | Get Crawler History | Retrieve the history of crawler jobs for your account. Returns paginated list of past crawler requests with their status, results, and metadata. | | `SCRAPEGRAPH_AI_GET_CREDITS` | Get Credits | Retrieve remaining and used credits for your ScrapeGraphAI account. Useful for checking credit availability before bulk scraping operations to avoid mid-run failures. | | `SCRAPEGRAPH_AI_GET_ENDPOINT_SUGGESTIONS` | Get Endpoint Suggestions | Tool to get AI-powered suggestions for creating scraping endpoints. Use when you need to identify what data can be extracted from a website and how to structure the scraping logic. | | `SCRAPEGRAPH_AI_GET_LIVE_SESSION_URL` | Get Live Session URL | Tool to get a URL for a live browser session. Use when you need to interact with a webpage in real-time through a controlled browser environment. | | `SCRAPEGRAPH_AI_GET_MARKDOWNIFY_HISTORY` | Get Markdownify History | Tool to retrieve the history of markdownify webpage-to-Markdown conversion jobs. Use when you need to view past markdownify requests and their statuses. | | `SCRAPEGRAPH_AI_GET_SCRAPE_HISTORY` | Get Scrape History | Retrieve the history of scrape jobs from your ScrapeGraphAI account. Use this to check the status of past scrapes, view results, and track credit usage. | | `SCRAPEGRAPH_AI_GET_SEARCHSCRAPER_HISTORY` | Get Searchscraper History | Get the history of searchscraper jobs with pagination support. Use this to retrieve past searchscraper requests, their status, and results. | | `SCRAPEGRAPH_AI_GET_SITEMAP_HISTORY` | Get Sitemap History | Tool to retrieve the history of sitemap extraction jobs. Use when you need to view past sitemap extraction requests, their status, and results. | | `SCRAPEGRAPH_AI_GET_SMARTSCRAPER_HISTORY` | Get Smartscraper History | Tool to retrieve the history of smartscraper jobs. Use when you need to view past scraping requests and their results. | | `SCRAPEGRAPH_AI_GET_USAGE_TIMELINE` | Get Usage Timeline | Tool to retrieve usage timeline statistics for your ScrapeGraphAI account. Use when you need to visualize or analyze service usage patterns over time. | | `SCRAPEGRAPH_AI_GET_WEBHOOK_LOGS` | Get Webhook Logs | Tool to retrieve webhook delivery logs for a crawler job. Use when you need to check the status and history of webhook notifications sent for a specific crawler execution. | | `SCRAPEGRAPH_AI_LIST_SCHEDULED_JOBS` | List Scheduled Jobs | Retrieve a paginated list of all scheduled scraping jobs for your account. Use this action to view and manage your scheduled jobs, including their configuration, cron schedules, and active status. Supports filtering by service type and active status. | | `SCRAPEGRAPH_AI_MARKDOWNIFY_STATUS` | Markdownify Status | Check the status and retrieve results of a Markdownify webpage-to-Markdown conversion job. Use this action to poll for the status of an async Markdownify request started via SCRAPEGRAPH_AI_MARKDOWNIFY. Note: The ScrapeGraph AI API typically returns completed results synchronously, so this status endpoint is primarily useful for long-running conversions of large or complex webpages. | | `SCRAPEGRAPH_AI_SAVE_ENDPOINT` | Save Endpoint Configuration | Tool to save custom scraping endpoint configurations to ScrapeGraphAI. Use when you need to create reusable scraping endpoints with specific parameters and extraction logic. | | `SCRAPEGRAPH_AI_SEARCH_SCRAPER` | Search Scraper | Perform AI-powered web searches with structured, parsed results. Some sites block scrapers and return empty bodies; treat these as unrecoverable for that URL. JS-rendered pages may yield incomplete content. | | `SCRAPEGRAPH_AI_SEARCH_SCRAPER_STATUS` | Check SearchScraper Status | Check the status and results of an asynchronous SearchScraper job. | | `SCRAPEGRAPH_AI_SMART_CRAWLER_STATUS` | SmartCrawler Status | Check the status and retrieve results of a SmartCrawler web crawling job. Use this action to poll for completion and get the extracted content from a previously started SmartCrawler job. Returns the job status, crawled URLs, page content in markdown/HTML format, and LLM extraction results (if enabled). Implement a polling timeout (e.g., max retries or elapsed time cap) to avoid indefinite loops when waiting for long-running jobs. | | `SCRAPEGRAPH_AI_SMART_SCRAPER_START` | Start Smart Scraper | Start AI-powered web scraping with natural language extraction prompts. When `wait` is false (default), returns a `request_id`; poll for results using SCRAPEGRAPH_AI_SMART_SCRAPER_STATUS. Check `error` and `job_status` fields in the response before using extracted data. | | `SCRAPEGRAPH_AI_SMART_SCRAPER_STATUS` | SmartScraper Status | Check the status and retrieve results of a SmartScraper web scraping job. Use this action to poll for completion after starting a SmartScraper job with wait=false. The request_id is returned by the Start SmartScraper action. Typical workflow: 1. Start a scraping job with SCRAPEGRAPH_AI_SMART_SCRAPER_START (wait=false) 2. Use the returned request_id to check status with this action 3. Poll until status is 'completed' or 'failed' 4. When completed, the 'result' field contains the extracted data. When completed, also check the 'error' field before consuming 'result', as 'failed' status populates 'error' instead of 'result'. | | `SCRAPEGRAPH_AI_START_SMART_CRAWLER` | Start Smart Crawler (Async) | Tool to start a multi-page web crawl using SmartCrawler for AI-powered data extraction. Use when you need to extract structured data from multiple pages of a website. Returns immediately with a task_id - use the status check action to monitor progress and retrieve results. | | `SCRAPEGRAPH_AI_SUBMIT_FEEDBACK` | Submit Feedback | Submit feedback and ratings for completed ScrapeGraphAI requests. | | `SCRAPEGRAPH_AI_SUBMIT_PRODUCT_FEEDBACK` | Submit Product Feedback | Submit product feedback for ScrapeGraphAI. Use to provide ratings, comments, suggestions, and other feedback about the product itself. | | `SCRAPEGRAPH_AI_TOONIFY` | Convert JSON to TOON Format | Tool to convert JSON data to TOON (Token-Oriented Object Notation) format. Use when you need to reduce token usage for LLM processing while maintaining data structure. | | `SCRAPEGRAPH_AI_VALIDATE_API_KEY` | Validate API Key | Validate your ScrapeGraphAI API key to ensure it is active and authorized. Use this action to check API key validity before making other API calls. | ## Supported Triggers None listed. ## Troubleshooting ### Why is my API key rejected? Copy the key directly from the Composio dashboard. Atomic Agent requires ASCII characters because it sends the key in an HTTP header. Remove any extra characters introduced when copying. ### Why are Composio tools unavailable after saving a key? Check that you use Atomic Agent v0.5.6 or later and that Composio is enabled in your configuration. A "composio": { "enabled": false } setting disables the integration while keeping the saved key on disk. ### Does Composio have a free plan? The Hobby plan includes 100,000 tool calls per month. Composio-managed OAuth apps have a lower included allowance. See [Composio pricing](/pricing) for the current limits that apply to your connection. ## Complete Code None listed. ## Conclusion ### Use Scrapegraph ai from Atomic Agent With your account connected, ask Atomic Agent to work with Scrapegraph ai in natural language. You can connect more apps through the same Composio integration as you need them. Visit the [Atomic Agent website](https://atomicagent.io/?cta_placement=atomic-agent-site) or inspect the [Atomic Agent source code](https://github.com/AtomicBot-ai/atomic-agent?cta_placement=atomic-agent-repo). ## How to build Scrapegraph ai MCP Agent with another framework - [OpenAI Agents SDK](https://composio.dev/toolkits/scrapegraph_ai/framework/open-ai-agents-sdk) - [Claude Agent SDK](https://composio.dev/toolkits/scrapegraph_ai/framework/claude-agents-sdk) - [Claude Code](https://composio.dev/toolkits/scrapegraph_ai/framework/claude-code) - [Claude Cowork](https://composio.dev/toolkits/scrapegraph_ai/framework/claude-cowork) - [Codex](https://composio.dev/toolkits/scrapegraph_ai/framework/codex) - [OpenClaw](https://composio.dev/toolkits/scrapegraph_ai/framework/openclaw) - [Hermes](https://composio.dev/toolkits/scrapegraph_ai/framework/hermes-agent) - [Google ADK](https://composio.dev/toolkits/scrapegraph_ai/framework/google-adk) - [LangChain](https://composio.dev/toolkits/scrapegraph_ai/framework/langchain) - [Vercel AI SDK](https://composio.dev/toolkits/scrapegraph_ai/framework/ai-sdk) - [Mastra AI](https://composio.dev/toolkits/scrapegraph_ai/framework/mastra-ai) - [LlamaIndex](https://composio.dev/toolkits/scrapegraph_ai/framework/llama-index) - [CrewAI](https://composio.dev/toolkits/scrapegraph_ai/framework/crew-ai) - [Pydantic AI](https://composio.dev/toolkits/scrapegraph_ai/framework/pydantic-ai) - [AutoGen](https://composio.dev/toolkits/scrapegraph_ai/framework/autogen) ## Related Toolkits - [Excel](https://composio.dev/toolkits/excel) - Microsoft Excel is a robust spreadsheet application for organizing, analyzing, and visualizing data. It's the go-to tool for calculations, reporting, and flexible data management. - [21risk](https://composio.dev/toolkits/_21risk) - 21RISK is a web app built for easy checklist, audit, and compliance management. It streamlines risk processes so teams can focus on what matters. - [Abstract](https://composio.dev/toolkits/abstract) - Abstract provides a suite of APIs for automating data validation and enrichment tasks. It helps developers streamline workflows and ensure data quality with minimal effort. - [AccuWeather](https://composio.dev/toolkits/accu_weather) - AccuWeather is a global weather service providing current conditions, forecasts, MinuteCast precipitation, alerts, and lightning data. Use it for fast, localized forecasts and real-time weather alerts. - [Adafruit IO](https://composio.dev/toolkits/adafruit_io) - Adafruit IO is Adafruit's cloud platform for connected device and IoT project data. It helps you store, visualize, and act on sensor feeds without running your own backend. - [Addressfinder](https://composio.dev/toolkits/addressfinder) - Addressfinder is a data quality platform for verifying addresses, emails, and phone numbers. It helps you ensure accurate customer and contact data every time. - [Agentql](https://composio.dev/toolkits/agentql) - Agentql is a toolkit that connects AI agents to the web using a specialized query language. It enables structured web interaction and data extraction for smarter automations. - [Agenty](https://composio.dev/toolkits/agenty) - Agenty is a web scraping and automation platform for extracting data and automating browser tasks—no coding needed. It streamlines data collection, monitoring, and repetitive online actions. - [Ambee](https://composio.dev/toolkits/ambee) - Ambee is an environmental data platform providing real-time, hyperlocal APIs for air quality, weather, and pollen. Get precise environmental insights to power smarter decisions in your apps and workflows. - [Ambient weather](https://composio.dev/toolkits/ambient_weather) - Ambient Weather is a platform for personal weather stations with a robust API for accessing local, real-time, and historical weather data. Get detailed environmental insights directly from your own sensors for smarter apps and automations. - [Anonyflow](https://composio.dev/toolkits/anonyflow) - Anonyflow is a service for encryption-based data anonymization and secure data sharing. It helps organizations meet GDPR, CCPA, and HIPAA data privacy compliance requirements. - [ApexVerify](https://composio.dev/toolkits/apexverify) - ApexVerify is a contact-data validation service for emails and phone numbers. Use it to normalize, verify, and clean leads before they enter your systems. - [Api ninjas](https://composio.dev/toolkits/api_ninjas) - Api ninjas offers 120+ public APIs spanning categories like weather, finance, sports, and more. Developers use it to supercharge apps with real-time data and actionable endpoints. - [Api sports](https://composio.dev/toolkits/api_sports) - Api sports is a comprehensive sports data platform covering 2,000+ competitions with live scores and 15+ years of stats. Instantly access up-to-date sports information for analysis, apps, or chatbots. - [Apify](https://composio.dev/toolkits/apify) - Apify is a cloud platform for building, deploying, and managing web scraping and automation tools called Actors. It lets you automate data extraction and workflow tasks at scale—no infrastructure headaches. - [Autom](https://composio.dev/toolkits/autom) - Autom is a lightning-fast search engine results data platform for Google, Bing, and Brave. Developers use it to access fresh, low-latency SERP data on demand. - [Beach Day API](https://composio.dev/toolkits/beach_day_api) - Beach Day API is a beach intelligence service for search, conditions, scores, rules, amenities, tides, and country coverage. Use it to power trip planning, surf reports, safety checks, and location-aware beach discovery. - [Beaconchain](https://composio.dev/toolkits/beaconchain) - Beaconchain is a real-time analytics platform for Ethereum 2.0's Beacon Chain. It provides detailed insights into validators, blocks, and overall network performance. - [Big data cloud](https://composio.dev/toolkits/big_data_cloud) - BigDataCloud provides APIs for geolocation, reverse geocoding, and address validation. Instantly access reliable location intelligence to enhance your applications and workflows. - [Bigpicture io](https://composio.dev/toolkits/bigpicture_io) - BigPicture.io offers APIs for accessing detailed company and profile data. Instantly enrich your applications with up-to-date insights on 20M+ businesses. ## Frequently Asked Questions ### What are the differences in Tool Router MCP and Scrapegraph ai MCP? With a standalone Scrapegraph ai MCP server, the agents and LLMs can only access a fixed set of Scrapegraph ai tools tied to that server. However, with the Composio Tool Router, agents can dynamically load tools from Scrapegraph ai and many other apps based on the task at hand, all through a single MCP endpoint. ### Can I use Tool Router MCP with Atomic Agent? Yes, you can. Atomic Agent fully supports MCP integration. You get structured tool calling, message history handling, and model orchestration while Tool Router takes care of discovering and serving the right Scrapegraph ai tools. ### Can I manage the permissions and scopes for Scrapegraph ai while using Tool Router? Yes, absolutely. You can configure which Scrapegraph ai scopes and actions are allowed when connecting your account to Composio. You can also bring your own OAuth credentials or API configuration so you keep full control over what the agent can do. ### How safe is my data with Composio Tool Router? All sensitive data such as tokens, keys, and configuration is fully encrypted at rest and in transit. Composio is SOC 2 Type 2 compliant and follows strict security practices so your Scrapegraph ai data and credentials are handled as safely as possible. --- [See all toolkits](https://composio.dev/toolkits) · [Composio docs](https://docs.composio.dev/llms.txt)