The Spider MCP server is an implementation of the Model Context Protocol that connects your AI agent and assistants like Claude, Cursor, etc directly to your Spider account. It provides structured and secure access to web crawling, scraping, search, screenshots, content transformation, and AI extraction tools, so your agent can crawl sites, scrape pages, search the web, extract structured data, and transform or capture web content on your behalf.
- AI-powered website extraction: Have your agent crawl websites or scrape specific pages, then extract structured information from them using natural-language instructions.
- Web search and discovery: Direct your agent to search the web, fetch relevant result pages, or discover links that match a specific description across a site.
- Browser-based research: Let your agent use the AI browser to navigate websites from a natural-language prompt and return the requested results.
- Page screenshots: Instruct your agent to capture a rendered webpage as a PNG, JPEG, or WebP image.
- Content transformation: Have your agent convert supplied HTML into markdown, text, or cleaned output without fetching a URL.