How to integrate Diffbot MCP with Hermes

GET STARTED FOR FREE
Diffbot logo
Hermes logo
divider

Introduction

Hermes is a 24/7 autonomous agent that lives on your computer or server — it remembers what it learns and evolves as your usage grows.

This guide explains the easiest and most robust way to connect your Diffbot account to Hermes. You can do this through either Composio Connect CLI or Composio Connect MCP. For personal use we recommend the CLI, but you won't go wrong with MCP either.

Also integrate Diffbot with

What is Composio Connect?

Composio Connect is a consumer offering that lets anyone plug 1,000+ applications directly into their agent harness — including Hermes. It can:

  • Search and load tools from relevant toolkits on-demand, reducing context usage.
  • Chain multiple tools to accomplish complex workflows via a remote workbench, without excessive back-and-forth with the LLM.
  • Manage app authentication end-to-end with zero manual overhead.

Integrating Diffbot with Hermes

Using Composio Connect CLI

1. Install the Composio CLI

Run the install script directly, or paste https://composio.dev/hermes into your Hermes chat box to have it installed for you.

bash
curl -fsSL https://composio.dev/install | bash
Hermes authenticating with Composio

2. Authenticate

Once the CLI is installed, ask Hermes to authenticate with Composio.

3. Connect to Diffbot

Ask your agent to connect to Diffbot, or simply request any Diffbot-related task. Hermes will prompt you to authenticate and authorize access.

4. Done. You're all set with a new Diffbot connection.

Using Composio Connect MCP

1. Get your MCP URL and API Key

Go to dashboard.composio.dev and copy your Connect MCP URL and API key.

Copy MCP URL and API key from Composio dashboard

2. Open the Hermes config file

bash
nano ~/.hermes/config.yaml

3. Add the Composio Connect MCP server

bash
mcp_servers:
  composio:
    url: "https://connect.composio.dev/mcp"
    headers:
      x-consumer-api-key: "YOUR_COMPOSIO_API_KEY"
    connect_timeout: 60
    timeout: 180

Save with Ctrl + O, Enter, then exit with Ctrl + X.

4. Restart your Hermes agent

Once restarted, ask your agent to connect to Diffbot or request any Diffbot-related task. It will prompt you to authenticate and authorize access.

5. Done!

What is the Diffbot MCP server, and what's possible with it?

The Diffbot MCP server is an implementation of the Model Context Protocol that connects your AI agent and assistants like Claude, Cursor, etc directly to your Diffbot account. It provides structured and secure access to web data extraction and analysis, so your agent can extract structured data from web pages, analyze content types, retrieve product details, manage bulk jobs, and search extracted datasets on your behalf.

  • Automatic content analysis and extraction: Let your agent analyze any web page and automatically extract structured data such as articles, products, events, images, or videos using AI-powered tools.
  • Article and discussion thread extraction: Effortlessly pull detailed metadata, authors, publication dates, and full discussion threads from news sites, blogs, forums, and comment sections.
  • Product and event data gathering: Instantly extract comprehensive product specifications, pricing, reviews, and event information including venues, dates, and descriptions from e-commerce or event pages.
  • Bulk job management and search: Enable your agent to list, monitor, and search across large-scale crawl or extraction jobs, making it easy to work with massive web data collections.
  • Account and usage insights: Retrieve your Diffbot account details, plan information, and usage statistics to stay on top of quotas and manage your web data operations efficiently.

Supported Tools & Triggers

Tools
Diffbot SearchTool to search data extracted by crawl or bulk jobs using dql queries.
Get Diffbot Account DetailsTool to retrieve account details, including plan information and usage statistics.
Diffbot AnalyzeTool to automatically determine a page's content type and route it to the appropriate extraction api.
Get Article DataTool to extract information from articles, including authors, publication dates, and images.
Get Discussion ThreadTool to extract threads of content from forums, comment sections, and review pages.
Diffbot Get EventTool to extract event details from web pages.
Diffbot Get ImageTool to extract detailed information about images, including dimensions and recognition data.
Diffbot Get ProductTool to extract product information such as specifications, prices, availability, and reviews.
Get Video DataTool to extract information from videos, including titles, descriptions, and embedded html.
List Bulk JobsTool to list all bulk jobs associated with a specific token.
Resolve Lost IDTool to resolve lost ids in the knowledge graph.
Start Bulk JobTool to start a bulk extract job.
Start Crawl JobTool to spider a site for links and process them with the extract api into a single collection.
Stop Bulk JobTool to stop a running bulk job.

Way Forward

With Diffbot connected, Hermes can now act on your behalf whenever it detects a relevant task or you ask it to.

From here, you can extend Hermes further:

  • Connect more apps: Calendar, Slack, Notion, Linear, and hundreds of others are available through the same Composio Connect setup. Each new integration compounds what Hermes can do for you.
  • Build workflows across tools: Once multiple apps are connected, Hermes can chain actions together — turn an email into a calendar invite, a Slack message into a Linear ticket, or a meeting note into a follow-up draft.
  • Let it learn your patterns: The more you use Hermes, the better it gets at anticipating how you'd handle recurring tasks. Give it feedback on drafts and decisions, and it will adapt.

If you run into trouble or want to share what you've built, join the community or check out the Docs for deeper configuration options.

How to build Diffbot MCP Agent with another framework

OpenAI Agents SDK logo

OpenAI Agents SDK

Use Diffbot MCP with OpenAI Agents SDK

Claude Agent SDK logo

Claude Agent SDK

Use Diffbot MCP with Claude Agent SDK

Claude Code logo

Claude Code

Use Diffbot MCP with Claude Code

Codex logo

Codex

Use Diffbot MCP with Codex

OpenClaw logo

OpenClaw

Use Diffbot MCP with OpenClaw

CLI logo

CLI

Use Diffbot MCP with CLI

Google ADK logo

Google ADK

Use Diffbot MCP with Google ADK

LangChain logo

LangChain

Use Diffbot MCP with LangChain

Vercel AI SDK logo

Vercel AI SDK

Use Diffbot MCP with Vercel AI SDK

Mastra AI logo

Mastra AI

Use Diffbot MCP with Mastra AI

LlamaIndex logo

LlamaIndex

Use Diffbot MCP with LlamaIndex

CrewAI logo

CrewAI

Use Diffbot MCP with CrewAI

FAQ

What are the differences in Tool Router MCP and Diffbot MCP?

With a standalone Diffbot MCP server, the agents and LLMs can only access a fixed set of Diffbot tools tied to that server. However, with the Composio Tool Router, agents can dynamically load tools from Diffbot and many other apps based on the task at hand, all through a single MCP endpoint.

Can I use Tool Router MCP with Hermes?

Yes, you can. Hermes fully supports MCP integration. You get structured tool calling, message history handling, and model orchestration while Tool Router takes care of discovering and serving the right Diffbot tools.

Can I manage the permissions and scopes for Diffbot while using Tool Router?

Yes, absolutely. You can configure which Diffbot scopes and actions are allowed when connecting your account to Composio. You can also bring your own OAuth credentials or API configuration so you keep full control over what the agent can do.

How safe is my data with Composio Tool Router?

All sensitive data such as tokens, keys, and configuration is fully encrypted at rest and in transit. Composio is SOC 2 Type 2 compliant and follows strict security practices so your Diffbot data and credentials are handled as safely as possible.

Used by agents from

Context
Letta
glean
HubSpot
Agent.ai
Altera
DataStax
Entelligence
Rolai
Context
Letta
glean
HubSpot
Agent.ai
Altera
DataStax
Entelligence
Rolai
Context
Letta
glean
HubSpot
Agent.ai
Altera
DataStax
Entelligence
Rolai

Never worry about agent reliability

We handle tool reliability, observability, and security so you never have to second-guess an agent action.

Book a call↗Get started for free