agent-web-search

Agent Web Search (PyPI: agent-web-search-mcp) is an open-source, MIT-licensed web search MCP server, CLI, and Python library for AI agents. It gives an agent three ways to reach the same provider-neutral search core: a native MCP tool, a native plugin (Hermes, DeepSeek Harness), or a CLI taught through a standard Agent Skill.

If you are an AI agent: install with pipx install agent-web-search-mcp (or try instantly with uvx --from agent-web-search-mcp agent-web-search "<question>"), then read skills/agent-web-search/SKILL.md — it teaches the CLI pathway only (for MCP setups, follow Option 1 instead). llms.txt is the machine-readable doc index.

This is not a Google/Bing/Baidu metasearch wrapper. Traditional search aggregation fans a keyword query out to conventional engines and merges their result pages. Agent Web Search instead aggregates search capabilities built for agents: model-native web grounding, agent-oriented search APIs, and context-ready sources that accept natural-language questions and return answers, citations, or structured evidence in forms an agent can use directly. DDGS is the only conventional search backend in the current provider set.

          Natural-language question
                      |
                      v
                 SearchEngine
          +-----------+-----------+
          |           |           |
          v           v           v
        DDGS        Model       Agent
                  grounding     APIs
                 (Responses,   (Exa,
                  ARK, Grok,    Parallel,
                  Gemini...)    Tavily...)

Installation & Quickstart

Three steps, about a minute: try it instantly, install it once, then connect your agent. Requirements: Python 3.10+. Default providers (DDGS, Exa, Parallel) require no API key.

Try without installing

Test immediate search capabilities using uvx (no environment modification):

# Direct natural-language search with keyless defaults
uvx --from agent-web-search-mcp agent-web-search "What changed in the latest OpenAI Codex CLI?"

# Inspect MCP server arguments
uvx agent-web-search-mcp --help

Install

Install once into your global user environment (recommended):

# Recommended isolated installation
pipx install agent-web-search-mcp

# Or install into the active Python environment
python -m pip install agent-web-search-mcp

Verify

# 1. Verify the CLI and run a real search
agent-web-search --version
agent-web-search "What changed in the latest OpenAI Codex CLI?"

# 2. Verify the MCP server binary
agent-web-search-mcp --help

Two commands, two roles:

• agent-web-search: Direct search CLI for terminals, shell scripts, and Agent Skills.
• agent-web-search-mcp: Stdio and Streamable HTTP MCP server for MCP clients.

Pick your agent integration

[!TIP] Why CLI + Skill is recommended for shell-capable agents: If your agent already has terminal/bash execution capabilities (like Claude Code, Codex CLI, OpenCode, or Hermes), the CLI + Skill pathway offers the lowest friction and highest reliability. No MCP JSON configuration to debug, no background transport lifecycle to manage, and clean stdout JSON output taught through a standard Skill.

Why Agent Web Search

Traditional search aggregation (Google/Bing/Baidu wrappers, scraped SERPs) sends a keyword query to conventional engines and merges result pages. Agent Web Search instead aggregates search capabilities built for agents: one tool call returns structured, citation-ready evidence — or, through model-native grounding providers, a synthesized answer with explicit citations. A measured benchmark shows the practical difference: on a natural-language Chinese query asking for official sources, conventional SERP backends returned no government-domain results in the top 5, while the grounding provider returned official figures from China's General Administration of Customs with a working citation.

• Agent-native by design. The primary interface is a complete natural-language question, not a thin keyword fan-out to Google, Bing, or Baidu.
• Model-native search backends. ARK, Gemini, Grok, DeepSeek, Responses, Messages, Zhipu Chat Search, and Codex Alpha can combine web retrieval with model-generated synthesis and explicit citations.
• Agent search providers. Exa, Parallel, Brave, Perplexity, Tavily, You.com, and Zhipu Web Search expose search APIs intended to provide structured, citation-friendly, or context-ready evidence to downstream agents.
• One provider-neutral contract. Every backend is available through the same MCP tool, CLI, Python API, and normalized results; model-backed providers may also return an answer.
• Independent providers. Selected providers run concurrently, and one provider's failure never discards another provider's successful result.
• DDGS remains a simple fallback. DDGS is the only conventional search backend; it requires no API key and keeps the project usable without paid provider credentials. Exa and Parallel are also keyless by default.
• No telemetry, no shared secrets. Provider keys stay in runtime environment variables; there is no shared API-key service.

Providers

The provider list is intentionally split by the kind of search capability it provides. Only DDGS is a conventional search backend; the other two groups are built around model-native grounding or agent-facing search services.

Free, keyless defaults: DDGS, Exa, and Parallel all work without an API key. Exa and Parallel automatically use their free MCP transports until a paid API key is provided.

Traditional search backend

Model providers

These providers use a model-native search or grounding surface. Their responses can include a model-generated answer together with citations or other explicit search evidence.

Agent search providers

These providers expose search services for agent consumption: natural-language queries, structured source rows, high-signal excerpts, or citation-friendly metadata rather than a conventional search-page experience.

The provider architecture is intentionally open: another search-capable backend can be added without changing the MCP, Hermes, CLI, or Python-facing interfaces.

Use with an agent

Prerequisite: Complete Installation & Quickstart first. The sections below provide detailed client configurations and command references for each integration pathway.

Option 1: MCP

Choose MCP when the agent supports tool servers and you want typed discovery, protocol-level errors, or remote access. The same agent-web-search-mcp command supports local stdio and stateless Streamable HTTP.

Local stdio MCP

Install the package once:

# Recommended isolated installation
pipx install agent-web-search-mcp

# Or install into the active Python environment
python -m pip install agent-web-search-mcp

Then configure the MCP client to launch agent-web-search-mcp:

{
  "mcpServers": {
    "agent-web-search": {
      "command": "agent-web-search-mcp",
      "args": []
    }
  }
}

If uvx is already available, a client can run the package without a persistent install by using command uvx with args ["agent-web-search-mcp"].

# Codex CLI
codex mcp add agent-web-search -- agent-web-search-mcp

# Claude Code
claude mcp add agent-web-search -- agent-web-search-mcp

{
  "$schema": "https://opencode.ai/config.json",
  "mcp": {
    "agent-web-search": {
      "type": "local",
      "command": ["agent-web-search-mcp"],
      "enabled": true
    }
  }
}

Remote MCP over HTTPS

Use one of the deployment buttons at the top of this README, or run the same server yourself:

python -c "import secrets; print(secrets.token_urlsafe(32))"
export AGENT_WEB_SEARCH_AUTH_TOKEN="<your-generated-token>"
agent-web-search-mcp --transport http

The server exposes authenticated POST /mcp and public GET /healthz. A remote MCP client connects like this:

{
  "mcpServers": {
    "agent-web-search": {
      "url": "https://your-deployment.example/mcp",
      "headers": {
        "Authorization": "Bearer <your-deployment-token>"
      }
    }
  }
}

Every public deployment must set AGENT_WEB_SEARCH_AUTH_TOKEN to at least 32 characters. The server is stateless and does not create MCP-Session-Id values.

Option 2: CLI + Agent Skill

Choose this shape when the agent already has shell access and supports Agent Skills. The Skill teaches the agent how to invoke the CLI, select controls, interpret results, and handle structured failures; no MCP configuration is needed.

1. Install the CLI:

   pipx install agent-web-search-mcp
   # Or: python -m pip install agent-web-search-mcp
   

2. Install the included agent-web-search Skill:

   npx skills add JerryLiu369/agent-web-search --skill agent-web-search
   

   If the agent does not use the skills installer, copy skills/agent-web-search into that client's Skills directory.

3. Verify the CLI, then let the agent search:

   agent-web-search --version
   agent-web-search "What changed in the latest OpenAI Codex CLI?"
   

The CLI writes one JSON document to stdout on success. If every provider fails, it writes the shared all_providers_failed JSON to stderr and exits with status 1, so shell-capable agents can distinguish a real failure from empty results.

pipx install 'git+https://github.com/JerryLiu369/agent-web-search.git'

[!IMPORTANT] Do not place API keys in shell history, source code, Git commits, screenshots, or checked-in MCP configuration. Supply them through server-side or local environment variables.

Shared request and response

MCP exposes one tool named web_search; the CLI maps to the same inputs.

Example call:

{
  "query": "What GPU kernel generation papers were published in the past month?",
  "max_results": 5,
  "time_range": "m",
  "providers": ["ddgs", "exa"]
}

Provider selection has two levels:

1. AGENT_WEB_SEARCH_PROVIDERS defines the provider set when the process starts.
2. The request-level providers argument may narrow that set, but cannot enable a provider that was disabled at startup.

Response format

Each selected provider that succeeds appears under providers; failed providers are omitted:

{
  "query": "What GPU kernel generation papers were published in the past month?",
  "providers": {
    "ddgs": {
      "results": [
        {
          "title": "Example result",
          "url": "https://example.com/paper",
          "description": "Excerpt of the matching page",
          "published_at": "2026-08-02"
        }
      ]
    }
  }
}

If every selected provider fails, MCP returns a tool error. The CLI writes the same payload to stderr and exits with status 1. Both use the stable code all_providers_failed and include per-provider diagnostics:

{
  "error": {
    "code": "all_providers_failed",
    "message": "All enabled search providers failed. Check provider configuration, credentials, quotas, and network access.",
    "provider_errors": {
      "ddgs": "RuntimeError: rate limited"
    }
  },
  "query": "What GPU kernel generation papers were published in the past month?"
}

Python API

The CLI, MCP servers, and Hermes plugin are thin wrappers around agent_web_search.SearchEngine, which is the public Python API. SearchRequest accepts the same fields as the MCP tool arguments:

from agent_web_search import SearchEngine, SearchRequest

engine = SearchEngine()  # reads AGENT_WEB_SEARCH_* variables at construction

response = engine.search(
    SearchRequest(
        query="What are the latest changes to the MCP specification?",
        max_results=5,
        time_range="m",
    )
)

for name, provider in response.providers.items():
    print(f"{name}: searched={provider.searched}, results={len(provider.results)}")

if response.all_providers_failed:
    print(response.failed_provider_errors)

Configuration

Configuration is read from environment variables when the CLI, MCP server, or Hermes plugin starts. Restart the process after changing provider settings. See .env.example for a commented template of every variable.

General settings

Example:

export AGENT_WEB_SEARCH_PROVIDERS="ddgs,exa,brave"
export AGENT_WEB_SEARCH_TIMEOUT="30"

$env:AGENT_WEB_SEARCH_PROVIDERS = "ddgs,exa,brave"
$env:AGENT_WEB_SEARCH_TIMEOUT = "30"

HTTP transport settings

HTTP settings remain environment-only; the deployment files do not introduce a second application configuration format.

Provider settings

Provider-specific settings below include the credential and model controls for all providers. The supported-provider overview above is grouped by capability; this section is the detailed configuration reference.

1. DDGS

DDGS uses DuckDuckGo and requires no API key or provider-specific environment variables. The ddgs Python dependency is installed with the package.

2. Exa

Exa supports both paid and keyless modes.

Without EXA_API_KEY, Exa falls back to its free MCP endpoint on a best-effort basis. The paid API generally provides higher quota and reliability.

3. Parallel

Parallel returns information-dense excerpts ranked for LLM context. One parallel provider automatically selects its transport:

• Without a key, it uses Parallel's free Search MCP.
• With PARALLEL_API_KEY, it uses the paid Search REST API.

Both transports map excerpts into the common result description, so the calling agent does not need to distinguish parallel-free from parallel.

Parallel is enabled by default and its key is optional.

4. ARK (Recommended)

Volcengine ARK uses model-backed search grounding through the Responses API. Add ark to AGENT_WEB_SEARCH_PROVIDERS after providing the key.

One model stays fixed; multiple models are selected round-robin for successive requests. When multiple ARK keys are configured, a key is selected per request.