chendefine/dsh-web-search-aggregation

dsh-web-search-aggregation

中文 · npm · GitHub

An aggregated web-search provider for DeepSeek Harness (DSH): the built-in web_search tool is served through one prioritized queue over AnySearch / TinyFish / Tavily / Brave / Exa / Firecrawl / Jina / SerpApi / Serper — each provider can hold a pool of API keys that rotate per request, every failed attempt falls through to the next provider or key, and the first success wins.

Features

• Prioritized queue, ordered fallback — entries are tried top-down per request; the first one that returns wins, and a failed entry falls through to the next. Order is edited live in the settings card; each provider kind can be queued at most once.
• Multi-key pools with rotation — every provider reads exactly one credential (ANYSEARCH_API_KEY / TINYFISH_API_KEY / TAVILY_API_KEY / BRAVE_SEARCH_API_KEY / EXA_API_KEY / FIRECRAWL_API_KEY / JINA_API_KEY / SERPAPI_API_KEY / SERPER_API_KEY) whose value holds all of that provider's keys joined by ,. Within one entry the keys are tried in rotating order (round-robin per successful request), spreading load and quota across the pool.
• Works out of the box — AnySearch allows anonymous access, so the queue answers searches before any key is configured. The shipped defaults simply enable all nine kinds — the order is whatever you arrange in the card; the key-required entries fall through until their credential is set.
• Budget-aware — each attempt gets its own deadline (default 10 s, 1–60 s), so one hung upstream cannot eat the tool-level budget; five to six full fallbacks still fit inside 60 s.
• Transparent failures — when every attempt fails, the error reports each one ([2] tavily/TAVILY_API_KEY#1: 401 unauthorized; …). An empty queue raises WEB_PROVIDER_UNAVAILABLE; caller cancellation raises WEB_ABORTED immediately.
• Secret hygiene — key literals are never logged and never appear in failure records (which cite REF / REF#N labels only); the settings card shows keys as masked tags and never echoes stored values back.
• Live configuration — the Plugins page (侧边栏 → 插件 → dsh-web-search-aggregation, dsh ≥ 0.1.7) shows the full form right on the plugin's package page (and on the row's own page via its configure control); it edits the queue, keys, endpoints, and timeout, and a committed change reaches the next search without a restart.

How it works

web_search (tool-web)
   └─ ctx.web.searchProvider = aggregated
        ├─ queue (top-down, first success wins — order user-arranged):
        │     anysearch · firecrawl · tavily · tinyfish · brave · exa · jina · serpapi · serper
        │       │   keys = credential split on ','   ← pool per provider
        │       │   anonymous attempt allowed for key-less AnySearch
        │       └─ rotation cursor per entry (resets when the endpoint changes)
        ├─ per attempt: adapter request under its own deadline (default 10 s)
        └─ all failed → WEB_PROVIDER_ERROR with a per-attempt summary

Nine adapters ship today; another upstream is one adapter module plus one registry row.

Requirements

• DSH web profile (dsh web), Node.js ≥ 22.19 — the same floor as DSH itself (^22.19.0 || >=24). Source installs from GitHub run the prepare build, which needs the same range.
• At least one reachable upstream: the default queue works with no credentials (AnySearch anonymous); Tavily / TinyFish / Brave / Exa / Firecrawl / Jina / SerpApi / Serper entries need their API keys to contribute.

Installation

From the npm registry (prebuilt — no build permission needed):

dsh plugin --profile web add dsh-web-search-aggregation

From a GitHub repository (source — pnpm runs the prepare build; allowlist the package in profiles/web/pnpm-workspace.yaml if pnpm blocks the build script):

dsh plugin --profile web add github:chendefine/dsh-web-search-aggregation

Or through the DSH plugin marketplace (设置 → DSH插件市场) — the repo carries the dsh-plugin topic and is indexed automatically.

After a bundle plugin is added to the profile layer stack, restart dsh web for it to load; uninstall with dsh plugin --profile web remove dsh-web-search-aggregation and restart again.

Configuration

The Plugins page (侧边栏 → 插件) opens the form one click from the list: the dsh-web-search-aggregation card's 查看 lands on the package page with the full form between its description and its rows; the row's own 配置 control opens the same form under the plugin's title. It edits the web-search-aggregation settings section — derived from the plugin entry's Config schema — live:

API keys are managed on each entry as masked tags: add one key at a time (+ or Enter), reorder by dragging tags off/on — tag order is the order a save writes and the runtime reads. Stored keys are never read back; a save replaces the whole pool, and closing every tag before saving clears the credential. Each provider's keys live in one fixed credential:

Combining with other provider plugins

A bundle patch replaces the web seam row's whole config, so the layer applied last determines the final searchProvider / fetchProvider pair. This plugin's patch states only searchProvider: aggregated — it owns the search selection and nothing else. Two consequences:

• With dsh-web-fetch-playwright installed before this plugin (or without it entirely), nothing needs adjusting: an unconfigured fetchProvider resolves to the single registered fetch provider automatically.

• If another plugin's layer is applied after this one and resets searchProvider, pin the combined selection in your profile's own profiles/web/cordis.patch.yml — the user layer wins over every bundle layer:

  - id: web
    config:
      searchProvider: aggregated
      fetchProvider: playwright
  

Development

pnpm install
pnpm typecheck   # tsc --noEmit
pnpm test        # vitest run (103 unit tests, no network)
pnpm build       # tsc declarations + tsdown (host ESM + client module-registration bundle)

Repository layout:

src/
├── index.ts               # host entry: registers provider + settings section
├── config.ts              # schemastery schema, defaults, queue normalization
├── provider.ts            # AggregatedSearchProvider: queue walk, rotation, deadlines
├── keys.ts                # ','-joined key-pool vocabulary (parse/format/mask)
├── types.ts               # queue entry, config, per-attempt failure records
├── adapters/              # AnySearch / Tavily / TinyFish / Brave / Exa / Firecrawl / Jina / SerpApi / Serper adapters + shared HTTP
└── client/                # browser half: settings card, form model, locales
tests/                     # config, keys/adapters, provider, client-controller

See CONTRIBUTING.md for the development and release workflow, and SECURITY.md for the security model and reporting policy.

Security

Search queries and API keys leave the machine only toward the endpoints configured per queue entry (the nine providers' official APIs by default). Keys are stored in the DSH credentials domain — never in the settings file, never logged, never echoed back to the client. Failure records and logs cite masked references (TAVILY_API_KEY#2), not literals. No result data is retained beyond the session that requested it.

License

MIT © 2026 chendefine