JohnXu22786/browser-automation

简体中文

Web Bridge — Browser Automation MCP Server (dsh plugin)

Web Bridge is an automation MCP server that uses a real browser. It gives agents a browser's eyes and hands: open web pages, read page structure, click, fill forms, take screenshots, and execute scripts. Its core mechanism is the accessibility-tree snapshot — rendering the page into a structured text tree with numbered interactive elements, so agents can interact with pages precisely without a vision model.

• Language: TypeScript (Node.js ≥ 20), communicating over the MCP protocol (stdio transport)
• Browser engines: Chromium / Firefox / WebKit (Chromium by default)
• Tool count: 22, all named with the web_ prefix

Features

• Snapshot driven: web_snapshot produces an accessibility tree with reference numbers (ref), so elements can be clicked/filled precisely without guessing selectors
• Real interactions: click, double-click, right-click, key combos, form filling, dropdown selection, hover, scroll, keyboard input
• Visual capabilities: viewport / full-page / element screenshots (PNG/JPEG), returned to the caller or saved locally
• Script execution: run JS in the page context; results are sanitized (circular references, BigInt, NaN, etc. safely converted)
• Multi-tab: create, list, switch, and close tabs; popups are captured automatically
• Fine-grained waiting: wait by load state / element state / URL match / fixed duration
• Session management: lazy browser startup, web_shutdown releases resources, automatic cleanup on disconnect

Installation

npm install          # install dependencies
npm run build        # compile to dist/
npx playwright install chromium   # install browser engine (required on first use)

Firefox / WebKit engines are also supported: npx playwright install firefox, switchable via configuration.

Quick Start

Option A: generic MCP client (stdio)

{
  "mcpServers": {
    "web-bridge": {
      "command": "node",
      "args": ["D:/path/to/browser-automation/dist/index.js"]
    }
  }
}

Option B: command line verification

node dist/index.js --help        # view usage
echo '{"jsonrpc":"2.0","id":1,"method":"initialize","params":{"protocolVersion":"2025-03-26","capabilities":{},"clientInfo":{"name":"probe","version":"1"}}}' | node dist/index.js

Option C: load as a dsh plugin (see next section)

Installing in DSH

dsh plugin --profile demo add github:JohnXu22786/browser-automation

Adds this plugin to the dsh demo profile from github:JohnXu22786/browser-automation. See the next section for authorization, loading, and lifecycle details.

Cordis bundle (dsh.bundle)

As an alternative to the dsh-plugin.json MCP-server manifest, the repo ships a Cordis bundle for hosts that consume dsh.bundle manifests: package.json declares dsh.bundle.patch → cordis.patch.yml, and index.js is the bridge a dsh profile loads. The bridge spawns the built MCP server (node dist/index.js) over stdio, performs the MCP handshake, and re-exposes the 22 web_* tools to the harness — the server itself is untouched.

In a source checkout dist/ is gitignored, so the bridge runs npm run build once on first load; installed npm packages ship dist/ in their files list and need no build step. Profile-level settings map to the same WEB_BRIDGE_* environment variables described below.

dsh Plugin Integration

This plugin follows the dsh "everything is a plugin" convention and describes itself via the dsh-plugin.json manifest in the root directory:

Harness loading flow (the dsh runtime integrates the plugin as follows):

1. Scan the plugin directory, read dsh-plugin.json, validate kind / entry / runtime;
2. Check that the Node version satisfies runtime.minVersion;
3. Spawn the plugin process with spawn('node', [entry]), connecting stdin/stdout to the MCP protocol (JSON-RPC 2.0, line-delimited);
4. Map harness config to environment variables injected into the child process according to config.envPrefix (e.g. actionTimeout → WEB_BRIDGE_ACTION_TIMEOUT);
5. After the MCP client handshake (initialize / notifications/initialized / tools/list), the tools in the tools list enter the agent's tool set automatically; each tool's schema and permission description are provided by the plugin in the tools/list response;
6. When the session ends or the process exits, the harness closes stdin and the plugin recycles the browser and exits automatically (see "Lifecycle").

Any standard MCP client (that does not support manifest) can also connect directly as in Option A.

Configuration

Priority: defaults < config file (--config or WEB_BRIDGE_CONFIG) < environment variables.

Environment variables (prefix WEB_BRIDGE_)

Config file

{
  "browser": { "name": "chromium", "headless": true, "viewport": { "width": 1280, "height": 720 } },
  "context": { "ignoreHTTPSErrors": false, "permissions": ["geolocation"] },
  "timeouts": { "action": 10000, "navigation": 60000, "settle": 500 },
  "sandbox": true,
  "testIdAttribute": "data-testid"
}

Start: node dist/index.js --config web-bridge.config.json.

Tool List

Full tool descriptions and permission notes are returned by MCP tools/list; key points are listed here.

Navigation

Interaction

Observation

Scripting and viewport

Tabs and session

Snapshot and ref Mechanism

web_snapshot renders the page as an indented structured text, for example:

title: Demo Home
url: http://localhost:8080/index.html
tree:
  banner
    navigation
      [1] link "首页"
      [2] link "文档"
  main
    heading "欢迎来到演示站点" (level: 1)
    form
      [3] textbox "用户名" (placeholder: "请输入用户名")
      [4] checkbox "记住我" (unchecked)
      [5] button "提交表单"

• [n] is a reference number (ref), assigned only to interactive elements (buttons/links/inputs/dropdowns/checkboxes, etc.);
• Subsequent web_click / web_fill / web_select calls can locate elements by ref (ref: 3), or by selector (selector: "#username", text=keyword, testid=value);
• The snapshot shows states: (checked/unchecked), (disabled), (selected), (expanded/collapsed), (password), (value: ...), (placeholder: ...);
• The value of password boxes is never shown;
• The selector parameter snapshots only a subtree, max_depth limits depth, and max_nodes limits the total node count (protects against token bombs).

ref is a positional path (determined by the DOM structure): after navigation, old refs automatically become invalid (reporting an error and asking for a re-snapshot); if the page script adds/removes DOM without navigation, old refs may resolve to other elements — re-snapshot if an operation fails.

Wait Strategies

web_open's wait_until: commit (request sent) → domcontentloaded (DOM ready) → load (resources loaded) → networkidle (network idle). Default is load.

Actions wait for settleMs (500ms) by default so async tasks (navigation, requests) settle.

Permissions and Security Notes

• This plugin is not a security boundary. web_evaluate can run arbitrary scripts, web_open can access any site, and save_to can write to any path — only use it in trusted environments, and configure per-tool permissions at the harness layer.
• Prefer least privilege: use web_snapshot for routine observation (zero network requests, zero side effects), web_screenshot when visual confirmation is needed, and only consider web_evaluate as a last resort.
• The browser process is managed by the plugin: client disconnect, web_shutdown, and SIGINT/SIGTERM all recycle the browser process.
• Headless mode is on by default; set WEB_BRIDGE_HEADLESS=false for visual debugging.

Known Limitations

• Shadow DOM / iframe content is not in snapshots: the walker only covers the regular DOM subtree of the main document; if the page hosts interactive elements inside shadow DOM, use web_evaluate or the page's own test hooks (testid=).
• ref is a positional path: see "Snapshot and ref Mechanism" above.
• Popups activate automatically: new tabs opened via target=_blank etc. automatically become the active page (the agent's intuitive click behavior); use web_tab_list + web_tab_switch to get other tabs back.
• The statement-sequence form of web_evaluate (multi-statement including function declarations) executes for side effects and returns no value; expression and function forms return results. Use the async function form for async logic.

Development and Testing

npm run build          # compile with tsc to dist/
npm test               # all tests (unit + browser integration + MCP protocol chain)
npm run test:unit      # unit tests only (config/tool functions/snapshot format/walker)
npm run test:integration  # browser integration tests only

Test coverage: config parsing, walker behavior, snapshot format, result sanitization, 16+ real browser integration scenarios (navigation/click/form fill/screenshot/tabs/history/wait/scroll/failure paths), and a real stdio process chain (including browser-recycle assertions after disconnect).

Project Structure

src/
  index.ts       entry: CLI args, config loading, startup of the MCP server
  config.ts      config (three-level default/file/env merge and validation)
  server.ts      MCP server assembly, error mapping, lifecycle cleanup
  browser.ts     browser session: lazy start / tabs / ref table / shutdown
  walker.ts      page-side accessibility-tree walker (single source shared by Node tests and the page)
  snapshot.ts    snapshot orchestration, ref assignment, text formatting
  scripting.ts   script execution and result sanitization
  locators.ts    ref/selector resolution
  util.ts        argument validation, URL validation, timeout error classification
  tools/         22 tool implementations (grouped by responsibility)
  registry.ts    tool registry
index.js          dsh Cordis bundle bridge (spawns dist/index.js over stdio)
cordis.patch.yml  dsh bundle install row (id: web-bridge, name: web-bridge-mcp)
dsh-plugin.json  plugin manifest (see README for dsh harness integration)
test/           tests and fixtures

License

MIT