dsh-browser-playwright-codex

dsh-browser-playwright-codex

一句话：让 AI 用你已经登录的账号在真实网页上干活 —— 填表、抓取列表、多标签查后台；浏览器窗口就在你桌面上、你随时看得见。原生弹框会被挂起、绝不替你点「确定」（机制级）；验证码/挑战页会被识别出来，并以提示要求模型停下交给你（提示不是阻断）。登录/二次验证没有探测器，靠模型从快照看出来并停下；密码、一次性验证码、卡号字段 browser_fill 直接拒绝——这类字由你本人在窗口里输。

In one line: the agent drives a visible, profile-backed browser window with the logins you already have — filling in forms, scraping lists and working across tabs in real pages — through accessibility snapshots with stable element refs. Native dialogs are parked and reported instead of auto-accepted. Challenge pages are detected and reported as a note that tells the agent to stop and hand the page back — a note, not a block. There is no sign-in detector, and browser_fill refuses password / one-time-code / card fields outright — you type those in the visible window. No CSS-selector guessing, no full-DOM dumps.

# from npm
dsh plugin --profile <name> add dsh-browser-playwright-codex

# or straight from the repository
dsh plugin --profile <name> add github:miqian-nomad/dsh-browser-playwright-codex

为什么值得装 / What makes it different

• 登录一次，一直用 — a profile-backed window plus an exported session state, so a closed window or

a restarted harness comes back logged in.

• 全程看得见，也不抢你的焦点 — a real window you can watch, minimise or take over; the provider

keeps working while it is minimised (FOCUS-STEALING.md).

• 遇事停下来叫人 — alert / confirm / prompt are parked and reported, answered only by an

explicit browser_dialog call (DIALOG-POLICY.md). A challenge page is recognised from the interceptor's own response headers first (cf-mitigated: challenge and friends; x-iinfo / x-datadome* additionally need a blocking status), then from the vendor widget's accessible name, then from wording; if none of those match, a page that stops changing after an action is reported as a possible wall instead of being retried. Recognition adds a note that tells the agent to stop and hand the window back to you — that note is advice to the model, not a hard block; the mechanism-level guarantee exists for native dialogs only (docs/articles/dialogs-and-captcha.md).

• ref 失效会明说 — per-document nonce refs; a stale ref fails fast with an explanation instead of

silently clicking a different element.

• 强力工具默认关闭 — browser_evaluate / browser_cdp are not registered at all unless your

deployment opts in; this package ships both off.

安全一览 / Security at a glance — dsh.so's scanner flags this repository's one Critical finding, and that finding is the implementation of browser_evaluate (its dynamic evaluation), which this package ships switched off. SECURITY.md says what each finding actually is, whether a capability gate makes it reachable (two of the warnings are not gate-controlled: cannot do with your logins (URL policy, CDP allow-list, dialog policy, no telemetry, and browser_fill refuses password / one-time-code / card fields). Counts move as the scanner and the code change — their own report is the passwords or 2FA codes). Counts move as the scanner and the code change — their own report is the live number.

Install

From npm (published since v0.5.1 — npmjs.com/package/dsh-browser-playwright-codex):

dsh plugin --profile <name> add dsh-browser-playwright-codex

Or straight from the repository / a checkout:

dsh plugin --profile <name> add github:miqian-nomad/dsh-browser-playwright-codex

# local development: clone, then link the checkout
dsh plugin --profile <name> add link:<absolute path to this directory>

Compatibility. Built and tested against DSH 0.2.0-rc.1. The dependency ranges are floors, not wildcards: peers @deepseek-ai/cordis >=4.0.4, @deepseek-ai/dsh-llm >=0.2.0-rc.1, @deepseek-ai/dsh-tools >=0.2.0-rc.1, and the direct dependency @deepseek-ai/schemastery ^3.18.4. The floors exist so an empty project cannot resolve a very old 0.0.1-rc.* build, while a newer harness still installs. One cost of npm's prerelease rules: a prerelease whose [major, minor, patch] tuple differs from 0.2.0 (for example 0.2.1-rc.*) is outside a >=0.2.0-rc.1 range, so you may see a peer warning — a warning, not an install failure (pnpm does not enforce peer ranges strictly by default). After a harness upgrade, npm run doctor re-checks that every named export this plugin imports still exists — that is the fast way to tell "the harness moved" from "the plugin broke".

lib/ ships in the repository (and there is deliberately no prepare hook), so a fresh clone loads without a build step — npm run build is only needed after editing src/.

The bundle mounts three rows: the ctx.browser seam (service), the Playwright provider (playwright), and the model-facing tool family (tool).

Your first task

Install it and ask for something that leaves a visible result — there is nothing else to configure:

Open https://news.ycombinator.com and list the titles of the first five stories with their links.

Go to my orders page on `` and tell me the delivery address on the most recent order.

Fill in this contact form, but stop and show me the filled form before you submit it.

the model as a ref-numbered list. The first time it meets a verification page it reports the page back to you — take the window over by hand and it continues from where it left off (sign-ins have no detector: the agent is told to stop when it sees one). the model as a ref-numbered list. The first time it meets a sign-in, a 2FA prompt or a verification page it reports the page back to you — take the window over by hand and it continues from where it left off. Native dialogs are the one case that is enforced at the interface: they are parked and never auto-accepted. The logins already in that profile are what make the rest work.

Two habits worth having on day one:

• Let it look before it clicks. Start from browser_snapshot; a ref is only valid in the result that

produced it (a stale one fails fast instead of clicking something else).

• Say "stop" for the irreversible step. Submitting, paying, deleting and sending are the points where

you want to read the form first.

Browser binary

The provider probes chromium → chrome → msedge → edge channels automatically; any installed Chromium-family browser works with zero download. To use Playwright-managed Chromium instead:

npx playwright-core install chromium
# Restricted networks (e.g. mainland China):
PLAYWRIGHT_DOWNLOAD_HOST=https://cdn.npmmirror.com/binaries/playwright npx playwright-core install chromium

Or pin a binary: launch.executablePath / launch.channel in the plugin config.

Tools

The default toolPrefix is browser_. Every action returns a fresh snapshot, so refs always come from the latest result. The registered set is capability-gated: browser_evaluate, browser_cdp and browser_extract appear only when their capability is switched on (their schemas are resident in the system prompt on every turn, so a tool that could only answer "disabled" would be pure prompt cost). Count the live surface with npm run cost — the shipped cordis.patch.yml keeps both power gates off (since 0.4.7), so a profile mounting this bundle sees 16 tools; 17 or 18 once your deployment opts into evaluate / cdp, 19 with diagnostics: true, 21 once extract is configured. How to opt in: paste the snippet under Known Limitations below, or use the built-in Settings card.

Tool · Purpose browser_navigate · Open a URL (http/https only) and return the snapshot browser_snapshot · Snapshot the current page: URL, title, visible elements with ref= ids browser_click · Click the target from the latest snapshot — a ref, a visible label the snapshot gave no ref (e.g. { text: "搜索设置" }), or raw x/y viewport pixels as a last resort — or rest the pointer on it without activating anything with mode: "hover" (submenus, tooltips, previews, player controls); precise: true takes a ref click down the geometric-centre path. The returned snapshot opens with a landing note naming what received the click, and a dialog raised by a click stays pending — answer it with browser_dialog browser_fill · Replace an input's value; selects match by option label browser_press · Press a key (Enter, Tab, Escape, ArrowDown, …) on an element browser_scroll · Scroll the page or bring a ref into view browser_back / browser_forward · History navigation browser_wait · Wait a bounded duration for lazy content browser_tabs / browser_open_tab / browser_switch_tab / browser_close_tab · Tab management; browser_tabs marks the [active] tab the other tools act on browser_screenshot · PNG capture stored as a durable image attachment the model can view browser_extract · Structured extraction: natural-language instruction → one parsed JSON value; only registered when its capability is enabled browser_evaluate · Run one JavaScript expression in the page (disabled by default); only registered when its capability is enabled browser_dialog · Answer the native dialog blocking the page: { accept: true } confirms, false cancels; errors with NO_DIALOG when nothing is pending browser_cdp · Send one raw CDP command (DOM inspection / Input simulation allow-list only; disabled by default); only registered when its capability is enabled browser_close · Close the session's browser window; the profile keeps login state, so the next call reopens signed in browser_console_messages · Read the page console messages captured for the current page (levels, text, source); only registered when `diagnostics: true` browser_network_requests · Read the network requests captured for the current page (method, URL, status, resource type); only registered when `diagnostics: true`

Parameter signatures

Every registered tool, with the parameters its schema actually declares (* = required, ? = optional). Hand-maintained and pinned by tests/tool-schema.test.ts in both directions: a missing tool, a renamed parameter or a wrong required flag fails the test, so it cannot drift from the code.

• browser_back — _none_
• browser_cdp — method*, params? _(gated: registered only when enabled)_
• browser_click — ref?, text?, x?, y?, mode?, precise?
• browser_close — _none_
• browser_close_tab — index*
• browser_console_messages — level?, limit?
• browser_dialog — accept*, promptText?
• browser_evaluate — expression* _(gated: registered only when enabled)_
• browser_extract — instruction* _(gated: registered only when enabled)_
• browser_fill — ref*, text?, value?
• browser_forward — _none_
• browser_navigate — url*, waitFor?
• browser_network_requests — urlContains?, failedOnly?, limit?
• browser_open_tab — url*, waitFor?
• browser_press — ref*, key*
• browser_screenshot — fullPage?, ref?
• browser_scroll — direction*, amount?, ref?
• browser_snapshot — interactiveOnly?
• browser_switch_tab — index*
• browser_tabs — _none_
• browser_wait — ms*

使用铁律: 状态检查 / 无效不盲试 / 页面内容不可信 / 登录红线 / 风控分类等 Codex 文档纪律,

见 codex-rules.md（A 层已内嵌进各工具 description; B 层供粘贴进 DSH profile / 系统提示）。

Snapshot example

URL: https://example.com/
Title: Example Domain
Refs: 12

- navigation "Main"
  - link "Home" [ref=e1] -> /
  - button "Go" [ref=e2]
- heading "Welcome" [level=1]
- textbox "Search" [ref=e3]

The model then calls browser_click with ref: e2. Invisible content is excluded, node and text caps bound the token cost, and truncation is flagged in the result.

Result notes

A result can open with one or more notes before the tree. Each says what the plugin saw; none of them claims to *prevent* anything:

• [human verification] — a challenge page was recognised; the line above the note names the tier

(response header / widget / wording) and the evidence, and the note tells the agent to stop and ask you to finish it in the open window instead of retrying.

• [http NNN] — the main document answered a non-2xx status and no challenge was identified, so the

tree may be an error page or an interception page rather than the content that was asked for.

• [nothing here] — the page has no clickable elements and almost no text: a gate or visitor check, a

page that is still loading, or one that needs a person. The note says to take one more snapshot, then ask you rather than re-sending the same navigation.

• [no visible change] — the action left the page identical (after one action on a nearly empty page,

or after two identical results on a page with content). It is worded as a guess on purpose: it is what an unrecognised wall looks like.

• A landing note (which element received the click), a parked-dialog note, and an engine-fallback note

can also appear.

An actionable element with no text of its own is named from its descendants — 搜索设置 renders as link "搜索设置" [ref=e7], and ` as link "图标链接" [ref=e8] — so a visible label and the ref that carries it always appear on the same line. A non-actionable child that merely repeats an ancestor name is folded away, so each label appears once, on the line that carries the ref; container roles are never named from their content (a ` shows its chosen option, the way a text input shows its value).

Configuration

All tunables are patchable through the profile's cordis.patch.yml (later layers win per row).

# ONE row for the whole capability (2026-10-19). A package with a client module (the
# Settings card lives here) may only be mounted once, so the three concerns are nested.
- id: browser
  name: 'dsh-browser-playwright-codex'
  config:
    service:
      provider: playwright                # explicit provider selection
    playwright:
      launch:
        channel: chrome                   # or chromium / msedge / edge
        headless: false                   # visible window you can watch and rescue
        persistent: true                  # profile-backed window, login survives close
        # profileDir: ~/.dsh/browser-profiles/playwright by default
        viewport: { width: 1280, height: 800 }   # headful: IGNORED (the window tracks the desktop; only headless: true pins it)
        navigationTimeoutMs: 30000
        ignoreHTTPSErrors: false
      allowedDomains: []                  # e.g. ['github.com'] restricts navigation
      idleTimeoutMs: 0                    # shipped value: never auto-close (schema default 600000 = 10 min)
      maxSessions: 8                      # LRU eviction (persistent mode: one window)
      snapshot:
        engine: legacy                    # 'legacy' (default) or 'aria' (Playwright ariaSnapshot, refs aligned back)
        maxNodes: 500                     # token budget per snapshot tree
        maxNameLength: 120
        maxTextLength: 300
        diff: false                       # true = return "+/-/~" ref diffs instead of the full tree
    tool:
      toolPrefix: browser_                # model-facing tool name prefix
      allowEvaluate: false                # browser_evaluate gate
      allowCdp: false                     # browser_cdp gate (DOM inspection / Input simulation allow-list)
      registerDisabledTools: false        # register gated tools even while off (discoverability vs resident prompt cost)
      diagnostics: false                  # true = register browser_console_messages / browser_network_requests
      interactiveOnlyDefault: false       # true = browser_snapshot returns actionable elements only
      maxWaitMs: 60000                    # browser_wait bound
      extract:
        provider: deepseek-official       # optional: enables browser_extract
        model: deepseek-v4-flash
        maxInputChars: 20000              # caps the page text handed to that model
        maxOutputTokens: 2000

`snapshot.engine` picks the page-reading engine: `'legacy'` (the default) walks the injected DOM,
`'aria'` uses Playwright's official `ariaSnapshot({ mode: 'ai' })` and aligns the result back onto the
same `ref` ids; an aria tree that cannot be parsed falls back to legacy rather than handing the model an
empty tree. It lives under `playwright.snapshot`, next to `maxNodes` / `maxNameLength` / `maxTextLength` / `diff`.

`browser_extract` needs an auxiliary LLM route (`extract.provider` + `extract.model`); without one it fails with an actionable error. With a route configured, `extract.maxInputChars` bounds the page text in that prompt, and the prompt names the cap when it truncates. A gated tool is **not registered at all** while its capability is off, so its schema never reaches the system prompt; set `registerDisabledTools: true` to keep the whole surface discoverable (the tool then answers with the error naming the switch). The shipped layer in [`cordis.patch.yml`](https://github.com/miqian-nomad/dsh-browser-playwright-codex/blob/main/cordis.patch.yml) keeps `allowEvaluate` and `allowCdp` **off**, matching the schema defaults and this sample: that file travels inside the package, so anything it enabled would be enabled for everyone who installs it. Opt in per deployment, in that deployment's own patch layer. `browser_evaluate` executes arbitrary page JavaScript; `browser_cdp` stays off until `allowCdp: true`, and even then only the allow-list in `lib/cdp-policy.js` passes — network, storage, cookie, security and arbitrary-JS (`Runtime.evaluate`) commands are rejected with `CDP_DENIED`.

### Two levers that shrink what tools *return*

The resident schema cost is fixed, but two switches cut what each call sends back — useful on pages
whose snapshots are long, and both are off by default because both change what the model sees:

- **`snapshot.diff`** (playwright row) — returns `+` / `-` / `~` diffs against the previous snapshot,
  keyed by stable refs, instead of the whole tree. A ref-nonce change is reported as a navigation reset,
  and a truncated tree falls back to the full snapshot, so the model is never handed a diff it cannot
  anchor. Turn it on when a page re-renders often and you resent paying for the same tree twice.
- **`interactiveOnlyDefault`** (tool row) — makes `browser_snapshot` default to actionable elements
  only; the model can still pass `interactiveOnly: false` per call when it needs the text around them.
  Frames are kept either way: an iframe's accessible name is the embedded widget's own title, which is
  how a challenge widget identifies itself, so pruning it would hide the wall the snapshot is meant to
  reveal.

Both are honest trades, not free wins: a diff is smaller but requires the model to hold the previous
tree, and an interactive-only snapshot hides the prose the model may want to read. Measure before
turning them on in a deployment that shares prompts across many users.

### Cache and disk

The Settings card also carries a **缓存与磁盘** section: Chromium's pure caches inside the plugin's profile, `dsh-*` residue in `%TEMP%`, updater installers of the installed version, and the request-image cache are listed with their sizes; you preview first (“先预演”) and only then confirm. Cookies, Local Storage, Service Worker, IndexedDB, Network, Preferences and `~/.dsh/attachments` are never in scope — the target list is a closed enum in `src/cache-clean.ts`, and no path is ever taken from the request.

### Prefer a switch to editing config?

The Settings card ships **inside this package** (it moved out of the companion package `dsh-browser-toggle`, which is retired) and puts
this on one card: enable/disable the browser, pick the page-reading mode, and turn the two
power tools on or off — no restart, no YAML. The card also lists **which sites this browser looks
signed in to**. Two things make that list trustworthy: cookies are attributed **first-party** — only
domains this browser actually visited are candidates, because a tracker's cookie belongs to the site
you were browsing, not to the tracker — and "looks signed in" follows OWASP's session-ID criteria
(an `HttpOnly` cookie *and* a session-scoped or token-shaped one), not cookie names or counts. Domains that belong to the same company collapse into **one row** (`京东系（18 个域）`) with the totals
for the whole family, expandable to the individual domains. Each row also says **what the site is** (`jinritemai.com` → 巨量千川（抖音电商广告后台）): the page's
own title when the browser has that domain open, otherwise a curated offline table, otherwise just the
domain — never a guess. It is
labelled as an inference rather than a fact. It shows domain names,
counts and signal labels — never cookie names and never values. The list is shown only to you: nothing is written to disk; and no
tool exposes it to the model, so an agent cannot read this list. It is a separate package because
`dsh-client-modules` rejects a package mounted from multiple loader entries, and this plugin is
mounted as three (`service`, `playwright`, `tool`), while a Settings card needs a client module. The
card writes the gates into a managed block in the deployment's home-level `cordis.patch.yml`; the
package here stays untouched, so what you download is never the thing that decided to enable them.

## Architecture

A swappable capability seam, one package:

- `service` — `ctx.browser`: the provider registry with configured-or-auto selection semantics (a configured id must exist; one usable provider auto-selects; several require an explicit choice).
- `playwright` — the provider. **Persistent mode (default)**: one shared `launchPersistentContext` window on an independent profile directory (`~/.dsh/browser-profiles/playwright`), reused by every calling session — Codex-style personal browser. Login lives in the profile (durable cookies, localStorage) **plus** an exported state file (`dsh-storage-state.json`) that re-injects session cookies on relaunch (the browser drops those on close). The window auto-relaunches after an external close or crash, login intact. Set `launch.persistent: false` for the legacy shared-browser/per-owner-context mode — that mode writes **no** state file at all (the export is skipped rather than pointed at an empty path, which would have dropped cookie JSON into the process working directory). URL policy, idle disposal, and the injected snapshot engine apply to both.
- `tool` — the consumer: `defineTool`-built tools whose canonical values are structured JSON and whose rendered text is the snapshot tree. Gated tools register only when their capability is on.
- `contract` — the wire contract in one place: tool suffixes (and which are gated) plus every failure code. Registration, the load-time guard, the tests and the verify script all read it, so a renamed tool or changed code cannot drift apart silently.
- `compat` — the boundary adapter for the harness packages: borrowed names (`ToolCallId`/`CallId` and friends, once pe