dsh-errkb

dsh-errkb

简体中文

Roadmap rebaselined on 2026-10-13. The ordered implementation source is now docs/implementation-plan.md, starting at EKB-001. The detailed sections below document the current DSH prototype and retain some legacy T/X task references until the README closeout card EKB-055.

Recycles errors: the same failure is diagnosed once, recorded, and handed back as a known fix the next time it appears.

dsh-errkb keeps one numbered, human-editable error knowledge base and delivers it in two forms:

Both forms share one set of rules (normalize, fingerprint, redact, match) and, once the toolbox ships, one knowledge base.

Status: P5 complete. The design document is written and under review — six of the eight decisions in its §17 are still open (Q2 and Q4 are decided) — and the work is broken into eighteen tasks across seven milestones. T01–T15 are done: the package installs, builds, type checks, lints, formats and tests (with the 99% coverage gate enforced), CI runs all five on every push to main and every pull request, and the pure local layer is complete: knowledge-base directory resolution (T05), error normalization and fingerprinting (T06), mandatory redaction (T07), the ERRORS.md store (T08) and matching (T09). The capture layer is wired: classification and the noise rule (T10), and the first two hooks, agent/error and tools/result (T11). The injection layer speaks: notice text, caps and fix trust (T12) are wired to the four injection points (T13), and resolution detection (T14) closes the loop. The five agent tools (T15) let the model look errors up and write fixes down.

The plugin records and injects, once installed. Loaded into a profile, apply logs where the knowledge base is, records tool failures, non-zero command exits and turn-level errors into ERRORS.md, and hands a recorded fix back to the model when the same error repeats, within the caps below. When a recorded error stops failing, its fix earns trust, and an entry with no fix gets one prompt asking the model to record it with err_record (T14, T15), which writes it into ERRORS.md. Still missing: LLM request-failure capture (T16), and installation into a profile (T17) — nothing installs it for you yet, so the tools only reach a model once you add the plugin to a profile yourself. Anything below tied to T16–T18 is still a description of the intended path.

• Implementation plan: docs/implementation-plan.md
• What the plugin will do: How it works
• Where your data will land: Where the knowledge base lives
• What is still open: Open decisions — Q2 and Q4 of §17 are decided, six remain

A second delivery form is planned. An external Windows toolbox (errkb.exe, Go + PostgreSQL, CUE for its schemas) will watch logs, wrapped processes and windows and serve any MCP client, so the knowledge base stops being dsh-only. None of it is written yet; the requirements, architecture decisions, 55 ordered task cards, and implementation order are consolidated in docs/implementation-plan.md.

Contents

• What works today
• Why
• Features
• How it works
• Where the knowledge base lives
• The ERRORS.md format
• Injection: when it speaks and how much
• Cost model
• Requirements and compatibility
• Installation
• Usage
• Configuration
• Privacy and redaction
• Failure modes and safety
• Known limitations
• Verifying it works
• External toolbox (planned)
• Roadmap
• Development
• Troubleshooting
• Uninstall
• Open decisions
• Future work
• Related projects
• Contributing
• License

What works today

Why

An agent has no memory of the last time it hit an error, so it re-derives the diagnosis from scratch: reasoning, a failed attempt, another failed attempt. That costs somewhere between 800 and 3000 tokens.

For a failure you see once, that is fine. For a failure you see every week — an EPERM from a locked node_modules, a CONTEXT_OVERFLOW, a misspelled provider name — it is pure waste. A human recognizes "this is the one from last time" instantly. The model does not, because the path changed, the line number changed, and the PID is different.

dsh-errkb closes that gap by keeping the knowledge on disk and injecting what it already knows.

Features

Designed, mostly implemented, not installed. Everything below is specified in the design document and covered by a test plan. Recording and injection run: stable IDs, one ID per error and mandatory redaction apply to what the T11 listeners write, a recorded fix is injected before diagnosis (T13), a resolved error is noticed and asked for its fix once (T14), and the five tools write the answer down and keep the ledger (T15). Nothing installs the plugin into a profile until T17 — see What works today.

How it works

1. CAPTURE      agent/error · tools/result · non-zero exits
                (agent/request-error: not yet wired, T16)
                observe only — never throws, never takes over retries
                                     │
                                     ▼
2. FINGERPRINT  normalize: strip ANSI; ts / pid / port / line / uuid / tmp → placeholders;
                absolute paths → <path>, filename kept
                signature = sha256(category + "\0" + message)[0:12]
                                     │
                                     ▼
3. MATCH        exact signature → Jaccard fuzzy (≥ 0.72) → short-message
                same-code fallback → otherwise assign a new ID
                                     │
                       ┌─────────────┴─────────────┐
                    hit│                           │miss
                       ▼                           ▼
4. INJECT     ≤ 120 tokens/notice             5. RECORD   append block to ERRORS.md
   ≤ 1/step, 3/turn, 2/ID/session                         assign next ID (append-only)
                       ▲                           ▲
                       └─────────────┬─────────────┘
                                     │
6. RESOLVE      model or user writes the fix via `err_record` (the sole entry point)
                resolution detection → one-shot prompt → status becomes `fixed`

The hard part is step 3, not step 2. Normalizing a message is mechanical; deciding that two different-looking messages are the same error is where a knowledge base either helps or quietly poisons the context with the wrong fix. Matching runs in three tiers — exact signature, then fuzzy token similarity, then a same-code fallback for messages too short to compare — and a near miss is labelled as one in the injected text rather than passed off as certain.

One rule holds the whole thing together: ERRORS.md is the single source of truth. errors.index.json is a derived cache that can be deleted and rebuilt; state.json holds local counters that can be lost without consequence. None of the three ever writes another's facts.

What earns an ID

Not every failure deserves a number. Transient ones are counted and only promoted when they keep happening, because a rate limit is not knowledge.

When it lands (T16), the listener on agent/request-error must await next() and return the downstream result unchanged. It observes; it never recovers.

What is wired today (T11). src/plugin.ts holds the listener bodies; apply registers two of them. agent/error hands its error to classification as a turn-level exception. tools/result records a failed result (result.isError) as a tool failure with result.error.message and result.error.info?.code; any other result has its text blocks sniffed for [exit code: N], with the command taken from the call's command, cmd or script argument. Each classified error is matched against the entries already in ERRORS.md: a hit bumps that entry's hits and last-seen in this machine's state.json and leaves ERRORS.md untouched (§4.3), a miss appends the next ID. A count-only error (a transient one below the threshold, or one after its promotion) only bumps an entry that already has its signature, the same way, and writes nothing otherwise. Transient counts are kept per session, by Agent.id; a payload with no agent counts under one plugin-wide counter. agent/request-error is not registered yet (T16), so no LLM failure reaches the pipeline.

Promotion happens once. A transient error's count is kept per session and per signature, and it takes an ID on the occurrence that reaches transientThreshold — not on every occurrence after it. With a source left out of capture, nothing is produced for it at all, not even a count.

One line decides the signature. Multi-line output — tsc with forty errors, a pnpm install log, a Python traceback — is reduced to one headline before fingerprinting, so a forty-first error or a reordered log keeps the same ID. The behavior is preserved by EKB-012 in the implementation plan. src/capture.ts picks:

1. for a Python traceback (Traceback (most recent call last):), the last non-empty line — checked first, because a traceback quotes source lines such as raise ValueError(...);
2. otherwise the first line matching ERR_[A-Z0-9_]+|E[A-Z]{2,}|[A-Z]\w*Error|error TS\d+ (log-level words such as ERR and ERROR do not count);
3. otherwise the last non-empty line.

The headline is capped at 200 characters, the code it names (ERR_PNPM_…, EPERM, TS2307, ModuleNotFoundError) becomes the entry's code when the source gives none, and the full text is kept as the raw sample. For a command, the harness's own [exit code: N] marker is left out of the headline. When a command's headline names no code — a silent failure reads exit code 1, a test runner's last line reads 1 test failed — the command that ran leads it (pnpm test → exit code 1, its first line, cut to 120 characters), so silent failures of two different commands never share an ID.

Where the knowledge base lives

kbDir resolves in three tiers, in order:

At startup the plugin logs the resolved path on one line, so the location is never a guess. err_stats prints it too.

Real entries belong in a private repository (§17 Q4). Redaction by text rules cannot be complete, so this public repository ignores errors/ entirely and only carries curated, redacted seed entries in seeds/ERRORS.seed.md. To sync your knowledge base across devices, clone a private repository of your own on each machine and point kbDir at it — an absolute path, or a path relative to the plugin root. The link: development install would otherwise land in errors/ inside this repository, where git now ignores it. Either way, no absolute path is ever written into the document.

Inside that directory:

A sensible .gitignore for that private knowledge base repository:

state.json
errors.index.json
.machine.json
.lock

The design document names state.json and errors.index.json explicitly; .machine.json and .lock are device-local or transient for the same reason.

That split is the whole cross-device story: knowledge merges by appending to ERRORS.md, counters stay on the machine that produced them, and the two never fight. See Roadmap for what is not built.

The ERRORS.md format

IDs start at E-0001, built from idPrefix (default E-) and idWidth (default 4), and are append-only — an ID is never changed once written, not even to resolve a merge conflict.

One entry, in the default English labels, with machine fields carried in an HTML comment so the rendered document stays clean and hand-editing stays safe:

## E-0007 · [tool:pwsh] EPERM: operation not permitted, rename


- Fingerprint: `3f2a1c9d0b71`
- Category: `tool / pwsh`
- First seen: 2026-09-14 09:12 · Last seen: 2026-09-14 15:40 · Hits: 5
- Trigger: `pnpm install` writing `node_modules` under a non-ASCII path while another process holds it
- Raw message:
  ```text
  EPERM: operation not permitted, rename '<path>\node_modules\.pnpm\<hash>'
  ```
- Fix:
  Close the editor or real-time antivirus scan holding the directory, then re-run `pnpm install`; if it persists, use `pnpm install --config.node-linker=hoisted`.
- Status: `fixed`
- Notes:

Labels: English by default. Set labels: 'zh' to write the Chinese labels of the design document's §8 instead (指纹, 分类, 首次, 最近, 命中, 触发, 原始信息, 解法, 状态, 备注). The parser reads both sets — and a full-width ： — whatever the setting, so a document may mix them, and an updated entry keeps the language it was written in. Machine keys in the comment are always English.

Hits and Last seen: the values written in a block are the ones it was created with (Hits: 1, the time of capture) or whatever a person typed there since. A repeat never rewrites the block: it is counted in this machine's state.json, keyed by ID, so two devices that both hit the same error never touch the same lines and thei