dsh-memory

Cairn

Cairn is long-term memory for the DeepSeek Harness, packaged as the @chenhw7/dsh-memory profile bundle — facts, preferences, corrections, and lessons survive across sessions and restarts.

This is a self-contained single package (not a multi-package workspace). It depends on dsh core services as peer dependencies (provided by the dsh installation you already have) and ships its own cordis.patch.yml so dsh plugin add activates it as one profile layer.

English | 简体中文

Interactive Deep-Dive

A single-file, zero-dependency illustrated walkthrough of the whole pipeline — signal capture, storage, retrieval, injection, and decay. The figures are not static: the extraction loop, the write-path security gates, the BM25 kernel ported from src/store/bm25.ts, and a decay simulator all run live in your browser. It is served from this repo via GitHub Pages:

→ Open the live page: memory-architecture.html · 简体中文版

Table of Contents

• Interactive Deep-Dive
• Features
• Install
• Update
• Uninstall
• Verify
• Configuration
• Architecture
• Known Limitations
• License

Features

Your dsh agent normally forgets everything when you close a session. This bundle fixes that:

• Persistent memory — facts, preferences, and conventions stored in a durable KV backend with audit logging.
• Three-layer scoping — global (cross-project), project (per-repo, auto-detected), and user (cross-project profile).
• Nine memory tools + the identity tool — memory_search, memory_add, memory_replace, memory_remove, memory_list, memory_get, memory_pin, memory_unpin, memory_forget, plus identity_update (below).
• Identity layer (opt-in) — the agent's two self-documents, SOUL.md (its character) and USER.md (its understanding of you), grown by the agent in conversation through identity_update and injected into every session (soul @80 / user-profile @81, coexisting with — and yielding to — the deployment persona; explicit conversation instructions always win). Seeded once from built-in Chinese copy, versioned with full history, never decayed. You hold no editor — only governance: a read-only Identity settings section with version history, a two-step revert valve (default on, separate from the remote write switch), and markdown export. Extraction never persists identity restatements (prompt rule + mechanical prefilter), and an eval slice + --identity axis pin the injection surface (see TECH_DESIGN §7.10).
• BM25 relevance search — dependency-free Okapi BM25 over CJK-aware tokenization (Latin word tokens; CJK unigrams + bigrams), pinned entries surfaced ahead of equal-relevance matches. Retrieval quality is not vibes: a fixed golden set (35 entries × 35 query sets, English + Chinese) is evaluated in CI — success@5 = 100%, MRR = 0.902 — with injection-cost numbers per prompt mode (see TECH_DESIGN §7.9).
• Automatic learning — a projection accumulator watches the conversation for explicit remember-intent, corrections, and verified failure streaks (repeated same-signature tool failures resolved by a success), then runs LLM extraction when enough candidates accumulate. Admission rules exclude anything the repository already records (code structure, git history, fixed-bug narratives), and model-handwritten date prefixes are stripped so timestamps always come from the program.
• Project notes prompt section — coding conventions and a pitfall log render into every session's system prompt (project-notes section). Nothing is written into your repository: memory lives entirely in the host-side store, is managed in the Memory settings UI, and upgrades automatically clean up notes files left by ≤0.5.x installs.
• Digest-first injection (default) — the system prompt carries only frozen guidance; the data rides step-tail messages instead: a one-time <memory-digest> inventory of the store (category counts + anchor topic words, re-appended once after compaction) on the session's first step, and per-step <recalled-memory> fences when the step's text matches stored entries. The prompt prefix stays byte-stable all session, and nothing resident has to be evicted on re-freeze.
• Dedup pipeline — two-stage deduplication (stop-word-filtered Jaccard prefilter + optional LLM judge with bounded merges) prevents near-duplicate accumulation; a low-frequency curator pass re-summarizes oversized entries.
• Two-tier memory lifecycle — pin important memories; overdue project-scoped entries are removed while overdue global/user entries are soft-decayed (hidden from standing injections, still searchable, un-stamped on recall); entries can also be manually archived from the UI; every write is audited.
• Step-level auto recall (on by default) — on each agent step, a BM25 search keyed on the step's user text appends a fenced <recalled-memory> message without touching the system prompt, keeping the KV-cache prefix stable. Fence hits refresh an entry's recency clock but never its access count, so query-word luck cannot inflate the eviction signal.
• Compaction-aware flush — when compaction shadows old context, the raw events are scanned for anything worth remembering.
• Security scanning, write and load time — API keys, tokens, prompt-injection patterns, and exfiltration attempts are blocked from being saved; anything that slips through is redacted ([BLOCKED: …]) wherever it would re-enter a prompt.
• Frontend-configurable — all settings exposed through five cards in the dsh settings UI, apply live.
• Optional human review before write — flip one setting (confirmBeforeWrite) and every extraction and tool write becomes a proposal in a pending queue (repeated signals accumulate hits and float up); adopting applies it (with your edits), rejecting discards it. The model never self-promotes: a proposed change to an existing entry rewrites nothing until you accept it.
• Memory manager UI — a dedicated "Memory" section in the dsh settings UI with three tabs: Overview (health dashboard), Review (the pending-proposal queue), and Manage — scope and workspace filters, BM25 search, category chips, a lazily loaded entry list, plus full write actions (edit / pin / archive / delete) — in English and Chinese.
• Time-window browsing + smart list view — memory_list returns newest-first with earliest/latest/hasStale metadata, accepts since/until epoch-ms bounds so "what did we learn last week" pages within the window, and suggests widening the filter when a narrowed query comes back empty over a non-empty store.
• Optional summaries for progressive disclosure — memory_add/memory_replace accept a summary, and extraction can emit a [summary:…] tag; index mode and auto-recall render summaries instead of truncated content.

Install

Prerequisites

• Node.js installed.

• The dsh CLI available (globally or from a source checkout):

  npm install -g @deepseek-ai/dsh
  

  (If you prefer, npx @deepseek-ai/dsh also works.)

  Or build dsh from source:

  git clone https://github.com/deepseek-ai/deepseek-harness.git
  cd deepseek-harness
  pnpm install
  pnpm run build
  

  With a source checkout, run dsh commands as pnpm dsh ... from the deepseek-harness directory (for example pnpm dsh web).

• pnpm (used by dsh to manage profile dependencies). If it is not installed yet:

  npm install -g pnpm
  pnpm --version
  

• A profile you want to add memory to (this guide uses web).

[!IMPORTANT] Harness channel: alpha (0.1.2-alpha.x). This bundle is built against the @deepseek-ai/dsh-settings 0.1.2-alpha API (ctx.settings.installSection) and declares ^0.1.2-alpha.2 peers. Install dsh from the alpha channel — the rc (next) line ships the removed module-level helpers and cannot load this bundle. See Known Limitations.

From npm (recommended)

One command. The published tarball ships prebuilt, so pnpm does not run any build script on your machine and no extra pnpm configuration is needed:

dsh plugin add --profile web @chenhw7/dsh-memory

With a source checkout, run it from the deepseek-harness directory instead:

pnpm dsh plugin add --profile web @chenhw7/dsh-memory

To pin a specific version:

dsh plugin add --profile web @chenhw7/dsh-memory@0.4.0

From a local checkout

If you want to hack on the plugin, clone and build it first, then install from the local path:

git clone https://github.com/chenhw7/dsh-memory.git
cd dsh-memory
npm install && npm run build
dsh plugin add --profile web file:.

pnpm does not run build scripts for file: dependencies, so no allowBuilds entry is needed — the install copies the files as they are, which is why the npm run build step above matters: a missing or stale lib/ is what you get installed.

From a tarball (no build permission needed)

If you'd rather not install from the npm registry, pack a tarball from a checkout with lib/ already built and install it — a tarball ships prebuilt, so no allowBuilds entry is needed:

cd dsh-memory
npm install && npm run build
npm pack                    # produces chenhw7-dsh-memory-0.5.0.tgz
dsh plugin add --profile web ./chenhw7-dsh-memory-0.5.0.tgz

Update

dsh plugin add always installs the latest version from npm on a fresh profile. But once a version is installed, running dsh plugin add again will not update it — pnpm sees the existing version range (e.g. ^0.2.0) is already satisfied by the latest (e.g. 0.2.1) and skips the update.

To update to the latest published version:

dsh plugin --profile web update @chenhw7/dsh-memory

With a source checkout:

pnpm dsh plugin --profile web update @chenhw7/dsh-memory

Upgrading from 0.5.x

Version 0.6 stops exporting memory into repository files. The docs/agent-memory/CONVENTIONS.md / PITFALLS.md renders and the managed AGENTS.md pointer block are gone; memory is prompt-injected and managed in the Memory settings UI only. On the first session in each project, the plugin removes what ≤0.5.x left behind: the managed pointer block is stripped from AGENTS.md (a pointer-only AGENTS.md is deleted) and the generated files under docs/agent-memory/ are deleted (only files the plugin generated; anything else in that directory is kept). You can also delete them by hand at any time — nothing writes there anymore.

Uninstall

Remove the plugin from a profile:

dsh plugin remove --profile web @chenhw7/dsh-memory

(with a source checkout: pnpm dsh plugin remove --profile web @chenhw7/dsh-memory from the deepseek-harness directory). This runs pnpm remove in the profile directory and reconciles the layer list, so the seven memory-* rows disappear from the composed config — you can confirm with the --dump-config check below.

Uninstall does not delete your saved memories. They live under dsh's storage area, named by the memory-store row's storage config: host-medium (default) keeps everything in memory.json; sqlite moves the store into the plugin-owned memory.db with a one-time migration (memory.json then holds only the migration marker):

# macOS/Linux
~/.dsh/storages/memory.json
~/.dsh/storages/memory.db
# Windows
%USERPROFILE%\.dsh\storages\memory.json
%USERPROFILE%\.dsh\storages\memory.db
# or $DSH_HOME/storages/memory.json (or memory.db) if you set DSH_HOME

Stop dsh, then delete the file for your backend (under sqlite: memory.db — leftover -wal/-shm sidecars are empty and harmless) to wipe all saved memories. Other files in the same directory belong to other features — do not remove the whole directory.

Verify

After install, confirm the seven memory rows are in the composed profile tree:

# Windows
dsh --profile web --dump-config | findstr memory
# macOS / Linux
dsh --profile web --dump-config | grep memory

You should see seven rows pointing at @chenhw7/dsh-memory/*:

- id: memory-root
  name: '@chenhw7/dsh-memory'
- id: memory-store
  name: '@chenhw7/dsh-memory/store'
- id: tool-memory
  name: '@chenhw7/dsh-memory/tool'
- id: memory-review
  name: '@chenhw7/dsh-memory/review'
- id: memory-notes
  name: '@chenhw7/dsh-memory/notes'
- id: memory-context
  name: '@chenhw7/dsh-memory/context'
- id: memory-remote
  name: '@chenhw7/dsh-memory/remote-service'

Then start dsh and check the settings UI shows the memory namespace:

dsh web

Configuration

The bundle owns five settings namespaces — one per plugin-configuration card (the host's plugins tab dispatches a card only when its slot key names a served settings namespace) — and they all apply live: a change takes effect on the next event or call, no restart:

• memory (card: Memory) — injection modes and the memory budgets. Owned by memory-context.
• memory-notes (card: Project Notes) — the project-notes prompt-section switch and budgets. Owned by memory-context.
• memory-autorecall (card: Auto Recall) — the step-level recall fence and the usage-hit signal. Owned by memory-context.
• memory-identity (card: Identity) — the identity layer's switch, budgets, and seed directory. Owned by memory-context.
• memory-review (card: Automatic Extraction) — extraction pipeline, model routing, dedup judge, pitfall streaks, curator pass. Owned by the memory-review plugin.

Every namespace resolves in layers: schema defaults → the composition config: entry (the base) → the user document in $DSH_HOME/settings.yaml. A field absent from the user layer inherits the composition value, so a deployment can pin a default and users override only what they need. With no settings service mounted (e.g. a headless profile), each plugin falls back to its composition entry exactly as composed.

memory namespace

memory-notes namespace

Settings migration: the 0.9.x combined key notesCharLimit still works — when it is the only budget key set, both halves derive from it (conventions 60% / pitfalls 40%); each new key you set wins its own half. Rename the key in $DSH_HOME/settings.yaml (or the composition config:) to migrate.

memory-autorecall namespace

memory-identity namespace

memory-review namespace

There is deliberately no separate tool-memory settings namespace: the tool plugin reads maxSearchResults live from the memory namespace above. Its composition config.maxSearchResults only serves as the fallback base when no settings service is mounted.

Setting via composition vs. UI

Every namespace accepts the same keys from both layers. A composition config: entry sets the base; the UI writes the user layer on top. The four memory-family namespaces share one composition row (memory-context's config: carries all their keys), while the user document keeps one section per namespace. For example, to pin maxSearchResults: 100 as the deployment default while still letting a user override it:

memory:
  config:
    maxSearchResults: 100

By default, extraction, judging, and curation use the same model the user is chatting with — the session's provider/model route. To run them on a dedicated cheaper model, set extractionModelProvider and extractionModelModel (in the composition config or the UI — the UI offers dropdowns fed by the host model catalog). Naming a provider here sends conversation excerpts and stored entries to it, so pick one you trust with conversation content; leaving both empty adds no data path beyond the model you are already talking to:

memory-review:
  config:
    extractionModelProvider: deepseek
    extractionModelModel: deepseek-chat

Example $DSH_HOME/settings.yaml (all namespaces):