AkinoHaruka/companion-memory

companion-memory

Native scoped Riko memory plugin for DeepSeek Harness

미분류dshmemory

description: "Native scoped memory plugin for DeepSeek Harness." kind: "package-bundle"

@deepseek-ai/dsh-riko-memory

English | 中文

Summary

Native DSH memory with stable-preset scope isolation, durable storage-domain records, explicit confirmation, canonical Wiki pages, bounded Resident Snapshots and a recoverable Dream pipeline. Secrets stay in credentials or process environment; the UI, control API and acceptance-only paths are explicitly scoped.

Table of Contents

What this package is

@deepseek-ai/dsh-riko-memory is the native DeepSeek Harness implementation of Riko-style scoped long-term memory. It connects directly to DSH lifecycle and storage seams: Cordis plugin loading, session events, stable Agent presets, storageDomain, system-prompt injection, explicit tools, credentials and background jobs. It is not an MCP memory server, not a second SQLite service and not a generic file-store adapter.

The governing promise is simple: a memory may influence a later reply only after it has a traceable source, passed the scope boundary, reached the canonical Wiki state machine and been selected for the bounded Resident Snapshot. Model output alone never becomes authority.

The canonical source is packages/bundle/riko-memory in the DSH Harness worktree. The GitHub repository AkinoHaruka/companion-memory is the user-facing source mirror of this bundle. Older standalone implementations and historical root-level copies are migration references, not parallel runtimes.

The normative V3.1 decision addendum is docs/v3.1-decision-semantics.md; its implementation status is tracked in docs/memory-v3-progress.md.

Architecture at a glance

User conversation
  -> DSH session/event listener
  -> L0 evidence: sessionId + event sequence + source span
  -> explicit tool or background Dream
   -> L1 Candidate: proposal with source record and consent state
  -> original user evidence or explicit management action
  -> L2 canonical Wiki page: durable, versioned authority
  -> Resident compiler
  -> L3 Resident Snapshot: bounded prompt projection
  -> next request for the same stable Agent preset
LayerNamePurposeDirect prompt input
L0Session evidenceWhat was actually said, with event source recordNo
L1CandidateWhat the extractor proposed, awaiting reviewNo
L2WikiCanonical pages, versions, links, sources, expiry and audit lineageIndirectly
L3Resident SnapshotRebuildable bounded projection for the next requestYes

The layers answer different questions: L0 is “what happened”, L1 is “what was proposed”, L2 is “what was accepted as durable memory”, and L3 is “what is safe and useful to inject now”. The Resident is derived and replaceable; the Wiki is the long-term authority.

Goals and non-goals

Goals

  • Share durable memory across sessions belonging to one stable Agent preset.
  • Isolate owners and presets in both runtime tools and the HTTP control plane.
  • Preserve session, event and source records so a page can be audited.
  • Require explicit evidence or explicit management action before Dream output becomes canonical.
  • Keep the chat hot path bounded: reading Resident does not call Dream, scan the full Wiki or wait for a background lock.
  • Make provider, parser, storage, Resident compilation and host restart failures recoverable.
  • Keep credentials outside source files, settings JSON and ordinary management payloads.
  • Expose Wiki, candidate, source and graph information for human review.

Deliberate v1 non-goals

  • Production-scale background vector-index compaction and multi-node index ownership beyond the optional bounded dense-recall provider and index seams.
  • Multi-node coordination, public multi-tenant hosting or distributed job ownership.
  • Complete cryptographic erasure of every evidence, audit and external backup copy.
  • Treating every conversational sentence as a memory.
  • Letting a model-generated “please remember this” sentence authorize itself.
  • Replacing DSH storage/lifecycle with Fastify, a fixed filesystem path, node:sqlite, MCP or a Rust sidecar.

Scope and isolation

Long-lived memory is keyed by:

MemoryScope = ownerNamespace + stableAgentPresetId

ownerNamespace identifies the owning installation or tenant boundary. stableAgentPresetId identifies the DSH Agent preset whose behavior and memory are meant to be shared. Session ID is an evidence record, not the long-term sharing key.

The rules are:

  1. Sessions under the same owner and stable preset share the same Wiki and Resident.
  2. Different stable presets remain isolated even when the owner is the same.
  3. Missing or unstable preset identity fails closed; there is no default global fallback.
  4. Session, bearer token, HTTP profile and storage record must resolve to the same scope.
  5. A model cannot enlarge scope by inventing a session ID, profile name or source reference.

“Global memory” therefore means global across sessions and projects within one stable Agent preset, not global across all Agents.

Versioned contracts

The core contracts are versioned and scope-aware:

  • MemoryScope: schemaVersion, owner namespace, stable preset and deterministic scope key.
  • storageDomain: riko_memory, per-record version 5, compatible with versions [1, 2, 3, 4].
  • EvidenceRef: schemaVersion, session ID, event sequence and optional source span.
  • MemoryCandidate: scope, content, status, consent, evidence, source (dream or manual), timestamps, expiry and supersession data.
  • WikiPage: versioned canonical body, status, consent, evidence, source candidates, update time and optional expiry.
  • ResidentSnapshot: bounded content, version, generated time, scope and source page IDs.
  • DreamJob: scope, session, event cursor, status, attempts, timestamps and sanitized last error.

Provider output cannot manufacture an authoritative EvidenceRef. The current DSH event stream remains the source of truth.

End-to-end lifecycle

1. Capture L0 evidence

The plugin observes DSH session events and writes bounded serialized evidence into the configured storageDomain. Session identity, event sequence and source information are retained. Scope is decided at capture time and is never reassigned from model text.

2. Choose an ingestion path

There are two paths:

  • Explicit path: the user directly asks the Agent to remember, correct or forget something. The tool validates the latest raw user message and applies the requested state transition immediately.
  • Dream path: after session activity, a debounced background job consumes the session cursor and asks the configured provider for structured Wiki files. Dream is asynchronous and does not block the normal reply path.

3. Produce a Candidate

Dream uses a controlled FILE protocol. Only approved wiki/ folders are accepted. The parser rejects malformed or empty blocks, ignores model-authored source authority, normalizes titles and wikilinks, bounds descriptions and bodies, and forces every extracted page to candidate with pending consent.

Repeated proposals are fingerprinted and merged. Matching candidates preserve the union of source conversations and the strongest available confidence instead of creating unbounded duplicates.

4. Confirm or reject

Confirmation has exactly two valid authorities:

  • Original user evidence: explicit tools require the claim or replacement text to appear in the latest raw user message.
  • Explicit management action: an operator can confirm, reject, edit or create a canonical page through the control API.

The Dream model's own status: confirmed, consent: true, locked: true, or a sentence saying “the user asked me to remember this” has no authority.

The candidateAutoConfirm policy can admit an extracted page without a manual confirmation step, and it defaults to user_grounded. A candidate is admitted only when its description exactly matches the normalized full text of one user-authored L0 event in the same scope. The canonical title, description and body are then all replaced with that exact user text, preserving negations and qualifications and discarding any model-authored additions or omissions. Dream-authored tags, categories, page types and temporal claims are also discarded when the canonical page is created. Candidate sensitivity is normalized locally from title, description and body: private claims become sensitive, identifier-shaped claims become provisional_sensitive, and only normal candidates can be admitted; a malformed sensitivity label fails closed as provisional. Conflicting candidates stay pending. off keeps every candidate pending. all skips grounding and is a deliberate deviation from the no-auto-promotion invariant, intended only for controlled dogfooding. Every automatic admission is audited as candidate-auto-confirmed with its mode and grounding reference.

5. Commit to the canonical Wiki

Confirmation creates or updates a versioned Wiki page, records source lineage and removes its pending Candidate. Corrections retain prior version information in the audit lineage. Superseding a page removes it from future Residents while retaining the page and evidence trail for review.

6. Compile and inject Resident

Only confirmed, consented, non-superseded, non-expired and Resident-eligible pages are compiled. The current runtime excludes sensitive pages by default; V3.1 additionally requires provisional_sensitive to be denied Resident use until an explicit policy permits it. Sensitive canonical pages remain auditable and available to explicitly gated recall, but do not enter ordinary Resident injection. Source pages, pending candidates and raw transcripts stay outside the hot prompt. The result is ordered, bounded by maxResidentChars, assigned a content-derived version and persisted with exactly the source page IDs represented by the selected blocks. DSH injects only the current stable preset's Resident as labeled memory data, not as instructions.

Explicit tools

memory_get_resident

Reads the current stable preset's Resident and version. Missing preset identity fails closed. This is a read-only hot-path operation and does not invoke Dream.

memory_remember

Stores one explicit preference or fact as managed, confirmed memory. The content must be non-empty and appear in the latest raw user message of the current session. It cannot save a model inference or a claim copied from another session.

memory_correct

Replaces the selected Wiki page's user-visible content after checking that the replacement occurs in the latest raw user message. The page remains confirmed, its lineage is retained and Resident is rebuilt immediately.

memory_forget

Requires an explicit deletion cue such as “forget”, “remove” or 忘记, plus the exact memory ID in the latest user message. It removes derived memory from future Residents and records the operation. v1 does not purge the original raw Session evidence.

Query-time Recall v1

The optional recallEnabled flag retrieves long-tail details that do not fit in Resident. The plugin defaults it off; the Web overlay enables it by default, with DSH_MEMORY_RECALL_ENABLED=false as the opt-out. Lexical ranking uses BM25 with IDF derived from the candidate corpus, so ubiquitous tokens such as a fixed adapter prefix or generic chatter cannot outrank rare discriminative terms. The budget first reserves up to min(recallMaxCandidates, recallAuthoritativeReserve) seats for non-evidence authoritative results, then considers the fused remainder with raw evidence capped by recallRawEvidenceMaxCandidates; canonical text is rendered as [authoritative] and selected supporting evidence as [supplement], while raw detail that repeats all selected non-evidence terms is suppressed. The rendered context remains bounded by recallMaxContextChars. embeddingProvider can add deterministic-local or OpenAI-compatible dense recall; provider failures degrade to lexical/RRF and never block chat. The keyless deterministic ablation measured 0/16 unique recall gain, 2/9 dense noise and 0/9 gate rejection in its companion comparison; the larger 12-scenario, 20-pages-per-scenario keyless chaos probe printed 0/12 unique gain, 60/60 noise and 36/96 gate rejection in this run. These are keyless routing and selection measurements, not BGE quality results. The BGE-gated semantic corpus (tests/dense-semantic-gain.spec.ts) measures paraphrase recall that lexical ranking structurally cannot reach: recorded against a local BGE-small-zh-v1.5 server, lexical-only reaches 0/5 targets, the deterministic hash provider 2/5, and BGE vectors 5/5. That run requires DSH_MEMORY_BGE_ENDPOINT, so a default package run reports no BGE quality measurement. The live hook also never supplies a MemoryReranker; reranking is reachable only through direct helper/store callers and tests. Recall accepts an explicit atTime or history mode for retained temporal lineage. A suppression cue is retained as policy evidence, drives suppression, and is excluded from ordinary raw recall while that suppression is active. See docs/recall-v1.md and docs/migration-phase-2-temporal-resident.md.

Observation management, graph expansion and raw-session purge are separately flagged with recallObservationEnabled, recallGraphEnabled and purgeEnabled, all defaulting to false. Opt-in Dream reflection can create anchored observation candidates, and candidate creation or evidence updates can auto-activate a still-candidate observation when it has normal sensitivity, no strong contradiction, at least the configured evidence and distinct-session minimums, and the configured confidence minimum. Separately, POST /memory/v1/observations/:id/activate is an authenticated management operation; it can explicitly activate a record after the store's minimum-evidence check, while the matching routes invalidate or suppress it. Neither activation path confirms a fact or grants explicit mention permission. Observation records require raw/confirmed anchors and remain epistemically distinct from Wiki facts. Ordinary memory_forget retains raw Session evidence.

Safe-use projections are rebuilt from canonical pages and observations, persisted in the projections table and read by the query-time recall path. The renderer applies their disclosure policy without per-turn sanitizing model calls; V3.1 forbids those calls.

Temporal validity and structured Resident

Canonical pages support observedAt, recordedAt, validFrom, validTo, supersession lineage and historical atTime/history recall. A temporal update closes the previous page interval and publishes a new page identity; a correction edits the existing page identity and retains audit lineage. Current Resident compilation excludes superseded, expired, source-only, unconsented and sensitive pages.

Resident content is compiled into bounded blocks (identity, preferences, relationships, currentState, communicationStyle, activePeople and openThreads). The persisted snapshot is the bounded value that is injected, with a deterministic version and selected source page IDs. Legacy unstructured Resident records remain readable during migration.

Dream pipeline and provider behavior

Dream is a recoverable background organizer, not the source of truth.

  1. turn/end and manual triggers enqueue after the evidence write barrier.
  2. Jobs are serialized within a scope; different scopes may progress independently.
  3. Persisted cursors prevent repeat ingestion and persisted jobs enable restart recovery.
  4. Transcripts are bounded and the provider is asked for only a small number of concise pages.
  5. Production Web defaults use Google's native models/{model}:generateContent contract. Explicit operator configurations may use OpenAI-compatible Chat Completions or Anthropic-compatible Messages; the protocol is inferred from endpoint shape, and these adapters are outside the formal Dream acceptance set.
  6. Requests use bounded provider fields, never put credentials in the URL, time out each provider attempt after 60 seconds, and switch in order on network errors, timeouts, HTTP 429/5xx, or invalid FILE output. HTTP 400/401/403 stop the chain.
  7. Responses pass strict FILE parsing before persistence. Invalid blocks, empty content, unexpected JSON and provider errors fail the job without replacing the old Wiki or Resident.
  8. System-generated extraction sessions do not recursively trigger another Dream pass.

Provider failures are classified and sanitized. The control plane exposes only configured/not-configured state and a credential reference, never a secret, Authorization header or complete external error body.

Persistence and recovery

The plugin uses DSH storageDomain as its persistence boundary. It does not expose a fixed path, node:sqlite database or public file-store API. Durable state includes scoped profile state, L0 sources, Wiki pages, Candidates, Resident metadata, audits and Dream cursors/jobs.

  • Storage initialization failure disables long-lived memory rather than silently using a global or local fallback.
  • Dream failure records a sanitized error and keeps the previous valid Resident readable.
  • Resident compilation failure keeps the previous valid projection.
  • Host restart can replay persisted evidence and jobs after the bundle loads again.
  • A provider-forged session ID is ignored; the job's session and cursor remain authoritative.

Security and privacy

Credentials

Dream and embedding secrets are resolved from DSH credentials or process environment at execution time. UI and HTTP configuration accept endpoint, model, token limit and credential reference, but reject raw apiKey or secret-like credential writes. Provider endpoints must use HTTPS and cannot contain embedded credentials.

Control plane

Loopback development can use the host boundary. Remote control requires a bearer token and matching x-dsh-memory-profile. apiTokens bind each token to one profile. In single-token mode, apiTokenProfile binds the token to one profile when set; when empty, the token is owner-wide admin and the control response reports scopeBinding: owner-admin. Cross-scope sessions are rejected.

Sensitive material

Dream-derived sensitive candidates remain pending until explicit review. Explicit user or trusted management actions have separate authority, but V3.1 treats sensitivity as a usage permission rather than a truth claim and requires the three-state policy to tighten conservatively. The extraction prompt forbids inferring secrets, diagnoses and instructions, but this is a guardrail rather than a substitute for deployment policy.

Deletion semantics

Ordinary “forget” means removing derived memory from the canonical Wiki projection and newly compiled Residents; it intentionally retains raw Session evidence. When purgeEnabled is explicitly enabled, POST /memory/v1/purge starts a journaled raw-session purge and reconciles session, source, page, candidate, observation, derived-index and Resident state. It requires a dry-run or the exact plan confirmation and reports verified completion. Interrupted journals can be retried on store reopen. Complete cryptographic erasure outside the configured storage domain remains a separate deployment responsibility.

Control API and UI

The UI is a management/audit surface, not a bypass around the state machine. It shows Resident content, Wiki pages, candidates, source sessions, graph edges, versions, confidence, consent, expiry and status. Replace /memory/v1 below with the configured apiPath when it differs from the default.

MethodPathPurposeAccess / flagsSurface
OPTIONS/memory/v1/*CORS preflightNo auth; no flagProtocol
GET/memory/v1, /memory/v1/ or /memory/v1/uiHuman-facing control and audit UINo auth for HTML; data API is auth-gatedAdmin-only
GET/memory/v1/configSafe endpoint/model/credential statusProfile auth; no flagAdmin-only
POST/memory/v1/configUpdate non-secret Dream settingsProfile auth; no flagAdmin-only
GET/memory/v1/wikiWiki snapshot, candidates, sources, graph and stateProfile auth; no flagAdmin-only
GET/memory/v1/wiki/pages/:idRead one Wiki page; sensitive/unknown bodies are redacted unless reveal=sensitiveProfile auth; no flagAdmin-only
GET/memory/v1/wiki/graphRead bounded Wiki graph; hop, evidence and includeEvidence are query optionsProfile auth; no flagAdmin inspection; graph recall is separately flag-gated
GET/memory/v1/wiki/searchLexical Wiki inspection; this is not live Agent recallProfile auth; no flagAdmin-only
GET/memory/v1/wiki/sourcesList scope-local source metadataProfile auth; no flagAdmin-only
GET/memory/v1/residentRead current bounded Resident and raw-retention disclosureProfile auth; no flagAdmin inspection
GET/memory/v1/sessions or /memory/v1/sessions/:idList sessions or read redacted L0 metadata; reveal=sensitive is auditedProfile auth; no flagAdmin-only
GET/memory/v1/candidatesList pending Dream CandidatesProfile auth; no flagAdmin-only
GET/memory/v1/observationsList inferred observations and statusesProfile auth; no flagAdmin/acceptance-only
GET/memory/v1/purgesRead scope-local purge journal metadataProfile auth; no flagAdmin/acceptance-only
GET/memory/v1/conflictsList contested and resolved conflict overlaysProfile auth; no flagAdmin-only
GET/memory/v1/auditsRead scope-local lifecycle auditsProfile auth; no flagAdmin-only
POST/memory/v1/recallRun bounded query-time recallProfile auth; recallEnabledAdmin inspection; live Agent recall uses the separate pre-step hook
POST/memory/v1/recall/debugRead recall plan, channels, gates and degraded modesProfile auth; recallEnabledAdmin/diagnostic-only
POST/memory/v1/observationsCreate an anchored observation candidateProfile auth; no flagAdmin/acceptance-only; not Dream/Agent creation
POST/memory/v1/observations/:id/activate, /invalidate or /suppressChange one observation statusProfile auth; no flagAdmin/acceptance-only
POST/memory/v1/conflicts/:id/resolveResolve a contested overlay with JSON `{ "resolution": "correction""temporal_transition""management" }; returns 400for another value and404` when no state changes
POST/memory/v1/purgePurge one raw session when the flag is enabledProfile auth; purgeEnabledAdmin/acceptance-only
POST/memory/v1/wiki/candidates/:id/confirm or /rejectConfirm or reject a CandidateProfile auth; no flagAdmin-only
POST/memory/v1/candidates/:id/confirm or /rejectCompatibility alias for Candidate actionsProfile auth; no flagAdmin-only
POST/memory/v1/wiki/pagesCreate a locked canonical pageProfile auth; no flagAdmin-only
PUT/memory/v1/wiki/pages/:idEdit one canonical p