dsh-history-fold

dsh-history-fold

Topics: dsh · dsh-plugin · deepseek-harness · history-fold · lossless-archive ·

context-compaction · non-summary · recoverable · extractive-index

Is a long DSH session costing you more on every single turn — and getting slower while it does?

Every request re-sends the whole visible history, and a context window is billed by the token. A conversation a few hundred turns deep pays for all of them, every time, whether or not any of it is still relevant.

This plugin folds the oldest part of that visible history into a single message backed by a lossless local archive — the request gets smaller, and nothing is thrown away.

• Inert by default. A fresh install reads nothing, writes nothing and logs nothing

until you configure it.

• Lossless. Every folded message is written to a local archive *before* the fold is

committed, and the replacement tells the model exactly where to look.

• No `node_modules` patching. It uses the same public surface-replacement mechanism as

the shipped compactor, so a DSH upgrade cannot break it.

• Also works on DSH 0.2.x (0.2.0-rc.2, 0.2.1-rc.2), which changed four things this plugin

depends on — two of which fail silently. See DSH 0.2.x for what changed and how each was measured.

• Declares what it supports. engines.dsh is set to >=0.1.0-rc.2 <0.3.0-0, so a loader that

checks compatibility can tell you *before* mounting it rather than after it misbehaves.

Why

Measured on one machine, a long session against a fresh one:

· long session · fresh session history tokens · 497,141 · small output / input ratio · 0.20% · 0.64% decode speed · 59 tok/s · 176 tok/s time to first token · ~2.5 s · ~2.5 s

So context length does not slow the first token; it slows decode, because attention runs over the whole KV cache. Shrinking the visible history is what brings the decode rate back.

What it does

Before a turn's first step, if the visible surface is above the trigger — triggerRatio × contextWindow, or an explicit absolute triggerSurfaceTokens, see how the thresholds are resolved — the plugin:

1. Picks the oldest run of nodes that ends on a tool-pairing balanced cut, leaves at

least keepRecentNodes behind, and brings the surface down to about the target (targetRatio × contextWindow, default 0.18).

1. Archives every folded node, verbatim, to

$DSH_HOME/history-fold/archive/.jsonl. If this write fails the fold is abandoned — history is never dropped without a copy on disk first.

1. Replaces that range with one user/message, using the same mechanism as the shipped

compactor:

session.append('user/message', message, {
     surfaceOp: { op: 'replace', start, end },
     sourceEventSeqs: [/* every shadowed seq */],
   })

dsh-session then splices the replacement into the old range's position, so chronology is preserved and the conversation still reads in order.

If that fold did not land, there is a second chance. When the next turn still finds the surface at or above emergencyTriggerRatio (0.8 of the window, 800k on a 1M window), the normal attempt clearly did not bring it down — keepRecentNodes covered the whole foldable span, the minFold* floors refused a small fold, or an archive write failed and the fold was abandoned. The plugin then retries with the recent window cut back to emergencyKeepRecentNodes (12) and the floors lowered to emergencyMinFoldNodes / emergencyMinFoldTokens. It is the same bounded fold: the same archival of every node, the same refusal to split a tool-call/tool-result pair. It just stops being polite about how much history it holds back — and every line of that fold carries emergency: true in the log, so a reader can tell the two paths apart afterwards.

The replacement is an index, not a summary

The message that enters the context window is a notice plus an extractive index — every sampled line is the real prefix of that message's own text. No model is called, and nothing is paraphrased:

[dsh-history-fold] 614 earlier messages (surface seqs 223995..387899) were folded out of the
visible history so this session stays fast.
NOTHING WAS DELETED. Their full text is archived at:
  /…/.dsh/history-fold/archive/session-….jsonl
To read any of it verbatim, search that file with the pwsh or grep tools, for example:
  rg -n "search words" "/…/.dsh/history-fold/archive/session-….jsonl"
The one-line index below is extractive, not a rewritten summary: read it to find what you
need, then search the archive for the exact text. Do not guess about folded content.
(sampled: every 3th of 614 folded messages)
[0] user/message (plugin) seq=223995: This is an automatically generated checkpoint …
[3] assistant/message (model) seq=224097: Now the fireWeapon for bolt does a spread burst …

If the index would exceed digestMaxChars it samples every *n*-th line rather than truncating, and says so.

Nested paths: a path inside a path

The in-context index is sampled once it overflows, and the sampling step is capped at 64 — so the more often a session folds, the coarser the index of its oldest spans becomes, until an old span is reduced to "somewhere in seqs A..B". That is the part that does not scale.

So every fold now does one more thing: it writes the unsampled index of the range to disk and leaves only its path in context.

the folded message in context
  |- path -> $DSH_HOME/history-fold/index/<session>/f100-4500.md  <- full index of that range
  |            \-- path -> f20-99.md                              <- an earlier fold's full index
  |                         \-- path -> f0-19.md
  \-- path -> $DSH_HOME/history-fold/archive/<session>.jsonl      <- the raw text of everything

Two rules make the chain hold:

1. Each folded message carries a machine-readable marker:

[fold-index] seqs=A..B messages=N file=. When that message is itself folded later, the marker is parsed back out — so the older span is represented by a pointer, not by a 160-character stub of an index.

1. That line is never sampled away. Sampling drops ordinary message lines only. Dropping a

fold-pointer would silently *orphan* everything older than it: nothing left in context could reach it.

Every tier file is self-contained on disk, and it lists the tier files of any older folds that it covers (## older tier files). Context keeps the shortest layer; walking down the paths reaches the raw text. Reading a tier file costs nothing until it is read — and it is never read unless it is needed.

Measured on a real 1,210-turn session (surface 213,919 tokens):

· folded · 614 messages / 184,709 tokens / 86.3% marker that enters context · 30,232 characters (~7.5k tokens) — about 24:1 archive written · 638 KB user-authored text preserved · verbatim assistant text preserved · verbatim (the archive is a superset — it also keeps reasoning)

Because the archive holds the real text and the marker tells the model to search it, folded content stays recoverable. Asked about a specific change that lived in the folded range, the model read the archive and answered with the exact values; where content genuinely was not in context it answered "not in context" instead of guessing.

Install

One command, straight from the repository:

dsh plugin add --profile web github:mili1360/dsh-history-fold

That resolves the package over git, adds it to the profile's dependencies, and registers the bundle in dsh.profile.bundles on its own — verified on 0.2.0-rc.2. Then restart DSH: a registered bundle is only picked up at boot.

Works the same way from a local clone, if you would rather build from source:

git clone https://github.com/mili1360/dsh-history-fold.git
dsh plugin add --profile web ./dsh-history-fold

Doing it by hand instead

This is a plugin for a DSH profile. Add it to the profile — the same layer every other DSH plugin uses.

1. Make the package resolvable by your profile, then add it to the profile manifest

~/.dsh/profiles//package.json:

{
     "dependencies": { "dsh-history-fold": "link:/path/to/dsh-history-fold" },
     "dsh": { "profile": { "bundles": ["@deepseek-ai/dsh-base", "dsh-history-fold"] } }
   }

Or, using the explicit patch layer, append to ~/.dsh/profiles//cordis.patch.yml:

- insert:
       - id: history-fold
         name: 'file:///path/to/dsh-history-fold/lib/index.js'

1. Restart DSH. Plugin rows load at boot.
2. It is already on. The shipped defaults fold for real — but only above triggerRatio

(0.75 of the model's context window), which sits just below the built-in compactor's 0.8, and every folded node is archived verbatim before it leaves the visible history. Installing it therefore does not leave the request untouched: it profiles from the first turn and folds once a window is 75% full.

To watch it without letting it write anything, create ~/.dsh/dsh-history-fold/config.json:

{ "dryRun": true }

dryRun: true logs what it *would* fold without writing anything. Watch ~/.dsh/logs/history-fold.jsonl until the numbers look right, then set "dryRun": false again (or delete the file and the default takes over).

To remove it, delete the row you added and restart. Nothing else is left behind; the archive directory is the only thing this plugin creates.

Requirements / compatibility

After installing, confirm it actually works — a profile that boots is not proof. A plugin can load cleanly, serve its routes and still break the host; we have seen exactly that. So the acceptance test is: create a new session and send it one message, then check $DSH_HOME/logs/history-fold.jsonl for a profile line (observer mode) or an idle / would-fold / folded line. If new sessions cannot be created at all, remove the mount line above and restart.

Developed and tested against DSH `0.1.0-rc.7` — a customised build, not a stock upstream release. Another version may register its services or name its events differently: this plugin touches ctx.on, ctx.effect, ctx.tokenMeter, ctx.reflect, ctx.llm and the agent/pre-step event.

DSH 0.2.x (0.2.0-rc.2, 0.2.1-rc.2) — supported

Verified on 0.2.0 end to end: three consecutive folds, every turn completing, the folded message persisted to session.v4.jsonl.zstd, the session reloaded after a restart and kept folding, and the nested tier link present on disk.

On 0.2.1-rc.2 the plugin was mounted into a clean profile in its own DSH_HOME and came up (log line {"mark":"boot","plugin":"dsh-history-fold","version":"...","tools":true}), which also proves ctx.tools is available there. Its hook signatures are unchanged in 0.2.1-rc.2: tools/pre-execute (exec, next), tools/post-execute (exec, result, next), session/event (session, event). One thing to know: `agent/pre-step` does not exist in 0.1.0-rc.8 at all (the string appears only in comments there) and does exist in 0.2.1 — so a listener on it is inert on 0.1.x and becomes live after the upgrade.

Four of its API changes had to be handled, and three of them fail silently:

what changed · 0.1.x · 0.2.0 · if you get it wrong event accessor · session.events, entries carry seq · no such member; the log is positional (eventAt(seq) is log[seq]), plus snapshotEvents() · every node reads as unknown / 0 bytes, no cut is ever balanced, no-safe-cut forever — no error surface-replace range · {op:'replace', start, end} · {op:'replace', **startSeq, endSeq**} · isReplaceOp throws: the fold is rejected surface head · no system message in the surface · node 0 is the system prompt and is protected · any replace reaching node 0 throws: "may be rewritten only by a system/message over exactly that node" message source · {kind:'plugin', plugin:X} · that wrapper is retired; a message needs a producer-owned kind — plugin: for a third-party plugin · the append succeeds, then ROW ADMISSION throws when the log is written ⇒ the turn never ends, the session stops persisting, and every later turn hangs too. Nothing is logged

The plugin picks the replace shape and the source spelling by host, and starts the fold *after* the protected head; the head's tokens stay in the reach-the-target arithmetic. Log lines name what was used (surfaceOpKeys) and how many leading nodes were skipped (protectedCount).

If your build does not match, the plugin deactivates and says so — it does not stop the profile from booting. Nothing is registered, and one line lands in logs/history-fold.jsonl:

{"mark":"deactivated","reason":"registration-failed","step":"services","error":"..."}

If the profile will not start at all after installing, remove the mounted lines from /cordis.patch.yml (or drop "dsh-history-fold" from dsh.profile.bundles, if that is how you installed it) and restart:

- insert:
    - id: dsh-history-fold
      name: 'file:///path/to/dsh-history-fold/lib/index.js'