dsh-hypercompact

dsh-hypercompact

Deterministic, zero-LLM, byte-budget context compaction for DeepSeek Harness (DSH), with byte-exact recall of everything it compacts.

• Source code: github.com/mrbeandev/dsh-hypercompact
• npm package: npmjs.com/package/dsh-hypercompact

Long DSH sessions eventually fail because every request re-uploads the whole conversation. dsh-hypercompact keeps that request small: when it grows past a byte limit, it compacts old history locally in milliseconds, with no extra model call. Everything you typed stays word for word, and the agent can restore any compacted tool output exactly with the recall tool.

Contents: Supported DSH versions · Installation · Preset not in the picker? · Check that it is working · Which sessions use it? · Uninstall · Why this plugin exists · Configuration · How it works · Recall tools · Requirements · Known limits

Supported DSH versions

dsh-hypercompact 0.2.4 works with DSH 0.1.5-rc.2 through 0.2.x (not 0.3.0 or later). Check your installed Harness with dsh --version.

Which dsh-hypercompact version do I need?

"Refused" means the plugin logs a clear error at startup and does not load; it does not attempt compaction. Releases in the allowed range have different preset installation models (Step 2 below), and an API check alone does not establish full runtime compatibility. After upgrading DSH, see Upgrading DSH.

Installation

Installing takes three steps: add the package, create a preset that uses it, and start a new session on that preset. The package on its own does nothing. DSH picks a compaction engine per preset, so the plugin runs only in sessions that use a preset containing it.

Step 1: add the package to your DSH profile

dsh plugin --profile web add dsh-hypercompact

# GitHub
dsh plugin --profile web add github:mrbeandev/dsh-hypercompact

# Local checkout (keep the folder in place: the profile links to it)
git clone https://github.com/mrbeandev/dsh-hypercompact.git
dsh plugin --profile web add "link:$PWD/dsh-hypercompact"

Step 2: create the "Hypercompact" preset

node ~/.dsh/profiles/web/node_modules/dsh-hypercompact/scripts/create-preset.mjs

This copies DSH's built-in standard preset and changes one row: the compaction engine. Your existing presets are not touched. The script detects your DSH version and does the right thing for it:

• DSH 0.1.7 and 0.2: it writes a small preset bundle to ~/.dsh/hypercompact/preset-hypercompact/ and installs it into the profile for you (it runs dsh plugin --profile web add itself). Output ends with:

  + dsh-hypercompact-preset link:~/.dsh/hypercompact/preset-hypercompact
  

• DSH 0.1.5: it writes the preset folder ~/.dsh/.agent-presets/hypercompact/:

  created preset "hypercompact" at ~/.dsh/.agent-presets/hypercompact
  

Not sure which DSH you have? Run dsh --version.

• Windows: node %USERPROFILE%\.dsh\profiles\web\node_modules\dsh-hypercompact\scripts\create-preset.mjs
• Custom DSH_HOME: use that directory instead of ~/.dsh.
• Local checkout: node /path/to/dsh-hypercompact/scripts/create-preset.mjs

Step 3: restart DSH and start a new session on the preset

1. Stop DSH web and start it again (dsh web).
2. Click New session.
3. Before you send the first message, open the preset picker and choose Hypercompact (standard).

To use it for every new session instead, make it the default:

• DSH 0.1.7 and 0.2: in the web UI's preset picker, set Hypercompact (standard) as your default preset.

• DSH 0.1.5: add this to ~/.dsh/settings.yaml and restart DSH:

  agent-presets:
    default: hypercompact
  

The preset does not appear in the picker

On DSH 0.1.7 and later, your profile's own ~/.dsh/profiles/web/cordis.patch.yml applies after every bundle. If it still contains a preset-hypercompact row, typically disabled: true left behind when the preset was disabled or uninstalled, or an inline declaration from an older version of the script, that row hides the preset.

Upgrade to dsh-hypercompact 0.2.3 or later and run the script again with --force. It removes those stale rows, keeps the previous file as cordis.patch.yml.bak-hypercompact, and prints what it removed:

dsh plugin --profile web add dsh-hypercompact@latest
node ~/.dsh/profiles/web/node_modules/dsh-hypercompact/scripts/create-preset.mjs --force

Then restart DSH. To fix it by hand instead, delete every preset-hypercompact row from that file (the preset bundle declares the preset itself).

Check that it is working

In the session, type:

/hypercompact

• It prints a status report (request size, trigger values, last compaction): the session is using dsh-hypercompact.
• The command is unknown: the session is on another preset. Start a new session and pick Hypercompact (standard) before the first message.

Compaction then happens on its own when the request grows past 5 MB. You can also run /compact at any time.

Which sessions use it?

Moving an existing conversation over: start a new Hypercompact session and mention the old one with @ (pick it from the list). DSH inserts a size-limited snapshot of that session so the agent can continue from it.

Upgrading DSH

After you upgrade DSH (for example 0.1.5 → 0.2), do two things:

1. Update the plugin, since a plugin release only accepts the DSH versions it knows:

   dsh plugin --profile web add dsh-hypercompact@latest
   

2. Regenerate the preset. It is a copy of DSH's standard preset at the time you created it, so re-run the script to follow the new version:

   node ~/.dsh/profiles/web/node_modules/dsh-hypercompact/scripts/create-preset.mjs --force
   

Then restart DSH.

Going from 0.1.5 to 0.1.7 or later, this is required: newer DSH no longer reads ~/.dsh/.agent-presets/, so the old preset silently disappears from the picker. The script creates the new-style preset and deletes the old folder it made.

Script options: --from ptc (copy another built-in preset), --id my-preset, --profile tui (default web), --force (overwrite / regenerate), --print (preview, write nothing), --no-install (0.1.7+: write the bundle but don't run dsh plugin add), --remove (delete everything the script created).

Uninstall

Remove the preset first. A preset that still names the package cannot load once the package is gone.

node ~/.dsh/profiles/web/node_modules/dsh-hypercompact/scripts/create-preset.mjs --remove
dsh plugin --profile web remove dsh-hypercompact

--remove uninstalls the preset bundle (0.1.7+) or deletes the preset folder (0.1.5). If you made Hypercompact your default preset, pick another default (or remove agent-presets.default from settings.yaml on 0.1.5). Sessions that were already compacted stay readable: their checkpoints are ordinary compaction checkpoints in the session log.

Add it to a preset by hand

To put the plugin in your own preset instead of using the script, replace the compaction-basic row inside the preset's compaction group. Keep the isolate block: /compact and the engine must share that group.

- id: compaction
  name: cordis:group
  group: true
  isolate:
    compaction: true
    toolResultPruner: true
  config:
    - id: hypercompact            # was: compaction-basic
      name: dsh-hypercompact
      config:
        maxRequestBytes: 5000000
        targetRequestBytes: 1500000
    - id: command-compact
      name: '@deepseek-ai/dsh-command-compact'
    - id: tool-result-pruner
      name: '@deepseek-ai/dsh-compaction-tool-result-pruner'

If you ever add a dsh-hypercompact row to a profile's cordis.patch.yml by hand, delete it before uninstalling: a row naming a missing package stops the profile from booting.

Why this plugin exists

Every model call in a long agent session re-uploads the whole conversation. On a slow or proxied connection the request body size, not the token count, is what fails: a 20 MB request can take minutes to upload and hit a proxy's 100-second timeout before the model ever sees it. DSH then retries the same oversized request.

The built-in engine (@deepseek-ai/dsh-compaction-basic) cannot help there:

• it triggers on a fraction of the model's context window, which a 1M-token model may never reach even while the body is far too large to upload;
• it summarizes by sending the whole history to an LLM, which is the same oversized upload that is already failing.

dsh-hypercompact measures the next request in bytes and compacts locally in milliseconds, with no model call. Measured on real sessions, inline screenshots were 50–92 % of the request body, and tool traffic most of the rest. The plugin targets both.

Overview

• What you wrote is never cut. Every human message stays verbatim in the checkpoint, through any number of later compactions. Your constraints ("never …", "do not …", "only …") are pinned at the top. The same protection covers a parent agent's task prompt in a subagent session, /goal round prompts, and team messages.
• Old tool calls collapse to one line each: tool name, key arguments, a +N/−M lines summary for edits, and a pointer to the original. Runs of read-only calls (read ×7: a.js b.js …) become one line.
• Old tool results collapse to status, size, a short head/tail excerpt, and a pointer. Failed results keep a larger excerpt.
• State of work. A block lists the last files edited, the last commands, and unresolved errors.
• Images. Older inline images become text labels, captioned with what the assistant said about them.
• Housekeeping. Between the target and the trigger, a light pass trims old tool results in place instead of rewriting history.
• Recall. The model gets recall, which restores any original byte-exact from the append-only session log (or just the matching lines of a large one), plus recall_search. You get /recall and /hypercompact.
• Drop-in. It implements the public ctx.compaction contract, so /compact, the token meter and context charts keep working. No client bundle, no settings UI, no patching of harness internals.

Measured results

Real sessions, measured offline with npm run measure (see Development) through pi-ai's own OpenAI-completions serializer, default configuration. No LLM calls are made.

Degradation 0 means only reasoning was dropped; 1 means tool-result excerpts were dropped (call lines and pointers stay). No entry was elided in any session. Every compacted request had 0 orphaned tool results and 0 unanswered tool calls, and recall returned a sampled tool result byte-identical to the log.

Configuration

All keys are optional. Unknown keys are rejected at mount, so a typo fails loudly instead of silently using a default.

Why these defaults

• 5 MB trigger. On a slow uplink (100–250 KB/s), 5 MB uploads in about 20–50 s, inside the common 100 s proxy/CDN timeout even at the low end.
• 1.5 MB target. About 6–15 s per request after compaction, with 3.5 MB of growth before the next compaction (each compaction is one cache miss).
• retainTurns: 2, retainBytes: 400000. The current and previous turn stay verbatim, so the agent never loses the work in progress.
• contextRatio: 0.85. A token safety net for a small-window model whose session stays under the byte trigger.
• keepRecentImages: 2. The screenshots the agent is currently looking at survive, and older ones, the dominant cost in measured sessions, do not.
• minCheckpointBytes: 300000. Measured: with a smaller floor, sessions whose retained tail is image-heavy got a tiny budget and elided hundreds of entries; at 300 KB every measured session compiled with degradation ≤ 1 and nothing elided, while still landing near the 1.5 MB target.

On a fast connection with a large-window model you can raise both byte values (for example 20 MB → 6 MB); the token trigger still protects the context window.

Per-model policies. None ship by default, because provider and model IDs are deployment-specific; provider and model must match your route exactly:

modelPolicies:
  # a route behind a slow proxy: keep every request under ~30 s at ~100 KB/s
  - { provider: my-gateway, model: my-model, maxRequestBytes: 3000000, targetRequestBytes: 1000000 }
  # a 128K-window model: fire the token trigger early
  - { provider: deepseek, model: deepseek-chat, contextRatio: 0.7 }

How it works

Trigger. Before every model step (agent/pre-step), the engine estimates the next request body: each surface message serialized as JSON, inline images as base64, plus the tool schemas and a fixed envelope. Per-message sizes are cached by log seq and the log is folded incrementally, so a step costs O(new events), not a rescan of the session. It compacts when the estimate reaches maxRequestBytes, or when a token trigger fires.

Selection. It replaces the oldest span of the surface. The span never includes the system prompt, never cuts between an assistant tool call and its results, prefers turn boundaries, and never touches the newest retainTurns turns or retainBytes bytes. It compacts enough that the result lands near targetRequestBytes.

Checkpoint. The span is compiled into one user message with three parts:

1. Pinned: one line per instruction message (seq + first line), every constraint sentence in them, verbatim, and the files changed with the seq of their last write. Carried forward and merged by every later compaction. Over pinnedBytes, the index is trimmed first (newest 40 lines plus one "N older user messages: seq a–b" line), then the files list, and constraints last.
2. History: the span in order. Instruction messages appear verbatim between [user seq N] and [/user seq N] (or [agent-message seq N], [goal seq N], [team-message seq N]). Assistant text is head + tail. Each tool call is one line with its result's status, size and excerpt; runs of read-only calls are one line. Harness-injected context (agent instructions, subagent reports) is shortened. Reasoning is dropped.
3. State of work: last files edited, last commands and their status, errors not followed by a success, and the last human request.

If the checkpoint exceeds its budget, the compiler degrades in fixed passes: drop success excerpts, then shorten assistant text (never below 1000 chars) and context, then elide the oldest unprotected entries (tool lines, assistant text, context) behind [N entries elided, seq A–B] lines. Human text, pinned lines, failed-call excerpts, and entries the model recently recalled are never elided. If they alone exceed the budget, the budget is exceeded (logged, and recorded in the stats) rather than dropping what the user said. An earlier checkpoint in the span is re-parsed, not copied: its human blocks stay protected with their original seqs, its pinned lines merge, and its tool lines become elidable, so repeated compactions stay bounded.

A compiled tool line looks like:

• edit file_path="src/net/retry.ts" old_string="const timeout = 30_000" new_string=<3.2 KB> [+84/−1 lines] (seq 1822 → result 1824 ok 23 B)
    "Edited src/net/retry.ts"

Commit. The same durable transaction as the built-in engine: compaction/start → compaction/summary → user/message (surface replace) → compaction/end, with no await in between. The summary event carries the standard fields (shadowedSeqs, shadowedTokenCount, …) that /compact and dsh-context read. The original events stay in the log. If the request is still above target, older images in retained tool results are replaced through the harness's content-only tool-result rewrite (th