billion-context

billion-context

npm install -g billion-context --prefix=~/.local

                                           

Cache health at a glance: a healthy session keeps a 95–97% prefix-cache hit rate — compression itself costs ≤2%. Sustained lower? Check attribution with /acp or /acp-cache (see FAQ); usual causes, in order: upstream cache TTL expiry · model switch · a bili bug (please report) · other/unknown.

Community

QQ Group: 1056132097 (full) 1108730198 (open)

📄 Paper / Preprint

• Model-Driven Incremental Hierarchical Compression: Training-Free Multi-Generational Context Management for Long-Lived Coding Agents (English, v0.2)

📝 The paper itself is open-sourced under the MIT License as part of the codebase (`paper/`). It is a living document — anyone may edit it; improvements are welcome via pull request.

A production-scale longitudinal study: 4.5 months, three hosts, 174,327 model calls, 18.76B cumulative input tokens (~24.7B across all hosts), zero window violations on 204,800-token models, marathon sessions of 8,584–12,049 calls.

billion-context sits between any agent and its model API, rewriting Anthropic/OpenAI streams with acp-kernel compression. The model decides when and what to compress into high-fidelity summaries — not a hard truncation limit.

Why

Long coding sessions blow up context. Each provider charges per token, and once you pass the context window the session degrades or dies. billion-context compresses consumed conversation into layered summaries so you can run a single session for days — billions of tokens through one context window.

Unlike a host's built-in summarizer, compression here is incremental, reversible, and prefix-cache friendly: summaries are written in small ranges, can be decompressed on demand, and the cache prefix stays intact.

How it works

Agent (Claude Code / Codex / Cursor / Aider ...)
        │  you point the agent's base URL at the proxy
        ▼
┌─────────────────┐
│  billion-context│   1. parse the request (Anthropic or OpenAI shape)
│     proxy       │   2. run acp-kernel compression on the conversation
│                 │   3. inject a `compress` tool + compression philosophy
│                 │   4. forward to the real model API
│                 │   5. rewrite the streaming response
└─────────────────┘
        │
        ▼
   real model API (Anthropic / OpenAI / compatible)

Context-management tools

The proxy injects four context-management tools into the conversation; the model calls them itself as context grows, and the proxy executes compress server-side so folded ranges stay summarized in history until restored:

• `compress` — fold a message range into a detailed summary.
• `decompress` — restore a compressed range when exact details are needed again.
• `search_context` — keyword search over compressed summaries and visible messages.
• `acp_status` — context-usage overview plus which ranges are still compressible.

Which do I need?

Pick by your client:

Client · Use pi · billion-context — bili pi (launcher) or bili plugin install pi (native); standalone billion-context-pi remains usable — details: CLIENTS.md opencode (1.x / 2.x) · billion-context — bili opencode (launcher) or bili plugin install opencode (native); standalone opencode-acp remains usable on 1.x — full guide: OpenCode omp · billion-context via bili omp (built-in plugin) or bili plugin install omp (self-spawning native plugin, no launcher) dsh · bili dsh (launcher — full native plugin via --patch) or bili plugin install dsh ≡ dsh plugin --profile add billion-context (one unified lane) — details: CLIENTS.md kimi · bili plugin install kimi (self-spawning native, Kimi Code ≥ 2.0.0) or bili kimi (cert-MITM) or /bili/ prefix — details: CLIENTS.md hermes · bili plugin install hermes (self-spawning native, Python plugin #958) or bili hermes (cert-MITM) zcode (Z.ai / bigmodel coding plan) · bili plugin install zcode (self-spawning native, #1145) or cert-MITM via the GUI's Settings → Network or /bili/ prefix — details: CLIENTS.md claude · bili claude (launcher) or bili plugin install claude (native posture, #964 — managed settings block + session-owned proxy; see the notes below) codex · bili codex (launcher — the full zero-config posture) or bili plugin install codex (MCP-shell tools companion: start bili first — the shell never spawns a proxy and never routes codex's own traffic) — details: CLIENTS.md jcode · billion-context via bili jcode (cert-MITM) or /bili/ prefix — no native mode (compiled Rust binary, no plugin seam, #962) gemini (Gemini CLI) · bili gemini (launcher, GOOGLE_GEMINI_BASE_URL /bili/ rewrite) or /bili/ prefix — launcher-only (no in-loop tool seam, #1043) iflow (iFlow CLI) · bili iflow (launcher, IFLOW_BASE_URL /bili/ rewrite) or /bili/ prefix qwen (Qwen Code) · bili qwen (launcher, cert-MITM) or /bili/ prefix antigravity (Antigravity CLI / agy, Google) · bili antigravity (launcher, CLOUD_CODE_URL /bili/ rewrite of daily-cloudcode-pa.googleapis.com — Antigravity's real default upstream, #3129) or /bili/ prefix — no plugin seam (closed Go language_server; its user-plugin surface is additive-only, #2115) — details: CLIENTS.md mcode (MiniMax Code) · billion-context via bili mcode (cert-MITM) or /bili/ prefix — no native mode (event hooks only, no model-request seam, #1050) aider · billion-context via bili aider (cert-MITM) or /bili/ prefix — no native mode (shell-command-only hooks, no tool-injection seam, #1048) copilot (GitHub Copilot CLI) · bili copilot (launcher, cert-MITM) — closed Go binary, no plugin seam (#1049) amp (Amp CLI) · bili amp (launcher, cert-MITM) — closed Go binary, no plugin seam (#1049) crush (Charm Crush) · bili crush (launcher, cert-MITM) — open-source Go binary, no plugin seam; built-in provider hosts whitelisted, custom base_urls auto-discovered from crush.json (#2340) zed (Zed editor) · bili zed (launcher, cert-MITM) — open-source Rust editor, no plugin seam; reqwest honors HTTPS_PROXY, CA via SSL_CERT_FILE (Linux env probing); built-in provider hosts whitelisted, custom api_urls auto-discovered from settings.json; loopback providers (ollama/lmstudio) stay direct via NO_PROXY (#2340) goose (Goose CLI) · bili goose (launcher) — rustls trusts no CA file, so no cert-MITM: openai/anthropic legs via OPENAI_HOST/ANTHROPIC_HOST, custom providers via a regenerated GOOSE_PATH_ROOT overlay (#1049) everything else (no context hook) · billion-context — bili (launcher, preferred) or /bili/ prefix

Native mode vs standalone extensions. The host-native plugins (bili plugin install …) and the standalone in-process extensions (billion-context-pi, opencode-acp) are mutually exclusive — both active means double compression. The installer makes the switch: it replaces the legacy entries (bare name, npm: alias, versioned, path form; array or object shape) and snapshots the original config to .bili-bak once; a project-local install is not touched — remove that one by hand. As a runtime safety net for manual installs, the native entries set BILLION_CONTEXT_NATIVE= synchronously at load so a standalone extension can stand down at action time. On the pi side the marker needs billion-context-pi 0.1.72+, and the pi-native entry scans both pi settings files once its proxy is up and warns loudly when it spots a co-resident legacy entry the installer never saw — that warning is the only visible signal while an old billion-context-pi silently double-compresses.

Install

Linux / macOS — install with a user-level prefix (no sudo, no npm config changes, and bili's self-update never hits permission errors):

npm install -g billion-context --prefix=~/.local

The bili command lands in ~/.local/bin — already on PATH in most distros; if not, add export PATH="$HOME/.local/bin:$PATH" to ~/.bashrc or ~/.zshrc. On nvm or Homebrew Node the default prefix is already user-owned — a plain npm install -g billion-context works as-is. On Windows the default prefix (%APPDATA%\npm) is also user-writable — plain npm install -g billion-context.

This installs the bili command (bili-proxy is kept as an alias). Hitting EACCES with an old root-owned prefix? Reinstall with --prefix=~/.local (pass the flag again on any future npm reinstall of bili) — that is the permanent fix; avoid sudo.

Prefer containers? A Dockerfile ships at the repository root — see Running in Docker below.

Quickstart

Three ways to use it — pick one:

• Native plugin (most native): bili plugin install — bili

becomes a plugin inside the client; start the client as usual.

• Launcher (config-free): one bili command brings up the proxy and

the client together — no real config file is ever touched.

• URL change (most universal): prefix your client's baseURL with the proxy

origin + /bili/.

Mechanism details behind these three options (plugin lifecycle, runtime-info protocol, injection priority) live in TECHNICAL-NOTES.md.

Ports, briefly (#1660): bili start (manual) owns 8787. Everything a lane spawns for you (native hooks, launcher lanes) lives in a separate self-managed zone starting at 18787 — collisions hop +1 and each lane remembers its drift, so zero-config installs never fight you for a port, and a deliberate bili start daemon is attached by default. Attach is lane-aware (#2905): a hook rides its own lane's instance first, NEVER another declared lane's — except lanes you list in native.attachLanePriority — and the 8787 user-zone daemon stays the LAST-resort fallback. An upgrade-restart that finds the previous build still draining on the lane's port waits for it to release (up to 5s) and rebinds the SAME port instead of drifting (#1723); only a genuinely occupied port hops +1 — and that hop is now logged loudly.

Option 1 — Native plugin (bili plugin install pi / omp / opencode / dsh / kimi / hermes / zcode)

The proxy lives inside the client: install once, then start the client exactly as you always do — no launcher command, no env vars, no fixed port, no URL edits. Supported today for pi, omp, opencode (1.x and 2.x), dsh, kimi, hermes and zcode:

bili plugin install pi          # registers a "billion-context" entry in pi's settings (npm form when bili itself was npm-installed)
bili plugin install omp         # registers an extensions entry in omp's config.yml (~/.omp/agent/config.yml)
bili plugin install opencode    # registers the plugin in opencode's real config + disables native auto-compaction
bili plugin install dsh         # runs 'dsh plugin --profile <name> add billion-context' for every existing profile
bili plugin install kimi        # writes $KIMI_CODE_HOME/plugins/managed/billion-context/kimi.plugin.json (+ installed.json record); per-session routing block lands in config.toml on first start (Kimi Code >= 2.0.0)
bili plugin install hermes      # copies the Python plugin into ~/.hermes/plugins/billion-context/ (+ machine-owned bili.json sidecar) and enables it via `hermes plugins enable billion-context`
bili plugin install zcode       # writes hooks.enabled + a SessionStart hook + mcp.servers.bili into ~/.zcode/cli/config.json; per-session routing lands in the bigmodel provider store on first start
bili plugin remove <client>     # undo (dsh removes through the same channel; config snapshots go to .bili-bak)

Where a client has its own plugin channel you can also install natively, skipping bili commands entirely:

• dsh: dsh plugin --profile add billion-context is the very

command bili plugin install dsh drives per profile (npm form only; a git checkout has no published entry — pasting a GitHub address installs source without dist/ and the bundle never loads) — same end state either way (pnpm into the profile, bundled patch layer mounted by dsh itself); remove through the same channel. See the dsh section below.

• opencode: add the bare npm name to your real config's plugin list —

"plugin": ["billion-context"] (npm form only; a git checkout has no published entry). The package publishes exports["./server"] → dist/agent/opencode-native.js, so opencode loads it through its own Npm.add machinery and the plugin self-spawns exactly like the bili-installed form. Do the two things the bili installer would have done for you too: set "compaction": { "auto": false } in the same config (otherwise OpenCode's native auto-compaction double-compresses) and keep a manual backup of the file first.

• claude: this repository doubles as a Claude Code plugin marketplace —

/plugin marketplace add ranxianglei/billion-context, then /plugin install billion-context@billion-context, then run /billion-context:bili-setup (it drives bili plugin install claude for you and tells you to restart). Same end state as the bili installer; the plugin ships no hooks or MCP entries of its own, so nothing double-registers.

For pi / omp / kimi / claude there is no client-side channel — bili plugin install writes their config entries for you (kimi's declarative kimi.plugin.json + registry record, claude's managed settings block, …).

Notes:

• Native mode is mutually exclusive with the standalone in-process extensions (billion-context-pi, opencode-acp) — the installer swaps the entries and snapshots the original config (.bili-bak).
• pi: native installs wire in built-in subagents by default (acp_delegate / acp_delegate_wait / acp_delegate_cancel) — how to disable them while keeping context compression: Pi built-in subagents.
• OpenCode legacy sessions, V1/V2 shapes and caveats: OpenCode.
• kimi reports runtime-info at bootstrap only (static headers can't carry per-request window/model values); subagent tool calls are routed by the proxy's outbound tool-use witness ring (#1685) — no model-visible conversation id.
• hermes's native plugin is Python: it points hermes' httpx stack at the proxy via env vars after a health check and stamps per-request headers through an llm_request middleware.
• codex is the one client a plugin install cannot make self-sufficient: codex routes model traffic via env only (no config-file routing seam for the default ChatGPT-login provider — a managed model_providers block would force API-key auth and drop subscription login), and an MCP server cannot inject env into its parent. bili plugin install codex writes a single [mcp_servers.bili] block into ~/.codex/config.toml (command = node, args = dist/mcp.js) exposing the four ACP tools; at session start the shell resolves a proxy — env BILI_MCP_PROXY > the live-instance record (any lane's proxy or a bili start daemon) > the 8787 user-zone default (#1660 removed the install-time origin bake, #403) — nothing reachable → tools/list fails with -32003. So: start bili first (bili start or any client's lane proxy), export HTTPS_PROXY yourself if you also want compression, or use bili codex for the zero-config full posture. Mechanics: CLIENTS.md.
• claude has a native posture (#964): managed settings block + SessionStart hook + MCP shell; the hook rides the self-managed port zone (#1660) and re-pins the managed URL to the live origin each session, so port drift self-heals. Since #1821 the MCP shell also watches the proxy mid-session (30 s tick × 3 failures) and respawns a dead lane proxy itself, so model requests recover within ~90 s instead of a new session (same semantics as the zcode lane). Opt out with BILI_NATIVE_CLAUDE=0 (passthrough). Mechanics: TECHNICAL-NOTES.md. Known limitation (#2290): the Claude Desktop app's Code tab (CLAUDE_CODE_ENTRYPOINT=claude-desktop) sets its own ANTHROPIC_BASE_URL on its embedded Claude Code, overriding the managed one — desktop sessions bypass the proxy entirely while the managed block's DISABLE_AUTO_COMPACT=1 still applies, so they get neither bili compression nor native auto-compaction. The hook warns loudly at session start (and records it for bili doctor, which also flags machines where Claude Desktop is installed alongside the native lane); terminal sessions are unaffected. Field-tested 2026-10-06 (#2290): putting HTTPS_PROXY in the settings env block produced zero CONNECT traffic to a live MITM-enabled proxy on Claude Desktop 2.19675.1 / embedded CC 2.1.288 (Windows 11) — the variable reaches no network stack there, consistent with the CLI finding that CC's undici ignores proxy env vars (why bili claude uses the /bili/ base-URL rewrite instead). As of that version there is no usable routing seam on the desktop Code tab; the lane stays tracked in #2290 in case upstream changes behavior. Desktop-only users can restore native auto-compact with bili plugin remove claude.
• zcode has a native posture (#1145): managed ~/.zcode/cli/config.json block + per-session provider baseURL rewrite. Full mechanics: CLIENTS.md.
• jcode and aider have no native mode (no plugin/MCP/tool-injection seam: #962, #1048) — use bili jcode / `bili