dsh-hyper-tools

dsh-hyper-tools

Keeps Charm Hyper model requests under the gateway's strict 10 MiB request-body cap.

Charm Hyper's HTTP gateway (Google Frontend in front of a Go backend) rejects any request body above 10 MiB with:

{ "error": { "message": "invalid request body", "type": "invalid_request_error", "code": null } }

Long conversations accumulate images, and every image is inlined as a base64 image_url part. Once the serialized body crosses the cap, the whole request fails before inference — even though the images are ordinary attachments. This plugin registers one llm/stream middleware for the configured Charm Hyper provider routes:

1. Before dispatch it projects durable history so the accumulated base64 image payload fits maxRequestImageBytes (default 6 MiB). The oldest images become the harness-standard deterministic placeholders ([image omitted to fit request image limits; …]) and the newest images stay. A request that already fits is dispatched untouched through next() — no extra provider attempt.
2. When the gateway still rejects the body (the projection under-counted non-image overhead) it swallows that terminal invalid request body failure and re-dispatches once with the image budget multiplied by recoveryBudgetFactor (default 0.5), which replaces more of the oldest images. If a tighter projection cannot remove anything further, the original failure is surfaced unchanged.

Accounting and placeholders come from the harness itself (offloadRequestImagesWithPolicy, offloadedImageText): base64 length of every image occurrence — nested tool-result images included — and the same deterministic placeholder text shipped adapters use.

Verified behavior

Checked live against https://hyper.charm.land/v1 (deepseek-v4.1-flash, six 1024×683 PNG screenshots):

The gateway cap was measured at 10 MiB (10,485,760 bytes): a 10,400,000-byte body passes the gateway, a 10,490,000-byte body is rejected. scripts/probe-hyper.py reproduces the probe; tests/live.test.ts reproduces the table above.

Install

dsh plugin --profile <profile> add dsh-hyper-tools

From a checkout or tarball:

dsh plugin --profile <profile> add ./dsh-hyper-tools
# or
npm pack && dsh plugin --profile <profile> add ./dsh-hyper-tools-0.1.0.tgz

From git (the built lib/ is committed, so pnpm needs no build permission):

dsh plugin --profile <profile> add github:samuelrubiodev/dsh-hyper-tools#main

The bundle patch (cordis.patch.yml) inserts one row (hyper-tools).

Configuration

Every field is optional; the row config carries them.

- id: hyper-tools
  name: 'dsh-hyper-tools'
  config:
    providers: ['charm-hyper']
    maxRequestImageBytes: 6291456

Notes and limits

• The budget counts image payload only — the platform's documented unit for request-image bounds. Text, tool schemas, and JSON structure ride outside it; the defaults leave several MiB of headroom under the 10 MiB cap, and the recovery retry covers a misestimate.
• The projection estimates each occurrence from durable attachment metadata (attachment.bytes); any provider-side re-encode only makes the actual payload smaller, so the estimate errs toward replacing slightly early.
• Only the routes listed in providers are touched; every other provider request passes through untouched.
• The plugin is adapter-agnostic: it only keys on GenerateOptions.provider, so it guards Charm Hyper whether it is served by @deepseek-ai/dsh-llm-pi-ai, a custom gateway adapter, or anything else.

Development

npm ci
npm run typecheck
npm test                 # unit + integration against the real cordis + @deepseek-ai/dsh-llm runtime
npm run build
HYPER_API_KEY=... npx vitest run tests/live.test.ts   # live check against Charm Hyper

The compiled lib/ is committed so github: installs need no build step; npm pack / npm publish refresh it through prepack.

Compatibility: @deepseek-ai/dsh-llm ^0.1.5-rc.1, @deepseek-ai/cordis ^4.0.1.

License

MIT