dsh-llm-quota-retry

dsh-llm-quota-retry

A DeepSeek Harness host plugin that answers an exhausted account quota or balance: it moves the session to another provider/model that still has allowance when one is configured, and otherwise keeps the step alive by retrying inside the same open turn.

When a model request fails because the account is out of allowance, the ordinary recovery policy gives up once its retry budget is spent and the turn ends. This plugin takes over at that point. If the session's route pool — a named, ordered list of interchangeable provider/model pairs — has a route with allowance, the retry goes there immediately, drawn in proportion to what each route has left; if none has, the same request is retried inside the same open turn once per configured interval — one hour by default, with no attempt limit — until a request succeeds or the turn is cancelled. It is off for a session that owns no setting, so this plugin never takes over a conversation a person is having and never redraws the model their picker chose: work the deployment declares to be delegated opts in on its own — see delegatedChildren. Which route has allowance comes from balance scripts and from the providers that report exhaustion themselves.

Any session can also be put in a pool by hand — the second composer chip or /quota-pool — and a session whose model happens to be pooled follows that pool without being told to. A pool's minimumRemaining decides which routes a session starts on, and the one it starts on is drawn from them in proportion to what each has left; once a session is running, only a provider reporting its allowance spent moves it.

That covers three shapes of the same condition: the canonical QUOTA code, and an AUTH or RATE_LIMIT failure whose text names a spent usage limit. The other two are how metered providers answer — a 403 permission error from one that meters a rolling window, a 429 rate limit from one that meters a Token Plan (已达到 Token Plan 用量上限：…, in Chinese) — and a plugin that keyed on QUOTA alone would miss exactly the cases it exists for.

Install

The package is a Cordis bundle with a Web composer half:

From the directory that contains your checkout of this repository:

dsh plugin --profile web add ./dsh-llm-quota-retry

Installation links the package into the profile and records it in dsh.profile.bundles, so the bundle layer is read on every later start. The host row applies without a restart: the profile patch reloads live, the row activates, and the bundle reports enabled with an active fiber.

The composer chip needs one host restart the first time. The dsh.client manifest is read by the client-module scan, which caches its verdict per package for the lifetime of the host — a package first scanned before it declared dsh.client stays cached as "not a client package", and no later rescan clears that entry. Restart dsh web once after installing, or after adding dsh.client to an already-installed package, and the chip appears.

Composition order

Compose the llm-quota-retry row after @deepseek-ai/dsh-llm-retry.

That order is the contract, not a preference. dsh-llm-retry runs earlier in the agent/request-error waterfall, spends its own retry budget first, and calls the next listener once that budget is exhausted. By the time a quota failure reaches this plugin, every earlier recovery policy has already declined it — which is exactly what "after the base quick retries" means. The bundle's own cordis.patch.yml appends the row to the composition root, so a bundle install lands in the right place automatically.

Row order does not decide listener order. A Cordis waterfall runs its listeners in registration order, and a plugin registers when its services appear rather than when its row is read; reloading a row moves that row's listener to the end. So in a deployment where this plugin happened to activate first, it gets the first word on a quota failure, and a failure the native policy lists as eligible gets no native retry at all. A provider named in nativeRetryFirstProviders is answered the same way either way.

Native quick retries for a named provider

Some providers report ordinary request throttling as a spent allowance — a QUOTA body behind a 429 that clears in seconds. Waiting out an hour for that is the failure this plugin exists to avoid, and on those providers a rate limit must also not count as evidence that the account is out of allowance.

nativeRetryFirstProviders names them. For a named provider, an exhausted allowance runs that provider's own native retry schedule first — the same bounded exponential backoff and the same attempt budget @deepseek-ai/dsh-llm-retry would have run, taken from the resolved retryPolicy the failed request carries (initialDelayMs, maxDelayMs, jitterRatio, maxRetries, and a providerRetryAfterMs the provider asked for while it fits under its own ceiling). Only when that budget is spent does this plugin mark the provider out of allowance, move the session to another route of its pool, and start the hourly wait.

The quick retries run before the exhaustion mark and before any failover, so a provider that was merely throttled keeps its allowance and the session stays on it. They hold the open step for a few seconds, like the long-term wait does, and they announce nothing: the entry, its events, and ctx.llmQuotaRetry state describe the long-term wait, which has not started yet. Progress is in the log:

llm-quota-retry: provider "qwen" returned QUOTA; native quick retry 2 in 4000ms

Three rules decide whether a quick retry happens at all, and each is a case where the native policy, not this plugin, owns the step:

• The provider is named in the row. Every other provider is unchanged.
• The native policy holds a bounded schedule. No retryPolicy at all, and mode: always, which has no budget to spend, leave the failure to the hourly wait. So does a providerRetryAfterMs beyond the policy's maxDelayMs, which the native policy answers by delegating rather than by retrying.
• The native policy has not already spent this step's budget. When this row really does run after dsh-llm-retry and the failure is one that policy retries, its own budget is spent first, and this plugin adds no wait of its own. The plugin reads that from the llm/retry events the Session already records — the one piece of cross-plugin state it depends on, and one it can simply lack: without @deepseek-ai/dsh-llm-retry the quick schedule runs, and with a native policy that never spent its budget, so does it.

A balance reading is deliberately not a fourth rule. qwen answers the same insufficient_quota failure whether the plan is spent or the request was merely throttled, so a reading showing nothing left cannot tell those two apart — and the quick retry is the only thing that lets a throttled request recover in seconds. Skipping the schedule for a low reading would turn one conflated error into an immediate failover, which is the opposite of what this list is for. The cost stays bounded and visible instead: a provider that really has nothing left spends one step's schedule and the long-term wait follows. Shortening that schedule is the provider's own maxRetries and maxDelayMs, not a rule here.

The quick budget is counted per step, like the native one: a step that runs out of it enters the long-term wait, and the next step starts a fresh budget.

- id: llm-quota-retry
  name: 'dsh-llm-quota-retry'
  config:
    # qwen reports a 429 rate limit as `insufficient_quota`, so its own bounded
    # backoff runs first: 2s, 4s, 8s … up to maxRetries, then the hourly wait.
    nativeRetryFirstProviders: [qwen]

What this plugin does not touch

The plugin is strictly additive for every failure except an exhausted allowance, and for that too in every session whose switch is off:

• Every other failure delegates unchanged — rate limits, server errors, timeouts, transports, and context overflows keep their ordinary handling, including compaction's own recovery. So does a credential the provider will keep rejecting, and so does ordinary throttling: an AUTH or RATE_LIMIT failure is read as a spent allowance only when its text names one, so a wrong or revoked API key still fails fast and a burst of concurrent requests still keeps its sub-minute retries, instead of either waiting an hour between attempts that cannot succeed.
• A session whose switch is off delegates an exhausted allowance unchanged. Turning the switch off is the opt-out; every session starts on, and it covers the route move as much as the wait.
• A pooled session's request can be moved before it is sent. That is what a pool is for, and it happens only when the route the request resolves to is unusable — an exhausted mark, or a reading below that provider's minimum — and another route of the session's pool is usable. A session with no pool and no assigned pool is never touched, and a model the user picked in the model picker is left alone as soon as it falls outside the session's pool.
• The native quick retry is never removed. dsh-llm-retry still runs first and its configured retryPolicy still governs. Note that QUOTA is not in the default retryableCodes, so by default an exhausted allowance triggers no native retry at all and this plugin is the only thing that answers it; a provider that lists QUOTA (or runs mode: always) keeps its own retries, and this plugin wakes up only after they are spent. A provider named in nativeRetryFirstProviders runs that same schedule from here when the native policy did not, which is additive rather than a replacement: see the named provider.
• One configuration is superseded rather than merely left alone. retryPolicy.mode: always has no eligible-code list and no budget, and it asks downstream first, so for an exhausted allowance the hourly schedule replaces its own sub-minute backoff. Every other configuration is untouched.

The per-session switch

The response to an exhausted allowance is on until the switch is turned off, and the switch covers the whole response: the move to another route of the session's pool, and the wait-and-retry when no route has allowance. Three paths move the same switch:

• Composer chip — beside the model, permission, and compaction chips, with a menu offering On, Off, and Default.
• /quota-retry <on|off|default|status> — the one write path the chip also uses, so a typed line and a click can never disagree.
• The setting is per session and durable through the llm_quota_retry storage domain, so it survives a host restart. Without a usable storage domain it degrades to a process-local value instead of failing the switch.

A session with no record of its own follows defaultEnabled, which is false unless the row sets it to true, with one exception: a session that is delegated work is on whatever the default says. A session is delegated work when its own header says so — delegationDepth above zero, or origin: 'subagent' — or when a delegator that keeps its children looking like full sessions publishes it as one of the projections delegatedChildren names. The order is the session's own record, then that, then the deployment default: a person who switched this off for a session keeps it off whatever that session turns out to be.

A delegator that creates ordinary root sessions leaves no trace in a session header — that header is what makes the Client keep showing the children as full sessions, each with its own model picker, and it names no parent either. It publishes the relation as a session projection, and this plugin collects it from the registry's change feed, which means a view published before this row mounted is not in the feed: such a delegation reads as absent until its delegator publishes again.

A forked or delegated session takes its nearest ancestor's explicit setting once, as a snapshot: the value applies to that session from then on, and default clears it back to the deployment default rather than re-inheriting.

Turning the switch off while a session is waiting ends the entry immediately — the pending timer is aborted rather than left to wake an hour later — and reports left with reason disabled.

The per-session route pool

Which pool governs a session is a setting of its own, in the same durable record as the switch and with the same inheritance rules. It is written the same two ways:

• Composer chip — a second chip beside the switch. Its menu lists every configured pool by name, plus Default and Off.
• /quota-pool <pool|off|default|status> — the one write path that chip also uses. status reports the effective pool, where it came from, and the route the session is on.

Three states, deliberately distinct:

Choosing a pool chooses a route. The moment a session is put in a pool — by the command, by the chip, or by the delegation plugin that created it from a pooled template — the plugin draws a route from it (each usable route in proportion to what it has left, so the threshold applies) and the session's next request uses it, even when the model it is on belongs to no pool. That choice is recorded on the session itself, as the model/selection event the model picker writes, so the Client's picker shows the route actually in use. After that the session is only moved again when a provider reports its allowance spent, and a model the user picks afterwards is left alone.

A quota failover never changes the deployment default for new sessions, even though it moves a session the same way the picker would. The picker's own write path saves the choice as the default as well; this plugin deliberately does not use it.

Configuration

Unknown keys and invalid values throw at activation, so a misconfigured row fails loudly instead of silently retrying on a wrong schedule.

A row that turns the default off makes the allowance response opt-in for the whole deployment; the row below also shortens the interval.

- id: llm-quota-retry
  name: 'dsh-llm-quota-retry'
  config:
    longTermDelayMs: 1800000
    defaultEnabled: false

Route pools and balance sources

A route is one provider/model pair. A pool is a named, ordered list of routes that can stand in for each other. A balance source is a script that reports what one provider has left. The scripts this deployment runs live in dsh-balance — one file per provider, no dependencies, each printing one JSON reading on stdout.

- id: llm-quota-retry
  name: 'dsh-llm-quota-retry'
  config:
    pools:
      medium:
        routes:
          - { provider: gateway-a, model: flash-1, reasoningEffort: high }
          - { provider: gateway-b, model: flash-1, reasoningEffort: high }
      advice:
        midSessionSwitch: false   # interchangeable to start on, never swapped under a live session
        routes:
          - { provider: gateway-a, model: model-max, reasoningEffort: high }
          - { provider: gateway-b, model: model-max, reasoningEffort: high }
    balances:
      - provider: gateway-a
        script: ['node', 'C:/resource/dsh-balance/gateway-a.mjs']
        apiKeyEnv: GATEWAY_A_API_KEY      # resolved through the credential service, then forwarded
        unit: percent             # what `remaining` counts, for a person reading it
        minimumRemaining: 5       # below this a route is not chosen, and never one already running
        refreshMs: 60000          # how long a reading answers a route draw without a new run
        timeoutMs: 5000           # and how long that draw waits for this script at most
        coefficients:             # how the windows this script reports are weighed
          five-hour: 1            # a window the map does not name weighs 1
          monthly: 0.25           # a month of credits matters a quarter as much
        capacities:               # what a window counts as full, for a script that
          monthly: 40             # reports an amount without the plan's cap
      - provider: gateway-b
        script: ['node', 'C:/resource/dsh-balance/gateway-b.mjs']
        apiKeyEnv: GATEWAY_B_API_KEY
        unit: usd-left
        minimumRemaining: 1
        capacity: 20              # this script reports no window: 20 usd counts as full

A route's provider is the harness's provider id

provider is the route key the harness resolves a model request through — the id itself passes to ctx.llm, not a provider's own marketing name and not the id some SDK underneath happens to use internally. A provider plugin registers one such id, and the name of its package, its row, or the vendor is not automatically that id.

This is the one mistake in this file that cannot be caught from here, so the plugin checks it rather than trusting it: see the provider check.

The provider check

A route's provider is written verbatim into the model/selection event this plugin emits, so a name no adapter serves does not fail where it was written — it fails later, on somebody's model request, as the harness's NO_ADAPTER. The plugin therefore holds every provider name in pools and balances against the adapters the composition registered, and reports a mismatch once rather than on every announcement:

llm-quota-retry: no LLM adapter is registered for provider "opencode-go" (named by
pools.medium.routes[0], balances[0]); a model request on one of them fails with
NO_ADAPTER. Registered providers: command-code-goat, dsh-opencode-go, kimi-coding

When it reports. Composition order decides when each adapter registers, and a provider plugin composed after this row registers its adapter after apply() has returned — so a name missing at activation says nothing about whether it will ever arrive. Nothing in the harness announces that a composition has finished, so the check answers at the earliest point that can: it follows every llm/adapters-updated announcement but reports from one only when two consecutive announcements agree on the missing set — every announcement registers at least one adapter, so an unchanged set means no configured provider is still on its way. Failing that, the first request to route settles it, which is also the moment the defect matters.

What it does to routing. A pool whose routes name a provider no adapter serves is refused: pickRoute throws for it, naming the pool and the provider, and the session that asked is not put in the pool. A balance source naming one is not refused — a balance name is only a lookup key into this plugin's own readings, so a mismatch there costs a reading rather than a request and a deployment whose balance script is misnamed still delegates.

/quota-routes prints the same sentence above the pools, so the state a person reads names the same defect the log does.

The balance script contract

The script is argv, never a shell line: script: ['node', 'x.mjs', '--json'] runs exactly that executable with those arguments, through the harness's own subprocess service, with the working directory the host was started in. It should print one JSON object on stdout:

{
  "remaining": 2.94,
  "windows": [
    { "name": "five-hour", "remaining": 14, "capacity": 14 },
    { "name": "weekly", "remaining": 2.94, "capacity": 35 },
    { "name": "monthly", "remaining": 37.94 }
  ],
  "resetsAt": "2026-10-01T00:00:00Z"
}

remaining is what the provider has left in whatever unit the row's unit field names — a percent left, a currency amount, a credit count. A script for a provider that reports used percentage converts it (remaining = 100 - used) before printing. resetsAt is optional context for a person; a provider's own refusal can be reported with error alongside a reading.

windows is optional, and it is what ranks one route against another: a plan meters several windows (a five-hour cap, a week, a month), and the one that will refuse the next request is the one to go by. Each entry names its own window (name), reports what is left of it (remaining), and reports that window's full allowance (capacity) when the provider publishes one. A window's remaining is in that window's own unit; the plugin divides it by that same window's capacity, so a percentage plan and a wallet in another currency compare. A window without a capacity — a prepaid wallet, a monthly credit an endpoint reports only as an amount — is kept for the person reading /quota-routes and skipped by the ranking.

Everything else is a missing reading, not an error: a non-zero exit, a timeout (timeoutMs), output that is not that JSON object, or no subprocess service in the composition. A missing reading never stops a request — the route that asked for it falls back to the pool's declared order under failOpen, and the exhaustion mark is what keeps a provider the pool has already been told about out of the way. A malformed windows entry is dropped rather than failing the reading; the top level remaining is what the row's minimumRemaining still measures.

The script owns its requests, and this plugin