dsh-llm-service-lite

@snnh/dsh-llm-service-lite

English | 中文

Community plugin. Maintained by snnh. It is not an official DeepSeek package: DeepSeek neither publishes nor supports it, and issues belong in this repository.

Install

A harness loads plugins from a profile's own installation, so this package is installed into a profile rather than into the harness itself:

# From the published package:
dsh plugin --profile web add @snnh/dsh-llm-service-lite

# Or straight from this repository. pnpm builds the checkout while installing,
# which the profile must allow first: run the command, add the exact key pnpm
# printed under `allowBuilds` in <profile>/pnpm-workspace.yaml, and re-run it.
dsh plugin --profile web add github:snnh/dsh-llm-service-lite

Mount

The package declares itself a bundle (dsh.bundle.patch), so the install above also mounts its llm-service-lite row: one command installs and mounts, and no hand-written entry is needed. The mount is dormant — no route exists until a profile declares one.

Two surfaces edit the same document. In the web UI, open Plugins, select this bundle, open its llm-service-lite row, and choose Configure: the page lists every provider, adds and removes profiles, edits the endpoint, protocol, capabilities and model catalog, stores an API key through the harness credential service, and asks an endpoint which models it serves. The same document can be edited by hand in the profile's own cordis.patch.yml ($DSH_HOME/profiles/<profile>/cordis.patch.yml; Settings → Open configuration file in the web UI opens it). Add the providers dictionary described under Configure provider profiles to the llm-service-lite row's config. The profile patch is applied after every bundle layer, so that entry's config replaces the row's empty default; the same entry also mounts the plugin in a composition that declares no bundle layer, because it names the module to load:

- id: llm-service-lite
  name: '@snnh/dsh-llm-service-lite'
  config:
    providers:
      # ... provider profiles, one per endpoint

After the install, refresh the Web page once: the browser half is served by the client module system, so the new page appears on the Plugins screen the next time the shell boots. It lives on the bundle's page and on its llm-service-lite row, where that row's Configure control opens it.

The plugin fills the Models page's adapter-family extension seats while the Host serves its settings namespace: every llm-service-lite provider card gets the plugin's own editor, and the section footer gets the same add-provider builder. The builder asks for a preset, route, protocol, endpoint, optional key, and runs a connection test before writing; the editor covers protocol, endpoint, credential, request behavior, a connection test, and the model catalog. The catalog is a filterable list with one expanded model editor; effort levels and thinking modes are localized toggles, and Fill capabilities applies the documented family defaults to fields a model left open. extraBody sits behind an advanced disclosure and blocks a save when its JSON is malformed. The plugin's page on the Plugins screen edits the same row config. Hand-written fields it does not render (headers, imageTokens, imagePixelBudget, imageMaxBytes, catalog, retryPolicy, and the per-model defaults) are preserved by every save and remain editable in the patch.

Credentials are named, not inlined, in a managed profile: set apiKeyEnv to the reference (DEEPSEEK_API_KEY, say) and store the secret through the harness credential service, export it in the launching environment, or keep it in the harness home's .env. The reference is resolved on every request, and a named reference that resolves to nothing fails the request with MISSING_CREDENTIAL instead of sending it unauthenticated. A profile imported verbatim from OWCode may keep its inline apiKey.

Mount it exactly once. When a composition already carries an llm-service-lite row (the archived dsh-lite fork's dsh-base bundle shipped one), that row already reaches this package: install without activating it as a bundle — dsh plugin --profile web add -D @snnh/dsh-llm-service-lite — and configure the existing row. Keep exactly one layer for the row: a later layer's same-id insert replaces the earlier row and its whole config — patch layers never deep-merge — so a second row silently discards what the first configured; it does not mount a second instance.

Route names

Route names must be unique across the adapters mounted in one composition. On stock upstream profiles the shipped llm-pi-ai catalog already declares common provider names (deepseek, openai, anthropic, and similar), so prefer a distinct route id that names the endpoint itself — deepseek-direct, company-gateway — rather than a name an installed catalog may own. A profile whose route name another adapter already owns is skipped with one warning naming the owner instead of failing the mount, the rest of the profiles register, and a newly written profile that claims such a name is refused where it is written; renaming its route is the repair. The plugin also refuses, by name, the routes the first-party DeepSeek channel owns; see Use this package.

Compatibility

This package drives the public seam of @deepseek-ai/dsh-llm and the @deepseek-ai/cordis service layer, so it runs against a published harness rather than a checkout: its peers are the harness packages, and the profile that loads it already owns the copies the runtime uses. The development baseline is harness 0.2.1-rc.2 — devDependencies pin those packages and the test suite runs against them — while the ^0.2.1-alpha.2 peer ranges also admit the 0.2.1 alpha line, so an older 0.2.1 harness loads the same package.

Two declarations of the archived dsh-lite fork are used when the running harness carries them and skipped when it does not: the model-defaults registration seam a configuration surface asks for declared model ids, and the outputModalities field on a model's info. Upstream 0.2.1-rc.2 carries neither, so both are inert there: the adapter keeps its family defaults on the request path, where they still fill the fields a profile left open, and the extra field is simply ignored by the runtime.

Summary

@snnh/dsh-llm-service-lite adapts third-party model endpoints: one route per configured provider profile, all answered over its transports — the OpenAI-compatible chat-completions wire, the Anthropic Messages protocol, and the OpenAI Responses API — so no provider SDK and no installed catalog sits in the request path. A profile is self-contained — wire protocol, endpoint, credential reference, request-body extras, concurrency, image input, and the models served — so a gateway, a national API, and a corporate proxy reach one code path. The shape ports OpenWebCode's model-provider layer onto the harness seam: one document describes every provider, and the route set follows it live.

Table of Contents

• Install
• Use this package
• Understand the implementation
• Further Exploration
• Model Experience
• Known Limitations and Deferred Work
• Dev Note

---

Use this package

Mount this plugin when a deployment wants to describe its model providers itself instead of naming providers an installed catalog already knows. The providers dictionary is the whole configuration surface: each key is the provider route a request selects with GenerateOptions.provider, and the value is everything that endpoint needs.

When to choose it

This plugin adapts third-party endpoints and nothing else: every route it registers comes from a profile a deployment wrote, and the first-party DeepSeek channel keeps its own adapter. DeepSeek is that channel by default, so the route names it owns — deepseek-official and deepseek-account, registered by @deepseek-ai/dsh-llm-deepseek-api-key and @deepseek-ai/dsh-llm-deepseek-account — are refused here by name, with the owner in the diagnostic, rather than claimed and left to fail plugin loading. DeepSeek itself keeps those modules — this adapter is not a substitute for that channel. Any other route name is free: a deployment that reaches a DeepSeek-compatible endpoint through this adapter names its own route and declares that endpoint's own capabilities. Choose this adapter when the same composition serves several endpoints — a national model API, a self-hosted vLLM or SGLang gateway, and a proxy on the corporate network — and each must be described completely rather than derived from a catalog. Choose dsh-llm-pi-ai when a pi-ai catalog entry already supplies the protocols and model capabilities. Both adapters can be mounted together: a profile whose route name another adapter already owns is reported and skipped at mount, and a write that claims such a name is refused where it is written (see Route names).

Configure provider profiles

- id: llm-service-lite
  name: '@snnh/dsh-llm-service-lite'
  config:
    providers:
      # A provider reached at its published API, with the key from the seam.
      # (`deepseek-direct`, not `deepseek`: the stock llm-pi-ai catalog already
      # declares the plain vendor names — see Route names above.)
      deepseek-direct:
        displayName: DeepSeek
        interfaceType: openai-chat-completions
        baseURL: https://api.deepseek.com/v1
        apiKeyEnv: DEEPSEEK_API_KEY
        includeUsage: true
        models:
          - id: deepseek-chat
            contextWindow: 131072
            maxTokens: 8192
          - id: deepseek-reasoner
            contextWindow: 131072
            maxTokens: 65536
            capabilities:
              effort: [low, medium, high]
              thinking: [enabled, disabled]
              reasoningContent: true
      # A self-hosted gateway: body fields only this server understands.
      vllm:
        interfaceType: openai-chat-completions
        baseURL: http://127.0.0.1:8000/v1
        extraBody:
          top_k: 40
        maxConcurrent: 1
        models:
          - id: Qwen3-32B
            contextWindow: 32768
            maxTokens: 8192
            capabilities:
              effort: [low, high]
              thinkingStyle: enable_thinking
      # An endpoint reached over the Responses API, with encrypted reasoning
      # replay declared.
      openai-direct:
        interfaceType: openai-responses
        baseURL: https://api.openai.com/v1
        apiKeyEnv: OPENAI_API_KEY
        models:
          - id: gpt-5
            contextWindow: 400000
            maxTokens: 128000
            capabilities:
              effort: [minimal, low, medium, high]
              thinking: [disabled]
              responsesEncryptedReplay: true
      # An endpoint reached over the Anthropic Messages protocol, with prompt
      # caching and its thinking modes declared.
      claude:
        interfaceType: anthropic-messages
        baseURL: https://api.anthropic.com/v1
        apiKeyEnv: ANTHROPIC_API_KEY
        promptCaching: true
        models:
          - id: claude-sonnet-4-5-20250929
            contextWindow: 200000
            maxTokens: 64000
            capabilities:
              thinking: [enabled, disabled]
              # This endpoint reads a later system message as the effective
              # prompt and activates tools inside the conversation.
              systemPromptUpdate: in-history
              toolUpdate: in-history
      # A gateway that routes by header and needs no credential of its own.
      proxy:
        interfaceType: openai-chat-completions
        baseURL: https://proxy.example.com/v1
        headers:
          x-tenant: acme
        streamIdleTimeoutMs: 60000

Each entry of models describes one model the route serves:

Each entry of models describes one model the route serves:

Start from an official configuration file

An official deployment publishes one provider file per provider — an API-keyed map of chat: entries. catalog converts such a file into this adapter's own vocabulary: the API becomes the route's interfaceType (openai-completions to openai-chat-completions, openai-responses, anthropic-messages), baseUrl becomes baseURL, a header set every model shares becomes the route's headers, and each chat entry becomes a model with its id, name, contextWindow, maxTokens, input modalities, and thinking levels. A model's samplingParams become its defaults, so an endpoint's own sampling settings arrive as configuration rather than as a caller concern. Fields this adapter has no counterpart for — cost, compat flags it does not model, inputLimits, and model types it does not serve — are reported one line each instead of guessed, and the route still serves. A header set every entry shares becomes the route's headers; only header sets that disagree, which one route cannot carry, are reported. The profile's own fields win over the conversion; an absent or empty models list keeps the catalog's models; and a file that mixes protocols or endpoints, names no chat model, or cannot be read leaves the route with a diagnostic rather than a half-converted one.

providers:
  kimi-coding:
    catalog: /etc/dsh/official/kimi-coding.json
    apiKeyEnv: KIMI_CODING_API_KEY

Declare what a model can do

A model entry's capabilities are declarations about the endpoint, never guesses: a capability left out is one this adapter does not claim, and one it declares is honoured or refused by name — never accepted with nothing behind it. effort lists the selectable reasoning levels and its values are the wire spellings, so a gateway with its own vocabulary declares that vocabulary. thinking lists the accepted thinking modes (enabled, disabled, adaptive). thinkingStyle says how the endpoint spells the switch: thinking: { type: … }, a top-level enable_thinking boolean, fixed for a model that always thinks, or effort_only — the explicit spelling of an endpoint that takes an effort level and has no switch, which is the same request an omitted style sends. reasoningContent declares that the endpoint returns reasoning in reasoning_content, which is what lets a later request replay prior thinking. replayReasoning decides whether that prior thinking actually travels back: absent defers to the model's family (see below), true sends it, and a family whose follow-up turns are rejected without it refuses to be turned off. tools declares whether tool declarations may be sent at all: a model that turns them off receives none rather than a list its endpoint would reject. modalities names what the endpoint accepts — text, image, video. image is carried: a retained occurrence is read from the attachment provider at the model's pixel budget (or its source dimensions), re-encoded to the model's byte target, and sent as an inline base64 image_url part, while an occurrence the session already offloaded contributes its placeholder text instead. A route's accumulated request is bounded by imageRequestMaxBytes, imageRequestMaxImages, imageOffloadByteQuantum, and imageOffloadCountQuantum; a request over budget fails with IMAGE_OFFLOAD_REQUIRED naming how many oldest occurrences to offload rather than dropping an image silently. imageOutput declares that the model answers with images of its own: the inline picture the endpoint returns is stored through the attachment provider and republished as a durable image block, an image from a model that never declared it fails by name, and the declaration is refused on anthropic-messages, whose assistant turn has no image part. video is accepted and carried as a file: a video occurrence reaches the model as the same handle text every file contributes, and a native part waits for the harness's own video content block. responsesEncryptedReplay is honoured by the Responses transport: the request asks for reasoning.encrypted_content and hands each reasoning item back verbatim on the next turn. systemPromptUpdate and toolUpdate declare the two mid-conversation reads a Messages endpoint can be asked for: the latest system message as the complete effective prompt, and a tool change as an activation or deactivation of a tool the declaration list already names. Declaring either is what makes the harness hand this route tool-addition and tool-removal blocks instead of restating the prompt and the tool list on every turn, and the declarations of the tools it defers then carry defer_loading. Both are refused on the two OpenAI wires, which have no mid-history part to read. imageTokens declares the endpoint's own visual-token accounting — { kind: area, per: 750 } for a flat pixel grid, or { kind: tiles, tile: 512, base: 85, perTile: 170 } for a base price plus a price per square tile — which is what the token meter reads to price an image-bearing context; a route that declares none keeps the meter's own structural heuristic rather than a number this adapter guessed.

Family defaults. A hand-written profile states what its author knows and leaves the rest to the model's family: src/model-defaults.ts is a reviewed table of what the mainstream families document — the image input a vision model takes, the effort ladder and the level its own platform applies by default, the switch spelling, and whether prior reasoning must travel back. The table only fills what the profile left open, so a profile field always wins; a family that shows no entry for a model leaves the adapter's own conservative answer in place; and nothing in it defaults a sampling parameter, because vendors disagree about the reachable ranges and the endpoint's own default beats an invented one. Each entry records the vendor documentation it came from, and only the model owner's own platform is ever a source: what a reseller or gateway requires of the same model id is not.

A model entry's defaults state the request parameters its endpoint wants when the caller states none: temperature, topP, topK, and maxTokens. Each is written only where the caller's own request left it unstated, so a default can never override an explicit choice; topK is refused on openai-responses, which has no top-k field; and defaults.maxTokens is what a caller that stated no cap receives. A model's own maxTokens is a capacity rather than a request default, and the three transports treat the output cap differently: chat-completions sends max_tokens only when the caller, or a declared defaults.maxTokens, supplied one, so an endpoint that defaults its own ceiling keeps it; anthropic-messages requires the field on every request and takes the caller's value, then the profile's extraBody.max_tokens, then defaults.maxTokens, then the model's family cap, then 64K for a model no family speaks for; openai-responses also always sends max_output_tokens, from the caller's value, then defaults.maxTokens, then the model's maxTokens (the route's defaultMaxTokens when the model declares none), raised to the API's 16-token floor.

An image request needs the attachment provider mounted (attachments); a deployment serving text-only routes never mounts one, and a route that declares image without it fai