dsh-minimax-asr

dsh-minimax-asr

MiniMax speech-to-text (asr-1.0) and text-to-speech (speech-2.8-hd) as a global DeepSeek Harness plugin: model-facing transcribe_audio and announce_speech tools, a settings card, voice input in the composer, spoken announcements, and handsfree conversation.

English | 中文 | Changelog |

30-second demo

The task finishes → the speaker reads one distilled sentence → you say something → it is transcribed and sent for you → the reply is read out, and the microphone reopens by itself. No keyboard, no mouse. (The part people remember: what is spoken is a line written for the ear, not a reading of the reply.)

Want to record your own? The shot list is in docs/demo-script.md.

What it adds

Formats MiniMax accepts: wav, aiff, flac, m4a/alac, mp3, aac, opus, ogg. Not accepted: bare PCM, webm. A file may be at most 500 seconds and 50 MB; MiniMax bills by the returned duration.

response_format is json (text + duration), verbose_json (adds speaker-separated timestamped segments), srt, or vtt. timestamp_level is sentence (default) or word; it is ignored for json.

Install

From GitHub (recommended):

dsh plugin --profile web add github:moluyao/dsh-minimax-asr

That lands the package as a bundle layer in the profile, so it activates on the next dsh start.

From a local checkout (development; hot-loads):

dsh plugin --profile web add <path-to-this-repo>

then activate it in the profile's patch layer ($DSH_HOME/profiles/web/cordis.patch.yml):

- insert:
    - id: minimax-asr
      name: 'dsh-minimax-asr'
      config:
        apiKeyEnv: MINIMAX_API_KEY
        baseURL: https://api.minimaxi.com
        model: asr-1.0

One activation site only. dsh plugin add also appends the package to dsh.profile.bundles whenever it declares dsh.bundle, and a bundle layer re-inserts id: minimax-asr. With the row above present that mounts the plugin twice (two registrations of one tool name), so a patch-layer deployment keeps the package as a dependency only. If a later dsh plugin invocation re-adds the bundle row, delete the insert row from cordis.patch.yml.

The host half (tool + settings namespace) hot-loads with the patch layer, and the host's client-module registry re-scans on every plugin mount, so both halves come up live — reload the browser page to pick up the recomposed boot graph.

Credential

The key never enters configuration: it is resolved per request through the credentials seam (reference MINIMAX_API_KEY by default), so a value already stored in ~/.dsh/.credentials.yaml, or exported in the environment, is reused with no extra setup:

MINIMAX_API_KEY=sk-...

Voice input (browser)

Chrome records webm/opus, which the endpoint does not accept, so the browser half decodes the recording and rebuilds it as WAV before uploading. The upload is bounded by the same maxFileMB the tool uses. A microphone permission prompt is expected the first time per origin; a refusal surfaces in the control's tooltip, and the tracks are always released.

Recording length defaults to 300 s (5 minutes) and is editable under Settings → Plugins → MiniMax ASR → Max recording (seconds) — anything from 10 to 500, taking effect on the next recording (a change never truncates one already in progress). While recording the control shows elapsed / cap, e.g. 1:23 / 5:00, and stops itself at the cap.

MiniMax's own ceiling is 500 s per file (about 8m20s) and it rejects longer audio with a 400 rather than truncating, so a 10-minute recording cannot be one request — 500 s is the most this plugin can offer. Size is not the constraint: 16 kHz mono 16-bit WAV is about 32 KB/s, so 500 s is ≈16 MB of the 50 MB budget.

Announcements (host → browser)

You are rarely looking at the screen when a long turn lands, so every finished turn is read out through the speakers. The point is that the sentence spoken is written for the ear, in that turn — it is not a slice of the reply.

Fallback: a turn that names no line has only its opening sentence spoken (the conclusion usually lives there) — never the whole reply. Nothing ever says "there is more, I will skip it": an announcement long enough to need that apology is an announcement that was not distilled, and the fix is to rewrite it shorter, not to read half and apologise. Anything past speakMaxChars (default 80) is cut at a sentence boundary with no explanation added.

announce_speech also refuses an essay-length line (over 600 characters) with a reason — "this is a summary of a report, not an announcement" — which forces the distillation; that turn then falls back to its opening sentence.

The default voice is male-qn-jingying; the card now offers the account's 303 voices as a picker, where Test the voice auditions the current selection immediately. The host half also exposes:

GET /minimax-asr/diagnostics reports listeners, which is the direct answer to "how many browsers are currently listening for announcements?".

Autoplay: a browser only allows audio after the page has had one user interaction. You type in this page, so it normally just plays; if it is blocked, the card says so and a click anywhere on the page clears it.

Handsfree conversation

The rightmost control in the composer tool row is the conversation switch. One click starts an exchange that needs no keyboard and no mouse:

The gate is adaptive: it measures the room, takes the larger of "3x the noise" and an absolute floor of 0.0025, and caps itself at 0.03 — so a quiet built-in microphone array works and a loud room does not deafen the loop. The measured numbers (level, floor, gate, loudest) are reported once a second to GET /minimax-asr/diagnostics, so a mis-tuned gate is visible from outside the browser.

Voices

The card's voice field is a picker, not a text box: the host half fetches the account's catalogue from MiniMax (POST /v1/get_voice — 303 system voices in testing), groups it by language (Mandarin / Cantonese / English / Japanese / Korean / …), and caches it for 10 minutes. A failed fetch falls back to a built-in shortlist so the picker always works. A cloned voice, or any id that is not in the catalogue, goes through "Custom voice id…".

Test the voice auditions whatever the picker currently shows, saved or not, so you can try voices until one fits and only then save. Under the hood /minimax-asr/speak takes optional voice/speed overrides.

Local routes

All five routes live under one fenced prefix — loopback Host, no cross-site marker, matching Origin, the posture the shipped /api gateway and the installed dsh-better-sidebar use (a DNS-rebinding/cross-site defense, not authentication):

GET /minimax-asr/diagnostics is the fastest way to answer "is the browser half loaded, did its controls mount, and is the speaker wired?" on a live deployment.

Verifying without the harness

MINIMAX_API_KEY=... node tests/smoke.mjs <audio-file> [response_format] [timestamp_level]
MINIMAX_API_KEY=... node tests/route-smoke.mjs <audio-file>
node tests/client-smoke.mjs
node tests/installed-check.mjs <profile-dir>
node tests/no-bom.mjs

tests/smoke.mjs loads lib/index.js, runs apply against a stub context, and executes the registered tool against the live endpoint. tests/route-smoke.mjs drives the registered route with synthetic node requests: every fence and size refusal, the diagnostics channel, a completed turn becoming one announce frame (a cancelled turn staying silent, and a hang-up dropping the listener), plus one real transcription and one real synthesis. tests/client-smoke.mjs loads client/client.js the way the browser does (a classic script calling window.__ModuleLoader__.load) under a stub React/module table, asserts the card's render and the exact path ops it writes, and runs the whole microphone pipeline with only the codec and the network faked — asserting the uploaded WAV's header, rate, and sample count. It then drives the whole announcement chain: SSE frame → synthesis request → playback → queueing → mute → mid-sentence interrupt → recovery after a failed synthesis. tests/installed-check.mjs resolves the installed package from the profile and asserts everything the host's client-package scanner needs (dsh.client.platform, ./client bytes, the bundle id, every declared inject, the host half's export shape) plus the single-activation-site invariant.

Verified

• Real transcriptions through asr-1.0: json, verbose_json with word-level segments and speakers, and srt — through the tool and through the local route.
• A fresh child agent inside the running Harness saw transcribe_audio in its registry and transcribed a file with it.
• The settings card rendered in a real browser on a throwaway profile composed from the same bundles: 8 fields resolved from the namespace, the credential badge read 已配置, an edit raised "未保存", and Save wrote minimax-asr: / language: zh into the profile's settings.yaml. That run is also what caught a real defect — the card read ctx.remote while declaring only remote.credentials, which Cordis rejects ("cannot get property "remote" without inject").
• The voice control rendered in a real session's composer tool row (aria-label="用 MiniMax 语音识别听写", supported: true), confirmed both in the DOM and by the deployment's own GET /minimax-asr/diagnostics report. It appears once a session is bound to a workspace; a workspace-less hero composer does not mount that zone.
• The microphone pipeline (record → decode → 16 kHz mono WAV → route → draft) runs green in tests/client-smoke.mjs with only the codec and the network faked.
• The announcement chain is verified on both sides: the host turns one completed turn into a real SSE frame (route-smoke), and real MiniMax TTS comes back through /minimax-asr/speak as 33 KB of audio/mpeg; the browser side (frame → synthesis → playback → queue → mute → interrupt → recovery) runs green in client-smoke. That run also caught two real defects: a failed synthesis was immediately masked by the "queue drained" idle write, and an interrupted playback never settled, which wedged the controller so no later announcement was ever spoken.
• The handsfree loop runs green in client-smoke: 30 s of silence releases the microphone without uploading, speech followed by silence ends the turn and submits by itself, a finished reply reopens the microphone, a cancelled turn still re-arms it, and switching off releases the microphone immediately. That run caught three more real defects: a loud first frame could raise the gate above the speaker's own voice (the loop went deaf), switching off while the microphone was still opening left it recording, and the state machine had no re-entrancy guard (unbounded recursion).
• The catalogue and the audition are verified against the live API: POST /v1/get_voice returns 303 system voices, /minimax-asr/voices serves them grouped, and the voice/speed override on /minimax-asr/speak synthesises real audio (female-tianmei → 21 KB of audio/mpeg).
• A real person speaking, all the way round — reported by the deployment's own diagnostics rather than by a test script: a genuine utterance (loudest 0.128, speechFrames 14) was transcribed and sent for the user (voice-submitted), the reply was read out, and the microphone reopened by itself — with one window rollover (handsfree-roll) and zero mic-idle-timeout events, which is the pause-then-keep-talking case working.

Notes

• The plugin runs in the Harness host process and reads the audio file itself, so the agent's file sandbox does not gate that read. Point it at a file the host can already read.
• A plugin cannot appear in the model/provider picker: MiniMax ASR has no chat-completions interface, so it is a tool, not a conversation model.
• A long-running instance needs one restart after a browser-half edit. The client bundle's bytes are read and revisioned when the module registry first activates the package, and a page reload alone keeps serving the old bytes. The host half has no such constraint: a settings or tool change reaches the next call without any reload.

License

MIT

• tests/no-bom.mjs scans the whole tree for a UTF-8 BOM. JSON.parse does not strip U+FEFF, so a BOM in package.json or the bundle patch aborts profile composition and the port never binds — and this plugin is installed into the profile as a link:, so this working tree is what the host reads.