dsh-live-voice

DSH Live Voice - You and Your Team / 实时语音 - 您和您的团队

English | Português (Brasil) | 简体中文

Talk to your AI assistant, listen to its answers, and bring meeting context into the conversation—without constantly switching back to the keyboard.

DSH Live Voice is a local-first voice plugin for DeepSeek Harness (DSH), with speech-to-text (STT), text-to-speech (TTS), and real-time meeting transcription.

_Speak to your assistant, listen to its answers, and control the conversation without returning to the keyboard._

_Built in Brazil 🇧🇷 and tested daily with Brazilian Portuguese on macOS._

⚡ Quick Start

Tested with DeepSeek Harness v0.2.1-rc.2.

Install the prebuilt package from npm with npx dsh into your DSH web profile (no source build required):

npx dsh plugin add --profile web dsh-live-voice

Open DSH Settings → Live Voice after your next DSH startup.

Interface translated into: 🇺🇸 English · 🇧🇷 Portuguese (Brazil) · 🇪🇸 Spanish · 🇫🇷 French · 🇮🇳 Hindi · 🇨🇳 Chinese. This refers to the plugin UI, not speech-recognition or text-to-speech language support.

What can you do with it?

• Work by voice: dictate a request, talk through code or documents, and hear the assistant’s response while staying focused on your work.
• Bring your AI assistant into the meeting: capture the discussion, review the transcript, and ask for help comparing proposals or summarizing decisions. Meeting audio capture and sharing the assistant’s voice require separate setup; this is not a native Meet or Teams integration.
• Stay in control: review before sending, choose manual or automatic delivery, and pause, resume, or stop spoken answers. See the conversation flow guide for listening and interruption controls.
• Choose your speech setup: use local recognition and speech engines where supported, or configure an HTTP endpoint. Local-first voice does not mean DSH or its AI provider is fully offline.

✨ Features at a Glance

• 🎙️ Voice Typing: Append final recognized speech to the end of the DSH composer without replacing manual edits.
• 👐 Hands-Free Conversation: Continuous dialogue that stays active across chat sessions.
• ❓ Spoken Structured Questions: Narrates DSH prompt questions and submits your spoken answer.
• ⌨️ Optional Hold-to-Talk (Push-to-Talk): Enable it in Settings, then hold Control anywhere on the page to speak; release to queue the message after the configured send delay. Disabled by default.
• 🎧 Acoustic Mode Isolation: Gated listening for speakers (no echo) and open-mic interruption for headphones.
• 🗣️ Spoken Commands: Control the chat using phrases like _"send"_, _"mute"_, _"clear"_, and _"stop speaking"_.
• ⭐ Dedicated Speech Bar: Smoothly scrolling approximate captions, a moving highlight, previous/next navigation, pause/resume, stop, and a current/total segment counter. Captions show the text sent to the speech engine, without claiming word-level alignment.
• 🔊 Speech Volume: Save a playback volume from silent to full in Speech settings for Browser, Say, and Qwen, without changing microphone input.
• 🧹 Markdown-Aware Speech: Remove formatting, announce links without reading full URLs, shorten file paths and line references, read checkbox states and table rows, and replace long code blocks with a localized notice. Preserve custom notices.
• ⚡ Responsive Conversation Settings: Automatic-send delays of 600 ms, 800 ms, or 1–6 seconds (4 seconds by default), plus an assistant response delay of 600 ms, 1.5, 2, 3, 4 or 5 seconds (1.5 seconds by default). Recognition and synthesis still contribute to overall latency.
• 🛠️ Optional Live Voice Debugger: Install dsh-live-voice-debugger separately to inspect runtime state and queues from the Developer tab in Live Voice Settings. The main plugin works without it.
• 💻 Recognition on this browser: download multilingual Whisper Tiny, Base, Small q4 (302 MB), Medium (684 MB), or Large v3 Turbo (762 MB), with non-f16 options in Speech recognition, or choose Qwen3-ASR with WebGPU. Models have explicit download, cancellation, progress, and removal controls. Whisper also offers an explicit WebAssembly CPU option. Audio stays in the browser for these engines; initial model downloads require network access. See browser model setup.
• 📝 Transcription context: optionally describe your vocabulary or speaking background in Speech recognition → Connection settings for Whisper HTTP and Qwen HTTP. The field starts empty; its English example is only a placeholder. Server support varies, and context does not restrict languages.
• 🗣️ Chatterbox reference voices: choose an authorized recording from a separately installed local MLX service in Speech settings. Existing playback, pause/resume and output controls are reused; this is complete-file synthesis, not streaming. See Chatterbox setup.
• 🏠 Local-First Speech Options: Use local engines such as Qwen or Whisper where supported. Browser recognition and configured HTTP services have their own processing requirements; do not assume every setup keeps audio on your machine.
• 🌐 Your DSH voice assistant, even on your phone: access your DSH over your home network or while away from home, and keep speaking to it (STT) and listening to its answers (TTS) from your phone’s browser. With host-backed engines, speech processing stays on the DSH computer while your phone captures your voice and plays the answers—no local model installation on the phone is needed. _I use Cloudflare Tunnel with authentication to access my DSH away from home._ Use HTTPS and authenticated access; microphone capture requires browser support and permission, including on your home network.

🧑‍💻 Meeting Mode — Real-Time Meeting Transcription and Live AI Q&A

Keep an editable transcript of a meeting as it happens, then ask the DSH assistant questions about the discussion without having to retell it afterward. Use it for code reviews, project discussions, or working through a spreadsheet together.

From discussion to an answer

1. Select Browser recognition when your browser supports shared audio tracks, or use Qwen HTTP / Whisper HTTP. Browser shared audio is checked at start and never silently falls back to your microphone; local-only processing and language-pack requirements still apply.
2. Click the Meeting mode button beside the composer microphone, then choose Share audio from a tab and enable audio in the browser sharing dialog. Right-click opens the same actions; opening the menu starts nothing. Capture your microphone too when you want to include your own contributions.
3. Review the transcript in the composer, add a question, and send the relevant context to DSH. For example: _“What decisions have we made so far?”_, _“Help us answer this point”_, or _“Compare this proposal with the project code.”_ Answers depend on the transcript you send and the code, spreadsheet, or documents you make available to the agent.
4. Read or listen to the answer while the meeting continues. For other participants to hear it, your meeting app must separately be configured to share the assistant’s playback audio.

You control what is captured and sent

• Shared audio and your microphone have separate controls and can be stopped independently. Normal voice commands and assistant speech remain available.
• With both sources capturing, source changes introduce Me: or Them:; consecutive chunks continue on the same line; configured longer pauses start new paragraphs. Customize the microphone and shared-audio names in Live Voice Settings → Conversation → Meeting mode (defaults: Me / Them). Names are saved when you leave the field; blank restores the default, with a maximum of 80 single-line characters. Changes apply to new source blocks without rewriting existing text. These labels distinguish audio sources, not individual meeting participants.
• An optional Timestamp clock toggle adds local timestamps such as [YYYY/MM/DD HH:MM:SS] to new source blocks, using chunk onset rather than recognition completion. It is off by default and never rewrites earlier text.
• Microphone and shared-audio bars mirror each other: the leading icon toggles Listening / Ignoring, with independent exact-command, timestamp, and Manual / Queue / Steer controls. Both use the same delivery countdown and cancel action. Ignoring does not end capture; stop sharing through Meeting mode → Stop audio sharing in the composer. The existing shared-audio shortcut still starts or stops capture directly. Enabled commands and automatic delivery follow each source’s controls; review commands before enabling them for participant audio.

Audio availability depends on the browser, operating system, and sharing surface. This is not a built-in Google Meet or Teams integration. Capturing meeting audio and sharing the assistant’s voice back into a call require separate setup. Use headphones to reduce feedback, and obtain appropriate participant consent before capturing a meeting. Transcription and AI responses have processing latency; “real-time” does not mean instantaneous or guaranteed accurate.

Experimental local Attendee input

Click or right-click Meeting mode beside the composer, then choose Join Google Meet · experimental. The menu contains only the Meet URL and Connect (or Enter), preparing reception and requesting entry with the saved Settings name/key. Configure bot name, API origin and key only in Settings, then choose Save API key once; saving never starts reception or creates a bot. The key it is saved on the host for that exact API origin and reused after reopening or restarting, without returning the secret to the browser. The password field stays empty when configured; typing a replacement and choosing Replace API key updates it, while blank input never deletes it. Use explicit Remove to erase the saved key without ending an existing bot lease. Storage is a plaintext host file with mode 0600, not OS-encrypted storage, shared by authenticated browsers using that host. Only the explicit URL/Connect action starts reception and creates a bot. Live Voice Settings → Conversation → Meeting mode contains the connection configuration and Default bot name, including while disconnected. The composer has no name override, credential field or configuration dialog. Participant names are visible to the left of their audio waves, with full names available when truncated. View participants and edit connection-local aliases in the composer panel; participant-labelled PCM streams use separate bars and independent VAD/STT pipelines. Aliases affect future transcript blocks and new capture metadata snapshots, not historical recordings. One Disconnect meeting action stops local reception/control immediately and returns to URL + Connect without a confirmation. It attempts remote departure with a short bound; if unconfirmed, a nonblocking warning remains and you can reconnect. Legacy released/expired state resets through the same button. The packaged Compose audio bridge manages local WSS and certificate trust without a separate host proxy; direct WSS remains an advanced option. Requires local Attendee API access and supported HTTP or browser-local STT. It does not guarantee isolated overlapping Meet voices; sparse streams are saved as capture bursts, not continuous meeting recordings. Start with the Google Meet quickstart; see setup, security and validation limits for technical details.

Recoverable audio sessions

With Qwen or Whisper, recording is enabled by default to retain compressed audio locally, including silence, independently of transcription. You can disable it in Speech recognition → Recording. Choose a retention count (default: one), quality, and transcription margins. For Attendee, the count means the N meeting groups with the newest capture activity, retaining all participant captures and bursts while their group is kept. Active writers, pending writes and stars protect the whole group without consuming an ordinary slot. A quiet gap does not mark a call ended or reserve a permanent place: a newer meeting can expire an older quiet group, and resuming cannot recover expired history. Legacy microphone/shared-audio recordings still keep N captures per source type. Browser quotas or storage eviction can limit availability even for retained or starred content. The optional debugger lists microphone/shared sources separately and lets you inspect pins, play speech or silence, review transcription results/errors, name starred sessions, inspect assistant playback and capture events with their actual timestamps, zoom the timeline, search transcribed segments, check total/per-session audio size in MB, download a single speech/silence segment as WAV, retranscribe finished speech with the currently selected recognition engine while preserving earlier attempts (without editing the composer), export a complete session, or delete a finished session. Clicking a timeline band loads that exact speech/silence range in the player; zoom remains independent. New HTTP attempts include measured WAV request bytes, samples, duration and response status without another audio copy; these describe the body prepared for DSH, not proof of complete model processing.

This requires browser WebCodecs Opus support; persistent recording also requires IndexedDB and Web Locks. Saving is incremental, not a guarantee against power loss, storage clearing, or loss of the newest unsaved samples. Recordings belong to this browser profile and exact site origin, including its port. Opus is lossy; obtain consent before recording other people. You can opt out of recording in Settings; disabling it does not delete saved recordings. Native Browser SpeechRecognition live capture still does not record original speech. Archive retries can process existing saved speech segments using Qwen HTTP, Whisper HTTP, Whisper · Browser or Qwen · Browser. Download the selected browser-local model in Settings first; retries use cached models only and never download models or fall back to HTTP. Native Browser can retry existing saved segments only when safe external-audio-track support is verified, with the selected local/remote-processing policy and language-pack requirements still applying. Unsupported browsers fail explicitly; retries never fall back to the microphone.

Compatible HTTP servers

See HTTP engine contracts for the routes and request/response payloads expected by the TTS and STT adapters.

Documentation

Detailed guides for deep-diving into engines and configurations:

• Configuration & Conversation Flow Guide — Settings overview, speaker vs. headphone modes, sequence diagrams, silence delays, and external engine setup.
• Choosing a Speech Recognition Engine — Comparison between Browser STT, Qwen3 ASR, and Whisper, with RAM footprints and OS compatibility.
• The Story Behind the Project — Why this project was built and the human story behind coordinating voice.
• Release notes — What changed in each version (canonical history in English).

Workspace layout (contributors)

This pnpm monorepo contains two independently installable plugins:

• packages/live-voice: the main voice plugin, with its own source, tests and manifest.
• packages/live-voice-debugger: the optional debugger, with its own src, tests and manifest.

Run pnpm install --frozen-lockfile, then pnpm build or pnpm test from the root. Build individual plugins with pnpm --filter dsh-live-voice build or pnpm --filter dsh-live-voice-debugger build. Root documentation covers both packages. The root English CHANGELOG.md is the canonical release history. Package READMEs link to that history. Generated packages/*/lib/ directories are ignored and never committed, including for releases. Users install prebuilt packages from npm, not directly from Git. Contributors must build before linking a local package directory to DSH; see development guidance.

🤝 Acknowledgments & Community

Listening and speaking should work together. A heartfelt thank you to GooDAnDReaDY for dsh-voice and Alan2Z for dsh-speak, which inspired this unified coordinator. Read the full story.

I use DSH Live Voice for at least 8 hours every day. Feedback, ideas, and contributions are welcome:

• Open an Issue
• Send a Pull Request

real-time meeting transcription, live meeting transcription, AI questions during meetings, meeting assistant, meeting Q&A, shared audio capture, speech-to-text, text-to-speech, local-first voice assistant, DeepSeek Harness plugin. transcrição de reuniões em tempo real, transcrição ao vivo, perguntas à IA durante reuniões, assistente de reuniões, perguntas e respostas sobre reuniões, captura de áudio compartilhado, reconhecimento de fala, síntese de voz, assistente de voz com prioridade local, plugin para DeepSeek Harness. 实时会议转写、实时语音转文字、会议中向 AI 提问、AI 会议助手、会议问答、共享音频采集、语音识别、语音合成、本地优先语音助手、DeepSeek Harness 插件。

License Apache-2.0