deepseek-harness-intelligent-ui

English · 中文

Demo · Quick start · How it decides · Components · Widgets · Extending · Security · Configuration · Quality · Develop

Generative, interactive UI for DeepSeek Harness (DSH). Ask in plain language and the agent decides on its own how to answer:

• plain text when that reads best;
• native components: dashboards, data grids, charts, scatter plots and heatmaps, timelines, live function plots, KaTeX math, Mermaid diagrams, code diffs, file trees, JSON trees, forms and choice cards;
• a fully custom sandboxed live widget: Three.js scenes, canvas simulations, mini-apps.

The UI renders progressively while the answer streams. Sorting, filtering, sliders, rotation, zoom and tabs respond instantly in the browser. Only explicit actions (a button, a form, a choice, Bridge.sendPrompt) go back to the agent, either in the current session or in a new one.

It ports the ideas of CopilotKit's OpenIntelligentUI (Open Generative UI + A2UI-style declarative components) onto DSH's own extension points, with no CopilotKit, LangGraph or external router.

✨ Highlights

🧠 Decides by itself

No command needed. Each turn the agent picks text, native components or a live widget, guided by a system-prompt section.

⚡ Streams progressively

Components appear while the JSON streams. Widgets morph into one iframe and finalize in place: no rebuild, no flash.

🖱️ Local first

Sort, search, filter, page, export, hover, sliders, rotation and tabs run in the browser and never call the model.

🧩 26 components

From cards and tabs to data grids, Scatter, Heatmap, Plot, Math, Mermaid, CodeDiff and Choice. Every UI is validated before it renders.

🔒 Sandboxed by default

Opaque-origin iframe, strict CSP, token-checked messages and a rate-limited bridge, all verified in a real browser.

💾 State that survives

Tabs, sort, filters, form values, sliders and UI.state survive turn folding, in-place edits, page reloads and stopped generations.

🎬 Demo

The real DeepSeek model, end to end

https://github.com/user-attachments/assets/e4eb0adc-659e-4c80-b473-449aa42ca00f

All 11 browser cases of the quality baseline, generated live by `deepseek-flash` in DSH Web 0.2.0-rc.1 (2026-10-16): text, chart, table, form round trip, Plot, Three.js with "Ask to fix", stop → regenerate, two in-place edits, Fast 3G, CDN offline and phone width. Failures are left in and marked ✗. The DeepSeek whale girl (鲸鱼娘) narrates (credits). Full video: docs/assets/real-model/demo.mp4 (2 min 27 s).

🖼️ Results at a glance (screenshots per case)

Multi-series chart: ✗ (call cut off mid-stream) · Two in-place edits with ui_patch: ✓ · *The chart is right, but the card says "generation interrupted"* · *"Make it a line chart", then "add 小程序 30/45/60/80": one card, version 3* Form submit → the agent uses the values: ✓ · Damped oscillation `Plot` with live sliders: ✓ · Three.js: ✗, then repaired with "Ask to fix" · Stopped mid-stream → regenerated: ✓ · Fast 3G, cache off: ✓ · CDN unreachable: ✓ (degrades as designed) · Phone width (390 px): ✓ · Sortable table: ✗ (agent researched instead) ·

GIF version for viewers that cannot play video (e.g. npm): demo.gif.

Feature tour

Recorded live in the DeepSeek Harness desktop app with real DeepSeek models through the magpie gateway; nothing is scripted. The camera follows the cursor and zooms into each interaction; waits for the model are fast-forwarded (the badge shows the speed).

`Choice`: pick one option, continue here or start it in a new session · DeepSeek-V4.1-Flash

https://github.com/user-attachments/assets/1317993c-17b1-4fca-9356-c365afcc6101

`Table` as a data grid: typed sorting, search, per-column filters, paging, CSV export · DeepSeek-V4.1-Flash

https://github.com/user-attachments/assets/be9c74e4-2ed3-402c-bd08-d57ea8516eec

`Scatter` and `Heatmap`: hover, tap and keyboard inspection; missing data hatched, never 0 · DeepSeek-Flash

https://github.com/user-attachments/assets/cb575946-bab5-4551-9621-489dee55df07

`CodeDiff`, `FileTree` and `JsonView`: code as text, collapsible context, JSONPath copy · DeepSeek-Flash

https://github.com/user-attachments/assets/75900041-3880-4e21-97f8-21bfd41872ad

🚀 Quick start

You need DeepSeek Harness 0.1.7 – 0.2.x (>=0.1.7-rc.1 <0.3.0) and Node.js ≥ 22.13, on Windows, Linux or macOS. Nothing is compiled during install: the package ships its prebuilt browser bundle and has no install scripts.

dsh plugin --profile web add deepseek-harness-intelligent-ui

Restart DSH Web (dsh web) and ask for something visual, for example *"compare Q1–Q4 revenue by channel"* or *"let me explore x²+y²+z²=2Rz in 3D"*. In the desktop app, use Plugins → Add plugin with the same package name.

The guided installer checks Node, the dsh version, pnpm and the profile first, then runs the same dsh plugin add:

npx deepseek-harness-intelligent-ui install     # --profile <name>, --dsh <path>, --dry-run
npx deepseek-harness-intelligent-ui doctor      # diagnose only

Task · Command Upgrade · dsh plugin --profile web add deepseek-harness-intelligent-ui@latest Try a prerelease (when a next dist-tag exists) · dsh plugin --profile web add deepseek-harness-intelligent-ui@next Pin a version · dsh plugin --profile web add deepseek-harness-intelligent-ui@0.2.7 Uninstall · dsh plugin --profile web remove deepseek-harness-intelligent-ui (or npx deepseek-harness-intelligent-ui uninstall) From a checkout · dsh plugin --profile web add file:/path/to/deepseek-harness-intelligent-ui

🩺 Troubleshooting

• *pnpm was not found*: DSH installs plugins with pnpm. Run corepack enable (it ships with Node) or npm i -g pnpm.
• *… is incompatible with dsh X*: your DSH is outside the supported range. Upgrade DSH (npm i -g @deepseek-ai/dsh@latest) or check compatibility for a plugin release that supports it.
• *Upgrading from 0.1.x (dsh-intelligent-ui via file:)*: run dsh plugin --profile web remove dsh-intelligent-ui first. Both packages insert the same intelligent-ui row, and doctor reports the conflict.
• *The UI does not appear*: restart DSH Web after installing, and make sure you started the profile you installed into.

Compatibility. CI installs the packed plugin and runs the full browser E2E on Windows, Linux and macOS against DSH 0.1.7-rc.2, 0.2.0-rc.1, latest, next and alpha, on every push and nightly, so new DSH releases are caught early. Details and local verification history: docs/compatibility.md.

🧠 How the agent decides

The plugin injects a system-prompt section (src/guidance.js) so the model routes each turn itself:

• Plain text for greetings, short facts, explanations, code to copy, and workspace/coding work.
• `ui_render` for structured answers: comparisons, data to explore, dashboards, processes, formulas, functions, code reviews, choices between alternatives, input collection.
• `ui_render_widget` for what the catalog cannot draw: 3D, simulations, animations, games, custom tools.
• Local first: anything computable in the browser must update instantly without a message to the model.

Why these rules were kept after a real-model A/B against a rewritten decision order, and how routing is measured: docs/presentation-routing.md.

/ui forces a UI answer.

Tool · Use it for · Rendering ui_render · Structured answers · Native React components from a fixed catalog, rendered progressively as the JSON streams ui_patch · Changing a UI shown earlier · Component-level operations; untouched components keep their state (details) ui_render_widget · Custom visuals, 3D, simulations, mini-apps · HTML/CSS/JS in an opaque-origin sandboxed iframe: CSS first, HTML morphed in live, JS last, all in one iframe

🧩 Component catalog (ui_render)

Category · Components Layout · Card, Stack, Grid, Tabs (keyboard navigable) Text & callouts · Text (markdown subset), Code, Alert, Badge, Divider Data · Stat, Progress, List, Timeline, `Table` as a data grid: typed sorting (numbers, ISO dates, booleans), search, per-column filters, paging, CSV/Markdown export Charts · `Chart` (bar / line / area / pie, stacked, legend toggles, hover/tap/keyboard tooltips, data export), `Scatter` (continuous x/y), `Heatmap` (sequential or diverging scale, explicit missing cells) Math & diagrams · `Plot` (y = f(x) with sliders that redraw instantly; a safe expression compiler, never eval), `Math` (KaTeX), `Mermaid` Code & structure · `CodeDiff` (multi-file before/after or unified patch), `FileTree` (status badges), `JsonView` (JSONPath, copy) Interaction · Button, `Choice` (single-choice card), Form (text, number, range slider, select, checkbox, date, textarea)

• Validated before it renders. A UI travels as a flat node list with id references (DSH's tool schema has no recursion). The Host rejects unknown types or props, dangling or duplicate ids, cycles, shape mismatches (chart series, heatmap rows), non-finite numbers, uncompilable Plot expressions and oversized data, returning ok:false with paths so the model fixes and retries.
• Text, never HTML. Cells, code, JSON and labels render as React text. CSV export neutralises spreadsheet formula injection and is RFC 4180 with a UTF-8 BOM.
• Accessible and responsive. Charts are focusable and keyboard-inspectable, trees use role=tree, touch targets grow on coarse pointers, and charts lay out at the card's real width so axis text stays legible on phones.
• `Math` and `Mermaid` render in the same sandbox as widgets from the CDN allowlist, with a static fallback when the CDN is unreachable, or fully offline with the offline resource package.

Choice and new sessions

Choice shows a question with option cards and two buttons. Continue in this session sends the picked option's prompt as your next message. Start new session creates a session in the same workspace, submits that prompt there and opens it; nothing from the current conversation is copied, so each option's prompt is self-contained. Button can do the same with sessionTarget: "new". The launch is idempotent: a double click, a retry after a failure or a reload never starts the task twice.

Roadmap and design notes for upcoming components (Map, Calendar): docs/component-roadmap.md.

🔁 Updating a UI in place (ui_patch)

When the user asks to change a UI shown earlier ("make that bar chart a line chart; leave the rest"), the agent patches it by component id instead of re-sending the whole tree.

• Identity: every ui_render result reports the UI's uiId and version 1; each accepted patch reports the new version and an id:Type outline.
• Operations: setProps (merge, unset to remove), replaceNode, insertChild and removeNode, applied in order and all-or-nothing. The next version then passes the same checks as a fresh ui_render.
• Concurrency: baseVersion is required; a stale patch is refused with the current version, and patches never run in parallel.
• Scope: the Host stores nothing. It rebuilds the current version by replaying the conversation's own history, so a patch can only address UIs of the same conversation.
• State: components keep their ids across versions, so untouched components keep their sort, filters, tab, form input, submitted lock and slider positions. A replaced component starts fresh.
• Display: while a patch streams, the previous version stays live. On success the new version renders beneath the reply and the older copy collapses to "Updated to version N". Refused, failed or stopped patches change nothing.
• Widgets are re-rendered rather than patched, and keep their UI.state.

🪟 Sandboxed widgets (ui_render_widget)

Fields follow the OpenIntelligentUI streaming contract, in this order: title, initialHeight, placeholderMessages, css, html, jsFunctions, jsExpressions. Placeholder lines show until css is complete; then a preview frame opens and the partial html is morphed in (idiomorph); when the arguments are complete, the same frame is finalized in place and the code runs.

Runtime helper · What it does UI.three(el, { camera, target, autoRotate }) · Renderer, scene, lights and OrbitControls (drag rotates, wheel/pinch zooms); renders on demand and handles resize UI.canvas(el, draw) · HiDPI 2D canvas, redrawn on resize and theme change UI.state / UI.setState() · Persisted state; inputs with an id are persisted and restored automatically UI.color('accent') / UI.css() · Opaque theme color for three.js/canvas / raw CSS value UI.onTheme(cb), UI.scheme, UI.reducedMotion · React to theme and motion preferences .ui-row, .ui-control, .ui-card, .ui-stage · Helper classes

• Bridge: Bridge.sendPrompt({ text }) disables the clicked control until the host answers; Bridge.openLink({ url }) opens links; Websandbox.connection.remote.* is kept for OpenIntelligentUI compatibility.
• Libraries: an importmap maps three, three/addons/…, d3, chart.js/auto, gsap and canvas-confetti to esm.sh.
• Recovery: script errors show a count with Reload and Ask to fix (which sends the errors to the agent).

Where the UI appears, and what survives

• While a turn runs, the UI renders inline in the tool call; one component instance spans preparing → running → result.
• When the turn ends, DSH folds tool calls into the "process" section, so the plugin re-presents each UI beneath the reply and the inline copy shrinks to a pointer.
• Interaction state is kept per UI in memory and in localStorage (bounded), so moving beneath the reply or reloading restores it. The UIs themselves are rebuilt from the durable session log.
• A stopped generation keeps the finished part (a static widget preview or the completed components) with a notice and Regenerate, even when DSH dropped a call that was still streaming.

🔌 Extending

· Third-party components · Trusted companion plugins register versioned, namespaced components with shared Host/browser props validation, React renderers and disable/uninstall lifecycle. Model output can never register or load code. Extension API and GitDiffViewer template Controlled data and actions · Table and Chart can share a scoped, read-only dataRef with local search/filter/sort/paging. Trusted hosts register schema-checked actions that run only after a native click, with permission checks, write approval and idempotent receipts. Integration guide Portable renderer · renderer/renderer.js renders accepted ui_render, ui_patch and widget artifacts outside DSH Web, e.g. in a mobile WebView. Remote-client contract Offline resources · An optional tarball embeds locked KaTeX/fonts, Mermaid, Three.js, D3, Chart.js and canvas-confetti, so formulas, diagrams and standard 3D scenes work without a CDN. Offline package

🔒 Security model

• Widget isolation: ` with srcdoc and **no** allow-same-origin. The E2E verifies in a browser that parent.document, localStorage and document.cookie throw SecurityError, fetch is blocked, and window.origin === "null"`.
• CSP: default-src 'none'; scripts, styles, fonts and images only from cdnAllowlist; connect-src 'none' unless allowNetwork; form-action 'none'; base-uri 'none'.
• Messages: accepted only from the frame's own contentWindow with a per-frame random token and a valid shape; forged messages are dropped (verified in E2E).
• `sendPrompt` limits: refused for 1 s after mount, at most one per 2 s, capped at sendPromptMaxChars, queued as a normal user message.
• `openLink`: https only, optionally restricted to openLinkAllowedOrigins.
• Catalog: text renders as React elements, never raw HTML; Plot expressions are compiled over a fixed math table with no eval, Function or property access; new-session launches and host actions need an explicit native click.

⚙️ Configuration

The bundle patch inserts the plugin row automatically. To change settings, add to your profile's cordis.patch.yml:

- id: intelligent-ui
  config:
    allowNetwork: true
    cdnAllowlist: [https://esm.sh, https://cdn.jsdelivr.net]

Option · Default · Meaning enableCatalog · true · Register ui_render and ui_patch enableWidget · true · Register ui_render_widget injectGuidance · true · Add the "when to use UI" system-prompt section cdnAllowlist · esm.sh, jsdelivr, unpkg · Origins widgets (and Math/Mermaid) may load resources from allowNetwork · false · Let widgets fetch and use WebSocket to those origins maxWidgetHeight · 2400 · Maximum frame height (px) maxWidgetBytes · 400000 · Reject larger widgets sendPromptMaxChars · 4000 · Longest message a widget, button, form or choice may send openLinkAllowedOrigins · [] · If non-empty, the only origins openLink may open

📊 Real-model quality baseline

A tool call whose arguments parse is not the same as a UI the user can use. npm run eval:real measures three layers separately; a pass in one is not evidence for another.

Layer · What it proves · Latest · 2026-10-16 · deepseek-flash Mock E2E · Rendering, streaming, interaction, persistence and isolation, with scripted answers · 30 / 30 Real model · API · Right tool, valid arguments on the first try, and the rendered UI contains the requested components and data · 30 / 32 (94%) Real model · browser · What the model generated works in real DSH Web + Chrome: renders, interacts, survives stop/resume and bad networks, no console errors · 9 / 11 automated8 / 11 after review

Timing (real model) · p50 · p95 First visible UI preview · 7.6 s · 70.2 s UI generation complete · 11.8 s · 73.4 s Browser-side execution after the stream · 26 ms · 2.9 s

The long tail comes from large Three.js widgets (10–20 KB of code before the preview starts). The failures are real model and agent behaviour: a call cut off mid-stream, an agent that researched on the web instead of rendering, Three.js scope slips repaired with "Ask to fix", and invalid first attempts the model fixed on retry. Each case, its checks and the full findings: docs/real-model-eval.md.