dsh-image-annotate

dsh-image-annotate

Draw on a pasted screenshot before it reaches the conversation — the "paste, mark it up, send" move from Cursor, as a DeepSeek Harness client plugin.

Paste a screenshot into the composer and an overlay opens over the picture. Press the pencil to get the tools, mark whatever needs pointing at, and press Send: the strokes are flattened into a PNG at the screenshot's native resolution and the file is filed into the composer as an ordinary attachment. Esc files the image exactly as it was pasted, byte for byte.

Nothing upstream is patched. The plugin registers into the official slot, locale and Conversation faces of the web client, so it survives DSH upgrades and uninstalls by removing one row.

What you get

• Paste to draw — Ctrl+V with a single image (no text on the clipboard) opens the overlay. A paste carrying text or several images is left to the shell.
• Tools — pen, arrow, rectangle, ellipse, and a text stamp.
• Style — six colours (including white, for dark screenshots) and three size steps; on the text tool the same steps set the type size (12 / 24 / 48 px in image coordinates).
• Text — click with the text tool to open a field where you clicked: Enter inserts, Shift+Enter starts a new line, Esc cancels the note (not the overlay).
• Undo / Clear — per stroke, or everything; Ctrl+Z works too.
• Send — Enter or the button: strokes are baked into a PNG 1:1 with the source image.
• Without drawing — Esc or the button: the original file is filed untouched, with no re-encode.
• Re-annotate later — a small pencil in the composer appears whenever the composer holds an image, and opens the overlay on that image without pasting it again.
• Honest failures — an image that cannot be decoded says so and still lets you file it as pasted, instead of leaving an empty frame.

Install

The package declares dsh.client (browser half) and dsh.bundle.patch (loader row), so it installs like any other DSH plugin:

# from a checkout, or from a published copy
dsh plugin --profile <profile> add /path/to/dsh-image-annotate
# after that, reload the page once (a NEW row is picked up when the page's module table is built)

Manual mounting into a profile, which is what this repository was developed against:

# 1. live link so edits are picked up by client HMR without reinstalling
New-Item -ItemType Junction -Path "$env:USERPROFILE\.dsh\profiles\<profile>\node_modules\dsh-image-annotate" `
         -Target (Resolve-Path .).Path

# 2. dependency of the profile:  ~/.dsh/profiles/<profile>/package.json
#    "dsh-image-annotate": "file:<path to this checkout>"

# 3. loader row:  ~/.dsh/profiles/<profile>/cordis.patch.yml
#    - insert:
#        - id: image-annotate
#          name: dsh-image-annotate

Rollback = drop the insert row, the dependency and the junction.

How it works

Four decisions worth keeping when editing:

1. The intercept runs in the capture phase on document, before Lexical hands PASTE_COMMAND to the shell. A "just an image" paste therefore never slips into the rail unannotated, and cancelling returns the original File to the composer with no re-encode.
2. The overlay is a portal onto document.body. The composer card is position: relative and the app frame carries its own transforms, so a position: fixed layer inside that tree can get stuck in an ancestor.
3. Sending uses the shell's own intake route, not faked input: conversation.createDrafts(sessionId, [file]) registers the draft (and its preview) and inputActions.addAttachments(ids) puts it in the rail — the same two calls the composer bar makes for a drop. If the rail refuses (a send is being adjudicated), the draft is released instead of leaking. When those faces are unavailable the plugin falls back to a synthetic ClipboardEvent on [data-composer-input], which is the path a real Ctrl+V takes.
4. A crashed slot entry is gone for good. A client plugin's slot entry that throws while rendering (or inside an effect) is retired from its cell by the framework's per-entry error boundary for the rest of the page's life — the plugin then looks dead until the page reloads. So: the overlay and the re-annotate chip live behind the plugin's own error boundary, handlers are wrapped in try/catch, and the standard session hooks are selector hooks (useInput(selectAttachmentIds), never a bare useInput(), which throws).

Strokes hold image coordinates; the canvas only scales them for display, which is why flatten writes a PNG at the screenshot's native size. Text is a stroke too ({tool: 'text', points: [anchor], text}) with textFontSize(width) = max(10, width·4) image pixels, so it participates in undo, clear and export like any other mark. The overlay palette is resolved in JavaScript from the live theme tokens with explicit fallbacks — an empty theme custom property makes the whole CSS declaration invalid, which is how an overlay ends up drawn but invisible.

Verify it yourself

# 1. syntax
node --check index.js && node --check lib/client.js

# 2. headless logic (64 checks): paste classification, stroke rasterisation, text stamps,
#    the flatten pipeline, the official attachment route plus every fallback, the palette,
#    the synthetic paste, slot registration and the diagnostics surface
node tools/smoke.mjs

# 3. every seam this plugin stands on, against a DSH build's own sources
#    (--root <@deepseek-ai root>, or let it discover extracted builds and npx caches)
node tools/check-api.mjs --all

# 4. a running GUI is serving THIS revision (needs the host's token URL or its launcher log)
node tools/verify-live.mjs --url "http://127.0.0.1:<port>/?token=<token>"

# 5. end-to-end in a real headless Chromium against that GUI — no model call, no clicking:
#    paste -> overlay -> pencil -> stroke -> Send -> draft in the rail -> Esc keeps the original
node tools/live-ui.mjs --url "http://127.0.0.1:<port>/?token=<token>"

# 6. no GUI at hand? stand up a throwaway host (own home, own port) and clean up after
node tools/iso-host.mjs up
node tools/live-ui.mjs --url "<URL printed above>"
node tools/iso-host.mjs down

# 7. the pre-publish gate: no personal data, no network/storage/eval in the payload
node tools/audit-public.mjs

tools/live-ui.mjs drives the real app: it dispatches a paste carrying a real PNG, checks that the overlay opens with its canvas, presses the pencil, paints a stroke with pointer events, sends, and asserts the composer rail gained a thumbnail; then it repeats with a broken image to prove the entry survives a failure, and finally exercises the text stamp. It sends no prompt.

Security and privacy

The plugin payload (index.js, lib/client.js, cordis.patch.yml) is deliberately boring, and tools/audit-public.mjs fails the build if that changes:

• No network access — no fetch, XMLHttpRequest, WebSocket, sendBeacon, no dynamic import.
• No storage — no localStorage, sessionStorage, indexedDB, no cookies: the drawing state lives in React state and dies with the overlay.
• No code or HTML injection — no eval, no new Function, no innerHTML; all UI goes through React elements, and user text is passed to canvas.fillText, never parsed as markup.
• Permissions — none requested; the plugin never touches the clipboard API (it only observes a paste the user already made).
• Where the pixels go — the pasted image is decoded from a blob URL created in the page, the annotated copy is produced on a <canvas>, and the resulting File is handed to the same attachment path a normal paste uses. Nothing leaves the page on the plugin's initiative; whatever the shell then does with an attachment is the shell's own behaviour.
• Contacts with the host — a running GUI's HTTP endpoint is queried only by the development tools in tools/ (read-only: the boot graph and the served bundle), and only at the URL you pass.

Personal data: the repository contains no user paths, no e-mail addresses, no tokens, and no screenshots. tools/audit-public.mjs scans both the payload and the whole tree for those, so a re-run is a one-liner instead of a promise.

Compatibility

• Verified live on DSH 0.2.0-rc.2 and 0.1.5-rc.1 client builds (42 end-to-end checks each).
• tools/check-api.mjs asserts the ten seams the plugin uses (slot declaration and scope, the inputActions provide face, the shell's addAttachments, createDrafts, composer data-* markers, the paste command's clipboard intake, the locale registry, standard-props spreading, the selector-hook contract of the standard hooks, and the dsh.client module-table contract).
• If a future DSH moves one of those seams, the plugin fails at check-api instead of half-working.

Limitations

• One image per paste: clips carrying text or several images are left to the shell.
• Dragging a file into the composer is not intercepted (only pasting is).
• Text is not draggable after it is committed, and drawings do not persist between overlay openings — after Send the result is an ordinary attachment.
• No crop, no blur/mosaic for sensitive areas, no extra shapes yet. Each tool is one branch in applyStrokes, so adding another is cheap.
• While the overlay is open it owns Esc, Ctrl+Z and Enter; while a text field is open it hands those keys to the field.

Diagnostics

window.__dshImageAnnotate in the page console answers "what happened": mounted (the plugin applied), opened (pastes intercepted), lastRoute (draft | paste | failed), lastName / lastBytes (what was filed), lastStroke, lastStrokes, and lastError (decode, decode-timeout, insert, flatten: …, overlay: …).

If a paste seems to be ignored, check document.querySelector('[data-slot-error]') first: a value of conversation.input.right means the slot entry was retired by an earlier crash, and only a page reload brings it back. tools/live-ui.mjs sections 8–9 keep that from regressing.

License

MIT — see LICENSE.

---

По-русски, коротко

Плагин для веб-интерфейса DeepSeek Harness: вставляете скриншот в композер (Ctrl+V), поверх картинки открывается оверлей, карандаш раскрывает инструменты (карандаш, стрелка, прямоугольник, эллипс, текст; 6 цветов, включая белый; 3 размера). «Отправить» впечатывает штрихи в PNG в натуральном разрешении и вкладывает файл в диалог; Esc вкладывает исходный файл без перекодирования. Уже вложенную картинку можно дорисовать — карандаш в панели композера. Ничего в самом DSH не патчится: только официальные точки расширения. Проверки и аудит — командами из раздела «Verify it yourself».