dsh-computer-use-en

dsh-computer-use-en

English port of @copylee/dsh-computer-use with DSH 0.2.1 unpacked koffi.node auto-discovery.

Let DeepSeek Harness directly observe and operate your Windows desktop: view the screen, click the mouse, type on the keyboard, launch or focus apps, manage windows, read Windows UI Automation (UIA) trees, and read/write the clipboard. While in control, an orange screen glow and status pill ("DeepSeek Harness is controlling your computer") appear on screen, and DeepSeek Harness minimizes into a draggable mini progress card in the bottom-right corner showing streaming replies and recent steps. Press Esc at any time to pause or resume, or click Stop to end the turn.

Windows 10 (2004+) / Windows 11 only.

Features

Feature · Description Screenshot coordinate space · The model works directly in pixel coordinates of the latest screenshot; DPI scaling and multi-monitor offsets are converted automatically (accurate even at 150%–200% scaling). Auto-screenshot after actions · Clicks, typing, keys, scrolling, and wait wait for the screen to settle (up to ~2.5s) and return a fresh screenshot automatically in a single round-trip. Batched actions (`computer_batch`) · Execute multi-step sequences (e.g., focus field → type text → press Enter → wait for dialog → save) in a single tool call. Target controls by name (`target`) · Pass target (e.g. "Save" or "button:OK") instead of coordinates — works even for menu items or dialog buttons that appear mid-batch (waits up to 3s). Wait for conditions (`until`) · wait with until ("window:Save As", "target:Save") proceeds as soon as the window or control appears (or disappears with gone: true). Read windows as text (`ui_elements`) · Pass text: true to ui_elements (or finish: "text" to computer_batch) to read documents, web pages, chat replies, or logs directly via UI Automation without scrolling screenshots. Per-app access control · Requests per-session permission via request_access (auto-approved when using the danger-full-access preset), or switch to Allow all apps in Settings. Never controls DeepSeek Harness itself or password managers. Mini progress card & overlay · Floating bottom-right progress card streams the AI's reply and step timeline, is excluded from AI screenshots (WDA_EXCLUDEFROMCAPTURE), and lets clicks pass through. Instant human handoff (`Esc`) · Press physical Esc to pause/resume; typing on the physical keyboard pauses the AI until you stop, and switching windows makes the AI re-check the screen before acting. Clean clipboard & streaming input · Long text is streamed paragraph-by-paragraph without triggering editor auto-format glitches, stays out of Windows Clipboard History (Win+V), and restores your clipboard afterward. App Skills (`app_skill`) · Per-application Markdown notes (~/.dsh/computer-use-skills/) automatically attached when open_application opens an app, with 10 built-in English skills (Chrome, Word, Excel, PowerPoint, File Explorer, Notepad, Obsidian, Typora, WeChat, QQ). Zero native build steps · Uses DSH's signed in-box koffi.node for fast in-process screenshots/input and compiles helper/CuHelper.cs on first run via Windows' built-in .NET Framework csc.exe.

Installation

From GitHub Release or GitHub:

dsh plugin --profile web add "https://github.com/ALAMBICCO/dsh-computer-use-en/releases/download/v0.8.1/dsh-computer-use-en-0.8.1.tgz"

Agent Tools

Tool · Description computer · Single desktop action: screenshot, left_click, double_click, triple_click, right_click, middle_click, mouse_move, left_click_drag, left_mouse_down, left_mouse_up, scroll, type, key, hold_key, wait, zoom, cursor_position. computer_batch · Run an ordered array of actions in one round-trip, stopping on the first error and returning one settled screenshot (or window text). open_application · Open an app by Start Menu name, desktop shortcut, .exe path, or URL; brings running windows to the front or restores tray apps from their notification-area icon. windows · list, focus, minimize, maximize, restore, close, or tidy top-level windows. ui_elements · Read clickable/editable UI Automation controls with screenshot-space coordinates, or pass text: true to read window text. clipboard · Read or write plain text to the Windows clipboard (restored after the turn). switch_display · List monitors or switch which display screenshots and coordinates target. app_skill · list, read, append, or write per-application operating notes. request_access / list_granted_applications · Request and inspect per-session app control grants.

Credits & License

MIT License. Based on @copylee/dsh-computer-use by copylee, translated to English and adapted for DSH 0.2.1 unpacked koffi.node resolution.