dsh-windows-remote-ssh

dsh-windows-remote-ssh

Control a remote Windows desktop over SSH from DeepSeek Harness. Same tool surface and safety model as the local dsh-click plugin — screenshots, UI Automation reads, click/type/scroll/key, app listing/launch — but every action runs on a Windows box reached over plain SSH instead of a local subprocess. The harness itself can run from Linux, macOS, or Windows; only the target needs to be Windows.

How it works

• No pre-installed agent on the target. The only thing this plugin assumes on the remote host is a working SSH server (Win32-OpenSSH, built into modern Windows). Everything else — window enumeration, UI Automation, screenshots, input delivery — is done by a single PowerShell script this package ships (native/win32/dsh-windows-remote-ssh-helper.ps1), staged onto the target over SFTP the first time it's needed and re-staged automatically if it ever changes (content-hash compared against a marker file). It uses only classes already in stock Windows (.NET's System.Windows.Automation, System.Drawing, and user32.dll/kernel32.dll P/Invoke) — no Python, no third-party modules, no separate installer.
• File-based request/response, not stdin/stdout. Each call writes a small JSON request file over SFTP, runs the helper once, and reads a JSON response file back over SFTP. Moving bytes through SFTP rather than piping them through the SSH exec channel's stdin/stdout sidesteps Windows console code-page and UTF-8/UTF-16 encoding pitfalls entirely.
• Runs in the user's real desktop session, not a hidden one. This is the detail that makes GUI automation over plain SSH actually work: Win32-OpenSSH runs a normal ssh user@host command in a non-interactive Session 0 / window station, completely isolated from the interactive desktop — from there, EnumWindows, UI Automation, and screen capture all come back empty even though the process runs fine. So instead of executing the helper directly, this plugin schedules it as a one-shot Scheduled Task bound to the connected user with the "only when logged on" flag (attaching it to their interactive session), triggers it immediately, and polls for the response file — the standard workaround for driving a Windows GUI over SSH. See Requirements below for what this means for the target account. The scheduled process is launched through a tiny VBScript wrapper (WScript.Shell.Run with a hidden window style), not powershell.exe -WindowStyle Hidden directly — that flag alone still lets a console flash on screen for a moment before it takes effect. Verified empirically: zero window sightings across repeated calls while continuously polling for one.
• Never steals the user's mouse/keyboard focus, by default. Actions prefer UI Automation invoke/value/scroll patterns; when an element exposes none, they fall back to posted window messages (WM_LBUTTONDOWN/WM_KEYDOWN/etc.) rather than moving the real cursor or bringing the window to the foreground. Bringing a window to the foreground is available only as an explicit, config-gated fallback (focusFallback: 'allow', default off). A further, more invasive opt-in (allowHardwareInput: true, also default off) lets a call pass hardware: true to deliver via real SendInput events instead — the only way to reach apps that ignore the posted-message queue entirely (games, DirectX/UWP surfaces, raw-input readers); see hardware: true below.

Requirements

• The target account must be a local administrator and must already be logged on interactively (an unlocked console/RDP session) when a tool call runs — the Scheduled Task attaches to that existing session; it does not create one. If nobody is logged on, calls fail with a clear SCHEDULE_FAILED / timeout error rather than hanging.
• Win32-OpenSSH (sshd) running and reachable on the target.
• Node.js ^22.19.0 || >=24.0.0 wherever the harness itself runs.

Installation

dsh plugin --profile web add "github:pureexe/dsh-windows-remote-ssh#main"

Then mount it with a profile patch (see cordis.patch.yml for every field, or override at bundle level):

- insert:
    - id: dsh-windows-remote-ssh
      config:
        ssh:
          host: <ip>
          user: <username>
          # Prefer a private key over a password where you can:
          # privateKeyPath: /path/to/id_ed25519
        autoApproveWindows: ['^Notepad']

Connection details may also come from environment variables instead of the profile (handy for CI, and for keeping credentials out of a checked-in patch): SSH_HOST, SSH_PORT (default 22), SSH_USER, SSH_PASSWORD, SSH_KEY_PATH, SSH_KEY_PASSPHRASE. Explicit config.ssh.* fields win over the matching env var when both are set.

Supplying the SSH target: two options

There are two ways to tell this plugin which Windows machine to control, and they can be mixed — a configured default, plus per-call overrides for the odd exception.

1. Configure ssh.host/ssh.user/ssh.password (or ssh.privateKeyPath) once, as shown above — recommended. The target and its credentials never appear in the model's context and never get written to the harness's own tool-call session logs. Every tool call just omits ssh entirely and uses this default. This is the right choice for a fixed, known target (your own desktop, a lab VM, a CI runner).

2. Leave config.ssh unset and let the model supply a target per call. Every entry-point tool (screen_shot, screen_read, app_list, app_launch) accepts an optional ssh argument — host, port, user, and either password or privateKeyPath — that the model fills in from whatever you tell it in chat ("connect to 10.0.0.12 as admin, password hunter2"). No config edit needed, which is handy when the target varies conversation to conversation or isn't known ahead of time. The tradeoff: that host/user/password is now part of the conversation — it flows through the model's context like any other tool argument and gets captured by the harness's own tool-call logging, the same way any other tool call's arguments are. If you can, prefer option 1, or at least a privateKeyPath in chat instead of a raw password, so the secret itself lives on disk rather than in the transcript.

If host/user are given (in chat, or in config.ssh) but neither password nor privateKeyPath is, the plugin falls back to the default SSH identity on the machine running the harness — ~/.ssh/id_ed25519, id_ecdsa, then id_rsa, the same lookup order the plain ssh CLI uses — before failing with "requires either a password or a privateKeyPath". This is what lets "connect to 10.0.0.12 as alice" work with no credential in the call at all, as long as that machine's default key is already authorized on the target.

click/type/scroll/key never need an ssh argument either way — they replay against whatever host the basedOn observation they cite came from, so a single conversation can safely juggle more than one remote machine without an action ever landing on the wrong one.

With no config.ssh default and no per-call ssh argument, a call fails immediately with a clear error asking for one or the other — it never silently guesses or hangs.

Tools

With lazyToolLoading on (the default), only pc_control is registered at startup; calling it once registers every other tool below for the rest of the conversation, so a session that never needs remote control never pays the prompt-token cost of the other ~20 schemas. Set lazyToolLoading: false to register everything immediately instead.

screen_shot — the image can be smaller than the real captured area

click/move/key/etc. all take real screen coordinates — the same space as window.rect and display_list — never a raw pixel position read off the screenshot image. The image itself can be downscaled from that real size for two independent reasons: this plugin's own maxSide cap, and/or the harness's attachment store shrinking it further on save. Whenever the returned image is smaller than the real captured area, the tool result carries a scaleNote stating the real size/origin and the exact pixel-to-screen-coordinate conversion, so a caller that estimated a click target visually can convert it correctly instead of clicking the wrong spot. cursor_location is also useful here — it reports where the real cursor actually is, in the same real screen coordinates, to sanity-check a computed target or confirm where a previous action actually landed.

filesystem_pull / filesystem_push — move files between the two machines

Download a file from the remote Windows host to inspect it here, and upload it back after editing — useful for a config file, a script, or an image.

• filesystem_pull(remotePath) downloads the file and, depending on what it is: an image comes back as a real image attachment (the same mechanism screen_shot uses — actually visible, not just bytes); small text comes back inline as a plain string the model can read and reason about directly; anything else becomes a generic file attachment. Every case returns a reference (or the text itself) that filesystem_push can consume to write it straight back.
• filesystem_push(remotePath, ...) takes exactly one of: content (a literal string — the natural way to write back an edited text/config file), or image/file (an attachment reference re-supplied exactly as an earlier filesystem_pull returned it, to write those exact bytes back unchanged — e.g. after some other tool produced an edited version of a pulled image). Creates missing parent directories by default.
• Both are gated by approval like a mutating action — unlike screen_shot/screen_read, reading (or writing) an arbitrary path isn't treated as a free "observer": it can expose content the operator never put on screen.
• maxFilesystemTransferBytes (default 10 MB) caps one transfer in either direction; going over it refuses outright rather than truncating (which would just corrupt binary content). maxInlineFilesystemBytes (default 100 KB) is the smaller threshold above which a pulled text file is stored as a file attachment instead of inlined, so a merely-large file doesn't bloat the model's context.

hardware: true — real SendInput delivery for apps that ignore posted messages

click, type, key, scroll, and move all accept an optional hardware: true argument. When set (and config allowHardwareInput: true, off by default), the action is delivered as real SendInput events — the actual OS mouse cursor moves and clicks, real keyboard-level key events are injected — instead of UI Automation invoke or posted window messages (WM_LBUTTONDOWN/WM_KEYDOWN/etc.). This exists because posted messages, however reliable for ordinary desktop apps, simply never reach some targets: games, DirectX/UWP-rendered surfaces, and anything else that reads raw input (GetAsyncKeyState, RAWINPUT) instead of processing the window message queue at all. SendInput is indistinguishable from a physical mouse/keyboard to the receiving app, so it reaches those too.

This is categorically more invasive than focusFallback: 'allow' (which only brings a window forward but still posts messages): hardware: true always brings the target window to the foreground and moves the real cursor / steals real keyboard focus, visible to anyone watching the remote desktop and disruptive to whatever the human at that machine was doing. It is off by default and must be turned on deliberately via allowHardwareInput: true; a call that passes hardware: true without that config set fails with a clear error rather than silently falling back. Once enabled, it is still gated by approval like every other mutating action. The action's result reports delivered: "hardware" when this path was actually used.

One side effect worth knowing: because it's a real hardware event rather than a message posted to one window, hardware: true on move's drag also reaches genuine OLE/shell drag-and-drop (e.g. dragging a file between two Explorer windows) — the one thing the README's honest limits on posted-message drag (below) say plain move/drag cannot do.

move — mouse move and drag (posted messages only, honest limits)

move posts WM_MOUSEMOVE to reach a point inside an observed window and, when drag: { toX, toY } (or toElementId) is given, also posts WM_LBUTTONDOWN at the source, several interpolated WM_MOUSEMOVE steps toward the destination, and WM_LBUTTONUP there — exactly like every other action in this plugin, the real OS cursor never moves and no window is brought to the foreground unless focusFallback: 'allow'.

Be honest about what posted-message drag can and cannot do. It reliably works for controls that react to simple mouse-move/button events — sliders, canvases, custom-drawn controls that track the mouse themselves. It is not real OLE/shell drag-and-drop: dragging a file between two Explorer windows (or anything else that relies on Windows' own drag-detection heuristics and IDropTarget/IDataObject negotiation) generally will not work through posted messages — that requires actual SendInput-driven physical mouse events, which this plugin does not generate by default. Use move/drag for UI manipulation within a single control, not for cross-application drag operations — unless hardware: true is set (see below), which does generate real SendInput events and does reach real drag-and-drop, at the cost of moving the real cursor and stealing focus.

wait_for — poll for a condition instead of guessing a fixed delay

wait_for polls the remote host roughly every 500ms until a condition is met or timeoutMs elapses (default waitForTimeoutMs, bounded 500..120000), entirely on the harness side (repeated screen_read-equivalent/app_list-equivalent calls) — no new native helper operation was needed for this one. Three condition kinds:

• kind: 'element' — target names a window (as screen_shot/screen_read do) and match (name/automationId/controlType, at least one, case-insensitive substring) names what to look for in its accessibility tree.
• kind: 'window' — match.title names a substring to look for across every top-level window's title.
• kind: 'foreground' — target names a window that must exist and be the current foreground window.

A timeout is not an error. The result always comes back as a normal tool result carrying met (false on timeout), timedOut, and whatever was last observed (observationId, window, elements, pixels) — the model needs to see what is actually on screen when a wait times out, not just a generic failure.

clipboard — get/set the remote clipboard text

Backed by Get-Clipboard/Set-Clipboard (built into PowerShell 5.1 on Windows, no extra module), run inside the same interactive Scheduled-Task session as every other operation — the clipboard is per-session, so a plain non-interactive SSH exec would see an entirely different, empty clipboard. action: 'get' is read-only and never needs approval (like app_list); action: 'set' is mutating and requires approval unless requireApproval is off, gated the same way powershell is (no window subject — the clipboard isn't scoped to any one window).

process — list or kill remote processes

Backed by Get-Process/Stop-Process. action: 'list' returns pid, name, executable path (when resolvable), and main window title (when any) for every running process — read-only, never needs approval. action: 'kill' takes pid or name (exactly one) plus optional force, and is gated by approval like powershell/clipboard set (no window subject).

display_list and multi-monitor / region screen_shot

display_list enumerates every monitor via [System.Windows.Forms.Screen]::AllScreens — index, screen-space rect, and whether it's the primary display. Read-only, never needs approval.

screen_shot gained two optional, backward-compatible parameters: display: N (with wholeScreen: true) captures one specific monitor's bounds instead of the primary screen, and region: { left, top, right, bottom } captures exactly that screen-space rectangle regardless of target/wholeScreen/display. Neither parameter changes anything about an existing call that supplies neither — a plain screen_shot() or screen_shot({ wholeScreen: true }) behaves exactly as before. Like wholeScreen, a region/display capture's windowId is the 0 sentinel and cannot be used as a basedOn target for a later action.

notify — a real Windows Action Center toast

notify shows an actual Windows 11 (and 10) Action Center toast notification — not a legacy balloon-tip/NotifyIcon popup — via the WinRT Windows.UI.Notifications.ToastNotificationManager APIs, loaded from PowerShell 5.1 through the standard reflection-load technique ([Windows.UI.Notifications.ToastNotificationManager, Windows.UI.Notifications, ContentType = WindowsRuntime]). It defaults to the well-known built-in Windows PowerShell AUMID ({1AC14E77-02E7-4E5D-B744-2EB1AE5198B7}\WindowsPowerShell\v1.0\powershell.exe, configurable via notifyAppId, overridable per call with appId) so it works out of the box on stock Windows 10/11 with no app registration — the standard community technique for toasting from PowerShell. Mutating: gated by approval like powershell (no window subject).

multi_action — a batch of click/type sub-actions, plus list selection

multi_action runs a sequence of click/type sub-actions against one cited basedOn observation in a single tool call and a single approval ask — rather than one click/type call (and one approval prompt) per step. Each step reuses the exact same click/type addressing and re-verification click/type already do on their own (an elementId step re-resolves that exact element by UIA RuntimeId right before acting; a coordinate step does not). By default the batch stops at the first failing step; continueOnError: true runs every step regardless and returns one outcome (or error) per step.

For list/grid multi-selection, a click-kind step may set selectionMode ('select'/'add'/'remove'/'toggle'): when the addressed element supports UIA's SelectionItem pattern, this invokes Select() / AddToSelection() / RemoveFromSelection() directly instead of posting a click — more reliable than synthesizing modifier-key-held clicks, and it never needs to hold a real modifier key down. Falls back to a plain posted click when the element doesn't support the pattern.

window_control — minimize/maximize/restore/move/resize/close

window_control changes one observed window's state via ShowWindow (minimize/maximize/restore) and MoveWindow (move/resize) Win32 calls, or posts WM_CLOSE (close). Requires basedOn and approval, exactly like click/scroll/key (participates in autoApproveWindows). Since a successful move/resize/minimize/maximize changes the window's own rect/state, a subsequent action against the same window needs a fresh screen_shot/screen_read first — the same freshness rule that already applies everywhere else. app_list's window entries (and the underlying WindowInfo shape) now also carry minimized/maximized booleans (IsIconic/IsZoomed), at no extra cost.

invoke — direct UIA pattern-based control interaction

Instead of posting a synthetic click or keystroke, invoke calls the target element's own UI Automation pattern method directly — more reliable for a control that reacts to its real pattern method but ignores posted input. Always addressed by elementId (never coordinates: this tool always targets a specific element). pattern is one of:

• 'invoke' — InvokePattern.Invoke()
• 'toggle' — TogglePattern.Toggle()
• 'expand' / 'collapse' — ExpandCollapsePattern.Expand()/.Collapse()
• 'select' / 'addToSelection' / 'removeFromSelection' — SelectionItemPattern.Select()/.AddToSelection()/.RemoveFromSelection()
• 'scrollIntoView' — ScrollItemPattern.ScrollIntoView()
• 'setValue' — ValuePattern.SetValue(string), requires a string value
• 'setRangeValue' — RangeValuePattern.SetValue(double), requires a numeric value

If the addressed element doesn't support the requested pattern, invoke fails with a clear error naming the pattern and the element's control type — it never silently no-ops. Like elementId-addressed click/type, the helper re-resolves the element by its UIA RuntimeId immediately before acting, so the whole-window tree hash adds no safety here. Mutating: gated by approval like click/type.

read_text / read_table — structured content via the Text/Grid/Table patterns

Two read-only tools that expose UI Automation content richer than the plain Name/Value screen_read already returns:

• read_text(basedOn, elementId) reads one text/document/edit element's full conte