dsh-olhos

dsh-olhos

Eyes for an agent inside DeepSeek Harness: capture a URL or a local HTML file, measure how much the screen drifted, and test written expectations against the image in the same 768-dimensional embedding space.

Every verdict comes with its score. "The screen is fine" without a number is the sentence this plugin exists to prevent.

One plugin line, two surfaces:

Install

npm install -g pnpm
npx @deepseek-ai/dsh plugin --profile web add github:songjongdevsecops/dsh-olhos

Restart the harness: the bundle layer is read at boot. Then check the loaded line:

npx @deepseek-ai/dsh --profile web --dump-config | grep -A6 olhos

Expected shape:

- id: olhos
  name: dsh-olhos
  config:
    capturas: temporarios/olhos
    limiar: 0.7344
    navegador: casa

The package declares dsh.bundle.patch, so dsh plugin add appends dsh-olhos to dsh.profile.bundles in the profile manifest. That line is the only activation.

Update: re-run the add command and restart. Uninstall: npx @deepseek-ai/dsh plugin --profile web remove dsh-olhos, then restart.

Requirements

• Node.js ^22.19.0 || >=24.0.0 (the range DeepSeek Harness supports).
• pnpm on PATH; the harness uses it to manage profile plugins.
• DeepSeek Harness itself, for example npx @deepseek-ai/dsh web.
• Python 3 on PATH, with websocket-client for the capture script: python3 -m pip install websocket-client.
• Chrome or Chromium listening on CDP. The default (navegador: casa) attaches to http://127.0.0.1:9222, so start the browser with --remote-debugging-port=9222. With navegador: proprio the plugin launches its own disposable headless browser (OLHOS_CHROME, default google-chrome) instead.
• An EmbeddingGemma engine checkout, which produces the scores. Point OLHOS_GEMMA_RAIZ at it (default ~/workspaces/tools/embeddinggemma). The directory must contain venv/bin/python, src/embeddinggemma.py, src/vetores.py, and the model cache under modelos/. Its own install script builds the CPU environment (torch, sentence-transformers) and downloads the 1.5 GB model.

The plugin does not bundle the embedding engine. Without that checkout a capture still works, but no score can be produced, and olhos_verificar fails with a message naming OLHOS_GEMMA_RAIZ.

The shell scripts (.sh) and Python companions run on Linux and macOS. On Windows, use WSL or Git Bash with Python 3 available.

Usage

Ask the agent to check something visible. It calls the tool with a target and one expectation per line:

Use olhos_verificar on http://127.0.0.1:3080/ with the expectations
"the panel shows three session cards" and "the error message is visible".

The tool answers with a score per expectation and a verdict:

limiar 0.7344
| expectation                         | score  | verdict   |
| the panel shows three session cards | 0.7225 | not found |
| the error message is visible        | 0.6254 | not found |
0/2 confirmed. Lowest score 0.6254. Capture: temporarios/olhos/.../20261013-163453.png

The threshold is strict: a score must be greater than the threshold to confirm. Scores below it are reported as not found, which is the deliberate precision-over-recall bias described under Limits.

In the chat:

/olhos http://127.0.0.1:3080/                       # capture and drift against the last reference
/olhos http://127.0.0.1:3080/ "three cards" "error"  # also test these expectations
/olhos marcar http://127.0.0.1:3080/                # capture now and store it as the good reference
/olhos ultima http://127.0.0.1:3080/                # drift only, no new capture
/olhos achar "login screen with an error"           # find an old capture by meaning
/olhos ajuda

Expectations should be short, concrete and visible. "The panel shows three session cards" works; "the layout is nice" does not.

A threshold passed to the tool (limiar) is stored for that target in limiar.json and used on the following calls.

Configuration

Every path is configurable and the defaults derive from your home directory, so nothing is tied to a specific machine. Override by environment variable or by the config of the olhos line in your profile patch. A patch replaces the whole config of the line, so restate every key you want to keep.

Profile patch example (~/.dsh/profiles/web/cordis.patch.yml):

- id: olhos
  config:
    capturas: '/home/you/.dsh/olhos'
    limiar: 0.7344
    navegador: casa

Verification

From a clone of the repository:

node --test tests/
scripts/olhos.sh selftest

node --test needs only Node.js: 20 tests covering the tool and command registration, the bundle contract, the strict threshold decision and the per-target key.

scripts/olhos.sh selftest needs the engine virtualenv installed (not the model weights) and checks the threshold math without loading the model.

To verify the install, check the --dump-config output above and ask the agent to run olhos_verificar against a page that is open in your browser.

Measured results

The default threshold and the warnings below come from a calibration run on this plugin: 20 written expectations, 22 captures, 440 labeled pairs, EmbeddingGemma, 768 dimensions.

The threshold is the highest negative score (max(negativos)): no false expectation passes it, which is why false positives are zero. The price is false negatives: true statements whose score falls below the threshold are reported as not found. In a live verification on the calibration page, the panel really had the three cards and scored 0.7225, below 0.7344.

The drift margin on that set is negative: the two groups overlap. A whole-screen cosine flagged a header path change and missed the three cards disappearing. That is why drift is opt-in and never a verdict on its own.

Limits

• The threshold is calibrated, not portable. It belongs to the EmbeddingGemma space and to a calibration set. Do not reuse a threshold measured for another model: the cosine range is different. Calibrate with scripts/olhos.sh calibrar --positivos <file.jsonl> --negativos <file.jsonl> and put the result in the target's limiar.json.
• Precision over recall. Zero false positives on the calibration and validation sets, at the cost of false negatives. If an expectation is true but scores below the threshold, the verdict is "not found".
• Whole-screen drift does not separate cosmetic from meaningful change. It is opt-in in the tool and should be read next to the pixel baseline, never alone.
• It does not replace a fine visual check. It catches gross changes and missing expected elements. It does not see 2 px overlaps, contrast problems, overflow or clipped text. Use a dedicated screenshot diff or visual inspection for that.
• Screenshots only. The engine truncates audio and video at about 30 s; that does not affect screen captures, but the same index is not a media index.
• /olhos achar reads an index, it does not build one. The command expects a collection named olhos-telas-gemma-v1 in the engine's indices/ directory. Building that index is outside this plugin.
• Cost. The model loads once per process (about 11 s) and each image costs about 5.6 s on CPU. A two-expectation verification was measured at 2 to 5 minutes of wall clock under load. Scores are deterministic for the same capture and model.
• Nothing is automatic. The plugin captures only when the tool or the command is called. There is no background timer.

Troubleshooting

• motor EmbeddingGemma não encontrado: set OLHOS_GEMMA_RAIZ to a checkout with venv/bin/python, src/ and modelos/, then restart the harness.
• ModuleNotFoundError: websocket: python3 -m pip install websocket-client.
• Capture is blank or fails: check that Chrome is listening on http://127.0.0.1:9222, or set navegador: proprio to let the plugin launch its own browser.
• Plugin installed but nothing appears: restart the harness; the bundle layer is read at boot, not at install time.

License

MIT