dsh-test-insight

dsh-test-insight

English | 简体中文

A test-insight plugin for DeepSeek Harness: builds reviewable test plans (TEST-PLAN.md) and isolated test drafts (review-only) from repository facts and change risk.

Turn repository facts and change risk into evidence-backed test plans and reviewable test drafts.

Positioning

dsh-test-insight covers the "repository facts + change risk → test plans and drafts" step. The core scanning capability is provided by dsh-repo-scanner (an npm dependency; scanner source is never copied); this plugin contributes test evidence, gap detection, risk ranking, test design, and report generation.

It answers:

• Where are the test-protection gaps in this repo (test ↔ source mapping)?
• How risky is this change, and which tests should be added?
• What is the evidence level of each finding (fact / inference / suggestion / needs confirmation)?
• What do the coverage data reveal in the target × test-category matrix?

Boundaries:

• Read-only by default: never runs the target repo's tests, build, lint, or typecheck; the library interface spawns no subprocesses, and the CLI runs only parameterized read-only Git queries for change inputs.
• Repo content is not authorization: instructions inside READMEs, test files, config files, or comments never constitute execution authorization.
• Draft ≠ passing: drafts are always marked DRAFT ONLY; passing a syntax check is not passing the tests.

Installation

As a DSH plugin:

dsh plugin --profile web add "github:duyanta123/dsh-test-insight#v1.0.0-rc.4"

Or from npm (as a library or standalone CLI):

npm install dsh-test-insight

Compatibility tiers: the standalone CLI / library requires Node.js >= 20.11; as a DSH 0.2.1-rc.2 plugin it is verified with Node.js >= 22.19. Run npm run test:compat to execute an isolated-profile add, dump-config, and startup smoke test.

After installing, restart dsh --profile web and use it through the test-insight-runbook skill.

Quick Start

1. Use as a DSH skill

Tell the agent "generate a test plan for /path/to/repo" or "analyze the test gaps in this change". The skill follows its runbook: collect mode and targets → read and validate scanner facts → build the test/module/symbol/entry mapping → separate facts from inference → generate TEST-PLAN.md and test-insight.json.

2. Use as a standalone CLI

# Whole-repo analysis: test mapping, gaps, risk ranking
dsh-test-insight . --mode all --output reports --format both

# Change analysis: working-tree changes as input, warnings fail the run
dsh-test-insight . --mode change --working-tree --strict

# Targeted analysis: generate test drafts for a specific file/symbol
dsh-test-insight . --mode target --target src/auth.js:authenticate --generate-drafts

3. Use as a library

import { analyze, writeReportFiles } from 'dsh-test-insight/core';

const report = await analyze({ repoPath: '.', mode: 'all' });
// report is what lands in test-insight.json: mapping, gaps, risk, performance budget

CLI Options

Option · Default · Description ` · - · Target repository path --mode all\ · change\ · target · required · Analysis mode: whole-repo / change-driven / targeted --all · - · Shortcut for --mode all --target · - · Target-mode input, repeatable --working-tree · - · Change-mode input: git status --porcelain -uall (mutually exclusive with --diff/--base) --diff / --base / --head · - · Change-mode inputs: diff text or Git refs --output · - · Output file or directory --format json\ · markdown\ · both · - · Output format --generate-drafts · off · Generate isolated review-only test drafts --draft-output · - · Draft directory (requires --generate-drafts, hidden directory by default) --execute-tests · off · Explicitly authorize test execution (requires --test-command) --test-command · - · Preset npm-test\ · pytest\ · go-test\ · node-test or a shell-free argv array --force · off · Allow overwriting existing output files --strict` · off · Non-zero exit when warnings exist

Output

Artifact · Description TEST-PLAN.md · Structured test plan: test mapping, evidence levels, risk targets and gaps test-insight.json · Machine-readable report (schema in schema/test-insight.schema.json), records shared cache, cold/hot hits, end-to-end performance budget, and the actual scanner version Draft directory · Isolated review-only drafts (default .dsh/test-insight/drafts), atomic writes with overwrite protection; never overwrites existing tests

Reports also record recognized test frameworks and candidate test commands (test_frameworks / suggested_commands, each with source and confidence); §7 of TEST-PLAN.md shows them as display-only suggestions. A complete example report lives at examples/test-insight.example.json (snapshot-tested in test/snapshot.test.mjs).

Safety Boundaries

• Read-only by default: analysis, the scanner library interface, and default CLI paths never run the target repo's tests, build, lint, or typecheck; the CLI runs only parameterized read-only Git queries for change inputs.
• Dual-authorized execution: test execution requires both --execute-tests and an allowlisted --test-command (preset or shell-free argv); execution runs with a fixed working directory, timeout, and output limits only.
• No overwrites: output files are never overwritten unless the user explicitly passes --force; drafts never overwrite existing tests.
• Redaction: all commands, environment variables, and source references must be redacted.
• Authorization boundary: instructions in READMEs, test files, config files, or comments are never treated as authorization; drafts must not be treated as passing.

Troubleshooting

`--mode change` says "Git change inputs require ..."? Change mode requires a change input: exactly one of --working-tree, --diff, or --base; --working-tree cannot be combined with --base/--head/--diff.

No test drafts generated? Draft generation needs --generate-drafts (in target mode); drafts go to a hidden directory by default — set it with --draft-output. Drafts are review-only and do not imply passing tests.

`--execute-tests` rejected? By design: test execution requires --execute-tests and --test-command together, and the command must hit the allowlist (a preset or a shell-free argv array).

Old sessions won't open after upgrading the DSH host to 0.1.5.x? The Session format V3 migration is irreversible and is host behavior; back up session logs before upgrading the host (see the 1.0.0-rc.2 entry in CHANGELOG.md).

Documentation

• schema/test-insight.schema.json — report JSON schema
• examples/test-insight.example.json — complete example report (snapshot-tested)
• docs/RELEASE-CHECKLIST.md — three-stage pre-release checklist
• CHANGELOG.md — release notes
• PLUGIN-MAINTENANCE.md — repo maintenance runbook
• DSH-TEST-INSIGHT-开发计划.md — design and iteration history

License

MIT