JimchengChina/dsh-frontier-repro

dsh-frontier-repro

Primary signals in. Auditable reproduction out.

中文 · Download · Star the project

An evidence-first frontier AI radar and reproduction gate for DeepSeek Harness.

This is not another generic news reader, arXiv search engine, or paper summarizer. It covers the missing path from a curated primary-source signal to a versioned release evidence bundle and then to an auditable reproduction decision. Executed attempts are retained—including failures and negative results—and a successful exact/scaled claim requires commands, artifacts, measured claim results, and independent verifier evidence.

[!TIP] If this makes a frontier release easier to trust or reproduce, give the repository a star. It helps more DSH users find the project.

What is different

Existing DSH plugins already cover literature discovery/full text (dsh-ai4scholar, dsh-literature), guided paper reading and notes (dsh-paper-workshop), general RSS reading (dsh-news), scientific-claim adjudication (dsh-research-plugins), and replayable experiment execution (dsh-science-workbench). This plugin does not duplicate those surfaces. It adds:

• a single timeline across arXiv, first-party lab releases, official model artifacts, verified personal blogs, and opt-in X API timelines;
• explicit identity provenance for person sources;
• explainable ranking rather than an opaque quality label;
• a mode-specific evidence matrix for exact, scaled, and behavioral reproduction;
• a hard separation between “prerequisites documented” and “reproduction executed”;
• run evidence that rejects unsupported success claims.
• Hugging Face paper-to-artifact expansion and immutable GitHub commit evidence;
• immutable arXiv versions and Hugging Face model/dataset repository SHAs;
• persistent source health with volume, structure, failure-streak, and staleness alerts;
• conservative lab-scoped clustering into Frontier Release Evidence Bundles, with corroboration and explicit missing evidence;
• capability/evaluation/license diffs, firstSeenAt / lastSeenAt / supersedesDigest version chains, and substantive-only watchlists;
• a first-party GitHub organization adapter for new repositories, releases, tags, immutable commits, and release-asset digests;
• claim-level execute_existing, partial_reimplementation, and from_scratch_replication protocols;
• preserved multi-attempt resource records (GPU/CPU/VRAM/time/cost/data scale/job URL) and mandatory verifiers for success;
• a file-only Hugging Face Trackio logbook scaffold instead of a duplicate experiment UI;
• serialized, journaled collection batches with guarded LIFO rollback;
• an explicit evidence dependency graph and canonical reproduction handoff manifest.

See docs/research.md for the overlap audit and docs/architecture.md for the Cordis spatiotemporal mapping.

Corporate/personnel announcements, placeholder titles, missing-date records from date-contracted sources, and personal-life posts are filtered before persistence. Sources may declare category allow/deny lists and known boilerplate titles. Recent Hugging Face model cards are inspected only to discover paper, code, data, and evaluation links; full card text is not archived. arXiv records retain both a stable paper id and the observed immutable vN. GitHub repositories and public Hugging Face models/datasets are resolved to full SHAs, while mutable popularity counts remain context only.

Built-in sources

• arXiv categories cs.AI, cs.CL, cs.LG, cs.CV, cs.RO, and cs.SE through the official Atom API.
• OpenAI News; Anthropic Newsroom, Research, and Engineering; Google DeepMind News; MiniMax Research; Kimi Blog; DeepSeek API News and Transparency; and Z.ai model release notes.
• NVIDIA Technical Blog, AMD ROCm Blog, and the filtered Intel Artificial Intelligence News feed for training, inference, accelerator, compiler, and benchmark signals.
• Verified DeepSeek, Moonshot AI, MiniMax, and Z.ai Hugging Face organizations for model artifacts, plus the official DeepSeek GitHub organization for repository/release/tag changes.
• Sam Altman's blog and Jack Clark's Import AI.
• Sam Altman, Dario Amodei, and Demis Hassabis on X, through X API v2 only.

No unverified DeepSeek-founder or GLM-person account is preloaded. Add a custom person source only with a first-party identityEvidenceUrl.

Tools

Install

Requires Node.js ^22.19 or >=24 and DeepSeek Harness 0.1.0-rc.7 or newer within the 0.1.x line. The in-app X credential card uses the plugin-settings extension introduced in rc.7.

dsh plugin --profile web add "https://github.com/JimchengChina/dsh-frontier-repro/releases/latest/download/dsh-frontier-repro.tgz"

For source development, clone the repository, run pnpm install, and add its absolute path instead. Headless profiles can install the same release with --profile headless.

Restart the selected profile. The default corpus is $DSH_HOME/frontier-repro/index.json.

During installation, the plugin reminds you that X API access is optional. In DSH Web, open Settings → Plugins → Plugin configuration → Frontier Repro to save an X API bearer token. If you do not configure one, X sources are disabled and skipped by default; arXiv, official lab blogs, GitHub, Hugging Face, and all other sources continue normally.

Tagged releases attach a verified package tarball and SHA-256 file. Storefronts can use https://github.com/JimchengChina/dsh-frontier-repro/releases/latest/download/dsh-frontier-repro.tgz without running a source build.

X access

Use Settings → Plugins → Plugin configuration → Frontier Repro in DSH Web, set X_BEARER_TOKEN in the launch environment, or write the same reference through the DSH credentials store. The token is resolved for every collection and never enters config, the corpus, or tool output. Without X API access, default collection skips X sources and every other source continues. Explicitly requesting an X source still returns its exact missing condition.

No HTML scraping fallback is used.

Configuration

- id: frontier-repro
  config:
    defaultDays: 90
    defaultLimit: 20
    maxRecords: 1000
    maxCollections: 20
    requestTimeoutMs: 20000
    maxResponseBytes: 5242880
    pageConcurrency: 3
    githubEnrichLimit: 8
    huggingFaceEnrichLimit: 20
    githubTokenEnv: GITHUB_TOKEN
    xBearerTokenEnv: X_BEARER_TOKEN
    # storagePath: /absolute/path/index.json
    # sourceFile: /absolute/path/sources.json
    promptGuidance: true
    promptOrder: 145

sourceFile is trusted local administrator configuration. Tool arguments never accept arbitrary source URLs. Custom sources may declare allowCategories, denyCategories, requirePublishedAt, boilerplateTitles, and healthStaleAfterDays. A github_org source declares an organization plus bounded releaseRepoLimit / releasesPerRepo; person sources require a name, role, and first-party identity evidence. GITHUB_TOKEN is optional and only raises public API limits; X API access remains explicitly required for X sources. See sources.example.json and docs/source-policy.md.

P1 workflow: release bundle to verified attempt

1. Collect first-party signals, then select a release with frontier_repro_events / frontier_repro_bundle.
2. Watch it. Later collections set changed_since_watch only when capability claims, evaluation evidence, licenses, or immutable artifacts changed; acknowledge after review.
3. Define independently testable claims with primary evidence and choose execute_existing, partial_reimplementation, or from_scratch_replication, plus exact, scaled, or toy equivalence.
4. Record every attempt. passed requires all required claim metrics and a passing verifier with identity/evidence. A toy pass is always stored as toy_only, never reproduced.
5. Export the Trackio scaffold to visualize the preserved attempts locally or, only when intended, in a Hugging Face Space. The scaffold does not execute the experiment or change verdicts.

Verification

pnpm run verify
node scripts/smoke.mjs
pnpm run smoke:plugin
node scripts/smoke.mjs openai-news google-deepmind-news zai-release-notes

The tests cover parsing, enrichment, event clustering/versioning, GitHub organization releases, watch rollback safety, claim grading, multi-attempt/verifier enforcement, Trackio export, digests, lifecycle serialization, transaction rollback, evidence topology, manifests, storage, and registration/disposal through the real DSH ToolRuntime. smoke.mjs exercises source adapters directly; smoke:plugin runs collection through the real tool runtime. Both perform read-only requests and never call X.

Limitations

• First-party sites change. Source failures are isolated and reported; successful records still persist.
• Source-health alerts indicate collection drift or staleness, not that an upstream publication is false or abandoned.
• OpenAI RSS is fetchable while article pages may block automated clients. The feed record remains usable and a selected page can be verified with DSH's browser/web capability.
• A primary source proves provenance, not correctness. Ranking is discovery priority, not a truth score.
• X requires the user's own API entitlement and budget; the plugin does not bypass access controls.
• Anonymous GitHub API requests have low rate limits. Configure the optional token or reduce githubEnrichLimit; Hugging Face pinning can be bounded with huggingFaceEnrichLimit. A pinning failure is reported and does not erase the source record.
• Release clustering is deliberately conservative and lab-scoped. Unrecognized model names remain one-record bundles rather than being merged by vague semantic similarity.
• A Trackio scaffold is an export. The plugin does not install Python packages, run jobs, upload to Spaces, or treat dashboard logging as verification.
• The gate checks evidence completeness and consistency, not whether every submitted evidence statement is true.
• Manifest integrity is a deterministic content digest, not an identity signature or remote artifact checksum.
• Scheduling is deliberately left to existing DSH schedule/cron plugins.

License

MIT