Roarpeng/GraphFlow

GraphFlow

English | 简体中文

The Token-First Surgical Code Engine for AI Coding Agents (v3.7.0)

An MCP server exposing exactly 4 surgical tools — gf_read, gf_edit, gf_exec, gf_macro — with true AST parsing for all six language families: TypeScript/JavaScript via @babel/parser, and Python, Go, Rust, C/C++, C# via optional tree-sitter grammars (the built-in structural analyzers remain the fallback). A self-written semantic index resolves call targets for blast radius/rename (no extra runtime deps), and precision-tuned doc↔code linking (BM25/IDF + PMI + caps + confidence) keeps the knowledge graph honest. Six-language parity is regression-tested (51-case matrix), bilingual retrieval scores 1.000 EN / 1.000 ZH on the isolated + packed 30+30 pair corpus, and local task reflection replays recorded shell instructions — by explicit record or by auto-harvesting an external instruction log — with no LLM/network calls. The knowledge graph rebuilds incrementally per file (only dirty files are recomputed, output identical to a full rebuild) and parsed ASTs are cached on disk, so a warm gf_read on a 274-file repo drops from 4.4 s to 1.4 s. Runtime dependencies: @babel/parser, @modelcontextprotocol/sdk and tree-sitter; the grammar packages are optional.

GraphFlow 3.0 — The Radical Rewrite

Version 3.0 replaced the entire 2.x architecture. The 10-tool knowledge-graph / context-compression MCP surface (graphflow_context, graphflow_plan, graphflow_run, graphflow_index, graphflow_insight, graphflow_skill_*, graphflow_diagnose, graphflow_artifact), the episodic-skill flywheel, the VS Code graph-memory extension, and LLM routing were all removed. In their place: 4 deterministic surgical tools over an AST/structural pipeline. MCP schema tax dropped 92.2% (5897 → 458 tokens per turn).

2.x (removed) · 3.x replacement graphflow_context(...) · gf_read({ targets, q, skeleton: true }) graphflow_plan / graphflow_run · Removed — deterministic pipeline, no LLM routing graphflow_index · Removed — the symbol index is built on demand graphflow_insight / graphflow_skill_* · gf_macro({ action: "mine" \ · "mine-git" }) graphflow_diagnose · graphflow status (CLI) graphflow_artifact · gf_macro({ action: "visualize" }) → .graphflow/knowledge-graph.html GRAPHFLOW_MCP_PROFILE env · Removed — the 4 gf_* tools are the only surface VS Code graph-memory extension · Thin prebuilt launcher (see VSIX-INSTALL.md) Dialogue / workbench memory features · Removed with the flywheel; gf_read({ targets: ["@brief"] }) covers cross-session handoff

**2.x users must switch tool calls graphflow_* → gf_*.** There is no compatibility layer.

Version · Notes 2.3.0 · Last legacy release (10-tool knowledge-graph MCP) 3.0.0 · Radical rewrite — breaking, 4 surgical tools 3.1.0 · 21-agent installer, 3-strike circuit breaker, git co-change miner, struct-field contract guard 3.1.1 · Thin prebuilt VS Code extension, VSIX on GitHub Releases 3.2.0 · Doc-graph (cross-agent brief, doc↔code linking, HTML knowledge graph) + bilingual tokenizer 3.2.1 · Fix pack: affected-tests honors package.json scripts.test, gf_exec run:true execute-vs-preview gating, brief hub in-degree for extension-ful imports 3.3.0 · Auto-register on VSIX activate (once per version, `graphflow.autoInstall` opt-out) + legacy registration prune on `install` (alias keys, stale launchers, removed-agent leftovers; `--no-prune` to skip) 3.3.1 · Method-aware blast radius & rename (member call sites, abort on unresolved callers) · real DSH Cordis-patch install (`cwd` + `GRAPHFLOW_WORKSPACE_ROOT`) · safe scoped `uninstall` (`--agents` / `--all`) · Python heuristic syntax guard · `exec` argument parsing · failure source window after `Error:` + stack frame · Go/Rust `struct` kind 3.3.2 · Real knowledge-graph edges (defines/imports/calls/doc-links-code/co-edit, class-qualified method ids, 5000-edge cap) · receiver-type-aware blast radius & rename (A/B same-name disambiguation, `REJECTED [ambiguous-symbol]`) · tsconfig/jsconfig alias + dynamic-import aware affected-test routing 3.3.3 · Type-aware knowledge-graph calls (receiver-matched targets, foreign receivers dropped, alias-resolved imports) · inheritance/`super`/`base` receiver inference (subclass→base dispatch, last-assignment-wins, generic/array/nullable unwrapping) · tsconfig/jsconfig `extends` chains, nested configs and project references for affected-test routing · `npm run bench` harness 3.3.4 · Safe host-hook merge on `install` (user `PreToolUse`/`PostToolUse` hooks preserved; idempotent reinstall) · `uninstall` removes only GraphFlow hooks · `graphflow status` per-agent registration state (`[MCP ✓] [Skill ✓] [Rule -] [Hook ✓]`) with `--agents` / `--all` · MCP stdio end-to-end test (initialize → tools/list 4 tools → gf_read/gf_exec → graceful unknown tool) 3.4.0 · True AST by default for Python/Go/Rust/C/C++/C# via optional tree-sitter grammars (`engine: "tree-sitter"`); structural fallback when bindings are missing, `GRAPHFLOW_TREE_SITTER=0` forces structural · C/C++ header/prototype handling and Python `super` receiver · dependency story: required `tree-sitter` core + optional grammar packages · refreshed benchmark measurements 3.5.0 · Doc↔code link precision (self-written BM25/IDF + PMI + reciprocal pruning + caps, `GRAPHFLOW_DOC_LINKS_MAX`; repo 728 → ≈150 edges, −80%) · `graphflow doctor parsers` + `npm run verify:parsers` + additive CI parser matrix · self-written L0 semantic index (confidence ≥0.75) with optional TypeScript/LSP providers · DSH config bundle `@roarpeng/graphflow-dsh` · release preflight (`npm run preflight` / `preflight:release`) · refreshed benchmarks 3.6.0 · 48-case six-language parity matrix + fixes (family-scoped `resolveMethod`/rename, Go embedding/grouped params, C typedef fields, Rust `&dyn`/`impl` unwrap, C/C++ header prototype↔definition rename) · bilingual recall 1.000 EN / 1.000 ZH on 30 pairs (pinyin/mixed/synonym 1.000; baseline ZH@1 0.967, pinyin 0.769) · local task reflection (`graphflow task`/`reflect`, `gf_macro` task-list/run/record) · Windows CRLF test fix; CI (6 jobs) + parser matrix (7 jobs) green · 458-token wire · 24 test files 3.6.1 · Six-language parity 51 tests: Rust trait dispatch CLOSED (trait methods with `parentClass`=trait; `dyn Trait`/`impl Trait` receivers resolve to the trait method, concrete receivers to the impl), C# `record`/`record struct` kinds + positional fields + namespace map CLOSED (Go same-package imports stays an honest gap) · packed-file bilingual corpus: packed EN 1.000 / ZH 1.000 @1/@3/@5 (ZH@1 was 0.900; ±7-line own-doc bleed fixed) · task auto-harvest (`.graphflow/instruction-log.jsonl`, `task log`/`task harvest`, `gf_macro` `task-harvest`; external writer needed) · 458-token wire · 24 test files 3.7.0 · Chinese→English recall outside the glossary: L1 PMI map from comments/commits, L2 curated colloquial lexicon (optional local ECDICT), L3 translator hook off by default · adaptive `gf_read` search budget · true incremental knowledge-graph rebuild (per-file contributions + shard manifest) · AST index cached on disk · rename no longer rewrites import paths · held-out 80 queries P@1 0.000 → 0.813 · `ts-zh-search` 3198 → 1077 tokens · wire 459 tokens · 39 test files 3.6.2 · Eval fix pack: class-qualified rename (`OrderService.cancel`) plus override-hierarchy `super` updates · Rust macro `impl Trait` blast · doc↔code constants / generic-token / test-symbol cap · `gf_read` source-size budget (`GRAPHFLOW_READ_BUDGET_RATIO`) · automatic instruction log from the PostToolUse hook and `gf_exec` `run:true` · MISS Jaccard + structural task-run `exitCode` · wire 459 tokens · 25 test files

The Four Surgical Tools

Tool · Purpose · Key arguments gf_read · Symbol slices with cross-file type closure + receiver-type-aware [Blast Radius] call sites (semantic index first, then inheritance/super heuristics), skeleton folding, bilingual search (1.000 EN/ZH parity recall), @brief cross-agent handoff · targets, q, topK, skeleton, dedup gf_edit · In-place / batch atomic edits, zero-LLM cross-repo rename (semantic target check first, receiver/inheritance heuristics as fallback; family-scoped staging, rejects ambiguous/foreign-receiver files; aborts if unresolved callers remain), struct-field contract guard · file, symbol, newCode, edits, renameTo gf_exec · Diff-to-test routing (honors package.json scripts.test; resolves tsconfig/jsconfig aliases, extends chains, nested configs and dynamic imports) + output dehydrator: passing suites → 1-line PASS; failures inline up to 5 lines of source context; run: true executes, otherwise preview · cmd, changedFiles, handle, run gf_macro · Habit macros, git co-change mining, knowledge-graph HTML export (type-aware calls, alias-resolved imports, BM25/IDF-capped doc-links-code; plus defines / co-edit edges), and local task reflection (task-list / task-run / task-record / task-harvest) · action: "list" \ · "run" \ · "save" \ · "mine" \ · "mine-git" \ · "visualize" \ · "brief" \ · "task-list" \ · "task-run" \ · "task-record" \ · "task-harvest"

Session read dedup skips unchanged files; 3 consecutive edit-test failures trigger a deterministic rescue packet (blast-radius contracts + recent git diffs). Syntax guards use the true-AST backends when available — @babel/parser for TS/JS and tree-sitter ERROR/MISSING diagnostics for the other five — with the structural analyzers' heuristic checks (brace/declaration, Python missing-colon/bracket diagnostics) as the fallback, including when GRAPHFLOW_TREE_SITTER=0 is set.

What's New in 3.7.0

• Uncovered Chinese still finds English identifiers — L0 pinyin, synonyms and the engineering glossary are unchanged, so the saturated 30+30 parity set stays 1.000. L1, at index time, pairs Chinese n-grams in comments and git log subjects with nearby identifier sub-tokens, keeps the high-PMI pairs (support 1 only when that English token does not also appear in other Chinese windows), and caches .graphflow/zh-identifier-map.json. A warm lookup is about 0.005 ms. L2 is an original colloquial programming list shipped with GraphFlow (Apache-2.0, not a copy of CC-CEDICT or ECDICT) and then filtered to English tokens that actually occur in the repo, so the cache stays well under 1 MB. A local ECDICT TSV (MIT; GRAPHFLOW_ECDICT or .graphflow/ecdict.csv) is optional and is not bundled. L3 is registerZhEnModel; it runs only when GRAPHFLOW_ZH_EN_MODEL=1. No model package is added. GRAPHFLOW_ZH_RECALL=0 / GRAPHFLOW_ZH_L1=0 / GRAPHFLOW_ZH_L2=0 turn layers off.
• Held-out set, 80 phrasings absent from L0 — baseline (L0 only) P@1 0.000 / MRR 0.000 / R@3 0.000. +L1: 0.375 / 0.375 / 0.375 (30 commit-taught queries go from 0 to 1.000). +L1+L2: 0.813 / 0.813 / 0.813 (35 colloquial queries go from 0 to 1.000). The other 15 phrasings stay at 0 — there is no comment, commit, or curated entry for them. L3 was not scored as a neural model; with the hook registered, one of those misses resolves. A shorter L1 n-gram can distract (搞定发货 → a commit that only taught 发货) until the longer L2 phrase outranks it.
• Adaptive search budget — when topK is omitted, a margin of at least 0.45 between the top two scores returns one slice; closer scores stay as a cluster of at most 3 (so 订单超时 still returns both the handler and OrderService.timeout). Those default slices skip type closure and blast radius. Passing topK keeps the previous full slice. On the 80 held-out queries the token sum drops 22157 → 4849 while adaptive P@1 stays 0.813. CLI ts-zh-search (three queries, ceil(chars/4)) drops 3198 → 1077 against a 1245-token file baseline; the required hits remain.
• True incremental knowledge-graph rebuild — the graph is now the sum of per-file contributions instead of one monolithic pass. Every edge has a single owner file (defines / imports / calls → the declaring / importing / calling file, doc-links-code → the code file), so buildKnowledgeGraphData splits into planKgRebuild + executeKgRebuild and only dirty files are recomputed. Invalidation is three-level, cheapest first: (1) fingerprint size:mtimeMs; (2) name-level reverse dependency — every file records the symbol names it queried (queriedNames) and a changed file's deltaNames is intersected with it, which is far more precise than pulling the whole import fan-in closure; (3) structural — class-parent changes fold the involved class names into deltaNames, and a file add/remove only touches files with unresolvedImports. A shard manifest (.graphflow/graph-shards.json) also stores each file's symbol table, so untouched files never even load their AST. A missing or corrupt manifest degrades to a full rebuild.
• AST index cached on disk, and the real hot paths fixed — the slicer's parse results now persist to .graphflow/ast-cache/.json, keyed by content hash plus a backend signature, so flipping GRAPHFLOW_TREE_SITTER invalidates correctly. GRAPHFLOW_AST_CACHE=0 turns it off. --cpu-prof also disproved the obvious hypothesis: parsing is not the bottleneck (274 files parse in ~950 ms), tokenizeEnglish (~46%) and normalizeWorkspaceRelPath (~43%) are. Both now use bounded memoization, and the fuzzy glossary scan is bucketed by first letter. Measured on axios (274 files): warm gf_read 4.4 s → 1.4 s, cold graph 16.1 s → 5.6 s.
• Doc↔code stores collapsed — the 274 per-file doc-link shards became three single-file stores (.graphflow/doc-links/symbols.json, concepts.json, sections.json): on Windows symbol-data load went 1000 ms → 121 ms and the document scan 920 ms → 49 ms. Candidate caches are keyed by the in-file ordinal (so) instead of a flat index, so inserting symbols earlier in a file no longer invalidates every later entry. The reciprocal top-K prune still runs globally per build, so the result is bit-for-bit what a full rebuild would produce.
• Fix pack (correctness) — gf_edit rename no longer rewrites import paths: comments and string/template literals are masked with equal-length placeholders first, so renaming a symbol can no longer turn ./combineURLs.js into ./joinURLs.js and break the module with ERR_MODULE_NOT_FOUND. The read-budget shrink order now prioritizes digestSymbolBlocks, so small functions are not folded away. Affected-test routing recognizes tests/ / __tests__/ / spec/ directories and skips pseudo-tests such as conftest.py. MCP tools/call on an unknown tool returns isError: true, and gf_exec surfaces the compiled local tool id through findCommandMacro(). CLI exit codes propagate, install / uninstall render their errors instead of swallowing them, and uninstall only removes GraphFlow-owned hooks and MCP rows.
• Version/state — 3.7.0; MCP surface stays 4 tools. Wire schema is unchanged at 459 tokens (1834 chars, ceil(chars/4)). The suite is 39 test files (10 new tests/fix-*.test.ts files). CI (6 jobs) + parser matrix (7 jobs) are unchanged.

What's New in 3.6.2

• Qualified rename follows overrides — gf_edit / graphflow edit --symbol OrderService.cancel --rename cancelOrder resolves class-qualified ids the same way gf_read accepts file#Class.method. A subclass override (VipOrderService.cancel, PartialRefundPolicy.refund) is no longer an ambiguous-symbol reject: the override declaration stays, and only the base call (super.cancel, super().refund) is rewritten. Foreign receivers (RefundService.cancel, settle_partial), import module paths (from billing.refund), and block-comment mentions stay untouched. new VipOrderService().cancel() resolves the constructed class, so the override call is not rewritten. Same-file duplicates and unrelated same-named classes are still rejected.
• Rust `impl Trait` calls inside macros — blast radius now sees receiver.method() tokens nested in macro_invocation bodies (for example describe(d: impl Device) { format!(..., d.read()) }). Direct &dyn Trait calls are unchanged. Trait+impl atomic rename remains a safe reject.
• Doc↔code links — backtick identifiers are no longer split, so ` OrderService.timeout does not steal a cap slot from TIMEOUT_WINDOW_MS. Explicit identifier mentions are exempt from the per-symbol cap of 3 (the per-section cap of 3 and the global cap still apply), so EVAL/README mentions of the same symbols cannot crowd out the section that defines them. Generic tokens 买家 / 卖家 / 会员 / 客服 (and their pinyin) are stop-listed, pinyin echoes of the same word count once, and concept-only links skip test symbols (setUp, tests/, test_*.py`).
• Read budget — when a gf_read response would be larger than the raw source of the primary files it sliced or searched (blast/closure neighbors are not added to the budget), the slicer compacts in place and does not hard-cut. It shortens blast snippets and symbol/closure bodies, drops duplicate closures and closures of symbols already shown, drops numbered blast lines that are not super/base, drops EVAL.md / README / results/ quotes, and keeps at most three // -> arrows (always in Hold() / in Available() / in Reservation() when present) plus the super/base call line. GRAPHFLOW_READ_BUDGET_RATIO defaults to 1; 0 disables the guard. Under budget, linked-doc lines and skeleton markers stay.
• Automatic instruction log — the graphflow install PostToolUse hook and gf_exec with run: true append .graphflow/instruction-log.jsonl and harvest it, so the second identical command promotes a local task without an external writer. GRAPHFLOW_AUTO_LOG=0 turns this off. Noise commands (echo, git status, …) are skipped. MISS text includes (jaccard=…), and MCP task-run returns structuredContent.exitCode (the tools/list schema is unchanged).
• Version/state — 3.6.2; MCP surface stays 4 tools. Measured wire schema is 459 tokens (1834 chars, ceil(chars/4); schema unchanged, still under the 460-token cap; 3.6.1 documented 458). The suite is 25 test files (tests/eval-regressions.test.ts). CI (6 jobs) + parser matrix (7 jobs) are unchanged.

What's New in 3.6.1

• Parity suite 48 → 51 tests — tests/language-parity.test.ts now closes two documented gaps: Rust trait dispatch (trait methods are emitted as methods with parentClass = trait name, so dyn Trait / impl Trait receivers resolve to the trait method and concrete receivers to the impl; cross-file trait/impl assertions plus structural-fallback trait scope) and C# record kinds (record → class, record struct → struct, positional record parameters exposed as fields). The self-written semantic index now builds a C# namespace → files map used for using/import, symbol and method-candidate resolution (with a file-stem fallback). Go same-package imports remains an honest gap: same-package sibling files still have no file-level imports edge (no invented edges; package-scope calls/edges resolve). Remaining residuals: trait+impl atomic rename is rejected safely, impl Trait for Type is not modelled as subclassing, ambiguous doc-graph KG calls now resolve through the semantic index's namespace → files map (confidence ≥0.75) — e.g. using Demo.Two; g.Cancel() targets two/Gadget.cs#Gadget.Cancel deterministically instead of one/Gadget.cs; the only residual there is the one-time cold semantic-index build on a first ambiguous workspace (~+120 ms, then cached) and the GRAPHFLOW_KG_SEMANTIC=0 escape hatch; C is function-level, Go only models the first embedded type, rename stays conservative, and C++ prototype targeting is best-effort.
• Packed-file bilingual corpus — the bilingual parity suite now adds a packed-file corpus (6 symbols per file, Chinese-only docstrings) beside the isolated 30-pair corpus. Packed EN 1.000 / ZH 1.000 at @1/@3/@5 (packed ZH@1 was 0.900 before the fix); own-doc-comment extraction no longer uses a fixed ±7-line window that bleeds into adjacent symbols, plus term caps/normalization. Isolated one-symbol, pinyin, mixed and synonym queries remain 1.000. Residuals unchanged: generic English still cannot disambiguate pinyin-named symbols at @1, and the glossary is curated.
• Task auto-harvest (still no LLM/network) — reflection can now be fed by an external instruction log: .graphflow/instruction-log.jsonl written with graphflow task log "" --step "" (append-only; --log override), then graphflow task harvest [--log ] promotes repeated normalized instructions into the same .graphflow/tasks.json + compiled .graphflow/tasks/.mjs runners. MCP equivalent: gf_macro({ action: "task-harvest", instruction, steps }) or a bare task-harvest to harvest the log. Harvesting is incremental via a byte-offset state file (.graphflow/task-harvest.json); the explicit graphflow task record path is unchanged and graphflow reflect still returns MISS when nothing matches. Limits: an external writer is still needed (shell wrapper / DSH skill appends to the log — there is no host hook or daemon), steps run locally with the recorder's privileges, {{argN}} parameters are manual, and concurrent records are last-writer-wins.
• Version/state — 3.6.1; MCP wire stays 458 tokens, the suite is 24 test files, CI (6 jobs) + parser matrix (7 jobs) remain green, and the benchmark numbers are the final 3.6.0 run (unchanged).

What's New in 3.6.0

• Six-language parity matrix & fixes — new tests/language-parity.test.ts (48 tests) asserts symbol/span/field parity for TS/JS, Python, Go, Rust, C/C++, C# across slicing + type closure, receiver-aware blast radius, cross-file rename, and knowledge-graph defines/imports/calls. Fixes: language-family-scoped resolveMethod/rename staging (no cross-language leakage), Go struct embedding + grouped params + embedded fields, C typedef struct fields, Rust &dyn/impl receiver unwrap, and the C/C++ header prototype↔definition rename cluster. Documented residuals: Go same-package siblings have no file imports edge; Rust trait declarations are not emitted as method symbols (trait dispatch yields 0, never a wrong target); C# record maps to interface kind; C is function-level (no inheritance); deep/multiple Go embedding is not chained; the structural fallback is not part of the parity matrix. (3.6.1 raises this suite to 51 tests and closes the Rust trait-dispatch and C# record gaps; Go same-package imports remains the honest gap — see above.)
• Bilingual recall 1.000 / 1.000 — new tests/bilingual-parity.test.ts measures 30 EN/ZH query pairs: EN and ZH recall are both 1.000 at @1/@3/@5, and pinyin / mixed / synonym families are 1.000 (baseline ZH@1 0.967, pinyin 0.769). Tokenizer fixes: morphological-rescue asymmetry, synonym forward-expansion pollution, a missing zh/pinyin glo