Dantwz7/dsh-doc-router

dsh-doc-router

English | 中文

Route before you read. A DeepSeek Harness

plugin that inspects a document's *real* format and a PDF's *page layout*, then hands it

to the extraction pipeline it deserves — converting multi-column papers with layout

awareness instead of shredding them.

Why · What you get · Install · Quick start · How the routing works · Configuration · Evidence · Troubleshooting · Development · Roadmap · License

Why this exists

Default converters get multi-column papers wrong, and the failure is not subtle. On one Nature Letter (nmat4749.pdf, two columns, 7 pages), measured with PyMuPDF 1.28.2 / MarkItDown 0.2.0:

Pipeline · Output · Multi-column handling pymupdf4llm (layout-aware) · 6 533 chars for page 1 · ✅ columns merged in reading order; headings → #, superscripts → `, _in situ_ italics kept pypdf · 30 632 chars, whole document · ⚠️ text is readable but **structure is gone** MarkItDown · 45 266 chars, whole document · ❌ body text fragmented into hundreds of \ · … \ · ` table rows

The same failure is reproducible from a clone. On this repository's and three-column.pdf — synthetic pages whose columns are individually marked — MarkItDown 0.1.5 reads straight down every column at once:

Fixture · pdf_markdown · MarkItDown 0.1.5 two-column-reading-order.pdf · L01…L15, then R01…R15 · L01 R01 L02 R02 … — 29 column alternations three-column.pdf · L01…L15, M01…M15, then R01…R15 · L01 M01 R01 L02 … — 44 column alternations

That is the whole argument in one line: one pipeline keeps the columns, the other interleaves them. npm test asserts the reading order in both pdf_markdown cells.

Asking a model to "just pick a converter" therefore produces quietly mangled source material, and the damage looks like success. The fix is not a better single converter — it is routing: look at the file first, then choose the pipeline.

What you get

Artifact · Purpose `doc_route` tool · Classifies a file and names the pipeline to use: format, page count, whether a text layer exists, single- vs multi-column, plus the per-page evidence behind the verdict `pdf_markdown` tool · Layout-aware PDF → Markdown via PyMuPDF; supports a page range and writing to disk `doc-routing` skill · Registered into the runtime skill layer, so it applies to every workspace without a per-workspace skills/ directory. Tells the model to route before reading

Every verdict is deterministic — no model call, no network, no token cost.

Install

Requirements

· DSH · >=0.2.0-rc.1 =0.2.1-rc.1 =24.0.0 Python · 3.9+ — needed only for PDF layout detection and pdf_markdown PyMuPDF · pip install -U pymupdf pymupdf4llm — optional; see license

The interpreter is looked up in this order, first hit wins:

1. config.pythonPath
2. the DOC_ROUTER_PYTHON environment variable
3. the runtime DSH ships
4. a runtime under ~/.dsh/dsh-runtimes
5. python on PATH

An explicit choice is taken verbatim, so a bare command name like python3 is allowed — and a wrong path is reported as *the path you named*, not as a mystery.

From npm (recommended)

plugin_manager action=install_bundle target="dsh-doc-router@0.6.1"

Pin the exact version. DSH's package manager applies a minimum-release-age policy, and installing by bare name can silently resolve to an older build. Restart DSH afterwards.

From the DSH CLI

dsh plugin --profile <YOUR-PROFILE> add dsh-doc-router@0.6.1

must be the profile DSH **actually boots**. plugin_manager` above targets the active profile for you, so prefer it if you are unsure. Two things this subcommand does *not* do:

• A wrong `--profile` name is not an error — it creates one. The profile is

initialized as a new, empty profile and the package is installed *there*, so the plugin never reaches the profile you are running. The active profile is the directory under $DSH_HOME/profiles whose package.json lists your plugins.

• `--help` is not a dry run. dsh plugin --profile --help initializes

that profile before printing pnpm's help.

From a checkout instead

plugin_manager action=install_bundle target="file:<ABSOLUTE-PATH-TO-REPO>"

A file: install is a copy, so editing the source does not take effect: bump the version, remove the bundle, reinstall and restart DSH. See CONTRIBUTING.md.

Enable layout detection

<the python the plugin resolved> -m pip install -U pymupdf pymupdf4llm

Without PyMuPDF the plugin still loads and doc_route still classifies by format; only layout detection and pdf_markdown are disabled, and both say so rather than failing obscurely.

Quick start

Route first, then convert. The examples below use this repository's own synthetic fixtures, so every line is reproducible from a clone.

1. Route it

doc_route({ path: "tests/fixtures/two-column.pdf" })

→ Routed tests/fixtures/two-column.pdf — format: pdf, pages: 1, text layer: yes, columns: multi-column (est 2)
  Recommended pipeline: pdf_markdown
  Multi-column body text (est. 2 columns) — markitdown shreds it into table fragments and pypdf drops the structure
  If formulas, super/subscripts or digits must be exact, render that page to an image and check it
  Probe detail: {"1":"0.91/2col"}

  dsh-doc-router v0.6.1

The two advisory lines above are English because that is the default as of

0.4.0 — see Configuration for noteLanguage: 'zh'. The

verdict line, recommend, and probe fields are language-neutral either way.

2. Convert it

pdf_markdown({ path: "tests/fixtures/two-column.pdf" })

→ Converted tests/fixtures/two-column.pdf (all 1 pages) — 3391 characters of Markdown. …

pdf_markdown({ path: "tests/fixtures/two-column.pdf", pages: "1", output: "paper.md" })

→ Converted tests/fixtures/two-column.pdf (pages 1) — wrote 3391 characters to paper.md.

Both tools accept workspace-relative paths. When inline output would exceed maxChars, pdf_markdown truncates and says so instead of flooding the context — pass output to get the whole document on disk.

Real verdicts, from this repository's fixtures

Every row below was produced by running doc_route on the checked-in fixture:

Fixture · Verdict · Pipeline two-column.pdf · pdf · 1 page · text layer · multi-column (est 2) · pdf_markdown three-column.pdf · pdf · 1 page · text layer · multi-column (est 3) · pdf_markdown two-column-watermark.pdf · pdf · 1 page · text layer · multi-column (est 2 — not 3) · pdf_markdown unknown-columns.pdf · pdf · 3 pages · text layer · unknown · pdf_markdown (safe side) dense-script-column.pdf · pdf · 1 page · text layer · multi-column (est 2) · pdf_markdown single-column.pdf · pdf · 1 page · text layer · single-column · markitdown → pypdf scanned.pdf · pdf · 1 page · no text layer · render_to_png → read_image sample.docx · docx · markitdown sample.png · png · read_image sample.txt · text · read

pdf_markdown refuses non-PDFs and names the pipeline to use instead:

pdf_markdown({ path: "tests/fixtures/sample.docx" })
→ pdf_markdown only converts PDFs — detected docx. Recommended pipeline: markitdown
  (run doc_route for the full verdict)

[!NOTE]

Advisory language. The verdict line, recommend, and probe fields are

language-neutral. Every *human- or model-facing string* follows

noteLanguage — the free-text advisory lines after the

verdict, pdf_markdown's refusal message, and the doc-routing skill body.

en by default since 0.4.0, zh for the previous behaviour.

How the routing works

All decisions are deterministic: zero model calls, zero network, zero tokens.

Decision · Basis Format · The file header's magic bytes, never the extension. ZIP containers are opened to tell docx / xlsx / pptx / epub / odt apart; BOM'd and BOM-less UTF-16/UTF-32 plain text is decoded, with a control-character guard so a binary is not mistaken for text Text layer · Fewer than 120 characters per page ⇒ treated as a scan Single vs multi-column · Text intervals are merged per horizontal band and the segments separated by *internal* whitespace are counted. Any body page that is multi-column makes the document multi-column. Edge artefacts — a publisher's rotated margin stamp, headers, footers — are excluded: they are narrow but run the full page height, so they land in *every* band

That last rule is a deliberate asymmetric bet: pymupdf4llm works fine on single-column pages too, whereas a missed multi-column page is handed to a converter that destroys the body text. One mistake is cheap; the other is not.

The same bet decides what happens when the column count cannot be measured — a page whose body is one large block, or many fragments too short to be body text, yields no usable band evidence. That is reported honestly as unknown and routed to pdf_markdown, and is never described as single-column: guessing "single column" is the expensive direction, so a file we cannot measure takes the cheap one.

"Too short to be body text" is measured in content, not codepoints. A block is discarded only below 25 *latin-equivalent* codepoints, where an ideograph counts as two — because a Chinese word is 1–2 ideographs while an English word is ~5 letters. Counting raw codepoints instead made the filter several times more aggressive on Chinese PDFs, and could throw away an entire column of real body text; see dense-script-column.pdf.

The routes

Verdict · Pipeline multi-column + text layer · pdf_markdown single-column + text layer · markitdown, falling back to pypdf column count unmeasurable (unknown) + text layer · pdf_markdown — the safe side no text layer · render to PNG → read_image docx / xlsx / pptx / epub / odt / csv / html / json / xml · markitdown images · read_image text / rtf · read — no conversion at all

Why multi-modal reading is not on the normal path

Format and column count are geometry problems. Code is fast, exact and free; a single 150 dpi page image is roughly 0.5 MB and a thousand tokens, and must be rendered page by page. Images are therefore reserved for the three things code cannot do:

1. No text layer — a scan.
2. Exactly verifying formulas, superscripts, Greek letters and units — text

extraction gets these wrong in ways that matter (an image shows Sr₃Al₂O₆ correctly; text extraction can only give Sr3Al2O6).

1. Understanding figures, layout, or which caption belongs to which figure.

Three routing bugs worth not reintroducing

1. "Count the text blocks on each side" → on a Nature first page the left column

held only 2 blocks, so it was misread as single-column. Fixed by finding gutters per line instead.

1. "Only look for the gutter down the middle of the page" → a 3-column Science

layout has body text in the middle, so it was missed. Fixed by counting columns.

1. "Take the majority across pages" → a single-column reference page and a figure

page tied, tipping the verdict to single-column. Fixed to *any body page multi-column ⇒ multi-column*, ignoring pages under 300 characters.