dsh-llm-wiki

LLM Wiki

English | 中文

An Obsidian-style personal knowledge base for DeepSeek Harness, built on the Karpathy Wiki methodology.

LLM Wiki turns DeepSeek Harness into a lightweight Obsidian: a three-column workspace (sessions · chat · wiki) where you drop source files in, the LLM incrementally builds and maintains a persistent wiki, and you can then ask questions in the chat that are grounded in your knowledge base.

---

Why LLM Wiki (vs. plain RAG)

Most RAG re-derives knowledge from raw documents on every query — nothing accumulates. LLM Wiki instead keeps a persistent, compounding wiki of interlinked markdown pages between you and your sources. Each new source is read once and integrated: entity/concept pages are updated, cross-references maintained, contradictions flagged, and the index kept current.

• You curate sources and ask questions.
• The LLM does the summarizing, cross-referencing, filing, and bookkeeping.

Inspired by Karpathy's "LLM Wiki" pattern — the wiki is just a git repo of markdown files.

---

Features

• Three-column workspace — sessions on the left, chat in the middle, the wiki panel on the right.
• One-click workspace — pick a directory, build a Karpathy-methodology wiki skeleton (raw/, wiki/, index.md, log.md, schema.md).
• Upload + confirm — drag/select files, review in a confirm dialog, write to raw/.
• Auto-ingest — the plugin drives the LLM to read raw/ and generate/update wiki/ pages, with concurrent processing and a progress bar. PDFs are text-extracted via pypdf.
• Knowledge graph — force-directed graph of [[wikilinks]], with zoom/pan, hover-highlight, and click-to-open.
• Self-check (lint) — reports orphan pages, isolated pages, and dead links.
• Read in place — markdown renderer + inline PDF viewer.
• Chat-grounded Q&A — a wiki_query tool + prompt section lets you ask the chat and get answers with page citations.

---

Architecture

A single package with two halves, plus a one-line preset contribution:

Wiki layout

<workspace>/llm-wiki/
├── raw/            # immutable source files (you own)
│   └── .ingested.json   # dedup bookkeeping
├── wiki/           # LLM-generated pages
│   ├── entities/
│   ├── concepts/
│   └── sources/
├── index.md        # content catalog (auto-rebuilt)
├── log.md          # append-only timeline
└── schema.md       # wiki conventions (you + LLM co-evolve)

---

Build & install

This package is authored as TypeScript/JSX against the DeepSeek Harness source tree, so it is built with that repo's toolchain:

# inside the deepseek-harness repo
pnpm install
pnpm --filter @vesna-strivozha-2026/dsh-llm-wiki build

# install into your profile (host + client)
dsh plugin --profile web add @vesna-strivozha-2026/dsh-llm-wiki

Then author a preset (copy of standard) that adds the chat-Q&A tool row, e.g.:

- id: tool-llm-wiki
  name: '@vesna-strivozha-2026/dsh-llm-wiki'

Restart the client — the wiki panel and wiki_query tool are now permanent.

---

Roadmap

• Core loop: workspace / ingest / graph / lint / read / chat Q&A
• Polished UI on the DSH design system (@deepseek-ai/dsh-client-ui-primitives)
• Publish to the DSH plugin store
• Open-source: CI, tests, contribution guide

Model Experience

Ingest engine

What the model sees

One LLM call per raw/ source: the source text (truncated to maxSourceChars) plus the schema.md conventions, asked to return a structured JSON {"pages":[{"path":"...","content":"..."}],"logEntry":"..."} that is written into wiki/ pages before index.md and log.md are rebuilt.

Token effect

Output is capped at ingestMaxTokens (default 16384) per source; input is truncated at maxSourceChars (default 60000); up to concurrency (default 3) sources run concurrently.

KV Cache effect

None; ingest reads source and wiki markdown from disk on each run.

wiki_query tool

What the model sees

The wiki_query tool searches wiki/ pages and returns the index.md summary plus the top matching pages; a llm-wiki-query prompt section tells the model to prefer the tool for knowledge-base topics.

Token effect

The tool result is truncated — index ≤ 1500 chars, each hit ≤ 1200 chars, at most 4 pages — so it stays small in context.

KV Cache effect

None; search reads markdown from disk on each call.

Known Limitations and Deferred Work

• macOS-only "open with specific app" — the PDF-app list (Preview / Adobe / Skim) and "open with app" are macOS-only; "open with default app" and "reveal in file manager" work cross-platform (macOS open, Windows Invoke-Item, Linux xdg-open).
• pypdf dependency — PDF ingestion (text extraction) needs Python with pypdf on any platform (in-panel PDF reading works without it).
• details slot takeover — the panel replaces the shipped tool-details column (tool results remain visible in the conversation cards).
• Hand-rolled markdown renderer and force-directed graph — no third-party markdown/graph library; large wikis may render slowly, and the graph offers only zoom/pan/click.
• Binary uploads go over JSON as base64 — large files upload slowly; there is no streaming upload.