nanobot-runtime/skills/llm-wiki/SKILL.md

---
name: llm-wiki
description: >
  Build and maintain an LLM-curated personal knowledge base — the "LLM Wiki" pattern. Use whenever
  the user wants to ingest a source (paper, article, transcript, PDF, notes) into a persistent,
  compounding knowledge base, ask a question against the accumulated notes, lint or audit such a
  base, or initialize a new one. Applies even when the user doesn't say "wiki" — any time they
  accumulate textual sources over time and want them organized. A deliberate, standalone store,
  distinct from agent memory (note / keep / MEMORY.md).
---

# LLM Wiki

A skill for building and maintaining an LLM-curated knowledge base inside a project, following the pattern Andrej Karpathy described in his April 2026 gist. The wiki is a directory of markdown files that the LLM owns and maintains; the user curates sources and asks questions, and the LLM does the bookkeeping.

## Nanobot adaptation — read this first

This skill is ported to run on this nanobot. The generic docs below describe a project-local wiki; the rules here pin it to this nanobot and override anything that conflicts.

**⚠️ On any add/save/ingest request: capture only.** Write the source to `cml/raw/<slug>.md`, confirm in one short line, and **STOP** — no reads, no scripts, no compiles. Full rules and the escape hatch in "Capture vs compile" below — read it before acting on any such request.

- **One wiki, fixed location.** This nanobot has exactly one wiki, at `cml/wiki/`, with raw sources at `cml/raw/` — both relative to your working directory (the workspace). Wherever the docs below say `wiki/` or `<project-root>/wiki/`, read `cml/wiki/`; `raw/` means `cml/raw/`.
- **Run scripts with `uv run`, never bare `python`.** Always `uv run skills/llm-wiki/scripts/<script>.py …`. The scripts default to `cml/wiki`, so for most you can omit the path argument.
- **Bootstrap once:** `uv run skills/llm-wiki/scripts/init_wiki.py . --wiki-dir cml/wiki --raw-dir cml/raw` creates `cml/wiki/` + `cml/raw/`. Idempotent — safe to re-run.
- **Separate store — not agent memory.** The wiki is a deliberate, standalone knowledge base of curated sources. It is **not** the agent's memory: keep it distinct from `note`, `keep`, and `MEMORY.md`, and do not fold wiki content into them (or vice versa). The Dream processor must **not** touch `cml/` — it is outside the memory and skills Dream curates. Do not wire the wiki into `MEMORY.md`; its location is documented here.
- **Lint is report-only — a lint turn never mutates the wiki.** A lint request ("lint", "what's broken", "clean up the wiki", or a HEARTBEAT lint) means exactly: run `wiki_lint.py` and — if the graph layer exists — `wiki_graph_lint.py`, present the findings as a summary of *proposed* edits, and **STOP the turn**. Forbidden during a lint turn (all of it is fixing, done later in a separate approved turn): creating or editing any page under `cml/wiki/`, writing debug/throwaway scripts, regenerating the graph (`wiki_graph_extract.py`), re-running lint in a loop, updating `index.md` / `log.md`. If you catch yourself editing a page or re-running lint to check your own fix, you are fixing inline — stop. Fixes happen only after the user approves, one category at a time (see the lint workflow).
- **Language.** This skill body is English; reply to the user in the user's own language.
- **Scope: local PoC.** Single machine, versioned by the git repo running over the workspace. No shared remote, no multi-client sync.

## Capture vs compile — the background pipeline

Ingest is split into two phases so the interactive turn stays instant. Full agentic compile takes a minute or more; doing it inline made capture unusable.

- **Capture (interactive default — instant).** When the user wants to add a source ("save this", "ingest this", "add X to the wiki"), do exactly three things and nothing more:
  1. Write the source into `cml/raw/<slug>.md` (pick a descriptive `<slug>`). Inline text → write as-is. URL → write the URL as-is (the compile step will fetch it).
  2. Confirm in **one short line** ("zachyceno — zkompiluju na pozadí").
  3. **STOP the turn.**
  Forbidden during a capture turn (all of this is compile, done later in the background): reading `SCHEMA.md` / `index.md` / any wiki page, creating or editing pages under `cml/wiki/`, running `init_wiki.py` / `wiki_lint.py` / `wiki_graph_*` / any script, rebuilding the graph, updating `index.md` or `log.md`. If you catch yourself about to read the schema or write a page, you are doing compile inline — stop and just capture.
- **Escape hatch.** Only if the user *explicitly* says "compile now" / "synchronously" / "do it now" / "hned" do you run the full Compile workflow inline in this turn. A normal "add this to my wiki" is **not** an escape hatch — it is capture.
- **Compile (drain — background, batched).** A system cron runs `scripts/wiki_compile.py` every minute; when `cml/raw/` has pending sources it invokes this skill with a drain goal. Compile processes **every** pending source in one batch (one index/graph update for many sources), then moves each processed source into `cml/raw/_done/`. This is the existing ingest workflow (below) applied per pending source. **Idempotency:** if `cml/wiki/sources/<slug>.md` already exists for a source, treat it as already compiled — skip re-processing and move the raw file to `cml/raw/_done/`. Always move a source out of `cml/raw/` once handled so the next cron tick doesn't re-process it.
- **`cml/raw/` layout.** Regular files directly in `cml/raw/` = the **pending inbox**. `cml/raw/_done/` = processed sources (move here after a successful ingest). `cml/raw/_hard/` = sources held back as ambiguous/conflicting (don't force-compile these; record why in `log.md`). `cml/raw/assets/` = downloaded images, never a source. The pre-check and compile both ignore `_done/`, `_hard/`, and `assets/`.

## Architecture: three layers, three operations

The wiki has three layers and three operations. Internalize this vocabulary because the rest of the skill assumes it.

The three layers are **raw sources** (the user's curated source material — articles, papers, PDFs, transcripts; immutable, the LLM reads but never modifies them), **the wiki** (a directory of LLM-generated markdown pages — entity pages, concept pages, comparisons, summaries; the LLM owns this layer entirely), and **the schema** (a `SCHEMA.md` file at the wiki root that documents the conventions for this particular wiki — page types, naming rules, tag taxonomy, ingest workflow customizations; co-evolved with the user).

The three operations are **ingest** (a new source arrives; the LLM reads it, writes a summary page, updates relevant entity and concept pages, appends to the log), **query** (the user asks a question; the LLM navigates the wiki via the index, reads the relevant pages, and synthesizes an answer — often filing the answer back as a new page so the exploration compounds), and **lint** (a periodic health check; the LLM scans for contradictions, stale claims, orphan pages, missing concepts, broken links).

For the canonical write-up of these operations, read `references/architecture.md`. For the step-by-step procedures, read `references/ingest-workflow.md`, `references/query-workflow.md`, and `references/lint-workflow.md` as needed.

## Graph layer (compiled, optional)

Pages can carry typed `graph:` metadata in frontmatter. A bundled extractor compiles every page into `wiki/graph/`: `nodes.jsonl`, `edges.jsonl`, `graph.sqlite`, `graph.graphml`. **Markdown is canonical**; the graph is a regenerable index. Pages without `graph:` still appear as nodes (derived from their `type`/`kind`) and contribute low-confidence `mentions` edges from body wikilinks. Typed semantic edges (e.g. `founded`, `proposed`, `depends_on`) require an explicit source and evidence quote — never emit one inferred from training data.

The conventions for the graph layer (predicate vocabulary, node id format, required fields) live in `wiki/graph/ontology.yaml`. The full reference is `references/graph-workflow.md`. Run the bundled scripts after substantive ingests:

```bash
uv run skills/llm-wiki/scripts/wiki_graph_lint.py cml/wiki/      # check ontology + evidence + alias collisions
uv run skills/llm-wiki/scripts/wiki_graph_extract.py cml/wiki/   # rebuild nodes.jsonl, edges.jsonl, graph.sqlite, graph.graphml
uv run skills/llm-wiki/scripts/wiki_graph_query.py cml/wiki/ neighbors --node product:konvy
```

If `wiki/graph/ontology.yaml` does not exist, the wiki is pre-graph and you should treat the graph step as a no-op — don't fabricate it.

## Default project layout

The wiki is at a fixed location on this nanobot (`cml/wiki/`, `cml/raw/`):

```
<workspace>/
├── cml/
│   ├── wiki/
│   │   ├── SCHEMA.md              ← conventions, the "config file" — read this FIRST
│   │   ├── index.md               ← entry point: catalog of all pages with one-line summaries
│   │   ├── log.md                 ← append-only chronological log of ingests/queries/lints
│   │   ├── indexes/               ← (appears once index.md shards) per-category indexes
│   │   ├── entities/              ← pages about specific things (people, products, papers, places)
│   │   ├── concepts/              ← pages about ideas, methods, frameworks
│   │   ├── sources/               ← per-source summary pages (one per ingested source)
│   │   └── synthesis/             ← cross-cutting analyses, comparisons, query results filed back
│   └── raw/                       ← the user's source material (PDFs, .md clippings, images)
│       └── assets/                ← downloaded images referenced by raw clippings
└── ...
```

## The scalability discipline

The single biggest failure mode of the LLM Wiki pattern is the wiki itself becoming a context bottleneck. Naive implementations break around a few hundred pages: the LLM either reads too many pages per query or starts hallucinating because it skipped the relevant ones. This skill's design is shaped almost entirely by avoiding that failure. The principles below are non-negotiable; ignoring them is what makes the pattern collapse at scale.

**Atomic pages.** Every wiki page is about one concept and stays small — soft cap 400 lines or roughly 2,000 words, hard cap 800 lines. When a page outgrows this, split it: extract sub-concepts into their own pages and have the parent link to them. A page that takes up 30% of the context window on its own is a design smell.

**Index-first navigation.** Never grep or glob the wiki blindly when answering a query. Always read `index.md` (or the relevant sharded index under `indexes/`) first to identify candidate pages, then drill into only those. The index is engineered to be cheap to read — one line per page, no bodies — and it is the cache that makes the whole pattern scalable.

**Sharded indexes.** When `index.md` itself exceeds ~300 lines or the wiki passes ~150 pages, shard it: move category-specific entries into `indexes/<category>.md` files (e.g. `indexes/entities.md`, `indexes/concepts.md`, `indexes/sources.md`, or finer domain shards), and have the top-level `index.md` become a directory of those shards. Now reading the index is a two-step lookup but each step is bounded.

**YAML frontmatter on every page.** Every wiki page begins with frontmatter that includes at minimum `type`, `tags`, `sources`, and `updated`. The bundled `wiki_search.py` script can filter on these without reading page bodies. See `references/page-conventions.md`.

**Surgical edits, not rewrites.** When updating a page (e.g. adding a new cross-reference because a freshly ingested source mentions an existing entity), use `str_replace` to touch only the relevant section. Rewriting whole pages is slow, expensive in tokens, and risks losing prior nuance.

**Backlink discovery via grep.** To find every page that references a given entity, run `grep -rl "\[\[entity-name\]\]" cml/wiki/` rather than reading pages to look for mentions. The bundled scripts make this easy.

**Chunked source ingestion.** Large raw sources (long PDFs, book chapters, lengthy transcripts) should be read in chunks during ingest, not loaded whole. The ingest workflow handles this — see `references/ingest-workflow.md`.

**Search script for large wikis.** Once the wiki passes ~300 pages, plain index lookup may not surface the right pages for fuzzy queries. Use `scripts/wiki_search.py` for BM25-ranked retrieval with optional frontmatter filters. It's a fallback, not the default — index-first is still cheaper when it works.

**Stats.** `uv run skills/llm-wiki/scripts/wiki_stats.py` gives a quick summary of page count by type and link density — useful for deciding when to shard the index.

For the full scaling playbook including thresholds and migration steps, read `references/scaling-playbook.md`.

## Initializing a new wiki

If the project does not contain a `wiki/` directory (or whatever the user calls theirs), run the bootstrap script:

```bash
uv run skills/llm-wiki/scripts/init_wiki.py . --wiki-dir cml/wiki --raw-dir cml/raw
```

This creates the directory structure, drops in templates for `SCHEMA.md`, `index.md`, and `log.md`, and seeds a starter page convention document. After bootstrapping, briefly walk the user through the schema and ask whether they want to customize anything (e.g. domain-specific page types, custom tags) before the first ingest. The schema is meant to evolve — encourage editing it.

Do **not** wire the wiki into an agent-memory file (`MEMORY.md` / `AGENTS.md`) on this nanobot — see the Nanobot adaptation rules: the wiki is a separate store and its location is documented in this SKILL.md, which the skill description already surfaces.

## The ingest workflow (summary)

**STOP — do not run this for a plain "add this to my wiki" request.** This is the **compile (drain)** step. It runs only from the background cron (the drain goal) or the explicit "compile now" escape hatch. If the user just asked to add/save/ingest a source, you are in *capture* — write to `cml/raw/` and stop (see "Capture vs compile"). The source is already captured in `cml/raw/` when compile runs, so don't re-write it; read it from there.

The full workflow is in `references/ingest-workflow.md`; what follows is the shape of it. Read the source — chunked if large — and write a single source-summary page in `cml/wiki/sources/`, named after the source slug, with full frontmatter and citations back to the raw file. Then identify which existing entity and concept pages this source touches; for each, surgically update the relevant section using `str_replace` rather than rewriting. Identify any new entities or concepts the source introduces and create new pages for them, linking from related existing pages so they don't become orphans. Update `index.md` (or the relevant shard) with the new pages. Append a single line to `log.md` with the date, operation type, and source title. After the source is fully ingested, move its raw file into `cml/raw/_done/`. When run interactively, discuss the takeaways with the user as a final step — what surprised them, what's worth following up on — and offer to file that discussion back as a synthesis page.

**Page naming — avoid slug collisions.** A source page and a concept/entity page must not claim the same slug, or their `[[wikilinks]]` collide (e.g. a paper *and* the concept it introduces both wanting `rotary-position-embedding.md`). Name **concept/entity pages after the short name of the idea** (`rope.md`, `transformer.md`), and **source pages after the source's own slug** (the title/filename of the raw source). If two pages still resolve to the same slug, suffix one to disambiguate.

## The query workflow (summary)

Full version in `references/query-workflow.md`. To answer a query against the wiki: read `index.md` (or the relevant shard) first; identify candidate pages from one-line summaries; read those pages (and any backlinks they list that look relevant); synthesize the answer with `[[wikilink]]` citations to the pages you used; offer to file the synthesized answer back into `wiki/synthesis/` so future queries benefit. If the index doesn't surface good candidates, fall back to `uv run skills/llm-wiki/scripts/wiki_search.py "query terms"` for ranked retrieval. If the wiki appears to lack coverage of the topic, say so plainly rather than confabulating — flag it as a candidate ingest target.

## The lint workflow (summary)

**STOP — a lint turn reports, it does not fix.** See the report-only gate in "Nanobot adaptation". Run the scripts, present the findings, stop. The scripts are fast (sub-second on a small wiki); if a lint turn runs long, you have wrongly slipped into fixing.

Full version in `references/lint-workflow.md`. Lint is best run on a cadence (after every N ingests or weekly), not on every operation. Run `uv run skills/llm-wiki/scripts/wiki_lint.py` for structural issues (orphan pages, broken `[[wikilinks]]`, oversized pages, missing/malformed frontmatter, stale `updated` dates) and — if the graph layer exists — `uv run skills/llm-wiki/scripts/wiki_graph_lint.py` for typed-edge issues. For the semantic checks that need an LLM (contradictions with older claims, concepts mentioned but lacking a page, coverage gaps) read at most the ~10 most-recently-updated pages — no blind globbing. Present everything as proposed edits for the user to approve; never apply them in the lint turn — the wiki is the user's, and silent rewrites erode trust.

**When the user approves fixes (a later turn):** fix in bounded batches, one category at a time. Do **not** re-read the lint script to reverse-engineer it, and do **not** loop edit↔lint — run lint once at the end to confirm, and if findings remain, report them and ask rather than continuing blind. Graph-lint findings are interdependent (fixing one edge can create an orphan); clearing a large backlog is its own task, not part of a lint turn.

## Failure modes to guard against

- **Silent corruption:** every wiki claim must carry a `sources:` frontmatter entry pointing back to the raw file. When in doubt during ingest, hedge ("the source claims X") rather than asserting.
- **Wiki-reads-its-own-output drift:** during ingest, when updating an existing page, re-read the relevant raw source for the existing claim before merging — don't take the wiki's word for what the source said.

## Reference files

The reference files are the source of truth for the detailed procedures. Read them when the relevant operation is happening, not preemptively.

- `references/architecture.md` — the three layers and three operations explained in depth, with examples of page formats and the rationale behind each design choice
- `references/ingest-workflow.md` — the step-by-step ingest procedure including chunked reading for large sources and the per-page-type templates
- `references/query-workflow.md` — navigation patterns from index → page → backlinks, when to fall back to the search script, and how to file answers back as synthesis pages
- `references/lint-workflow.md` — what to check, how to present findings, and the cadence
- `references/page-conventions.md` — frontmatter schema, page naming, link syntax, page-type definitions, sizing rules
- `references/scaling-playbook.md` — thresholds at which to shard the index, when to introduce the search script, signals that the wiki has outgrown its current conventions
- `references/graph-workflow.md` — the optional graph layer: ontology, frontmatter schema, when to add typed edges vs plain wikilinks, and the extract/lint/query flow

## Templates

The templates in `assets/` are starting points — they get copied into the user's wiki on bootstrap and then evolve under the user's editing.

- `assets/SCHEMA.md.template` — the canonical schema document for a new wiki
- `assets/index.md.template` — the empty index file
- `assets/log.md.template` — the empty log file
- `assets/page.md.template` — a generic wiki page with the frontmatter scaffold
- `assets/ontology.yaml.template` — starter graph ontology copied to `wiki/graph/ontology.yaml`
- `assets/graph_README.md.template` — explainer for `wiki/graph/` (canonical vs generated files)
- `assets/graph_gitignore.template` — `.gitignore` for `wiki/graph/` (ignores `graph.sqlite` and `graph.graphml` by default)