# Architecture

This document explains the three-layer / three-operation architecture in detail. The main `SKILL.md` summarises it; this is the reference you reach for when you need to know *why* a design decision exists or how to handle an edge case.

## The three layers

### Raw sources

Raw sources are the user's curated input material. They live in `cml/raw/` (or wherever the project's `SCHEMA.md` declares). They are **immutable** — the LLM reads from them but never modifies them. This immutability is load-bearing: it means the wiki can always be re-derived from the raw sources if it gets corrupted, and it gives the user a stable ground truth they can audit independently.

What goes in `cml/raw/`: PDFs, web articles converted to markdown (the Obsidian Web Clipper is one popular path), transcripts, code repos, dataset descriptions, screenshots, hand-typed notes the user wants the wiki to incorporate. What does *not* go in `cml/raw/`: anything the LLM generated — that all belongs in the wiki layer.

A useful convention is one source per file (or per directory if the source has multiple pieces, e.g. a paper plus its appendix), with a slugified filename that ends up matching the wiki's source-summary page name. This makes the back-pointer from the wiki to the raw source trivial.

### The wiki

The wiki is a directory of LLM-generated markdown files. The LLM owns this layer entirely — it creates pages, updates them when new sources arrive, maintains cross-references, and keeps everything consistent. The user reads it (typically through Obsidian, but any markdown viewer works); the LLM writes it.

The wiki directory is conventionally split into subdirectories by **page type**:

- `cml/wiki/sources/` — one summary page per ingested source. Captures what the source said, in the LLM's words, with a citation back to the raw file. These are append-mostly: you write one when you ingest a source and rarely modify it after.
- `cml/wiki/entities/` — pages about specific things: people, products, papers, places, companies, events. Anything that has a proper noun or could plausibly be a Wikipedia article subject. These accumulate updates as new sources mention the entity.
- `cml/wiki/concepts/` — pages about ideas, methods, frameworks, abstractions. These are the most heavily cross-referenced pages and the ones most likely to evolve as understanding deepens.
- `cml/wiki/synthesis/` — cross-cutting analyses, comparisons, query answers filed back. This is where exploration compounds: a comparison the user asked for becomes a page the next query can build on.

`SCHEMA.md` may declare additional types (e.g. `cml/wiki/decisions/` for an engineering team, `cml/wiki/characters/` for a fan wiki, `cml/wiki/experiments/` for a research lab). Adding a new type costs nothing — make the directory, document it in the schema, update the index template.

### The schema

`SCHEMA.md` lives at the wiki root and is **the configuration file** that turns a generic LLM into a disciplined wiki maintainer for *this specific* knowledge base. It documents the page types in use, the tag taxonomy, the naming conventions, any custom workflow steps, and the user's stylistic preferences (e.g. "always include a 'Why this matters' section on concept pages", "never use bullet lists in summaries").

The schema is **co-evolved** with the user. On bootstrap it starts from the default template in `assets/SCHEMA.md.template`, but every user will customize it as they discover what fits their domain. When you notice a recurring pattern in the user's edits or feedback that isn't in the schema, propose adding it. When the schema starts contradicting itself or growing unwieldy, propose pruning it.

The schema is the first file you read when entering an existing wiki. Its conventions override the defaults documented in the skill.

### The graph (optional, compiled)

The wiki may carry a fourth layer at `cml/wiki/graph/`: a compiled, queryable view of the typed `graph:` metadata in page frontmatter and the body wikilinks. It contains a hand-edited `ontology.yaml` (the contract: which node types and predicates exist) plus generated artifacts (`nodes.jsonl`, `edges.jsonl`, `graph.sqlite`, `graph.graphml`) produced by `wiki_graph_extract.py`. **Markdown is canonical**; the graph can be deleted and regenerated from the markdown without losing knowledge.

Its purpose is to make typed, provenance-backed relationships machine-queryable — "who founded what", "what does Konvy depend on", "shortest path from A to B" — without giving up the editability and human-legibility of markdown. Typed edges require an explicit `source` (a source-page slug) and `evidence` quote; the extractor never invents them. Plain `[[wikilinks]]` in the body produce low-confidence `mentions` edges, which are useful for navigation but not for evidence.

Use the graph layer when the user's questions are predominantly relational and the cost of maintaining typed metadata is paying for itself. Skip it for purely textual wikis. Full reference: `graph-workflow.md`.

## The three operations

### Ingest

A new source has arrived. The user has dropped a file in `cml/raw/` (or pasted content and asked you to file it). The job is to integrate this source into the wiki such that future queries can benefit from it.

The shape of an ingest: read the source (chunked if large), discuss the key takeaways with the user briefly, write a summary page in `cml/wiki/sources/`, identify which existing pages are touched, surgically update those pages, create new pages for any new entities or concepts, update the relevant index, and append to the log.

The temptation to skip the discussion step is strong, but resist it — the user's reaction to the takeaways often reveals what should be emphasized in the wiki versus left out. Ingest is not a batch import; it's a collaborative reading.

For the full procedure, see `ingest-workflow.md`.

### Query

The user asks a question. The job is to answer it from the wiki, with citations, and to file the answer back if it represents new synthesis.

The shape of a query: read the index to identify candidate pages; read those pages; if needed, follow `[[wikilinks]]` from those pages or grep for backlinks; synthesize the answer with `[[wikilink]]` citations; offer to file the answer into `cml/wiki/synthesis/` if it's substantive enough to be worth keeping.

The compounding effect of the wiki only works if good answers get filed back. A comparison you generated, a connection you discovered, an analysis you produced — these should not evaporate into chat history. Default to offering to file; let the user decline if the answer was too trivial or too transient.

For the full procedure, see `query-workflow.md`.

### Lint

Periodic health check. The job is to find structural and semantic problems before they compound.

Structural problems are mechanical and the bundled `wiki_lint.py` script catches them: orphan pages with no inbound links, broken `[[wikilinks]]` to nonexistent pages, oversized pages that need splitting, missing or malformed frontmatter, suspicious staleness (a page that hasn't been updated despite many recent ingests touching its topic).

Semantic problems need the LLM: contradictions between pages, claims that newer sources have superseded, concepts mentioned in many pages but lacking their own page, cross-references that should exist but don't, knowledge gaps the user might want to fill.

Lint findings are presented as proposed edits, not silent rewrites. The user approves changes. This is essential for trust — a wiki that mutates under the user is not a wiki the user can rely on.

For the full procedure, see `lint-workflow.md`.