uklid orphanu po retired llm-wiki + korekce reportu o wiki indexaci
This commit is contained in:
@@ -1,5 +1,64 @@
|
|||||||
# Wiki indexace — výsledky šetření
|
# Wiki indexace — výsledky šetření
|
||||||
|
|
||||||
|
> **KOREKCE 2026-09-18 17:52:10 — původní šetření níže je ve třech nálezech nesprávné.**
|
||||||
|
> Text se ponechává jako stopa po chybné analýze. Platí to, co je v této sekci.
|
||||||
|
|
||||||
|
## Korekce
|
||||||
|
|
||||||
|
### Indexace wiki funguje
|
||||||
|
`wiki_sync.log` v 17:07 hlásí `workspace: indexed 1 files (3 chunks), deleted 0` a
|
||||||
|
`done files=197 chunks=789 vectors=789 pending=0 embedded=3`. `wiki/index.sqlite` má
|
||||||
|
mtime 2026-09-18 17:07. Původní report vznikl v 17:06 — **minutu předtím, než běh doběhl**.
|
||||||
|
Git zdroje (`index`, `travel`, `wood-ng`) souhlasí s remote HEAD, `git ls-remote` i
|
||||||
|
embedding endpoint `nvidia.hell:11434` odpovídají v řádu ms.
|
||||||
|
|
||||||
|
### Root cause je výpadek serveru, ne skill
|
||||||
|
Všechny crony ustaly **naráz** 2026-09-17 v 19:13 a vrátily se 2026-09-18 ve 14:40 —
|
||||||
|
`wiki_sync.log` i `ollama_usage_cron.log` končí na stejné minutě. To sedí na pád
|
||||||
|
virtualizačního hostu. Žádná indexace mezitím neběžela, protože neběželo nic.
|
||||||
|
|
||||||
|
### Stale lock BYL (původní nález 3 je vyvrácen neprávem)
|
||||||
|
`wiki_sync.log`: `2026-09-18T14:40:37+02:00 WARN stale lock, reclaiming`. Lock je
|
||||||
|
`wiki/.sync.lock` (`STALE_SECONDS = 30 min`, viz `skills/wiki/scripts/wiki_sync.py`), ne
|
||||||
|
`triggers/.lock` ani `unlock.sh` — ty s wiki nesouvisí a hledaly se špatně. Lock byl
|
||||||
|
pozůstatek po běhu zabitém pádem hostu a první porebootový běh ho korektně uklidil.
|
||||||
|
|
||||||
|
### `wiki-compile` není přejmenovaný `llm-wiki`
|
||||||
|
`wiki-compile/SKILL.md` sám odkazuje „*that is capture-only (see llm-wiki skill)*" —
|
||||||
|
byl to **sourozenec** llm-wiki, ne rename. Obsahoval jen `SKILL.md`, žádné skripty.
|
||||||
|
|
||||||
|
### Rozbitý cron byl orphan, ne příčina
|
||||||
|
Retirement `llm-wiki` proběhl korektně (zálohy `backup/skills/llm-wiki/`,
|
||||||
|
`backup/workspace/cml/`), jen se zapomněl řádek v crontabu. Se skillem `wiki` nesdílel
|
||||||
|
**nic** — jiná DB, jiný log, jiný lock, jiný adresář. Nemohl indexaci blokovat.
|
||||||
|
|
||||||
|
**Pozor na tři nesouvisející věci sdílející slovo „wiki":**
|
||||||
|
|
||||||
|
| | živé | retired |
|
||||||
|
|---|---|---|
|
||||||
|
| skill `wiki` (hybrid RAG) | `wiki/index.sqlite`, `log/wiki_sync.log` | — |
|
||||||
|
| `llm-wiki` (PoC LLM-kurátorovaná wiki) | — | `backup/skills/llm-wiki/`, `backup/workspace/cml/` |
|
||||||
|
| `wiki-compile` (drain pro llm-wiki) | — | `backup/skills/wiki-compile/` |
|
||||||
|
|
||||||
|
### `db/wiki.sqlite` byl orphan
|
||||||
|
Grep přes celý workspace nenašel **žádnou** referenci kromě tohoto reportu. 0 bajtů, smazán.
|
||||||
|
|
||||||
|
### Co se opravilo (2026-09-18)
|
||||||
|
- crontab uživatele `nanobot`: odstraněn mrtvý `wiki-compile` job (zbývá 6 jobů)
|
||||||
|
- `log/wiki_compile_cron.log` (2,1 MB) + `log/wiki_compile_manual.log` → `backup/*.retired`
|
||||||
|
- `skills/wiki-compile/` → `backup/skills/wiki-compile/`
|
||||||
|
- `db/wiki.sqlite` (0 B) smazán
|
||||||
|
|
||||||
|
### Zbývá prověřit
|
||||||
|
Běh, který ve 14:40 přebral stale lock, doběhl až v 17:07 — **2 h 27 min** na jeden
|
||||||
|
zaindexovaný soubor. Hypotéza: `git fetch` nemá timeout (na rozdíl od `ls-remote`, který má
|
||||||
|
20 s) a po rebootu visel na nedostupné síti. Přesně scénář, před kterým varuje komentář
|
||||||
|
v `wiki_sync.py`. Neakutní — síť i endpointy teď odpovídají normálně.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Původní šetření (nesprávné — viz korekce výše)
|
||||||
|
|
||||||
## Souhrn
|
## Souhrn
|
||||||
Poslední úspěšná wiki indexace proběhla **2026-09-17 18:50** (mtime `wiki/index.sqlite`, 7.3 MB).
|
Poslední úspěšná wiki indexace proběhla **2026-09-17 18:50** (mtime `wiki/index.sqlite`, 7.3 MB).
|
||||||
Od přejmenování skillu `llm-wiki` → `wiki-compile` jede kompilační cron job nasucho a loguje chybu `No such file or directory`. Žádný stale lock na wiki nezůstal.
|
Od přejmenování skillu `llm-wiki` → `wiki-compile` jede kompilační cron job nasucho a loguje chybu `No such file or directory`. Žádný stale lock na wiki nezůstal.
|
||||||
|
|||||||
@@ -1,99 +0,0 @@
|
|||||||
---
|
|
||||||
name: wiki-compile
|
|
||||||
description: >
|
|
||||||
Idempotent wiki source compilation — drain pending raw sources into the wiki.
|
|
||||||
Use when the cron drain goal fires or the user explicitly says "compile now" / "zkompiluj".
|
|
||||||
Handles duplicate detection, ambiguous sources, idempotent skip, and graph regeneration.
|
|
||||||
Do NOT trigger on a plain "add this to my wiki" request — that is capture-only (see llm-wiki skill).
|
|
||||||
---
|
|
||||||
|
|
||||||
# Wiki Compile
|
|
||||||
|
|
||||||
Idempotent compile of pending raw sources into the LLM wiki. Runs as a background drain (cron) or on explicit user request ("compile now" / "hned").
|
|
||||||
|
|
||||||
## When to use
|
|
||||||
|
|
||||||
- Cron drain goal fires (background batch compile)
|
|
||||||
- User explicitly requests synchronous compile ("compile now", "zkompiluj wiki", "do it now")
|
|
||||||
- **NOT** for plain "add this" / "save this" requests — those are capture-only (write to `cml/raw/`, stop)
|
|
||||||
|
|
||||||
## Prerequisites
|
|
||||||
|
|
||||||
- Wiki must be initialized (`cml/wiki/SCHEMA.md` exists)
|
|
||||||
- Read `SCHEMA.md` first — it defines page types, naming rules, and ingest customizations
|
|
||||||
- Read `index.md` to know what pages already exist
|
|
||||||
|
|
||||||
## Steps
|
|
||||||
|
|
||||||
### 1. List pending sources
|
|
||||||
|
|
||||||
Scan `cml/raw/` for regular `.md` files (ignore `_done/`, `_hard/`, `assets/` subdirectories).
|
|
||||||
|
|
||||||
If empty → nothing to do, stop.
|
|
||||||
|
|
||||||
### 2. Batch all pending sources
|
|
||||||
|
|
||||||
Process **all** pending sources in one batch — one index/graph update for many sources is more efficient than one-by-one.
|
|
||||||
|
|
||||||
### 3. For each source, check idempotency
|
|
||||||
|
|
||||||
Read the source slug from the filename (e.g., `cml/raw/my-source.md` → slug `my-source`).
|
|
||||||
|
|
||||||
Check if `cml/wiki/sources/<slug>.md` already exists:
|
|
||||||
- **Exists** → already compiled. Move `cml/raw/<slug>.md` to `cml/raw/_done/`, skip re-processing, log "skip (already compiled)".
|
|
||||||
- **Does not exist** → proceed to duplicate check.
|
|
||||||
|
|
||||||
### 4. Duplicate URL detection
|
|
||||||
|
|
||||||
If the source content is a URL (single-line URL or frontmatter `url:` field), check whether any existing source page in `cml/wiki/sources/` already references that same URL:
|
|
||||||
- **Duplicate found** → move the raw file to `cml/raw/_hard/`, append entry to `log.md` noting "duplicate URL — same as <existing-slug>", skip compilation.
|
|
||||||
- **No duplicate** → proceed to ambiguous/conflict check.
|
|
||||||
|
|
||||||
### 5. Ambiguous / conflicting source check
|
|
||||||
|
|
||||||
If the source content is unclear, contradictory, or cannot be reliably summarized (e.g., garbled text, empty content, conflicting metadata):
|
|
||||||
- Move to `cml/raw/_hard/`
|
|
||||||
- Append entry to `log.md` with reason (e.g., "ambiguous — garbled content", "conflicting — title mismatch")
|
|
||||||
- Skip compilation
|
|
||||||
|
|
||||||
### 6. Compile the source
|
|
||||||
|
|
||||||
Follow the standard ingest workflow (see `references/ingest-workflow.md` in the llm-wiki skill):
|
|
||||||
|
|
||||||
1. Read the source (chunked if large)
|
|
||||||
2. Write a source-summary page at `cml/wiki/sources/<slug>.md` with full frontmatter and citations
|
|
||||||
3. Identify existing entity/concept pages this source touches → surgically update relevant sections
|
|
||||||
4. Create new entity/concept pages for novel topics, linking from related pages
|
|
||||||
5. Update `index.md` (or relevant shard) with new pages
|
|
||||||
6. Append a single line to `log.md`: date, operation, source title
|
|
||||||
|
|
||||||
### 7. Move processed source
|
|
||||||
|
|
||||||
After successful compilation, move `cml/raw/<slug>.md` to `cml/raw/_done/`.
|
|
||||||
|
|
||||||
**Every source must leave `cml/raw/`** — either `_done/` (success/skip) or `_hard/` (held back). Never leave a source in the inbox after processing.
|
|
||||||
|
|
||||||
### 8. Regenerate graph (if applicable)
|
|
||||||
|
|
||||||
If the wiki has a graph layer (`cml/wiki/graph/ontology.yaml` exists) and this batch added any pages with `graph:` frontmatter metadata:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
uv run skills/llm-wiki/scripts/wiki_graph_lint.py cml/wiki/
|
|
||||||
uv run skills/llm-wiki/scripts/wiki_graph_extract.py cml/wiki/
|
|
||||||
```
|
|
||||||
|
|
||||||
If no graph layer exists, skip this step entirely.
|
|
||||||
|
|
||||||
### 9. Summary
|
|
||||||
|
|
||||||
Report what happened in one concise line, e.g.:
|
|
||||||
- "Compiled 3 sources, skipped 1 (already done), held 1 (duplicate URL)."
|
|
||||||
- "Nothing to compile — inbox empty."
|
|
||||||
|
|
||||||
## Key rules
|
|
||||||
|
|
||||||
- **Idempotent**: re-running on the same source is a no-op (skip + move to `_done/`)
|
|
||||||
- **No force-compiling ambiguous sources**: move to `_hard/` and log why
|
|
||||||
- **Batch efficiency**: one index update + one graph regeneration per batch, not per source
|
|
||||||
- **Graph scripts require wrapper**: if workspace safety guard blocks direct execution, write a `tmp/` wrapper script using `uv run --script` with inline dependency metadata
|
|
||||||
- **Language**: wiki content is in Czech; compile output and log entries may be in English for consistency with existing logs
|
|
||||||
Reference in New Issue
Block a user