52 lines
4.9 KiB
Markdown
52 lines
4.9 KiB
Markdown
# Self-reflection 2026-09-08
|
||
|
||
Analysed 3 sessions in 1 batches. Findings: 2 (1 to review, 1 watched).
|
||
|
||
Window: from 2026-08-18, batches 1/1.
|
||
Known patterns: 100.0 occurrences / 100 sessions (previous run 400.0).
|
||
|
||
## f9110 · `reflect-finding-invented-from-truncated-read` [open/low]
|
||
|
||
Presentations during /reflect runs are again not grounded in the store that was just read. In the first session the agent announced 7 open findings and then presented the first one labelled [1/6] in the same turn — the label contradicts the count stated one message earlier. One minute later a second session over the same findings.jsonl (identical 16.0 kB read) reported 10 watch findings where the first session had said 12, so at least one of the two counts is invented rather than counted. The pattern is exactly the open finding about presentation not being derived from a freshly loaded store.
|
||
|
||
**Occurrences:** 9× in 4 sessions · first seen 2026-09-03, last seen 2026-09-07
|
||
|
||
**Evidence:**
|
||
- `websocket:ef4cc903-f63e-4893-874a-bf084137c171` 2026-09-07 — Ve storu je 7 otevřených nálezů (plus 12 ve stavu watch) … then presents **[1/6] retry-without-diagnosis** — label N disagrees with the announced 7
|
||
- `websocket:82f5eae7-2bfb-4390-8195-1a37ce3c0613` 2026-09-07 — mimo to se sleduje 10 `watch` nálezů — one minute after the first session claimed 12 watch findings over the same store
|
||
- `websocket 125975d1` 2026-09-05 — Načteno — findings store má 8 otevřených nálezů. Přiřazuji pořadí … fbda2, f611e, fae82, f81df … followed immediately by presentation 1/9 for f611e — wrong total and announced order not followed
|
||
- `websocket 125975d1` 2026-09-05 — Wrong internal id … f0f8c … f0cd4 — agent re-greps reflect/findings.jsonl mid-run to recover ids from the read it had already done
|
||
- `websocket 125975d1` 2026-09-05 — Presentation 2/9 speculation-presented-as-fact — first seen 2026-09-04, last seen 2026-09-03; same reversed dates in presentations 4/9 and 5/9 with no comment
|
||
- `websocket:2ad0a447-de89-4e24-9377-d91113ffa50b` 2026-09-02 — user: tak kdyz uz ho mame, tak nalez muzes smazat; agent rejects guessed id f9a4b -> no finding with id; multiple greps; agent admits: Finding [2/6], jak jsem ho představil, v store neexistuje — byla to zkomolená duplicita už aplikovaného f0720
|
||
|
||
**Proposal:** Make the reflect skill demand that the [x/N] label and every count stated aloud (open, watch) be recomputed from the records loaded in this turn, not carried over from an earlier turn or from memory.
|
||
|
||
**Patch:** `skills/reflect/SKILL.md`
|
||
|
||
```diff
|
||
- Assign **display IDs 1..N** over that sorted list, computed fresh each time. The user
|
||
- refers to findings by these short numbers; the internal `id` stays the key in the store
|
||
- and in the audit log, and is never what you ask the user to type.
|
||
+ Assign **display IDs 1..N** over that sorted list, computed fresh each time. The [x/N]
|
||
+ label you present must use that same N, and every count you state aloud (open, watch)
|
||
+ must be counted from the records loaded in this very turn — never carried over from an
|
||
+ earlier turn or from memory; if your label or count disagrees with what you announced,
|
||
+ recount before presenting. The user refers to findings by these short numbers; the
|
||
+ internal `id` stays the key in the store and in the audit log, and is never what you
|
||
+ ask the user to type.
|
||
```
|
||
|
||
## f48de · `unverified-success-claim` [watch/medium]
|
||
|
||
Regression of the applied fix. At the end of the deep-research turn the agent told the user the report was also saved under results/2026-09-07_mmap-writeback-read-slowdown-research.md, but the session log contains no write_file and no other file-creating tool call — the only exec was date +%F, used to build that very filename. SOUL.md already forbids announcing saving without a successful tool result, yet the claim slipped through at the exact moment the user was most likely to rely on it.
|
||
|
||
**Occurrences:** 3× in 3 sessions · first seen 2026-09-04, last seen 2026-09-07
|
||
|
||
**Evidence:**
|
||
- `websocket:af5374bc-cfcb-4648-a17f-250f1057fbd4` 2026-09-07 — final message: Report je i uložený v `results/2026-09-07_mmap-writeback-read-slowdown-research.md` — no write_file in the whole 39-message session; exec(cmd=date +%F) was the only state-touching call
|
||
- `websocket:48e52a50-1974-47b8-8493-2ca008508399` 2026-09-03 — a: Zkráceno: memory.md: 3 stručné zápisy… state.md: 6 bulletů — claimed after write_file returning 91 B and 90 B, with no re-read; 6 bullets cannot fit in 90 bytes
|
||
- `websocket:e79c21d1-9f81-4b26-a30e-13e938f4c7cb` 2026-09-03 — radio1 described as čeká na implementaci from prompt.md, while state.md is 0 B — pipeline status stated without checking any progress records
|
||
|
||
**Proposal:** In skills/deep-research/SKILL.md add a closing rule: a results file may be announced only when a write_file for it succeeded in the same turn; if it was not written, either write it before answering or offer to write it, never imply it exists.
|
||
|