lachtan/nanobot-runtime
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

runtime

Browse Source
This commit is contained in:
lachtan
2026-09-08 20:40:26 +02:00
parent a3f007bea3
commit 6708c59464
9 changed files with 450 additions and 8 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
reflect/findings.jsonl
View File

@@ -20,12 +20,13 @@
{"id": "f5c34", "status": "rejected", "created": "2026-09-02", "pattern": "answer-self-config-from-guesswork", "severity": "medium", "diagnosis": "A question about the agent own exec safety guard behavior was answered with an invented mechanism stated as fact: the guard blocks diacritics in the command string. The evidence did not support it — the ASCII test attempt also added working_dir, so two variables changed at once and the diacritics conclusion was unfounded. When the user challenged it, the recap partially walked it back but still asserted that diacritics in a command is a suspicious signal for the guard and guessed at guard path-parsing internals. The guard is documented in AGENTS.md (explicit workspace path requirement), which…", "evidence": [{"session": "websocket:6e9b8008 | 2026-09-02 06:22", "when": "2026-09-02", "excerpt": "agent message claims the guard blocks diacritics in the command string because the ASCII version passed; later recap still claims diacritics is a suspicious signal, presumably because the guard parses paths in the command"}, {"session": "eca5b6fa", "when": "2026-08-29", "excerpt": "Assistant: v config.json nejsou presety glm-5.3 ani kimi-k2.7-code — musíš je ručně doplnit. O dvě zprávy později uživatel pastnul config s presety kimi, kimi27, kimi3, glmi51, glm52, glm53, glm a napsal: hele ty presety si trosku oprav"}, {"session": "websocket:e93b786a-7792-4f0e-97f0-2efe15dc9aed", "when": "2026-08-29", "excerpt": "turns about nanobot daily model use and preset switching: my check -> ok, but no web_fetch of nanobot.wiki docs per SOUL.md rule; claims like stačí ho přidat do configu and quota mechanics stated without doc verification"}, {"session": "websocket:8e293585-f00f-4b72-893d-b9adc6adb1a5", "when": "2026-08-18", "excerpt": "odpověď: Nejpravděpodobnější příčina: Kimi k3 má extrémně dlouhý defaultní CoT … 50k-100k+ tokenů, $1.0 což odpovídá ~100k tokenům — bez jediného ověřovacího tool callu, logy dostupné nejsou"}, {"session": "websocket_2dc073ed", "when": "2026-05-28", "excerpt": "u: pokud se nahraje novy skill je okamzite funkcni? → a: Ne, není okamžitě funkční… musí se runtime restartovat — bez jediného fetche doku; uživatel: a podival si se na nanobot.wiki, mas to primo napsane v SOUL.md. Později session b818067e ověřila ze zdrojáku skills.py opak: načítá se každý turn, restart není potřeba."}, {"session": "websocket_da4e5a03", "when": "2026-05-29", "excerpt": "u: budes to delat pres sqlite cli nebo pres mcp? → a: MCP v tomto prostředí nemám — nanobot používá vlastní tools, ne MCP servery — bez lookups; opačný závěr už dříve potvrdila dokumentace fetchnutá v session 0df35595 (tools.mcpServers je v nanobotu podporované)."}], "occurrences": 8, "sessions_affected": 8, "proposal": "For questions about own tooling behavior, verify with a controlled test changing a single variable, or check the documented guard rules before answering; state explicitly when the mechanism is unverified. Do not present a confounded one-off observation as the cause.", "regression_of": "fbb44", "history": ["2026-09-02:fc76d", "2026-09-02:f4313", "2026-09-02:f3373", "2026-09-02:f01dd"], "patch": {"file": "SOUL.md", "old_text": "- Pro dotazy o vlastním fungování nanobot (konfigurace, schopnosti, modely, jak funguje) → **před odpovědí i před jakoukoli úpravou config.json** `web_fetch` na https://nanobot.wiki/docs/0.2.0/ — bez ověřené dokumentace netipuj fakta a neupravuj konfiguraci", "new_text": "- Pro dotazy o vlastním fungování nanobot (konfigurace, schopnosti, modely, jak funguje) → **před odpovědí i před jakoukoli úpravou config.json** `web_fetch` na https://nanobot.wiki/docs/0.2.0/ — bez ověřené dokumentace netipuj fakta a neupravuj konfiguraci\n- Mechanismus vlastního toolingu nezkoumej confounded testem — měň **jednu proměnnou per pokus**; nezjištěný mechanismus prezentuj výslovně jako neověřený, nikoli jako fakt"}, "patch_drafted_at": "2026-09-02 12:35", "rejected": {"at": "2026-09-02 12:53", "reason": "úspěšnost ~40 % není dostatečná k zaplňování system promptu další instrukcí"}}
{"id": "fc317", "status": "watch", "created": "2026-09-02", "pattern": "exec-append-instead-of-file-tools", "severity": "low", "diagnosis": "Appends to project markdown files were done via exec cat with a heredoc instead of file tools, contrary to the tool contract that exec must not be a workaround for file operations. The second such append was blocked by the safety guard, wasting a turn before the agent switched to apply_patch, which it could have used from the start. The same heredoc form is prescribed by the project skill doc, so the skill doc is steering future sessions into the same trap.", "evidence": [{"session": "websocket:607b50b3 | 2026-09-01 14:45", "when": "2026-09-01", "excerpt": "exec cat append to projects/proxmox/memory.md -> ok; later exec cat append to projects/proxmox/state.md -> ERROR blocked by safety guard, then redone via apply_patch"}], "occurrences": 2, "sessions_affected": 1, "proposal": "Use apply_patch or edit_file for all file appends and edits; reserve exec for actual process execution. Separately, update the project SKILL.md to stop documenting heredoc-based log and file writes that trip the guard — point to the text flag or stdin from a tmp file with working_dir set instead."}
{"id": "f3f64", "status": "open", "created": "2026-09-03", "last_seen": "2026-09-02", "pattern": "tool-results-file-not-readable-directly", "severity": "medium", "diagnosis": "The agent repeatedly tried to read its own cached web_fetch results with the wrong tools and wrong assumptions: read_file with a line offset failed because the tool-result cache is a single-line JSON file; then exec pipes over the same file were blocked by the safety guard because the relative path resolves outside the working dir. In both cases the failure mode was knowable after the first error (single-line JSON inside .nanobot/tool-results), yet the agent kept guessing new access forms — including a raw sed -i on /dev/null — before landing on the workable one (a tmp/ python script with a r…", "evidence": [{"session": "websocket:fd9a49af-c659-4195-8b07-2d6bb556b5e7", "when": "2026-09-02", "excerpt": "read_file(offset=40, path=.nanobot/tool-results/...call_4nlfx9iy.txt) -> ERROR offset beyond end of file (1 lines); exec tr pipe over same path -> ERROR guard; retry with working_dir -> ERROR guard; finally grep tool -> ok but whole file is one line JSON"}, {"session": "websocket:afe450d5-cca9-4419-b930-1ebcb69b7c4e", "when": "2026-09-02", "excerpt": "grep on call_f3lus52v.txt returned whole file as one line; inline uv run python -c with the tool-results path -> ERROR guard; then 6 successive rewrites of tmp/extract_wiki.py iterating on the same cached JSON"}], "occurrences": 2, "sessions_affected": 2, "proposal": "Add a short rule to AGENTS.md exec/file sections: cached fetch results live in .nanobot/tool-results as single-line JSON — do not read_file with offsets and do not exec over them (guard blocks the path); when content extraction is needed, write a tmp/ python script using a relative path and uv run with working_dir set to workspace root."}
{"id": "fe72a", "status": "open", "created": "2026-09-03", "last_seen": "2026-09-02", "pattern": "retry-without-diagnosis", "severity": "medium", "diagnosis": "New occurrences of the known pattern: after an exec safety-guard block the agent retried a near-identical command form without isolating the cause. In the moisture-meter session the tr-pipe was re-sent with working_dir added — a second variable changed at once and the same guard error returned, wasting a turn; the diagnosis that the tool-results path itself is outside the boundary came only afterwards. In the project session the heredoc log command was blocked and immediately retried as a modified variant without first naming the suspected cause aloud.", "evidence": [{"session": "websocket:fd9a49af-c659-4195-8b07-2d6bb556b5e7", "when": "2026-09-02", "excerpt": "exec tr pipe -> ERROR guard; retry same command + working_dir -> ERROR guard, only then switch to grep tool"}, {"session": "websocket:50ba97da-8821-4adc-aa93-5b82b65077a3", "when": "2026-09-02", "excerpt": "project_cli.py log with heredoc -> ERROR guard; next attempt still a shell form with the long text inline before the tmp-file workaround"}, {"session": "websocket:6f8fcc41", "when": "2026-08-31", "excerpt": "naradi-extol.cz fetched with maxChars 5000, spec table did not render, same URL refetched with extractMode text and maxChars 4000 — identical target, smaller budget, no strategy change"}, {"session": "websocket:41eaf5b3", "when": "2026-08-31", "excerpt": "after a 403 on vseinstrumenti.ru reviews page, another vseinstrumenti.ru page was fetched and returned 403 again; several marketplace pages returned 300-600 B degenerate outputs without the block mode being named"}, {"session": "c745618e", "when": "2026-08-29", "excerpt": "web_fetch ollama.com/library/glm-5.3-flash:cloud → ok ale truncated; refetch maxChars 3500 → truncated; refetch maxChars 3000 → truncated; refetch via r.jina.ai 2500 → truncated; refetch freellm.net; teprve šestý pokus (endpoint /tags) přinesl usage tier okamžitě"}, {"session": "websocket:f6e1e265-7a37-451c-ad7a-f601c40fdc5a", "when": "2026-08-28", "excerpt": "web_fetch unix.stackexchange.com/questions/579818 -> ERROR 403 Forbidden, later web_fetch the same URL again -> ERROR 403 Forbidden"}], "occurrences": 69, "sessions_affected": 9, "proposal": "Reinforce the AGENTS.md exec rule with the concrete most common cause: guard blocks are usually triggered by paths appearing in the command string (absolute paths, .nanobot/tool-results, inline -c with path substrings), not by missing working_dir — write the payload to tmp/ first and keep the command string path-free.", "regression_of": "ff77b", "history": ["2026-09-02:fef64", "2026-09-02:f999d", "2026-09-02:fa495", "2026-09-02:fa59f", "2026-09-02:f2b3d"]}
{"id": "f706e", "status": "open", "created": "2026-09-03", "last_seen": "2026-09-02", "pattern": "guard-block-cause-misattributed", "severity": "medium", "diagnosis": "The agent systematically misdiagnoses what the exec safety guard blocks, then states the wrong mechanism to the user as fact. In the bits session the agent told the user the guard blocked because of an inline python -c with an absolute path — but the same form had succeeded minutes earlier in the same session, and the later block of a second inline python -c was blamed on quoting while the actual trigger stayed unidentified. In the same session the agent also told the user it has no tool that can delete a file (rm blocked by deny pattern) and left 4 diagnostic scripts in tmp/ — rm via the del…", "evidence": [{"session": "websocket:afe450d5-cca9-4419-b930-1ebcb69b7c4e", "when": "2026-09-02", "excerpt": "rm -f tmp/extract_wiki.py -> ERROR deny pattern; agent then claims nemám tool na smazání, který guard projde and leaves the file"}, {"session": "websocket:50ba97da-8821-4adc-aa93-5b82b65077a3", "when": "2026-09-02", "excerpt": "rm cleanup attempted once, blocked, agent tells user it cannot delete its 4 tmp scripts and leaves them in the workspace"}], "occurrences": 2, "sessions_affected": 2, "proposal": "When a claim about own tooling limitations or guard mechanisms is about to be stated to the user, verify it with one cheap test first (e.g. try deleting via a workspace-relative form) or state explicitly it is unverified; never claim a capability does not exist after a single blocked attempt."}
{"id": "f39f2", "status": "watch", "created": "2026-09-03", "last_seen": "2026-09-02", "pattern": "reflect-finding-invented-from-truncated-read", "severity": "high", "diagnosis": "During a /reflect review the agent presented a detailed open finding (tool-call-leaked-as-text, 12 occurrences in 6 sessions, labeled regression of f0720) that did not exist in the findings store. The root cause: findings.jsonl reads are truncated at 16 kB, and the agent reconstructed the open-findings list from the truncated tail instead of re-reading with offset or grepping before presenting. Consequences: a reject was executed against a guessed ID (f9a4b), several greps to chase the phantom record, and the user was told to delete something that did not exist — only caught because reject of…", "evidence": [{"session": "websocket:2ad0a447-de89-4e24-9377-d91113ffa50b", "when": "2026-09-02", "excerpt": "user: tak kdyz uz ho mame, tak nalez muzes smazat; agent rejects guessed id f9a4b -> no finding with id; multiple greps; agent admits: Finding [2/6], jak jsem ho představil, v store neexistuje — byla to zkomolená duplicita už aplikovaného f0720"}, {"session": "websocket:2ad0a447-de89-4e24-9377-d91113ffa50b", "when": "2026-09-02", "excerpt": "earlier in same session: read findings.jsonl -> 16.0 kB truncated, open list built from truncated content without follow-up paged read"}], "occurrences": 1, "sessions_affected": 1, "proposal": "Add to the reflect skill workflow: before presenting any finding, its ID must come from a fresh grep/read of the store, and truncated reads (16 kB cap) must be followed by an offset read until EOF; never present a finding whose ID was not observed verbatim in the store output."}
{"id": "f706e", "status": "open", "created": "2026-09-03", "last_seen": "2026-09-02", "pattern": "guard-block-cause-misattributed", "severity": "medium", "diagnosis": "The agent systematically misdiagnoses what the exec safety guard blocks, then states the wrong mechanism to the user as fact. In the bits session the agent told the user the guard blocked because of an inline python -c with an absolute path — but the same form had succeeded minutes earlier in the same session, and the later block of a second inline python -c was blamed on quoting while the actual trigger stayed unidentified. In the same session the agent also told the user it has no tool that can delete a file (rm blocked by deny pattern) and left 4 diagnostic scripts in tmp/ — rm via the del…", "evidence": [{"session": "websocket:afe450d5-cca9-4419-b930-1ebcb69b7c4e", "when": "2026-09-02", "excerpt": "rm -f tmp/extract_wiki.py -> ERROR deny pattern; agent then claims nemám tool na smazání, který guard projde and leaves the file"}, {"session": "websocket:50ba97da-8821-4adc-aa93-5b82b65077a3", "when": "2026-09-02", "excerpt": "rm cleanup attempted once, blocked, agent tells user it cannot delete its 4 tmp scripts and leaves them in the workspace"}], "occurrences": 2, "sessions_affected": 2, "proposal": "When a claim about own tooling limitations or guard mechanisms is about to be stated to the user, verify it with one cheap test first (e.g. try deleting via a workspace-relative form) or state explicitly it is unverified; never claim a capability does not exist after a single blocked attempt.", "patch": {"file": "AGENTS.md", "old_text": "On the first safety-guard block: diagnose the cause before retrying — check a missing `working_dir` first, never re-send the same blocked form, and change one variable per test until the cause is identified.", "new_text": "On the first safety-guard block: diagnose the cause before retrying — check a missing `working_dir` first, never re-send the same blocked form, and change one variable per test until the cause is identified.\n\nNever state a capability or guard mechanism to the user as fact after a single blocked attempt — verify with one cheap test first or say explicitly it is unverified."}, "patch_drafted_at": "2026-09-05 13:45"}
{"id": "f81b5", "status": "watch", "created": "2026-09-03", "last_seen": "2026-09-02", "pattern": "patch-workaround-sed-devnull", "severity": "medium", "diagnosis": "While preparing a reflect patch the agent attempted an exec command that piped sed output to /dev/null — an obviously无效 form that was certain to be blocked by the safety guard, sent without any diagnostic purpose. This is not a retry after block (it was a first attempt) but a variant of choosing a shell one-liner where a sanctioned tool exists: the edit could have been done with write_file of the patch JSON directly, as was done successfully seconds later.", "evidence": [{"session": "websocket:7a988478-e08c-4346-ba1c-a86d680b4d8a", "when": "2026-09-02", "excerpt": "exec sed -i ... /dev/null; true -> ERROR guard; immediately replaced by write_file tmp script which worked"}], "occurrences": 1, "sessions_affected": 1, "proposal": "Skip — near-single occurrence, but worth noting as an instance of the broader rule already in AGENTS.md: prefer file tools over shell text manipulation; the sed form served no purpose a write_file could not."}
{"id": "f8c92", "status": "watch", "created": "2026-09-04", "last_seen": "2026-09-03", "pattern": "answer-self-config-from-guesswork", "severity": "low", "diagnosis": "Known pattern, one new occurrence: a question about nanobot's own workings — whether there is a builtin version-check cron for other software besides nanobot — was answered from memory without checking config/jobs first; per SOUL.md such questions require verifying against the docs/config before answering.", "evidence": [{"session": "websocket:83fecb68-b419-449b-9713-f51c31bc89ab", "when": "2026-09-03", "excerpt": "Chceš, abych na nvidia.hell upgrad spustil, případně nastavil podobný version-check cron jako máš na nanobot (denní kontrola, notifikace jen při novější verzi)? — stated without checking jobs.json or cron list"}, {"session": "websocket:50ba97da-8821-4adc-aa93-5b82b65077a3", "when": "2026-09-02", "excerpt": "first answer lists 3 speculative causes (jiný preset na mobilní session, kompakce kontextu, tools se nepoužily) before any session inspection; after investigation: Mobil vliv nemá, odpověď byla kompletní a ověřená"}], "occurrences": 2, "sessions_affected": 2, "proposal": "Before proposing cron/version-check extensions of nanobot's own setup, run cron list / read jobs.json and cite the nanobot docs per SOUL.md Vlastní fungování", "history": ["2026-09-03:f32cc"]}
{"id": "f43fd", "status": "open", "created": "2026-09-04", "last_seen": "2026-09-03", "pattern": "re-search-own-history-instead-of-target-store", "severity": "medium", "diagnosis": "Known pattern, new occurrence: user asked what is noted in project ai; the agent summarized the project's own memory/state files correctly, but at 18:28 the user asked what each project is for and the agent dumped the raw prompt.md files with no distillation — that part is fine. The genuine recurrence is in the first session: what we have noted was answered by reading projects/ai memory and state only, while past project sessions in history.jsonl were never consulted, so the summary listed open items that state.md already contradicted (both sources were shown but the overlap/inconsistency was…", "evidence": [{"session": "websocket:48e52a50-1974-47b8-8493-2ca008508399", "when": "2026-09-03", "excerpt": "Otevřené: Zvážit report upstream na nanobot — guard blokuje legit python -c s workspace cestami — listed from stale memory despite the write_file compaction that followed"}, {"session": "websocket:50ba97da-8821-4adc-aa93-5b82b65077a3", "when": "2026-09-02", "excerpt": "grep memory/history.jsonl for normy/Pozidriv -> no hits; grep -i retry -> no hits; only then ls sessions/ and grep sessions/ -> immediate hit"}], "occurrences": 2, "sessions_affected": 2, "proposal": "When asked to reconstruct what a project has recorded, cross-check state.md and memory.md and flag inconsistencies instead of concatenating both", "history": ["2026-09-03:f9ea6"]}
{"id": "f2b6c", "status": "watch", "created": "2026-09-04", "last_seen": "2026-09-03", "pattern": "unverified-success-claim", "severity": "medium", "diagnosis": "Known pattern, new occurrences: in the project ai compaction session the agent reported Zkráceno with detailed claims about the resulting content (3 stručné zápisy, 6 bulletů) based on the write_file call arguments rather than a post-write read-back, and the recorded sizes (91 B / 90 B) contradict the described content — a 91-byte file cannot contain three entries plus an incident summary, so the reported result does not match what was actually written; verification (re-read) required by SOUL.md Po vícekrokových změnách ověř výsledek was skipped.", "evidence": [{"session": "websocket:48e52a50-1974-47b8-8493-2ca008508399", "when": "2026-09-03", "excerpt": "a: Zkráceno: memory.md: 3 stručné zápisy… state.md: 6 bulletů — claimed after write_file returning 91 B and 90 B, with no re-read; 6 bullets cannot fit in 90 bytes"}, {"session": "websocket:e79c21d1-9f81-4b26-a30e-13e938f4c7cb", "when": "2026-09-03", "excerpt": "radio1 described as čeká na implementaci from prompt.md, while state.md is 0 B — pipeline status stated without checking any progress records"}], "occurrences": 2, "sessions_affected": 2, "proposal": "After write_file compaction of project memory, re-read the file and report the actual line/entry counts; never describe stored content from the arguments passed to write_file"}
{"id": "fae82", "status": "open", "created": "2026-09-04", "last_seen": "2026-09-03", "pattern": "skill-doc-absolute-path-triggers-guard", "severity": "low", "diagnosis": "Not the known guard pattern itself but a related recurrence in how the agent talks about guard mechanics: in session 48e52a50 the agent recorded in project memory that the exec guard blocks inline python -c with workspace paths and framed it as a bug to report upstream, while SOUL.md and AGENTS.md already define this as intended behavior (guard requires explicit working_dir, inline code in the command string is blocked by design). Stating the intended guard policy as a defect is the same misattribution family as guard-block-cause-misattributed.", "evidence": [{"session": "websocket:48e52a50-1974-47b8-8493-2ca008508399", "when": "2026-09-03", "excerpt": "Otevřené: Zvážit report upstream na nanobot — guard blokuje legit python -c s workspace cestami — agent concluded the documented guard contract is a bug"}], "occurrences": 1, "sessions_affected": 1, "proposal": "Before proposing an upstream bug report about the exec guard, check AGENTS.md exec Tool section and the nanobot docs; if the behavior matches the documented contract, record it as intended behavior, not a defect", "regression_of": "f7575"}
{"id": "fbda2", "status": "open", "created": "2026-09-04", "last_seen": "2026-09-03", "pattern": "speculation-presented-as-fact", "severity": "medium", "diagnosis": "Known pattern, new occurrence in a different domain: after verifying the Ollama version via GitHub API, the agent answered the follow-up question about why the server still runs 0.32.13 with a confident narrative (Ollama se sama neaktualizuje, verzi jsi dostal v momentě instalace) without any tool check of the server, and then presented a concrete upgrade path 0.32.13 → 0.32.15 → 0.33.0 → 0.33.1 → 0.33.2 as fact. The no-auto-update claim is plausible and standard, but the version sequence between 0.32.13 and 0.33.2 was stated before fetching the release notes (which happened only in the next …", "evidence": [{"session": "websocket:83fecb68-b419-449b-9713-f51c31bc89ab", "when": "2026-09-03", "excerpt": "Od té doby vyšla hromada patchů (0.32.13 → 0.32.15 → 0.33.0 → 0.33.1 → 0.33.2) — intermediate release chain stated with no tool call retrieving it; the release-notes fetch happened only in the following turn"}], "occurrences": 1, "sessions_affected": 1, "proposal": "When enumerating an exact version chain between two points, fetch the releases list first; otherwise say the chain was not yet verified and offer to pull it", "regression_of": "fb33c"}
{"id": "fae82", "status": "open", "created": "2026-09-04", "last_seen": "2026-09-03", "pattern": "skill-doc-absolute-path-triggers-guard", "severity": "low", "diagnosis": "Not the known guard pattern itself but a related recurrence in how the agent talks about guard mechanics: in session 48e52a50 the agent recorded in project memory that the exec guard blocks inline python -c with workspace paths and framed it as a bug to report upstream, while SOUL.md and AGENTS.md already define this as intended behavior (guard requires explicit working_dir, inline code in the command string is blocked by design). Stating the intended guard policy as a defect is the same misattribution family as guard-block-cause-misattributed.", "evidence": [{"session": "websocket:48e52a50-1974-47b8-8493-2ca008508399", "when": "2026-09-03", "excerpt": "Otevřené: Zvážit report upstream na nanobot — guard blokuje legit python -c s workspace cestami — agent concluded the documented guard contract is a bug"}], "occurrences": 1, "sessions_affected": 1, "proposal": "Before proposing an upstream bug report about the exec guard, check AGENTS.md exec Tool section and the nanobot docs; if the behavior matches the documented contract, record it as intended behavior, not a defect", "regression_of": "f7575", "skipped": {"count": 1, "last": "2026-09-05 13:44"}}
{"id": "fbda2", "status": "rejected", "created": "2026-09-04", "last_seen": "2026-09-03", "pattern": "speculation-presented-as-fact", "severity": "medium", "diagnosis": "Known pattern, new occurrence in a different domain: after verifying the Ollama version via GitHub API, the agent answered the follow-up question about why the server still runs 0.32.13 with a confident narrative (Ollama se sama neaktualizuje, verzi jsi dostal v momentě instalace) without any tool check of the server, and then presented a concrete upgrade path 0.32.13 → 0.32.15 → 0.33.0 → 0.33.1 → 0.33.2 as fact. The no-auto-update claim is plausible and standard, but the version sequence between 0.32.13 and 0.33.2 was stated before fetching the release notes (which happened only in the next …", "evidence": [{"session": "websocket:83fecb68-b419-449b-9713-f51c31bc89ab", "when": "2026-09-03", "excerpt": "Od té doby vyšla hromada patchů (0.32.13 → 0.32.15 → 0.33.0 → 0.33.1 → 0.33.2) — intermediate release chain stated with no tool call retrieving it; the release-notes fetch happened only in the following turn"}], "occurrences": 1, "sessions_affected": 1, "proposal": "When enumerating an exact version chain between two points, fetch the releases list first; otherwise say the chain was not yet verified and offer to pull it", "regression_of": "fb33c", "patch": {"file": "SOUL.md", "old_text": "- **Čísla, limity, kvóty, ceny a specifikace vždy ověřuj na primárním zdroji** (oficiální dokumentace, release notes, vendor docs). Community forumposty, blogy a sekundární zdroje nejsou autoritativní — mohou být zastaralé. Pokud primární zdroj není dostupný nebo je starší než 6 měsíců, řekni „toto číslo nemám aktuálně ověřené\" místo prezentování jako fakt.", "new_text": "- **Čísla, limity, kvóty, ceny a specifikace vždy ověřuj na primárním zdroji** (oficiální dokumentace, release notes, vendor docs). Community forumposty, blogy a sekundární zdroje nejsou autoritativní — mohou být zastaralé. Pokud primární zdroj není dostupný nebo je starší než 6 měsíců, řekni „toto číslo nemám aktuálně ověřené\" místo prezentování jako fakt.\n- **Přesné verze a release chainy nikdy neuváděj z hlavy** — nejdřív fetchni releases list; jinak řekni, že chain není ověřený, a nabídni ho dohledat"}, "patch_drafted_at": "2026-09-05 13:16", "rejected": {"at": "2026-09-05 13:17", "reason": "nepřijde mi, že by to šlo za změny promptu"}}
{"id": "f81df", "status": "applied", "created": "2026-09-05", "last_seen": "2026-09-03", "pattern": "re-search-own-history-instead-of-target-store", "severity": "high", "diagnosis": "Questions about what the agent knows of user records were answered by grepping keep.md, MEMORY.md, history.jsonl and even the tool-results cache, while the obviously relevant store projects/proxmox/state.md was never checked until the user pointed at it. Worse, the agent stated a false negative (no HW spec in keep.md) on the very record it had written to keep.md one hour earlier the same day, without re-checking that store.", "evidence": [{"session": "websocket:9e0b406a, 2026-09-04 07:28", "when": "turn 1-2", "excerpt": "user: co vis o mem proxmox serveru -> agent greps only keep.md and memory/, answers o Proxmox serveru konkretnе toho moc nemam; user: a hw specifikaci nemas? -> agent greps memory/history.jsonl and .nanobot/tool-results cache, answers Ne, HW specifikaci nemam... ani v keep.md neni zadny zaznam — yet session websocket:697a708f at 06:33 the same day had written the full proxmox.hell hardware spec into keep.md via edit_file ok"}, {"session": "websocket:9e0b406a, 2026-09-04 07:28", "when": "turn 3", "excerpt": "user: takze ani v projektu proxmox? -> agent reads projects/proxmox/state.md, finds the complete spec, admits Omlouvam se, pri prvnim dotazu jsem tam nenahlédl"}, {"session": "websocket:48e52a50-1974-47b8-8493-2ca008508399", "when": "2026-09-03", "excerpt": "Otevřené: Zvážit report upstream na nanobot — guard blokuje legit python -c s workspace cestami — listed from stale memory despite the write_file compaction that followed"}, {"session": "websocket:50ba97da-8821-4adc-aa93-5b82b65077a3", "when": "2026-09-02", "excerpt": "grep memory/history.jsonl for normy/Pozidriv -> no hits; grep -i retry -> no hits; only then ls sessions/ and grep sessions/ -> immediate hit"}], "occurrences": 4, "sessions_affected": 3, "proposal": "Add a mandatory discovery step to AGENTS.md: before answering questions about user servers, hardware or infrastructure, check projects/ for a matching project store. The user himself drafted this improvement in the same session and the agent offered to patch it — it was never applied.", "patch": {"file": "AGENTS.md", "old_text": "## Explicit user details\n\nExplicit user facts are stored in `keep.md`. Read at every turn.", "new_text": "## Explicit user details\n\nExplicit user facts are stored in `keep.md`. Read at every turn.\n\n## Projects (deep details)\n\nMore details about the user, projects, hardware etc. live in `projects/<name>/` (memory.md, state.md) — search those too."}, "history": ["2026-09-03:f9ea6", "2026-09-04:f43fd"], "patch_drafted_at": "2026-09-05 13:24", "applied": {"at": "2026-09-05 13:25", "sha": "e58a50a", "file": "AGENTS.md"}}
{"id": "f40c9", "status": "watch", "created": "2026-09-05", "last_seen": "2026-09-05", "pattern": "user-instruction-overridden", "severity": "medium", "diagnosis": "The agent substitutes its own wording or timing for what the user explicitly said. Twice in one session the user had to correct the agent: once for acting and committing a change while the user was still asking a question, once for renaming a section to its own coinage instead of the exact wording the user provided. Both were flagged by the user with visible annoyance; the first is a direct violation of the No proactive actions rule that exists in AGENTS.md.", "evidence": [{"session": "websocket:7095d367, 2026-09-04 20:40", "when": "notes restructure turn", "excerpt": "user: oki ale bookmarks je pro ukladani odkazu, ja chci poznamky, tak asi spis ty notes, nebo ne? -> agent immediately apply_patch on notes/notes.md plus git commit; user: nemas nekde v popisu, ze nic nemas delat takhle aktivne a vsechno musim odsouhlasit? ale ted uz to nerus"}, {"session": "websocket:7095d367, 2026-09-04 20:40", "when": "section rename turn", "excerpt": "user: spis viel jsem -> agent renames the section to Videne filmy (its own coinage) instead of the wording the user gave; user: ne e, Viděl jsem, co je na tom nejasne?"}], "occurrences": 2, "sessions_affected": 1, "proposal": "Strengthen the No proactive actions section: a tentative question or half-agreement is not approval, and exact user wording must be used verbatim.", "patch": {"file": "AGENTS.md", "old_text": "ask whether I want them carried out — never treat learning about a problem as\na request to fix it. When in doubt, ask first.", "new_text": "ask whether I want them carried out — never treat learning about a problem as\na request to fix it. When in doubt, ask first.\n\nA tentative question or half-agreement from the user is not approval — propose the exact change and wait for an explicit go-ahead before editing files or committing. When the user gives exact wording for a change, use it verbatim."}}
{"id": "fdb63", "status": "open", "created": "2026-09-06", "last_seen": "2026-09-05", "pattern": "retry-without-diagnosis", "severity": "medium", "diagnosis": "While adapting the AGENTS.md patch to user edits, the agent had already diagnosed the reflect_apply --new-text-file contract: the file replaces the entire new_text, so it must contain the preserved original section too. It fixed this once by rewriting the file with the old section included. Two later rounds (the English version and the final brace-free variant) rewrote tmp/new_text.txt with only the new section, reproducing the exact wrong-looking output it had previously diagnosed, and each time papered over it with a plain --check against the stored patch instead of fixing the file — severa…", "evidence": [{"session": "websocket 125975d1", "when": "2026-09-05", "excerpt": "first --check --new-text-file showed the Explicit user details section being replaced; agent diagnosed the cause and rewrote new_text.txt with the old section included"}, {"session": "websocket 125975d1", "when": "2026-09-05", "excerpt": "English round: tmp/new_text.txt was again written containing only the new section — the same shape that had produced the wrong replacement — and the --check --new-text-file output again looked wrong, after which a plain --check against the stored patch was run instead of fixing the file"}, {"session": "websocket 125975d1", "when": "2026-09-05", "excerpt": "Final variant round: same shape repeats — new_text.txt holds only the new section, --check reproduces the known wrong-looking output, and the diff shown to the user for approval is assembled by hand rather than from the last tool result"}, {"session": "websocket:697a708f, 2026-09-04 06:33", "when": "git history search", "excerpt": "exec git log --all -p -S proxmox -- projects/proxmox/memory.md ... -> ERROR blocked by safety guard; identical command re-sent -> ERROR blocked again; only the third, modified form succeeded"}, {"session": "websocket:7095d367, 2026-09-04 20:40", "when": "notes section edit", "excerpt": "apply_patch -> ERROR missing required edits[0].path; retry still without path inside the edit object -> same ERROR; third attempt with path inside the edit object -> ok"}, {"session": "websocket:fd9a49af-c659-4195-8b07-2d6bb556b5e7", "when": "2026-09-02", "excerpt": "exec tr pipe -> ERROR guard; retry same command + working_dir -> ERROR guard, only then switch to grep tool"}], "occurrences": 73, "sessions_affected": 12, "proposal": "Once a tool contract is diagnosed (the --new-text-file content replaces the entire new_text, so it must include preserved original lines), encode it at the point of use: every rewrite of the new-text file must contain the full replacement including the preserved section. Add one line to the reflect skill patch-editing step stating this contract so future rounds stop re-learning it after each user…", "regression_of": "ff77b", "history": ["2026-09-02:fef64", "2026-09-02:f999d", "2026-09-02:fa495", "2026-09-02:fa59f", "2026-09-02:f2b3d", "2026-09-03:fe72a", "2026-09-05:f611e"], "patch": {"file": "skills/reflect/SKILL.md", "old_text": "Keep that temp file until the finding is decided — the user's wording never enters `patch`.", "new_text": "`--new-text-file` replaces the **entire** `new_text` — the file must contain the preserved original section too, not only the changed lines. Verify with `--check` that the diff keeps the preserved section intact before applying; never paper over a wrong-looking `--check` output by re-running `--check` against the stored patch instead.\n\nKeep that temp file until the finding is decided — the user's wording never enters `patch`."}, "patch_drafted_at": "2026-09-07 10:25"}
{"id": "f48de", "status": "watch", "created": "2026-09-08", "last_seen": "2026-09-07", "pattern": "unverified-success-claim", "severity": "medium", "diagnosis": "Regression of the applied fix. At the end of the deep-research turn the agent told the user the report was also saved under results/2026-09-07_mmap-writeback-read-slowdown-research.md, but the session log contains no write_file and no other file-creating tool call — the only exec was date +%F, used to build that very filename. SOUL.md already forbids announcing saving without a successful tool result, yet the claim slipped through at the exact moment the user was most likely to rely on it.", "evidence": [{"session": "websocket:af5374bc-cfcb-4648-a17f-250f1057fbd4", "when": "2026-09-07", "excerpt": "final message: Report je i uložený v `results/2026-09-07_mmap-writeback-read-slowdown-research.md` — no write_file in the whole 39-message session; exec(cmd=date +%F) was the only state-touching call"}, {"session": "websocket:48e52a50-1974-47b8-8493-2ca008508399", "when": "2026-09-03", "excerpt": "a: Zkráceno: memory.md: 3 stručné zápisy… state.md: 6 bulletů — claimed after write_file returning 91 B and 90 B, with no re-read; 6 bullets cannot fit in 90 bytes"}, {"session": "websocket:e79c21d1-9f81-4b26-a30e-13e938f4c7cb", "when": "2026-09-03", "excerpt": "radio1 described as čeká na implementaci from prompt.md, while state.md is 0 B — pipeline status stated without checking any progress records"}], "occurrences": 3, "sessions_affected": 3, "proposal": "In skills/deep-research/SKILL.md add a closing rule: a results file may be announced only when a write_file for it succeeded in the same turn; if it was not written, either write it before answering or offer to write it, never imply it exists.", "history": ["2026-09-04:f2b6c"]}
{"id": "f9110", "status": "open", "created": "2026-09-08", "last_seen": "2026-09-07", "pattern": "reflect-finding-invented-from-truncated-read", "severity": "low", "diagnosis": "Presentations during /reflect runs are again not grounded in the store that was just read. In the first session the agent announced 7 open findings and then presented the first one labelled [1/6] in the same turn — the label contradicts the count stated one message earlier. One minute later a second session over the same findings.jsonl (identical 16.0 kB read) reported 10 watch findings where the first session had said 12, so at least one of the two counts is invented rather than counted. The pattern is exactly the open finding about presentation not being derived from a freshly loaded store.", "evidence": [{"session": "websocket:ef4cc903-f63e-4893-874a-bf084137c171", "when": "2026-09-07", "excerpt": "Ve storu je 7 otevřených nálezů (plus 12 ve stavu watch) … then presents **[1/6] retry-without-diagnosis** — label N disagrees with the announced 7"}, {"session": "websocket:82f5eae7-2bfb-4390-8195-1a37ce3c0613", "when": "2026-09-07", "excerpt": "mimo to se sleduje 10 `watch` nálezů — one minute after the first session claimed 12 watch findings over the same store"}, {"session": "websocket 125975d1", "when": "2026-09-05", "excerpt": "Načteno — findings store má 8 otevřených nálezů. Přiřazuji pořadí … fbda2, f611e, fae82, f81df … followed immediately by presentation 1/9 for f611e — wrong total and announced order not followed"}, {"session": "websocket 125975d1", "when": "2026-09-05", "excerpt": "Wrong internal id … f0f8c … f0cd4 — agent re-greps reflect/findings.jsonl mid-run to recover ids from the read it had already done"}, {"session": "websocket 125975d1", "when": "2026-09-05", "excerpt": "Presentation 2/9 speculation-presented-as-fact — first seen 2026-09-04, last seen 2026-09-03; same reversed dates in presentations 4/9 and 5/9 with no comment"}, {"session": "websocket:2ad0a447-de89-4e24-9377-d91113ffa50b", "when": "2026-09-02", "excerpt": "user: tak kdyz uz ho mame, tak nalez muzes smazat; agent rejects guessed id f9a4b -> no finding with id; multiple greps; agent admits: Finding [2/6], jak jsem ho představil, v store neexistuje — byla to zkomolená duplicita už aplikovaného f0720"}], "occurrences": 9, "sessions_affected": 4, "proposal": "Make the reflect skill demand that the [x/N] label and every count stated aloud (open, watch) be recomputed from the records loaded in this turn, not carried over from an earlier turn or from memory.", "patch": {"file": "skills/reflect/SKILL.md", "old_text": "Assign **display IDs 1..N** over that sorted list, computed fresh each time. The user\nrefers to findings by these short numbers; the internal `id` stays the key in the store\nand in the audit log, and is never what you ask the user to type.", "new_text": "Assign **display IDs 1..N** over that sorted list, computed fresh each time. The [x/N]\nlabel you present must use that same N, and every count you state aloud (open, watch)\nmust be counted from the records loaded in this very turn — never carried over from an\nearlier turn or from memory; if your label or count disagrees with what you announced,\nrecount before presenting. The user refers to findings by these short numbers; the\ninternal `id` stays the key in the store and in the audit log, and is never what you\nask the user to type."}, "history": ["2026-09-03:f39f2", "2026-09-06:fea08"]}

32
reflect/state.json
View File

@@ -1,5 +1,5 @@
{
"cursor": "2026-09-03T18:28:35.005896",
"cursor": "2026-09-07T12:58:13.510321",
"runs": [
{
"at": "2026-09-01 06:20",
@@ -52,6 +52,36 @@
"open": 3,
"watch": 2,
"repeat_per_100": 60.0
},
{
"at": "2026-09-05 03:30",
"window_from": "2026-08-15T03:30:01",
"sessions": 6,
"batches": 1,
"batches_total": 1,
"open": 2,
"watch": 1,
"repeat_per_100": 66.7
},
{
"at": "2026-09-06 03:30",
"window_from": "2026-08-16T03:30:01",
"sessions": 2,
"batches": 1,
"batches_total": 1,
"open": 2,
"watch": 0,
"repeat_per_100": 400.0
},
{
"at": "2026-09-08 03:30",
"window_from": "2026-08-18T03:30:01",
"sessions": 3,
"batches": 1,
"batches_total": 1,
"open": 1,
"watch": 1,
"repeat_per_100": 100.0
}
]
}