Files
nanobot-runtime/skills/reflect/SKILL.md
2026-09-02 10:36:37 +02:00

9.7 KiB
Raw Blame History

name, description
name description
reflect Review findings from your own self-diagnosis and apply approved fixes, one at a time. Triggers ONLY on the explicit `/reflect` command. Never infer this skill from conversational mentions of reflection, findings, self-improvement, or "what did you learn" — it can edit your own configuration, so it must never start by accident.

Reflect

Interactive review of findings produced by the unattended analysis run (scripts/reflect_auto.py, daily via cron). You show the user one finding at a time, wait for a decision, and apply only what they approve.

Answer in the language the user writes in.

STOP gates — read before anything else

  1. This skill never runs the analysis. Analysis is a cron script. If the user asks for a fresh analysis, tell them to run ~/.local/bin/uv run --script skills/reflect/scripts/reflect_auto.py — do not distil sessions or diagnose patterns yourself in this turn.
  2. Never edit anything without approval of that specific finding. Not "the user seemed positive earlier", not "finding 3 is obviously right". One ok approves exactly one finding.
  3. One finding per step. Never present two findings at once, never apply a batch, even if the user says "apply everything" — in that case apply the first, show the result, and continue to the next.
  4. Never edit a target file yourself — always through scripts/reflect_apply.py. That script refuses a patch whose original text is missing or ambiguous, and commits only the file it touched. If it refuses, relay the reason; never hand-edit around it. reflect/findings.jsonl is never edited by hand either — not with a one-off script, not with a heredoc, never. It is the audit trail; every write to it belongs to reflect_apply.py. Drafting a patch during the review goes through --set-patch.

Data

Path What
reflect/findings.jsonl one JSON object per line — the findings store
reflect/state.json cursor and per-run statistics (owned by the script)
results/<date>_reflect.md full report of a run
log/reflect.log append-only audit of every decision — applied, rejected, skipped

A finding has: id, status, created, pattern, severity, diagnosis, last_seen, evidence, occurrences, sessions_affected, proposal, optional patch (file / old_text / new_text), optional patch_drafted_at (the patch was drafted during a review, not proposed by the analysis), optional regression_of, optional history, and once decided one of applied (at / sha / file, plus new_text and edited_by_user when the user rewrote it), rejected (at / reason), skipped (count / last).

Statuses: watch (seen once, not worth the user's attention yet) · open (waiting for review) · applied · rejected.

Two things about the numbers, both of which you must not overstate to the user:

  • A run analyses a window of recent sessions, not the whole history. state.json holds the window it used; the dated report repeats it in its header.
  • occurrences and sessions_affected are cumulative across runs — the sum of what the model counted in each slice, not a figure anyone measured over the whole corpus. last_seen is the newest date in the evidence, history[0] when the pattern was first filed, created only when the record was last rewritten. Older records may lack last_seen; fall back to created for those.

Procedure

1. Load

Read reflect/findings.jsonl. Take the records with status: open, sorted by severity (high, medium, low), then by last_seen and then by occurrences, all descending. occurrences alone would let a stale pattern with a large cumulative count outrank a fresh one.

A finding whose last_seen is older than the window of the latest run (state.json, window_from) is stale: no run looks that far back any more, so nothing will refresh it. Say so when you present it — it is a candidate for rejection rather than a patch.

Assign display IDs 1..N over that sorted list, computed fresh each time. The user refers to findings by these short numbers; the internal id stays the key in the store and in the audit log, and is never what you ask the user to type.

If there are none: say so, mention how many watch findings are being tracked, and stop.

2. Present one finding

Show exactly one, in this shape:

[1/4] retry-without-diagnosis · medium · 7× in 4 sessions
first seen 2026-09-01, last seen 2026-09-02 · deferred 2× already
<diagnosis>

Evidence:
- websocket_e5a6aa… (2026-07-11) — web_fetch → ERROR 403 ×4
- …

Proposal: <proposal>

Write the labels in the user's language, not necessarily as shown here.

The second line carries what the count alone hides: history[0] for when the pattern was first filed, last_seen for when it last actually occurred, and skipped.count when the user has already deferred it. Drop the skip part when there is none; a finding deferred several times is worth saying so about, because rejecting it is cleaner than a list that keeps re-presenting it.

Mark a finding with regression_of clearly as a regression — this pattern was fixed before and came back after the fix. Mark a stale finding (see step 1) as such too, and say what it means: the evidence predates the current window, so the pattern may well be gone already.

If the finding has a patch, get the diff from the script — never by reading the file and judging for yourself, that is the duplication gate 4 exists to prevent:

uv run --script skills/reflect/scripts/reflect_apply.py --id <internal id> --check

It changes nothing and prints the diff; show the user what it printed. Exit code 2 means the patch no longer applies — relay the printed reason and offer only skip / reject.

Then ask for a decision and wait.

3. Accept a decision

Input Meaning
ok, apply apply this finding's patch
edit: <text> the user rewrites new_text; show the new diff and ask again
skip record the deferral, leave it open, move to the next
reject ask why first, then close it for good — it never opens again
stop end the review

Accept the equivalents in whatever language the user writes in — the words above are the meanings, not a required vocabulary.

Rejection needs a reason and the script will not take it without one. Ask for it in one short question and pass the user's own words through — half of the first findings were rejected, and that number only says something about the analysis if the reasons are on the record. Do not invent a reason, and do not talk the user out of rejecting.

A finding without a patch cannot be applied. Offer to draft one, and file it with --set-patchnever by writing to reflect/findings.jsonl yourself (gate 4):

# {"file": "SOUL.md", "old_text": "<copied character-for-character>", "new_text": "<the fix>"}
uv run --script skills/reflect/scripts/reflect_apply.py --id <internal id> --set-patch tmp/patch.json

old_text must be copied from the current file and occur in it exactly once — copy it, never retype it, and watch the quotation marks. The script verifies the patch before storing it, so a refusal (exit 2) leaves the finding exactly as it was; relay the reason and draft again. On success it prints the diff — show that, then ask for the ok. It changes no file and decides nothing, so applying still needs step 4.

4. Apply (only after ok)

Never edit the file yourself. scripts/reflect_apply.py does it, and it is what enforces the gates — it refuses a stale or ambiguous patch, commits only the touched file, writes the audit line and updates the store. Editing by hand would bypass all of it.

uv run --script skills/reflect/scripts/reflect_apply.py --id <internal id>
  • The --id is the finding's internal id, not the display number you showed the user.
  • User rewrote the text (edit:)? Write their version to a temp file and pass --new-text-file <path>.
  • skip--skip. Changes no file and no status; it only counts the deferral.
  • reject--reject --reason "<user's words>". It changes no file.
  • Exit code 2 means refused: relay the printed reason and move on. Do not work around it, do not edit the file to make the patch fit.

On success the script prints the commit SHA and the revert command. Pass that on, then continue with the next finding.

5. Close

When the user stops or the list is exhausted, summarise: how many applied, skipped, rejected, and how many remain open. If anything was applied, remind the user that the change lives only on the server and should be pulled back into the tracking repo.

Decision history

Asked what was already decided, or what a past decision changed: read log/reflect.log — it is the source of truth for the list, one line per decision, oldest first. Then fill each line in from findings.jsonl:

  • applied → the patch (the model's proposal), applied.new_text when the user rewrote it, and revert: git revert <applied.sha>. Use git show <sha> for the real diff.
  • rejected → rejected.reason. Records decided before reasons existed carry a flat rejected_at and no reason; say the reason is missing rather than guessing one.
  • skipped → skipped.count, still open.

Do not reconstruct this list from memory or from the findings store alone.

When a finding is wrong

Findings come from an LLM reading its own logs and can be plain wrong. That is expected — rejection is a normal outcome, not a failure. If several findings in a row are noise, say so plainly; that is a signal the analysis prompt needs tuning, and it is worth telling the user rather than working through a list of nonsense.