upravy projektu a skillu
This commit is contained in:
lachtan
205
skills/reflect/SKILL.md
Normal file
205
skills/reflect/SKILL.md
Normal file
@@ -0,0 +1,205 @@
|
||||
---
|
||||
name: reflect
|
||||
description: >
|
||||
Review findings from your own self-diagnosis and apply approved fixes, one at a time.
|
||||
Triggers ONLY on the explicit `/reflect` command. Never infer this skill from
|
||||
conversational mentions of reflection, findings, self-improvement, or "what did you
|
||||
learn" — it can edit your own configuration, so it must never start by accident.
|
||||
---
|
||||
|
||||
# Reflect
|
||||
|
||||
Interactive review of findings produced by the unattended analysis run
|
||||
(`scripts/reflect_auto.py`, daily via cron). You show the user one finding at a time,
|
||||
wait for a decision, and apply only what they approve.
|
||||
|
||||
Answer in the language the user writes in.
|
||||
|
||||
## STOP gates — read before anything else
|
||||
|
||||
1. **This skill never runs the analysis.** Analysis is a cron script. If the user asks
|
||||
for a fresh analysis, tell them to run
|
||||
`~/.local/bin/uv run --script skills/reflect/scripts/reflect_auto.py` — do not
|
||||
distil sessions or diagnose patterns yourself in this turn.
|
||||
2. **Never edit anything without approval of that specific finding.** Not "the user
|
||||
seemed positive earlier", not "finding 3 is obviously right". One `ok` approves
|
||||
exactly one finding.
|
||||
3. **One finding per step.** Never present two findings at once, never apply a batch,
|
||||
even if the user says "apply everything" — in that case apply the first, show the
|
||||
result, and continue to the next.
|
||||
4. **Never edit a target file yourself — always through `scripts/reflect_apply.py`.**
|
||||
That script refuses a patch whose original text is missing or ambiguous, and commits
|
||||
only the file it touched. If it refuses, relay the reason; never hand-edit around it.
|
||||
**`reflect/findings.jsonl` is never edited by hand either — not with a one-off script,
|
||||
not with a heredoc, never.** It is the audit trail; every write to it belongs to
|
||||
`reflect_apply.py`. Drafting a patch during the review goes through `--set-patch`.
|
||||
|
||||
## Data
|
||||
|
||||
| Path | What |
|
||||
|---|---|
|
||||
| `reflect/findings.jsonl` | one JSON object per line — the findings store |
|
||||
| `reflect/state.json` | cursor and per-run statistics (owned by the script) |
|
||||
| `results/<date>_reflect.md` | full report of a run |
|
||||
| `log/reflect.log` | append-only audit of every decision — applied, rejected, skipped |
|
||||
|
||||
A finding has: `id`, `status`, `created`, `pattern`, `severity`, `diagnosis`,
|
||||
`last_seen`, `evidence`, `occurrences`, `sessions_affected`, `proposal`, optional `patch`
|
||||
(`file` / `old_text` / `new_text`), optional `patch_drafted_at` (the patch was drafted
|
||||
during a review, not proposed by the analysis), optional `regression_of`, optional `history`,
|
||||
and once decided one of `applied` (`at` / `sha` / `file`, plus `new_text` and
|
||||
`edited_by_user` when the user rewrote it), `rejected` (`at` / `reason`), `skipped`
|
||||
(`count` / `last`).
|
||||
|
||||
Statuses: `watch` (seen once, not worth the user's attention yet) · `open` (waiting for
|
||||
review) · `applied` · `rejected`.
|
||||
|
||||
Two things about the numbers, both of which you must not overstate to the user:
|
||||
|
||||
- A run analyses a **window** of recent sessions, not the whole history. `state.json`
|
||||
holds the window it used; the dated report repeats it in its header.
|
||||
- `occurrences` and `sessions_affected` are **cumulative across runs** — the sum of what
|
||||
the model counted in each slice, not a figure anyone measured over the whole corpus.
|
||||
`last_seen` is the newest date in the evidence, `history[0]` when the pattern was first
|
||||
filed, `created` only when the record was last rewritten. Older records may lack
|
||||
`last_seen`; fall back to `created` for those.
|
||||
|
||||
## Procedure
|
||||
|
||||
### 1. Load
|
||||
|
||||
Read `reflect/findings.jsonl`. Take the records with `status: open`, sorted by severity
|
||||
(`high`, `medium`, `low`), then by `last_seen` and then by `occurrences`, all descending.
|
||||
`occurrences` alone would let a stale pattern with a large cumulative count outrank a
|
||||
fresh one.
|
||||
|
||||
A finding whose `last_seen` is older than the window of the latest run (`state.json`,
|
||||
`window_from`) is **stale**: no run looks that far back any more, so nothing will refresh
|
||||
it. Say so when you present it — it is a candidate for rejection rather than a patch.
|
||||
|
||||
Assign **display IDs 1..N** over that sorted list, computed fresh each time. The user
|
||||
refers to findings by these short numbers; the internal `id` stays the key in the store
|
||||
and in the audit log, and is never what you ask the user to type.
|
||||
|
||||
If there are none: say so, mention how many `watch` findings are being tracked, and stop.
|
||||
|
||||
### 2. Present one finding
|
||||
|
||||
Show exactly one, in this shape:
|
||||
|
||||
```text
|
||||
[1/4] retry-without-diagnosis · medium · 7× in 4 sessions
|
||||
first seen 2026-09-01, last seen 2026-09-02 · deferred 2× already
|
||||
<diagnosis>
|
||||
|
||||
Evidence:
|
||||
- websocket_e5a6aa… (2026-07-11) — web_fetch → ERROR 403 ×4
|
||||
- …
|
||||
|
||||
Proposal: <proposal>
|
||||
```
|
||||
|
||||
Write the labels in the user's language, not necessarily as shown here.
|
||||
|
||||
The second line carries what the count alone hides: `history[0]` for when the pattern
|
||||
was first filed, `last_seen` for when it last actually occurred, and `skipped.count` when
|
||||
the user has already deferred it. Drop the skip part when there is none; a finding deferred
|
||||
several times is worth saying so about, because rejecting it is cleaner than a list that
|
||||
keeps re-presenting it.
|
||||
|
||||
Mark a finding with `regression_of` clearly as a **regression** — this pattern was fixed
|
||||
before and came back after the fix. Mark a stale finding (see step 1) as such too, and
|
||||
say what it means: the evidence predates the current window, so the pattern may well be
|
||||
gone already.
|
||||
|
||||
If the finding has a `patch`, get the diff from the script — never by reading the file and
|
||||
judging for yourself, that is the duplication gate 4 exists to prevent:
|
||||
|
||||
```bash
|
||||
uv run --script skills/reflect/scripts/reflect_apply.py --id <internal id> --check
|
||||
```
|
||||
|
||||
It changes nothing and prints the diff; show the user what it printed. Exit code 2 means the
|
||||
patch no longer applies — relay the printed reason and offer only *skip* / *reject*.
|
||||
|
||||
Then ask for a decision and wait.
|
||||
|
||||
### 3. Accept a decision
|
||||
|
||||
| Input | Meaning |
|
||||
|---|---|
|
||||
| `ok`, `apply` | apply this finding's patch |
|
||||
| `edit: <text>` | the user rewrites `new_text`; show the new diff and ask again |
|
||||
| `skip` | record the deferral, leave it `open`, move to the next |
|
||||
| `reject` | **ask why first**, then close it for good — it never opens again |
|
||||
| `stop` | end the review |
|
||||
|
||||
Accept the equivalents in whatever language the user writes in — the words above are the
|
||||
meanings, not a required vocabulary.
|
||||
|
||||
Rejection needs a reason and the script will not take it without one. Ask for it in one
|
||||
short question and pass the user's own words through — half of the first findings were
|
||||
rejected, and that number only says something about the analysis if the reasons are on
|
||||
the record. Do not invent a reason, and do not talk the user out of rejecting.
|
||||
|
||||
A finding without a `patch` cannot be applied. Offer to draft one, and file it with
|
||||
`--set-patch` — **never by writing to `reflect/findings.jsonl` yourself** (gate 4):
|
||||
|
||||
```bash
|
||||
# {"file": "SOUL.md", "old_text": "<copied character-for-character>", "new_text": "<the fix>"}
|
||||
uv run --script skills/reflect/scripts/reflect_apply.py --id <internal id> --set-patch tmp/patch.json
|
||||
```
|
||||
|
||||
`old_text` must be copied from the current file and occur in it exactly once — copy it,
|
||||
never retype it, and watch the quotation marks. The script verifies the patch *before*
|
||||
storing it, so a refusal (exit 2) leaves the finding exactly as it was; relay the reason
|
||||
and draft again. On success it prints the diff — show that, then ask for the `ok`. It
|
||||
changes no file and decides nothing, so applying still needs step 4.
|
||||
|
||||
### 4. Apply (only after `ok`)
|
||||
|
||||
**Never edit the file yourself.** `scripts/reflect_apply.py` does it, and it is what
|
||||
enforces the gates — it refuses a stale or ambiguous patch, commits only the touched
|
||||
file, writes the audit line and updates the store. Editing by hand would bypass all of it.
|
||||
|
||||
```bash
|
||||
uv run --script skills/reflect/scripts/reflect_apply.py --id <internal id>
|
||||
```
|
||||
|
||||
- The `--id` is the finding's internal `id`, not the display number you showed the user.
|
||||
- User rewrote the text (`edit:`)? Write their version to a temp file and pass
|
||||
`--new-text-file <path>`.
|
||||
- *skip* → `--skip`. Changes no file and no status; it only counts the deferral.
|
||||
- *reject* → `--reject --reason "<user's words>"`. It changes no file.
|
||||
- Exit code 2 means refused: relay the printed reason and move on. Do not work around it,
|
||||
do not edit the file to make the patch fit.
|
||||
|
||||
On success the script prints the commit SHA and the revert command. Pass that on, then
|
||||
continue with the next finding.
|
||||
|
||||
### 5. Close
|
||||
|
||||
When the user stops or the list is exhausted, summarise: how many applied, skipped,
|
||||
rejected, and how many remain `open`. If anything was applied, remind the user that the
|
||||
change lives only on the server and should be pulled back into the tracking repo.
|
||||
|
||||
## Decision history
|
||||
|
||||
Asked what was already decided, or what a past decision changed: read `log/reflect.log`
|
||||
— it is the source of truth for the list, one line per decision, oldest first. Then fill
|
||||
each line in from `findings.jsonl`:
|
||||
|
||||
- applied → the `patch` (the model's proposal), `applied.new_text` when the user rewrote
|
||||
it, and `revert: git revert <applied.sha>`. Use `git show <sha>` for the real diff.
|
||||
- rejected → `rejected.reason`. Records decided before reasons existed carry a flat
|
||||
`rejected_at` and no reason; say the reason is missing rather than guessing one.
|
||||
- skipped → `skipped.count`, still open.
|
||||
|
||||
Do not reconstruct this list from memory or from the findings store alone.
|
||||
|
||||
## When a finding is wrong
|
||||
|
||||
Findings come from an LLM reading its own logs and can be plain wrong. That is expected —
|
||||
rejection is a normal outcome, not a failure. If several findings in a row are noise, say
|
||||
so plainly; that is a signal the analysis prompt needs tuning, and it is worth telling the
|
||||
user rather than working through a list of nonsense.
|
||||
Reference in New Issue
Block a user