lachtan/nanobot-runtime
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

nanobot: 2026-09-14 12:02:18

Browse Source
This commit is contained in:
lachtan
2026-09-14 12:02:19 +02:00
parent 15a735e9bf
commit 59bddbff63
10 changed files with 169 additions and 82 deletions
Download Patch File Download Diff File Expand all files Collapse all files

3
memory/history.jsonl
View File

@@ -423,3 +423,6 @@
{"cursor": 423, "timestamp": "2026-08-28 13:26", "content": "- [ephemeral] User switched model preset to `kimi27` (`kimi-k2.7-code:cloud`) for the current session.", "session_key": "websocket:f6e1e265-7a37-451c-ad7a-f601c40fdc5a"}
{"cursor": 424, "timestamp": "2026-08-31 09:17", "content": "- [skip] Background cron drain task: compiled 17 inbox files from notes/inbox/ into notes/notes.md (created new, 7 thematic sections), moved all sources to notes/done/\n- [skip] Per note SKILL.md, the notes/ store is separate from agent memory (keep/MEMORY.md) — Dream must not touch notes/, so the compiled note contents (chata shopping list, DT Glass product details, ZOT work items, 25.5 cm chair-to-table measurement, etc.) live only in notes/notes.md and are not mirrored here\n- [skip] Two DT Glass product URLs (UNIVERSAL and Amber wine-bottle glasses, 69 Kč) fetched successfully — details filed under \"## DT Glass\" section in notes/notes.md\n- [skip] notes/notes.md did not exist before this compile; it was created fresh with sections: DevOps/Infra, Nanobot, Chata, DT Glass, Měření, 3D tisk, Work/ZOT", "session_key": "note-compile"}
{"cursor": 425, "timestamp": "2026-09-08 20:54", "content": "(nothing)", "session_key": "telegram:8826147089"}
{"cursor": 426, "timestamp": "2026-09-14 08:33", "content": "- [ephemeral] Active plan: run qwen3-embedding:0.6b as a static llama.cpp server on nvidia.hell:8081 (instead of via ollama) to isolate wiki embeddings from ollama's unstable load path; parity test script prepared at tmp/embed-parity-test.py — embed same chunks via both endpoints, cosine > 0.999 → switch wiki/config.yaml endpoint without reindex\n- [durable] qwen3-embedding:0.6b GGUF can be reused from ollama blob store: /usr/share/ollama/.ollama/models/blobs/sha256-06507c7b42688469c4e7298b0a1e16deff06caf291cf0a5b278c308249c3e439 — copy to own path (e.g. ~/models/) before use, ollama pull can swap the blob\n- [durable] On nvidia.hell, `llama` binary v0.4.0-dev build 10909 installed via official `curl -LsSf https://llama.app/install.sh | sh` (llama.app is the official llama.cpp installer; GitHub nightly releases have NO Linux CUDA build, CUDA prebuilts exist only via llama.app on Linux)\n- [durable] Verified `llama serve` on nvidia.hell supports all required embedding flags: --embeddings, --pooling {none,mean,cls,last,rank}, -c/--ctx-size, -ngl; launch command: `llama serve -m ~/models/qwen3-embedding-0.6b.gguf --embeddings --pooling last -ngl 999 -c 1024 --host 0.0.0.0 --port 8081` (needs --host 0.0.0.0 so nanobot can reach it)\n- [correction] nanobot.hell is NOT a persistent load source on local ollama — the batch requests over ~10 models were one-time tool-calling tests; in normal operation nanobot uses local ollama only for (a) wiki embedding (qwen3-embedding:0.6b) and (b) :cloud model proxying — report todo #1 \"stop nanobot batches\" is not an operational risk\n- [durable] Ollama on nvidia.hell root cause (from two saved Claude Code reports, artifacts in projects/devops/artifacts/): broken mmap decision + load interruption leaks pinned host memory (only reboot frees it, not rmmod); verified mitigation is cgroup limit MemoryMax=6G + MemorySwapMax=0 (no MemoryHigh); earlyoom rejected — external killer makes the leak worse\n- [durable] Embedding models need KV buffer sized only for max input length (-c 1024 suffices for wiki chunks); llama.cpp default reads full context from model metadata and over-allocates for embedding use\n- [skip] User will confirm when the llama.cpp server is running so the agent can run the parity test; exact HF repo name for Qwen3-Embedding GGUF not confirmed from memory", "session_key": "websocket:0d81522a-4774-490a-a2c3-10d72dcd4680"}
{"cursor": 427, "timestamp": "2026-09-14 08:42", "content": "- [durable] Ollama vs llama.cpp embedding parity test for qwen3-embedding:0.6b (GGUF blob sha256-06507c7b...) failed — cosine only 0.87–0.93, not >0.999. Special tokens (EOS 151643, `</s>` 128247), BOS, chat templates (`<|im_start|>`), query prefixes, and normalization were all systematically ruled out. Main hypothesis: ollama uses mean pooling while llama.cpp server runs `--pooling last`; next step is restarting llama server with `--pooling mean` and re-running the parity test.\n- [durable] Model oddity: the GGUF blob has `</s>` at token 128247 flagged as \"control-looking token, probably a bug in the model\"; llama.cpp auto-appends Qwen EOS 151643 on tokenize (add_special=true).\n- [durable] llama.cpp server per-request `pooling` field in `/embedding` payload is ignored — server-level `--pooling` flag wins.\n- [durable] Workspace pitfall: `tmp/queue.py` in the nanobot workspace shadows Python stdlib `queue` module (breaks urllib3 imports in uv scripts); run test scripts from a separate subdirectory like `tmp/embed-test/` instead.\n- [durable] llama.cpp `/v1/embeddings` response shape: top-level is a list of items with keys `['index', 'embedding']` (OpenAI-style), not `{\"embedding\": ...}`.\n- [durable] Environment: ollama at http://nvidia.hell:11434 (qwen3-embedding:0.6b), llama.cpp server at http://nvidia.hell:8081; llama server run with `--embeddings --pooling last -ngl 999 -c 1024 --host 0.0.0.0 --port 8081`, n_slots=4.\n- [permanent] User preference: do not switch the wiki embedding pipeline from ollama to llama.cpp without resolving the parity issue (decision: \"NEPREPINAT bez reseni\").", "session_key": "websocket:0d81522a-4774-490a-a2c3-10d72dcd4680"}
{"cursor": 428, "timestamp": "2026-09-14 08:44", "content": "- [durable] llama.cpp server and ollama produce fundamentally different embeddings for `qwen3-embedding:0.6b` even on identical single-token input (`cat` → cosine 0.36); parity by reusing the ollama blob is not achievable — the forward pass itself yields different vectors.\n- [durable] Both engines read the same blob `sha256-06507c7b…` (Q8_0, `general.file_type=7`, `pooling_type=3`), so quantization was ruled out as the cause; suspected different GGUF build/tokenizer handling in ollama's vendored llama.cpp.\n- [durable] `tmp/queue.py` in the nanobot workspace shadows the stdlib `queue` module and breaks Python scripts importing `requests`/`urllib3` — run scripts from a subdirectory (e.g. `tmp/embed-test/`) instead.\n- [durable] ollama 0.34.0 at `nvidia.hell:11434` has no `/api/tokenize` endpoint; `/api/embed` options `raw`, `truncate`, `num_ctx` (512/1024/8192) have zero effect on output vectors.\n- [durable] llama.cpp server is fully deterministic (5× same input → cosine 1.000000) and `--pooling last` is the best-fitting pooling for qwen3-embedding (mean pooling measured worse: 0.50–0.71 vs 0.87–0.93 cosine).\n- [durable] Reference: GitHub issue ollama/ollama#14121 documents the same qwen3-embedding mismatch; note llama-cpp-python `n_batch=512` chunk-splitting corrupts embeddings for inputs >512 tokens (not the cause here, inputs were ~79 tokens).\n- [ephemeral] Decision pending: three paths offered for the wiki embedding index — (1) pull the same GGUF directly in llama.cpp and retest parity, (2) reindex wiki with llama.cpp server (port 8081, `--pooling last`) as the new source of truth (assistant's recommendation), (3) stay on ollama for embeddings (loses the goal of isolating embeddings from ollama's load-path instability). User has not yet chosen.\n- [skip] Context: this debugging is part of an ongoing embed-parity test effort to migrate embeddings away from ollama's unstable load path.", "session_key": "websocket:0d81522a-4774-490a-a2c3-10d72dcd4680"}