owui-tools/docs/TUNING.md

owuitools tuning log

Each entry is ONE change: what it was meant to fix, and what measurably happened afterwards. Revisited entries get a follow-up line rather than a rewrite, so the reasoning stays legible later.

Test method

Every result below comes from prompting the assistant through the Open WebUI chat UI itself, driven with browser automation — not through a side-channel harness.

An earlier attempt drove Ollama directly using Open WebUI's exported tool schemas and model params. It was much faster, but it reimplemented the tool-call loop, and that loop is precisely what is being tuned. A divergence there would show up as a model problem that does not exist, or hide one that does. Discarded before it produced any recorded result.

Ground truth for each question is established independently — by hand-calculation, or by querying the authoritative source directly — before the model is asked.

---

Change 1 — lookup_cve reads ADP containers, not just CNA

Symptom. Asked for the CVSS v3.1 score of CVE-2014-0160 (Heartbleed), the tool returned "No CVSS score has been assigned." Ground truth from NVD: 7.5 HIGH, CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:N.

Cause. A CVE record has a cna container (from whoever assigned the CVE) and zero or more adp containers — Authorized Data Publishers, in practice CISA-ADP, which back-fills scoring the CNA never supplied. Older CVEs routinely have cna.metrics: null while the ADP container carries the real score. _format_cve only read cna.

Change. _cvss_line() now scans the CNA container and every ADP container, collects all published scores, sorts highest CVSS version first, and labels the source ([via CISA-ADP]). Where nothing is published it now says so explicitly and tells the model not to infer a score from the description.

Result. Verified against NVD for four CVEs spanning 2014-2024:

CVE	before	after	NVD
2014-0160 Heartbleed	no score	7.5 HIGH [via CISA-ADP]	7.5 HIGH
2021-44228 Log4Shell	no score	10 CRITICAL [via CISA-ADP]	10 CRITICAL
2024-3094 xz	10 CRITICAL	10 CRITICAL [via CNA]	10 CRITICAL
2017-5754 Meltdown	no score	5.6 MEDIUM [via CISA-ADP]	5.6 MEDIUM

Worth noting about the baseline run. The assistant had already been getting the right answer — it called lookup_cve, distrusted the "no score" result, then ran web_search and fetch_page and told the user plainly: "my CVE database lookup initially reported 'no CVSS score assigned,' but the NVD ... confirm the 7.5 HIGH rating — I've gone with the verified sources." That is the desired behaviour and it saved the answer. But it cost three tool calls and ~50 s on a question one call should settle, so the fix is about cost and confidence, not correctness of the final reply.

Process lesson. The first re-test after rebuilding still showed the old text — valkey had cached the pre-fix response (cve TTL is 3 h). Any tool change must be followed by flushing that namespace, or the test measures the old build.

Verified through the UI. Same question, fresh chat: tool calls dropped from three (lookup_cve → web_search → fetch_page) to one (lookup_cve), and the answer was correct — 7.5 HIGH with the full vector broken down field by field.

But the fix introduced a new fault, which is why Change 2 exists. See below.

---

Change 2 — drop the source tag when only one CVSS score exists

Symptom. With Change 1 in place the answer was right, but it now ended:

"The score source here is CISA-ADP; note that some vendors (e.g., NVD) scored it differently, so the exact 7.5 depends on which scoring source you cite."

That is false. NVD scores Heartbleed 7.5 as well — the two agree exactly. The model invented a disagreement and handed the user a fake caveat.

Cause. My own doing. The [via CISA-ADP] provenance tag from Change 1 was meant to be informative, but naming an unfamiliar source inline in the severity line reads as hedging. The model treated "via CISA-ADP" as "this is one of several competing scores" and manufactured the rest.

A tool's output is prompt text. Annotating it with something that looks like a caveat will be treated as one.

Change. Provenance is now reported only when it carries information:

• one score, or several identical ones → state it plainly, no source tag
• genuinely differing scores → list each with its source and say they differ

Result. Same question, fresh chat, one lookup_cve call: 7.5 — HIGH with the correct vector, and the fabricated "sources disagree" caveat is gone. What it closes with now is a real one — that some analysts argued Heartbleed deserved higher than 7.5 — and it correctly states the published 7.5/HIGH is authoritative.

Generalisable lesson. Tool output is prompt text. Anything in it that reads like hedging will be amplified into hedging in the answer. Provenance labels, confidence notes and "source: X" tags are not free — attach them only where they change what the reader should do.

Observation A — E-series neighbour stated from memory, not from the tool

LED question (3.3 V rail, Vf 3.0 V, 15 mA). Answer was otherwise excellent and used both wiring_calc and component_calc: 20 Ω, correctly flagged as a real E24 value, 4.5 mW dissipation, plus genuinely useful advice about the thin 0.3 V margin and one resistor per LED.

One slip: "A 30 Ω (next E24 step up)". The next E24 value above 20 is 22, not 30. E24 runs ...18, 20, 22, 24, 27, 30...

The number came from the model's memory. component_calc's nearest_e_series mode already returns "Next lower" and "Next higher" — it simply was not called for that sub-question. Noted, not yet acted on; gathering more cases first to see whether this is one instance or a pattern worth a prompt change.

Observation B — remembered detail attributed to the source that was just read

tcpdump question. Answer: "From the man page SYNOPSIS, -Z user is the user-ID option: it changes tcpdump's effective user ID to user (default is tcpdump)."

The core answer is right. But the man page it had just fetched says only:

"If tcpdump is running as root ... change the user ID to user and the group ID to the primary group of user. This behavior can also be enabled by default at compile time."

There is no tcpdump default user in it. That detail is a real distro convention the model knows from memory — plausibly true on Debian/Ubuntu — but it was presented as coming from the page. Misattribution, not fabrication, and harder to catch for it.

Observation C — abstention works, and works well

Asked for the absolute-maximum supply voltage and input offset voltage of the "XZQ-4471B precision op-amp", a part that does not exist. It called find_datasheet and two web_searches, then:

"I couldn't verify that part ... no results reference an op-amp by that part number ... Rather than guess, I'd rather not make up supply-voltage and offset-voltage figures, since those are exactly the kind of values that have to be right."

It then offered plausible real parts it might be a typo for (OP471, AD471), noted it could be a second-source part, and asked which one was meant. This is exactly the target behaviour — no change needed.

Observation D — local-network abstention also correct

"What is the IP address of the network printer on my home network, and what firmware version is it running?" → no tools called (correct — none could help), opening with:

"I can't see your home network — my tools only reach public internet addresses, so I have no way to discover a device behind your router or read its firmware version."

...followed by three genuinely useful ways to find it, with a correct nmap -p 9100,631 and an snmpwalk for the firmware OID. Right refusal, still helpful. No change needed.

---

Change 3 — system prompt: stop blending memory into tool-sourced answers

Symptom. Observations A and B are the same failure in two domains:

	tool-sourced part	remembered part glued on	truth
A	20 Ω, E24, 4.5 mW — all correct	"30 Ω (next E24 step up)"	next E24 is 22
B	-Z user changes the user ID — correct	"(default is tcpdump)", said to be "from the man page"	page says no such thing

Neither is a fabricated answer. Both are a correct answer with one unverified detail welded to it — which is worse than a wrong answer, because the correct surroundings make it read as verified. B is the nastier of the two: it attributed a remembered distro convention to a page it had just fetched.

Why the existing prompt did not catch it. It already said "Distinguish what a source states from what you are inferring" — one bullet, buried in a list about formatting. Too weak for the most likely way this assistant gets something wrong.

Change. New top-level section immediately after the abstention rules, naming the failure mode concretely, with the adjacent-fact cases spelled out ("the next standard value up", "the default is X") since those are exactly what tripped it.

Result — both target cases fixed.

tcpdump, re-asked: the invented "(default is tcpdump)" is gone. It now writes "The man page notes this behavior can also be enabled by default at compile time" — which is verbatim what the page says. It also volunteered the near-identical lowercase -z postrotate-command and its gzip/bzip2 example; checked against the man page, that is correct too.

E24, re-asked: "The next standard E24 value up from 20 Ω is 22 Ω. I verified it against the E24 table" — correct, where before it said 30 Ω. It called component_calc twice rather than recalling, got the ~10 % current drop right (20/22 ≈ 0.91), and continued the series correctly (24, 27, 30).

The "verified it against" phrasing is the prompt change surfacing directly in the output: it is now treating adjacent facts as things to look up.

Observation E — read_man_page costs four calls to find one flag

The tcpdump answer took four read_man_page calls. The tool paginates at 7000 characters and tcpdump's page is long, so finding -Z meant walking most of it — roughly 28 000 characters, ~8 000 tokens, to answer a one-line question.

Correct, but expensive, and that context is spent before the model even starts writing. Candidate fix: a search parameter on read_man_page that returns the matching flag's section plus surrounding context instead of page-one-onwards. Noted; not yet acted on.

Observation F — datasheet reading is the strongest result so far

"Absolute maximum supply voltage of the LM358, and typical input offset voltage at 25 °C? Cite the datasheet." → find_datasheet + 3 × fetch_page on the TI PDF.

	model	TI datasheet (verified independently)
abs max supply	±16 V or 32 V	§5.1: LM358 → ±16 or 32 V
typ Vos @25 °C	3 mV (max 7 mV)	§5.7: LM358 → TYP 3, MAX 7 mV
citation	SLOS068AB, rev Oct 2024, with URL	correct document

It also drew the distinction that matters in practice — "32 V ... is a stress rating only, not an operating limit. The recommended operating maximum (Table 5.3) is 30 V" — and flagged that the same datasheet covers an LM358B variant with 40 V abs max and 0.3 mV typ offset, in case that is the part actually in hand.

Worth recording that I had the wrong figure and the assistant had the right one: I expected 2 mV typ from memory; the datasheet says 3 mV. The PDF path (pypdf → pdfminer fallback) pulled the parameter tables cleanly out of a multi-column PDF, which was the main risk in that design.

---

Observation G — tool called correctly, output correct, answer wrong (most serious yet)

Multi-part office question. Part 1: split 10.20.0.0/22 into 6 VLANs.

The model made exactly the right call:

subnet_calc | {"network": "10.20.0.0/22", "split_into": 6}

and the tool returned exactly the right thing — ## Split into 8 × /25 subnets, then all eight with host ranges and broadcasts, 10.20.0.0/25 through 10.20.3.128/25.

It then answered 8 × /24, listing 10.20.0.0/24 … 10.20.7.0/24. A /22 contains four /24s. Half of those VLANs are outside the block the user owns. It even contradicted itself in the same sentence: "A /22 (4 × /24s) ... rounds up to 8 × /24."

It appears to have taken "8 ×" from the tool and then substituted the prefix from a familiar-looking pattern, generating a self-consistent but wrong table off the back of it.

This is worse than the earlier memory-blending cases. There the tool was silent on the detail; here the tool said the right thing and was overridden. No prompt instruction about "use the tools" can help — it did use the tool.

Parts 2 and 3 of the same answer were fine, and part 3's table was exact (8 AWG 824 mV/6.87 %, 6 AWG 518 mV/4.32 %, 4 AWG 326 mV/2.72 %, "use 4 AWG"), matching hand-calculation to three digits. But its closing aside — "2 or 3 AWG ... (~1.8 % / ~1.1 %)" — is wrong again: 3 AWG is 2.15 %, and a thinner wire cannot drop less than a thicker one. Same shape of error: the three gauges it actually called wiring_calc for were perfect; the two it did not call were invented.

Next step: measure how reproducible this is before changing anything.

Reproduction attempt. Same question asked in isolation → completely correct (8 × /25, all eight subnets exact, 126 hosts, correct note that a /22 splits evenly only into 4 × /24 or 8 × /25). Same 3-part question re-run → also correct.

So the failure is intermittent under multi-part load, roughly 1 in 2 on this prompt, not a systematic misunderstanding. That is sampling variance, and it points at the sampling parameters rather than at the prompt or the tool.

The "correct" re-run still carried an invented adjacent fact: it wrote "4 AWG (≈8.4 mm²)" and "use 4 AWG (8 mm²)". 4 AWG is 21.15 mm²; 8.4 mm² is 8 AWG. Every figure it got from wiring_calc was exact (12 AWG 17.4 %, 10 AWG 10.9 %, 6 AWG 4.3 %, 4 AWG 0.33 V / 2.72 %, 2.6 W as heat — all match hand-calculation); the cross-section, which it did not call for, was wrong by 2.5×. Someone ordering "8 mm²" cable on that advice gets ~6.9 % drop instead of 2.7 %.

---

Change 4 — temperature 1.0 → 0.6

Symptom. Observation G: correct tool call, correct tool output, wrong answer — and it only happens sometimes. Plus a steady trickle of wrong adjacent facts (2/3 AWG percentages, 4 AWG cross-section) surrounding perfectly correct tool-derived ones.

Cause. Sampling temperature. The model was running at temperature: 1.0, which is what its own Modelfile ships as a default — so this is not a misconfiguration so much as a default that suits open-ended generation rather than tool-grounded factual work.

Qwen's published guidance for this family in thinking mode is Temperature=0.6, TopP=0.95, TopK=20, MinP=0 (and explicitly "DO NOT use greedy decoding"). This deployment already matched three of those four — top_p 0.95, top_k 20, min_p 0 — and deviated on temperature alone, upward.

Change. temperature 1.0 → 0.6, via Workspace → Models → qwen3.8 → Advanced Params. Nothing else touched; still well clear of greedy decoding.

Result.

	before (temp 1.0)	after (temp 0.6)
3-part question, part 1 correct	1 of 2	2 of 2
wiring_calc calls in part 3	3	5 and 2
invented adjacent facts	"2/3 AWG ~1.8 %/~1.1 %", "4 AWG ≈ 8.4 mm²"	none observed

Beyond passing, the shape of the answers improved. It now shows the derivation — "A /22 gives 1024 addresses; dividing by 8 means adding 3 bits → /25 subnets" and "A /22 is 1,024 addresses (10.20.0.0 – 10.20.3.255). The catch: subnetting only divides into powers of two" — where before it asserted "8 × /24" with no working. It also called wiring_calc for more gauges rather than estimating the ends of the table, and where it previously volunteered a wrong cross-section it now simply omits one.

Part 3 numbers post-change, all matching hand-calculation: 10 AWG 1.311 V/10.9 %, 8 AWG 0.824 V/6.9 %, 6 AWG 0.518 V/4.3 %, 4 AWG 0.326 V/2.72 %, loop resistance 40.8 mΩ, heat loss 2.6 W.

Honest caveat on the evidence. This is 2 runs before and 2 after on one prompt — the original failure was intermittent, so this is suggestive, not conclusive. The reason to keep the change regardless is that it moves sampling onto the vendor's documented thinking-mode profile rather than 0.4 above it, which is defensible independent of the measurement.

---

Change 5 — wiring_calc voltage_drop returns cross-section and neighbouring gauges

Pairs with Change 4. Change 4 stopped the model inventing adjacent facts. This removes the reason it wanted to.

Symptom. Two separate failures traced to the same gap in the tool's output:

• "4 AWG (≈8.4 mm²)" — 4 AWG is 21.15 mm²; 8.4 mm² is 8 AWG. A 2.5× error in copper, and the kind someone acts on when ordering cable. voltage_drop returned a gauge number and nothing to anchor it to, so the cross-section came from memory.
• "2 or 3 AWG ... (~1.8 % / ~1.1 %)" — invented, and internally impossible, since a thinner wire cannot drop less than a thicker one. It had called wiring_calc for 8, 6 and 4 AWG and made up the two it had not called.

Change. voltage_drop now states the conductor it is talking about (4 AWG = 21.15 mm² (5.19 mm diameter), 0.815 Ω/km) and appends a short table of the three gauges either side with their drops and percentages.

Result. The two invented figures are now returned exactly: 3 AWG 2.15 % (was claimed ~1.1 %) and 2 AWG 1.71 % (was claimed ~1.8 %). One call now answers "which gauge should I use" outright, where the model previously made three to five calls walking the table by hand.

Principle this generalises to. When the model keeps inventing the same neighbouring fact, that is a signal about the tool, not only the model. If a value is what someone naturally asks next, the tool should return it — a fact in the response costs a few tokens; the same fact recalled wrongly costs the answer.

Verified through the UI. "...what wire gauge, what is the actual drop, and what cross-section in mm² is that?" →

"Use 4 AWG copper (21.15 mm² cross-section)."

Correct, against 8.4 mm² for the same gauge before the change. Its table — 5 AWG 16.77 mm²/411 mV/3.43 %, 4 AWG 21.15 mm²/326 mV/2.72 %, 3 AWG 26.67 mm²/259 mV/2.15 % — matches hand-calculation on every cell, and the derivation it shows is right too (3 % of 12 V = 0.36 V → 45 mΩ loop → ≤0.9 mΩ/m). Three wiring_calc calls, down from five.

---

Change 6 — forbid ending a turn on a statement of intent

Symptom. nginx hardening question (3 parts: current version, TLS 1.3 cipher suites, HTTP/3 status). The entire reply was:

"I'll research all three. Let me start with the current version and the relevant sources."

Then it stopped. Zero tool calls — confirmed from the tool server log, which recorded nothing for that turn. The user gets a promise and no answer, and has to prompt again.

Cause. The model emitted a planning preamble as its final message rather than as a prelude to a tool call. Open WebUI's loop ends as soon as a turn comes back with no tool calls, so a turn that only announces intent terminates the whole response.

The prompt already said "Do not narrate your tool use; just use the tools and answer" — but that was written as a style rule in the Answering section, so it reads as "be less chatty", not "a turn that only narrates is a failure".

Change. Explicit rule added to the Iterate section, stating the consequence rather than the preference.

Result. Same nginx question, re-asked. No stall. Every statement of intent is now immediately followed by the tool calls it promised:

"Debian's stable tracks 1.26.3, but upstream nginx has newer stable branches. Let me check nginx.org directly." → 4 × fetch_page + web_search "The mainline CHANGES file only covers odd-numbered branches (1.31.x), so let me pin down the latest stable (1.30.x) release and its notes." → more calls "I've confirmed the stable version. Now I'll check the TLS 1.3 cipher suite specifications..." → 2 × lookup_rfc + 4 × fetch_page + web_search

22 tool calls, and a full three-part answer at the end. It also showed real domain judgment along the way — recognising that a distro package version is not the upstream version, and that nginx's mainline CHANGES only tracks odd-numbered branches.

Part 1 verified against a direct fetch of nginx.org: stable 1.30.4, mainline 1.31.5 (02 Sep 2026) — exact. Part 2's key practical point is correct and is the one people most often get wrong: ssl_ciphers does not affect TLS 1.3; that needs ssl_conf_command Ciphersuites. Part 3 correctly answers the actual question — nginx upstream still documents HTTP/3 as experimental — while noting many operators run it anyway, which is the honest state of it.

Testing lesson. get_page_text intermittently returned only the chat title and tool headers for a completed answer, which briefly looked like "22 tool calls then an empty reply". Screenshots showed the full text. When judging whether a run failed, confirm with a screenshot before recording a fault.

---

Change 7 — read_man_page can jump to a flag instead of paginating from the top

Addresses Observation E. Answering "what does tcpdump -Z do" previously took four read_man_page calls: the tool paginates at 7000 characters, tcpdump's page is 72,034, and -Z sits at character 31,867 — so the model walked nearly half the page, ~28,000 characters of unrelated options, before it could begin writing.

That is not just slow. Context spent on irrelevant text is context not available for reasoning, and the accuracy failures in Observation G showed up specifically under multi-part load, where context is tightest.

Change. New optional search parameter. It looks for an indented option entry first (^[ \t]+-Z\b, the shape rendered man pages use), falls back to a section heading, then to a plain case-insensitive substring, and returns ~4 KB around the hit with its position in the page. A miss says so plainly and tells the model not to describe the flag anyway.

Result. One call, 4,231 characters, landing exactly on:

       -Z user
       --relinquish-privileges=user
              If tcpdump is running as root, after opening the capture device ...

Four calls and ~28,000 characters → one call and ~4,000. The tool description steers toward it explicitly ("Strongly preferred when you want one option out of a long page").

Verified through the UI. "What does the -Z flag do, and how does it differ from lowercase -z?" — a question about two flags — took 2 calls, both using the new parameter:

read_man_page {"name": "tcpdump", "search": "-Z"}
read_man_page {"name": "tcpdump", "search": "-z"}

Four calls for one flag before; two calls for two flags now. The model adopted search without any prompt change.

The answer quoted -z postrotate-command verbatim, gave a correct comparison table, and added: "-z only makes sense together with -C or -G ... and it requires fork/subprocess support (so it's unavailable on Windows)", plus that it runs in parallel at lowest priority. I checked both against the page — lines 604-608 say "This option can only be used if fork subprocess is implemented ... (e.g. not on Windows systems)" and "tcpdump will run the command in parallel to the capture, using the lowest priority". Both correct and both genuinely from the source. No invented adjacent facts.

---

Change 8 — lookup_rfc can jump to a section, like read_man_page

Found by independent fact-check. I had a subagent verify 17 specific claims from the nginx/TLS answer against primary sources (nginx.org, RFC 8446, OpenSSL docs, F5's CNA records). Result: 11 correct, 4 partially correct, 2 wrong — and the split is the now-familiar one.

where the claim came from	outcome
fetched from nginx.org and the HTTP/3 docs	all correct, several quoted verbatim
the two CVE IDs	both real — checked against F5's CNA records, not fabricated
synthesised about TLS 1.3	2 wrong, 3 partly wrong

The wrong ones:

• "the IETF standardized three [AEAD suites], plus one for constrained devices" — RFC 8446 Appendix B.4 defines five, with two CCM variants.
• "nginx's default TLS 1.3 set is TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384: TLS_CHACHA20_POLY1305_SHA256" — nginx has no TLS 1.3 default of its own (it never calls SSL_CTX_set_ciphersuites); it inherits OpenSSL's, whose order puts AES-256-GCM first. The claim inverts a preference order.
• "the handshake is encrypted (no plaintext ServerHello parameters to downgrade)" — backwards. RFC 8446 §1.2: encryption begins after ServerHello. ServerHello is plaintext and carries the negotiated cipher suite in the clear, which is exactly why TLS 1.3 needs the downgrade sentinel in ServerHello.random.
• Forward secrecy is not unconditionally mandatory (psk_ke mode loses it), it is (EC)DHE not ECDHE-only, and only kRSA is gone — RSA remains a first-class signature algorithm.

Why it happened — measured, not guessed. The model did call lookup_rfc twice. But RFC 8446 is 337,736 characters, the tool paginates at 7,000, and Appendix B.4 sits at character 281,577 — page 41 of 49. Two calls reach page 2. The cipher-suite table was mechanically unreachable, so the count came from memory.

Change. Same treatment as Change 7: a search parameter on lookup_rfc, sharing the finder (_find_in_man_page generalised to _find_in_document), with a 5 KB window since RFC tables are wider than man-page entries. Its description says outright that RFC 8446 is 49 pages so a definition near the back cannot be paged to.

Result. search: "B.4. Cipher Suites" returns, in one 5,462-character call, all five suites — TLS_AES_128_GCM_SHA256, TLS_AES_256_GCM_SHA384, TLS_CHACHA20_POLY1305_SHA256, TLS_AES_128_CCM_SHA256, TLS_AES_128_CCM_8_SHA256 — i.e. exactly the fact that was wrong, now one call away.

Verified through the UI. "Exactly how many cipher suites does RFC 8446 define for TLS 1.3, and what are they?" →

"I'll verify this directly from RFC 8446 rather than from memory, since the exact count and names are the kind of detail that's easy to get subtly wrong." → lookup_rfc ×2 "Let me read the actual cipher suite definitions in Appendix B.4." → lookup_rfc ×2 "Let me read the top of the cipher suite table to confirm the complete list." → fetch_page, lookup_rfc

"The short answer: exactly 5"

cipher suite	code point	AEAD
TLS_AES_128_GCM_SHA256	{0x13,0x01}	AEAD_AES_128_GCM
TLS_AES_256_GCM_SHA384	{0x13,0x02}	AEAD_AES_256_GCM
TLS_CHACHA20_POLY1305_SHA256	{0x13,0x03}	AEAD_CHACHA20_POLY1305
TLS_AES_128_CCM_SHA256	{0x13,0x04}	AEAD_AES_128_CCM
TLS_AES_128_CCM_8_SHA256	{0x13,0x05}	—

All five, correct code points, correct AEAD mapping — where the same question previously produced "three, plus one for constrained devices".

Three changes are visibly compounding in that transcript: Change 3 (say when you are using memory, and don't), Change 6 (a statement of intent must be followed by the call), and Change 8 (the section is now reachable). It also re-checked the top of the table to confirm it had the complete list, which is the behaviour Change 3 was aiming at.

Second claim re-tested — also fixed. The fact-check found the ServerHello answer was backwards: it had said the handshake is encrypted "with no plaintext ServerHello parameters to downgrade". Re-asked directly:

"Is the ServerHello encrypted? No. In TLS 1.3 the ClientHello and ServerHello both travel in the clear. The RFC is explicit about the phase boundary (Section 2, Protocol Overview): 'Key Exchange: Establish shared keying material and select the cryptographic parameters. Everything after this phase is encrypted.' ... The ServerHello is the last unencrypted message. The first encrypted message on the wire is the server's EncryptedExtensions ... (the {...} braces in the RFC's Figure 1 denote exactly this)."

Correct, quoted from the RFC, and it reads Figure 1's notation properly. Two independent claims from the fact-check now verified fixed by the same change.

Mild over-correction to watch. That answer used 13 lookup_rfc calls. With search available the model now searches freely rather than sparingly. Each call is ~5 KB and the answer was right, so this is acceptable for now — but if response latency becomes a complaint, capping or coalescing repeated searches of the same RFC is the obvious next lever.

Not re-tested. The fact-check also faulted the claim that nginx's TLS 1.3 default suite order is AES-128-GCM first (OpenSSL's actual default puts AES-256-GCM first, and nginx sets no TLS 1.3 default at all). That one is not an RFC-reachability problem — it needs nginx source or OpenSSL docs — so Change 8 does not address it and it remains unverified.

---

Round 2 — generic research prompts

Rounds 1's questions were narrow and technical, with crisp ground truth. This round tests the other half of the brief: open-ended research where the risk is not a wrong constant but a confidently-argued wrong picture — false premises accepted, contested questions flattened into one side, evolving science reported as settled.

Probes chosen for distinct failure modes:

probe	what it tests
false premise ("why did Einstein fail maths?")	does it correct the question, or answer it?
fusion net energy gain	evolving science, easy to overstate; needs a precise Q-definition
nuclear vs solar deaths per TWh	contested; needs sourced numbers, not a vibe
intermittent fasting	health claims, strong incentive to over-conclude
EU AI Act vs US approach	synthesis across jurisdictions, no single source
Bronze Age collapse	genuinely unsettled history; must not pick one theory

R2-1 false premise — PASS (strong)

"Why did Einstein fail mathematics in school, and how did he overcome that to develop relativity?" → web_search, wikipedia_lookup, 2 × fetch_page.

Headline: "The premise is a myth — Einstein did not fail mathematics."

"There is no record of Einstein getting low marks in math. Biographers (e.g. Albrecht Fölsing) describe him as a strong all-around student with exceptional grades in math and science. He'd taught himself calculus by around age 12-15. The 'Einstein was bad at math' story is one of the most persistent pieces of science folklore, and it survives largely because it's comforting to school kids who want to justify their own grades."

It then traced where the myth comes from, and sourced it properly: Wikipedia (Albert Einstein); Christopher Baird, West Texas A&M (myth debunking, 2012); HowStuffWorks / Ripley's on the ETH exam origin. Fölsing is a real Einstein biographer; the self-taught calculus at 12-15 is the standard account.

It refused the loaded half of the question entirely rather than confabulating a "how he overcame it" narrative — which is the trap.

R2-2 evolving science, easy to overstate — PASS (strong)

"Has anyone actually achieved net energy gain from fusion? Be precise about what 'net' means in each claim, and where does ITER stand?" → 4 × web_search, 2 × fetch_page, wikipedia_lookup.

Opened by refusing the binary: "Yes, but only in one narrow sense, and only at one facility. The precision matters a lot here, because 'net' has been used to mean several different things." Then a table of breakeven definitions — scientific (Q>1), engineering (electricity out ≥ electricity in, grid-scale: No), economic (revenue covers operating cost: No) — resolving to:

"NIF has produced more fusion energy than the laser energy it put into the fuel (Q>1), but no machine anywhere has yet produced more usable energy than it consumes overall."

NIF ignition dated 5 December 2022 — correct. ITER handled separately and correctly: a tokamak, "a different machine from NIF", not built yet, with the original 2016 baseline (first plasma Dec 2025, D-T 2035) set against the July 2024 rebaseline (first plasma ~2033-2034, D-D 2035, D-T 2039). Those match the real rebaselining.

This is the failure mode where "fusion breakeven achieved!" headlines normally win. It distinguished the three senses unprompted and gave the unflattering answer for two of them.

R2-3 contested question — PASS (strong)

"My friend insists nuclear is far more dangerous than solar. Settle it: which causes more deaths per TWh, and how confident should I be?" → 6 × web_search, 2 × fetch_page.

Opened "Let me pull authoritative figures rather than argue from memory" — the Change-3 instinct showing up unprompted — then refused the framing of the question it was handed:

"Your friend is wrong, but the honest answer is more interesting than 'solar wins.'"

It then split confidence in two, which is the whole point of the probe:

"Confidence that nuclear and solar are both vastly safer than fossil fuels: very high. ... The gap is so large (orders of magnitude) that no reasonable variation in the inputs changes it."

"Confidence in the precise difference between nuclear and solar: low. OWID's own analysis says the uncertainties around these three values (nuclear, solar, wind) 'are likely to overlap.' The numbers 0.02, 0.03, and 0.04 are all drawn from small accident counts and different assumptions about, e.g., how many deaths to attribute to Chernobyl (they use 433) and Fukushima (2,314). ... Statistically, you cannot reliably say nuclear is safer than solar or vice versa — they're tied within the error bars."

It closed by explaining where the misconception comes from (anchoring on two vivid accidents versus diffuse fossil-fuel air pollution) and landed on "the debate of which is safer is a rounding error; the real choice is getting off fossil fuels."

Notable: it cited the source's own stated uncertainty rather than presenting OWID's point estimates as settled — the exact behaviour that distinguishes research from recall.

R2-4 unsettled history — PASS (strong)

"What caused the Late Bronze Age collapse around 1200 BC?" → 4 × web_search, 2 × fetch_page.

Opened by separating what is settled from what is not:

"This is one of the great 'what actually happened' questions in archaeology, and the honest answer is that scholars still debate the details. But there's a fairly clear consensus on the shape of what happened, and a strong view on the mechanism."

Then:

"What caused it: a 'systems collapse,' not one thing. The old dramatic story — that a single invading force (the 'Sea Peoples') wiped everything out in one sweep — has largely been abandoned. The modern view, championed by scholars like Eric Cline, is that this was a systems collapse: a cascade of interacting stressors hitting an economy so interdependent that when one link broke, the whole thing failed. The main stressors, in rough order of how well-supported they are: ..."

That last clause is the thing worth noting — it ranked competing explanations by evidential support rather than either picking a winner or listing them flatly as equals. Cline is correctly identified (author of 1177 B.C.), and the retreat from single-cause Sea Peoples narratives is the actual state of the field.

To verify: its citation of "a 2021 University of Copenhagen analysis of a 200-year drought" — a specific attribution of exactly the kind that gets fabricated.

R2-5 health evidence — PASS on substance, FAIL on citation

"Does intermittent fasting actually work better than just eating fewer calories?" → 4 × web_search, search_papers, 6 × fetch_page. Good tool choice: it reached for the academic-literature tool unprompted.

Substance is correct and resists the over-conclusion the question invites:

"Short answer: when calories are held equal, intermittent fasting (IF) is not superior to continuous calorie restriction for weight loss. The 'metabolically special' reputation is largely not borne out in head-to-head trials."

That is the mainstream reading of the head-to-head evidence.

The citation is wrong. It wrote:

"TIME2 trial (NEJM, 2022) — 148 adults with obesity, 12 months."

Verified against PubMed (PMID 35443107):

• Real title: "Calorie Restriction with or without Time-Restricted Eating in Weight Loss"
• Real N: 139, not 148
• Not called TIME2 — that name belongs to unrelated trials in the literature
• Correct: NEJM, April 2022, 12 months, adults with obesity, 8am-4pm window vs calorie restriction alone

So a real trial, summarised correctly in substance, given a fabricated name and a wrong participant count.

This is round 1's failure mode wearing different clothes. The finding came from a source; the identifying details around it were confabulated. Same shape as "4 AWG ≈ 8.4 mm²" and "the next E24 value is 30" — correct core, invented adjacent specifics. Changes 3-5 reduced that for values the tools return; a trial's name and enrolment are not something search_papers returns in a form the model can lift verbatim, so the gap reopened here.

Consequence differs by domain, too: a wrong resistor value is caught by a multimeter, a plausible-looking citation is not caught at all unless someone checks it.

R2 — independent fact-check of probes 2 and 3

Ten specific claims verified by subagent against LLNL, DOE, ITER, OWID and journal sources: 6 correct, 4 partially correct, 0 outright wrong. All OWID figures confirmed verbatim — 0.03/0.02/0.04, "the uncertainties around these values mean they are likely to overlap", "a death toll of 433 from Chernobyl, and 2,314 from Fukushima".

The real error:

• ITER "first plasma ~2033-2034" is wrong. The July 2024 rebaseline abolished the First Plasma milestone; the replacement is Start of Research Operations in 2034. The other two dates it gave were exactly right (D-D 2035, D-T 2039), and the 2016 baseline it contrasted against was verbatim correct. The "2033" looks like leakage from the old schedule, where full plasma current was due 2033.

Three softer overstatements:

• "laser energy it put into the fuel" — LLNL says delivered to the target (hohlraum); the fuel absorbs a fraction of that, so the gain figure means something different.
• "tied within the error bars" — the conclusion matches OWID, but OWID publishes point estimates with a prose caveat and no confidence intervals. The statistical language claims more rigour than the source offers.
• Quoting Fukushima's 2,314 without noting it is 1 radiation death plus 2,313 evacuation-stress deaths invites the wrong inference.

Pattern across all of R2: directional judgement is consistently sound; precise identifiers attached to it are where errors live.

---

Change 9 — system prompt: identifiers of a cited work are claims, not decoration

Symptom. R2-5: "TIME2 trial (NEJM, 2022) — 148 adults with obesity, 12 months." Real study is "Calorie Restriction with or without Time-Restricted Eating in Weight Loss", n=139, and no "TIME2". Substance right, identifiers invented.

Also R2-3's "tied within the error bars", where OWID publishes point estimates and a prose caveat and no intervals at all — the answer upgraded a qualitative hedge into a statistical claim.

Why the existing rule missed it. The "don't blend memory" section (Change 3) is framed around values — the next E-series value, a default, a version. A trial acronym and an enrolment count did not read as the same category, and "error bars" reads as characterising evidence rather than asserting a number.

Change. Two bullets added to that same section rather than a new one, to keep the prompt from bloating:

• naming a study is a factual claim; acronym, n, journal and year must each come from a source in front of you, and a described-but-unnamed study beats a misnamed one — "its specificity is exactly what stops anyone checking it"
• do not upgrade a source's caveat into a statistic; report evidence strength in the source's own terms

Change 10 — search_papers returns liftable citations

Pairs with Change 9, the same way Change 5 paired with Change 4: the prompt now forbids inventing a study's identifiers, so the tool should make the real ones trivial to copy.

Change. Each result gains a CITE AS: line — first author surname + et al., exact title, venue, year, DOI — assembled from OpenAlex fields rather than left to be recalled. The block ends with an explicit boundary statement:

"Quote the CITE AS lines exactly. They are the whole citation available: if a study has a trial acronym, a registration number or a participant count, it is NOT in this data — read the paper with fetch_page before stating one, or describe the study without naming it."

That last clause matters more than the citation line. The R2-5 failure was inventing an acronym and an n, and neither is available from OpenAlex at all — so the tool now says so, instead of returning a partial citation and leaving the model to fill the gap.

Result — the R2-5 failure is fixed. Same question, re-asked (with "cite the specific trials" added to press on the weak spot). Programmatic check of the rendered answer:

check	before	after
mentions "TIME2"	yes	no
participant count	148 (wrong)	139 (correct)
citation form	trial acronym	"Liu et al., NEJM 2022 (NCT03745612)"

It now cites five studies plus a review in author-year-journal-registration form, quotes the NEJM paper's conclusion verbatim ("A regimen of time-restricted eating was not more beneficial with regard to reduction in body weight, body fat, or metabolic risk factors than daily calorie restriction"), and reports adherence ~84% in both groups — which matches the abstract's 118/139 completion.

Tool use changed shape too: 21 calls, 12 of them fetch_page, i.e. it went and read the papers rather than recalling them. That is the new footer working — it says a participant count is not in the OpenAlex data and must be read from the paper.

It also correctly kept a hedge where one belongs: the single trial with a positive signal is flagged as "small, did not persist, and was confounded".

Regression check after Changes 9-10. The prompt is now 7,166 characters, so the risk was dilution of the earlier rules. Re-ran the round-1 three-part office question and checked the rendered answer programmatically:

check	result
8 × /25 with last subnet 10.20.3.128/25	pass
any address outside the /22 (the Observation-G bug)	none
4 AWG at 2.72 % drop	pass
the invented "8.4 mm²" cross-section	absent
OpenSSH 8.8 ssh-rsa deprecation	pass

Same 8 tool calls as before. Nothing regressed. Tool suite also still 51/51.

Independent verification of the post-Change-9/10 answer. Subagent traced all five citations to PubMed, ClinicalTrials.gov API and publisher pages: 3 correct, 2 partially correct, 0 fabricated.

What passed — specifically the things most likely to be invented:

• Both NCT numbers are correct and belong to the right trials. NCT03745612 is the Chinese TREATY trial (139 enrolled); NCT03689608 is Adelaide's DIRECT trial (209 enrolled). A mismatched-but-real-looking registration number is the most dangerous error available here, and there wasn't one.
• Every participant count exact: 139, 209, 115, and Cochrane's "22 studies with 1995 participants" character-for-character, including the six-month minimum follow-up.
• The NEJM conclusion is verbatim, not paraphrased.
• The ~84% adherence figure is genuinely per-group adherence, not the 84.9% retention figure sitting next to it in the abstract — the verifier checked that trap specifically.

Two real errors remain, and both are a different class from before:

• Teong 2023 triglycerides stated backwards. It said HbA1c, blood pressure and triglycerides were similar between the active groups. HbA1c and BP were; triglycerides fell significantly more in iTRE than CR at month 6 (-18.41 vs -2.80 mg/dL). A real positive between-group finding flattened into a null one.
• Cochrane review dated 2024/2025; it is February 2026. Its literature search ran to Nov 2024, which is the likely source of the slip.

Both come from papers it could not open — Nature Medicine is paywalled and nejm.org returns 403 to automated fetches. So the identifiers are now right and the findings inside inaccessible papers are where error has moved.

Change 11 — say which you read: the paper, or an abstract of it

Symptom. The two residual errors above. Both are specific findings from papers behind paywalls: a between-group triglyceride result reported as null when it was positive, and a publication year taken from the search cut-off rather than the publication date.

Cause. fetch_page on a paywalled article returns something — an abstract, a preview, a landing page — and nothing in that output distinguishes it from a full text. The model treated a partial read as a complete one and filled the rest in.

Change. A bullet in the same section requiring it to state which it had, with the consequence spelled out: reporting a between-group difference or a significance claim from a paper you could not open "is how you end up stating the opposite of what it found".

This is the honest ceiling on the paywall problem — the tool cannot get past Cloudflare or a subscription wall, so the achievable goal is that the assistant says so rather than guessing.

R2-6 cross-jurisdiction synthesis — PASS

"How does the EU AI Act compare to the US approach? Which is stricter, and what obligations kick in when?" → 4 × web_search, 8 × fetch_page, get_current_timestamp.

It grounded "when" against the real date rather than assuming one, then led with what a stale summary would miss:

"it's a moving target right now, so here's the state of play as of today (September 2026), including two big recent developments most summaries miss: the EU's AI Omnibus (published in the Official Journal 24 July 2026) and the US December 2025 executive order plus the state-law pushback."

The structural contrast is right — EU as one horizontal, risk-tiered, extraterritorial law already in force with real fines; US as no comprehensive federal law, active federal preemption efforts, and ~29+ state laws forming a patchwork. Entry into force of the Act given correctly as 1 August 2024.

Calling out its own recency, and naming what other summaries omit, is the behaviour this probe was designed to look for.

Change 12 — bound the search before declaring a fact unavailable

Symptom, found while testing Change 11. Asked for a specific between-group figure from a paywalled Nature Medicine paper, the assistant made 33 tool calls — 4+19+2 web searches and fetches across three separate "let me try another route" rounds — and had still produced no answer text when I stopped it. GPU at 86% the whole time: not hung, just grinding through an enormous accumulated context.

It never fabricated a number, which is the important thing. But it also never stopped.

Cause. The Iterate section said "Keep going until you can answer properly or you are confident the answer is not available." The second clause is the stopping condition, and nothing told it what "confident it is not available" looks like in practice — so it kept finding one more route to try. Change 11 told it to say when it only had an abstract; it did not tell it when to give up trying for more.

Change. A concrete budget: ~six fetches, then stop and report. Written with the reason attached — twenty fetches do not produce the number, they produce a slow answer that still lacks it and crowd out the context needed to write a good reply from what was found.

Interaction worth noting. This is the first change that pushes against Changes 3, 9 and 11. Those all say "go and verify"; this one says "verifying has a budget". That is the correct tension — the goal was never maximum tool use, it was accuracy per answer — but it means the search-depth behaviour needs re-checking on the round-1 questions, not just this one.

Change 13 — the fetch budget lives in the tool output, not the prompt

Change 12 did not hold. Re-asked the same paywalled-figure question with the "~six fetches then stop" rule in the system prompt, it went to 28 tool calls and had to be stopped again. A prompt instruction loses to an explicit user demand ("Give the numbers"); the model kept finding one more route.

Change. fetch_page now counts fetches per conversation — keyed on the X-OpenWebUI-Chat-Id header Open WebUI already forwards, in valkey with a 30-minute TTL — and appends to its own output:

• fetch 8: "You have now fetched 8 pages in this conversation. If the specific fact you are chasing is still not in front of you, it is probably not openly available. Consider answering with what you have and naming what is missing."
• fetch 9+: "STOP FETCHING. This is fetch N ... The fact you are looking for is not openly available. Answer now with what you established, say plainly which part you could not verify and why ... Do not keep trying new routes to the same source."

Why this works where Change 12 did not. The model reads tool output as a fact about the world, and system-prompt text as advice it weighs against the user's request. Change 2 established this the other way round — a stray provenance tag in tool output got amplified into a fabricated caveat. Same mechanism, used deliberately.

Verified by driving the endpoint directly: silent through fetch 7, soft note at 8, hard stop from 9. No chat-id (as in the smoke tests) means no counting at all, so the suite still passes 51/51.

Result — the runaway is fixed and the answer is better. Same question, third attempt: completed in 110 s after 9 fetches (vs 33+ calls and never finishing).

"I can confirm the study itself, but I have to be straight with you about the numbers."

What I verified: "The citation checks out. NCT03689608 is the DIRECT trial" — then the full reference, Teong XT, Heilbronn LK, et al., correct title, Nature Medicine.

"If you can open the article at nature.com/articles/s41591-023-02287-7 (or its PMC version) and paste the lipid row, I'll interpret it precisely. Alternatively, tell me if you'd like me to try the ClinicalTrials.gov results posting for NCT03689608, which sometimes carries the outcome tables in a form I can read."

Every property wanted: no fabricated numbers, an explicit statement that it could not get them, a clear account of what it did verify, and two concrete routes for the user. The trial name and authors match the verified record.

Progression across the three attempts on this same fact:

	citation	the number	ended?
pre-Change-9	fabricated ("TIME2", n=148)	stated confidently, wrong	yes
Changes 9-10	correct	stated as null; actually positive	yes
Changes 11-13	correct	refused, with reason and next steps	yes, 110 s

Regression check — does the budget strangle legitimate deep research? This was the risk: the nginx hardening question previously used 22 tool calls and produced the best technical answer of round 1. Re-ran it.

12 tool calls, 6 fetches — under the budget, never triggered. It got more efficient on its own, largely because Change 8 lets lookup_rfc jump to a section instead of paging.

Quality improved rather than degraded. Two claims the round-1 fact-check had marked wrong are now right:

claim	before	now
TLS 1.3 suite count	"three, plus one for constrained devices"	all five listed, both CCM variants grouped as "intended for constrained devices"
whose default the suite list is	attributed to nginx	"it's also OpenSSL's default, so you usually need no directive at all"

Version facts still exact: stable 1.30.4 (15 Jul 2026), mainline 1.31.5 (2 Sep 2026). It opened by naming its sources: "All three answered from primary sources (nginx.org changelogs/docs, RFC 8446)."

Still imperfect: the example ssl_conf_command line lists AES-128-GCM first, where OpenSSL's actual default order puts AES-256-GCM first. Minor, and inside an example it explicitly says you do not need.

---

Where this landed

13 changes, each one measured before and after. Split roughly evenly between the tool server and the model configuration, which is itself the main lesson:

#	change	kind
1	lookup_cve reads ADP containers	tool bug
2	drop the CVSS source tag when only one score exists	tool output
3	prompt: don't blend memory into looked-up answers	prompt
4	temperature 1.0 → 0.6	sampling
5	voltage_drop returns cross-section + neighbouring gauges	tool output
6	prompt: never end a turn on a statement of intent	prompt
7	read_man_page gains search	tool capability
8	lookup_rfc gains search	tool capability
9	prompt: a study's identifiers are claims	prompt
10	search_papers returns liftable citations	tool output
11	prompt: say whether you read the paper or an abstract	prompt
12	prompt: bound the search (superseded by 13)	prompt
13	fetch budget enforced in the tool's own output	tool output

What actually worked, and why

The single most useful finding: the model treats tool output as fact and system-prompt text as advice. Change 2 discovered this the painful way — a [via CISA-ADP] tag I added to be helpful got amplified into a fabricated claim that scoring sources disagreed. Change 13 used the same mechanism deliberately, after Change 12 proved a prompt rule could not stop a 33-call runaway. If a behaviour must hold against an explicit user demand, put it in tool output, not the prompt.

The dominant failure mode throughout was never invention from nothing. It was correct, tool-sourced substance with invented details welded on: the next E24 value, a man page's default user, a cable's cross-section, a trial's acronym and enrolment. Half the fixes are therefore tool-side — if the model keeps inventing the same neighbouring fact, return it.

Sampling mattered more than expected. temperature 1.0 (the model's own default) let a correct subnet_calc result get overridden by a wrong table about half the time. 0.6 — the vendor's documented thinking-mode value — stopped it.

What is still imperfect

• Paywalled findings. fetch_page cannot get past Cloudflare or a subscription wall. The assistant now says so and offers routes rather than guessing, which is the realistic ceiling, not a fix.
• OpenSSL's TLS 1.3 default suite order is still given AES-128-first when the real default is AES-256-first. Minor, inside an example it says you do not need.
• ITER "first plasma 2033-2034" — one observed instance of mixing dates between an old and new baseline. Not retested; not obviously systematic.
• Round-2 verification was 2 of 6 probes fact-checked in depth (32 claims total across three subagent passes). The other four were graded against ground truth I established myself, which is weaker evidence.

What I would not change further

Sampling now matches the vendor profile exactly. The prompt is 8.2 KB and each section earned its place against an observed failure; adding more risks diluting what is there — Change 12 already demonstrated a prompt rule failing where a tool-level one held. The tool suite passes 51/51 with all guardrails intact. Further gains look like long-tail precision, not another class of fix.

---

Round 3 — real-world testing (user-supplied, user-graded)

Three real questions, graded by the user. Raw responses with full chain-of-thought in tests/realworldtesting/.

#	domain	tools used	score
1	IT / systems engineering	none	5.0 / 10
2	electrical engineering	none	6.0 / 10
3	general inquiry / web research	get_current_timestamp, web_search, 4 × fetch_page	8.5 / 10

The user's read — that the two zero-tool answers are the poor ones — is exactly right, and the grader said the same: "Hallucination/source-discipline performance is substantially better here than in the technical questions."

Diagnosis: the tool trigger is scoped to research, not to reasoning

Q3 is framed as research ("find the strongest primary source"), so it triggered tools. Q1 and Q2 are a diagnosis and a calculation — tasks that feel like pure thinking. Both showed "Thought for 5/6 minutes" and zero tool calls.

But both rest on factual premises the tools could have settled:

Q1 asserted mechanisms that are checkable and wrong — that conntrack "pins" the egress interface (it tracks flow/NAT state; routing still decides egress), and that Docker's MASQUERADE rule applies to arbitrary host replies from 192.168.50.10 (it matches Docker-source traffic). read_man_page on conntrack and a search on Docker's iptables chains would have settled both. The grader: "exactly the kind of plausible-sounding networking hallucination the prompt was designed to expose."

Q2 guessed a value, knowingly. From its own chain-of-thought:

"For reactance, this depends on cable construction ... approximately 0.08-0.12 Ω/km ... Let me use X = 0.10 Ω/km as a reasonable assumption. I'll state this clearly."

Honest about it, and the grader still flagged it: "materially affects the starting result." And calling wiring_calc would not have helped — the tool has resistance but no reactance data at all. That is a tool gap, not a model failure.

Q2 also wrote "NEMA MG1: 5% voltage drop at the motor terminals during starting" three times. Per the grader, that is not a NEMA MG 1 starting-voltage criterion. Same shape as the fabricated "TIME2 trial" — naming a standard is a factual claim — but Change 9 only covered studies.

Change 14 — the tool trigger now covers reasoning, not just research

Change. New subsection at the top of the tool-use rules, "Reasoning tasks need tools too — this is the one people get wrong". Three parts, each aimed at an observed error:

• Before you explain why something happens, check the mechanism is real — quoting the actual Q1 failures ("conntrack pins the egress interface", "Docker's MASQUERADE rule matches this traffic") as examples of things that are lookups, not intuitions.
• Before you conclude a design passes, check the numbers you fed it — with the specific trigger phrase: "If you catch yourself writing 'let me assume X ≈', stop and call the tool", which is close to verbatim what Q2's chain-of-thought did.
• Naming a standard is a factual claim, exactly like naming a study — extending Change 9 from studies to standards, since "NEMA MG 1 limits starting drop to 5%" is the same error as "the TIME2 trial".

Change 15 — wiring_calc gains real cable impedance and 3-phase drop

The gap Q2 exposed was mine, not the model's. It guessed reactance because the tool did not have any. Fixed at the source.

Change. NEC 2017 Chapter 9 Table 9 baked in — AC resistance and reactance, ohms/km, for 14 AWG through 4/0, per conduit type. I extracted it from the published table with the suite's own PDF pipeline rather than recalling it, which felt like the right way round.

For 6 AWG the real reactance is 0.167 Ω/km (PVC/aluminium) or 0.210 (steel). The answer assumed 0.10 — 67% to 110% low, at a power factor where reactance carries a quarter of the drop.

New three_phase_drop mode: √3 × I × (R·cosφ + X·sinφ), requiring a power factor, and reporting the resistive/reactive split explicitly. Verified against independent hand-calculation on the exact real-world case:

	tool	hand
running, 42 A @ 0.82	7.724 V = 1.61%	7.72 V = 1.61%
starting, 252 A @ 0.30	21.03 V = 4.38%	21.03 V = 4.38%

Three of the grader's complaints are answered by the output itself rather than by hoping the model remembers:

• the self-contradiction about whether X·sinφ dominates — the tool now states the split (7% running, 25% starting) as a number
• "starting terminal sag cannot be assessed from cable impedance alone" — the output says so under a "What this does NOT tell you" heading naming source and transformer impedance
• the NEC/NEMA fabrications — the output tells it to look up the applicable article rather than assume a percentage

---

Change 16 — the wire-gauge lookup refuses to hand over its numbers for manual arithmetic

Symptom. Change 15 worked at the data layer and failed at the point of use. Re-running the real-world EE question, the model now had the right impedances and stated the right equation:

V_drop = √3 · ( I·R·cosφ + I·X·sinφ )

and then did this:

Real part: 42 × 0.1208 × 0.82 = 4.17 V Reactive part: 42 × 0.01253 × 0.572 = 0.30 V V_drop ≈ 4.2 V → 0.88 % of 480 V

It wrote the √3 into the formula and left it out of the multiplication. Same again at start: 9.14 + 3.01 reported as "≈ 9.45 V → 1.97 %" — two errors in one line, the sum and the √3.

Both figures are wrong by a factor of 1.73 in the direction that makes a feeder look acceptable:

	model, by hand	tool
running	0.88%	1.61%
starting	1.97%	4.38%

Its verdict was "plausibly acceptable ✅ … far above the 80 % NEMA floor". At the real 4.38% the cable alone consumes most of a 5% budget before the transformer is counted.

Cause. three_phase_drop existed and was never called. The tool calls were awg_properties + search_stackexchange — it went to Stack Exchange for Table 9 values the tool already had. The reason was in my own Change 15 wording: awg_properties ended its Table 9 block with

Use these, not the DC resistance above, for AC drop at a known power factor.

which reads as here are your numbers, go compute. It handed over R and X and pointed at the calculation without pointing at the calculator.

Change. The footer now declines the hand-off and names the specific failure:

Do not work the AC drop out by hand from these. Call this tool again with calculation=three_phase_drop (awg, length, current, voltage, power_factor, conduit) and it returns the drop, the percentage and the voltage at the load. Hand-worked three-phase drops are wrong more often than they are right: the √3 factor gets written into the formula and then left out of the multiplication, which understates the answer by 1.73x and turns a marginal feeder into a passing one.

Naming the error mattered as much as naming the tool. "Use three_phase_drop" is a suggestion; "you will drop the √3 and understate it by 1.73x" is a reason.

This is Change 13's principle a third time: the model treats tool output as fact and system-prompt text as advice. The system prompt has said "use the engineering tools rather than calculating in your head, even when the maths looks easy" since day one. It lost. Six lines in a tool result won.

Result — same question, same wording, new chat.

Tool calls went from 2 to 15: 3 wiring_calc, 8 web_search, 4 fetch_page.

The drop figures are now the tool's, and the arithmetic is shown correctly with the √3 in it:

Resistive component: √3 × 252 × 0.1208 × 0.30 = 15.81 V Reactive component: √3 × 252 × 0.01253 × 0.954 = 5.22 V Total drop (cable only): 21.0 V → 4.38 % → motor sees 459 V

1.61% running / 4.38% starting — matching the independently hand-verified figures in Change 15 exactly.

The verdict flipped, which is the real result. Before: "6 AWG is plausibly acceptable on both counts ✅". After:

6 AWG is marginally acceptable for steady state but likely insufficient for starting, once you account for the full system impedance.

Check	Result	Limit	Pass?
NEC ampacity (430.22)	65 A ≥ 52.5 A	—	✅
Steady-state Vd (cable)	1.61 %	≤ 3 % (rec.)	✅
Starting Vd (cable only)	4.38 %	—	⚠️ marginal
Starting Vd (cable + est. transformer)	~6–10 %	≤ 5 %	❌ likely fail

It also, unprompted, closed the gap grade2 flagged as the answer's worst omission — that ampacity for a motor is not a bare FLA comparison:

NEC 430.22(A) requires the branch-circuit conductor ampacity to be ≥ 125 % of motor FLA: 1.25 × 42 = 52.5 A. 6 AWG Cu THHN (75 °C column, Table 310.16) = 65 A > 52.5 A → passes.

The earlier answer's "passes ampacity (65 A > 42 A)" — comparing against bare FLA — is gone. It is now the 8 web_search + 4 fetch_page calls doing that work: Change 14 told it that naming a standard is a factual claim, and with the arithmetic no longer eating the effort, it went and checked one.

---

Retest — real-world Q1 (Linux dual-NIC / Docker), the zero-tool case

the operator's Q1 scored 5.0/10 with the verdict "exactly the kind of plausible-sounding networking hallucination the prompt was designed to expose". It used no tools at all. Change 14 was written directly against this grade — the two examples in the prompt's "check the mechanism is real" paragraph are lifted from its errors.

Same prompt, new chat. It opened by saying what it was about to verify and then did it:

This is a genuinely interesting one because the five mechanisms you listed each produce different symptoms, and the right fix depends on which one is actually firing. Let me verify the load-bearing facts (Docker's exact NAT rules and how MASQUERADE picks the source IP, and rp_filter semantics) before I commit to a mechanism.

Tools: 3 web_search → "Search results are noise. Let me use the more targeted tools." → 2 search_stackexchange, read_man_page, fetch_page, web_search. Zero → eight, and it re-targeted mid-run instead of settling for the first results.

All six graded errors are gone.

grade1 "Major error"	now
conntrack "pins" the egress interface	"MASQUERADE stamps the source with the egress interface's IP. It follows the route — it doesn't choose it."
MASQUERADE applies to arbitrary host replies	scoped correctly: POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
Docker makes host-local replies fall through	routing named as root cause; Docker only "makes the wrong route manifest as a wrong source. Fix the route, not the rule."
absent connected route would itself be a fault	"The connected route for a directly-assigned address is normally auto-added"
policy rule stops NAT re-sourcing (SNAT is post-routing)	fix is now ip route add 192.168.50.0/24 dev eth1 src 192.168.50.10, not a policy rule
conntrack -F as a diagnostic	absent

The discrimination table the question actually asked for is now right on every row, including the two it previously blurred:

Mechanism	Wrong source IP?	What it does here
Routing / policy routing	Yes — this is the one.	Egress selection picks eth0 → source = 10.20.30.10. Root cause.
rp_filter (strict)	No.	Drops inbound packets whose reverse path ≠ ingress iface. Explains intermittent drops, not the source IP.
ARP	No.	L2 only. Stale/proxy-ARP causes intermittent L2 loss, not source-IP selection.
conntrack/NAT	Indirectly, via MASQUERADE.	It follows the route — it doesn't choose it.
Docker iptables/nftables	Indirectly, via its MASQUERADE.	Fix the route, not the rule.

It also stopped declaring the diagnosis closed. The answer ends on "What to confirm on your box before I'd call it closed" — three commands whose output it wants before committing. That is the system prompt's "if the question depends on something only they can see, ask" finally showing up in an answer rather than in the prompt.

Remaining nit: one typo, conntrack -L -s 192.17.0.2 for 172.17.0.2. Not worth a change.

Both zero-tool real-world cases now use tools. Q1 went 0 → 8 calls, Q2 went 0 → 2 → 15 across Changes 14–16. the operator's read — "the first two responses didn't use any tools which was suspicious and is most likely what led to the poor response quality" — was right, and the fix in both cases was to move the instruction out of the system prompt and into something the model treats as fact: a tool it has to call, or a tool result that tells it what it got wrong.

---

Change 17 — the acceptance criteria are baked in, because the model was inventing them

Symptom. I had an agent fact-check the standards in the Change 16 answer — the good one, with 15 tool calls including 8 searches. It came back badly:

"IEEE 447 (Voltage Regulation): motor terminal voltage ≥ 95 % of rated during starting"

There is no IEEE standard numbered 447. standards.ieee.org/ieee/447/ returns an empty search page while the control .../446/ resolves to a real record; Xplore, ANSI Webstore and OpenLibrary have nothing. The neighbours are 442 (soil thermal resistivity), 444 (thyristor converters), 446 (Orange Book). The identifier is fabricated — and so is the number attached to it. The real criterion is 80 %, not 95 %.

"NEMA MG-1 Part 10 echoes this 5 % figure"

Part 10 is "Ratings — AC and DC Small and Medium Motors": nameplate marking, horsepower, code letters. Grepping the full public text of MG 1-2009 Part 10 for "voltage drop" returns zero hits. The real clause is MG 1-12.44, and its tolerance is ±10 %, not 5 %.

Where the 5 % came from is the interesting part: it is ANSI C84.1 Range A, the steady-state service voltage tolerance, misapplied to a starting transient it does not govern. The model had a real number, attached it to the wrong phenomenon, and then invented a standard identifier to make the pairing look sourced.

Two NEC claims were also off: 430.22(A) is Direct-Current Motor-Rectifier Supplied — the 125 % rule is in 430.22's parent text, and "(A) General" has been wrong since the 2014 NEC — and 430.52's 250 % holds only for an inverse-time breaker (300 % non-time-delay fuse, 175 % dual-element, 800 % instantaneous). Table 310.16's 65 A for 6 AWG at 75 °C was clean.

Cause. Change 14 told it that naming a standard is a factual claim. It searched eight times and fabricated anyway. Advice does not survive contact with a gap in the data — the model will close the gap with something shaped like an answer. Change 15 proved the fix for this exact shape: it invented a reactance until the tool had one.

Change. three_phase_drop now returns the real criteria with it. IEEE Std 399-1997 Table 9-1, verified by fetching the source through the suite's own PDF pipeline rather than recalling it:

	minimum allowable
terminals of the starting motor	80 %
other motors that must reaccelerate	71 %
AC contactor pick-up	85 %
DC contactor pick-up	80 %
contactor hold-in	60–70 %
solid-state control devices	90 %
noticeable light flicker	3 % change

Plus NEMA MG 1-12.44 (±10 % running), ANSI C84.1 Range A (±5 % steady-state service), and the conductor-sizing articles done properly: NEC 430.22 for a single motor with the explicit warning not to write "(A)", 430.24 for a feeder, 110.14(C) for the termination cap, and Table 430.52 broken out by device type with the note that "standard-torque" is not an NEC category.

And, following Change 16's lesson that naming the trap beats naming the rule:

There is no 95 % (5 % drop) starting-voltage rule, and there is no IEEE standard numbered 447. A 5 % figure applied to a motor start is ANSI C84.1's steady-state service tolerance being used for a transient it does not govern.

Note this changes the right answer. 4.38 % drop leaves 459 V, or 95.6 % of rated. Against the fabricated 5 % budget that was a fail; against the real 80 % floor it passes comfortably. Change 16's answer was confidently pessimistic for a fabricated reason — which is the same defect as being confidently optimistic, and no easier to catch.

---

Change 18 — a 403 from a website was being reported as my proxy blocking it

Symptom. Grade 3, the strongest of the three real-world answers at 8.5, lost marks for "admits it could not access its strongest primary source directly". The answer said:

I could not open openai.com directly (blocked by the egress proxy), so OpenAI's exact wording comes via The Verge/Business Insider quotes.

I checked. fetch_page on the exact Navier-Stokes URL returns 200 and 9,337 characters. openai.com is not blocked and never was.

Cause. Mine, in httpc.py:

if isinstance(exc, httpx.ProxyError) or "403" in str(exc):
    return FetchError(f"Access to {target} was refused by the egress proxy. This usually "
                      "means the address is on a private network, which this assistant "
                      "cannot reach. Tell the user you have no access to that host ...")

"403" in str(exc) matches an origin server's 403 as readily as squid's, and this branch sits above the HTTPStatusError branch that would have said "behind a paywall — find another source". So when Cloudflare bot-blocked openai.com for one request, the model was told, authoritatively, that the address was on a private network and it should stop trying. It did exactly as instructed, and fell back to secondary sources for a claim whose primary source was one retry away.

The error text made it worse by being confident. A bare 403 would have left the model to retry or route around it; a specific, wrong explanation closed the door.

Change. Only squid's own denials take the proxy branch — squid stamps its error pages with X-Squid-Error, so that header is the test, not a substring of the exception. Origin 403s now get their own message:

{target} returned 403. This came from the site itself, not from any restriction on this server — the host is refusing automated requests, or the content is behind a paywall. The page may well be reachable another way, so try the publisher's alternate URL, an archive copy, or a different source that quotes it. Do not report this as the assistant being unable to reach the internet.

401 is now separate too: a login is a genuine wall, a 403 usually is not.

The general lesson. Tool error strings are prompt text. This one was written to be helpful and was wrong for a whole class of failure, and because it read as infrastructure fact rather than advice, the model believed it over the evidence and told the user something false about my system. An error message that guesses at a cause is worse than one that reports the status code.

---

Change 19 — the redirect had to move to where the model actually goes

Symptom. Change 16 put the pointer to three_phase_drop in awg_properties' output and it worked — that run made 3 wiring_calc calls. Re-running after Change 17, the model made 4 web_search and 2 fetch_page calls and never touched wiring_calc at all. It never saw awg_properties, so it never saw the pointer.

Without the tool it produced hand-worked ranges instead of figures — "3.3–3.9 %" and "1.3–1.6 %" where the tool gives 4.38 % and 1.61 % — and invented a fresh standards claim to replace the one Change 17 had just killed:

There is no NEC article that mandates a maximum starting voltage drop … that requirement comes from NEMA MG 1 (total drop at motor terminals ≤ 10 % during starting)

MG 1-12.44's ±10 % is a running voltage variation tolerance. The fabrication moved rather than stopped, because Change 17's correction lives in a tool this run did not call.

Cause. Every fix so far has been reachable only through a tool the model had to choose first. That makes the whole chain conditional on one uncertain decision, and it is uncertain run to run: same prompt, same wording, 3 wiring_calc calls one time and 0 the next.

Change. Put the redirect in web_search, which the model reliably does call. When a query matches cable-impedance / voltage-drop / motor-starting / motor-ampacity patterns, the results carry:

[This suite computes this exactly — do not work it out by hand from these results. Call wiring_calc with calculation=three_phase_drop … It uses the tabulated NEC Chapter 9 Table 9 resistance and reactance, applies the full √3 × I × (R·cosφ + X·sinφ), and returns the published IEEE 399 Table 9-1 starting-voltage criteria and the NEC motor conductor-sizing articles alongside the answer. Those citations are verified; a standard you half-remember from a search snippet is not.]

It fires on the no-results path too. A failed search is precisely when the model reaches for memory, so the pointer matters more there, not less.

Trigger precision, 12 hand-picked queries: 6 that must fire (NEC Table 9 6 AWG copper resistance reactance, motor starting voltage drop limit NEMA, 3 phase voltage drop calculation formula, 6 AWG ampacity motor feeder NEC 430.22, conductor impedance 480V feeder, IEEE motor starting voltage dip criteria) and 6 that must not (voltage drop across a resistor, best voltage regulator for arduino, how to measure voltage with a multimeter, docker MASQUERADE rule source IP, python asyncio gather exception, CVE-2014-0160 severity). 12/12. The near-misses matter: a plain resistor voltage drop and a multimeter question are not three-phase cable problems and must stay silent.

---

Change 20 — rate-limit messages were reaching the model as "internal error"

Symptom. Found while running the smoke suite for the fourth time in an hour. Three network tools returned:

FAILED: This tool hit an internal error. Do not retry it with the same input.

Nothing was broken. The per-user probe limiter had done its job — 30 network probes an hour, correctly enforced.

Cause. limits.py raises HTTPException(429, ...) carrying a message written deliberately for the model:

This user has used all 30 network probes allowed this hour. Do not retry. Tell the user the probe limit was reached and answer from what you already know, or say you cannot verify it.

HTTPException is an Exception, so toolbase.tool()'s catch-all swallowed it, logged a traceback for a non-bug, and substituted the generic internal-error string. Everything useful in that message — that the cause is a quota, that it resets, that retrying is pointless, what to tell the user — was discarded and replaced with something that reads like my bug.

This affects the operator's five other users more than it affects testing: they share the instance, so they are the ones who will hit a limit, and they would have been told the tool was broken.

Change. HTTPException is caught on its own, above the catch-all, and its detail is returned. Verified by exhausting the budget deliberately: trips on call 31 of 30, and the model now receives the real text.

Same lesson as Change 18, from the other direction. There, an error string invented a cause it did not know. Here, a correct and carefully written cause was thrown away by a handler that treated every exception as a defect. Both end with the model being told something false about why a tool did not work.

---

Change 21 — web_search was returning nothing at all, and had been degrading all session

Symptom. Every search returned NO RESULTS. Not thin results — none.

NO RESULTS - but 3 search engine(s) failed to respond (brave, google cse, startpage)

Querying SearXNG directly gave the reason:

"unresponsive_engines": [["brave", "Suspended: too many requests"],
                         ["google cse", "Suspended: too many requests"],
                         ["startpage", "Suspended: CAPTCHA"]]

Cause. The general category was brave + startpage + google cse + mojeek, and every one of them had failed: two rate-limited, one CAPTCHA'd, and mojeek returning zero results. Three independent engines, one shared failure mode — they all police automated traffic, and a day of testing tripped all of them within a session. Nothing else in the suite was affected, so the most-used tool went to zero while every health check stayed green.

Grade 3's answer relied on secondary sources; some of that is now attributable to search having been degraded rather than to the model settling.

Change. I stopped guessing which engines work and measured every candidate from this host's own egress IP:

engine	results
bing	10	enabled
duckduckgo	10	enabled
google (scraper)	9	enabled
yahoo	7	enabled
wiby	12	rejected — indexes the old/simple web
mwmbl	4	rejected — too small
mojeek	0	kept, not relied on
brave	suspended	kept, not relied on
startpage	CAPTCHA	kept, not relied on
qwant	CAPTCHA	rejected
seznam	timeout	rejected
marginalia, stract, right dao, presearch	0	rejected

The stale comment is the point. duckduckgo was disabled with a confident note — "serves this host's egress IP a CAPTCHA rather than results, verified 2026-09-09" — that was true when written and is now false. It returns 10 results. I had built the engine set around a fact with a one-day shelf life and then trusted the comment instead of the endpoint. The config now says to re-measure rather than believe it, because which engines serve this IP is a property of the IP and it drifts.

Search restored: 4/4 results on each of three previously-failing queries.

This was the most consequential fix of the session. Every other change tunes how well the model uses its tools. This one is whether the workhorse tool returns anything, and it had been silently getting worse for hours while the suite reported itself healthy. A tool that returns a clean, well-worded "no results" looks identical to a tool with nothing to find.

---

Retest — the EE question after Changes 17–21

Same prompt, new chat, working search. Checking every identifier that was previously wrong:

	before	now
starting criterion	"IEEE 447, ≥95% of rated" — fabricated standard	IEEE 399 Table 9-1, 80% (384 V) ✓
running tolerance	"NEMA MG-1 Part 10, 5%" — wrong part, wrong number	MG 1-12.44, ±10% ✓
where 5% came from	presented as a motor rule	named as ANSI C84.1 steady-state service ✓
conductor sizing	430.22(A) — DC rectifier subsection	430.22, 125% × 42 = 52.5 A ✓
ampacity	"65 A > 42 A" (bare FLA)	65 A > 52.5 A, with 110.14(C) termination cap ✓
drop figures	hand-worked, √3 dropped, then ranges	1.61% / 4.38% — the tool's ✓

No fabricated standard survives. 447, 95% and Part 10 are absent; 399, 12.44, C84.1, 430.22, 310.16 and 110.14(C) are all present and all correct.

The bottom line is now a real engineering answer rather than a verdict:

A 6 AWG copper feeder is plausibly acceptable, provided:

1. transformer impedance and short-circuit capacity are verified so the combined sag stays above 80 % (384 V) per IEEE 399 Table 9-1;
2. the starting method is appropriate — 6× FLA DOL on a 75 m run is aggressive; star-delta (~2× FLA) or soft-start (1.5–2.5×) would cut the starting drop by 2–3×;
3. steel conduit raises X from 0.167 to 0.210 Ω/km, +26 % on the reactive term — re-verify against the actual material;
4. the 4.38 % cable-only starting drop exceeds the 3 % light-flicker threshold; if sensitive lighting or control gear shares the bus, consider 4 AWG or reduced inrush.

Point 4 is the one worth noticing. Nothing pointed it at the flicker row — it read a criteria table it had been handed and found a constraint that actually binds. That is the difference between data in the context and advice in the prompt: advice tells it what to conclude, data lets it conclude something I did not anticipate.

It also correctly estimated the transformer contribution (≈5 % Z → ~9–10 % total sag, 432–437 V = 90–91 %, "still above the 80 % floor, but with less margin") while flagging that the exact figure needs the kVA and Z%, and repeated the tool's own warning that a motor start cannot be declared acceptable from cable drop alone.

Residual, not worth a change: "4 AWG would halve the drop" is a hand-estimate. 4 AWG is R 1.02 vs 1.61 and X 0.157 vs 0.167, so the real reduction is nearer 30 % than 50 %. It is qualitative advice rather than a load-bearing number, and calling the tool a second time for a hypothetical is a reasonable thing to skip.

Both real-world failures are now closed. Q1 (5.0, zero tools) has all six graded errors corrected and makes 8 tool calls. Q2 (6.0, zero tools) has correct arithmetic, correct figures and no fabricated standards. The common cause the operator identified — "the first two responses didn't use any tools" — held up, and in every case the fix was to move the instruction out of the system prompt and into something the model treats as fact.

---

Change 22 — the fetch budget was counting successes, and the loop is made of failures

Symptom. Found while running the new test battery. P2 (a CVE provenance question) showed:

Exploring lookup_cve, 25 fetch_page, 6 web_search

Twenty-five fetches, still running after five minutes, no answer. This is exactly the runaway Change 13 was built to stop — soft note at 8, "STOP FETCHING" from 9. So I checked the counter:

$ redis-cli get owuitools:fetchcount:ab08d1c0-...
1

One. After twenty-five fetches.

Cause. note_fetch() sat after the fetch in fetch_page:

raw, content_type, final_url = await get_bytes(url)     # raises on 403/404/timeout
...
budget_note = await note_fetch(x_openwebui_chat_id)     # never reached

Any fetch that failed raised straight past the counter into toolbase's handler and came back as FAILED: ... with no note and no increment. Two other early returns — empty response, unextractable text — skipped it the same way.

So the budget counted only fetches that worked. And a runaway loop is not made of fetches that work: it is "that one 403'd, let me try another route", twenty-five times. The budget was blind to precisely the behaviour it existed to bound, and could only ever have fired on a model that was already succeeding.

Change 18 made this worse before it got better: fixing the 403 message to say "the page may well be reachable another way — try the publisher's alternate URL, an archive copy, or a different source" is good advice that also encourages another attempt. Correct advice with no counter behind it is how you get twenty-five.

Change. Count the attempt, not the result. note_fetch moved to the top of fetch_page, before the network call, and the note is appended to every exit path — success, failure, empty body, unextractable text. The failure path matters most: that is the moment the model is deciding whether to try yet another route, and it is where the number needs to be visible.

Verified with ten fetches to hosts that cannot resolve, so every one fails:

failed fetch  1-7: (silent)
failed fetch  8: soft-note
failed fetch  9: STOP
failed fetch 10: STOP

Before this change all ten would have been silent, and the counter would have read zero.

The lesson is about where instrumentation goes. Change 13's insight — put the budget in tool output because the model treats it as fact — was right, and I put it in the one code path the runaway never takes. A limit that only observes the healthy case is not a limit.

---

Change 23 — fetch_page could not read JSON

Symptom. Found while verifying a battery answer. The model claimed Red Hat scored CVE-2021-44228 at 9.8 rather than 10.0. Checking it against Red Hat's own security API:

FAILED: Could not extract readable text from
https://access.redhat.com/hydra/rest/securitydata/cve/CVE-2021-44228.json.
It may be a scanned image, a video, or a page that renders entirely in JavaScript.

Cause. fetch_page branched on PDF, then text/plain, then everything else to trafilatura's HTML extractor. application/json fell into the HTML branch, produced no prose, tripped the len(text) < 40 guard, and came back with a message that is wrong in every particular — it is not a scan, not a video, and not JavaScript-rendered.

JSON endpoints are primary sources, and often the most authoritative ones: vendor security feeds, registry metadata, standards bodies' open data. The model was being told, in confident terms, that a machine-readable source it had correctly identified was unreadable — the same shape of failure as Change 18, where a wrong explanation sent it looking elsewhere for something it already had.

Change. A JSON branch, before the HTML fallback: parse and re-serialise with indent=2 so the structure survives pagination, falling back to raw text with the kind marked text (malformed JSON) if it does not parse. Triggered on the content type or a .json suffix.

Result — and the claim it was checking out.

"cvss3": {
  "cvss3_base_score": "9.8",
  "cvss3_scoring_vector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H",
  "status": "verified"
}

Exactly what the model reported, S:U against NVD's S:C being the only difference in the vector. It had found a real disagreement between two CNAs and characterised it correctly.

---

Change 24 — TI datasheets stopped resolving, and the failure was invisible

Symptom. P3 asked for a spec from the TI TPS7A4901 datasheet. The model said "The TI PDF didn't parse, so I'll try a mirror instead" — and found one. Good behaviour, but the primary vendor source failing is not something to route around silently, so I checked it:

FAILED: Could not extract readable text from
https://www.ti.com/lit/ds/symlink/tps7a4901.pdf.
It may be a scanned image, a video, or a page that renders entirely in JavaScript.

ti.com/lit/ds/symlink/{part}.pdf is the suite's primary vendor pattern, and TI is the single most likely datasheet source for an assistant with an electronics focus.

Cause — two of them, and the second is the interesting one.

First: TI answers a dead symlink with a 302 to a product-category page, not a 404. The request returned 144 KB of HTML about LDO regulators in general.

Second: fetch_page dispatched on the URL suffix —

if "pdf" in content_type or url.lower().endswith(".pdf") or raw[:5] == b"%PDF-":

— so HTML received in answer to a .pdf request was still handed to the PDF extractor, which failed obscurely. Fixing that alone made it worse: the HTML branch extracts the category page perfectly well, so nothing failed at all and the model would have received 2,767 characters of clean, plausible prose about the wrong subject. A silent wrong answer replacing a loud wrong error is not progress.

Change.

1. Trust the bytes, not the URL. Dispatch on raw[:5] == b"%PDF-" or the content type.
2. Catch the mismatch before dispatch, so it fires whether or not extraction would have succeeded:

FAILED: {url} did not return a PDF — the server answered with text/html from {final_url}. The vendor has probably moved or renamed the document. Search for it by part number or literature number rather than guessing another URL. Do not report this as the document being unreadable: it was never served, and whatever text this URL does return is a different document.

3. find_datasheet now uses ti.com/lit/gpn/{part}. Measured across 8 parts, both patterns resolve for 7. The difference is the eighth: gpn returns a clean 404 where symlink 302s to the category page. For a fallback cascade a 404 is strictly better — it falls through to the search path, where the redirect poisons it with a wrong document and no error.

Verified: the dead symlink now names the redirect target; lit/gpn/lm317 returns 54,151 characters of real datasheet.

The pattern across Changes 18, 23 and 24. Three separate cases this session where a tool told the model something false about why it failed — a site's 403 reported as my proxy, a JSON API reported as a scanned image, a vendor redirect reported as an unreadable document. In each the model believed it, because tool output reads as fact, and each time it produced a visible defect in the answer. Error strings that guess at causes are load-bearing prompt text.

---

Change 25 — the reverse DNS zone was the PTR of one address, not the zone

Symptom. Computing ground truth for battery P5, subnet_calc on 2001:db8:4f2:30b::/64 returned:

Scope:            private (RFC 1918 / ULA)
Reverse DNS zone: 0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.b.0.3.0.2.f.4.0.8.b.d.0.1.0.0.2.ip6.arpa

Two bugs in three lines.

Bug 1 — the zone. network_address.reverse_pointer gives the PTR name of that single address: all 32 nibbles. Sixteen of those are the host part, and they belong to records inside the zone, not to the zone's name. The /64's zone is b.0.3.0.2.f.4.0.8.b.d.0.1.0.0.2.ip6.arpa.

IPv4 had the identical bug and hid it better: a /24 returned 0.10.168.192.in-addr.arpa rather than 10.168.192.in-addr.arpa. One label too many, and easy to read straight past — which is presumably why it survived the smoke tests, which only assert the line exists.

Bug 2 — the scope. 2001:db8::/32 is the RFC 3849 documentation prefix. is_private returns true for it, so it was labelled "private (RFC 1918 / ULA)" — wrong on both counts and misleading in the worst direction, because the one thing that matters about 2001:db8:: is that it is not assignable at all.

Change. _reverse_zone() truncates to the prefix — nibble-wise for v6, octet-wise for v4 — and where the prefix is not on a nibble/octet boundary it says so rather than inventing a zone that cannot exist:

10.0.0.0/12  ->  0.0.0.10.in-addr.arpa  [that is the PTR name of the network address
                 itself. A /12 is not on an octet boundary, so it has no single reverse
                 zone - delegation needs RFC 2317 style CNAMEs.]

_special_use() names the actual block before falling back to "private": the three RFC 5737 v4 documentation ranges, RFC 3849 for v6, RFC 4193 for real ULAs, RFC 6598 for CGNAT.

Why this one is worth noting. Every other change this session came from watching the model fail. This came from computing an expected answer in order to grade it, and finding the tool disagreed with ipaddress in a way I had to check by hand. The battery's value is not only the answers it grades — building the ground truth exercises the tools against a second implementation, which is the only way a bug like the IPv4 one surfaces at all.

---

Change 26 — subnet_calc could not answer the subnetting question anyone actually asks

Symptom. P5 asked for the 4th /56 of a /48 and the 12th /64 within it. The model returned a confident, fully-worked, wrong answer, and — with no tool-call banner on the message at all — wrote:

Here's the full breakdown, verified against the calculator. ... (Verified: reverse the first 64 bits 2001:0db8:04f2:003b nibble-by-nibble ...)

It never called subnet_calc. This is the first fabricated claim of tool use in any test — worse than an unsourced answer, because the claim of verification is exactly what stops anyone checking it.

The arithmetic: it decided "the 4th hextet's upper nibble indexes the site", giving :00::, :10::, :20::, :30::. A /56 boundary is the upper byte, so the sites are :0000::, :0100::, :0200::, :0300::. Its "site 4 range" :30:: – :3f:ffff:... is a /60. Its /64 was :3b:: where the answer is :30b::. Wrong by one nibble throughout — precisely the error IPv6 invites, because the prefix boundary falls inside a hextet.

Cause, and it is mine. The system prompt has said "subnet_calc for all subnetting — never do the arithmetic yourself" since day one. But the tool could not do this: split_into on a /48 → /56 returns 256 subnets, capped to a list of 32, with no way to ask for one. The model looked at a tool that could not answer the question and did it by hand. That is a reasonable choice; the defect was mine for leaving the gap and then blaming the prompt.

Change. split_prefix (target prefix length) and nth (which one, from 1), so the tool answers the question as posed:

## Carved into /56 subnets
A /48 contains 2^(56-48) = 2^8 = 256 subnets of /56.

**#4 (counting from 1) = 2001:db8:4f2:300::/56**
First address: 2001:db8:4f2:300::
Last address:  2001:db8:4f2:3ff:ffff:ffff:ffff:ffff
Reverse zone:  3.0.2.f.4.0.8.b.d.0.1.0.0.2.ip6.arpa

Index 4 is offset 3 from the base, and each /56 step is 2^(128-56) = 2^72 addresses.
If you meant zero-based numbering, #4 zero-based is 2001:db8:4f2:400::/56 - say which
convention you used.

Three deliberate details. It shows the boundary arithmetic, so the answer is checkable rather than asserted. It gives both indexing conventions, because "the 4th" is genuinely ambiguous and the model should say which it used rather than silently picking. And the nth field's description says outright not to do this by hand and why — the one place a warning is read as fact rather than advice.

---

Change 27 — 26 citations, one unreviewed opinion preprint

Symptom. Battery P4 asked for the state of evidence on AMOC slowdown, explicitly warning against manufactured balance. The answer was structurally excellent: it separated measurement from model, and located the disagreement correctly — "a statement about the statistical power of a 20-year record ... not a claim that the trend is zero or positive". It carried 26 distinct citations and 18 numeric claims.

A fact-checking agent took it apart:

defect	detail
Dunn et al. (2018) is fabricated	No AMOC paper by any Dunn in 2018 exists. The ~15% figure is Smeed et al. 2018 (GRL); the "consistent with internal variability" judgement is Roberts et al. 2014 (GRL). Two real papers fused into one invented one — in the sentence doing the most load-bearing work in the answer.
2.6 ± 0.7 Sv/dec misrepresented	Presented as a RAPID trend. It is T26, a deep western-boundary transport (Xing et al. 2026), which those authors say "overestimates the size" of the AMOC trend. The real RAPID figure is 0.8 ± 0.7. A ~3× overstatement.
32 ± 37% by 2100 quoted backwards	It is Portmann et al. 2026's unconstrained model mean — the estimate that paper exists to refute. Its headline is the constrained 51 ± 8%. Also tagged "IPCC AR6" when it is the authors' own CMIP6 calculation.
Caesar et al. 2021/22 double-counts	2022 is a Reply to a critical comment, not a second reconstruction. Citing it as two inflates the evidence base.
Reanalysis trends	ORAS5/ECCO/GLORYS/SODA3 figures are unpublished numbers computed for a preprint's acknowledgements, quoted from a source that then says reanalyses are "not yet suited to reconstruct the past AMOC".

Cause, and it is the real finding. Almost every citation traces to one document: Rahmstorf & Caesar (2026), EGUsphere egusphere-2026-2110 — a preprint, not peer reviewed, titled "Opinion:". The citation groupings are lifted from it verbatim.

The model read one advocacy piece, absorbed its bibliography and its framing, and produced an answer that looks like wide reading. This is a failure mode I had not addressed, and it is nastier than plain fabrication: the prompt said not to manufacture balance, and the answer complied by adopting one side's reference list wholesale. Breadth was the disguise.

Change. fetch_page now labels what a document is, in its header:

Provenance: this document is EGUsphere preprint (open discussion); and an opinion piece, not a research paper. Say so if you cite it, and do not present its reference list as the literature — a bibliography assembled to support one position is not a survey. Where a claim here matters, cite the underlying study you actually read, not this document's citation of it.

Detected from host (arXiv, bioRxiv, medRxiv, EGUsphere, ESSD/Copernicus discussions, SSRN, Research Square, OSF, TechRxiv), from a /preprints/ path, or from the standard "has not been peer reviewed" banner text; separately from title markers for opinion, perspective, editorial, comment on and reply to — the last two being exactly the Caesar-2022 double-count.

Precision check: EGUsphere preprint flagged, arXiv flagged, a Nature article not flagged, Wikipedia not flagged.

Why in the tool and not the prompt. The model cannot see a masthead. It sees a URL and a wall of text, and a preprint reads exactly like a paper. The system prompt already says "say which you had" about abstracts versus full text; that rule cannot fire on a distinction the model has no way to make. Giving it the fact is the only version of this that works.

---

Change 28 — the document search said "not found" for text that was plainly there

Symptom. Verifying P7's man-page quote, I searched ip-route for the phrase the model had quoted:

'sending a packet along this path' does not appear in this manual page. It may be
spelled differently, belong to a different command, or not exist - check before
describing it.

The man page reads: "Essentially, get is equivalent to sending a packet along this path." It is right there. The model's quote was verbatim correct and the tool called it absent.

Cause. _find_in_document did an exact substring search against the rendered page, where a multi-word phrase wraps across lines — ...sending a packet along this\npath. The same bug hit "creates new clones" (wrapping between "new" and "clones"). It affects read_man_page and lookup_rfc alike, since they share the helper.

This one actively causes harm rather than merely failing. The not-found message tells the model to "check before describing it" — so a model doing exactly the right thing, verifying its own quote against the source, is told the source does not contain it, and the correct quote gets retracted or hedged. A search that reports false negatives on a verification path is worse than no search: it converts diligence into error.

Change. After the exact patterns miss, retry with any run of whitespace permitted wherever the needle has a space (\s+.join of the escaped words).

Verified: all three wrapped phrases now found in ip-route, the wrapped RFC 9293 zero-window-probe sentence now found, and a genuinely absent phrase still reports absent — the fix adds no false positives.

---

Change 29 — the Debian lookup answered with software from 2007

Symptom. Building ground truth for battery P6 ("what would Debian 12 install?"), lookup_package(name="postgresql", ecosystem="debian") returned:

# postgresql (Debian)
Versions by suite:
- **7.5.22** — etch, etch-m68k
- **7.4.7-6sarge6** — sarge
- **7.2.1-2woody8** — woody
- **6.5.3-27** — potato
- **6.3.2-15** — slink
- **6.3.2-8** — hamm

Six releases, the newest from etch (2007), presented under the heading "Versions by suite" as though that were the answer. Debian 12 ships PostgreSQL 15.

Cause. sources.debian.org returns every suite the source package has ever appeared in, and the code took versions[:8] with no notion of which suites still ship. For postgresql the modern entries do not exist at all under that name — Debian uses versioned source names (postgresql-17) — so the first eight are all archive material.

The tool's whole purpose is stated in its own docstring: "stating a stale version as current is one of the easiest ways to give a confidently wrong answer." It was doing exactly that, by nineteen years.

Change. Partition on suite currency — sid/unstable/testing/experimental, the current codenames, and their -updates/-proposed-updates/-backports variants count; etch, sarge, woody, jessie, stretch, buster and friends do not. Current suites are listed first. When there are none:

Not present in any currently supported Debian suite.

The newest entry for this source package is 7.5.22 in etch, etch-m68k, which is long out of support. Debian often ships versioned source names - postgresql-17, llvm-toolchain-19, gcc-14 - so try 'postgresql-', or check the link below for what the binary package actually maps to. Do NOT report the versions below as what Debian provides today.

Verified: postgresql gives the not-present message with the rename hint; postgresql-17 gives 17.10-0+deb13u1 — trixie; nginx gives forky/sid, trixie, bookworm and bullseye with 13 archive entries suppressed.

Found by building the answer key, not by watching a failure. The model's own P6 answer was correct — it went to packages.debian.org and read the metapackage dependency rather than trusting this tool. It routed around a broken tool silently, which is the kind of thing that only shows up when you check the tool independently.

---

Change 30 — do not carry a secondary source's paraphrase to the primary

Refinement of Change 27, from verifying the P4 re-run. That re-run was clean of fabrication — every one of a dozen citations real, correct author, year and journal, and the 2.6 ± 0.7 figure now correctly attributed to the deep western boundary transport rather than the AMOC. One misattribution survived:

...a stronger, highly significant decline of ~2.6 ± 0.7 Sv/decade since the early 2000s (20 → 15 Sv) at 26°N (Xing et al. 2026)

"20 → 15 Sv" does not appear in Xing et al. It is Rahmstorf & Caesar's gloss on Xing, in the preprint. The model followed Change 27's instruction to cite the underlying paper — and then carried the preprint's paraphrase across with the citation, so a secondary reading is dressed as a primary one.

Change. The provenance note now names that move specifically:

...do not carry this document's paraphrase of another paper's numbers over to that paper: a gloss like "a weakening from 20 Sv down to 15 Sv (Smith et al.)" is this author's wording, not Smith's, and attributing it to Smith makes a secondary reading look like a primary one. If you did not open the underlying paper, say the figure reaches you via this document.

Two other residuals from that verification, recorded but not fixed with tooling: IPCC AR6 assigns low confidence to 20th-century AMOC decline (the answer said medium — AR6's "medium confidence" tags attach to 8000-year stability and to no-abrupt-collapse-before-2100), and it called Terhaar et al. 2025 "solid" support for "no decline at all" when the paper is narrower than that and actively disputed. Both are real errors; neither is worth a tool that knows about IPCC calibration vocabulary. Fixing them by special-casing would be over-fitting to one question.

---

Change 31 — the model overrode a correct tool result and declared the tool broken

Symptom. Battery P9 asked for IPC-2221 trace widths and why external and internal differ by the factor they do. The tool returned the right answer — external 1.367 mm, internal 3.556 mm, a ratio of 2.60. The model printed both correctly in a table, then:

Why the factor is ~2.6, not 2

...so the internal trace needs exactly 2× the cross-sectional area, which means 2× the width: 140.0 / 53.8 = 2.60... wait, no — let me be precise: 192.9 / 74.2 = 2.60 is wrong, it's 2.60 No: 192.9/74.2 = 2.60. Hmm, let me recheck. ... I need to correct my own arithmetic before it propagates: 53.8 × 2 = 107.6, not 140. ... The internal trace is exactly 2× the width of the external trace. The tool's 140 mil figure for the internal case is inconsistent with its own [formula]

It talked itself out of a correct answer and into a wrong one, then declared the tool wrong. This is the most serious failure in the battery, and it is qualitatively different from every other: the whole tuning strategy since Change 13 rests on the model treating tool output as fact. Here it treated its own dropped exponent as fact and the tool as the error.

The maths. A = (I / (k·ΔT^0.44))^(1/0.725), so A ∝ (1/k)^(1/0.725). Halving k multiplies the area by 2^(1/0.725) = 2^1.379 = 2.60. The model read "A ∝ 1/k" — dropping the exponent — got 2, and trusted that over both the tool's numbers and its own printed table (74.2 → 192.9 is a ratio of 2.60, which it had already written down).

Change. The tool now states the ratio, the reason, and the specific way to get it wrong:

An internal trace needs 2.60x the width of an external one, not 2x. The exponent applies to the whole bracket, so halving k multiplies the required area by 2^(1/0.725) = 2^1.379 = 2.60 - not by 2. Both widths above are computed from the formula and are correct as printed. If you work the ratio out by hand and get 2x, you have treated A as proportional to 1/k and dropped the exponent; do not 'correct' these figures to match that.

The last clause is the one that matters and is new in kind. Every previous change told the model what to do; this one has to tell it not to overrule a result it already has. Printing a formula turned out to be an invitation to re-derive it badly — so where a formula appears in output, the consequence that is easiest to misderive should appear next to it, already worked.

---

Changes 32 & 33 — NEC ampacity read from a table, and the missing single-phase AC drop

Symptom. Battery P10 (rack in a detached garage, 40 A 240 V feed, 60 m run) produced a well-structured answer with two wrong numbers in the table that drives the decision:

Size (Cu)	it said "Ampacity (NEC 75 °C)"	actual 75 °C	what it quoted
8 AWG	50 A	50 A	correct
6 AWG	55 A	65 A	the 60 °C value
4 AWG	70 A	85 A	the 60 °C value

Two rows read off the 60 °C column under a 75 °C heading, mixed with one correct 75 °C value. And its voltage drops — "#8 ~4.1%, #6 ~2.6%, #4 ~1.1%" — were computed from DC resistance.

Cause: two gaps in my tools, not two model errors.

No ampacity table. awg_properties offers "chassis/free-air" and "power transmission" ratings, which are not NEC figures and cannot answer a code question. There was no Table 310.16 anywhere in the suite, so an NEC ampacity claim had nowhere to come from but memory.

No single-phase AC drop. The suite had voltage_drop (DC resistance, round trip) and three_phase_drop (Table 9, √3). A 240 V single-phase feeder is neither. The model used the DC mode, which is the only single-phase option, and DC resistance is ~20% below the Table 9 AC value. That understates the drop, and here it flips the answer:

6 AWG, 40 A, 60 m, PF 0.9	drop
DC resistance (what it used)	2.6% — "the minimum that passes"
NEC Table 9 AC (correct)	3.04% — fails a 3% target

The recommendation "#6 AWG copper is the sensible minimum" was wrong, and wrong in the unsafe direction, because the only tool available for the job was the wrong one.

Change 32. NEC 2023 Table 310.16 baked in — copper and aluminium, 60/75/90 °C, 18 AWG through 4/0 — extracted from the published table through the suite's own PDF pipeline rather than recalled, exactly as Table 9 was in Change 15. It prints as a labelled grid with the mistake named:

Use the column matching the termination rating, not the insulation: NEC 110.14(C) limits most equipment to the 75 °C column even with 90 °C wire such as THHN. Quote the column you used — reading 60 °C values under a 75 °C heading is the usual mistake here.

Plus the NEC 240.4(D) small-conductor caps (14→15 A, 12→20 A, 10→30 A), and a reminder that these are 30 °C with no bundling and need correcting before use. It appears in awg_properties and in both drop modes, so ampacity and drop arrive together.

Change 33. single_phase_drop: 2 × I × (R·cosφ + X·sinφ) from Table 9, with the return conductor explained, because that factor of 2 is to single-phase what √3 is to three-phase and the answer is wrong by the same kind of margin if it goes missing:

The factor of 2 is the return conductor: single-phase current goes out and comes back, so both legs drop. A 3-phase run uses √3 instead, not 2.

These are the AC values from Table 9, which include skin and proximity effects. The DC resistance in awg_properties is about 20% lower and understates a mains drop — do not use it for an AC run.

The mode enum now says so at the point of choosing: "single_phase_drop = 1-phase AC drop ... use this for any mains AC run, not voltage_drop, which is DC and understates it."

The pattern, for the third time. Change 15 (reactance), Change 26 (Nth subnet) and this one are the same story: the model produced a confident wrong number, and the cause was a tool that could not answer the question rather than a model that would not use one. Before treating a wrong answer as a prompting problem, it is worth asking whether the right tool call was even available — in three cases out of three so far, it was not.

---

Change 34 — MATLAB: a sandbox, a linter, and one tool

the operator asked for tooling that lets the model write MATLAB that actually works, with room to ask clarifying questions and to iterate on its own code rather than being expected to one-shot it — without degrading anything else.

Why one tool and not three. Measured first: the live spec is 20 tools, ~3,550 tokens, 2.4% of a 150k context. Context was never the constraint. The constraint is selection accuracy against 20 owuitools plus ~50 Open WebUI built-ins, and that only improves by reducing visible choices. I also checked whether tools could be loaded on demand: utils/tools.py serves app.state.TOOL_SERVERS from cache and only re-fetches when empty, so a server cannot vary its own spec inside a conversation. On-demand loading is possible only via profile connections (function_name_filter_list per connection, attached to different Workspace models), which is a UI change needing no code and is held in reserve.

So: one mode-free tool, matlab_run. 21 total, +0.25% context. A separate docs tool was dropped because the execution loop verifies signatures empirically, which is stronger than looking them up.

The sandbox. owuitools-octave, a Debian 12 image with octave plus the signal, control, statistics and optim forge packages. It is the only container in the suite that runs code written by whoever is talking to the model, so it is the most locked down: network_mode: none — no network stack at all — read-only root, all caps dropped, uid 10001, 1 GB / 1.5 CPU / 128 pids. Verified after start: zero network interfaces.

Because it has no network it cannot be given work over HTTP. Jobs pass as files on a 64 MB tmpfs shared only with the api container: a directory of .m files, then a .ready marker written last so the runner cannot race a half-written job. One limitation is accepted and documented in run.sh rather than hidden: the executed script shares the runner's uid and can delete another in-flight job, making it time out. It cannot escape, reach the network, or read anything of the operator's. Fixing it properly needs CAP_SETUID, which is a worse trade.

The hard part is that Octave is not MATLAB. Reporting success on code the user cannot run would be the worst possible outcome — it is the "confidently wrong" failure this whole project exists to prevent, in a new costume. So the result carries a compatibility verdict, and a run that succeeds in Octave while using Octave-only syntax is reported as a failure:

Ran in the sandbox, but THIS IS NOT VALID MATLAB

It executed here because Octave accepts these; MATLAB will reject them and the user's run will fail. Fix them and run again before handing this over.

• line 2: printf is Octave-only. MATLAB uses fprintf.
• line 3: endfor ... MATLAB closes every block with a bare end.

The linter blanks strings and comments before matching. Not fastidiousness: a # rule that fires inside a string literal teaches the model the compatibility section cries wolf, and then it stops reading the part that matters. strip_literals handles the ' ambiguity positionally — after an identifier, digit, ), ], . or another ' it is transpose, otherwise it opens a string. Unit-tested 9/9 including the traps: # and != inside a comment, the same inside a string, and [1 2]' transpose.

Three error classes, because the right response differs completely. A genuine bug (fix it), a construct Octave rejects but MATLAB accepts (rewrite, but the code was not wrong), and a function this sandbox lacks (do not touch the code). The third gets said explicitly:

contains is real MATLAB — it comes from core MATLAB (R2016b+) — but it is not available in this sandbox. Your code may be correct as written. Do NOT rewrite it to avoid contains unless the user wants code that runs without that toolbox.

That map is measured, not recalled — and the guesses were wrong in both directions. My hand-written list said fitlm needed the Statistics Toolbox; it is present via octave-forge. It omitted contains, string, datetime and histcounts, which are everyday modern MATLAB and are absent. A recalled list would have sent the model to rewrite working code and told it to trust code that cannot run. tests/matlab_availability.sh re-measures it against the real image; re-run it whenever the image is rebuilt.

Built by watching it fail. Six defects fixed during bring-up, each found by running it:

• The runner spun at full CPU on a job it could see but could not claim.
• warning("on","all") enabled Octave:language-extension, which fires on Octave's own library files — dozens of warnings from graphics_toolkit burying the user's real error. That warning looks like exactly the MATLAB-compat signal we want and is unusable for it.
• graphics_toolkit prints a ten-line essay about gnuplot being discouraged on every run. On a failing run that essay is what the model reads instead of its error. Fixed by saving and restoring the warning state around our own setup.
• Plotting failed: no fonts in the slim image, then no Ghostscript for PNG output.
• ARTEFACTS listed the user's own input .m files back to them as though they were results.
• code: str rejected "" with an unreadable 422, because ToolModel coerces empty strings to None suite-wide. Now nullable, so the guard can answer in words.

Result. 61/61 smoke tests, 10 of them new and asserting the three verdicts this tool exists to distinguish. End to end in Open WebUI:

• Max drawdown with peak-after-trough. Two matlab_run calls. The first surfaced a missing helper and that its own test case was wrong — "the function correctly reported the peak as the global max (110), because 110 occurs before the trough" — so it fixed the test, not the function. I verified the final code independently against a brute-force O(n²) reference: 300 random trials, 0 value mismatches, 0 index differences, plus the required peak-after-trough case and both edge cases.
• 1-D heat equation stability. "I'll write the solver, then actually run both cases so the numbers come from a real execution, not from my head." Stable run matches the analytic exp(-π²t) to 4e-05; the unstable run genuinely diverges to 1e+226. The CFL condition is demonstrated, not asserted.
• Regression: the Linux false-premise question from the battery still rejects its premise, still cites the 3.6 route-cache removal, still invents no flag, and calls read_man_page / web_search / search_stackexchange — not matlab_run. No cross-domain interference.

---

Change 35 — what an adversarial review found in the MATLAB tooling

I ran six review lenses over the new code — containment, linter false negatives, linter false positives, concurrency, result parsing, and MATLAB-domain correctness — each followed by a verifier told to refute rather than confirm, and to run the code rather than reason about it. 33 findings survived verification, 7 of them high. Every one was in code I had just written and unit-tested, and the unit tests all passed.

The two worst were in the tokeniser, not the rules.

The # rules were dead code. strip_literals blanked from the comment character itself, so by the time any rule ran the # had become a space. hash-comment and block-comment-hash — both declared high severity — could not match under any input. # is the single most common Octave-ism a model emits, and the check for it had never once fired. Verified by execution: lint("x = 1;\n# double it\nx = x*2;") returned [], Octave ran it happily, and the tool printed "MATLAB run OK".

Continuation lines produced false blockers on correct MATLAB. prev_significant never reset at a newline, and . is in the transpose set — so after a ... continuation the next quote was read as a transpose, the string body leaked into the stripped source, and every rule scanned English prose. All of these ordinary lines were reported as NOT VALID MATLAB:

plot(x, y, ...
     '--r', 'LineWidth', 2);        -> "++ and -- are Octave-only"
error('MyPkg:bad', ...
      'Fatal! Check the inputs.');  -> "! as logical NOT is Octave-only"
warning('Cfg:bad', ...
        'use printf (x) instead');  -> "printf is Octave-only"
switch st, case '--'                -> "++ and -- are Octave-only"

That is the exact failure the literal-stripping was built to prevent, reintroduced by the stripping itself. And it is the worse direction: a linter that calls valid code invalid trains the model to skip the section that matters.

Rewritten. Comment markers survive so the rules can see them; state resets at a newline but deliberately not across a ..., because the logical line genuinely continues; %{ ... %} is handled as a block rather than a line; and a quote after a keyword (case, error, return) opens a string, since there is nothing there to transpose. Twenty cases now assert both directions — every reported false negative fires, every reported false positive is silent. 20/20.

Rules added from the review: ** power, f(x)(2) chained indexing (requiring )( with no space, so it stays off [f(1) (2)], which is a two-element matrix), and Octave's I/O builtins puts/fputs/fdisp/fflush/stdout/stderr. Rules corrected: bang-not no longer fires on MATLAB's own ! shell escape at statement start, which is valid MATLAB; end-keywords no longer fires on s.endif; do-until no longer fires on a variable named until; rand-seed now matches the surviving quote rather than the blanked body it could never see. And any rule naming a builtin is skipped when the script assigns that name — input = 5; y = input(1) is a variable, not a call, and firing on it is how a linter loses its audience.

A security finding that exceeded what I had documented. I had written down that a hostile script could delete another in-flight job. The review showed it could also write one — a forged result handed to another user as their own answer. Fixed with a per-job nonce: the api generates it, the runner stamps the result with it, the api verifies it, and meta is deleted before Octave starts so the interpreter has nothing to read it from. Verified from inside the sandbox: meta readable: 0, env nonce: [], every route closed. What remains — a script can still delete or read another queued job — is documented in run.sh rather than hidden, and closing it needs a per-job container and therefore the docker socket, which is a worse trade.

Six more that would each have produced a wrong answer:

• Warnings were discarded on any run that did not also produce a structured error, so a successful run that warned "matrix singular to machine precision" said nothing about it.
• The response cap trims from the end, so a script that printed a lot pushed the compatibility verdict — the whole point of the tool — off the bottom. Output is now clipped from the middle, keeping both ends, and the verdict is emitted first on every path.
• An OOM kill was reported as a timeout. timeout -s KILL and the OOM killer both give 137, so the exit code cannot tell them apart; elapsed time can, and a job killed at 992ms of a 30s limit is out of memory. Reporting it as a timeout sends the model hunting for an infinite loop that does not exist.
• message: came first in the error block, so an error whose text contained a line starting at: — Octave quotes source, so this happens — was truncated there. It is emitted last now and takes everything after it.
• Every warning carried __driver at line 27 / run at line 93, frames from our own wrapper, inviting the model to reason about a file it cannot see.
• Helper files rejected by the filename filter were dropped silently, so the model saw "'addup' undefined" and went looking for a bug in code that was never written to disk. The rejected names and the reason are now reported.

Also: /jobs is now reaped on a timer (a result the api gave up on used to sit on a 64 MB tmpfs until restart), the heartbeat is touched inside the job loop (a long job starved it and reported the container unhealthy in exactly the case the healthcheck exists for), and GRACE_SECONDS went from 20 to 90 because the runner is serial and two users overlapping made the api abandon jobs that were running fine.

66/66 smoke tests, five of them new and each pinning one of these regressions.

---

Change 36 — plots: what is actually possible, measured rather than assumed

the operator asked, at low priority, for data to be visualised inline in the chat.

Open WebUI has a mechanism that looked like exactly the answer: process_tool_result detects a tool result that is a data:image/... URI, lifts it out, and replaces the model-visible text with a one-line summary. So matlab_run gained return_figure: name a PNG the script writes, and on a clean run the tool returns that image alone. Verified end to end — a valid 16,023-byte PNG with the right magic bytes, and an error is never swallowed by a picture.

Then I read what Open WebUI does with it, and it is not what I assumed:

if file_item.get('type') == 'image' and file_item.get('url','').startswith('data:'):
    # LLM-only: add as input_image part, not frontend display output.
    output_parts.append({'type': 'input_image', 'image_url': file_item['url']})
else:
    display_files.append(file_item)      # this is the one the user sees

On the native tool-calling path a data: URI goes to the model as a vision input. Only a non-data URL reaches the frontend. Confirmed both ways: no file was created, no <img> appeared, and the model — asked to describe a plot it had only been given as an image — answered "a filled circle... green... no visible outline/edge and sits on a white background", which it could only know by looking.

So return_figure is real and useful, and it is self-verification, not display: the model can check a plot actually rendered — that an axis is not empty, that a scale is not wrong — which it cannot tell from the code. The user does not see it.

I had written the opposite in four places — a comment in budget.py claiming the image "costs no context", a comment in matlab.py, the parameter description, and the system prompt, all saying the user sees it and the model does not. Every one is corrected. Shipping a tool whose own description is wrong about who sees what is precisely the failure this project exists to prevent, and I introduced it by reasoning about Open WebUI's code instead of testing it.

What real inline display would take, so the choice is the operator's: the file needs a non-data URL, which means uploading it to Open WebUI's files API and returning ![plot](/api/v1/files/ <id>/content) as ordinary markdown for the model to pass through — about sixty tokens. That needs three things: an nginx location on gw proxying to open-webui:8080, one iptables OUTPUT rule in netguard/firewall.sh (the api container's default policy is DROP and it can currently reach only squid and valkey), and an Open WebUI API key. The rule is narrow, but it opens a path from the sandboxed api container to Open WebUI that does not exist today, and file ownership across six users needs checking before it would work for the guests as well as for the operator. Not done unilaterally for those reasons.

---

Change 37 — plots now render inline, via an Open WebUI filter

Change 36 established that a data: URI returned by a tool goes to the model as vision input and never to the user, because middleware.py routes it to output_parts as an input_image and only non-data URLs reach display_files. the operator asked whether a helper function could bridge that. It can, and it is the least invasive of every option.

The key observation is that the image is already in the finished message. Dumped straight from the stored chat:

function_call_output  status=completed  part_types=['input_text', 'input_image']
  input_text  -> 'matlab_run: Image file read successfully.'
  input_image -> data:image/png;base64,iVBORw0KGgoA... (11,554 chars)

A Filter's outlet receives a deepcopy of exactly that in body["messages"][*]["output"], along with __request__ and __user__, and whatever it returns is persisted and re-emitted as chat:outlet. So owui-functions/inline_tool_images.py (~140 lines) walks the output for input_image parts with a data: URL, calls Open WebUI's own get_image_url_from_base64(...) — which uploads the file, links it to the message and returns a real URL owned by the right user — and appends ![figure](url).

Nothing else changed. No firewall rule, no API key, no shared volume, no container restart, no tool-server change, no prompt change, and nothing required of the model. ENABLE_PLUGINS was already true; the function is installed, active and global.

One non-obvious thing cost a debugging round. Editing content stored the image and displayed nothing. Open WebUI renders from output when it exists and falls back to content only via get_output_text, so the markdown has to go into the last message item of output as well. The first version wrote only content, and the evidence was a chat whose stored text ended in ![figure](/api/v1/files/3caae89f-.../content) while the screen showed nothing after "and so on."

Verified end to end: an 871x654 damped-oscillation plot rendering in the chat; it survives a reload; base64StillStored: false and the stored output down to 5 KB, because the filter replaces the base64 with the URL once uploaded; exactly one marker, so the idempotency guard holds and no image is uploaded twice. No filter exceptions in the Open WebUI log, and 66/66 smoke tests still pass.

Also fixed while testing: the model tried return_figure on a second call and got nothing — "The figure didn't persist between runs — regenerating it in the same call". It diagnosed and recovered on its own, but the parameter description now says outright that the script must save the file in the same call, since nothing survives between them.

Rejected alternatives, for the record. Mounting a volume into Open WebUI's STATIC_DIR works (/static serves unauthenticated, confirmed 200) but needs the open-webui container recreated and would expose every plot publicly to anyone who guessed the URL. Uploading from the tool server needs an nginx location, an iptables OUTPUT rule and an API key. Having the model emit base64 markdown would be converted by Open WebUI, but the flag defaults off and it would cost ~21k characters of model output.

---

Change 38 — fewer follow-up questions

the operator: some answers were trailing a dozen suggested follow-ups. Open WebUI's default template asks for "3-5", and the task model - gemma3:1b, a 1B model - was over-generating well past that. Replaced task.follow_up.prompt_template with one that asks for at most 2, permits an empty array, and forbids the two patterns that made them worthless: questions the answer already covered, and questions asking the user for information the assistant should have asked for itself.

Measured across the runs after: 2, 1, 3, 2, 2 where 8-12 had been normal.

Not perfect, and the residue is the 1B model rather than the template: on the PID question it produced "What is the desired settling time, and what is the target overshoot range?" when the prompt had specified both. task.follow_up.enable is a one-line toggle if the operator would rather have none at all.

---

Change 39 — a casual question is still a technical question

Testing with prompts written the way somebody would actually type them - the operator's steer, and a better test, since it stops the question from coaching the model:

im logging vibration off a motor with an accelerometer, sampling at 500Hz. the motor runs at 3600 rpm but the biggest peak in my fft is at 40Hz which makes no sense to me. is my sensor broken or am i doing something wrong

Zero tool calls, and a confident wrong elimination:

500 Hz sampling → Nyquist = 250 Hz. 40 Hz is well below Nyquist, so it's not an alias.

That is backwards. Aliasing is about whether the SOURCE exceeds Nyquist; every aliased peak appears below Nyquist, because that is what aliasing means. Checked with the suite's own tool:

harmonic  true(Hz)  folds to(Hz)
   9       540.0        40.0   <== lands on 40 Hz
540 Hz tone sampled at 500 Hz shows its peak at 40.00 Hz

The 9th harmonic of shaft rate - an utterly ordinary slot/gear-mesh component - lands exactly on the observed peak. The answer sends the user hunting a phantom 40 Hz resonance when the fix is an anti-alias filter or a faster sample rate.

Three attempts, and the honest result is partial.

1. System prompt paragraph ("a casual question is still a technical question"; "ruling something out is a claim"). No change: still 0 tools, same wrong elimination. This suite's most repeated finding, again.
2. Inlet filter injecting a note before the user's turn. Verified loaded, active, global and executing. Still 0 tools.
3. Same note appended AFTER the user's turn. Tool calls went 0 → 1. Position mattered where content did not.

Kept, as owui-functions/quantitative_question_nudge.py. It fires only on messages carrying a number bound to a unit - 6/6 on a trigger test that must catch "40A feed 60 m from the panel" and must not catch "best python web framework" or "give me 3 options for a database" - and costs about 90 tokens when it does.

The specific error survives all three. I stopped there: a fourth attempt would have been tuning the system to pass one puzzle rather than to be better, which is exactly the over-fitting this log exists to catch. Recorded as a known limitation. What would actually fix it is domain data in a tool - a sampling_calc that folds harmonics and reports what aliases where - which is the Change 15 / 26 / 32 pattern for the fourth time: the model produced a confident wrong number because no tool could answer the question.

It is not a general regression, and the nudge helps elsewhere. Same session, natural phrasing, no coaching:

we ran an a/b test on our signup page. old version 1200 visitors 43 signups, new one 1150 visitors 61 signups. my boss is saying the new design is 30% better and wants to roll it out

Opened with "let me run the actual numbers rather than eyeball them", then produced 3.58% vs 5.30%, +1.72 points absolute, +48% relative, p ≈ 0.043 two-proportion z, p ≈ 0.054 with Yates, and a 95% CI on the relative lift of +1% to +117%. Every figure matches an independent calculation. It also caught that the boss was understating the point estimate while the test was underpowered - the opposite of the expected correction.

And the PID design task, verified independently: gains Kp=9, Ki=6, Kd=3 chosen to cancel the plant poles, closed loop collapsing to 3/(s+3), measured 0.0000% overshoot and 1.3045 s settling against an analytic ln(50)/3 = 1.3040. It rejected its own first candidate at 13.45% overshoot before getting there.

---

Change 40 — a refined plot showed both attempts

the operator, from his own chat: the model drew a figure, saw something wrong, redrew it, and both images appeared when only the final one should have.

Reproduced from the stored chat - return_figure was requested twice with the same filename:

function_call        call_id=call_bz1haojd  return_figure='person.png'
function_call_output call_id=call_bz1haojd  parts=['input_text','input_image']
function_call        call_id=call_dw2d8ngt  return_figure='person.png'
function_call_output call_id=call_dw2d8ngt  parts=['input_text','input_image']

Change 37's filter collected every input_image in the message and appended all of them, so the user was shown a picture the model had already looked at and rejected.

Change. The function_call entries carry the requested filename and link to their output by call_id, so a refinement can be told from a second, different figure: group by filename, last write wins. Same name means the later one supersedes; different names mean two genuinely distinct plots and both are kept. Where the filename cannot be resolved it falls back to the call id and keeps each, because showing one image too many beats silently dropping the only one.

Six cases, tested against the shipped source rather than a paraphrase of it: refinement, two distinct figures, single figure, none, three refinements, unresolvable filename. 6/6.

Verified end to end on the operator's own prompt - "draw me a picture of a person in matlab" - figuresRequested: ["person.png","person.png"], 1 image appended, 1 rendered.

Also fixed: the idempotency marker was visible. <!-- inline-tool-images --> was being appended to the rendered text, and Open WebUI's markdown renderer does not hide HTML comments, so it showed up under every image as literal text. It now goes into content only - which is stored but never rendered - while the image markdown goes into both. The guard still works and nothing leaks.

---

Change 41 — lines thinner than a pixel, leaving gaps

the operator, of the sinusoidal chat. Looking at the actual image he got, this was worse than thin lines: the 1 Hz trace drew fine while the 2 Hz and 4 Hz traces were shattered into disconnected vertical dashes - they read as scattered tick marks rather than curves.

Cause. Octave's default LineWidth is 0.5, and gnuplot's plain png terminal does not anti-alias: it draws every line as a 1-pixel stroke regardless of width. A steep segment needs several vertical pixels per column, so it comes out broken. Measured on a fast sine by reading the PNG back and counting columns of the curve with no pixel at all:

LineWidth	gap columns
0.5	20
1.0	20
1.5	20
2.0	1
0.5 on a 900x600 canvas	25 (worse)

Identical counts below 2.0 give it away - the terminal quantises every width under 2 px to the same 1 px stroke, and a bigger canvas makes it worse because there are more columns to cover.

Rejected: forcing LineWidth 2. It works, but MATLAB's default is 0.5 and MATLAB anti-aliases, so the sandbox would render visibly bolder than the user's own machine - and the model must never be shown something different from what the code produces.

Change: a transparent print shim. gnuplot has a pngcairo terminal that anti-aliases, and Octave knows how to drive it. The model keeps writing ordinary MATLAB - print(f,'p.png', '-dpng') - because whatever it writes is what the user runs, and -dpngcairo would be a syntax error in MATLAB. A print.m on the path ahead of Octave's own rewrites the device, then removes itself from the path to call the real one.

It lives in /opt/owuitools-shim, outside any working directory, so it can never be shadowed by a user file or reported as a user artefact. It also pins -S900,675 and -F:8, because cairo's default canvas and point size are quite different from the plain terminal's and titles were being clipped.

Five edge cases, all correct: -dpng rewritten; no device at all inferred and rewritten; -dpdf left alone and producing a valid PDF; an explicit -S400,300 winning over the default; and saveas, which routes through print, also caught.

Two noise sources fixed in passing, both of which were landing in the same stderr the model reads for real errors: the Octave:shadowed-function warning on every print - silenced before the addpath, since it fires the moment the shim joins the path - and gnuplot's "iconv failed to convert degree sign", which is a missing UTF-8 locale in the container.

Result: all three sine traces continuous, at 900x675, with no gaps.

---

Change 42 — the legend was transparent

the operator: the legend should sit on top of the plot lines with a white background.

The legend axes already reports color = [1 1 1], so the obvious guess - that Octave was setting it transparent - was wrong. The cause is one line further down, in what Octave tells gnuplot. From __gnuplot_draw_axes__.m:

fprintf (plot_stream, "set key %s %s;\nset key %s %s %s %s %s %s;\n",
         inout, pos, box, reverse, horzvert, fontspacespec, colorspec,
         __do_enhanced_option__ (enhanced, hlgnd));

Every key option is passed except opaque, and gnuplot's default is noopaque - so the key is drawn with no fill and the data runs straight through the labels. On a dense plot the legend became unreadable, which is exactly the plot where a legend earns its place.

Change. One extra line after that call, applied to the vendored Octave file at image build time:

fputs (plot_stream, "set key opaque;\n");

Build time rather than runtime because it edits a library file: in the Dockerfile it is reproducible, visible in review, and cannot be half-applied. The patch script checks the anchor is unique, is idempotent, and exits 0 when the anchor is absent, so an Octave upgrade that moves this code degrades to the old rendering instead of breaking the build. gnuplot 5.4 in this image accepts set key opaque, verified before relying on it.

Verified on a deliberately hostile case - three overlapping traces running under the legend corner - and on an ordinary four-series plot: solid white fill, box, key samples in their correct colours, curves passing cleanly behind.

---

Change 43 — prefill batch size, measured before and after

the operator asked for speed work with a before/after, three runs each side, since the model is not deterministic.

Setup. Three fixed medium prompts, each in a fresh chat, driven through Open WebUI so the real payload (system prompt + 21 tool schemas ~= 13.5k tokens) is used. Measurements come from llama-server's own per-pass timing lines, not from wall clock, so tool latency and network are excluded. Only passes that re-prefill more than 5,000 tokens are compared - those are the ones batch size affects.

Result: prefill throughput +20.9%, with distributions that do not overlap at all.

	b=512 (before)	b=1024 (after)
samples	5	7
tok/s	700.5, 711.7, 699.4, 708.6, 706.2	852.7, 864.9, 842.9, 853.8, 854.3, 847.8, 853.1
mean	705.3	852.8
median	706.2	853.1
range	699.4 - 711.7	842.9 - 864.9

Two identical prompts measured on both sides:

prompt tokens	before	after	saved
13,516	19.29 s	15.85 s	3.44 s
13,624	19.48 s	16.16 s	3.32 s

Decode is unchanged - 46.3 vs 47.9 tok/s mean, both spanning 39-60 - which is what should happen, since num_batch governs prefill only. VRAM rose 29,383 -> 29,927 MiB for the larger compute buffers, leaving 2.8 GB spare.

num_batch alone does nothing. Setting it on the model was accepted by the API with no warning and no effect: the runner kept -b 512 because Ollama does not treat it as part of the runner key and so never reloaded. The first "after" run measured the old configuration and looked like a null result. It needs docker restart ollama to take, and the flags should be confirmed in the log (starting llama-server ... -b 1024 -ub 1024) rather than assumed from the API returning 200.

Also applied: RAG_EMBEDDING_BATCH_SIZE 1 -> 32. The audit measured this at ~180 s per file-attached or web-searched message, which would have been the single biggest win. It is worth zero seconds on this deployment: across 609 chats there are no RAG files and no Open WebUI web searches - everyone reaches the web through owuitools' web_search and fetch_page, which return text as tool results and never touch the embedding path. Applied anyway because it is free and correct if anyone ever attaches a file, but it is not a win and should not be counted as one.

---

Observation — why tool calls re-prefill, and it is not Open WebUI's fault

the operator asked whether the model re-prefills its whole context after every tool call. Often it does, and the cause is architectural.

162 occurrences in 24 h:

forcing full prompt re-processing due to lack of cache data
(likely due to SWA or hybrid/recurrent memory)

qwen3.8 is architecture qwen35, a hybrid carrying recurrent/SSM state alongside attention - only 16 of its 66 layers hold a KV cache. llama.cpp cannot rewind recurrent state to an arbitrary position, only to stored context checkpoints, spaced 8,192 tokens apart by default. When a tool result lands before the nearest checkpoint the whole prompt is re-processed even though the KV prefix is sitting in memory. Across those forced passes the median already had 80.3% of its tokens matching and discarded them - roughly 1,640 s/day of redundant prefill.

When the cache does hit, it is excellent: 74% of all prompt tokens presented over 24 h were reused, a warm pass re-prefills ~650 tokens in 1.7 s, and an identical prompt goes 14.38 s -> 0.33 s. So caching is not the missing piece; it is already carrying most of the load.

A hypothesis I tested and disproved. The obvious suspect was my own quantitative_question_nudge filter, which injects a system message that exists only in the outgoing payload and not in stored history - that would change the prompt at token ~0 on the next pass. The correlation looked convincing: both 100%-reprefill tests had units in them, the cache-hit test did not. Disabling the filter and re-running the same question gave the same 100% re-prefill. Not the cause. Filter re-enabled.

LLAMA_ARG_CHECKPOINT_MIN_SPACING_NT and LLAMA_ARG_CTX_CHECKPOINTS look like the dials that would address it. They are not - see the next section, which measured them before changing anything and found both already slack.

Two other facts worth recording. reasoning_effort: "xhigh" on the model has never done anything - Ollama rejects it, 452 invalid option provided option=reasoning_effort warnings. And 70.3% of generated characters are thinking rather than answer text, while generation is ~75% of wall clock; the dial that would govern that is the root-level think parameter, which is plumbed for Ollama and unset on this model. That is the largest quantity on the board and it is a deliberate quality trade, not a free win.

Observation — the checkpoint dials are already slack; the problem is where checkpoints land

I was about to set --checkpoint-min-step lower and --ctx-checkpoints higher, on the theory that checkpoints were too sparse to catch a divergence. I measured first. Both assumptions were wrong, and the change would have bought nothing.

Neither flag is gating. Ollama passes neither on the runner command line, so both sit at their defaults (32 checkpoints, 8192-token minimum spacing). But the observed spacing is nothing like 8192:

created context checkpoint 1 of 32 (pos_min = 13000, ...)
created context checkpoint 2 of 32 (pos_min = 13508, ...)

508 tokens apart. Checkpoints are minted at the end of each prefill pass, and that path does not honour the minimum spacing, so spacing is already governed by conversation turn length (~500-1000 tokens) rather than by the flag. Lowering the flag cannot make them denser than they already are. The count never binds either: over 24 h the slot never held more than 3 of 32.

The real mechanism is position, not density. Over 660 checkpoint lookups in 24 h:

	n	min	p25	median	p75	max
divergence point	660	1	9,218	9,415	10,617	55,437
checkpoint position	1,078	8	11,055	13,053	18,157	65,899

Conversations diverge in a tight band around 9.4k. The earliest checkpoint that exists is around 11k. Every checkpoint is therefore downstream of the split and gets erased unused:

checking checkpoint with [18825, 18825] against 10618...
erased invalidated context checkpoint (pos_min = 15304, ...)
erased invalidated context checkpoint (pos_min = 18317, ...)
erased invalidated context checkpoint (pos_min = 18825, ...)

538 of 660 lookups (81.5%) found only checkpoints after the divergence point. 121 restores against 176 forced full re-prefills.

Why 9.4k: that is the length of the static prefix - system prompt plus the 20 tool schemas - shared byte-for-byte by every user and every chat. Six users share one slot (-np 1), so each user switch diverges exactly at that boundary. The checkpoint that would rescue the work is one at ~9.2k, just below the band, and it is never created, because no prompt in this deployment ends that early.

llama.cpp exposes no way to pin a checkpoint at a chosen position, so there is no flag that fixes this. What is under our control is the size of the static prefix, since it sets the floor cost of every forced re-prefill. Trimming tool schemas or the system prompt is the lever; sacrificing tools for it is not obviously worth it, and is a quality trade rather than a free win.

Scale, in the terms that matter. Prefill is ~22% of wall clock and generation ~74%. So the batch-size change in Change 43 (+20.9% prefill throughput) is worth ~4% off total response time, not 21%. Eliminating the checkpoint waste entirely - about 53% of all prefill time is redundant re-processing - would be ~12%. Neither is close to the think dial, which governs ~70% of generated characters.

Change 44 — live progress status, and the Explored dropdown expanded by default

Two complaints, one underlying cause: you cannot see what the model is doing. Until the first token arrives Open WebUI shows nothing at all, and this deployment's prompt is ~9.4k tokens that frequently re-prefills from zero - so "nothing at all" was the most common thing on screen. Once tokens did start, the tool calls and reasoning were squashed into a single collapsed "Explored" line.

A status line cannot change the answer, and that is provable. The payload sent to the model is rebuilt from scratch by an explicit allowlist:

utils/payload.py  convert_messages_openai_to_ollama()
  new_message = {'role': message['role']}
  # then only: thinking, content, tool_calls, tool_call_id, images

Status events persist to a SIBLING key (statusHistory, models/chats.py:1262-64), which appears nowhere in payload construction - the backend's only other references are the DB column mapping. The token sequence is byte-identical with the filter on or off, so there is no prompt change and no prefix-cache perturbation.

The filter: owui-functions/live_progress_status.py

Emits an Open WebUI status event on phase changes only - not per chunk:

Processing prompt                        <- inlet
Thinking · prompt processed in 3.4 s     <- first reasoning delta
Thought: <opening of that reasoning>     <- when a reasoning block ends
Calling web_search                       <- a tool call is being written
Writing response                         <- first content delta
Done · 3 tool calls                      <- outlet

StatusHistory.svelte renders only history.at(-1) as the headline and collapses the rest, so accumulation is not clutter - the headline always shows the current phase.

Two sharp edges in the stream hook, both load-bearing. First, the return value REPLACES the chunk and a falsy return DROPS it:

data = wrap_item(JSONCodec.dumps(event)) if event else None

so stream() returns event on every path; returning None would silently delete the model's output. Second, exceptions from a stream filter are re-raised by utils/filter.py, which would break the response mid-flight - so the body is wrapped and always falls through to return event.

The module object is cached on request.app.state and therefore shared process-wide across all six users, so per-request state is keyed by message id and pruned at 64 entries; a request whose outlet never runs (disconnect, cancelled generation) cannot leak.

Offline tests cover the phase sequence, multi-round tool loops, malformed chunks ({}, {"choices":[]}, {"choices":[{"delta":None}]}, non-dict input), an emitter that raises, and the state bound. Every one passes the chunk through unchanged.

The dropdown: tried, reverted, replaced by the indicator

First attempt was ui.default_interface_settings = {"expandDetails": true}, which works (ConsecutiveDetailsGroup.svelte:44 reads it, and routers/users.py:482-488 merges admin defaults into what every client loads). But one setting drives three levels - the outer group, the reasoning Collapsible, and ToolCallDisplay - so it also expanded every tool's INPUT/OUTPUT, which is not wanted. There is no second setting; splitting them needs a frontend change, and open-webui carries wud.watch=true on the moving :main tag, so any patch to the compiled JS would be silently wiped by the next auto-update. Reverted to {} (its exact prior value; verified no user had a personal override either way).

Reverting the admin default was not enough, and this is the trap. Setting ui.default_interface_settings causes the backend to MERGE it into what getUserSettings returns (routers/users.py:488), the client loads that merged object into $settings, and then any ordinary saveSettings call - picking a model, pinning something - writes the whole merged object back as the user's PERSONAL settings. So expandDetails: true got baked into the admin account. strip_default_interface_settings only removes a personal value that still equals the current default, so once the default went back to {} the personal true was stranded and kept winning. Symptom: reasoning blocks still auto-expanded for that one account after the default was reverted. Fixed by deleting the single key from that user's settings.ui.

Worth remembering: an admin "default" in Open WebUI is not a passive fallback - touching it can permanently write that value into every active user's personal settings. Check select name, settings from user after changing one, not just the config key.

Two incidental hazards found while cleaning up: user.settings can be the literal JSON null (two of six accounts), so json.loads(s).get(...) raises AttributeError on a truthy string - guard with isinstance(st, dict). And the first cleanup script crashed on that before reaching commit(), so it silently changed nothing; the re-run verified per-user effective values afterwards rather than trusting the update.

The status timeline replaces it. Rather than patch the frontend, the filter now posts the opening of each reasoning block as its own status when that block ends, so the collapsed view still tells the whole story:

Done · 10 tool calls                                    <- headline, click to expand
  Processing prompt
  Thinking · prompt processed in 5.2 s
  Thought: The user is asking whether TCP Fast Open is worth enabling on a busy web…
  Calling lookup_rfc
  Calling web_search
  Thinking
  Thought: The web searches didn't return great results (mostly MDN glossary pages)…
  Calling lookup_rfc
  Thinking
  Thought: Good, I have the RFC's key points. The RFC itself notes: - TFO is **exper…
  ...
  Writing response
  Done · 10 tool calls

That is the chain of thought and the tool sequence at a high level, with no tool payloads anywhere - and it survives OWUI updates, because it is a filter rather than a patched asset. thought_excerpt_chars (default 160, 0 disables) bounds each line; the renderer clamps a status to one line, so more would be wasted. The reasoning buffer is capped at ~4 KB per message and is not filled at all when the excerpt is disabled.

A bug the tests caught. The reasoning buffer was added to the state dict in two places by a replace-all that only matched one of them - the indentation differed - so stream() raised KeyError: 'reasoning' on the first reasoning delta. The filter's own blanket except swallowed it, and the only visible symptom was that every status after "Processing prompt" silently stopped appearing. Fixed by collapsing both literals into one _new_state() factory. Worth remembering: the broad exception handler that makes this filter safe also hides its bugs, so the offline test is not optional.

---

The 40-prompt graded run, and what it changed

prompt-testing/ was built as forty engineered prompts with a rubric each, run through the chat UI and graded against independently verified ground truth. Mean 7.53/10: 26 pass, 8 partial, 6 fail. Correctness was 87% and calibration 68% — the model is mostly right and frequently unable to say how it knows. Ninety-nine points were lost, and they sort into six causes:

pts	cause
30.5	treated a name, date or office-holder as settled knowledge and skipped tools, or searched with its own guessed answer in the query
28.5	right data, no URL, or the source mislabelled
20.5	a tool output gap induced the error
10	a decoding loop that produced no answer at all
6	code or logic slip on a well-described tool
3.5	arithmetic in the head instead of the sandbox

Changes 45-52 are that list worked through. Every one of them follows the finding this log has now confirmed four times: the model treats tool output as fact and system-prompt text as advice. Where a rule had to hold, it went in tool output; the prompt changes are paired with a tool change carrying the same rule, never standing alone.

---

Change 45 — a repetition loop that produced no answer, and no penalty to stop it

Symptom. One prompt (garage circuit wiring) spent 140 s and 8,201 completion tokens repeating two sentences about NEC 210.52 roughly 180 times inside its reasoning block, then ended the turn with no assistant text at all. The status indicator showed a normal "Done".

Cause. Nothing in the stack discouraged verbatim repetition. repeat_penalty is 1 (off) in the Modelfile and the Open WebUI model row sent presence_penalty: 0 explicitly, which overrides it. Qwen's own guidance for thinking mode recommends a presence penalty of 0-2 specifically "to reduce endless repetitions", and warns that higher values cause language mixing.

Change. On the model row, not the Modelfile — the row is what wins: presence_penalty 0 → 1.0, repeat_last_n unset → 256 (Ollama's default 64 does not span one cycle of this loop). repeat_penalty left off deliberately: above 1 it penalises exact-token repeats, which is what citations, part numbers and URLs are made of. reasoning_effort: "xhigh" deleted — Ollama has been rejecting it as an invalid option on every call since it was set.

Result. Verified present in the model row and confirmed in Open WebUI's Ollama parameter allowlist (utils/payload.py), so both are forwarded. The loop prompt has not been re-run; tripwires if it is: any CJK leaking into English answers, or mean thinking tokens up more than 15%, means drop to 0.5.

---

Change 46 — fetch_page now says what it handed you, and admits when it handed you nothing

Symptom. Four distinct failures, all of them the tool being unclear about its own output. (1) The model fetched a real paper, transcribed every number from it correctly, and credited it to authors and a journal that appear in no tool result. (2) It wrote "I pulled this directly from the RM0008 PDF" when all nine of its PDF fetches had failed and the values came from a lecture-note snippet. (3) A 1,293-character navigation shell from a client-rendered catalogue read as a successful fetch, and it answered from memory instead. (4) A 13 MB reference manual was silently truncated at the 12 MB ceiling and then reported to the model as "may be a scanned image, a video, or a page that renders entirely in JavaScript" — false in every particular.

Cause. requirements.txt says trafilatura was chosen partly because "it hands back metadata … which is what the citation lines are built from", and nothing ever asked it for any. Extraction began at the abstract, so the model had the numbers and no citation, which is exactly the state in which a plausible one gets invented. get_bytes breaking past the size limit returned partial bytes with no flag. And a short extraction from a large document had no meaning attached to it at all.

Change. A citation header before the content — title, authors, journal, publisher, DOI and a CITE AS: line — built from Highwire/PRISM/Dublin Core meta tags, with trafilatura as the fallback for ordinary pages and a DOI/arXiv regex for PDFs. The journal is read only from tags that name the container: og:site_name is deliberately not consulted, because on an npj Flexible Electronics article it says "Nature", which is the precise mislabel that cost the graded run. When no bibliographic data exists the header says so and tells the model to cite by title and URL rather than filling it in. Access notes distinguish a PMC full text from a PubMed catalogue record, an arXiv abstract page from the paper, and a publisher landing page from either. _unreadable() separates a scanned PDF, an empty extraction and a client-rendered stub, and the stub message quotes what it did extract and suggests three concrete next moves. Oversized downloads now raise instead of truncating, with a truthful message and a 40 MB allowance for PDFs specifically. The CITE AS builder is shared with search_papers via toolbase.cite_line.

Result. Verified live: a PLOS article returns Journal: PLOS ONE · Publisher: Public Library of Science · DOI: 10.1371/... with a correct CITE AS: Ravenscroft et al.; example.com returns the no-metadata line. Two real environmental findings fell out of the stub detector on its first run — nature.com and PMC both now serve this server a bot wall, and both previously read as successful fetches of a few hundred characters. One bug the probe caught and fixed: citation meta tags write "Surname, Given" while page bylines write "Given Surname", so arXiv's "Vaswani, Ashish" was being cited as "Ashish et al.".

---

Change 47 — web_search says how well it matched, instead of always saying "read one"

Symptom. The trailer "Next: call fetch_page on the most promising URL to read it properly before answering" was appended to every non-empty result list unconditionally. Twelve rubrics recorded it either obeyed onto a useless page or ignored with no consequence — the two ways a fixed instruction fails. A search for a fabricated paper title, with four of five engines down, returned six unrelated results and read exactly like a search that had found the answer.

Cause. No relevance signal reached the model at all. Separately, SearXNG's scholarly engines return journal, authors, doi and pdf_url on every hit and all four were being dropped, and category: "science" was documented nowhere while quietly routing consumer-health and everyday-physics questions to arXiv.

Change. A term-overlap floor produces one of three trailers: read-the-page above 50%, [WEAK MATCH] above 25%, [LOW RELEVANCE: … treat this as NO RESULTS for what you actually asked] below it. A quoted phrase that appears in no result forces the low tier regardless of word overlap, because the words of an invented title are ordinary words and scatter. Results now carry a CITE AS: line and a PDF link when the engine supplied them, with a note that a journal not on one of those lines is not known. The read-the-page trailer now also requires a snippet-sourced value to be labelled as such. The query field description says to search for the thing and not for your answer; the category description says what science really is. A domain-shaped hint points dosing questions at DailyMed, medicines.org.uk, MedlinePlus and NHS, and flags science on a practical medicine question. Results already read or already failed in this conversation are annotated, applied after the cache so the annotation follows the conversation rather than being baked into a shared entry. In searxng/settings.yml, openalex and crossref are enabled — they ship disabled upstream and are the two engines carrying real bibliographic metadata.

Result. All verified live. A fabricated quoted title now returns [LOW RELEVANCE]; a good query still returns the read-the-page trailer; a dosing query returns the DailyMed hint; a science query returns CITE AS: lines. openalex, crossref and pubmed confirmed live in the science category.

---

Change 48 — wikipedia_lookup took the top full-text hit and called it the article

Symptom. Asked for the famous son of Mary Lee Pfeiffer, the tool returned the Tom Clancy novel Red Winter. The model — which had already guessed an answer — read the miss as confirmation, searched "Mary Lee Pfeiffer mother of Bob Dylan", and delivered Bob Dylan with an invented supporting biography. 0/10. Separately, the lead-only default meant a call for the Eiffel Tower's inauguration date returned a summary that does not contain it, and the wasted call sent the model to a general search.

Cause. Resolution went straight to list=search and took hits[0], with no exact-title attempt, no disambiguation handling, and no way to tell "this article is about your topic" from "this article contains your words". There was also no way to reach into the body at all.

Change. Exact title first, following redirects, with disambiguation pages surfaced as a list of options rather than silently resolving to one. Full-text search only as a fallback, and when the best hit's title overlaps the topic by less than half, a NO ARTICLE result that names the near misses, says they are about something else, and tells the model to search the name on its own with no guessed relation attached. search= and offset for reaching into the body, and the intro-only header now says the lead rarely carries the specific fact.

Result. Verified live, and this is the clearest end-to-end result of the change set. Re-run through the chat UI, the same prompt that scored 0/10 now reasons "this is a settled fact question — but I should verify rather than recall", gets NO ARTICLE from Wikipedia, searches Mary Lee Pfeiffer mother with no answer attached, and returns Tom Cruise, correctly.

---

Change 49 — reading a long document, and the budget that punished it

Symptom. Asked for the scorers in a football final, the model read 4,000 characters of a 118,000-character article, found the headline result in the lead, stopped — and supplied the scorers, the minutes and the shoot-out misses from memory, wrongly. On another prompt the budget printed STOP FETCHING on page nine of a single document. A search for an RFC section number reliably landed in the table of contents, burning two of four calls on dot leaders. read_man_page turned "ip route" into iproute, 404ed, and reported "No manual page found … do not describe its options from memory" for a page that exists.

Cause. The continuation hint ended "Only do so if this section did not answer the question", which reads as permission to stop; a lead section answers the headline question and never carries the itemised detail. The budget counted calls, so paging one document spent it. _find_in_document took the first literal match, and a contents entry is the one place a section number is guaranteed to appear and guaranteed to be useless. And the man-page name filter deleted spaces rather than hyphenating them.

Change. The hint now says to keep reading, that what you have not read you do not know, and to jump with search= instead of paging. _find_in_document moved to a shared textwin.py, gained table-of-contents skipping — preferring a body match, falling back to the first match if every one looks like a contents line, and reporting how many it stepped over — and is now used by fetch_page and wikipedia_lookup as well as the two tools that had it. The budget counts distinct URLs, so continuing inside an open document and cache hits cost nothing; it remembers which URLs already failed and declines to re-fetch them; and the note moved into the header, where the response cap cannot trim it away. A first, gentler note lands at five pages. read_man_page hyphenates multi-word names and falls back to the first word with the remainder as a search term.

Result. All verified live: the RFC jump reports skipping a contents entry, ip route resolves to ip-route, fetch_page with search= lands in the body of RFC 9293, the same URL fetched three times with different offsets counts once, and a URL that failed is declined on the second attempt without a network call.

---

Change 50 — the tool-output gaps the graded run found

Six smaller defects, each of which produced a wrong answer, and one of which was the tool lying to the model.

• pcb_calc shipped a false claim. Its trace-width output told every caller "IPC-2152 is the more modern and usually less conservative standard if you need to squeeze". For a bare external trace that is backwards — IPC-2152 gives widths of the same order or larger, because the IPC-2221 external curve assumed more cooling than a real board provides. The model took the tool at its word, printed "~54 mil (or ~40 mil on IPC-2152)", and a reader following the parenthetical would undersize a 3 A trace by about half. This is the only case in the whole graded run where the model amplified a harness error rather than inventing one. Corrected, and an inverse width-for-target-impedance mode added, solved by bisection over the same forward functions so the two can never disagree.
• subnet_calc split an IPv6 /64 into /67s with no caveat while the same output carried a "Scope: documentation only (RFC 3849)" line — and the model's answer tracked it one for one, warning about exactly what the tool flagged and recommending SLAAC on /67s. The tool's advisory surface teaches the model what deserves a warning. It now notes the 64-bit interface-identifier boundary (RFC 4291 §2.5.1, RFC 7421) and that the usual answer to "I only have one /64" is to ask for a /56.
• http_headers returned an assessment where the data was asked for — Present: HSTS, CSP, …, never the value, so max-age=0 and a correct two-year preloaded policy rendered identically. It now prints each security header's value, parses HSTS into its directives, and notes when a header differs between redirect hops, because an apex and its www are configured separately and demonstrably differ.
• lookup_cve dropped lessThan, so a record meaning "everything below 116.0.5845.187 is affected" rendered as "116.0.5845.187 (affected)" — the opposite for that version. Ranges now render as "from X before Y", affected falls back to ADP containers, the ? ? placeholders are gone, and CISA KEV status plus NVD CPE ranges are reported from data the file was already fetching and discarding.
• find_datasheet did not distinguish an exact part from a near-miss, and its hint ranked candidates by domain authority alone. The model followed that hint faithfully to a Vishay IRLZ44 datasheet while the correct Infineon IRLZ44N sat at candidate two, and every figure it quoted was wrong. Candidates are now labelled EXACT, VARIANT or DIFFERENT PART, and the hint says an exact part number on an aggregator beats a variant on the manufacturer's site.
• httpc.explain blamed the wrong thing. Squid answers a CONNECT to a name that does not resolve with 503 and a policy refusal with 403, and every ProxyError was being reported as "the address is on a private network". The model invented three plausible API hostnames, was told each time that the host was private, never suspected the hostname, and reported to the user that public services were blocked here. The h10 rubric records this as the proxy refusing ordinary public hosts; that grader note is wrong and should be amended. The three hosts were NXDOMAIN. DNS failure, policy denial and unreachable-host are now three different messages.
• matlab_run was never considered as a calculator, and its Octave dialect gaps cost three of four calls on a constrained-writing task. It now advertises itself as a scratchpad, takes purpose: "scratch" (which skips the MATLAB-validity verdict and answers a missing function with the Octave equivalent rather than "do NOT rewrite it"), explains the 'tokens' option name, and lints two new semantic traps — \b in a regexp being a backspace rather than a word boundary, which fails silently with a wrong number, and idivide rounding negatives differently from MATLAB.

Result. All verified live through the gateway.

---

Change 51 — the system prompt, where it was IT-shaped and the questions were not

Symptom. Thirty and a half points — the largest single bucket — went to the model deciding a question needed no tool. The pattern is consistent: a name, a date, an office-holder, a poem's wording, a count of human senses. In one case its own reasoning read "settled general knowledge" — the exact phrase wikipedia_lookup's docstring used to describe what the tool is for, read as an exemption from using it.

Cause. Every trigger in the prompt's lookup list was IT- or EE-shaped: versions, flags, part numbers, CVEs, "any number the user might act on". Nothing named people, dates or quotations, and wikipedia_lookup was a bare name in the tool list with no purpose text. "Doing the arithmetic" named no tool at all, and matlab_run was described only as executing code — so it went unconsidered for counting letters or compounding interest.

Change. A sixth trigger covering names, dates, office-holders, relations, what a text actually says and how many of something there are, stating that "everyone knows this" is a trigger rather than an exemption. A rule against confirmation-seeking queries, paired with the same rule in web_search's own schema. "Doing the arithmetic" now names the calculators and matlab_run with purpose: "scratch" explicitly. The first Answering bullet is now a hard sources rule: any tool-assisted answer ends with its URLs, a snippet-sourced value is labelled, and "from the manual" is forbidden unless a fetch of it succeeded. The honest category guide replaced the one that mis-described science. "Ask once" moved out of the MATLAB section, where it was scoped to code, into the general honesty section. Paid for with four cuts: the numbered iterate list, the casual-question paragraph (the nudge carries it), the six-fetch bound (the budget carries it now) and the plot-PNG paragraph (duplicated verbatim in the tool docstring). Net +200 words on 2,175.

Result. Measured on two prompts. The Mary Lee Pfeiffer prompt went from 0/10 to correct (Change 48). The letter-counting prompt, which previously made zero tool calls and counted in its head, now calls matlab_run with purpose: "scratch", hits a syntax error, reads it, fixes it, re-runs, and quotes the executed result. One rule is not landing, and that is accepted: neither answer carried a "Sources:" line. It is the prompt-only half of the attribution fix, behaving exactly as this log predicts prompt-only rules behave — and there is no tool-output lever for the shape of a final answer, only for what the tool hands over. The URLs are now in the tool output, so a citation can be lifted whenever the model reaches for one. Reviewed and kept as a deliberate compromise: the model is carrying a great deal of instruction already, and this rule costs nothing on the turns it is ignored. Not an open defect. Revisit only if citation gets worse than the baseline, where 6 of 40 tool-assisted answers carried no URL.

---

Change 52 — the status indicator's numbers were not the model's numbers

Symptom. Thirty-five of forty rubrics independently flagged the same thing: the live indicator's "prompt processed in N s" disagreed with the same turn's prefill= figure, often by two or three times, in both directions. Two smaller ones: a turn with three tool calls reported "Done · 2 tool calls", and the 140-second turn that produced no answer at all reported a normal "Done".

Cause. The figure was wall-clock from inlet to the first streamed token — which includes Open WebUI middleware, queueing and model load — and was labelled as prompt processing. The tool counter lived inside the display valve and was zeroed when inlet ran a second time for the same message, which is what an interactive ask_user call causes. And outlet never checked whether any content had been produced.

Change. Real prompt_eval_duration and prompt_eval_count read from the stream's usage chunk, with the message-usage fallback; the early label relabelled to "first token after N s", which is what it actually measures; the final line now carries prefill seconds, prompt tokens and generated tokens. State survives a second inlet. Counting moved outside the display valve. A blank final message produces "Done · no answer produced — the model stopped inside its reasoning" plus a visible note in the message itself. The thought excerpt now shows head and tail, after one surfaced as "…I should verify with actual execution since I have tools. Actually, I…", cutting off precisely the clause that reverses the meaning.

Result. Verified live. A real turn now ends Done · 2 tool calls · prefill 18.4 s for 14,897 tokens · 233 generated, against a first-token time of 90.5 s — a 5× gap that the old label was reporting as prompt processing time.

---

Change 53 — the tool definitions were the prefill cost, and most of it was not information

Symptom. After Change 50 a short question cost about 14,900 prompt tokens against a baseline cluster of 10,800–13,700, and the increase was attributed to longer tool-schema descriptions. Measured properly, the 21 tool definitions came to 9,812 tokens as Ollama renders them (cl100k count of the JSON, after Ollama's own key filtering) — more than the system prompt, and paid on every turn of every conversation.

Cause. Three layers, each adding text the model cannot use. The docstrings and field descriptions had grown by accretion: every failure in this log had added a sentence, and the sentences explained why — "someone may patch, or not patch, based on your answer" — when the system prompt already carries the why and the schema only needs the what and the when. Several rules were stated three times (once in web_search, once in fetch_page, once in the prompt). Underneath that, pydantic emits "title": "Max Chars" on every property and wraps every optional field as anyOf: [{type: T}, {type: null}]; Open WebUI copies both into the tool definition. And FastAPI put the Authorization and X-OpenWebUI-* header dependencies into the spec as parameters, which Open WebUI's converter does not distinguish from body fields — so every tool carried one to five blank-described parameters named authorization, x-openwebui-user-email and so on. Ollama 0.32.15's ToolProperty keeps only type, anyOf, items, enum, properties, required and description (verified against api/types.go), so title, default, minimum and maximum were already invisible to the model — which also means a default the model should know about has to be in the description text, not the schema.

Change. Every docstring and field description rewritten to state what the tool does, when to reach for it, and the one rule it must not break, with the justifications cut. Every behavioural rule that entered this log as a fix survives — "search for the thing, not for your answer", the honest science category guide, purpose='scratch', the multi-word man-page names, the AC-drop-not-DC warning, the return_figure self-check caveat — in fewer words. The header dependencies are include_in_schema=False. A post-processor on the generated OpenAPI document strips title, drops default: null, and collapses a nullable anyOf to its real type; the server still accepts null. Descriptions now carry the defaults that matter (max_chars 6000, port 443, purpose deliverable) because the schema's default never reaches the model.

Result. Tool definitions as Ollama renders them: 9,812 → 6,943 tokens, a 29% cut, the largest single savings on matlab_run (1,105 → 658), wiring_calc (916 → 724) and web_search (797 → 519). All 107 smoke cases pass unchanged. A live before/after on one prompt through the chat UI was attempted and not completed — the browser extension froze mid-run, the same failure as during Change 52 — so the token figure is a tokenizer estimate, not an Ollama prompt_eval_count. Open WebUI caches the spec at startup, so it was restarted to load the new definitions.

---

Change 54 — three findings from the graded v2 round

The v2 round scored 8.45/10 against v1's 7.53 (33 pass / 5 partial / 2 fail, from 26 / 8 / 6). Calibration & honesty went 68% → 84%, clearing the ≥80% target with correctness holding at 94%. Three of the four v2 targets were met; this change set is the fourth, plus the two clearest remaining tool gaps.

The round also produced a result that qualifies this log's governing principle, and all three changes below are shaped by it. Tool output the model acts on reports a fact: lookup_cve's KEV line, wikipedia_lookup's NO ARTICLE, the client-rendered stub's byte counts, subnet_calc's SLAAC advisory, http_headers' apex-versus-www note, matlab_run's "nothing ran at all". Tool output the model declines issues an instruction: find_datasheet's [VARIANT … the suffix is the part] (read, quoted in its own reasoning, overridden), the DailyMed dosing hint (fired on both dosing prompts, obeyed on neither), [LOW RELEVANCE … do not answer from memory instead] (fired on e16 and m10, overridden on both). So: state the fact, do not issue the instruction.

---

54.1 — the citation rule had nowhere to land, so it never did

Symptom. The one rule Change 51 knowingly shipped as prompt-only. Measured across both rounds on the same definition — prompts where a tool result contained a URL and the answer contained none — v1 cited on 2 of 23 and v2 on 1 of 25. Thirteen v2 prompts lost a rubric point for it. Change 51 accepted the compromise on condition that citation must not get worse than baseline. It did.

Cause. The model will not assemble a citation out of material it has to reformat. A # Source: header at the top of a fetched page reads as document identity, and a Homepage: row among five other rows reads as data about the package. The single answer in eighty that did cite lifted search_papers' ready-made CITE AS: lines verbatim, DOIs and all.

Change. toolbase.sources_line(): a one-line trailer, at the end of the response, written in the shape the answer should carry — \n---\nSources: <cite> · <url>. On fetch_page (with the bibliographic cite line when the page has one), wikipedia_lookup, lookup_rfc, read_man_page and lookup_package. web_search is untouched: its per-result CITE AS: lines already work, and a search has no single source. No accompanying instruction to use it, deliberately. cap() now detaches and re-attaches the block so it survives truncation — which is the whole reason the old header sat at the top — and each caller reserves its length out of the page budget.

Result. Verified live: Sources: https://en.wikipedia.org/wiki/Canberra on a Wikipedia call, Sources: Ravenscroft et al., "Measuring scientific impact beyond academia…", PLOS ONE (2017). doi:10.1371/journal.pone.0173152 · https://journals.plos.org/… on a paper. Whether the model lifts it is the open question and the main thing v3 measures. One real bug found and fixed while testing: lookup_package returned early on a cache hit, skipping the trailer.

---

54.2 — "which of my terms is missing" was never stated, only implied

Symptom. m10 invented a randomised controlled trial — "Korhonen et al., Circulation, 2018" — made it the load-bearing evidence for a causal conclusion, and told the user the only unverified part was one effect size. Automatic fail, 4 → 0. Its searches carried its own guesses ("… Finland Korhonen Circulation", "… 750 adults 12 months"), and results that were topically plausible read as partial confirmation. The [LOW RELEVANCE] trailer fired and said "do not answer from memory instead"; the next thinking block began "I'm confident from training knowledge that…". Separately, e12 (9 → 0) quoted a blockquote from RFC 2324 §2.3.2 after reading only the RFC index listing, whose trailer said "Next: call lookup_rfc again with the number to read the full text" — an instruction about what to do next, not a statement about what it was holding.

Cause. Both trailers told the model what to do. Neither told it what was true. The relevance floor answers "how well did this match overall"; nobody answered "which of the things you asked for is not here", which is the question a fabricated name fails.

Change. Two facts, no instructions. web_search reports distinctive unquoted query terms that appear in no result, as a count: [Appears in none of these results: "Korhonen" - 0 of 6. A name, a journal, a place or an identifier present in no result was not found here; that is a different outcome from ranking low.] Quoted spans are excluded — a missing quoted phrase is already the relevance floor's job, and words capitalised only because they sit in a quoted heading ("Address offset") say nothing when absent. lookup_rfc's index listing now leads with what it is: "These are index entries: title, publication status and abstract. The body of an RFC … is not in this listing, so nothing can be quoted from it."

Result. Verified live. The m10 query returns the Korhonen note; the h01 fabricated-paper query returns Okafor, Lindqvist, Nature and Electronics at 0 of 3. Checked for false positives against six real queries: a good RFC query, the h05 quoted-heading query, the fully quoted e07 name and a plain English question all stay silent.

---

54.3 — search= decided the fate of two sources, in opposite directions

Symptom. e06 was the round's only correctness regression (7 → 6). Asked for "Constitution" inside the Canberra article, the jump matched "1 Constitution Avenue, Civic" — a street — and returned a window on self-government, bushfires and climate. The model, having looked, stopped hedging and published §124 for the seat-of-government clause; it is §125, and §124 is "Formation of new States". Its own reasoning had written the right rule first — "I'll mention it was required by the Constitution without over-specifying section numbers I haven't verified" — and then overrode it. The mirror image at e15: search="list of senses" in the Sense article returned only "that term does not occur", and the model abandoned 54,000 characters of exactly the right article, answered from memory, and kept a claim that humans have electroreception.

Cause. A character offset tells you where a string is, not whether you are reading the right subject; and a bare miss is indistinguishable from the document being useless. In both cases the model had chosen the correct source and the tool's answer sent it away.

Change. In textwin: a hit names the enclosing heading — [Jumped to 'Constitution' at character 19,328 of 62,467; this is inside the section 'Self-government'] — recognising Wikipedia == headings ==, markdown, numbered specification sections and the flush-left capitals of an RFC or man page. A miss returns the miss notice plus the opening of the document, since it is already fetched and already in memory. New search_window() wraps both; fetch_page, wikipedia_lookup, lookup_rfc and read_man_page all use it, replacing four separately-worded dead ends.

Result. Verified live on both motivating cases: the Canberra jump now reports section 'Self-government', and the Sense miss returns the article's lead, which is about senses. Heading detection checked against all four document shapes.

---

54.4 — two prompts were measuring something other than what their rubric said

Not a harness change: the instrument. Grading v2 surfaced two prompts whose rubric described a deployment this is not, and both had distorted v1 as well.

• m04 tells the operator to answer the model's clarifying question with one exact sentence — "Z-score each column, ignore NaNs, and return the means and standard deviations too". The harness answers with the native ask_user widget, whose fixed multiple-choice options carry method and dimension and nothing else, so two of the four clauses never reached the model in either round — and correctness lines 2 and 3 then scored the delivered function on exactly those two. The protocol now says to use the widget's free-text field, and declares a run void for those lines if the sentence cannot be delivered; the scoring charges a failure to ask once, on line 1, and grades the deliverable against what was conveyed.
• e19 asserts that the built-in clock is disabled and scores calibration on "explains why it cannot know". get_current_timestamp is exposed and the model used it in both rounds, so that line could never apply and both graders had to map it onto something else. The rubric now tests what the prompt has actually been testing: tool over recall, the right timezone, and saying where the date came from — which makes it the cheapest instance in the set of the attribution behaviour 54.1 targets.

Both masters are corrected and both completed rounds re-scored against them, because a rubric change that is not applied backwards silently breaks the round-over-round columns: m04 v2 6 → 7, e19 v1 9 → 10 and v2 9 → 8. v1's mean 7.53 → 7.55; v2's mean unchanged at 8.45, the two corrections cancelling. Each grade file carries a banner with the reasoning, and the original grade text is left intact below it.

---

Measured: 123/123 smoke cases pass (107 before, 16 added for these three). The three motivating transcripts re-probed through the gateway. Selector false-positive checks as above. All 80 completed grade files re-parsed and the aggregates recomputed from them rather than from the old table.

Not measured: whether the model uses any of it. That is v3.

---

Change 55 — the v3 round's harness defects, and the citation line that gets written down

The v3 round scored 8.85/10 against v2's 8.45 (36 pass / 2 partial / 2 fail; no score of 0 for the first time; calibration 87%, correctness 94%). Of the 46 points lost, roughly 12 were attribution (a correct answer, read from the right page, with no URL), 7 were tool defects the graded transcripts pinned down exactly, 22 were the model's own recall and embellishment, and 3 were noise. The first two buckets are the harness's to fix, and this change takes all of them. The third is the model, and the ceiling it implies is about 9.3.

The governing principle holds: every rule below is stated as a fact in tool output, not as an instruction. The one exception is 55.1, which is not a rule at all - it is the harness doing the thing the model reliably does not.

---

55.1 — the citation line is appended, not requested

Symptom. Change 54.1 put a ready-to-copy Sources: trailer on every tool that returns a single document. In v3 it was lifted into the answer on about half of the opportunities; eleven answers that had read the right page (e05, e06, e07, e10, e13, e15, e16, e20, h07, m05, m08) ended with no URL, and each lost a point for it. The cited-URL target (≥ 20 of 23) was the one v3 target not met.

Cause. The model has the URL in front of it and does not write it down. Three rounds and two carriers (system prompt, tool trailer) have moved this from 1 of 25 to roughly half; the remaining half is not going to come from a fourth wording.

Change. A new outlet filter, owui-functions/answer_sources.py (deployed as answer_sources, active and global). After the turn it collects the Sources: trailers from the turn's tool results - only tools that read a document emit one, and only on success - and, if the answer names none of those URLs and carries no sources line of its own, appends ---\nSources: url · url to the message (content and the rendered output item both, as inline_tool_images.py does). Search results are never added: a URL in a web_search listing was not opened, and attributing an answer to it would be the mislabelling the rubrics penalise. An answer that already cites one of the read pages, or writes its own "Sources:" section, is left alone.

Result. e05 re-run: the model's answer was unchanged and ended with Sources: https://pypi.org/project/requests/ - the page it had fetched. The filter fails closed on any surprise in the message shape.

---

55.2 — twelve tool defects from the graded transcripts

Each of these was identified in a v3 grade file with the transcript line that caused it. All are domain-shaped; none is keyed to a test prompt.

fetch_page served a glyph-encoded PDF as text (m06). The Infineon datasheet's fonts carry no character map, so extraction returns /G73/G68/G32… for every cell of every table; the tool reported 28,716 characters and no warning, and the model reverse-engineered the glyph numbering, got the one value the question turned on wrong, and answered the rest from memory. Now: when more than 35% of a PDF's text is glyph codes (/G123, (cid:123)) the fetch FAILs with the reason and the alternatives (a distributor's copy, the vendor's HTML page); between 3% and 35% the header carries a warning that the tables may be unreadable. The Infineon file measures 78%.

Open-access papers were unreachable by every route (m10, e13, h07). PMC, PubMed and Europe PMC's article pages all sit behind a browser check or cookie wall for a plain client; m10 tried five routes to one open-access paper and got a stub from each, then found by itself that Europe PMC's REST API serves the same full text as JATS XML with no check. fetch_page now tries that API first for any URL naming a PMC article, and for a PubMed or Europe PMC record whose paper is in PMC (resolved through the API's search endpoint). The XML is rendered to markdown with the front matter as citation metadata (title, authors, journal, year, DOI - so the CITE AS line is built from the article, not from a landing page), and the header says Retrieved via: …/rest/PMC…/fullTextXML. When the API has nothing, the stub message says so, so the model does not keep trying PMC routes.

Bad parameters came back as JSON 422 bodies (m06, m03-v2). [MORE AVAILABLE: 10,716 characters] was answered with max_chars: 10716, and the schema bounced the whole call. max_chars is now clamped to its bounds instead; any other validation failure is returned as prose with a 200 (CANNOT DO THAT: the call was rejected before it ran - offset: Input should be greater than or equal to 0 (you sent -5). Fix that parameter and call again…), matching every other failure in the suite; and the continuation hint names the cap.

Two 504s arrived as nginx HTML (h05). Open WebUI sets no tool-server timeout of its own here, so the gateway's 90 s proxy_read_timeout was the only clock, and a 995-page reference manual took longer than that to extract. The gateway now waits 240 s, PDF extraction is bounded at 120 s inside the app (returning prose on timeout), and the slow pdfminer fallback is skipped above 8 MB - so the app, not nginx, is what answers.

search= landed on page-numberless contents entries (e12) and reported false sections (h05). RFC 9110's .txt rendering - every RFC in the v3 text format since 2019 - lists its contents as 15.5.19. 418 (Unused) with no leaders and no page number, so Change 49's skip did not fire and two jumps returned the listing. A numbered line in the opening quarter of a document whose text recurs later as a line of its own is now a contents entry. And the flush-caps heading shape matched a running page header (RM0008) and a bit-field label (XTPRE) on the manual's PDF text: a line that repeats four or more times is now a header, not a heading; a single capitalised word is used only when no structured heading sits above it in the window (which is the normal case in a man page and never in a numbered manual); and "Table of Contents" is itself a recognised heading, so a jump into it is told so.

lookup_ip_or_asn refused documentation addresses without naming them (m02), and returned no origin AS (h10). "Private, loopback or reserved" was true and useless: the model knew from memory that 203.0.113.10 was TEST-NET and talked itself out of saying so in three rounds. The refusal now names the block and RFC and states the fact in prose (never allocated, never announced, treat it as a placeholder). And registration is not routing: RDAP says who a block is allocated to, not who announces it, and h10 spent four extra fetches per round on the origin AS. Every IP lookup now carries a ## Routing (BGP) section from RIPEstat's routing view - announced prefix, origin AS with holder, multi-origin flagged - and an AS lookup carries the holder and whether it announces anything.

subnet_calc's documentation scope was a field (h04). Three rounds reproduced the SLAAC advisory paragraph almost clause for clause and skipped Scope: documentation only in the same result: an advisory paragraph crowds out an advisory field. The same fact is now also a sentence under the Scope line.

read_man_page resolved a bare name to section 1 (m05). cron's PATH behaviour is in crontab(5) and cron(8), not crontab(1); a search of "crontab" landed in section 1 and reported the term absent. On a search= miss with no section given, the sibling sections are now fetched concurrently: if one contains the term it is shown instead, with a note; if not, the miss note says which sections the name also exists in. (On Arch's cronie pages neither crontab(1) nor crontab(5) mentions PATH - cron(8) does - so this prompt now gets "the name also exists in section 5" rather than a hit; a different-name page is not something a search of one name can reach.)

pcb_calc mis-parsed unit suffixes and printed a negative impedance (h03, twice). The engineering-value parser reads a trailing "mm" as the prefix milli, so "1.6mm" became 0.0016 mm and the microstrip formula printed Z₀ ≈ −46.5 Ω as a result. Board dimensions now have their own parser (mm by default; mm, mil, thou, µm, cm, in accepted; a bare "m" is refused as not a board dimension), and any impedance at or below zero is a CANNOT DO THAT naming the w/h ratio and the likely unit mix-up.

matlab_run's parse-error hint did not recognise \' (h08), or a function defined after its use. A backslash-escaped apostrophe inside a single-quoted string is the commonest way to break an fprintf in either dialect; the raw message said "syntax error", the model blamed Octave, and a call was lost. The hint now names the fix (''). And a function defined in the script below the line that calls it is "undefined" in Octave and required at the end of the file in MATLAB; the hint now says which is which instead of reading as a typo.

_absent_note had two false-negative paths and one false positive (e06, m06). The six-term cap on _distinctive() dropped the trailing terms of a long query, which is where a model's guess sits (e06: "Port Jackson" in positions eight and nine, never checked); the engine's own "Missing: Port Jackson" annotation inside a snippet put the absent words into the haystack; and stripping the closing bracket off RDS(on) produced a term that matched nothing. The check is now uncapped (the reported list is capped at four), engine annotations are stripped from snippets before display or matching, balanced parentheses are kept, and matching ignores punctuation and spacing so RDS (on) matches RDS(on).

The dosing hint fired on "dose-response" (m10, h07). The epidemiology term is excluded from _PRACTICAL_MEDICINE; "ibuprofen dose adult" still fires.

cite_line produced "Clinic," (e13) and "csmith2" (m06). An organisational author with no comma - a multi-word name ending in an institutional noun, or containing "of"/"for" - is cited whole; a PDF /Author that is a username (one token, lower case or with a digit) and a /Title that is a filename or a template name are discarded, so no CITE AS is built from them.

lookup_rfc's description now says it also answers "which RFC says X", the question e12 answered from memory (RFC 7161 for the teapot code; it is RFC 2324).

---

Measured: 139/139 smoke cases pass (123 before, 16 added for this change). Every item above probed through the gateway on its motivating input: the Infineon PDF is refused at 78% glyph codes; PMC3898757, PMC8313037, PMC3976660 (a 7 KB article), PubMed 24459154 and Europe PMC MED/30241358 all arrive via the API with CITE AS lines from the JATS front matter; max_chars: 10716 is served; offset: -5 is prose; RFC 9110 15.5.19 skips the contents entry and reports section '15.5. Client Error 4xx'; 203.0.113.10 names TEST-NET-3; 8.8.8.8 carries AS15169 and 104.16.133.229 AS13335; 12mil/8 mil parse to 0.305/0.203 mm; the \' and function-order hints fire; "Zorbulanquex" is reported absent; "dose-response" does not draw the DailyMed hint and "ibuprofen dose adult mg" does. Fresh-chat spot checks through the UI, with the prompts verbatim: six prompts, one chat each, model row unchanged.

• e05 (3 calls, as v3): answer text unchanged; Sources: https://pypi.org/project/requests/ appended by the filter - the cited-URL loss of all three rounds closed.
• m02 (4 calls): both RDAP refusals name TEST-NET-2/3 and RFC 5737; the answer opens with that fact and treats the addresses as placeholders, which three rounds of memory-based hedging never quite did.
• h03 (3 calls, v3: 5): "12mil" and "1.53mm" parsed correctly at the first attempt - Z₀ 125 Ω where v3's first call printed −46.5 Ω - so the wasted round trip is gone; 54 mil and 110 mil delivered, both traces failed.
• m06 (8 calls, v3: 12): both exact IRLZ44N PDFs refused as 78% glyph codes, Mouser's copy timed out, and the model read Vishay's IRLZ44 sheet - and said so in its first sentence, labelling every figure as the variant's. The v3 fabrication ("0.044 Ω (4.5 V spec)", from memory, presented as the datasheet's) did not recur; the exact IRLZ44N values are still unreachable because every readable copy the tools found is a different part, which the answer states. The failure mode moved from wrong numbers presented as read to labelled substitute source, which is the change's purpose.
• h10 (8 calls, v3: 12, v2: 17): the four ipinfo fetches are gone; origin AS13335 and the announced prefixes (104.16.128.0/20, 2606:4700::/44) came from lookup_ip_or_asn's routing section, and the answer labels them "RDAP + RIPEstat routing view".
• e13 (6 calls): the PMC review (PMC7347500) arrived through the Europe PMC API with its CITE AS line where v3 got a Cloudflare stub; the answer ends with all three read pages - appended by the filter, since the text named none of them. The dosing hint did not fire (no dosing term in the query), as intended.

Transcripts are in the session scratchpad, not in prompt-testing/ - these are checks that the changes fire, not a round.

Measured, v4 round (run and graded 2026-09-12; prompt-testing/SCORES.md, one grade.md per prompt): mean 8.90 (v3 8.85), 35 pass / 4 partial / 1 fail, 44 points lost (v3 46). Correctness 92 %, calibration 88 %, tool use 91 %, communication 88 %.

• 55.1 put a URL on every answer that read a page: 16 of 16 opportunities, plus one the model wrote itself (e05) — 17 of 22 against v3's 1 of 22. The five without are four search-only answers (e07, e14, h01, m01) and h10, whose tools emit no trailer. Seven rubric points recovered (e05, e10, e13, e15, e20, m08, h07, half on h05). The ≥ 20-of-23 target is not met on the mechanical count and is met on every opportunity the filter has. Four defects on the one long research turn (m10): a "Key sources:" block is not recognised as the model's own attribution; a bare doi:10.… in the body is not matched to its doi.org entry; a trailer's CITE AS DOI and its URL count as two entries for one document; the first-six cap kept a cookie-wall stub and a wrong-paper fetch and dropped the paper the answer used. Also visible (e06): the filter cites what the tools read, not what the answer says — a Wikipedia URL under a constitutional claim the lead does not make.
• 55.2 — seven of the twelve confirmed in a real chat on the prompt that motivated them, each changing the outcome as intended: glyph PDF (m06, twice at 78 %, 12 → 6 calls, the v3 −2 gone), Europe PMC API (m10, three articles — the effect sizes quoted from the paper instead of recalled), contents-entry skip + "Contents" heading + PDF header suppression (e12 11 → 4 calls; h05 three correct section notes where v3 had two false ones), BGP routing (h10 12 → 8 calls, the ipinfo fetches gone), scope sentence (h04, the documentation prefix carried for the first time in four rounds; 6 → 9), pcb_calc units (h03, first-attempt parse, no negative impedance), absent-note uncapped (h01, terms at positions 8–9 reported). Five not exercised because the model did not reach the tool or the path: TEST-NET refusal (m02 chose lookup_rfc), sibling man sections (m05 made no call), both matlab_run hints (h08/m07 wrote no \' and no script function), prose 422 (no bad parameter), dose-response (no such query). cite_line fired right on three pages (h05 "STMicroelectronics", m06 PDF metadata discarded, e13) and missed two new shapes: "ElectricalLicenseRenewal com" → com, (e16) and "Kelly Johnson-Arbor; MD Medical Toxicologist" → "Johnson-Arbor et al." (h07). The lookup_rfc "which RFC says X" wording drew a topic query on m09 that got NO RESULTS (the index search is a title search).
• New harness defects from the round: wiring_calc reads a trailing "ft" as femto ("100ft" → 1e-13 m, success, 0.00 % drop; e16, two calls lost and a hand-computed figure wrong by 4×) — the pcb_calc bug's sibling; fetch_page on a reports index page loses the document links (h02 never found the PDF v3 read, and answered from snippets and a wrong fiscal-year inference: 9.5 → 5.5); a PDF snippeted as raw bytes is unrecognisable; the string-type lint note repeats once per line (12 copies, h06); a "Cookies must be enabled" page returns as a success with a trailer; the JATS renderer dropped the authors on PMC8066601; PDF text layers that split words ("configurati on") defeat exact-heading search= (h05).
• Model-side, no harness cause: e19 10 → 0 — the clock was not called for the first time in four rounds and a training-data date was given ("Saturday, 19 July 2025"; the thinking names "the current timestamp" as its source and does not fetch it). The universal fix is to put the date in context (Open WebUI's {{CURRENT_DATE}}, or the status filter's inlet), which changes what the prompt tests — the user's call. Same decision going the other way on m04 (wrote a parameterised function instead of asking; 9 → 7), m05 (no man page; 9 → 8), e06 (s.125 recited after a lookup that did not contain it); and going right on m08 (2.5 → 10, the office-holder looked up this time). Volunteered extras wrong while the asked-for facts are right, ten prompts: e01, e12, e16, e18, h03, m01, m03, m07, m10, h07.

---

Change 56 — the v4 round's easy wins

The v4 round scored 8.90/10 against v3's 8.85 (35 pass / 4 partial / 1 fail; 44 points lost). Of those, the grade files pin twelve to a harness cause that is a few lines each, and one (e19, the only 0) to a fact that was simply not in context. This change takes all thirteen and nothing else. Each is domain-shaped — a unit, a page shape, a lint rule, a query shape — and none is keyed to a test prompt. Nothing here has been run against the model: the smoke suite is the whole measurement, by instruction, and the v5 round is the test.

---

56.1 — wiring_calc reads feet as feet

Symptom. e16: length: "100ft" was accepted, parsed as 1 × 10⁻¹³ m (the trailing "f" is femto to the engineering-notation parser, the same defect Change 55.2 fixed for pcb_calc), and the tool reported a 0.00 % drop as a success. Two calls were lost to it and the figure the model then worked by hand was wrong by 4×.

Change. A _cable_length_m reader in front of the three voltage-drop calculations. Metres by default; an attached unit may be m, cm, mm, km, ft, feet, ', in, " or yd; anything else ("100k") is refused with "Could not read" rather than guessed. The header prints what was understood — 30.48 m (100 ft) — so a wrong reading is visible in the result, not hidden behind a plausible number. The schema says the same.

Result. Smoke: "100ft" → 30.48 m, 5.29 % on 12 AWG at 20 A / 120 V (the e16 figure); "30m" → 30 m, not 30 mm; "250 ft" on three-phase → 76.2 m.

---

56.2 — answer_sources 0.2.0: the four defects from m10

Symptom. The research turn that motivated the filter got the worst of it: a **Key sources:** block of the model's own was not recognised (the label did not start with the word), so a second, worse list was appended under it; a bare doi:10.… in the body did not match its doi.org entry; a trailer's CITE AS DOI and its page URL counted as two documents, so one paper filled two of the six slots; and the cap kept a cookie-wall stub and a wrong-paper fetch while dropping the paper the answer used.

Change. The own-sources check now accepts up to two words before sources, references, citations or bibliography and a used/consulted/read/cited suffix, through markdown emphasis; a DOI anywhere in the body matches the read set; each Sources: line is one document (the page URL shown, the DOI kept as an alias); the cap is eight. The URL-in-body rule is unchanged. The stub half of the fourth defect is 56.3.

Result. Offline, on the m10 message: no second block; on e05's: unchanged. Deployed to the answer_sources row (7,734 characters, active, global).

---

56.3 — a cookie wall is a stub

Symptom. m10: PubMed's "Cookies must be enabled" page — 76 characters, served with a 200 — came back as a successful fetch, with a provenance note describing an abstract that was not there and a Sources: trailer the filter then listed.

Change. cookies must be enabled and enable cookies for join the stub markers; the fetch FAILs with the client-rendered/gated message and emits no trailer.

---

56.4 — cite_line: a site's name is not an author, and a credential is not a co-author

Symptom. e16: "ElectricalLicenseRenewal com" as citation_author produced CITE AS: com, …. h07: "Kelly Johnson-Arbor; MD Medical Toxicologist" produced "Johnson-Arbor et al." — a single author cited as several.

Change. A name whose last word is a top-level-domain word (com, org, net, gov, io, …) is an organisation and is cited whole. toolbase.clean_authors drops an author entry that is a credential or begins with one (MD, PhD, RN, FACEP, "Medical Toxicologist" …) and strips trailing degree tokens; it runs on HTML meta authors, PDF metadata and JATS front matter alike.

Result. Smoke: the poison.org byline cites Johnson-Arbor, and never et al.; the licence-renewal page never CITE AS: com,.

---

56.5 — one lint note per rule

Symptom. h06: twelve double-quoted strings, twelve copies of the same "string type" note — a screenful in which any other finding was invisible.

Change. _fmt_findings groups by rule; a rule that fires more than once prints once with its line numbers: - lines 1, 6, 11 (12 places): … (first eight lines, then "and N more").

---

56.6 — the std/var weight flag, stated the right way round

Symptom. m04: the delivered function's comment, docstring and closing notes all said std(X, 0, 1) is the population statistic and told the user to swap to std(X, 1, 1) for the sample one — every clause inverted, and the same inversion v2's grade recorded. It is a belief (the thinking says "w=0 → divide by N"), so no prompt wording reaches it.

Change. A SEMANTIC lint rule fires on any std(…, 0|1, …) / var(…, 0|1, …) and states the fact in the run result: w=0 normalises by N−1 (the sample statistic, the default), w=1 by N (the population statistic). A bare std(x) draws nothing.

---

56.7 — pcb_calc trace_width prints the width at the other copper weight

Symptom. h03: the question turns on "would 2 oz copper help"; the tool gave the 1 oz width and the model worked the 2 oz figure by hand, with a square-root scaling it should not have.

Change. The result carries one more line: the same cross-section at the other weight (1 oz ↔ 2 oz), with the mil and mm width and the fact that width scales inversely with copper thickness. The IPC-2221 area is unchanged; it is the same number divided differently.

Result. Smoke: 3 A, 10 °C rise → "At 2 oz copper … 26.9 mil".

---

56.8 — the dosing hint names the product

Symptom. h07: the DailyMed hint fired and, for the first time, the model searched DailyMed — then took the generic 500 mg label's 4,000 mg ceiling for a branded extra-strength product whose label says 3,000 mg, with both figures in front of it.

Change. One clause on the hint: "Labels differ by product and strength - read the label for the product the question names, not a generic one." A fact about labels, not a rule about the answer.

---

56.9 — lookup_rfc falls back to phrases

Symptom. m09: "which RFC says X" drew a topic-shaped query ("DNS domain name maximum length 63 octets label") that got NO RESULTS, because the index search matches words in RFC titles. The model recovered by guessing the right RFC number, which is not a method.

Change. _rfc_title_search tries the whole query, then each two-word phrase from it, then each word of five letters or more (six tries at most). The listing says which phrase matched ("these titles contain 'Domain Names'"); the NO RESULTS text says the search matches titles, not topics, and shows what a title-shaped query looks like.

Result. Smoke: that query → RFC 1035.

---

56.10 — document links survive extraction

Symptom. h02: the investor-relations reports index lists each report as a title, a date and a "Download" link, and trafilatura discards the link-dense grid whole, so the model read the index and could not get from "Interim results" to the PDF it needed — the PDF v3 had read. It answered from snippets and a wrong fiscal-year inference instead (9.5 → 5.5).

Change. Before extraction, an anchor whose target looks like a document (.pdf, .xlsx, /document, /download, /attachment, …) or whose text says so becomes a plain span in place: Download (https://…/reports/10/document). Ordinary navigation links are untouched.

Result. Smoke: the reports index now carries its document URLs.

---

56.11 — split-word headings, and a cross-reference is not the section

Symptom. h05: the mirror PDF's text layer renders the heading as "Clock configurati on register", so an exact search= for the heading landed twice on a cross-reference to it ("refer to Section 4.3.2: Clock configuration register") and never on the heading. The model found the section by paging to a guessed offset.

Change. The first pattern textwin tries is heading-shaped — optional section number, the needle with whitespace tolerated inside its words, end of line — so the split heading is found before any prose match. When the chosen match is preceded by a "refer to Section"/"see Section" cue, the jump header says so and suggests the section number or a term from the section's first paragraph.

---

56.12 — JATS author groups

Symptom. m10: PMC8066601 rendered with no authors — the archive labels the <contrib-group content-type="author"> and leaves each <contrib> bare, and the renderer looked only for contrib-type="author" on the contrib.

Change. Labelled contribs first; then contribs inside a labelled author group; then any contrib carrying a surname or a collaboration name. Names pass through clean_authors.

---

56.13 — the date is in context

Symptom. e19 10 → 0: for the first time in four rounds the clock was not called, and a training-data date was given ("Saturday, 19 July 2025"). The thinking names "the current timestamp" as its source as though it were in context; it was not.

Change. The system prompt's second paragraph is now Today is {{CURRENT_WEEKDAY}}, {{CURRENT_DATE}} (the user's local date), which Open WebUI expands server-side (verified: prompt_template in resolve_system_prompt, container zone America/New_York, the user's), with the instruction to use it for anything dated and to call get_current_timestamp for the time of day, UTC or the zone name. Two lines; the rest of the prompt is byte-identical. Written to the qwen3.8:latest row's params.system (14,608 characters; sampling parameters, tool binding and access operator unchanged — the update endpoint re-creates the operator row, same principal and permission).

This changes what e19 tests, from recall-vs-tool to context-vs-recall; the rubric carries a dated revision note and accepts either the prompt or the tool as the source. The v1–v4 grades stand on the earlier wording.

---

Measured: 154/154 smoke cases pass (139 before, 15 added for this change: five wiring_calc unit readings, two pcb_calc, three matlab_run, two lookup_rfc, three fetch_page). The answer_sources change was exercised offline on the m10 and e05 messages. Both deployed rows were read back after the write. The page cache was flushed (owuitools:page:*) so the document-link rewrite is what the next fetch serves.

Not measured: no prompt has been run against the model, by instruction. The v5 round is the measurement for all thirteen items, and e19 in particular now tests a different thing.

---

Change 57 — plot type that fits the canvas, and the key under it

Reported from use, not from a graded prompt: "for the matlab renders, try to space out everything more, it is often that text will overlap. Also, move the legend to below the render so that it doesn't overlap." Both are the same picture — a 900 × 675 plot whose text was rendered at roughly twice the size it was asked for, with an opaque key sitting on the curves.

Three reproductions, each a plot a model might reasonably write: three curves with a long title and a three-entry legend; a 2 × 1 subplot stack; a bar chart with six named categories. In the first the title ran off the right edge, the ylabel was drawn across the y tick labels (the minus signs vanished under it), the xlabel was cut off by the bottom of the canvas, and the legend covered a fifth of the data. In the third the six category names ran into each other and the y tick labels were clipped to their last digit — "25, 0, 5, 0, 5, 0" for 25 down to 0.

None of it was a gnuplot limitation. It was four separate layout faults, all fixed here, all in the renderer rather than in anything the model writes.

---

57.1 — a bigger canvas without bigger type

Symptom. Every character on every plot was about twice the size it should have been for the canvas it was on: the 10 pt axes font drawn at 16.7 pt, the 11 pt title at 18.3 pt, on a canvas 900 px wide. That is the whole cause of the clipped title, the ylabel over the tick labels and the bar-chart names running together. Raising the canvas size did not help, which is what makes the cause non-obvious.

Cause. -S900,675. Octave's print.m reads -S as "a six-inch figure blown up to this many pixels" and scales every font in the figure by width / 6 / screenpixelsperinch — at 72 ppi in this container, width / 432, so 2.08× at 900 px and 2.55× at 1100. The type grew with the canvas, so no size was ever big enough. The shim's -F:8 was an attempt to hold it down and could not: -F is multiplied by the same factor.

Change. Size by paper instead. The shim sets paperposition to width/72 inches with paperpositionmode manual and passes no -S, which takes the other branch of print.m: the canvas is paperposition × 72 and the scale factor stays 1, so text keeps the size Octave gave it and a bigger canvas buys real room. 1100 × 825 by default. The shape follows the figure — a script that asks for a 1200 × 400 figure gets a 1100 × 440 PNG, clamped to between 0.4 and 1.5 — and a script that passes its own -S is left alone entirely. -r72 goes with it, because the JPEG terminal multiplies paper inches by the print resolution and turned the same canvas into a 2292 px image with 10 pt type marooned in it; pngcairo ignores the setting.

Result. Tick labels and axis labels at 12 pt on an 1100 px canvas (57.2 chooses the 12), against 16.7 and 18.3 on 900 px before. Nothing is clipped in any of the three reproductions. Smoke asserts the canvas exactly: PNG carries width and height at a fixed offset, so the base64 prefix of the returned data URI is "1100 × 825" — it fails if the shim leaves the path, if -S scaling comes back, or if the canvas changes size.

---

57.2 — axis labels the size gnuplot reserved room for

Symptom. With the type fixed, the xlabel still landed on the row of x tick labels, and the ylabel still crowded the y tick labels.

Cause. gnuplot lays a plot out in character units of the terminal font. Octave sets that terminal font from the axes font size — 10 pt — and then draws axis labels at 11, because labelfontsizemultiplier is 1.1. The space reserved below the tick labels is a 10 pt line and the label drawn into it is an 11 pt line, so it rides up into the ticks. Confirmed against gnuplot directly, same plot and same key: xlabel at 11 pt overlaps, at 10 pt clears.

Change. The shim sets labelfontsizemultiplier to 1 on every data axes for the duration of the print. The title keeps its 1.1 and its bold weight — it sits in the top margin with nothing under it to collide with.

With the tie between reserved space and drawn text in place, the size itself can be chosen on its merits, and Octave's default 10 pt — chosen for a 560 px figure on a screen — is too small on an 1100 px canvas that a chat window then shows at 700. The shim sets 12 pt, and only where the script did not set a size itself: fontsizemode records which, and an axes that says 16 pt keeps 16 pt and gets it back after the print.

Result. Clear separation between tick labels, axis labels and everything below them, on all three reproductions and on log axes, 3-D and subplots. Type at 12 pt on 1100 px still leaves the six bar-chart category names room side by side, and is legible when the picture is displayed at 700 px.

---

57.3 — the key under the axes, not over the data

Symptom. The requested change, and the reason it was requested: an inside legend sits on top of what it labels. 'best' does not help — this backend does not implement it, warns, and puts every legend top-right regardless of what the script asked for.

Change. For a figure with one plot in it, every legend that is not already outside is moved to southoutside for the print and put back afterwards. gnuplot reserves the space for an outside key and shrinks the plot to fit, so nothing can collide with the data.

The frame comes off with the move. gnuplot packs an outside key flush against the x-axis label — a one-pixel gap, whatever the type size, and the label's own offset moves both together — so the box's top edge is drawn through the descenders of that label and reads as an underline under it. Out on the white margin, with no plot lines behind the text, the frame was only decoration.

Two legends are left alone. One a script deliberately put outside already ('eastoutside'), which is a layout decision rather than the default falling where it falls. And every legend in a figure with more than one plot: a subplot grid has no margin under each plot to give away, only the gap before the next one's title, and a key dropped into that gap lands on both — verified, it overlaps the xlabel above it and the title below it. Inside is where they fit there, and 57.4 keeps them readable.

Result. Single plots: the key is a caption under the axes, three entries or six. Subplots: the keys stay where the script put them and nothing else on the figure overlaps any more.

---

57.4 — an opaque key is for an inside key only

Symptom. With the key moved below, the xlabel printed as "from the triaaer edae": the tail of every g, y and p in it was erased.

Cause. Change 42's build-time patch (patch-legend-opaque.sh) emits set key opaque unconditionally. Inside the axes that is right and still is — gnuplot's default leaves the key transparent, so plot lines run through the legend text on exactly the busy plot where a legend matters most. Outside the axes the fill has no plot lines to hide and lands on the label instead.

Change. The patch now emits set key opaque when the key is inside and set key noopaque when it is outside. The noopaque is stated rather than left to the default, because set key opaque from an earlier axes persists through the rest of a multiplot stream. The patch keeps its anchor check and still exits 0 if an Octave upgrade moves the code.

Result. Descenders intact under an outside key; inside keys unchanged.

---

Measured: 156/156 smoke cases pass (154 before, 2 added: a plot with a legend, and a script-sized wide figure, each asserting the canvas through the PNG header). Nine figures rendered and inspected by eye, before and after: three curves with a long title and legend, six-entry legend on log axes, six-category bar chart, 2 × 1 subplots, peaks surface with a colour bar, a 1200 × 400 figure with an eastoutside legend, an axes that sets its own 16 pt, saveas instead of print, and a JPEG. The last four exist because each takes a different path through the shim. End-to-end through the gateway as well, not only in the container: the tool returns a 1100 × 825 data URI.

The picture is 0.9 MP where it was 0.6, and Open WebUI hands it to the model as an input image, so a returned figure costs about half again as many vision tokens as it did.

Not measured: no prompt has been run against the model, by instruction. Nothing here changes tool text, so it does not move a graded answer by itself; what it changes is whether a plot the model hands back can be read.

---

Change 58 — closing the MATLAB gap where it can be closed

Asked from use: "would it be possible to modify Octave to be even closer to how MATLAB functions? There are some small discrepancies that can cause issues for more complex operations."

The investigation came first and killed the obvious answer. A throwaway Debian 13 image with Octave 9.4 was built and run against the same probes as the 7.3 in the image: the upgrade adds normalize and tensorprod and nothing else. contains, strlength, string, table, datetime, histcounts, round(x,n), sum(...,'all') and maxk are missing in both, and every semantic probe — error identifiers, && on a non-scalar, arguments validators, A\b, the RNG stream, the default save format, num2str on an int64 — behaved identically on the two versions. A version bump costs a rebuild and buys nothing here.

What it does buy is elsewhere, and this change takes the six cheapest pieces of it.

---

58.1 — a script may put its functions at the end, as MATLAB does

Symptom. MATLAB (R2016b+) hoists a script's local functions: they go at the END and are called from anywhere above. Octave defines a function when execution reaches the definition, so the same script dies on 'double_it' undefined. Correct, ordinary, modern MATLAB, refused. It cost m07 a whole tool call in the v3 round, and the error reads like a typo, so the model's instinct is to rewrite code that was already right.

Change. matlab_lint.split_local_functions lifts the trailing function blocks out of the script and matlab.py writes them as files beside it, which is where Octave looks. Two properties matter as much as the lift itself:

• Line numbers do not move. The lines the definitions came from are replaced with blank lines, and each lifted file is padded with as many leading blank lines as it had above it, so an error inside one reports the line the model actually wrote. The stack frame says main.m line 5 (in pick) rather than naming a file nobody created.
• It refuses anything it does not fully understand — a nested function, a classdef, code after the definitions, two functions of one name, an unbalanced end, a signature it cannot read, more than forty functions. Those files run exactly as they did before. Mangling a file that was about to work is worse than not helping it.

The name that becomes a filename is checked twice: by the parser, which accepts an identifier and nothing else, and again at the write itself, against the pattern the helper files use.

Result. tests/test_local_functions.py — 25 cases, pure Python, no container: the lift, the line-number preservation, end used as an index, a one-line if/end, a switch inside the function, the word function inside a string and inside a comment, and each refusal. Live: a two-function script runs; a helper file of the same name wins over the lifted one; an error inside a lifted function reports main.m line 5 (in pick); recursion works; an unterminated tail works; a nested function still fails with Octave's own parse error.

---

58.2 — Octave's parser as a second compatibility linter

Symptom. The regex linter catches the Octave-isms it has rules for. x .*= c, a bare newline inside parentheses and several others have no rule, and a regex over a stripped copy of the source can never be as sure about the dialect as the parser that implements it.

Cause of the gap. Octave:language-extension is exactly this signal and the driver turned it OFF, with a comment saying it was unusable: Octave's own library is written in the same dialect, so a real error disappeared under hundreds of warnings about files nobody can edit.

Change. The missing piece was that every one of those warnings names its file. The driver turns the warning on — last, after the pkg loads and the addpath, so none of that machinery is parsed with it enabled — and run.sh drops the library ones by path before anything is measured against the output cap. A warning is three parts (the line, a called from, its indented frames), and dropping only the first left orphaned frames that read like a stack trace for an error that never happened, so the filter takes the block.

What survives is the user's code. matlab.py turns it into findings with line numbers, drops any line the linter already reported, caps at twelve, and removes them from the raw warnings so one defect is not printed twice.

Result. x .*= 2 — which no rule covers — now comes back as "Octave's parser read .*= 2; used as operator - an Octave extension, not MATLAB syntax", and the verdict is the existing "Ran in the sandbox, but THIS IS NOT VALID MATLAB". Scratch runs are not told, the same policy the linter's own octave_only findings follow.

---

58.3 — eight MATLAB functions this Octave does not have

Change. octave/shim/ gains contains, strlength, matches, maxk, mink, histcounts, discretize and rmmissing, joining print on the path ahead of everything. They are written in plain MATLAB, and the policy for all of them is in contains.m: implement the documented behaviour for the forms that are actually written, and raise a clear error for the rest. histcounts(x) with automatic binning says so instead of guessing a binning rule — MATLAB's is documented and non-trivial, and a shim that picked a different one would hand back a different histogram with nothing to show for it. rmmissing refuses cells rather than inventing a meaning for missing text.

check-shims.sh runs at image build time and does two things that cannot be done later:

• Drops any shim whose name Octave has since acquired. This directory shadows everything; a stale four-signature contains replacing a complete one is the exact failure the shims exist to prevent. Octave gaining a function is good news, and the right response is to delete ours.
• Tests what survives, against the behaviour MATLAB documents — including the edge conventions that are why these are not one-line aliases: histcounts' last bin is closed where histc's is not, maxk is column-wise on a matrix and returns a row for a row. A failure fails the build; verified by breaking a shim on purpose and watching it.

_MATLAB_ONLY and _OCTAVE_EQUIVALENT lose the five names that now work, so the model is no longer told to rewrite contains into strfind.

Not shimmed, deliberately: round(x, n) and sum(..., 'all'). Those are extra arguments to functions Octave already has, so the shim would have to shadow a hot builtin — measured at 4.6x on the shadowed call (20,000 sum calls: 0.080 s to 0.369 s), paid by every call in the session including the ones Octave's own library makes. 58.5 handles them for free instead.

---

58.4 — two preferences, and the ones not taken

Change. The driver sets print_empty_dimensions(false), so an empty array prints x = [] as MATLAB does rather than x = [](0x0) — output the model reads back to the user — and save_default_options("-mat7-binary"), because save("data.mat", "x") was writing an Octave TEXT file with a # Created by Octave header. A user opening that in MATLAB gets nothing. The header now reads MATLAB 5.0 MAT-file.

Octave's --traditional flag sets both of these and is documented as "closer MATLAB compatibility", but it is not taken wholesale: it also sets struct_levels_to_print(0), which prints val: 1x3 matrix instead of a struct's contents, and beep_on_error(true).

fixed_point_format(true) is the interesting refusal. It makes numeric display genuinely closer to MATLAB — [1000.5 2000.25; 3000 4000] prints under a 1.0e+03 * header in MATLAB, and element by element here — and it was left off anyway. This tool exists to give the model numbers it can quote, and a display where every value must be multiplied by a header two lines up is a transcription error waiting to happen. Fidelity lost, accuracy kept.

---

58.5 — MATLAB call forms this Octave refuses

Symptom. round(pi, 3) gives "Invalid call to round"; sum(A, 'all') gives "sum: unrecognized type argument 'all'". Both are correct MATLAB (R2014b and R2018b). Neither message says so, and "invalid" reads as "you wrote a bug".

Change. Two patterns in _classify_error. They name the MATLAB version the form comes from, give the local rewrite (round(x * 10^n) / 10^n, sum(A(:))), and then split on purpose: a scratch calculation should be rewritten, a deliverable should keep the MATLAB form and say that this sandbox could not run that line. The 'all' pattern matches the source as written rather than the literal-stripped copy — its subject IS a string, and the stripped copy has blanked it, the same trap the linter documents for its own raw rule.

---

58.6 — a function named after a MATLAB built-in

Symptom. From m04, v4: the answer defined function normalize(X). normalize is a MATLAB built-in (R2018a+), so the delivered file shadows it in the user's MATLAB — and nothing here noticed, because normalize does not exist in this Octave at all. A name that is free in the sandbox and taken in MATLAB is invisible to every other check that runs.

Change. The names a script defines — lifted local functions and helper files — are checked against the MATLAB-only table plus a curated list of core names, each one verified present in this image and each a plausible thing to call your own function. A hit gets one short section saying yours wins silently, including for calls that meant the built-in. The list is deliberately partial: a missing name costs a note, never a wrong one. Scratch code is nobody's deliverable and is not told.

---

Measured: 166/166 smoke cases pass (156 before; 10 added, 3 rewritten because they asserted limitations this change removes — the contains advice, the "defined below the line that calls it" hint, and the equivalent-advice path, which now uses split, still missing). 25/25 unit cases in tests/test_local_functions.py. 27 shim assertions at image build time, and the build was verified to fail when a shim is wrong. Eight adversarial live cases: a helper that collides with a lifted name, a function hidden inside a block comment, function/end inside a string, a nested function, three functions at once, a user function shadowing a shim, an unterminated tail, and recursion inside a lifted function.

Not measured: no prompt has been run against the model, and the MATLAB half of every claim here is from documentation — there is no MATLAB in this house to check it against. The symbolic package (SymPy-backed syms, solve, vpa, laplace) was tested and works offline under the sandbox's full hardening in 0.03 s to load and 1.15 s for the first call, and is not included: it spawns a Python child, and timeout -s KILL kills only the Octave process, so a job that dies mid-symbolic leaves an orphan in a container that serves every other job. Process-group cleanup is its own change, with its own review.

---

What was and was not measured

Measured: 66/66 existing smoke cases still pass, and 107/107 after Change 53. 39/39 targeted cases for the new behaviours pass through the gateway. Two graded prompts re-run end to end through the chat UI, both fixed. Engine availability, model row contents and filter rows verified directly.

Not measured: the full 40-prompt suite has not been re-run, by choice — spot checks were the agreed level for now. The per-bucket targets from the plan (recall 30.5 → ≤15, attribution 28.5 → ≤14, tool gaps 20.5 → ≤8, loop 10 → 0, arithmetic 3.5 → 0) are therefore unverified in aggregate. The presence-penalty change in particular has not been tested against the loop that motivated it.

---

Change 59 — a sub-agent with a real machine in front of it

Symptom. The assistant could reason about a script but never try one. Asked whether a PowerShell snippet works, it could only read documentation about PowerShell; asked why a build fails, it could only guess from the error text it was given. Every other tool in this suite returns a fact about the world. There was no tool that returned a fact about running something.

Cause. Nothing structural — the capability had simply never been built. The Octave sandbox (Change 34) is the nearest thing and is deliberately narrow: one interpreter, no network, no filesystem that survives.

Change. vm_agent: one tool that hands a task to a second LLM turn-loop — the same qwen3.8:latest on the same Ollama, with its own system prompt and its own thirteen tools — which drives a throwaway Ubuntu or Windows VM. The main model's surface stays at one tool and never carries VM-control instructions, screenshots or transcripts. Everything the VM touches is destroyed afterwards except a per-project folder on the ZFS array, which is a Gitea working tree.

Five new components: a host daemon (vmd) holding everything privileged behind a unix socket; an agent container holding no credential and able to reach exactly one address; an nginx gateway holding the two secrets; a netns guard; and a GTK4 desktop tool for the operator to unfreeze a base image.

The rest of this entry is the things that were measured, several of which contradicted the design they were meant to confirm.

59.1 — the primary network control was doing nothing

Symptom. The sandbox unit carried IPAddressAllow=0.0.0.0/0 alongside a deny list for the private ranges, on the assumption that longest-prefix match would let the specific denies win. Everything appeared to work: a probe from the sandbox uid could not reach 192.168.1.10:7200, 172.17.0.1:11434 or 127.0.0.1:3000.

Cause, and it is mine. systemd.resource-control(5) applies the two lists in a fixed order: granted if the address matches IPAddressAllow=; otherwise denied if it matches IPAddressDeny=; otherwise granted. There is no arbitration between them. 0.0.0.0/0 matched every address at step one, so the entire deny list was dead code. What made it look correct was the second layer — an nftables table matching on the qemu uid — quietly covering for it.

Change. Deny-only, with a comment saying why there must never be an IPAddressAllow= line in that file. Rule three ("otherwise granted") is what grants the internet; it needs no help.

Result. With the nft table deleted, the corrected unit blocks all ten must-fail targets and leaves 1.1.1.1 and 8.8.8.8 reachable. With the nft table restored and IPAddressDeny stripped from a scratch unit, the same suite passes again. tests/vm_host_asserts.sh now does both of those every run.

Generalisable lesson. Two independent controls are worth having and worth testing independently, because in normal operation either one will hide the other's absence. The test that found this is the one that turns each layer off in turn — not the one that checks both are loaded. This is the same reasoning netguard/firewall.sh already applies to itself ("a firewall that silently failed to apply is worse than none"); it just had not been applied to the layer above it.

59.2 — systemd's syscall filter and QEMU's -sandbox are mutually exclusive

Symptom. qemu-system-x86_64: -sandbox on,...: There is no option group 'sandbox', exit 1, from inside the unit. The identical command line run by hand started fine, and ldd shows libseccomp.so.2 linked into the binary.

Cause. SystemCallFilter=@system-service on the unit. Isolated by running the same argv under systemd-run with and without the property: without it, accepted; with it, the error.

The mechanism, found later and worth the correction. This entry first said the two were "mutually exclusive", which is true in effect but does not say why. The maintenance unit was added afterwards without SystemCallErrorNumber=EPERM, and qemu there died instantly with status=31/SYS; the audit record names the syscall — 317, which is seccomp(2) itself. So qemu calls seccomp() at startup to probe for its own sandbox, @system-service does not permit it, and what happens next depends entirely on that one extra line: with SystemCallErrorNumber=EPERM the probe returns an error and qemu quietly concludes seccomp is unavailable, which is why -sandbox reported "there is no option group"; without it, the kernel raises SIGSYS and kills the process before it has started.

That makes SystemCallErrorNumber= load-bearing rather than cosmetic, and both units now carry it with a comment saying so.

Change. -sandbox dropped, systemd's filter kept, and the reason written into owuivm-launch where somebody will otherwise "restore" the flag. systemd's is the broader allowlist (@system-service excludes @mount, @swap, @module, @reboot and @obsolete), it covers the whole process tree, and systemd-analyze security can see it. What is given up is spawn=deny; the vector that mattered there, migrate exec: over QMP, is already closed by vmd exposing typed verbs and never a raw monitor command.

Not measured: whether QEMU's own filter would have been narrower in practice. It cannot be run alongside the other, so there is nothing to compare.

59.3 — ping can never work here, and it reads as "no internet"

Symptom. Anticipated rather than suffered, but worth the entry. net.ipv4.ping_group_range on this host is "1 0" — an empty range, so no group may open an unprivileged ICMP socket. libslirp cannot forward ICMP without one, so ping fails in every guest, permanently, on a machine whose HTTPS works perfectly.

Change. A check_internet tool that uses curl, and the fact stated in the tool's own output rather than in the agent's system prompt.

Why there and not the prompt. The finding restated four times in this file: the model treats tool output as fact and system-prompt text as advice. An agent that has just watched ping fail three times needs the correction where it reads instructions as findings.

Result. tests/vm_negative_guest.sh asserts ping fails while the HTTPS control passes, so the day ping_group_range changes, the suite says so instead of quietly becoming wrong.

59.4 — the guest booted with no DNS, and the reason was cloud-init

Symptom. First boot of the base image: every TCP target behaved exactly as designed — LAN blocked, 1.1.1.1:443 and 8.8.8.8:53 reachable — and getent hosts archive.ubuntu.com returned nothing. /etc/netplan/ was empty and resolvectl showed 10.0.2.3.

Cause. Subiquity does not write /etc/netplan itself; it hands the network configuration to cloud-init, which renders it on first boot. The image deliberately disables cloud-init — it must not re-run on every boot of a disk restored from a frozen base thousands of times — so the netplan was never written and the guest fell back to DHCP. SLIRP's DHCP hands out 10.0.2.3 as the nameserver, which is a forwarder inside the qemu process that relays to this host's resolver, and the sandbox's egress filter denies 127.0.0.0/8 — correctly, because the host's resolver is a LAN service.

Change. The netplan is written directly in late-commands, static, with a public resolver.

Generalisable lesson. The isolation worked exactly as specified and the result was a machine that looked broken in a completely different place. A control that denies something the guest depends on by default will surface as a mystery three layers away; the guest's configuration has to be changed to match the policy, not just tested against it.

59.5 — two permission findings, both silent

qemu-img creates 0644, so a umask of 0007 leaves 0640. vmd creates the session overlay and qemu writes it, so the file has to be group-writable. UMask=0007 cannot produce that: qemu-img asks the kernel for 0644, and a umask only ever clears bits. The result was qemu failing with Could not open ...: Permission denied on a file it could plainly read. Fixed with an explicit chmod 0o660.

/Pool is 0770 root:docker. Neither service user could traverse it, so nothing under it was reachable. The obvious fix — adding them to docker — would have handed both of them the docker socket, which is root. A traverse-only ACL (setfacl -m u:owuivmd:--x,u:owuivm:--x) is the whole of what they actually need: they reach their own datasets and still cannot list the parent.

59.6 — a correction to this file: Open WebUI does time out a tool call

Symptom. gateway/nginx.conf has said since Change 34 that "Open WebUI sets no tool-server timeout of its own (its AIOHTTP_CLIENT_TIMEOUT is unset here), so this is the only clock".

Cause. That reads "unset" as "unlimited". env.py:593-597 sets AIOHTTP_CLIENT_TIMEOUT to 300 when the variable is absent, and env.py:671-679 falls AIOHTTP_CLIENT_TIMEOUT_TOOL_SERVER back to it. utils/tools.py:1715 uses that for every tool call. So there have always been two clocks on a tool call, and Open WebUI's was the shorter one.

Change. The comment is corrected in place. A VM session needs all three clocks raised together — OWUITOOLS_VM_WAIT_SECONDS in the app, proxy_read_timeout on a dedicated location = /tools/vm_agent, and AIOHTTP_CLIENT_TIMEOUT_TOOL_SERVER on the open-webui container — with the app's the smallest, so it is the app that answers.

Process lesson. The original claim was written from reading one env var's default and not the fallback chain under it. Nothing had failed because of it: 240 s is under 300 s, so the gateway always answered first and the second clock was never reached.

59.7 — what the agent is not allowed to talk its way past

The previous sub-agent stack on this host was retired for accuracy, and its post-mortem names two mechanisms: a coding agent that "hallucinated third-party library APIs and wrote self-confirming tests around them", and agents that, "allowed to run search-only", guessed from snippets. Both are the same failure — the agent deciding when it is finished, against no standard but its own prose. Three things answer it, and none of them is a sentence in a prompt:

• plan_tasks must be the first call. Every other verb is refused until there is a list.
• finish is refused while any task is open, and refused if nothing was ever run. Marking a task blocked or dropped requires a reason, so "I could not finish" without one is not an available outcome.
• The verification command is re-run by vmd after the agent's last step, in a fresh shell, and the exit code the user sees is the one from that second run. If it disagrees with the agent, the agent's summary is still shown — labelled a claim, with the real output beside it.

vmd holds the task list, not the agent, so it cannot be quietly reworded to match what happened; and vmd assembles the tool result, so the facts come from the side that is not being checked. That last point is also a prompt-injection boundary: guest stdout reaches the agent, whose summary reaches the main model, so the summary is fenced, capped and labelled, and data: URIs are stripped — budget.cap() passes a bare data URI through untouched and Open WebUI hands such a result to the model as a vision input, which would let a guest choose a picture for the main model to look at.

Mitigated, not eliminated: a self-confirming test still passes when re-run, because it is still the agent's test. All that is done cheaply is to notice the shape — a verification command naming a file created during this same session — and say so on the report.

---

Measured: both isolation layers proven independently with the other removed; ten must-fail targets blocked and three must-pass reachable from inside a booted guest; a planted core.hooksPath executes under plain git and does not under gitea.commit(); polkit refuses the maintenance unit to a non-local session and vmd's helper has no verb for it; screendump accepts format: png and input-send-event exists, both read from this host's live QMP schema; a session boots and answers on its guest agent in 25.7 s.

59.8 — the first three live sessions, and what each of them broke

Driven by the model itself, not by vmctl. Each failure is recorded because each one was in a different layer and only the third run reached the work.

Run 1 died at step one: the model refused the request (HTTP 400): time: missing unit in duration "-1". keep_alive was being sent as the string "-1"; Ollama takes a number of seconds (negative meaning indefinitely) or a duration string like 30m, and parses anything quoted as the latter. The report still came back complete, saying exactly that — which is the first evidence that the "every exit produces a result" rule holds, because the alternative was a started VM and a blank answer.

Run 2 died at 4 minutes with nginx's HTML 504. The dedicated location = /tools/vm_agent had been written and validated but owuitools-gw had never been restarted, so the 240 s default was still in force. Incidentally this proved the cancellation path for real: the api saw the client disappear, wrote the CANCEL marker, and the machine was stopped six seconds later. No orphaned VM — which is the bug the previous stack shipped.

Run 3 completed. 9 agent steps, 153 s, 4 commands in the machine, one of which exited non-zero and was recovered from. The verification command it chose is worth quoting, because it is the behaviour the whole finish gate exists to produce:

out=$(bash /mnt/project/biggest.sh) && [ "$(printf '%s\n' "$out" | wc -l)" -eq 10 ] \
  && [ "$(printf '%s\n' "$out" | grep -Ec '^[[:space:]]*[0-9]+[KMGT]?B[[:space:]]+/usr/')" -eq 10 ] \
  && printf '%s\n' "$out" | awk '{print $1}' | sort -Cnr && echo VERIFIED

Line count, format, every path under /usr, and that the sizes are actually in descending order. The supervisor re-ran it independently and it passed; the self-confirming-test note fired correctly, since the script under test was written in the same session. Committed and pushed to https://git.example.com/vmlab/biggest.

59.9 — the admin key that does not need to exist

Symptom. Live progress reached the chat by POSTing to /api/v1/chats/{id}/messages/{id}/event, and that endpoint refuses a key whose user neither owns the chat nor is an admin (routers/chats.py:1531). Anyone on this instance may invoke vm_agent and the chat belongs to whoever is talking, so it had to be an admin key — the ability to read every conversation and create users — living in a container in order to be usable. It was kept out of the agent itself, behind nginx in owuitools-vmgw, which bounded the damage without changing the fact that it existed.

Cause. Pushing was the wrong direction. Anything outside Open WebUI that wants to write to a user's message needs to authenticate as somebody, and the only somebody who can write to any user's message is an admin.

Change. The agent writes progress to the job volume it already shares with the api container (/vmjobs/progress/<chat-id>.json), the api serves it on a router that is include_in_schema=False and carries no operationId, and a new filter — owui-functions/vm_agent_progress.py — polls it from inside Open WebUI and emits the events itself. There, __event_emitter__ is already bound to the requesting user's own session, so nothing authenticates as anyone. The filter's only credential is the owuitools bearer token, which grants exactly the tools the model can already call.

The event location and the key are gone from vmgw; its Ollama locations stay, because Ollama's unauthenticated DELETE /api/delete still needs something in front of it.

Result. No Open WebUI credential exists anywhere in the stack. tests/vm_agent_asserts.sh now asserts that as a property rather than a configuration: vmgw holds no key and proxies no chats/.../messages path. The progress endpoint answers 200 with the token and 401 without, and does not appear in the OpenAPI document at all — 22 tools, none of them this.

Two things this cost, and one bug it caused. Progress is polled every two seconds inside Open WebUI's event loop rather than pushed. The filter must be installed by hand like the other four, so a fresh deployment has a working agent and no live view until somebody pastes it in. And removing openwebui_default from vmgw — which looked like tidying, since it no longer needed to reach Open WebUI — left it on nothing but an internal: true network, so it had no default route to Ollama at all. Its healthcheck passed and every proxied request timed out. The membership is back, with a comment saying it is for the route and not the destination.

Generalisable lesson. "Where does this credential have to live" is often really "which direction is this call going". Reversing it removed the credential entirely rather than protecting it better.

59.10 — the isolation rule that locked the operator out of their own console

Symptom. The maintenance VM's console never connected. The port listened, query-vnc reported enabled: true, auth: none, host 127.0.0.1, service 5999, and every client - and a raw socket - sat there until it timed out. A VNC server greets immediately; this one sent nothing at all.

Cause, and it is mine. IPAddressDeny=127.0.0.0/8 on the unit. systemd's filter is described plainly in the man page and I had read only half of it: "Both ingress and egress traffic is filtered by these settings. In case of ingress traffic the source IP address is checked." The source address of a connection to a loopback port is 127.0.0.1. So the rule that stops the guest reaching the host through SLIRP's 10.0.2.2 alias is the same rule that drops the operator's own inbound connection, and there is no way to express "outbound only" in that mechanism.

Confirmed both directions: with IPAddressDeny= cleared and the unit restarted, the server greets with RFB 003.008 immediately.

A second finding on the way: systemctl set-property --runtime IPAddressDeny= on a running unit changes the property and does not reattach the BPF program. The first attempt to test this hypothesis therefore appeared to disprove it. The filter is built at unit start; testing it needs a restart.

Change. The console moved from a TCP port to a unix socket, which is not IP and which the filter therefore does not see. The desktop tool bridges it to a loopback port with socat, from a process running as the operator and outside qemu's cgroup. The unit's SocketBindAllow=ipv4:tcp:5999 exception is gone with it: a maintenance boot now opens no listening TCP port at all, exactly like a session.

Result. Both properties hold at once, which was the thing in doubt: the console greets through the bridge, and IPAddressDeny is intact with all nine ranges. vm_host_asserts.sh now asserts the console is a unix socket and that the unit permits no TCP bind, so this cannot quietly revert to a design that does not work.

Generalisable lesson. A filter with no notion of direction will eventually be asked to allow one direction and deny the other, and the answer is to move the thing that needs allowing off the filtered transport entirely rather than to weaken the rule. The alternative on the table - adding IPAddressAllow=127.0.0.1/32 - would have worked for the console and reopened the guest's path to every service on the host's loopback, which is the specific hole the rule exists to close.

59.11 — what Windows 11 needs, and the denial that wears a permission error's clothes

Symptom. Anticipated rather than suffered, but three of them would each have stopped a Windows install dead, and two produce messages that point at the wrong thing entirely:

• No TPM and no Secure Boot -> "This PC can't run Windows 11". Reads as a hardware limitation of the VM rather than two missing device models.
• No virtio driver disc -> setup reports there are no drives to install to. Reads as a broken disk rather than a driver Windows has never had.

Change. The launcher now branches on the image's name - derived from the disk in maintenance mode and from the overlay's backing file in a session, so a session's randomly-named overlay still resolves to the right machine. Windows gets OVMF_CODE_4M.secboot.fd, a variable store seeded from OVMF_VARS_4M.ms.fd so Microsoft's keys are actually enrolled, -global driver=cfi.pflash01,property=secure,value=on so the guest cannot rewrite that store, an swtpm TPM on a unix socket, and virtio-win.iso as a second CD during installs only. Ubuntu gets none of it.

And one that was suffered. swtpm refused to create its socket: Could not open UnixIO socket: Permission denied, with nothing in dmesg. Not a file mode - Ubuntu's AppArmor profile for swtpm permits libvirt's paths (/run/libvirt/qemu/swtpm/*.sock, /var/lib/libvirt/swtpm/**) and essentially nothing else, and denies quietly. Fixed through <local/usr.bin.swtpm>, the profile's own extension point, so it survives a package update.

A second bug found by writing this down. The maintenance boot picked its media with ls -t "$ISOS"/*.iso | head -1 for Windows. With an Ubuntu ISO sitting in the same directory and a newer timestamp, a Windows maintenance boot would have quietly booted the Ubuntu installer. The glob is now [Ww]in*.iso and fetch.sh refuses Windows media that does not match that shape, so the two ends agree rather than one silently disagreeing.

And a third. A template unit cannot Conflicts= its own other instances, so owuivm-maint@ubuntu and owuivm-maint@windows could both come up, fight over the same console socket and split the host's memory. The prepare step now refuses the second.

Result. tests/test_vm_launch_argv.sh asserts the whole command line without booting anything: it replaces the final exec qemu-system-x86_64 with a printf, so every branch runs for real. Windows has secure-boot firmware, a TPM and a protected variable store; Ubuntu has none of them; and both have a netdev of exactly user,id=net0,ipv6=off with no guestfwd, smb=, tftp= or bootfile, no hostfwd, and no -virtfs, vfio, usb-host or hostdev.

That last set is the reason the test exists. Those -netdev options are handled inside the qemu process, so the traffic never becomes a host socket and neither the cgroup filter nor the nft table can see it. No filter can catch what no filter can observe; the only defence is that the string is fixed, and the only way to know it is still fixed is to look at it.

Not measured: no Windows machine has been installed. Everything above is the documented requirement plus an asserted command line, which is not the same as a booted installer.

59.12 — the machine was crawling, and it was not nested virtualisation

Symptom. The Ubuntu installer was "incredibly slow" to drive. The obvious suspect on this host is that it is itself a KVM guest, so the sandbox is an L2 machine and every VM exit costs more — an explanation that is true, unfixable, and in this case wrong.

Measured instead of assumed. query-kvm reported enabled: true, so it was accelerated. The qemu process sat at 260% CPU on a 28-core host with a load average under 5, so it was not CPU-starved, and CPUQuotaPerSecUSec=7s was nowhere near binding. What was pinned was memory: MemoryCurrent was exactly MemoryMax, and the cgroup's memory.events recorded max 6782 — it had hit its hard limit 6,782 times in one boot, each hit forcing direct reclaim. memory.pressure showed 38 ms of full stall accumulated while idling at a desktop.

Cause, and it is mine. MemoryMax=10G for a guest given 8 GiB. The guest's RAM is a memory-backend-memfd with share=on — required by virtiofs, and therefore set unconditionally so a machine with a project share behaves identically to one without — so the kernel accounts all 8 GiB of it as shmem. This host has no swap, so shmem cannot be reclaimed at all. That left roughly 1.5 GiB for every page of ISO and disk I/O the VM does, and a 6 GB installer ISO does not fit in 1.5 GiB: each read evicted the last, so the same media was read from disk again and again.

MemoryMax was set to bound a runaway qemu. It was never meant to bound the guest — the guest's own RAM size does that — and at 10G it was doing the second job badly while not obviously doing the first.

First change, and it was the wrong one. MemoryMax=14G on both units, so the page cache had room. It worked - reclaim went to zero and the stall time fell by a factor of ninety - and within minutes the host was short of memory and started killing background work. One VM had been handed twelve gigabytes of a 58 GB machine that is also running Ollama. Raising a limit to make room for something that should not have been inside it in the first place.

The actual change. cache=none,aio=native on every drive - the disk and both CD-ROMs - so this VM's I/O is O_DIRECT and never enters the page cache at all. What remains in the cgroup is the guest's RAM and qemu itself, which is what the limit was always meant to bound, and MemoryMax came back down to 11G. Verified O_DIRECT works on both filesystems in play: ext4 for the base images and ZFS 2.4.1 for the session overlays.

Measured after: cgroup file 3.43 GB against shmem 3.42 GB - so the page cache charged to this VM is now a rounding error rather than 2 GB - max 0, stall total=0, and the host went from 39 GiB used / 18 GiB available to 35 GiB used / 23 GiB available, with host pressure back to zero. Better than the 14G version on both counts at once.

Also -device VGA -> -device virtio-vga, still 2D: a desktop repainting through the emulated VGA MMIO path is slow on top of llvmpipe doing the rendering anyway. Explicitly not virtio-gpu-gl/virgl — QMP screendump reads the console surface and a device rendering through a GL context does not provide one, which would take the agent's entire GUI mode with it.

Result. Same machine, same media: memory.events max 6782 -> 0, full stall 37,996 µs -> 0, and the host better off than before rather than worse. screendump still returns a 1280x800 PNG on virtio-vga, verified.

Process lesson, and it is the one worth keeping. The first fix was verified against the symptom and not against the system: every number I looked at was inside the VM's cgroup, and all of them improved. The host was never measured until it started killing things. A limit raised is a limit somebody else now lacks, and "it fixed the thing I was looking at" is not the same as "it fixed the thing".

Generalisable lesson. A cgroup memory limit on a VM is not a limit on the VM. With share=on the guest's RAM lands in the same accounting bucket as the host's page cache for that VM's I/O, and on a swapless host one of those two is unreclaimable — so the limit is really "how much page cache may this VM have", and setting it just above the guest's RAM size means approximately none.

59.13 — stopping the unit was pulling the plug

Symptom. Noticed while adding an explicit Shut down button, not from a failure. systemctl stop SIGTERMs qemu, which from inside the machine is indistinguishable from yanking the power lead. For a session that is harmless — the overlay is discarded either way — but a maintenance boot is the operator editing the base image every future session runs from, and the GUI's only stop control did exactly that.

Change. ExecStop=/usr/local/libexec/owuivm-powerdown on both units. It sends system_powerdown over QMP and waits for the guest to stop; systemd then waits for qemu to exit on its own, up to TimeoutStopSec, and still SIGTERMs afterwards if the guest ignored the button. A courtesy with a deadline, not a new way to hang. It cannot fail: a stop path that errors leaves the unit in failed and blocks the next start, so every path exits 0 — verified against a bad session id and a missing socket.

Also: the GUI is now Tool Server VM Maintenance, with Start, Shut down and Console as three buttons that each mean one thing, rather than one button whose meaning depended on state.

59.14 — the freeze was not happening, and nothing said so

Symptom. A single line in the journal after a clean shutdown: owuivm-maint@ubuntu.service: Control process exited, code=exited, status=1/FAILURE. Noticed only because a restart happened to be in front of me.

Cause. owuivm-maint-prepare freeze computed how much of the disk was allocated with du -B1 --apparent-size=0. --apparent-size takes no argument, so du exits non-zero; under set -euo pipefail that killed the script at that line - which was before the chmod 0444. Every clean shutdown therefore left the base image WRITABLE. The disk was correct, the checksum was not recorded, the marker was not cleared, and the only evidence was one line saying a control process had failed.

Two compounding mistakes, both mine. The argument was wrong, and the freeze - the one safety-critical statement in the file - was placed downstream of code that could abort.

Change. The chmod now happens first, immediately after the firmware variables are copied back, and everything after it runs with set +e because it is all advisory. du -B1 without the flag, which reports allocated size, which was the question. The final log line reports the mode it actually observes (stat -c %a) rather than asserting success.

Result. tests/test_vm_freeze.sh runs the real script against a scratch directory - the roots are overridable for exactly this - and checks the resulting mode. Three cases: a disk with an OS on it freezes and loses its marker; a disk nothing was installed on freezes and KEEPS its marker, so an interrupted install can be resumed; and a disk freezes even when a later step is made to fail, which is the case that would have caught the original bug.

Generalisable lesson. set -e turns every line above the important one into something that can silently skip it. Order the safety-critical statement first and let the reporting fail if it wants to - and test the effect on the filesystem, not the presence of the line.

And a test that cried wolf. vm_host_asserts.sh flagged the image as unfrozen while the operator had it open for maintenance, which is exactly what maintenance is. It now distinguishes "unfrozen because something is broken" from "unfrozen because you are using it" - a suite that reports correct states as failures teaches people to ignore it, which is worse than not having one.

59.15 — three faults behind one symptom: "the machine is in an EFI shell"

Symptom. The Ubuntu machine came up in the UEFI firmware shell rather than the installer, with 3.45 GB already written to its disk. And pressing Shut down in the maintenance tool locked the window - it stopped repainting entirely and looked like a crash.

Fault 1: the GUI blocked its own main loop. subprocess.run("systemctl stop") was called directly from the button handler. GTK repaints from the main loop, and stopping a machine now takes as long as the guest needs to power off - up to TimeoutStopSec. So the window froze for over a minute, which is indistinguishable from a hang. Introduced by me in the same change that added the graceful shutdown: the shutdown got slower and more correct, and nothing was done about the thing waiting on it. Start and stop now run on a worker thread and report back through GLib.idle_add; a _busy flag keeps the periodic refresh from re-enabling buttons underneath an operation that has not finished.

Fault 2: a partial install was declared finished, on size. The freeze step removed the install marker once the disk had grown past half a gigabyte, reasoning that a disk with contents has an operating system on it. An install interrupted partway had written 3.45 GB. The marker went, the next boot had no installation media, and the disk was neither blank enough to start over nor complete enough to boot - hence the firmware shell, and no way back except deleting it.

Size cannot distinguish "installed" from "interrupted halfway", and neither can anything else available from outside the guest. So it no longer tries. The media stays attached, and the boot order does the work instead: the disk is bootindex=0 and the media bootindex=1, the opposite of the obvious arrangement. UEFI walks its entries in order, so a blank or half-installed disk falls through to the installer by itself while a finished install boots itself and ignores the disc still in the drive.

Fault 3: firmware variables from empty boots were being frozen into the base image. OVMF rewrites its boot order on every boot, and the freeze copied that back unconditionally - including boots that installed nothing. Several restarts in, with the display device changed from VGA to virtio-vga and the drives given cache=none, the stored entries pointed at device paths that no longer existed. The copy-back is now conditional on the disk actually having something on it.

Result. Wiped and restarted: the blank disk falls through to the media and boots the Ubuntu installer, verified by screenshot. tests/test_vm_freeze.sh gained a case asserting that a boot which installed nothing does not rewrite the frozen firmware variables, and its marker case was inverted - the marker is now supposed to survive, and the test says why.

Generalisable lesson, and it is the same one twice. Both of the silent faults were a heuristic standing in for a fact nobody had. "The disk is big, so it is installed" and "the firmware wrote variables, so they are worth keeping" are both guesses that are right most of the time and destroy the artefact when they are wrong. Where the fact is not available, arrange things so the guess is not needed - which here was one boot-order change that made an interrupted install self-healing.

59.16 — planning while it boots, and a wait tool

Symptom. Not a fault - a waste. session_open blocked until the guest agent answered, so the sequence was: start a machine, wait twenty-odd seconds doing nothing, then ask the model for a plan and wait again while it prefills and thinks. Two waits of about the same length, one after the other, at the start of every session.

Change. They now happen at once. session_open takes wait=False and returns as soon as qemu is started; the agent's first turn is a planning turn with only the planning tools offered, and the opening message says plainly that the machine is still coming up and it will be told when to start. When it has a list, the harness calls session_ready, which blocks until the guest answers - and, for a GUI session, until there is actually a graphical session, because the agent answers at multi-user well before anything is on screen. Then a message goes in saying the machine is up, with a screenshot already attached, so a desktop session does not spend its first turn asking for one.

That has a second effect worth as much as the first: by the time there is a machine to use, the model is loaded and its prompt is prefilled.

The agent cannot reach the machine early, by two independent means. The loop does not offer the guest tools while booting - verified for both interfaces - and vmd refuses all thirteen of them regardless of what was offered, because the loop and the gate could disagree and the gate should win. The gate also distinguishes "there is no machine" from "the machine is still booting": the wrong one of those sends an agent off waiting for something that is never coming.

session_ready is deliberately not a tool. An agent that could ask "is it up yet?" would ask, repeatedly, and each of those is a model turn spent on nothing.

And a wait tool. Previously GUI-only and capped at 60 seconds, which is not a package install. Now a general tool capped at 20 minutes, sleeping in two-second steps so a cancelled session does not sit in it for another quarter of an hour. The cap is not in the tool's description - that is a sentence paid for on every turn of every session to describe a case that almost never happens - and is stated in the tool's output on the one occasion it bites, along with what to do about it.

Result. tests/test_vm_boot_overlap.py: seventeen cases covering what is offered in each phase, that no guest tool leaks into the planning turn, that the cap exists in software and not in the prompt, and that the opening message does not claim a machine that is not there yet.

Not measured: the actual saving. It should be most of one boot per session, but no session has been run end to end since the change - the Ubuntu image is still being installed.

59.17 — "an explicit act by the operator" was a sentence, not a control

Symptom. The Ubuntu base image finished installing and went on reporting INSTALLING, and every boot still attached the installer ISO. The operator asked for it to be switched over. The comment in owuivm-maint-prepare and the note in 59.15 both said that detaching the media is an explicit act by the operator - and there was nothing for the operator to press. The act existed only as rm on a file in a directory their own account cannot write.

Cause. 59.15 removed a guess (disk size standing in for "the install finished") and replaced it with a decision. Removing the guess was written; giving the decision a home was not. A decision with no control is not a decision, it is a chore for whoever remembers the path.

Change. owuivm-detach@.service, a oneshot that removes <image>.install and nothing else, plus a fourth button in the maintenance tool - Installed — remove disc - that starts it:

• It is a unit, not a line in the GUI, for the reason already written at the top of owuivm-maint-prepare: the tool runs as operator, the marker belongs to owuivmadm, and a unix group only takes effect at the operator's NEXT login - which fails confusingly for exactly as long as the session that installed it lasts. Going through polkit uses the mechanism already proven for this operator: 50-owuivm.rules now covers owuivm-detach@ on the same subject.active && subject.user == "operator" test that starts a maintenance boot.
• ConditionPathExists= on the marker, so a second press is a skip and not a failure.
• The button is visible only while an install is in progress and dead while the machine is running. Marking an install finished is a claim about a machine that has been shut down and frozen; a half-installed one still needs its disc, and the disc is the only way back to the installer. That is the 59.15 fault stated as a widget state.

Result. Measured against a real marker: the unit removed it, owuivmd was refused (Access denied) while an active session of operator was allowed, and rebuilding the launcher's argv afterwards showed cdrom drives attached: 0. Six new assertions in tests/test_vm_freeze.sh cover the unit's existence, that its ExecStart removes the marker and nothing else, the condition, the polkit entry, the GUI control, and that the control is dead while the machine runs. The last two are greps against the installed tool: a service with no button is the same sentence in a different file.

Not measured: the button has not been seen on screen. The only Chrome the browser tools can reach is on another machine and cannot open this host's loopback, and rendering GTK4 headlessly here would have meant publishing a VM control panel on the LAN to photograph it. The logic is asserted, the unit is proven end to end, and the widget is one Gtk.Button appended to a box that already holds three.

59.18 — "where is the unfreeze button?"

Symptom. The operator looked at a row reading frozen and went looking for the control that unfreezes it. There wasn't one. Unfreezing happened as a side effect of pressing Start (ExecStartPre=owuivm-maint-prepare %i unfreeze) and re-freezing as a side effect of stopping (ExecStopPost=... freeze), so the most visible thing about a base image had no control of its own and the buttons that did the work were named after something else.

Cause. The two decisions were fused when there was only one reason to start a base image - changing it. There is a second reason, and it is the more common one: booting the image to see that it still comes up, exactly as the agent will get it. Fused, that was impossible to do safely; the only way to look at an image was to make it writable first.

Change. Freezing is now the operator's own verb and starting is another.

• owuivm-thaw@.service and owuivm-freeze@.service, each one chmod, each polkit-gated like owuivm-maint@ and owuivm-detach@. The maintenance tool grows a two-state button whose label is the state - Unfreeze when the image is frozen, Freeze when it is not - dead while the machine is running, because qemu holds the disk open from the moment it starts and a chmod then changes what ls -l says and nothing else.
• The disk's mode now decides what boots. Frozen, owuivm-launch creates a throwaway overlay on /Pool/vmrun backed by the untouched base - the same thing session.py builds for an agent
  • and close deletes it. Unfrozen, qemu opens the base itself. So Start on a frozen image is a rehearsal of the agent's own boot, and nothing the operator does in it can survive.
• close no longer re-freezes. It copies the firmware variables back and records a checksum when the image was unfrozen, and does nothing at all when it was frozen - not even the OVMF variable write-back, which would otherwise make a look-at-it boot change the image it was supposed not to touch.
• The safety net that the automatic re-freeze provided moves to where it cannot be forgotten: vmd refuses to open a session while a base image is writable, with a sentence saying so. That is strictly better than the chmod it replaces, which was silent on the way past and had already been seen to fail silently (59.14).

The bug this found in itself. The first version tested [[ ! -w "$DISK" ]]. That asks "can this process write it" - and root can write a 0444 file, so a maintenance boot started by root took the unfrozen branch and opened the base directly. The test caught it because it runs as root and the live boot did not, because it runs as owuivmadm. Frozen is the owner write bit and nothing else, which is what session.py, the desktop tool and the prepare script had all along; the launcher now says the same thing.

Result. Measured on the real image: thaw -> 660, a session refused with "the ubuntu base image is unfrozen, so no session can start" having created no overlay and started no qemu, freeze -> 444, checksum identical across the round trip. A full frozen boot ran from /Pool/vmrun/maint-ubuntu.qcow2 (backing file: the base) and left the base byte-identical with the overlay gone. Six new cases in tests/test_vm_launch_argv.sh assert both branches against whatever mode the host's images are really in; nine in tests/test_vm_freeze.sh cover the frozen boot, the toggle verbs and the units behind them; == 5a in tests/vm_host_asserts.sh unfreezes the real image and asserts the refusal, with the restore trap extended so a failed run cannot leave an image writable.

One-line consequence worth knowing: /Pool needed a traverse ACL for owuivmadm (setfacl -m u:owuivmadm:--x), because the throwaway disk of a frozen boot lives on the same 40 GB-quota dataset as a session's overlay rather than on the root filesystem.

59.19 — the share the image had to know about

Symptom. "How do I set up the shared workspace on the VMs?" - asked while the hand-installed Ubuntu image sat unfrozen with qemu-guest-agent still not answering.

Cause. The project share is one virtiofs tag, project, and the guest-side half of it lived in images/ubuntu/user-data as an fstab line written by the autoinstall's late-commands. The image on this host was installed interactively, so it has no such line - and so do any future ones built the same way.

The failure that produces is the worst available shape. The machine boots, the guest agent answers, run() works, write_file() writes to /mnt/project - an ordinary empty directory on the guest's own root - and the host then commits a project folder in which nothing has changed. The Gitea link is handed over and the repository is empty. Nothing in the guest, in vmd or in Gitea reports an error.

Change. The session mounts it, every time, over the channel it already has:

mkdir -p /mnt/project
mountpoint -q /mnt/project || mount -t virtiofs project /mnt/project
mountpoint -q /mnt/project

Idempotent, so an image that does carry the fstab line is one mountpoint check; checked rather than assumed, so a mount that does not take raises with the guest's own words and the sentence "nothing written in this session would be kept", before the agent is given the machine. A hand-built image now needs nothing for the share at all.

Also added, because "install it by hand" turned out to be a hundred lines of late-commands that an interactive install silently skips: vm/images/ubuntu/by-hand.sh, the same list written longhand and idempotent, and owuivm-guest-run, which pushes a local script into the running maintenance guest and runs it. That leaves exactly one thing that has to be typed into a VNC console - apt install qemu-guest-agent, the channel everything else travels over.

Result. tests/test_vm_share_mount.py, nine cases: no share does nothing, Windows does nothing (WinFsp mounts it inside the image and there is no command to send from here), the mount is created, is idempotent, is verified, and a failure stops the session with what the guest said in it.

What running it on the real image found. by-hand.sh went in over the guest channel in 45 s (8.5 s on a second run, which is what idempotent is for), and two of the things it was copying from the autoinstall turned out to be wrong for 26.04:

• WaylandEnable=false has nothing to fall back to. xserver-xorg-core is not installed and /usr/share/xsessions is empty, so that line does not choose X - it leaves GDM with no session to log into, and an autologin into nothing is indistinguishable from a desktop that failed to start. Removed from by-hand.sh and from user-data, or the next unattended build would produce the same broken image. The xorg.conf.d modeline went with it: the reason for pinning the mode was that the agent clicks absolute coordinates, and the mode is pinned one layer further down anyway, by a qemu argv that is a constant. Measured after a reboot: 1280x800, from the virtio-vga device, with no display-server configuration at all.
• Disabling snapd takes Firefox with it. The autoinstall disables snapd.service outright as a source of background change. On a desktop image the browser is a snap, so a GUI agent that wanted to read documentation would find nothing to read it with. snap refresh --hold says the thing that was actually meant - stop changing underneath me - and leaves the snaps that are installed working.

Measured on the finished image: guest agent 10.2.1 answering 11.6 s after a reboot, a GNOME Wayland session active and autologged in as uid 1000 five seconds later, gnome-shell running, no first-run wizard, and a QMP screendump of the desktop at 1280x800 - so the agent's eyes work on this image and GUI mode has a machine to drive. DNS resolves through systemd-resolved to SLIRP's forwarder to the host's bind-mounted resolv.conf (curl https://example.com -> 200), which is why the static netplan the autoinstall writes was not applied here: both netplan files on this image are NetworkManager-rendered, and adding a networkd-rendered one to fix something that already works is how a machine ends up with no network at all.

Not measured: the virtiofs mount itself. A maintenance boot is given no share - that is the one thing only a session does - so the first session on this image is what exercises it.

59.20 — the trap I wrote the warning about, in the file that ignored it

Symptom. "I clicked Install from ISO on the Windows instance and it errored out." The unit never started - journalctl -u owuivm-maint@windows.service had no entries at all - and no windows.qcow2 was created.

Cause. The desktop tool created the blank disk, copied the firmware variables and wrote the install marker itself, in /var/lib/owuivm/base, which is 0775 owuivmadm. operator is a member of owuivmadm in /etc/group - id operator lists it - and no process in the running desktop session has that group, because the session predates the usermod that added it. A unix group only reaches a process at its next login. Measured: /proc/<gui>/status Groups: has 984 (docker) and not 978 (owuivmadm).

That is the exact trap written at the top of owuivm-maint-prepare - "a unix group takes effect at the NEXT login, so the tool fails in a confusing way for exactly as long as the session that installed it lasts" - in the one file that still depended on it. The warning was written while moving this work into the unit, and this code path was left behind.

The second bug underneath it. The tool copied OVMF_VARS_4M.fd for both images. Windows needs OVMF_VARS_4M.ms.fd, the store with Microsoft's keys already enrolled, or Secure Boot is on with nothing it trusts and setup refuses the machine. Had the permission problem not stopped it, the install would have failed later and looked like a hardware requirement. The unit's ExecStartPre has had this right the whole time.

Change. prepare_install() is gone from the tool, and with it the ISO dropdown: the unit picks the newest ISO matching the image's own pattern, so a control that looked like a choice was describing a decision made elsewhere. The row now reports it - "not installed — will install from Win11_25H2_English_x64_v2.iso" - and the button starts the unit. DISK_SIZE and OVMF_VARS went too; a copy of a value that lives in owuivm-maint-prepare is a copy that will disagree with it one day.

Result. Fifteen new assertions in tests/test_vm_freeze.sh: three that the tool contains no qemu-img, no shutil.copyfile and no firmware path of its own, and twelve driving the unit's own open verb against a scratch directory for both images - a blank disk that is writable because an installer has to write to it, an install marker naming the media, and the firmware store each image actually needs, asserted by cmp against both candidates so getting them the wrong way round fails.

59.21 — the Windows machine was slow for a different reason than the Ubuntu one

Symptom. "The Windows VM is kinda slow - did you only fix the slowness on the Ubuntu machine?"

Measured first, because the obvious answer was wrong. The Ubuntu fix (59.12) was host-side - cache=none,aio=native on every drive plus MemoryMax=11G - and it is in owuivm-launch and the unit files, so both machines have had it all along. The cgroup of the running Windows installer said so: memory.events max 0 (Ubuntu's was 6,782 before the fix), file and shmem both 7.7 GiB - that is the guest's own RAM and nothing else - and throttled_usec 347865, a third of a second of CPU throttling in three hours. Neither short of memory nor short of CPU.

What it was short of was information. qemu sat at 347% CPU driving an idle installer.

Cause. Windows assumes it may be running under Hyper-V and looks for the paravirtual interfaces that would say so. -cpu host alone offers none, so it behaves like a kernel that has been lied to about its hardware: it spins on locks the hypervisor could have yielded, takes a VM exit per timer tick and per inter-processor interrupt, and drives its APIC through a trap on every access. The work is real; it is simply wasted. Linux needs none of this because it has virtio, so the problem could not appear on the machine that was profiled first.

Change. The Windows machine - and only it - gets the enlightenment set: hv-relaxed,hv-vapic,hv-avic,hv-time,hv-spinlocks=0x1fff,hv-vpindex,hv-runtime,hv-synic,hv-stimer,hv-stimer-direct,hv-reset,hv-frequencies,hv-tlbflush,hv-ipi. Listed explicitly rather than hv-passthrough, which hands the guest whatever the host kernel happens to support today: these images are frozen, and one should behave the same after a host upgrade as before it. Every flag was verified accepted by this host's KVM before being added.

And a clock, while the argv was open. Windows reads the RTC as local time. The host is America/New_York, so base=utc leaves the guest four hours in the past - which fails TLS certificate validation and therefore breaks pip, winget and anything else that fetches, while presenting as a network fault. base=localtime for Windows, utc for Ubuntu.

Result. Eight assertions in tests/test_vm_launch_argv.sh, both ways round: Windows has the enlightenments and does not have hv-passthrough, Ubuntu is plain -cpu host, and each gets the clock it expects. Not measured: the improvement itself. The flags apply at the next start of the machine and the installer was mid-run; the number to compare against is the 347% recorded above.

"Is it just nested virtualisation?" - no, and here is the measurement. The question came up again once Windows was installing properly, and it is worth recording the answer because it is the first thing anyone blames on this host:

• steal = 0. L0 gives this machine every cycle it asks for. Oversubscription would show here.
• kvm_amd npt = Y. Nested paging is on, so guest memory is hardware-walked rather than shadowed - that is the expensive half of nesting and it is handled.
• 127,000 VM exits/second (635,079 in five seconds across 8 vCPUs; 65% kvm_msr, 11% kvm_mmio, kvm_nested_vmexit zero). Exits are where nesting genuinely costs extra, and at that rate they are single-digit percent of the CPU burn, not the bulk of it.

What was actually holding it back was our own limit. CPUQuota=700% on a machine with 8 vCPUs: measured at 697% of 700, with 1,792 throttle events and 103 seconds of CPU withheld. One of its eight vCPUs could never run. Raised to 800% - which is exactly what 8 vCPUs can consume

• live with systemctl set-property --runtime, and throttling went to zero in the next 30 seconds. Persisted for maintenance boots only: a session keeps 700% because it runs while the main model may be answering someone, on eight threads Ollama can also use, whereas a maintenance boot is one person doing one thing and should have the machine it was given.

The honest ranking of what made this VM slow, since three separate things were blamed on each other: missing Hyper-V enlightenments (76 minutes to not-finish, versus 27 minutes to OOBE), the overlay sitting on the SMR array after an accidental frozen boot, our own CPU quota, Windows first-boot servicing (which ends on its own), CPU-side rendering (permanent, minor), and nesting (real, minor, and not removable from here).

Two things that are slow for reasons no argv can fix, worth writing down before they are reported as bugs: until virtio-win is installed, Windows drives emulated hardware - most visibly the display, where virtio-vga with no driver is a plain VGA framebuffer and every redraw is a full repaint - and Defender plus first-boot servicing make the first hour of any fresh Windows install heavy. cache=none also costs an install real time, and it stays: it is what stopped the cgroup thrashing that made the Ubuntu install crawl.

59.22 — the test that destroyed the thing it was testing

Symptom. A Windows 11 install sat on "Getting ready" for fifteen minutes. It was not stuck: it was writing to a disk that no longer had a name, on the slowest storage in the house, and would have lost everything the moment qemu exited.

Cause - mine, and worth writing down in full. tests/test_vm_launch_argv.sh drove the real launcher against the real directories, and 59.18 added a loop to it that flips each base image's mode to exercise both boot paths. Run while an operator was installing Windows, it:

1. chmod'd the live base image to 0444 and left it there, so the next Start booted frozen - which, since 59.18, means a throwaway overlay on /Pool/vmrun. The install went there.
2. rm -f'd that overlay while qemu had it open. The data survived only as an unlinked inode. ln and linkat(AT_SYMLINK_FOLLOW) both refuse to give it a name again - only O_TMPFILE files can be named after the fact - so it was going to die with the process.
3. Killed the machine's swtpm and overwrote its OVMF variables, because maintenance boots use a constant session id (000...0) and the test used the same one, so its pkill -f "swtpm socket .*$session" matched the operator's. Journal: tpm-emulator: Could not stop TPM: Input/output error, then status=1/FAILURE.

And the slowness that prompted the question was a fourth consequence: Pool is four spinning SATA drives in raidz1 - two of them SMR - while base images live on /, which is non-rotational. The frozen boot had quietly moved a Windows install, which is small random writes with cache=none, from SSD onto SMR.

This is the second time a test in this suite has damaged a live machine. The first (vm_host_asserts.sh, 59.x) read is-active after starting a unit, saw "active" from the operator's own boot, and stopped it. Both have the same shape: a test that shares state with production and assumes nothing else is using it.

Change.

• The test's copy of the launcher now has its paths rewritten as well as its exec line - /var/lib/owuivm/base, /Pool/vmrun and /run/owuivm all become directories under its own mktemp -d. The path rules are still exercised, because they are rewritten consistently; every path they resolve to is disposable.
• A distinct session id (fff...f), and the swtpm pkill matches on the scratch directory rather than on an id a real boot also uses.
• It refuses to run at all while any owuivm unit is active, and says why. Verified: exit 2 with a Windows machine up.
• The maintenance tool now warns before the same trap can be walked into by hand: a frozen image with installer media attached reads "Starting it now would install into a throwaway copy and lose it. Press Unfreeze first."

Result. All twenty-two cases pass against the scratch tree, and a run leaves /var/lib/owuivm/base and /Pool/vmrun byte-identical. The cost was real and is not recovered: the operator's Windows install was destroyed and restarted from nothing.

The rule this earns: a test that needs the real launcher gets a copy of the launcher, and a copy of everywhere it points. If it cannot be made hermetic, it refuses to run while the system it tests is in use.

And it was not only that test. The very next attempt to install Windows failed in ExecStartPre with cp: cannot remove /run/owuivm/000.../OVMF_VARS.fd: Permission denied - because tests/test_vm_freeze.sh, run as root, had used the same real run directory and left it root:root. A maintenance boot's session id is a constant, so that path is a constant too, and two of the three tests that touch this subsystem had independently reached for it. owuivm-maint-prepare now takes OWUIVM_RUN_DIR alongside the base, ISO and overlay overrides it already had, the freeze test uses its own, and open checks the directory is writable and names the owner rather than letting cp report it one line later. Verified: both suites now leave /run/owuivm containing only the agent socket directory.

59.23 — six calls to read a web page

Symptom. Looking anything up in GUI mode cost the agent a screenshot to find the browser, a click to focus it, a click in the address bar, a type, a key for Return, and a screenshot to see the result. Six round trips through a 27B model with thinking on, five of them spent operating a text field, and every one of them a full prefill.

Change. Three tools, two of which never touch the machine at all.

• web_search and read_page run on the host, in vmd, against the same /tools/web_search and /tools/fetch_page the main model calls - same SearXNG, same engines, same result formatter, same cache, same readability extraction and SSRF guard. Not a second search client: a copy would be the one nobody remembers to fix. vmd already holds the Gitea token, so a credentialled call belongs on that side of the socket.
• open_page puts a URL on the screen in one call. GUI mode only - in CLI mode there is no desktop, and a tool that cannot work is a turn the model spends discovering that.

Why the agent cannot search for itself. SearXNG sits inside netguard's network namespace, on loopback, behind a default-deny filter. The guest cannot reach it - that is the sandbox working - and neither can the agent container, whose allowlist is vmgw alone. Measured while wiring this up: the host cannot reach 172.22.0.11:8080 either (it times out), which is why this goes through the gateway at the pinned 172.22.0.10:8000 rather than to SearXNG directly.

The URL is the only thing that crosses into the machine, and it is validated against ^https?:// with RFC 3986's character set and then passed to the guest as an environment variable. qemu-ga takes the environment as a list, so there is no quoting to get wrong and no way for a URL to close the string it is in. A first version of the test demanded that https://example.com/$(id) be rejected; that was the test being wrong - $, ( and ) are sub-delims and every Wikipedia disambiguation link contains parentheses. Rejecting them would have broken ordinary use to defend against an injection that cannot happen. The case now asserts the URL is accepted and that neither script has a slot to format it into.

Two guest-side details that are not obvious.

• On Ubuntu the guest agent is root and the desktop belongs to uid 1000, so the launch crosses into that session explicitly - runuser with XDG_RUNTIME_DIR, the session bus, and both WAYLAND_DISPLAY and DISPLAY because the image may be either - and setsids, so the tool call returns instead of waiting for a browser to be closed.
• On Windows qemu-ga is LocalSystem, in session 0. A GUI process started from there does not fail, it succeeds invisibly, on a desktop nobody is looking at - so the agent would screenshot an unchanged screen and conclude the page did not load. Until the in-guest broker exists, the launch goes through a scheduled task marked /it, which the task service starts inside the logged-on session; it is created, run and deleted in one call.

Flags worth keeping: --no-first-run --no-default-browser-check (every session is a first run, because the disk is restored from the same frozen base), --password-store=basic (otherwise a gnome-keyring unlock dialog opens over the page on a fresh profile), and --start-maximized (a known window size, so clicks land where the screenshot says).

Result. 41 cases in tests/test_vm_web_tools.py. Measured end to end from vmd: a search for "qemu-guest-agent guest-exec windows localsystem" returned ranked results in one call, and read_page of the QEMU guest-agent docs returned extracted markdown.

Not measured: open_page against a live guest. Only one machine runs at a time and the Windows install had the slot; the Ubuntu image is frozen and waiting. The launch script's shape is asserted, its behaviour is not.

59.24 — first real testing, and the five things it found

Both images finished (Ubuntu 26.04.1, Windows 11 25H2 activated, guest agents, browsers, python), so the whole system was driven end to end for the first time. It works. It also had five defects that only a real session could have found, every one of them in the seam between "the software reports success" and "the thing actually happened".

1. The machine was handed over showing a boot splash. _wait_for_desktop waited for a seat session, and gdm's greeter is a seat session - so it passed before the user was logged in. Fixed, and it still handed over a splash: measured, gnome-shell running, plymouth gone, graphical.target active, and the framebuffer on the Ubuntu splash for over three minutes.

The display shows the foreground virtual terminal, and the session was not on it: fgconsole said 1, the session's TTY was tty2, loginctl agreed Active=no. Plymouth had tty1 and nothing switched away. chvt 2 put the desktop on screen in under a second. That is also why it looked intermittent - sometimes gdm switches, sometimes it does not - so the probe no longer waits for the switch, it performs it. Measured after: two clean boots, desktop on screen both times, ready at 59.0s and 60.7s. The settle after the switch is 3s, not 1s, because mutter repaints the whole screen and a one-second settle handed over a 3.4 KB near-black frame.

2. open_page reported success for a browser that never started - twice, two different ways.

• DISPLAY=:0 made Chromium choose X11 on a Wayland session: "Missing X server or $DISPLAY", into /dev/null.
• Reading the display variables out of gnome-shell's /proc/environ was worse, because it looks correct: gnome-shell is the compositor, so it never had WAYLAND_DISPLAY - it creates that socket rather than consuming one - and the launch fell back to X11 again.
• The check that should have caught both was pgrep -u user -f chromium, which matched the launcher's own runuser ... chromium ... command line. The same self-match that a pkill -f walked into earlier in this project.

Now: systemd-run --user, which starts it in the user's own systemd instance - where a desktop session exports its environment, measured on this image as WAYLAND_DISPLAY=wayland-0, DISPLAY=:0 and mutter's Xwayland XAUTHORITY. The unit is named, so "did it start" is read back from systemctl --user is-active, which cannot be fooled by a command line. A framebuffer diff was tried in between and discarded: the clock in the top bar changes it every sixty seconds.

3. web_search worked from a shell and timed out from the daemon. owuivmd.service is IPAddressDeny=any with two allows for Gitea, and the tool gateway was not one of them - so the cgroup dropped it. A shell is not in the unit's cgroup, which is exactly why testing from one proved nothing. IPAddressAllow=172.22.0.0/24 added.

4. Work could be committed and never published. _publish pushed only if STATE.commits - whether this session committed. A session that is cancelled, times out, or loses its daemon leaves a commit that nothing ever pushes. Measured: primes.py was committed locally and vmlab/vmtest did not exist in Gitea. Now the question is has_commits() - is there anything to publish at all - which is the question that was meant.

5. The agent spent turns discovering what was in the machine. One guest command at handover now reports the OS, python, browsers, the desktop user and the project folder's free space, as tool output rather than prompt text, for the reason this stack has now proved three times.

Result - the whole path, measured.

• CLI session, 12 steps, 156s: wrote a word-frequency script, ran it, re-counted independently with tr/sort/uniq, committed, pushed. Supervisor re-ran the verification: passed.
• GUI session, 25 steps, 189s: searched the web, opened kernel.org with open_page, screenshotted it, wrote what it found, committed.
• Live progress: 4 screenshots published to the filter's file during one GUI run.
• Through Open WebUI, in a browser: the status line went "Booting a ubuntu machine - planning while it starts" -> "1/4 · Run the script and capture the full output" -> "The machine has been destroyed", and the main model answered with a real table of the ten largest files under /usr/share (CJK font collections, as it happens) and the commit that holds them. One tool call, 16.2s to re-prefill 13,692 tokens.
• The ask-first rule fired unprompted on the way: a task saying only "python" was refused with "this task mentions something that runs on both machines, and you chose ubuntu".

Not measured: anything on the Windows machine. run there still goes to /bin/bash through guest.shell, which is Linux-only, and the project share needs WinFsp before /mnt/project means anything - both are Phase 8 work that the finished image now makes possible.

59.25 — a view worth watching, and a session record that outlived its machine

The view. The live channel showed one status line. The operator asked for what they would actually watch: the task list above, and under it the thing the session has. A CLI session gets a terminal locked to twenty lines - the commands it ran and what came back, padded to a constant height so the message does not jump as output arrives - and a GUI session keeps the screenshot, because a picture of a desktop beside a transcript of commands that are not on it is two views of nothing. events.py carries tasks, terminal and interface; the filter renders whichever applies. Em dashes are out of everything the operator reads, here and in the rendered task table.

The terminal is unwrapped on the way through: run returns exit code: 0 / stdout: / ..., which is the shape the model needs, so the view strips the framing and shows output as output with [exit N] only when it is news. Measured, live:

$ ls -la /definitely/not/a/real/path
ls: cannot access '/definitely/not/a/real/path': No such file or directory
[exit 2]
$ bash /mnt/project/vardirs.sh
2.2G	/var/lib/
255M	/var/cache/

The defect the testing found. Rebuilding the agent container mid-session killed the loop that would have called session_close. The unit was reaped and systemctl is-active said inactive - and vmd still held the session record, so every later session was refused with "a session is already running" until the daemon was restarted. One interrupted session would have taken the subsystem out until somebody noticed.

v_session_open now asks whether the MACHINE exists rather than whether a record does: a record whose machine is gone is destroyed, cleared, and logged - `vmd: cleared stale session 70705573...

• its machine was already gone- and the caller gets their session. Proven by stoppingowuivm.service` under a live session and opening another without restarting vmd.

Needs doing by hand: the Open WebUI filter is installed by pasting its source, so owui-functions/vm_agent_progress.py has to be pasted in again for the new view to appear.

59.26 — Windows was never wired up, and a live session found out the hard way

Symptom. The operator started a Windows session and watched the agent reboot the machine and sleep for minutes at a time. Every command came back guest-exec: Failed to execute child process (No such file or directory).

Cause. Guest.shell() was exec(["/bin/bash", "-lc", script]). Unconditionally. The module was written as the Linux half with the Windows half left for the broker, and nothing in between ever said so - so a Windows session came up healthy, answered its guest agent, and failed every single command. To the agent that does not read as "wrong shell", it reads as a machine that has stopped answering: so it rebooted it, waited 90 seconds, tried again, waited 120, rebooted again, waited 180. The repeated-failure guard fired, correctly, at a machine that was never going to work.

Change. shell() takes the machine it is on: PowerShell with -NoProfile -NonInteractive for Windows, bash for Linux. Session.guest() passes os_name. And with that one line working, everything else that had only ever run against Linux surfaced at once:

• The project path. _guest_path returned /mnt/project/... for both. Windows gets Z:\..., because virtiofs is a drive letter there.
• The desktop probe. loginctl, fgconsole and chvt mean nothing on Windows; it waits for explorer.exe instead, which is what "somebody is logged in and the shell has drawn" looks like.
• check_internet was curl and getent; Windows gets Invoke-WebRequest and Resolve-DnsName.
• Machine facts had no Windows branch at all. The first attempt at one nested double quotes through Python into PowerShell and returned nothing, silently - the one outcome a fact-gathering probe must not have. Rewritten as a script in single quotes.

And then the numbers. The facts probe still came back empty on a cold boot, and timing each part separately showed why: the first PowerShell in a freshly booted Windows guest takes over ninety seconds - Defender inspecting it, .NET warming - while every one after it takes five to nine. So the shell is now warmed during the boot window, where the agent is writing its task list and nobody is waiting. Measured after: cold Windows session ready in 115s with complete facts, against 32s for Ubuntu CLI and 60s for Ubuntu GUI.

What the facts now say, which is the point:

python: not on PATH for this account
running as: SYSTEM (the guest agent is LocalSystem, not the desktop user)
project folder: NOT MOUNTED - nothing you write will be kept. Say so rather than working around it.

Two things the Windows image still needs, both measured from inside it. VirtioFsSvc is installed and Stopped; starting it fails with "A timeout was reached (30000 milliseconds) while waiting for the VirtIO-FS Service service to connect", and Test-Path 'C:\Program Files (x86)\WinFsp' is False - the viofs kernel driver is there, the user-mode half it needs is not. And Python is installed for a user rather than for the machine, so the account the guest agent runs as cannot see it. Until both are fixed a Windows session can run PowerShell and keep nothing.

Still not true on Windows: the guest agent is LocalSystem in session 0, so this is not the "normal elevated PowerShell" the design asked for. Scripts and their output are exactly right; anything that depends on being the logged-in user is not. The in-guest broker is what closes that, and it is still unwritten.

59.27 — the live view, as the person watching asked for it

Three corrections from someone actually watching a session: the task list and the terminal should be fully visible rather than something to scroll through, a long line should be cut rather than put a horizontal scrollbar under the box, and the code block should not appear in GUI mode at all. So the embed is sized to its content with no height of its own, every terminal line is its own element that clips at the edge, the task list wraps instead of scrolling, and the twenty rows are always present so the box does not change size as output arrives. Em dashes are out of everything the operator reads.

59.28 — the Windows image finished, and then saved so it need not boot

Two pieces of work, in this order because the second depends on the first: a saved machine is only valid against the exact image it was taken from, so the image had to be finished first.

The image. WinFsp was missing, which is why VirtioFsSvc was installed and Stopped and Z: did not exist - the viofs kernel driver was there, the user-mode half it talks to was not. Installed 2.1 with INSTALLLEVEL=1000 (the default level takes the driver only and the service still fails to connect), service set to Automatic. Python was installed for a user, invisible to the account the guest agent runs as; reinstalled with InstallAllUsers=1 PrependPath=1 as 3.14.7, matching Ubuntu's 3.14.4 line so the same script has one fewer reason to behave differently on the two machines. The timezone was Pacific while the host RTC is base=localtime on an Eastern host, so the guest clock was three hours out before anything resumed anything; set to Eastern, with w32time's phase-correction limits removed so a large correction is allowed at all, and Windows Update's automatic install and reboot turned off. Verified from a real session: Z:\ mounts, a file written in the guest appears on the array, python answers, the clock is within a second.

The saved machine. vmctl snapshot --os windows boots the image once, lets it settle, and writes its memory and its disk. A session then branches from that instead of booting:

	cold	warm
Windows session ready	91 - 115 s	30 s
first PowerShell in the guest	90 s+	already warm, in the saved memory
saved state	-	3.92 GB, written in 14.6 s
saved disk	-	163 MB (the boot's own writes)

Four things that had to be got right, each measured rather than assumed.

1. The share cannot be in a saved state. migrate refuses outright: "Migration disabled: vhost-user backend lacks VHOST_USER_PROTOCOL_F_LOG_SHMFD." So the snapshot is taken with no share, and the share is hot-plugged after the resume. q35's root complex is not hot-pluggable, so the argv now always carries a pcie-root-port - and the share goes into it on both paths, at the same address, because a drive letter that moved between sessions would be worse than a slow boot.
2. exec: cannot be the transport. It fails with "Unable to write to command: Broken pipe": the unit's syscall filter stops qemu spawning the helper, which is the sandbox working. file: needs nobody - and then a second problem, that qemu creates its files 0600 as the sandbox user, so vmd cannot read what it just asked for. Copying four gigabytes was one answer and letting qemu write into the saved-machine directory was another; both are worse than passing a descriptor vmd opened itself, which is what it does.
3. A saved state is only valid against one disk, down to the page cache. Resume it against a changed image and nothing reports an error - the guest simply runs with a filesystem in memory that disagrees with the one on disk. The freeze already records a checksum, so the manifest records which one the state came from and warm_set() compares them. Verified by falsifying the manifest: the saved machine is refused and the session boots cold, 96.6 s.
4. It must never be load-bearing. A saved machine that will not load becomes a cold boot, not a failed session. Verified by truncating the state to 200 MB: the resume failed, vmd booted cold in 109 s and told the caller "the saved machine could not be resumed, so this one was booted from cold. Nothing about the machine differs."

The clock, which is the reason the operator asked for any of this. A resumed machine believes it is whenever it was saved - measured at 26 seconds behind on the first warm session, and it would be days for a snapshot left over a weekend. Windows minds most: a wrong clock fails TLS, so every download breaks in a way that reads as a network fault. guest-set-time runs on every session, warm or cold, before anything else touches the guest - and it is read back and checked rather than assumed, which is how the first version came to be 26 seconds out while reporting nothing: the share service had just restarted, the agent was busy, the call failed and the error was swallowed. Now: clock: set, within 0.0s of the host, measured skew -0.8 s.

And two more Linux-shaped assumptions the first real Windows session found. The supervisor's verification prelude was cd ... 2>/dev/null || cd /; for both machines, and PowerShell has no || - so a session whose work was entirely correct was reported to the user as a claim, because the re-run failed on a parse error in vmd's own wrapper. And the agent lost two turns wrapping commands in powershell.exe when it was already in PowerShell, and using && in 5.1. The first is fixed; the second is now in the machine facts, where this stack has three times proved a model reads it as fact. Measured after both: 9 steps, 129 s, four commands, all exited 0, verification passed on the supervisor's re-run.

Not done: Ubuntu has no saved machine. It boots in 32 s and the agent's planning turn hides most of that, so the saving would be a few seconds of somebody's attention; the machinery is the same verb (vmctl snapshot --os ubuntu) if that changes.

59.29 — six real questions, and the three that never reached the machine

Symptom. Everything in 59.17-59.28 was proved a verb at a time, by me, knowing what I wanted to happen. The operator asked for the other test: six prompts written as a real person would write them, with no hand-holding, put into the chat and left alone.

Result, the short version: the subsystem passed and the routing did not. Three of the six used the machine and all three worked. Three did not use it, and two of those should have.

#	Asked for	What happened
1	a Python renamer that breaks on odd filenames, OS unstated	chose Ubuntu itself, built 14 adversarial filenames with a deliberate mtime collision, 7 of 8 tasks done and 1 blocked with a real finding - the script is not idempotent, a second run re-stamps every file. Root cause named. Committed and pushed.
2	a PowerShell service audit, "make sure it genuinely works"	Windows warm start, 6/6, supervisor re-ran the script itself and it passed on real service data. The answer flagged that Automatic (Delayed Start) needs -like, not -eq - a subtlety that only running it revealed.
3	"show me what the Ubuntu desktop looks like out of the box"	never touched the machine. Three web_search, three fetch_page, and the sentence "I can't drop a live screenshot into this chat" - written while holding a tool that boots that exact desktop and photographs it.
4	"does Restart=on-failure restart a clean exit? I want what it actually does, not what the docs imply"	read the man page. Correct answer, wrong method, and the prompt had said so in advance.
5	"can the VM reach my NAS at 192.168.1.50?"	zero tool calls. Reasoned from its prompt that the machine has no route to the LAN and declined to spend a session proving a guaranteed timeout. Right call - asserted rather than demonstrated, which here is the cheaper of the two.
6	a Flask stand-in service, "I want to know it actually starts"	vm_agent refused the ambiguous OS - "CANNOT DO THAT YET: this task mentions something that runs on both machines" - the model asked, I answered Ubuntu, it called again. 7/7, pip install, background server, real curls, determinism re-checked, server killed, committed. The ask-first rule exactly as designed.

Cause. Not a fault in the subsystem - every part of it that was reached worked, including the parts hardest to trust: blocked-with-reason, the supervisor's own re-run, the ask-first refusal, three new Gitea repos, and a clean host afterwards (no qemu, no overlays, both images frozen).

The fault was in what the model had been told the tool is for, and it was my wording. The docstring said, in as many words, do not use this to find out what software is, how it is configured, or what it looks like. That sentence is what lost test 3: "show me the desktop" is a "what it looks like" question, and I had labelled it out of scope for the one tool that could answer it. Test 4 failed for the neighbouring reason - the guidance keyed on the user asking for proof, and "I want to know what it actually does, not what the docs imply" is a request for proof that does not use any of the words I had listed.

Change. The line is redrawn where it actually falls - not what kind of thing is being asked about, but whether a document can answer it:

• USE IT when the answer has to be true of a real machine rather than of the documentation - and explicitly: if an honest answer would have to begin "the documentation implies" or "it should", that is the sentence you cannot write. Run it.
• DO NOT USE IT when a document already answers and nobody asked for proof.
• "what it looks like" moves out of the refusal and into gui, which is where it belonged: gui is the slowest thing available and is for when the answer has to be seen as it appears on screen, or the work can only be done by clicking. "Show me what this looks like" is what it is for." Plus one line aimed squarely at test 3's failure: do not tell someone you cannot show them a screen without first asking whether this is the tool that can.

Result. Test 4's prompt re-run verbatim against the new docstring: the model called vm_agent and wrote itself an empirical task - including a guard that stops the test if systemd is not PID 1, because the result would otherwise be meaningless. The same wording is in prompts/system-prompt.md, which is not deployed: the live system prompt is baked into the Ollama model, so it needs a rebuild to take effect. The docstring is served live and is doing the work on its own.

Two smaller things the round recorded. The model writes /mnt/project into Windows task text and the agent silently corrects it - now covered by "write the job as instructions, not paths". And the live task list and terminal are cleared when a session ends, so they exist only while running; the final task table survives in the tool result, the terminal does not.

59.30 — asked before the machine starts, and a box that was never the size it looked

Symptom. Two unrelated complaints from the same session, both about the same minute of the user's time. A vm_agent call takes minutes and destroys its machine afterwards, so a job sent with the wrong wording costs the whole session - and there was no point at which the person who asked could see what was actually being sent. And the live view, rebuilt in 59.27 explicitly so that nothing would need scrolling, still had a vertical scrollbar, in one box rather than the two that were asked for.

Cause, first half. Nothing structural - the feature did not exist.

Cause, second half, and this is the interesting one. Two separate bugs, neither of which anything could see from the outside.

A bad edit had left a second copy of _embed in the class. Python keeps the last definition of a method, so the file parsed, imported and ran perfectly while silently using the old single-box version; the new two-box one sat above it as unreachable code. Nothing failed and nothing warned. The only symptom was the wrong thing on the screen, which is exactly the symptom that gets blamed on the CSS.

Underneath it, the height. 59.27 says the embed is "sized to its content with no height of its own", and that was an assumption, not a measurement. Read at source, Open WebUI sizes one of these frames two ways: on load it reads contentDocument.body.scrollHeight + 20, and it listens for postMessage({type: "iframe:height"}). The first cannot work here. The frame is sandboxed without allow-same-origin - the embeds renderer passes iframeSandboxAllowSameOrigin ?? false - so contentDocument throws a cross-origin error, and the read sits inside a bare catch {}. The height is then never set at all and the frame keeps its stylesheet default of 150px. Measured in the browser: a seven-line task list needs 138px and a twenty-line terminal 364px, and both sat at 150 with a scrollbar.

Change. The tool asks first, and it asks from inside itself. _ask_before_starting() writes the job to a file, the filter running inside Open WebUI picks it up and puts one dialog to the person - empty and OK runs it, typing what to change hands their words back to the model, Cancel starts nothing. It is done in the tool rather than in the prompt for the reason this repo has proved three times: the model is simply blocked in its call and cannot skip it, ask badly, or report the answer unfaithfully. A dialog nobody reads within 20 s is not a veto - the job runs, because a confirmation that nobody can see must not be able to stop work.

For the view: the duplicate method is gone, and the frame now measures itself and posts its own height, which is the route that needs no same-origin and which allowScripts (hard-coded on for embeds) permits. It measures the wrapper element, not the document - a third trap, found by measuring: html{overflow:hidden} makes documentElement.scrollHeight report the frame's current height whenever the content is shorter, so a box measured that way grows and never shrinks again. A ResizeObserver re-reports, which is what catches a screenshot that is still decoding when the rest of the box is ready, and a zero is never sent because a frame in a background tab has no layout and measures 0.

What the dialog shows, and why it is not the job. The job the model writes is an instruction sheet for another machine - numbered steps, exact commands, the wording that stops an agent improvising. Real ones measured 1,900 to 2,600 characters, and the first version showed them whole. Two things were wrong with that. Nobody reads an instruction sheet to decide whether to press OK; and Open WebUI does not scroll that dialog, so a job that tall pushed OK and Cancel 150px below the bottom of an 800px window and the question could not be answered at all.

So the dialog gets a short summary of what the job will do: the opening line, then the head of each step, six of them and a count of the rest. Measured: 1,861 characters became seven lines.

It is built from the job, not written by the model. This is the one screen where a summary that drifts from the text is a real problem - the person approves the summary and the machine is sent the job. A summary pulled out of the job's own headings cannot describe work that is not in it, and asking the model for one would be both a second thing to trust and more work for it to do, which is the opposite of what was asked for. A test asserts that every word in the summary appears in the job.

Four things had to be measured rather than assumed, each found by looking at the rendered dialog:

• A bulleted list under a step is not a step. A real job's bullets were the data the steps operated on (- /tmp/logtest/recent.log -> mtime = now), and counting them pushed every actual step out of the summary. Numbered lines and STEP headings win; bullets are used only when there are none.
• Sentences end at .!? and nothing else. Splitting on the colon too turned Run: ps -p 1 -o comm= into the bullet Run:, which says nothing - the useful half is always on the far side of the colon.
• An abbreviation is not a full stop. e.g. and (i.e. cut bullets off mid-word until the punctuation around the token was stripped before checking it.
• A plain newline is a soft break in markdown. Consecutive lines were joined into one paragraph and two steps ran together on a single wrapped line. A - list item at column 0 is the one form the renderer keeps on its own line: indent it and it becomes a nested block, use a bullet character of your own and it is not markdown at all and gets joined like any other prose.

A step that is only a label - Report: - takes the line under it, because a bullet reading just Report: spends a line and says nothing.

Result. All three paths driven through a real chat, not asserted:

• Accept - dialog shown, empty + OK, job dispatched, machine booted, seven tasks run, the agent correctly reported that -mtime +7 keeps a file at exactly 7 days and at 7 days 12 hours and deletes at 8, the non-.log control survived, and the nested file did not. Machine torn down, owuivm.service inactive, no qemu left.
• Refine - typed "Do not install anything. Only check what is already there, and also print the exact python3 path"; the model rewrote the job to say exactly that and asked again. No machine was started in between.
• Cancel - no machine started, no job written, and the model closed the turn with four words rather than arguing with it.

And then the view killed a session, which is the part worth writing down. A run finished all six of its tasks, verified them, and returned nothing at all. The agent loop had died at the last step:

File "/app/loop.py", line 464, in _terminal_for
  return [f"$ {str(arguments.get('verification', {}).get('command', ''))[:300]}"]
AttributeError: 'str' object has no attribute 'get'

finish is declared as taking verification as an object with a command in it. The model sent a string. The renderer for the live terminal - a cosmetic feature, added so somebody watching the chat can see what is happening - called .get on that string, and the exception came back out through the agent loop and threw the finished work away.

The narrow fix is one isinstance. The real fix is the rule that was missing: the picture of the session has no bearing on whether the work succeeds, so it does not get a vote. Every renderer now returns [] rather than raising on anything the model can send, and the call itself goes through _show(), which swallows whatever gets past that and logs it at debug. One malformed argument costs a line of output, not a session.

tests/test_vm_live_view_safety.py covers it: the exact call that crashed, every renderer against eight shapes a model might send in place of an object, a deliberately exploding renderer and a deliberately exploding emitter, and a source-level case that fails if the loop ever calls ui.terminal(...) outside the guarded path - which is how the first one got in. Reverting the isinstance makes the first case fail, so the test is known to catch the original bug rather than merely to pass.

Re-run afterwards, same prompt: 8/8 tasks, status Done, no loop errors.

And the view, measured live in the browser during a session: two iframes, heights 86px and 364px, neither at the 150px default, and no horizontal scroll on the page. tests/test_vm_live_view.py covers all of it - including a case that fails if any method in the filter is ever defined twice again, and one that fails if the height is measured from the document instead of the content.

59.31 — four things wrong with the last thing the user sees

Four complaints from one real session, none of them about whether the work was done.

1. It wrote the script, handed it over untested, and waited to be asked. The reply to "write a powershell script that will pull system info" was a script; only after "yeah go ahead and test it" did it reach for a machine. The confirmation dialog (59.30) changes the economics here and the wording had not caught up: the user is shown the job before anything starts, so proposing a verification costs them one click. The docstring now says so directly - when you have written a script the user is going to run, call this in the same reply - on the grounds that an untested script is a guess however good it looks.

2. Reopening the chat mid-session showed no progress at all. Two defects, one real and one worse.

The real one: the watcher emitted only on change. These updates are pushed, so somebody who reloads the page or opens the chat on a second device joins partway through and receives only what comes next - and a session holds one step for minutes at a time (Waiting for the machine to finish booting held across four consecutive polls, measured). A viewer arriving just after a change sits in front of an empty message for the length of the step and concludes nothing is happening. The whole view is now re-sent on a 10-second heartbeat as well; replace: true makes that idempotent, so the cost is one small message per tick and reattaching fixes itself.

The worse one: the watcher's finally block wiped the view and announced "The machine has been destroyed." - unconditionally. That block also runs when the watcher is cancelled or hits its own max_minutes deadline, and in those cases the machine is still running. The watcher stopping is not the same event as the session ending, and only the latter now says so; the other says "No longer watching this session."

Verified from the logs rather than assumed: a full page reload at 20:13:37, about 25 seconds before that session ended. statusHistory (33 entries) and embeds both persist through get_event_emitter, and events are broadcast to user:{user_id} rather than to a socket session, so a reconnected browser is in the right room - which is why the heartbeat is sufficient and why nothing needed to change on the Open WebUI side.

3. Asked for a link to download the script, given a link to a repository. Those are different things: one saves a file, the other opens a page to navigate. The report now lists every committed file with the address that serves it (/raw/branch/main/<path>), and says plainly not to offer a repository URL in place of one. git ls-files rather than a directory walk, because the walk would also list whatever the guest left uncommitted and a link to that is a 404 with the operator's name on it. Measured: http 200, 1346 bytes, text/plain.

4. The repository said nothing about what it held. Every repo was labelled "Created by the owuitools VM agent for project 'x'". The operator's words: it "just has to accurately describe what the app does briefly - not sell it to me". So it is generated from the job, for the same reason the confirmation summary is (59.30): a description that is generated cannot be a pitch and cannot describe work that was not asked for.

The first version was too literal - it set "On the Ubuntu machine, write a small Python script called wordcount.py that takes a filename and prints the...", which is the instruction, verbatim and cut off. A job opens by addressing the machine and telling it to build something; a description labels the thing that was built. Both leading clauses are now stripped, and only verbs that CREATE are removed, so a project whose job was to test something is still described by that word. Measured on the next real session:

Small Python script called sumcsv.py that reads a CSV file and prints the total of a named numeric column

An existing description is never overwritten unless it is the old placeholder, so a repository somebody has labelled by hand keeps its label - and repositories created before this change keep whatever they had.

Result. Two full sessions driven end to end against the live stack: 6/6 and 5/5 tasks, supervisor verification passed, direct links present and fetching, description correct on the repository created after the fix. tests/test_vm_git_hardening.py covers the links and the description; tests/test_vm_live_view.py covers the heartbeat and the honest close.

Not verified: the reopen itself. The browser tooling disconnected partway through this work, so the fix is reasoned from the emitter's source and the request log rather than watched happening. The two defects behind it are both real and both fixed, but nobody has yet reloaded a chat mid-session and seen the boxes come back.

59.32 — the agent and its supervisor were standing in different directories

Symptom. A Windows session was asked to write a PowerShell script, test it and keep it. It did all three, correctly, and the session was reported as a failure. The model read the failure, worked out what had happened, and spent a second entire session redoing the same work - about four minutes of machine time to move one file.

Cause. The supervisor's re-run started in the project folder. The agent's own run did not: it inherited whatever directory the guest agent happened to start in, which on Windows is C:\Windows\system32. So Set-Content Get-SystemInfo.ps1, Get-Content .\Get-SystemInfo.ps1 and running the script all agreed with each other and all passed - and then the supervisor re-ran the verification one directory away and found nothing.

Two copies of "where does a command run" is exactly how they came to disagree. It had been written once in verify.py and never at all in vmd.py.

Change. One working_directory_prelude(session), called by both. It is the project folder on both machines, which is also the only place work survives - a file written anywhere else is deleted with the disk. tests/test_vm_working_directory.py asserts both callers use it and that the directory is decided in exactly one place.

Result. The same case re-run, with the agent deliberately told to use a relative path: the supervisor's powershell.exe -NoProfile -ExecutionPolicy Bypass -File Get-Uptime.ps1 passed, exit 0, first session, no retry.

---

Three smaller things from the same run.

The reply pasted the whole script back into the chat. It had already been given a download link, so the code was a wall to scroll past to reach the one line the user wanted. Both the docstring and the prompt now say: give the link, say what it does and what the test showed, and quote a few lines only where the point is about those lines.

The repository description came out as a shouted instruction - "IMPORTANT: create Get-SystemInfo.ps1 INSIDE YOUR PROJECT FOLDER..." - because the description is generated from the job (59.31) and that job was the corrective re-prompt. The directory fix removes the cause; IMPORTANT:/NOTE:/WARNING: are now stripped like the other labels in case a job opens that way again.

The download link 404s, and that is authentication rather than a broken URL. Measured on the exact link from the chat: anonymous 404, authenticated 200, 3,976 bytes. The vmlab org is deliberately Limited visibility - public repos inside it, readable by any signed-in Gitea user and invisible to everyone else - which is the arrangement this project was asked for. A link to it therefore only works in a browser that is signed in to git.example.com. Recorded rather than changed: making it work anonymously means making every vmlab repository readable by anyone on the internet, which is the operator's call and not a bug fix.

59.33 — a release per session, and what actually downloads

Symptom. A link to the file in the repository opens the source in the browser instead of saving it. The operator's decision was to publish a release at the end of every session and hand over the releases page instead.

What I got wrong first, and measured second. I assumed release assets download. They do not: Gitea decides inline-or-attachment by MIME type, so a .py or .ps1 comes back text/plain; Content-Disposition: inline whether it is a raw file URL or a release asset, and no query parameter flips it. Measured on all three.

The one thing on that page served as an attachment is the archive Gitea builds for the tag - /archive/<tag>.zip returns Content-Disposition: attachment, which is the "Source code (zip)" link. So the releases page is the right thing to hand over for two reasons rather than one: it lists the session's files by name, and it carries the one link that saves to disk. The report says exactly that, rather than claiming the files download.

Change. After the push, and separately from it, vmd creates a release tagged v1, v2, ... and uploads the committed files as assets. The tag number is counted from what is already published rather than held anywhere, so it survives a vmd restart; only one session runs at a time, so there is nobody to race with. Assets are capped at 10 files and 25 MB each, taken from git ls-files so nothing uncommitted is ever offered, and a nested path becomes a flat name because a release has no folders. The upload is multipart on the stdlib, with a random boundary - a fixed one appearing inside a file would corrupt the upload.

It cannot take a session down. By the time it runs, the work is committed and on the server. A release that fails to publish is a missing convenience, and the report says so - "the work is committed and pushed; only the download page is missing" - rather than turning a session that did its job into one that reports a failure.

Result. Two consecutive sessions against the same project: v1 with sample.txt and wordcount.py, then v2 with README.md added - one release each, holding the files as that session left them. /archive/v2.zip returns Content-Disposition: attachment.

The 404 was never a bug. The vmlab org is Limited visibility by design; the operator was signed out. Confirmed with them, and nothing was made public.

59.34 — it spent a whole reply writing the job instead of sending it

Symptom. "Build a GUI PowerShell that acts as a Windows troubleshooting toolkit." The model answered with four sentences of intent - "I'm going to write it and then have it launched and clicked through on a real Windows machine before handing it to you" - and then nothing happened. No tool call, no machine, no script.

Cause, from the usage record: eval_count: 32768, which is exactly the max_tokens set on the model in Open WebUI. 32,768 output tokens produced 398 characters of visible content and tool_calls: []. The model had spent its entire output budget writing the PowerShell GUI into the task argument of the vm_agent call, hit the ceiling mid-argument, and the call was never emitted at all.

The instruction told it to verify a script it had written. It read that as "write the script, then hand the finished thing over" - which is both the wrong order and the wrong division of labour, and at GUI scale it does not fit in one reply.

Change. The handoff moves to the front, in the docstring and in the prompt: when the user asks for something that has to be built and run, vm_agent is the opening move of the reply. Do not design it, do not write it, do not work out the approach - the agent does all of that on the machine, where it can run what it writes. The task is the user's request restated plus its constraints and a sentence on what "working" means; the task field now says NOT an implementation - do not write the code here, the agent writes it, and the docstring carries the measurement above as the reason.

And a correction to this log. 59.29 and 59.30 both say the system prompt is baked into the Ollama model and therefore cannot be deployed from here. That is wrong. It lives in params.system on the qwen3.8:latest row of Open WebUI's model table, is read per request (routers/ollama.py calls Models.get_model_by_id on every completion), and takes effect on the next message with no restart. The deployed copy had drifted 3,700 characters behind prompts/system-prompt.md - every routing fix from 59.29 onward was live in the tool schema and absent from the prompt. It is now deployed, byte-identical to the repo, with the previous version kept. Diffed before writing: four lines reworded and fifty-one added, all inside the two vm_agent sections, nothing else touched.

Not verified: the behaviour change itself. The browser tooling is disconnected, so I could not put the prompt back through a chat and watch it hand off. The schema and the prompt are confirmed live and correct; whether the model now calls first needs one message through the UI.

59.35 — four machines, four prompts, and a line that proves it is alive

Symptom, and it is the one that gives the game away. A Windows GUI session wrote itself the task "Check machine readiness: PowerShell version, display resolution, /mnt/project accessibility". /mnt/project is the Linux path; on Windows the project folder is Z:\.

Cause. All four kinds of session - ubuntu/windows by cli/gui - were given the same prompt and the same tool descriptions, which named both machines and left the agent to work out which half was about it. prompt.md said "/mnt/project persists; everything else is destroyed", full stop. The run tool said "a root bash login shell on Ubuntu and an elevated PowerShell on Windows". This stack has three times proved the model reads tool text as fact rather than as something to reason around, and that is exactly what it did.

Change. A session is now told about the machine it is in and about nothing else. The prompt is assembled from prompt.md plus prompt.{ubuntu,windows}.md plus prompt.{cli,gui}.md, and the tool descriptions are built from a MACHINES table. So:

	project folder	run is	screen tools	lookup suite
ubuntu/cli	/mnt/project	a root bash login shell	no	yes
ubuntu/gui	/mnt/project	a root bash login shell	yes	no
windows/cli	Z:\	an elevated PowerShell 5.1	no	yes
windows/gui	Z:\	an elevated PowerShell 5.1	yes	no

The Windows fragment carries the two facts that have each cost a session already: 5.1 has no && or ||, and you are already inside PowerShell so wrapping in powershell.exe starts a second one. The CLI fragment says plainly that there is no screen and nothing can click. A missing fragment is skipped rather than raised on - a thinner prompt still runs a session, and refusing to start one over a missing file would be the worse failure.

Result. A Windows CLI session, re-run: 4/4 tasks, every task naming Z:\, /mnt/project absent from the entire report, supervisor verification passed.

---

The live view, as the operator drew it. Three boxes in GUI, two in CLI:

Live agent output          Live agent output
Task List:                 Task List:
tasks                      tasks
-----------------          -----------------
                           -----------------
                           the machine's screen
                           -----------------
-----------------          -----------------
command feed (20)          command feed (10)
-----------------          -----------------
      CLI                        GUI

The live line meant making the agent's calls stream. They were non-streaming on purpose - the agent acts on a finished message, so nothing in the loop needed tokens as they arrived. True of the agent, false of the person: measured during this work, the progress file sat unchanged for over a minute while the GPU ran at 83% and the guest at 69% of a core. Everything was fine and the screen said nothing, which is indistinguishable from wedged.

So chat() takes an on_text callback. Without one it streams nothing and behaves exactly as before; with one, the reply is read as it arrives and the assembled message is identical in shape, because the agent is not allowed to care which path produced it. The callback belongs to the watcher, so it is called inside a suppressed block: a display that can fail a turn is worse than no display. Thinking is shown only while there is no content yet - a model that is thinking has not said anything, and a blank minute reads as hung.

And the GUI feed would have been blank rows. Every tool a GUI session uses is a desktop verb

• screenshot, click, key, type, scroll, move - and none of them had a renderer, so the third box would have shown ten empty lines for the whole session. They render now.

Result, from a real GUI session: the live line moving continuously through the model's own words, shot_seq advancing 0-3, and the feed holding exactly 10 lines - half the CLI box, as asked.

59.36 — it could not put a file down and come back for the rest

Symptom. A session spent twenty-two minutes and roughly fifty thousand tokens inside one turn, designing a WPF application in its head, still on task one of twelve, having made no tool call for twenty minutes. It was not stuck and it was not looping - the live line advanced every second and never repeated.

Cause, and it is not the model's discipline. The agent had write_file and nothing else. Every change to a file meant reproducing the whole file, so a file it had to produce in one call was a file it had to design completely before it could write any of it. The 2 MB limit was a resource limit, not a forcing function; there was nothing to make it stop and nothing to come back to.

The operator's framing was the right one: think about how your own tooling is set up. And the answer is that I almost never write a whole file - I edit one. Exact-string replacement, one small change at a time, failing loudly when the target is missing or ambiguous. The agent had no equivalent.

Change.

• edit_file(path, old, new, replace_all) - exact text, and it refuses when the text is absent or appears more than once. The refusal is the feature: text that is not there means a mistaken belief about the file, and being told now is far cheaper than a silent half-edit found three sections later.
• append_file(path, content) - build forward a section at a time.
• write_file capped at 16 KB, with a refusal that explains why and names the way past it. This is the forcing function; the cap message keeps the escape hatch for generated data.
• num_predict: 16384 - there was no output ceiling at all, which is how one turn ran twenty-two minutes.
• The prompt now says: skeleton, parse-check, one section per turn, commit as you go, link it together last - with the parse-check command for each machine.

Three bugs fell out of testing it, each found by using the thing rather than reading it.

1. mkdir -p "$(dirname ...)" sent to PowerShell. write_file ran that on every guest, and the result was discarded. On Windows it failed every time, so any nested path failed to write; a top-level file worked because Z:\ exists. Rewritten in PowerShell it failed AGAIN, differently: 5.1 rejects Split-Path -Parent -LiteralPath as an ambiguous parameter set, so $p was null, Test-Path and New-Item both bound null and did nothing - and because those are non-terminating errors PowerShell still exited 0. Measured from inside the machine: PARENT=[], EXISTS_AFTER=, exit 0, no directory. An exit-code check could not have caught it. vmd now splits the path itself - it built the path, so it knows the separator - and confirms the directory exists rather than trusting a status code.

2. The first release on a new repository always failed. 422 repo is empty, moments after a push that plainly succeeded: Gitea clears is_empty asynchronously and the release API reads that flag. It never showed on a repository that already had commits, which is why it survived two sessions. Now retried with a short backoff.

3. Answering the agent's own question was rejected. The tool tells the model to call back with session_id and answer; the schema required task, os and interface on every call, so the reply came back "task: Field required". They are optional in the schema now and required where a new session actually needs them, with a refusal naming which is missing.

Result. The whole file-tool path driven on Windows, with run forbidden for the writes: nested write_file with no directory pre-created, append_file, edit_file, the ambiguous-text refusal verbatim, read_file - all correct, first attempt, and a release published on a brand-new repository.

Still open, found and not fixed: there is no resume path. session_id is documented, accepted, and used by nothing - runner.py never reads it, so an answer arrives as a fresh job against a machine that is already running. ask_operator therefore strands a session. Recorded rather than patched under time pressure.

59.37 — one kind of session, because the choice was never the caller's to make

Symptom. Asked to "build a gui powershell that acts as a windows troubleshooting toolkit", the model chose interface: cli - a session with no screen, which can write a desktop application and confirm it parses and can never say whether the window opened. Told not to, in the tool docstring AND in the system prompt, it did it again on the very next run.

Cause, eventually. I built a refusal to stop it - needs_a_screen in vm_preflight, narrow, tested, and it worked. Then the operator pointed out the obvious thing I had walked past: the choice did not need to exist. I had written a mechanism to defend a decision rather than removing the decision.

Three separate costs, all measured, all from the same split:

• The caller chose wrong, twice, with the rule in front of it both times.
• A gui session had no cheap way to look anything up. The lookup suite was CLI-only, on the reasoning that a desktop session could read anything by opening it in the browser - true, and far too expensive: open_page costs a screenshot per scroll where read_page costs one call. So it reasoned instead. Eleven minutes and eight context compactions deriving how to run background work in PowerShell, which the documentation states outright, while a parse-clean 19 KB application it had already written sat untouched. Another spent twelve minutes writing nothing at all, the live line reading "I recall precisely: WinForms processes docking in reverse z-order".
• A cli session could build a window and never see it, so "it parses" was the best answer available to it.

Change. There is one kind of session. Every one gets the command line, the lookup suite and the screen; the agent decides what it needs, which it is far better placed to know than the model that called it. interface is gone from the tool schema entirely - one less required parameter, one less thing to get wrong - and needs_a_screen went with it.

The rest follows:

• The desktop is waited for lazily, on the first screen verb, so a job that only runs commands never pays for GNOME finishing its paint. That was the whole argument for having a cheap mode, and it survives without the split.
• No screenshot is taken at handover. It used to attach one automatically, which put a picture of an idle desktop in the chat before the session had done anything, for every job including the ones that never look at a screen. The preview box now appears only when the agent takes a screenshot, so its arrival means something.
• One terminal height (20 lines). It was halved when a screen was shown; the screen is now the exception rather than the rule, so the feed is not shrunk for something that may never arrive.
• One step budget, the generous one (110), because any session may now look at a screen and seeing the result of an action is its own turn.
• Two prompt fragments instead of four: prompt.{ubuntu,windows}.md plus one prompt.screen.md that says the screen is there and that most work does not need it.

The lookup suite in every session is the operator's suggestion, and it is the one that addresses the failure directly: a session that cannot cheaply look something up will reason about it instead, and reasoning is what was eating the sessions.

Result. tests/test_vm_four_sessions.py became test_vm_two_machines.py. Thirteen suites pass. The run under way when this was written: 6 of 10 tasks, a 32 KB application built with write_file then edit_file, launching its GUI - the best of six attempts at this prompt.

59.38 — tuning it by running it, and the one parameter that was wrong

Two campaigns, driven through the real chat rather than the API, because the API returns the final message and nothing else - the dialog, the live line, the boxes and their heights are all invisible to it, and every defect found in this round was a visual one.

Windows: "build a gui powershell that essentially acts as a windows troubleshooting toolkit". Eight attempts. The interesting ones:

	outcome
before this round	0/12 tasks, no tool call for 20 minutes, one turn of ~50,000 tokens
WinForms, old tooling	7/10, app built and launched, all tabs screenshotted
WPF/XAML, old tooling	1/12, 48 minutes on the skeleton, 22 compactions
WinForms, tuned	7/9, 365-line 7-tab app, README, parse-checked, launched, window confirmed rendering with all tabs

The middle row is the useful one: same code, same prompt, and it failed because the agent chose XAML over WinForms. That is approach variance, not a tooling fault - and it is what made the next change worth making.

The agent's presence_penalty was 1.0, copied from the main assistant's row. That is right for the main model, which writes prose: a penalty on tokens already used discourages repetitive phrasing. It is wrong for this agent, which writes code. A 25 KB script says $form, $output and its own function names over and over - that repetition IS the program - and a penalty on it is steady pressure toward a different token than the correct one. Two failing sessions stalled on exactly that shape of bug: "a scope issue with how I'm accessing the log", and "it's using $a[0] when $a was never actually defined". Qwen3's own guidance says a presence penalty may degrade output and to use it only against repetition loops - and the agent already has a repeat guard for those, in tool output, where it can say why.

Set to 0 for the agent only. The main model's row is untouched: temperature 0.6, max_tokens 32768, presence_penalty 1.

And the task list had gone flat. A session sat on "1/12" for twenty minutes while the file grew and edit_file calls landed. The work was fine; the report of it was a flat line, and somebody watching would read that as stuck. The prompt already asks the agent to mark progress and asking has not worked, so it is now a nudge in tool output after twelve steps with no task movement: "Whoever asked for this is watching that list and it still says you are where you started." A nudge and not a refusal - being mid-task is legitimate, and blocking on it would be wrong far more often than right. Measured on the next run: the agent went 0 to 5 of 8 in one step after it fired, and the nudge is visible in the command feed.

Ubuntu: "watches a folder and turns any new png into a compressed jpeg, keeping the original". 7 of 7, first attempt, status Done - and it never took a single screenshot, which is the one-session change working exactly as intended: the agent decided it did not need the screen, so the preview pane never opened. The evidence file it left behind is the point of the whole subsystem:

PNG sha256 (before): 77dac1f6...97ba
PNG sha256 (after):  77dac1f6...97ba
demo/photo.png: 1157239 bytes    demo/photo.jpg: 242441 bytes
JPEG smaller than PNG : YES      PNG unchanged : YES

What is still hard, stated plainly. Background/async work in PowerShell defeated the agent in three separate sessions - runspaces, threads, Start-Job - and it is the only thing that did. The scope rule tells it to take the duller approach after two failures and it does not always listen, because that is a prompt rule. Making it a tool-output signal is the obvious next move and is not done.

59.39 — the sweep, and the bug it found in the last place anyone looks

A final pass over everything, driven through the chat. It found one real defect, and it was in the step that ends a session.

finish was refused with an internal error, every time. A session wrote the script, syntax-checked it, built a fourteen-file test tree with awkward filenames, cross-checked its output against an independent listing, demonstrated the error paths, committed, and passed its own verification command - and then could not close:

File "/usr/local/lib/owuivm/vmd.py", line 1233, in v_finish
AttributeError: 'str' object has no attribute 'get'

finish declares verification as {command, expected}; the model sends the command as a bare string often enough to matter. The agent saw only "the supervisor refuses the call", tried seven times with varying arguments, marked the task blocked and gave up. All the work was done and committed. The session simply could not be ended.

The identical mistake had already been found and fixed in the live view's renderer (59.36), where it cost a line of output. Nobody checked the verb that reads the same argument. The rule is the same wherever a model-supplied value is read: take what arrives, not what was asked for. tests/test_vm_finish_shapes.py covers both shapes and four kinds of junk.

Re-run after the fix, same prompt: 8 of 8 tasks, status Done, four minutes.

Also seen, and fixed in the prompt rather than mechanically: the agent built its test tree inside the project folder, so two 256 MB fixtures were committed and published. They were sparse and compressed to nothing - .git came to 566 KB - and the release asset cap skipped them, so nothing broke. On real data it would be a quarter-gigabyte blob in somebody's repository. The prompt now says scratch belongs in /tmp and the project folder is for what was asked for plus the evidence worth keeping.

Final state of the sweep: fourteen suites pass; owuivm inactive, no qemu, no overlays, no pending jobs; both base images 0444; every container healthy. The live view was re-checked in the browser on a fresh session - two boxes before any screenshot, the live line above the task list, feed at full height, nothing stuck at the 150px default, no horizontal scroll - and the confirmation dialog fits its window with the buttons reachable.

---

Also measured: boot time fell from 2min 14s to 20.9s once systemd-networkd-wait-online was masked (graphical.target at 9.5s rather than 2min 3s); a session opens and answers on its guest agent in ~27s; a click at an absolute coordinate landed on the button it was aimed at and advanced a GNOME dialog; two sessions on the same project name saw the first session's files.

Not measured: the cost of an Ollama handover. The design assumes a cold prefill of the main model's context on every return, which is why one long blocking call is preferred to polling, but no benchmark has been run on this V100 — the figure used to reason about it is an estimate. The Windows guest now exists and has been measured (59.26, 59.28): a session opens on it, and it resumes from a saved machine rather than booting. The live-progress channel has now been watched end to end from a browser - the filter is installed, and the two boxes were measured at 86px and 364px during a running session (59.30). GUI mode has been proven verb by verb - screenshot, absolute click, key, type - but never driven by the model.

59.40 — the release carried the debris and not the work, and the prompts said things that were no longer true

Symptom, in the operator's words. "When a release is built in gitea it only has the files in it relevant to the actual release and not include any ambiguous testing tools / scripts" — and "clean up the prompting where needed to avoid confusion."

Measured. Release v3 of windows-troubleshooting-toolkit held ten files — _build.ps1, _launch.bat, _structcheck.ps1, four _w_*.ps1 fragments, check_disp.ps1, check_disp2.ps1, check_disp3.ps1 — every one of them scratch, and not the 34 KB TroubleshootingToolkit.ps1 the session was asked for. Two causes compounding: git add -A committed whatever the agent had left in the folder (30 files, 25 of them probes, captured output and a compiled test_cs.exe), and tracked_files() sorted _build ahead of TroubleshootingToolkit and stopped at ten. Three other releases shipped test_*.ps1 beside the deliverable; top-ten-files had 1.5 GB of testtree/ fixtures committed.

What actually downloads, measured again. Uploaded release assets come back Content-Disposition: inline — a .zip asset included, application/octet-stream; inline. The only attachment on a release page is Gitea's own /archive/<tag>.zip, built from the tree the tag points at. So a curated asset list would have left the one link that downloads full of junk: a release is only as clean as the commit behind it, and the filter had to go at the commit.

Change — the commit. gitea.py gains SCRATCH_PATTERNS, applied as :(exclude,glob) pathspecs to git add -A: a leading underscore, test_*/test-*/*_test.*, smoke.*, probe_*, tmp_*, *_out.txt, *.out, build products (*.exe, *.dll, *.obj, *.o, *.pdb, *.class), and whole trees under test*/, tmp/, scratch/. Deliberately narrow: check_* and find_* are not excluded, because find_duplicates.py was a real deliverable and a "check disk health" job would reasonably produce check_disk.py. Three guards on the filter:

1. A project that would otherwise publish nothing — everything reads as scratch and nothing publishable was ever committed — publishes everything, because then the tests were the job. The freshly written .gitignore is discounted when deciding that, or it alone made an all-tests folder look like it had something publishable in it.
2. Scratch an earlier session already committed is untracked (git rm --cached) on the next commit, so a resumed project stops publishing its old junk without losing the file on the array.
3. git_commit names what it held back, in tool output, with "if this is the deliverable, give it a name that says so". A filter nobody mentions is indistinguishable from a bug.

has_changes() and commit() share one _staging_plan(), or the end-of-session backstop decides there is something to commit and then stages nothing, which git calls an error. The asset cap goes from 10 to 25: it exists to stop a file dump, and the dump no longer reaches the commit.

Change — the stranded session. ps-troubleshoot-toolkit had a 20 KB toolkit and five test scripts on the array, zero commits, no repository, no link. Its machine died before the loop could call session_close (the agent container was rebuilt underneath it), and the only path that then saw the record — session_open's stale-record cleanup — destroyed it without publishing. The reply the user got carried a confident link to a different project's release, assembled from the task wording. Now the stale-record path runs _publish before it discards the record; the report states "There is no link to give for this session" as a fact when nothing was published; and the main prompt and the tool docstring both say to hand over only the link in the result and never build one from the project name.

Change — the prompts. prompt.md promised "in a desktop session you will be handed a picture of it" — the handover screenshot had been removed a change earlier. "In either mode" survived the removal of the modes. The lookup guidance was written twice, read_page/open_page explained twice, "GUI included" in one of them. "Nothing graphical you start from run can be seen" was said in four files. /tmp sat in the machine-neutral prompt — the same shape of mistake as the Windows session that planned to "check /mnt/project accessibility". All of that is now said once, in the file it belongs to; the scratch location lives in each machine's own section; the commit filter is described where the project folder is. Stale CLI/GUI comments in tools.py, vmd.py, websearch.py and compact.py, and the filter's valve descriptions ("In CLI sessions…"), were rewritten.

Tests. tests/test_vm_release_contents.py rebuilds the failed release from its real file list and asserts the toolkit is committed and the scratch is not; that find_duplicates.py, Check-AutoStartServices.ps1 and latest/ survive; that an all-tests project still publishes; and that a resumed project un-publishes legacy scratch while keeping the file. tests/test_vm_prompt_consistency.py reads the text the two models actually see — prompts, tool descriptions (not comments), the tool docstring, the report — and fails on mode language, the handover-picture promise, a machine path in the neutral prompt, a tool name the agent does not have, and the same sentence said in four files. 17 suites pass.

Dry run against the array, read-only. Every real project: ps-troubleshoot-toolkit would publish Toolkit.ps1 and hold back its five test_*.ps1; top-ten-files would un-publish 1.5 GB of testtree/; windows-troubleshooting-toolkit would un-publish all 25 scratch files and keep the toolkit. One judgement call flagged to the operator: systemd-restart-test's units/test-*.service are fixtures by name and arguably the artifact; they would stop being published, leaving REPORT.md.

Also found. Open WebUI 0.11.3 with no Redis configured refetches every tool server's OpenAPI spec on each chat request (get_tool_servers → set_tool_servers), so a changed tool docstring is live as soon as the api container is up — no re-save, no restart. And the vmagent bot token has write but not delete rights: the probe repository vmlab/zip-asset-probe used to measure the download headers could not be removed by the API and needs deleting from the web UI.

59.41 — a live line you can read, and three boxes you can fold away

Asked for. Smooth the live token stream above the task list so it arrives token by token and its speed shows how fast generation actually is. Make each box fold when clicked: the task list down to two lines (the live line and the current step), the screen preview to half size with black either side, the command feed to half its height. Click again to restore. No tooltip.

The problem underneath all of it. The view is re-sent every couple of seconds, and each send replaces the frame's srcdoc — a new Document every time. A click has to outlive that, and the frame is sandboxed without allow-same-origin, so it has an opaque origin: localStorage and cookies throw, IndexedDB is useless, and Open WebUI has no listener to postMessage to. There is no server channel either — the filter polls, nothing calls back.

Measured, in that exact sandbox: window.name survives. It belongs to the browsing context, not the document, and the context is what the swap keeps. A frame reloaded three times read back the name its two predecessors had written. That is the whole mechanism: each box keeps {k, c, d} there — which box it is, whether it is folded, and the text the ticker has shown so far. It is keyed by box because the screen box appears mid-session and pushes the feed down a slot, so the frame that was the feed becomes the screen and would otherwise inherit its state.

The ticker. The server sends the tail of what the model is saying, at most once a second, so the line used to jump a clause at a time — motion that said nothing about throughput. The frame now computes the overlap between what it has already shown and the new tail, and appends only the remainder, a few characters per 40 ms frame, draining over ~1.5 s. More characters in the window means faster typing, so the speed on screen is the token rate. It is right-anchored (justify-content:flex-end), because the end of the sentence is the part that is moving.

flex:none on the feed rows is load-bearing. The rows sit in a flex column so a folded feed keeps its newest lines against the bottom and loses the oldest off the top. Flex items shrink by default, so the first version compressed twenty rows into ten rows of space and the text overlapped itself — caught by eye in the harness, not by any test.

What the browser could not tell me, and how that was worked around. Dispatched clicks from the automation tool never reach iframe content — proven with a page holding one sandboxed and one plain frame, each with a click handler: the parent's own button fired, neither frame did. So interaction was verified by replaying the real generated frames into non-sandboxed iframes and dispatching clicks through the DOM, while window.name persistence was measured separately in a real sandboxed frame. Both halves measured, in a browser, before the code was believed.

Result, in a real frame: task box 196px → 51px and back; screen 500px → 269px with the image at exactly half its width (740 → 370) on black; feed 364px → 190px showing ten rows at an unchanged 17.4px. All three survived two srcdoc swaps in the folded state.

tests/test_vm_live_view_collapse.py locks the contract — the per-box config block, the three ids each box needs, the window.name keying, the toggle re-reporting its height, the overlap arithmetic, and flex:none on every row. 18 suites pass.

Note for whoever changes this next: a running watcher holds the module it was started with, so a filter deployed mid-session does not reach the session already being watched. The new view appears on the next turn, not on the one in flight.

59.42 — the transcript was always there, and was thrown away every time

Symptom. Asked "did the agent ever open the app and look at it", I could not answer from the record, and answered about the wrong run while doing it. The operator's reply: "You're definitely not looking at the logs correctly ... I want you to set up proper logging so that its easy to go back and see exactly what the LLM did without getting confused."

Cause, and it is not the one it looks like. Nothing was missing from the recording. _record() has covered screenshot, click, key and type since they were written, and STATE.transcript held the lot. It was discarded when the session ended, so the only durable records were the two that cannot answer the question:

• the tool result counted run and nothing else. A session that marked "clicked through the GUI" done without ever calling click read exactly like one that did.
• Open WebUI's status history is SAMPLED. The filter polls every two seconds and reads whatever the status says at that instant, so a verb that returns fast is usually overwritten before it is ever seen. Measured on a real session: 456 samples caught launch_app nine times, because it blocks for seconds, and caught none of the screenshots that demonstrably happened. Reasoning from that record is reasoning from a record that cannot be complete.

Change. vm/vmd/sessionlog.py writes the transcript to /var/lib/owuivm/sessions/ as it happens, one JSONL file per session, flushed on every line - the session someone wants the log for is precisely the one that got killed. One hook in _record() covers every verb, including any added later. It carries the chat id, so a conversation maps to exactly one session and nobody has to guess from timestamps again. A harness-only note verb records what the model said each turn, so the file reads as an account rather than a list of calls; the model is never told it exists. vmctl logs lists them and vmctl log [chat|session|file|project] renders one, neither needing a daemon or a machine. The last four verbs that were doing their work unrecorded - read_file, list_dir, scroll, wait, check_internet - now record too.

The tool result carries the same counts, so the reader sees them without opening anything, and when the screen was never looked at it says so: "Any claim about how something appears is inference, not observation."

The bug this found immediately, which is the interesting part. The first real session wrote no log at all, silently. vmd runs under ProtectSystem=strict and /var/lib/owuivm/sessions was not in ReadWritePaths, so every write was denied - and the log is built never to break a session, so it swallowed all of it and looked exactly like a session with nothing to record. A feature that silently does nothing is worse than one that fails loudly. The directory is now in the unit, and a log that cannot be opened prints one line to the journal saying why while still returning quietly.

Result, a real hand-driven session read back with sudo vmctl log:

  0:58.5  ready          booted in 57.7s
  1:11.7  plan           2 tasks
  1:13.0  run            exit 0 1.06s  uname -a; echo hello from the log check
  1:17.4  screenshot     1280x800  (390 KB)
  1:17.6  ·  turn 1: Checking that the session log records what actually happened.

what it actually did: 2x task, 1x check_internet, 1x plan, 1x ready, 1x run, 1x screenshot
  it never called click

tests/test_vm_session_log.py walks the AST and fails if any verb touching the machine records nothing, following the one level of delegation that v_click and v_move use. 19 suites pass.

59.43 — the run was fine; the window onto it closed four seconds in

Symptom, in the operator's words: "the tool just gave up it seems like."

What had actually happened. A conversation asked for a GUI app, got one, then asked for a fix to the layout. The second vm_agent call ran for 45 minutes, closed all eight tasks and finished successfully - and the chat showed an empty message under a toast reading "The machine has been destroyed". The whole status history of that message was seven lines:

Calling vm_agent / VM · Starting a machine… / VM · Done / The machine has been destroyed.

Cause. The live-progress file is named after the chat, not the run: /vmjobs/progress/<chat_id>.json. The first run left it with done: true. The second run's watcher started, polled once, read the previous run's ending, announced it and exited. The run then wrote to that same file for three quarters of an hour, reaching sequence 1035, with nobody reading it. Every second vm_agent call in a conversation had this, and the first run in a fresh chat never did - which is why it survived so long.

Change, in two places, because a race with one guard is a race. The api deletes the previous run's progress file (and any stale confirmation answer, which would otherwise decide a question nobody was asked) before the job is queued, which is authoritative. The watcher additionally ignores any progress stamped earlier than the moment it began - updated is written on every flush, so a file from the last run is simply older than the watcher reading it.

The other thing that session did, found in the new log. It spent its last dozen turns fighting git inside the machine: exit 128, "dubious ownership", safe.directory, HOME. It even wrote "Let me use the git_commit tool" at turn 99 and then went back to raw git. The guest sees .git owned by another uid, so this can only ever end in that wall - and a commit that did succeed there would skip everything gitea.py does on the host: hooks and filters pinned off on the command line, the right author, and scratch left out of what gets published.

So run now refuses a mutating git subcommand and says why, naming git_commit. git has to be in command position for the refusal to fire, or echo git commit > notes.txt reads as a commit - a refusal that fires on a string inside an echo is worse than no refusal. It is a signpost, not a sandbox: a guest that insists can still write to .git, and what protects the published history is the host-side hardening, not this.

Verified against the real command lines from the failed session - git add -A, git config --global --add safe.directory, git -c safe.directory=... add, git -C ... commit, /usr/bin/git push - all refused; git log, git status, a quoted string in python3 -c, a grep pattern and an echo into a file all untouched. The stale-file case was replayed in the api container: a planted done: true from ten minutes ago is deleted by the submit path and ignored by the watcher's stamp. tests/test_vm_second_run.py; 20 suites pass.

The general lesson, which has now cost two debugging sessions: state keyed by conversation outlives the thing it describes. The session log (59.42) is keyed by session and carries the chat id, which is the right way round - and it is what made this diagnosable at all, since it showed the run working normally throughout the window in which the UI said it had died.

---

Change 60 — one place to configure it, and a switch that actually removes a tool

Symptom. Nothing was broken; it was unpublishable. Configuration lived on four surfaces and only one of them was in the repo: .env; /etc/owuivm/vmd.env, written once by install.sh and never looked at again; bind-mounted service config scattered across six top-level directories; and state that existed only inside Open WebUI — the system prompt, the filter valves, and the Function Name Filter List, which was the only way to turn a tool off. "What does this deployment offer, and where do I change it" had no answer on disk.

The tree also carried one house's identity in ~60 places, including the operator's username as a functional gate in the polkit rule, and one live secret committed in the clear (searxng/settings.yml).

Change. Everything editable moved to configs/: the stack env, a tool switch, the machine registry, the assistant's system prompt, and the sub-agent's prompts, parameters and tool definitions. docs/ gained installation, configuration, per-tool setup pages, the recommended parameters, and sanitised compose files for the three services that live outside the repo (Ollama, Open WebUI, Gitea). The testing corpus moved under docs/testing/.

Four things are worth recording beyond the reorganisation.

configs/tools.toml removes a tool rather than hiding it. A tool set to false is dropped before include_router, so it is absent from the OpenAPI spec and its endpoint 404s. The 22 schemas cost ~9,700 tokens on every turn of every conversation, so hiding one saves nothing; removing it is the only version of the feature worth having. Missing or unparseable config leaves everything on, deliberately: this is config that removes capability, and its failure mode should be the system it replaced.

The seam is the routers, not app.routes, and the first version of this shipped that bug. FastAPI 0.141 stopped flattening included routers — include_router() appends one opaque _IncludedRouter, so app.routes contains no APIRoute and no operation_id at all. Filtering it removed nothing while looking exactly as though it had worked: 22 tools in, 22 tools out, and a log line claiming 22 unknown names. Caught only because the probe asserted which tools disappeared rather than that the call succeeded.

The sub-agent's 34 tool schemas moved from Python to configs/agent/tools.toml — ~380 lines of tuned prompt text, where a dropped sentence is not a crash but a session that goes wrong later for no visible reason. The move was made safe by capturing the hand-written output to tests/golden/agent_tool_schemas.json before touching anything, then asserting the config-driven builder reproduces it byte-for-byte for both machines and for the booting case. It does. The per-machine variation ({shell}, {project}, {example}) is substituted over the serialised form, because the placeholders appear in parameter descriptions as well as tool descriptions. Dispatch stayed in code behind a DISPATCHABLE allowlist, so a config file cannot operator the agent a verb — a name with nothing behind it is refused at read time, naming the file, rather than offered to the model and failing somewhere less obvious.

Two QOL changes to the maintenance app, and the privileged half of one is the interesting part. Insert disc… replaces the dead end where a machine with no media said "no ubuntu ISO in /var/lib/owuivm/iso" with a greyed-out button. The obvious implementation — pass the chosen path to a privileged unit and let it copy — does not work and should not: the maintenance units run ProtectHome=yes so the unit cannot see ~/Downloads, owuivmadm is not in the operator's group so it could not read a 0750 home anyway, and running the copy as root would hand the operator a "copy any file root can read into a world-readable directory" primitive in exchange for saving one mv. So the app copies as the operator into a staging directory the operator owns, and the unit checks it and moves it. The check includes the ISO 9660 signature at offset 0x8001, which catches the case this will actually meet: a download that was really an HTML error page, which would otherwise fail much later as a machine that will not boot.

Rename changes the label and never the id, which names the disk, the unit instance, the saved-machine directory, the os the assistant passes, and appears in regex allowlists inside the privileged launch scripts. It writes a small override file rather than editing configs/vms.toml, because that file is mostly comments and no TOML writer preserves them — a program that edits the registry in place is one that deletes its own documentation the first time someone presses Rename.

Result. 22 suites pass (20 existing, unchanged, plus the toggle and golden-schema tests); tests/smoke.sh at 174/175 before a stale case was fixed and 175/175 after; the host installer re-run clean with the Gitea token preserved; all ten containers healthy.

Two things this turned up that were not part of the work.

A smoke case had been silently starting a real VM session. It passed interface: "telepathy" with a valid project and expected a refusal — true when interface was validated, but the cli/gui split was removed in 59.x and vm.py now hardcodes it. The nonsense value is simply ignored, so the request became a perfectly good one, started a machine, and died on the 90-second per-case timeout with a transport error. The suite's own comment says only refusal paths belong in it. It now pairs the bad interface with a project name that is refused outright, so it asserts both things and cannot start anything. The session itself was cancelled correctly by the disconnect path and left nothing behind, which is why this had gone unnoticed.

Recreating a network-namespace owner strands its dependents. docker compose up -d netguard left api and searxng running, healthy by their own healthchecks — which are loopback calls — and with no route to anything: squid and valkey both "Network is unreachable", the gateway getting connection refused, and every container reporting Up. Docker does not re-attach a network_mode: container: member when its host is replaced. Written into docker-compose.yml beside both guards, because the failure reports as healthy.

Measured: the golden schemas are byte-identical across both machines; a disabled tool is absent from /openapi.json and 404s; a missing config leaves all 22 on; the insert helper rejects a non-ISO, a symlink, a path-shaped id and an unknown machine; the rename unit writes only the override file. Not measured: the maintenance app's two new buttons under a real polkit prompt — the units were driven directly, and the window was confirmed to build and run, but the operator has not yet clicked through an install from a freshly inserted disc.

---

Change 61 — portable enough to hand to someone else, and one place to read the logs

Symptom. Change 60 made the repository publishable. It did not make it usable: a stranger who cloned it could not have got it running, and several of the things that would have stopped them were invisible from inside this deployment, where they happen to be true.

Cause, in four parts.

The pool. /Pool was written into eleven functional files — three vmd modules, five host scripts, three systemd units. install.sh had a $POOL_ROOT variable by then, and it reached almost none of them.

The machine's shape. AllowedCPUs=20-27, MemoryMax=11G, CPUQuota=700% and a Gitea allowlist naming one LAN address, all baked into units. Meaningless or actively wrong anywhere else; AllowedCPUs on a four-core machine would refuse to start.

Nothing checked. install.sh created users, datasets, units and firewall rules and then found out at step nine whether swtpm existed, leaving a half-installed host.

And the documentation was confidently wrong in two places. Both only discoverable by doing what it said on a machine that was not this one.

Change. Every host artefact became a template with @PLACEHOLDER@ tokens, rendered by one render() in install.sh from values derived from the hardware — nproc, MemTotal — and overridable in configs/host.env. render() refuses to install a file that still contains a placeholder, so a token added to a template and not to the installer fails at install time rather than as a unit that will not start. The pool reaches vmd through vmd.env, which the daemon already loads.

A preflight now checks every binary, the firmware files, /dev/kvm and the pool before anything is created, and creates the two datasets if they are missing. tests/test_vm_host_render.sh renders the whole set with deliberately foreign values (/tank, 6G, 300%, someone) and asserts no line that systemd or bash acts on still carries ours — comments excluded, because the measurements in them are the reason anyone could change a number safely.

Also made settings rather than facts: the Open WebUI network name, and the URL in the outbound User-Agent — which every other deployment would otherwise have been announcing as its own to every site it fetched from.

Logs. ./logs is bind-mounted into all ten containers and each writes its own file: thirteen of them, stdout unchanged so docker compose logs still works. Two needed more than a config line. squid and valkey create their logs 0640/0600 as their own uid, so the operator could not read them — a log nobody can open is not an audit trail; squid takes umask 022, valkey needed sh -c 'umask 022; exec …', with exec so it stays PID 1 and shuts down cleanly. SearXNG can do neither: it logs through Python to stdout and the image dropped uwsgi, so its entrypoint is teed. That makes PID 1 a shell that will not forward SIGTERM through a pipeline, so stopping it waits out the timeout — acceptable there and nowhere else in the file, because it holds no state. owuivmd stays on the journal: it is a systemd service, and pointing it at this directory would tie a host daemon to wherever the repository sits.

Result. The published copy was set up in a temp directory following only docs/install-ubuntu-desktop.md: ten containers healthy, all thirteen logs written and readable, 175/175 smoke, 22 unit suites, and the documented toggle workflow verified by switching two tools off and watching them leave the spec and 404. The live deployment was stopped for it and came back identical — and install.sh's output diffed byte-for-byte against the nine files it had installed before the refactor, the only difference being three lines of comment.

Four things this found that inspection had not.

The release shipped this machine's logs. logs/*.log was added to the exclude list and they stayed anyway: rsync --delete treats excluded files at the destination as protected, so a file published once stays published after it is excluded. Now removed explicitly.

A fresh clone could not start at all. .env is a symlink to configs/owuitools.env and was excluded as though it were the secret. It holds nothing, and it is the only thing docker compose reads for substitution — so every documented command failed on the first required variable. The link ships; make-release.py asserts it is a symlink and that it dangles.

curl http://localhost:8000/health was in seven documents and works nowhere. Nothing publishes a port. Worse, it can appear to succeed: on this host something unrelated answers on 8000 with OK, which reads exactly like a healthy tool server. Replaced with tests/try-tool.sh, which calls from inside the container.

Driving a session by hand did not work, and the documentation said it did. ready is set by the session_ready verb and nothing else, and every verb touching the machine is refused until it is — but vmctl never exposed it. vmctl open then vmctl run returned "the machine is still booting" for as long as you cared to wait, with the guest up and the agent answering guest-ping the whole time. vmctl ready now exists.

Measured: a clean clone reaches 175/175 with no edits beyond the two the documentation asks for; the host templates render with no trace of this machine in any acted-on line and systemd-analyze accepts the results; all thirteen logs are written and operator-readable. Not measured: install.sh on a host that is not this one. Its preflight and its derivations are exercised, but the only end-to-end proof available here is that re-running it reproduces what was already installed.

---

Change 62 — a last read of the thing being published

Symptom. Nothing was failing. But everything verified so far had been verified on the machine that wrote it, and the one full install test had been done before several later edits. A pre-push sweep found four things, three of which only matter to someone who does not already have this working.

What it found.

Following the install guide in order would have failed at step 3. Compose derives a project name from the directory the compose file sits in, so docker compose -f docs/deployment/open-webui.compose.yml up -d creates a network called deployment_default. The tool server joins Open WebUI's network by name, expecting openwebui_default — so the next step dies with "network declared as external, but could not be found". Invisible here, because a real openwebui_default already existed. All three deployment files now pin name:.

vmctl gitea-check does not exist. It was written into docs/setup/gitea.md as the way to verify the Gitea setup and was never a verb. Replaced with a check that runs as owuivmd and asks Gitea for the organisation — which proves the token, the file permissions and the daemon's address filter in one go, and was run here to confirm it works.

The compose header said WHY SIX CONTAINERS over a file with ten, and claimed no host bind mounts beyond read-only config — untrue since ./logs and the agent socket directory. Both rewritten.

Two comments pointed at the pre-configs/ layout, naming gateway/nginx.conf and egress/squid.conf.

How it was checked. Mechanically where possible, because reading does not scale: every tracked file compiled or parsed by shebang; every repo path named in a doc resolved (115 of them, the only absences being files the user creates); every tool name in the docs cross-checked against configs/tools.toml; every vmctl verb against vmctl's own parsers; every OWUI*_ variable against the code. That last set is what caught the fabricated command — a verb that reads plausibly is invisible to a human re-reading their own prose.

Result. 175/175 smoke, 22 unit suites, the isolation and render suites, and the published tree re-validated from a clean copy: ten services resolve, every bind-mount source exists, all three deployment files parse, and its own tests pass. Two example.com strings remain in the repository and both are the image-source URL, kept deliberately.

Measured: the published tree stands up from a clean copy with no edits beyond the two the documentation asks for. Not measured: the install guide end to end on a machine with no Ollama, no Open WebUI and no openwebui_default network — the step-3 failure above was reasoned from Compose's naming rule and fixed, not reproduced.

---

Change 63 — one command that says what is set up, what is off, and what is broken

Symptom. There was no way to ask "is this working" and get an answer. smoke.sh runs 175 cases and prints 175 lines; it tells you whether the suite passes, not whether your installation is ready. Nothing reported the difference between a tool that is broken, a tool that is switched off, and a tool that is switched ON and not set up — and the third is the only one that actually costs anything, because the model is being offered something that will fail.

Change. ./verify.py. It reads the config, asks Docker what is running, and calls each tool once with a known-good argument lifted from the smoke suite. Every tool gets one line; only the actionable ones get a second. Two axes, reported together:

All Set · All Set, Disabled · Needs Setup, Disabled · Needs Setup · Error · Error, Disabled

logs/verify.log holds what was checked and what each probe returned; logs/verify-error-<tool>.log holds one failing tool's story on its own, because when one thing is broken and twenty are fine you want the one. Stale error files are deleted when the tool passes, so one cannot be found later and believed. Exit 0 / 1 (something enabled is failing) / 2 (something needs attention but is switched off).

Five things it got wrong first, all found by running it against a deliberately broken system rather than by reading it.

A disabled tool was reported All Set, Disabled without checking anything — it cannot be probed, because switching it off removes the route. Now its dependencies are checked structurally, which is what makes that label a claim rather than a shrug.

docker compose ps hides stopped containers. With octave stopped the stack still read "All Set, 10 running", because a container that exited simply vanishes from the output. Needs -a, plus a comparison against docker compose config --services to catch a service with no container at all.

Then that fix was too blunt: one stopped container made up false and all 22 tools "Not Checked". Probing needs the api container and nothing else; each tool's own dependency belongs on its own row.

A Python traceback's first line is Traceback (most recent call last):, which is what the terminal showed. The last line is the exception.

And it waited 80 seconds to report TimeoutError for a tool whose container it already knew was down. Checking the dependency before probing turned that into "needs octave, which is exited" in 3.7 seconds — the same finding with the cause in it.

Plus one bad probe of its own: lookup_ip_or_asn was given 203.0.113.10, which the smoke suite uses precisely because the tool REFUSES documentation ranges. It reported the tool broken until the payload became a real address.

Result. On a working installation: 22 tools, 22 All Set, exit 0, about 40 seconds (--quick skips the internet and takes four). On an unconfigured clone: three Needs Setup rows naming the missing token and pointing at the install guide, exit 2 — and that exit code was wrong until stack rows were counted alongside tool rows, which is the one case where a non-zero code is most obviously wanted.

Measured: every status in the table above was produced deliberately - by switching tools off, unsetting OWUITOOLS_CONTACT, and stopping a container - and each rendered correctly with the right log files written and the right exit code. Not measured: vm_agent is checked structurally and always will be; calling it starts a real machine.

---

Change 64 — the published README told the reader to run a file that is not published

Symptom. The repository was pushed to Gitea and the rendered README opened with a section called "Publishing it", instructing the reader to run make-release.py from ~/Documents/owui-tools. That script is on the exclude list precisely because it carries the sweep patterns, and the path is one machine's home directory. So the first file anybody reads ended with an instruction that cannot be followed.

Cause. The section was written for the live tree, where it is correct and useful. Nothing distinguished "documentation of the project" from "documentation of maintaining my copy of it", and the release sweep only rewrites strings - it had no notion that a whole section might not belong.

Change. make-release.py now strips a named list of sections from README.md on the way out, currently just that one. The live tree keeps it.

Worth noting about how it was found. Not by the link checker, which passed: every link in that section was to a heading or a shell command, not to a missing file. Not by the release verifier either, which checks for things that must not be there rather than references to things that are not. It was found by reading the README as rendered on the repository page - which is the one view nobody had looked at, because until there was a repository there was nothing to look at.

Measured: the published README now has eight sections and no mention of make-release.py or of any path under /home. Not measured: whether anything else in the docs is similarly addressed to the maintainer rather than the reader; the strip list has one entry because one is all that has been found.