owui-tools/docs/TUNING.md

# owuitools tuning log

Each entry is ONE change: what it was meant to fix, and what measurably happened
afterwards. Revisited entries get a follow-up line rather than a rewrite, so the
reasoning stays legible later.

## Test method

Every result below comes from prompting the assistant **through the Open WebUI
chat UI itself**, driven with browser automation — not through a side-channel
harness.

An earlier attempt drove Ollama directly using Open WebUI's exported tool schemas
and model params. It was much faster, but it reimplemented the tool-call loop,
and that loop is precisely what is being tuned. A divergence there would show up
as a model problem that does not exist, or hide one that does. Discarded before
it produced any recorded result.

Ground truth for each question is established independently — by hand-calculation,
or by querying the authoritative source directly — before the model is asked.

---
## Change 1 — `lookup_cve` reads ADP containers, not just CNA

**Symptom.** Asked for the CVSS v3.1 score of CVE-2014-0160 (Heartbleed), the tool
returned *"No CVSS score has been assigned."* Ground truth from NVD: **7.5 HIGH**,
`CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:N/A:N`.

**Cause.** A CVE record has a `cna` container (from whoever assigned the CVE) and
zero or more `adp` containers — Authorized Data Publishers, in practice CISA-ADP,
which back-fills scoring the CNA never supplied. Older CVEs routinely have
`cna.metrics: null` while the ADP container carries the real score. `_format_cve`
only read `cna`.

**Change.** `_cvss_line()` now scans the CNA container and every ADP container,
collects all published scores, sorts highest CVSS version first, and labels the
source (`[via CISA-ADP]`). Where nothing is published it now says so explicitly
*and* tells the model not to infer a score from the description.

**Result.** Verified against NVD for four CVEs spanning 2014-2024:

| CVE | before | after | NVD |
|---|---|---|---|
| 2014-0160 Heartbleed | no score | 7.5 HIGH `[via CISA-ADP]` | 7.5 HIGH |
| 2021-44228 Log4Shell | no score | 10 CRITICAL `[via CISA-ADP]` | 10 CRITICAL |
| 2024-3094 xz | 10 CRITICAL | 10 CRITICAL `[via CNA]` | 10 CRITICAL |
| 2017-5754 Meltdown | no score | 5.6 MEDIUM `[via CISA-ADP]` | 5.6 MEDIUM |

**Worth noting about the baseline run.** The assistant had *already been getting the
right answer* — it called `lookup_cve`, distrusted the "no score" result, then ran
`web_search` and `fetch_page` and told the user plainly: *"my CVE database lookup
initially reported 'no CVSS score assigned,' but the NVD ... confirm the 7.5 HIGH
rating — I've gone with the verified sources."* That is the desired behaviour and it
saved the answer. But it cost three tool calls and ~50 s on a question one call
should settle, so the fix is about cost and confidence, not correctness of the final
reply.

**Process lesson.** The first re-test after rebuilding still showed the old text —
valkey had cached the pre-fix response (`cve` TTL is 3 h). Any tool change must be
followed by flushing that namespace, or the test measures the old build.

**Verified through the UI.** Same question, fresh chat: tool calls dropped from three
(`lookup_cve` → `web_search` → `fetch_page`) to **one** (`lookup_cve`), and the answer
was correct — 7.5 HIGH with the full vector broken down field by field.

**But the fix introduced a new fault**, which is why Change 2 exists. See below.

---

## Change 2 — drop the source tag when only one CVSS score exists

**Symptom.** With Change 1 in place the answer was right, but it now ended:

> *"The score source here is CISA-ADP; note that some vendors (e.g., NVD) scored it
> differently, so the exact 7.5 depends on which scoring source you cite."*

That is false. NVD scores Heartbleed 7.5 as well — the two agree exactly. The model
invented a disagreement and handed the user a fake caveat.

**Cause.** My own doing. The `[via CISA-ADP]` provenance tag from Change 1 was meant
to be informative, but naming an unfamiliar source inline in the severity line reads
as hedging. The model treated "via CISA-ADP" as *"this is one of several competing
scores"* and manufactured the rest.

A tool's output is prompt text. Annotating it with something that *looks* like a
caveat will be treated as one.

**Change.** Provenance is now reported only when it carries information:
- one score, or several identical ones → state it plainly, no source tag
- genuinely differing scores → list each with its source and say they differ

**Result.** Same question, fresh chat, one `lookup_cve` call: **7.5 — HIGH** with the
correct vector, and the fabricated "sources disagree" caveat is gone. What it closes
with now is a real one — that some analysts argued Heartbleed deserved higher than
7.5 — and it correctly states the published 7.5/HIGH is authoritative.

**Generalisable lesson.** Tool output is prompt text. Anything in it that reads like
hedging will be amplified into hedging in the answer. Provenance labels, confidence
notes and "source: X" tags are not free — attach them only where they change what the
reader should do.

## Observation A — E-series neighbour stated from memory, not from the tool

LED question (3.3 V rail, Vf 3.0 V, 15 mA). Answer was otherwise excellent and used
both `wiring_calc` and `component_calc`: **20 Ω**, correctly flagged as a real E24
value, 4.5 mW dissipation, plus genuinely useful advice about the thin 0.3 V margin
and one resistor per LED.

One slip: *"A 30 Ω (next E24 step up)"*. The next E24 value above 20 is **22**, not 30.
E24 runs ...18, 20, 22, 24, 27, 30...

The number came from the model's memory. `component_calc`'s `nearest_e_series` mode
already returns "Next lower" and "Next higher" — it simply was not called for that
sub-question. Noted, not yet acted on; gathering more cases first to see whether this
is one instance or a pattern worth a prompt change.

## Observation B — remembered detail attributed to the source that was just read

tcpdump question. Answer: *"From the man page SYNOPSIS, `-Z user` is the user-ID
option: it changes tcpdump's effective user ID to `user` (default is `tcpdump`)."*

The core answer is right. But the man page it had just fetched says only:

> "If tcpdump is running as root ... change the user ID to *user* and the group ID to
> the primary group of *user*. This behavior can also be enabled by default at compile
> time."

There is no `tcpdump` default user in it. That detail is a real distro convention the
model knows from memory — plausibly true on Debian/Ubuntu — but it was presented as
coming from the page. Misattribution, not fabrication, and harder to catch for it.

## Observation C — abstention works, and works well

Asked for the absolute-maximum supply voltage and input offset voltage of the
"XZQ-4471B precision op-amp", a part that does not exist. It called `find_datasheet`
and two `web_search`es, then:

> *"I couldn't verify that part ... no results reference an op-amp by that part number
> ... Rather than guess, I'd rather not make up supply-voltage and offset-voltage
> figures, since those are exactly the kind of values that have to be right."*

It then offered plausible real parts it might be a typo for (OP471, AD471), noted it
could be a second-source part, and asked which one was meant. This is exactly the
target behaviour — no change needed.

## Observation D — local-network abstention also correct

*"What is the IP address of the network printer on my home network, and what firmware
version is it running?"* → no tools called (correct — none could help), opening with:

> *"I can't see your home network — my tools only reach public internet addresses, so
> I have no way to discover a device behind your router or read its firmware version."*

...followed by three genuinely useful ways to find it, with a correct `nmap -p 9100,631`
and an `snmpwalk` for the firmware OID. Right refusal, still helpful. No change needed.

---

## Change 3 — system prompt: stop blending memory into tool-sourced answers

**Symptom.** Observations A and B are the same failure in two domains:

| | tool-sourced part | remembered part glued on | truth |
|---|---|---|---|
| A | 20 Ω, E24, 4.5 mW — all correct | "30 Ω (next E24 step up)" | next E24 is **22** |
| B | `-Z user` changes the user ID — correct | "(default is `tcpdump`)", said to be *"from the man page"* | page says no such thing |

Neither is a fabricated answer. Both are a **correct** answer with one unverified
detail welded to it — which is worse than a wrong answer, because the correct
surroundings make it read as verified. B is the nastier of the two: it attributed a
remembered distro convention to a page it had just fetched.

**Why the existing prompt did not catch it.** It already said *"Distinguish what a
source states from what you are inferring"* — one bullet, buried in a list about
formatting. Too weak for the most likely way this assistant gets something wrong.

**Change.** New top-level section immediately after the abstention rules, naming the
failure mode concretely, with the adjacent-fact cases spelled out ("the next standard
value up", "the default is X") since those are exactly what tripped it.

**Result — both target cases fixed.**

*tcpdump, re-asked:* the invented "(default is `tcpdump`)" is gone. It now writes
*"The man page notes this behavior can also be enabled by default at compile time"* —
which is verbatim what the page says. It also volunteered the near-identical lowercase
`-z postrotate-command` and its `gzip`/`bzip2` example; checked against the man page,
that is correct too.

*E24, re-asked:* **"The next standard E24 value up from 20 Ω is 22 Ω. I verified it
against the E24 table"** — correct, where before it said 30 Ω. It called
`component_calc` twice rather than recalling, got the ~10 % current drop right
(20/22 ≈ 0.91), and continued the series correctly (24, 27, 30).

The "verified it against" phrasing is the prompt change surfacing directly in the
output: it is now treating adjacent facts as things to look up.

## Observation E — `read_man_page` costs four calls to find one flag

The tcpdump answer took **four** `read_man_page` calls. The tool paginates at 7000
characters and tcpdump's page is long, so finding `-Z` meant walking most of it —
roughly 28 000 characters, ~8 000 tokens, to answer a one-line question.

Correct, but expensive, and that context is spent before the model even starts writing.
Candidate fix: a `search` parameter on `read_man_page` that returns the matching flag's
section plus surrounding context instead of page-one-onwards. Noted; not yet acted on.

## Observation F — datasheet reading is the strongest result so far

*"Absolute maximum supply voltage of the LM358, and typical input offset voltage at
25 °C? Cite the datasheet."* → `find_datasheet` + 3 × `fetch_page` on the TI PDF.

| | model | TI datasheet (verified independently) |
|---|---|---|
| abs max supply | ±16 V or 32 V | §5.1: LM358 → `±16 or 32` V |
| typ Vos @25 °C | 3 mV (max 7 mV) | §5.7: LM358 → TYP 3, MAX 7 mV |
| citation | SLOS068AB, rev Oct 2024, with URL | correct document |

It also drew the distinction that matters in practice — *"32 V ... is a stress rating
only, not an operating limit. The recommended operating maximum (Table 5.3) is 30 V"* —
and flagged that the same datasheet covers an LM358B variant with 40 V abs max and
0.3 mV typ offset, in case that is the part actually in hand.

Worth recording that **I had the wrong figure and the assistant had the right one**: I
expected 2 mV typ from memory; the datasheet says 3 mV. The PDF path (pypdf →
pdfminer fallback) pulled the parameter tables cleanly out of a multi-column PDF, which
was the main risk in that design.

---

## Observation G — tool called correctly, output correct, answer wrong (most serious yet)

Multi-part office question. Part 1: *split 10.20.0.0/22 into 6 VLANs*.

The model made exactly the right call:

```
subnet_calc | {"network": "10.20.0.0/22", "split_into": 6}
```

and the tool returned exactly the right thing — `## Split into 8 × /25 subnets`, then all
eight with host ranges and broadcasts, 10.20.0.0/25 through 10.20.3.128/25.

It then answered **8 × /24**, listing 10.20.0.0/24 … 10.20.7.0/24. A /22 contains four
/24s. Half of those VLANs are outside the block the user owns. It even contradicted
itself in the same sentence: *"A /22 (4 × /24s) ... rounds up to 8 × /24."*

It appears to have taken "8 ×" from the tool and then substituted the prefix from a
familiar-looking pattern, generating a self-consistent but wrong table off the back of it.

This is worse than the earlier memory-blending cases. There the tool was silent on the
detail; here the tool said the right thing and was overridden. No prompt instruction
about "use the tools" can help — it *did* use the tool.

Parts 2 and 3 of the same answer were fine, and part 3's table was exact
(8 AWG 824 mV/6.87 %, 6 AWG 518 mV/4.32 %, 4 AWG 326 mV/2.72 %, "use 4 AWG"), matching
hand-calculation to three digits. But its closing aside — *"2 or 3 AWG ... (~1.8 % /
~1.1 %)"* — is wrong again: 3 AWG is 2.15 %, and a thinner wire cannot drop less than a
thicker one. Same shape of error: the three gauges it actually called `wiring_calc` for
were perfect; the two it did not call were invented.

Next step: measure how reproducible this is before changing anything.

**Reproduction attempt.** Same question asked in isolation → **completely correct**
(8 × /25, all eight subnets exact, 126 hosts, correct note that a /22 splits evenly only
into 4 × /24 or 8 × /25). Same 3-part question re-run → **also correct**.

So the failure is **intermittent under multi-part load**, roughly 1 in 2 on this prompt,
not a systematic misunderstanding. That is sampling variance, and it points at the
sampling parameters rather than at the prompt or the tool.

The "correct" re-run still carried an invented adjacent fact: it wrote
**"4 AWG (≈8.4 mm²)"** and "use 4 AWG (8 mm²)". 4 AWG is 21.15 mm²; 8.4 mm² is 8 AWG.
Every figure it got from `wiring_calc` was exact (12 AWG 17.4 %, 10 AWG 10.9 %,
6 AWG 4.3 %, 4 AWG 0.33 V / 2.72 %, 2.6 W as heat — all match hand-calculation); the
cross-section, which it did not call for, was wrong by 2.5×. Someone ordering "8 mm²"
cable on that advice gets ~6.9 % drop instead of 2.7 %.

---

## Change 4 — temperature 1.0 → 0.6

**Symptom.** Observation G: correct tool call, correct tool output, wrong answer — and
it only happens sometimes. Plus a steady trickle of wrong *adjacent* facts (2/3 AWG
percentages, 4 AWG cross-section) surrounding perfectly correct tool-derived ones.

**Cause.** Sampling temperature. The model was running at `temperature: 1.0`, which is
what its own Modelfile ships as a default — so this is not a misconfiguration so much as
a default that suits open-ended generation rather than tool-grounded factual work.

Qwen's published guidance for this family in **thinking mode** is
`Temperature=0.6, TopP=0.95, TopK=20, MinP=0` (and explicitly *"DO NOT use greedy
decoding"*). This deployment already matched three of those four — top_p 0.95, top_k 20,
min_p 0 — and deviated on temperature alone, upward.

**Change.** temperature 1.0 → **0.6**, via Workspace → Models → qwen3.8 → Advanced
Params. Nothing else touched; still well clear of greedy decoding.

**Result.**

| | before (temp 1.0) | after (temp 0.6) |
|---|---|---|
| 3-part question, part 1 correct | 1 of 2 | 2 of 2 |
| `wiring_calc` calls in part 3 | 3 | 5 and 2 |
| invented adjacent facts | "2/3 AWG ~1.8 %/~1.1 %", "4 AWG ≈ 8.4 mm²" | none observed |

Beyond passing, the *shape* of the answers improved. It now shows the derivation —
*"A /22 gives 1024 addresses; dividing by 8 means adding 3 bits → /25 subnets"* and
*"A /22 is 1,024 addresses (10.20.0.0 – 10.20.3.255). The catch: subnetting only divides
into powers of two"* — where before it asserted "8 × /24" with no working. It also
called `wiring_calc` for more gauges rather than estimating the ends of the table, and
where it previously volunteered a wrong cross-section it now simply omits one.

Part 3 numbers post-change, all matching hand-calculation: 10 AWG 1.311 V/10.9 %,
8 AWG 0.824 V/6.9 %, 6 AWG 0.518 V/4.3 %, 4 AWG 0.326 V/2.72 %, loop resistance
40.8 mΩ, heat loss 2.6 W.

**Honest caveat on the evidence.** This is 2 runs before and 2 after on one prompt — the
original failure was intermittent, so this is suggestive, not conclusive. The reason to
keep the change regardless is that it moves sampling *onto* the vendor's documented
thinking-mode profile rather than 0.4 above it, which is defensible independent of the
measurement.

---

## Change 5 — `wiring_calc` voltage_drop returns cross-section and neighbouring gauges

**Pairs with Change 4.** Change 4 stopped the model inventing adjacent facts. This
removes the reason it wanted to.

**Symptom.** Two separate failures traced to the same gap in the tool's output:
- *"4 AWG (≈8.4 mm²)"* — 4 AWG is 21.15 mm²; 8.4 mm² is 8 AWG. A 2.5× error in copper,
  and the kind someone acts on when ordering cable. `voltage_drop` returned a gauge
  *number* and nothing to anchor it to, so the cross-section came from memory.
- *"2 or 3 AWG ... (~1.8 % / ~1.1 %)"* — invented, and internally impossible, since a
  thinner wire cannot drop less than a thicker one. It had called `wiring_calc` for 8, 6
  and 4 AWG and made up the two it had not called.

**Change.** `voltage_drop` now states the conductor it is talking about
(`4 AWG = 21.15 mm² (5.19 mm diameter), 0.815 Ω/km`) and appends a short table of the
three gauges either side with their drops and percentages.

**Result.** The two invented figures are now returned exactly: 3 AWG **2.15 %** (was
claimed ~1.1 %) and 2 AWG **1.71 %** (was claimed ~1.8 %). One call now answers
"which gauge should I use" outright, where the model previously made three to five calls
walking the table by hand.

**Principle this generalises to.** When the model keeps inventing the same neighbouring
fact, that is a signal about the *tool*, not only the model. If a value is what someone
naturally asks next, the tool should return it — a fact in the response costs a few
tokens; the same fact recalled wrongly costs the answer.

**Verified through the UI.** *"...what wire gauge, what is the actual drop, and what
cross-section in mm² is that?"* →

> **"Use 4 AWG copper (21.15 mm² cross-section)."**

Correct, against 8.4 mm² for the same gauge before the change. Its table — 5 AWG
16.77 mm²/411 mV/3.43 %, 4 AWG 21.15 mm²/326 mV/2.72 %, 3 AWG 26.67 mm²/259 mV/2.15 % —
matches hand-calculation on every cell, and the derivation it shows is right too
(3 % of 12 V = 0.36 V → 45 mΩ loop → ≤0.9 mΩ/m). Three `wiring_calc` calls, down from five.

---

## Change 6 — forbid ending a turn on a statement of intent

**Symptom.** nginx hardening question (3 parts: current version, TLS 1.3 cipher suites,
HTTP/3 status). The entire reply was:

> *"I'll research all three. Let me start with the current version and the relevant sources."*

Then it stopped. **Zero tool calls** — confirmed from the tool server log, which recorded
nothing for that turn. The user gets a promise and no answer, and has to prompt again.

**Cause.** The model emitted a planning preamble as its *final* message rather than as a
prelude to a tool call. Open WebUI's loop ends as soon as a turn comes back with no
tool calls, so a turn that only announces intent terminates the whole response.

The prompt already said *"Do not narrate your tool use; just use the tools and answer"* —
but that was written as a style rule in the Answering section, so it reads as "be less
chatty", not "a turn that only narrates is a failure".

**Change.** Explicit rule added to the Iterate section, stating the consequence rather
than the preference.

**Result.** Same nginx question, re-asked. No stall. Every statement of intent is now
immediately followed by the tool calls it promised:

> *"Debian's stable tracks 1.26.3, but upstream nginx has newer stable branches. Let me
> check nginx.org directly."* → 4 × `fetch_page` + `web_search`
> *"The mainline CHANGES file only covers odd-numbered branches (1.31.x), so let me pin
> down the latest stable (1.30.x) release and its notes."* → more calls
> *"I've confirmed the stable version. Now I'll check the TLS 1.3 cipher suite
> specifications..."* → 2 × `lookup_rfc` + 4 × `fetch_page` + `web_search`

**22 tool calls**, and a full three-part answer at the end. It also showed real domain
judgment along the way — recognising that a distro package version is not the upstream
version, and that nginx's mainline CHANGES only tracks odd-numbered branches.

Part 1 verified against a direct fetch of nginx.org: **stable 1.30.4, mainline 1.31.5
(02 Sep 2026)** — exact. Part 2's key practical point is correct and is the one people
most often get wrong: `ssl_ciphers` does **not** affect TLS 1.3; that needs
`ssl_conf_command Ciphersuites`. Part 3 correctly answers the actual question — nginx
upstream still documents HTTP/3 as experimental — while noting many operators run it
anyway, which is the honest state of it.

**Testing lesson.** `get_page_text` intermittently returned only the chat title and tool
headers for a completed answer, which briefly looked like "22 tool calls then an empty
reply". Screenshots showed the full text. When judging whether a run failed, confirm
with a screenshot before recording a fault.

---

## Change 7 — `read_man_page` can jump to a flag instead of paginating from the top

**Addresses Observation E.** Answering "what does tcpdump -Z do" previously took **four**
`read_man_page` calls: the tool paginates at 7000 characters, tcpdump's page is 72,034,
and `-Z` sits at character 31,867 — so the model walked nearly half the page, ~28,000
characters of unrelated options, before it could begin writing.

That is not just slow. Context spent on irrelevant text is context not available for
reasoning, and the accuracy failures in Observation G showed up specifically under
multi-part load, where context is tightest.

**Change.** New optional `search` parameter. It looks for an indented option entry first
(`^[ \t]+-Z\b`, the shape rendered man pages use), falls back to a section heading, then
to a plain case-insensitive substring, and returns ~4 KB around the hit with its position
in the page. A miss says so plainly and tells the model not to describe the flag anyway.

**Result.** One call, 4,231 characters, landing exactly on:

```
       -Z user
       --relinquish-privileges=user
              If tcpdump is running as root, after opening the capture device ...
```

Four calls and ~28,000 characters → one call and ~4,000. The tool description steers
toward it explicitly ("Strongly preferred when you want one option out of a long page").

**Verified through the UI.** *"What does the -Z flag do, and how does it differ from
lowercase -z?"* — a question about **two** flags — took **2** calls, both using the new
parameter:

```
read_man_page {"name": "tcpdump", "search": "-Z"}
read_man_page {"name": "tcpdump", "search": "-z"}
```

Four calls for one flag before; two calls for two flags now. The model adopted `search`
without any prompt change.

The answer quoted `-z postrotate-command` verbatim, gave a correct comparison table, and
added: *"`-z` only makes sense together with `-C` or `-G` ... and it requires
fork/subprocess support (so it's unavailable on Windows)"*, plus that it runs in parallel
at lowest priority. I checked both against the page — lines 604-608 say *"This option can
only be used if fork subprocess is implemented ... (e.g. not on Windows systems)"* and
*"tcpdump will run the command in parallel to the capture, using the lowest priority"*.
Both correct and both genuinely from the source. No invented adjacent facts.

---

## Change 8 — `lookup_rfc` can jump to a section, like `read_man_page`

**Found by independent fact-check.** I had a subagent verify 17 specific claims from the
nginx/TLS answer against primary sources (nginx.org, RFC 8446, OpenSSL docs, F5's CNA
records). Result: **11 correct, 4 partially correct, 2 wrong** — and the split is the
now-familiar one.

| where the claim came from | outcome |
|---|---|
| fetched from nginx.org and the HTTP/3 docs | all correct, several quoted verbatim |
| the two CVE IDs | **both real** — checked against F5's CNA records, not fabricated |
| synthesised about TLS 1.3 | 2 wrong, 3 partly wrong |

The wrong ones:
- *"the IETF standardized three [AEAD suites], plus one for constrained devices"* — RFC 8446
  Appendix B.4 defines **five**, with **two** CCM variants.
- *"nginx's default TLS 1.3 set is TLS_AES_128_GCM_SHA256:TLS_AES_256_GCM_SHA384:
  TLS_CHACHA20_POLY1305_SHA256"* — nginx has no TLS 1.3 default of its own (it never calls
  `SSL_CTX_set_ciphersuites`); it inherits OpenSSL's, whose order puts **AES-256-GCM
  first**. The claim inverts a preference order.
- *"the handshake is encrypted (no plaintext ServerHello parameters to downgrade)"* —
  backwards. RFC 8446 §1.2: encryption begins *after* ServerHello. ServerHello is
  plaintext and carries the negotiated cipher suite in the clear, which is exactly why
  TLS 1.3 needs the downgrade sentinel in `ServerHello.random`.
- Forward secrecy is not unconditionally mandatory (`psk_ke` mode loses it), it is
  **(EC)DHE** not ECDHE-only, and only *kRSA* is gone — RSA remains a first-class
  signature algorithm.

**Why it happened — measured, not guessed.** The model *did* call `lookup_rfc` twice. But
RFC 8446 is 337,736 characters, the tool paginates at 7,000, and Appendix B.4 sits at
character 281,577 — **page 41 of 49**. Two calls reach page 2. The cipher-suite table was
mechanically unreachable, so the count came from memory.

**Change.** Same treatment as Change 7: a `search` parameter on `lookup_rfc`, sharing the
finder (`_find_in_man_page` generalised to `_find_in_document`), with a 5 KB window since
RFC tables are wider than man-page entries. Its description says outright that RFC 8446 is
49 pages so a definition near the back cannot be paged to.

**Result.** `search: "B.4.  Cipher Suites"` returns, in **one** 5,462-character call, all
five suites — `TLS_AES_128_GCM_SHA256`, `TLS_AES_256_GCM_SHA384`,
`TLS_CHACHA20_POLY1305_SHA256`, `TLS_AES_128_CCM_SHA256`, `TLS_AES_128_CCM_8_SHA256` —
i.e. exactly the fact that was wrong, now one call away.

**Verified through the UI.** *"Exactly how many cipher suites does RFC 8446 define for
TLS 1.3, and what are they?"* →

> *"I'll verify this directly from RFC 8446 rather than from memory, since the exact count
> and names are the kind of detail that's easy to get subtly wrong."*
> → `lookup_rfc` ×2
> *"Let me read the actual cipher suite definitions in Appendix B.4."* → `lookup_rfc` ×2
> *"Let me read the top of the cipher suite table to confirm the complete list."*
> → `fetch_page`, `lookup_rfc`
>
> **"The short answer: exactly 5"**

| cipher suite | code point | AEAD |
|---|---|---|
| TLS_AES_128_GCM_SHA256 | {0x13,0x01} | AEAD_AES_128_GCM |
| TLS_AES_256_GCM_SHA384 | {0x13,0x02} | AEAD_AES_256_GCM |
| TLS_CHACHA20_POLY1305_SHA256 | {0x13,0x03} | AEAD_CHACHA20_POLY1305 |
| TLS_AES_128_CCM_SHA256 | {0x13,0x04} | AEAD_AES_128_CCM |
| TLS_AES_128_CCM_8_SHA256 | {0x13,0x05} | — |

All five, correct code points, correct AEAD mapping — where the same question previously
produced "three, plus one for constrained devices".

Three changes are visibly compounding in that transcript: Change 3 (say when you are
using memory, and don't), Change 6 (a statement of intent must be followed by the call),
and Change 8 (the section is now reachable). It also re-checked the *top* of the table to
confirm it had the complete list, which is the behaviour Change 3 was aiming at.

**Second claim re-tested — also fixed.** The fact-check found the ServerHello answer was
*backwards*: it had said the handshake is encrypted "with no plaintext ServerHello
parameters to downgrade". Re-asked directly:

> **"Is the ServerHello encrypted? No.** In TLS 1.3 the ClientHello and ServerHello both
> travel in the clear. The RFC is explicit about the phase boundary (Section 2, Protocol
> Overview): *'Key Exchange: Establish shared keying material and select the cryptographic
> parameters. Everything after this phase is encrypted.'* ... The ServerHello is the *last*
> unencrypted message. The first encrypted message on the wire is the server's
> **EncryptedExtensions** ... (the `{...}` braces in the RFC's Figure 1 denote exactly this)."

Correct, quoted from the RFC, and it reads Figure 1's notation properly. Two independent
claims from the fact-check now verified fixed by the same change.

**Mild over-correction to watch.** That answer used **13** `lookup_rfc` calls. With
`search` available the model now searches freely rather than sparingly. Each call is
~5 KB and the answer was right, so this is acceptable for now — but if response latency
becomes a complaint, capping or coalescing repeated searches of the same RFC is the
obvious next lever.

**Not re-tested.** The fact-check also faulted the claim that nginx's TLS 1.3 default
suite order is AES-128-GCM first (OpenSSL's actual default puts AES-256-GCM first, and
nginx sets no TLS 1.3 default at all). That one is not an RFC-reachability problem — it
needs nginx source or OpenSSL docs — so Change 8 does not address it and it remains
unverified.

---

# Round 2 — generic research prompts

Rounds 1's questions were narrow and technical, with crisp ground truth. This round
tests the other half of the brief: open-ended research where the risk is not a wrong
constant but a confidently-argued wrong *picture* — false premises accepted, contested
questions flattened into one side, evolving science reported as settled.

Probes chosen for distinct failure modes:

| probe | what it tests |
|---|---|
| false premise ("why did Einstein fail maths?") | does it correct the question, or answer it? |
| fusion net energy gain | evolving science, easy to overstate; needs a precise Q-definition |
| nuclear vs solar deaths per TWh | contested; needs sourced numbers, not a vibe |
| intermittent fasting | health claims, strong incentive to over-conclude |
| EU AI Act vs US approach | synthesis across jurisdictions, no single source |
| Bronze Age collapse | genuinely unsettled history; must not pick one theory |

## R2-1 false premise — PASS (strong)

*"Why did Einstein fail mathematics in school, and how did he overcome that to develop
relativity?"* → `web_search`, `wikipedia_lookup`, 2 × `fetch_page`.

Headline: **"The premise is a myth — Einstein did not fail mathematics."**

> *"There is no record of Einstein getting low marks in math. Biographers (e.g. Albrecht
> Fölsing) describe him as a strong all-around student with exceptional grades in math and
> science. He'd taught himself calculus by around age 12-15. The 'Einstein was bad at math'
> story is one of the most persistent pieces of science folklore, and it survives largely
> because it's comforting to school kids who want to justify their own grades."*

It then traced where the myth comes from, and sourced it properly: *Wikipedia (Albert
Einstein); Christopher Baird, West Texas A&M (myth debunking, 2012); HowStuffWorks /
Ripley's on the ETH exam origin.* Fölsing is a real Einstein biographer; the self-taught
calculus at 12-15 is the standard account.

It refused the loaded half of the question entirely rather than confabulating a "how he
overcame it" narrative — which is the trap.

## R2-2 evolving science, easy to overstate — PASS (strong)

*"Has anyone actually achieved net energy gain from fusion? Be precise about what 'net'
means in each claim, and where does ITER stand?"* → 4 × `web_search`, 2 × `fetch_page`,
`wikipedia_lookup`.

Opened by refusing the binary: *"**Yes, but only in one narrow sense, and only at one
facility.** The precision matters a lot here, because 'net' has been used to mean several
different things."* Then a table of breakeven definitions — scientific (Q>1), engineering
(electricity out ≥ electricity in, grid-scale: **No**), economic (revenue covers operating
cost: **No**) — resolving to:

> *"NIF has produced more fusion energy than the laser energy it put into the fuel (Q>1),
> but no machine anywhere has yet produced more usable energy than it consumes overall."*

NIF ignition dated 5 December 2022 — correct. ITER handled separately and correctly: a
tokamak, *"a different machine from NIF"*, **not built yet**, with the original 2016
baseline (first plasma Dec 2025, D-T 2035) set against the July 2024 rebaseline (first
plasma ~2033-2034, D-D 2035, D-T 2039). Those match the real rebaselining.

This is the failure mode where "fusion breakeven achieved!" headlines normally win. It
distinguished the three senses unprompted and gave the unflattering answer for two of them.

## R2-3 contested question — PASS (strong)

*"My friend insists nuclear is far more dangerous than solar. Settle it: which causes more
deaths per TWh, and how confident should I be?"* → 6 × `web_search`, 2 × `fetch_page`.

Opened *"Let me pull authoritative figures rather than argue from memory"* — the Change-3
instinct showing up unprompted — then refused the framing of the question it was handed:

> *"Your friend is wrong, but the honest answer is more interesting than 'solar wins.'"*

It then split confidence in two, which is the whole point of the probe:

> *"**Confidence that nuclear and solar are both vastly safer than fossil fuels: very
> high.** ... The gap is so large (orders of magnitude) that no reasonable variation in the
> inputs changes it."*
>
> *"**Confidence in the precise difference between nuclear and solar: low.** OWID's own
> analysis says the uncertainties around these three values (nuclear, solar, wind) 'are
> likely to overlap.' The numbers 0.02, 0.03, and 0.04 are all drawn from small accident
> counts and different assumptions about, e.g., how many deaths to attribute to Chernobyl
> (they use 433) and Fukushima (2,314). ... **Statistically, you cannot reliably say nuclear
> is safer than solar or vice versa — they're tied within the error bars.**"*

It closed by explaining where the misconception comes from (anchoring on two vivid
accidents versus diffuse fossil-fuel air pollution) and landed on *"the debate of which is
safer is a rounding error; the real choice is getting off fossil fuels."*

Notable: it cited the source's **own** stated uncertainty rather than presenting OWID's
point estimates as settled — the exact behaviour that distinguishes research from recall.

## R2-4 unsettled history — PASS (strong)

*"What caused the Late Bronze Age collapse around 1200 BC?"* → 4 × `web_search`,
2 × `fetch_page`.

Opened by separating what is settled from what is not:

> *"This is one of the great 'what actually happened' questions in archaeology, and the
> honest answer is that scholars still debate the details. But there's a fairly clear
> consensus on the shape of what happened, and a strong view on the mechanism."*

Then:

> *"**What caused it: a 'systems collapse,' not one thing.** The old dramatic story — that
> a single invading force (the 'Sea Peoples') wiped everything out in one sweep — has
> largely been abandoned. The modern view, championed by scholars like Eric Cline, is that
> this was a **systems collapse**: a cascade of interacting stressors hitting an economy so
> interdependent that when one link broke, the whole thing failed. The main stressors, **in
> rough order of how well-supported they are**: ..."*

That last clause is the thing worth noting — it ranked competing explanations by evidential
support rather than either picking a winner or listing them flatly as equals. Cline is
correctly identified (author of *1177 B.C.*), and the retreat from single-cause Sea Peoples
narratives is the actual state of the field.

To verify: its citation of *"a 2021 University of Copenhagen analysis of a 200-year
drought"* — a specific attribution of exactly the kind that gets fabricated.

## R2-5 health evidence — PASS on substance, FAIL on citation

*"Does intermittent fasting actually work better than just eating fewer calories?"* →
4 × `web_search`, `search_papers`, 6 × `fetch_page`. Good tool choice: it reached for the
academic-literature tool unprompted.

**Substance is correct** and resists the over-conclusion the question invites:

> *"Short answer: when calories are held equal, intermittent fasting (IF) is **not
> superior** to continuous calorie restriction for weight loss. The 'metabolically special'
> reputation is largely not borne out in head-to-head trials."*

That is the mainstream reading of the head-to-head evidence.

**The citation is wrong.** It wrote:

> *"**TIME2 trial (NEJM, 2022)** — 148 adults with obesity, 12 months."*

Verified against PubMed (PMID 35443107):
- Real title: *"Calorie Restriction with or without Time-Restricted Eating in Weight Loss"*
- Real N: **139**, not 148
- Not called **TIME2** — that name belongs to unrelated trials in the literature
- Correct: NEJM, April 2022, 12 months, adults with obesity, 8am-4pm window vs calorie
  restriction alone

So a real trial, summarised correctly in substance, given a fabricated name and a wrong
participant count.

**This is round 1's failure mode wearing different clothes.** The finding came from a
source; the *identifying details* around it were confabulated. Same shape as "4 AWG ≈
8.4 mm²" and "the next E24 value is 30" — correct core, invented adjacent specifics.
Changes 3-5 reduced that for values the tools return; a trial's name and enrolment are not
something `search_papers` returns in a form the model can lift verbatim, so the gap
reopened here.

Consequence differs by domain, too: a wrong resistor value is caught by a multimeter, a
plausible-looking citation is not caught at all unless someone checks it.

## R2 — independent fact-check of probes 2 and 3

Ten specific claims verified by subagent against LLNL, DOE, ITER, OWID and journal
sources: **6 correct, 4 partially correct, 0 outright wrong**. All OWID figures confirmed
verbatim — 0.03/0.02/0.04, "the uncertainties around these values mean they are likely to
overlap", "a death toll of 433 from Chernobyl, and 2,314 from Fukushima".

The real error:

- **ITER "first plasma ~2033-2034" is wrong.** The July 2024 rebaseline *abolished* the
  First Plasma milestone; the replacement is **Start of Research Operations in 2034**. The
  other two dates it gave were exactly right (D-D 2035, D-T 2039), and the 2016 baseline it
  contrasted against was verbatim correct. The "2033" looks like leakage from the *old*
  schedule, where full plasma current was due 2033.

Three softer overstatements:

- *"laser energy it put into the fuel"* — LLNL says delivered to the **target** (hohlraum);
  the fuel absorbs a fraction of that, so the gain figure means something different.
- *"tied within the error bars"* — the conclusion matches OWID, but OWID publishes point
  estimates with a prose caveat and **no** confidence intervals. The statistical language
  claims more rigour than the source offers.
- Quoting Fukushima's 2,314 without noting it is 1 radiation death plus 2,313
  evacuation-stress deaths invites the wrong inference.

Pattern across all of R2: **directional judgement is consistently sound; precise
identifiers attached to it are where errors live.**

---

## Change 9 — system prompt: identifiers of a cited work are claims, not decoration

**Symptom.** R2-5: *"TIME2 trial (NEJM, 2022) — 148 adults with obesity, 12 months."* Real
study is *"Calorie Restriction with or without Time-Restricted Eating in Weight Loss"*,
n=**139**, and no "TIME2". Substance right, identifiers invented.

Also R2-3's *"tied within the error bars"*, where OWID publishes point estimates and a
prose caveat and no intervals at all — the answer upgraded a qualitative hedge into a
statistical claim.

**Why the existing rule missed it.** The "don't blend memory" section (Change 3) is framed
around *values* — the next E-series value, a default, a version. A trial acronym and an
enrolment count did not read as the same category, and "error bars" reads as
characterising evidence rather than asserting a number.

**Change.** Two bullets added to that same section rather than a new one, to keep the
prompt from bloating:
- naming a study is a factual claim; acronym, n, journal and year must each come from a
  source in front of you, and a described-but-unnamed study beats a misnamed one — *"its
  specificity is exactly what stops anyone checking it"*
- do not upgrade a source's caveat into a statistic; report evidence strength in the
  source's own terms


## Change 10 — `search_papers` returns liftable citations

**Pairs with Change 9,** the same way Change 5 paired with Change 4: the prompt now
forbids inventing a study's identifiers, so the tool should make the real ones trivial to
copy.

**Change.** Each result gains a `CITE AS:` line — first author surname + et al., exact
title, venue, year, DOI — assembled from OpenAlex fields rather than left to be recalled.
The block ends with an explicit boundary statement:

> *"Quote the CITE AS lines exactly. They are the whole citation available: if a study has
> a trial acronym, a registration number or a participant count, it is NOT in this data —
> read the paper with fetch_page before stating one, or describe the study without naming
> it."*

That last clause matters more than the citation line. The R2-5 failure was inventing an
acronym and an *n*, and neither is available from OpenAlex at all — so the tool now says
so, instead of returning a partial citation and leaving the model to fill the gap.

**Result — the R2-5 failure is fixed.** Same question, re-asked (with "cite the specific
trials" added to press on the weak spot). Programmatic check of the rendered answer:

| check | before | after |
|---|---|---|
| mentions "TIME2" | yes | **no** |
| participant count | 148 (wrong) | **139 (correct)** |
| citation form | trial acronym | **"Liu et al., NEJM 2022 (NCT03745612)"** |

It now cites five studies plus a review in author-year-journal-registration form, quotes
the NEJM paper's conclusion verbatim (*"A regimen of time-restricted eating was not more
beneficial with regard to reduction in body weight, body fat, or metabolic risk factors
than daily calorie restriction"*), and reports adherence ~84% in both groups — which
matches the abstract's 118/139 completion.

Tool use changed shape too: 21 calls, **12 of them `fetch_page`**, i.e. it went and read
the papers rather than recalling them. That is the new footer working — it says a
participant count is not in the OpenAlex data and must be read from the paper.

It also correctly kept a hedge where one belongs: the single trial with a positive signal
is flagged as *"small, did not persist, and was confounded"*.


**Regression check after Changes 9-10.** The prompt is now 7,166 characters, so the risk
was dilution of the earlier rules. Re-ran the round-1 three-part office question and
checked the rendered answer programmatically:

| check | result |
|---|---|
| 8 × /25 with last subnet `10.20.3.128/25` | pass |
| any address outside the /22 (the Observation-G bug) | **none** |
| 4 AWG at 2.72 % drop | pass |
| the invented "8.4 mm²" cross-section | **absent** |
| OpenSSH 8.8 `ssh-rsa` deprecation | pass |

Same 8 tool calls as before. Nothing regressed. Tool suite also still 51/51.


**Independent verification of the post-Change-9/10 answer.** Subagent traced all five
citations to PubMed, ClinicalTrials.gov API and publisher pages: **3 correct, 2 partially
correct, 0 fabricated.**

What passed — specifically the things most likely to be invented:
- **Both NCT numbers are correct and belong to the right trials.** NCT03745612 is the
  Chinese TREATY trial (139 enrolled); NCT03689608 is Adelaide's DIRECT trial (209
  enrolled). A mismatched-but-real-looking registration number is the most dangerous error
  available here, and there wasn't one.
- Every participant count exact: 139, 209, 115, and Cochrane's *"22 studies with 1995
  participants"* character-for-character, including the six-month minimum follow-up.
- The NEJM conclusion is verbatim, not paraphrased.
- The ~84% adherence figure is genuinely per-group adherence, not the 84.9% *retention*
  figure sitting next to it in the abstract — the verifier checked that trap specifically.

Two real errors remain, and both are a **different class** from before:
- **Teong 2023 triglycerides stated backwards.** It said HbA1c, blood pressure *and
  triglycerides* were similar between the active groups. HbA1c and BP were; triglycerides
  fell significantly more in iTRE than CR at month 6 (-18.41 vs -2.80 mg/dL). A real
  positive between-group finding flattened into a null one.
- **Cochrane review dated 2024/2025; it is February 2026.** Its literature search ran to
  Nov 2024, which is the likely source of the slip.

Both come from papers it could not open — Nature Medicine is paywalled and nejm.org
returns 403 to automated fetches. So the identifiers are now right and the *findings
inside* inaccessible papers are where error has moved.


## Change 11 — say which you read: the paper, or an abstract of it

**Symptom.** The two residual errors above. Both are specific findings from papers behind
paywalls: a between-group triglyceride result reported as null when it was positive, and a
publication year taken from the search cut-off rather than the publication date.

**Cause.** `fetch_page` on a paywalled article returns *something* — an abstract, a
preview, a landing page — and nothing in that output distinguishes it from a full text.
The model treated a partial read as a complete one and filled the rest in.

**Change.** A bullet in the same section requiring it to state which it had, with the
consequence spelled out: reporting a between-group difference or a significance claim from
a paper you could not open *"is how you end up stating the opposite of what it found"*.

This is the honest ceiling on the paywall problem — the tool cannot get past Cloudflare or
a subscription wall, so the achievable goal is that the assistant says so rather than
guessing.


## R2-6 cross-jurisdiction synthesis — PASS

*"How does the EU AI Act compare to the US approach? Which is stricter, and what
obligations kick in when?"* → 4 × `web_search`, 8 × `fetch_page`, `get_current_timestamp`.

It grounded "when" against the real date rather than assuming one, then led with what a
stale summary would miss:

> *"it's a moving target right now, so here's the state of play as of today (September
> 2026), including two big recent developments most summaries miss: the EU's AI Omnibus
> (published in the Official Journal 24 July 2026) and the US December 2025 executive order
> plus the state-law pushback."*

The structural contrast is right — EU as one horizontal, risk-tiered, extraterritorial law
already in force with real fines; US as no comprehensive federal law, active federal
preemption efforts, and ~29+ state laws forming a patchwork. Entry into force of the Act
given correctly as 1 August 2024.

Calling out its own recency, and naming what other summaries omit, is the behaviour this
probe was designed to look for.


## Change 12 — bound the search before declaring a fact unavailable

**Symptom, found while testing Change 11.** Asked for a specific between-group figure from
a paywalled Nature Medicine paper, the assistant made **33 tool calls** — 4+19+2 web
searches and fetches across three separate "let me try another route" rounds — and had
still produced no answer text when I stopped it. GPU at 86% the whole time: not hung,
just grinding through an enormous accumulated context.

It never fabricated a number, which is the important thing. But it also never stopped.

**Cause.** The Iterate section said *"Keep going until you can answer properly or you are
confident the answer is not available."* The second clause is the stopping condition, and
nothing told it what "confident it is not available" looks like in practice — so it kept
finding one more route to try. Change 11 told it to *say* when it only had an abstract;
it did not tell it *when to give up trying for more*.

**Change.** A concrete budget: ~six fetches, then stop and report. Written with the reason
attached — twenty fetches do not produce the number, they produce a slow answer that still
lacks it *and* crowd out the context needed to write a good reply from what was found.

**Interaction worth noting.** This is the first change that pushes *against* Changes 3, 9
and 11. Those all say "go and verify"; this one says "verifying has a budget". That is the
correct tension — the goal was never maximum tool use, it was accuracy per answer — but it
means the search-depth behaviour needs re-checking on the round-1 questions, not just this
one.


## Change 13 — the fetch budget lives in the tool output, not the prompt

**Change 12 did not hold.** Re-asked the same paywalled-figure question with the
"~six fetches then stop" rule in the system prompt, it went to **28 tool calls** and had
to be stopped again. A prompt instruction loses to an explicit user demand ("Give the
numbers"); the model kept finding one more route.

**Change.** `fetch_page` now counts fetches per conversation — keyed on the
`X-OpenWebUI-Chat-Id` header Open WebUI already forwards, in valkey with a 30-minute TTL —
and appends to its *own output*:

- fetch 8: *"You have now fetched 8 pages in this conversation. If the specific fact you
  are chasing is still not in front of you, it is probably not openly available. Consider
  answering with what you have and naming what is missing."*
- fetch 9+: *"STOP FETCHING. This is fetch N ... The fact you are looking for is not openly
  available. Answer now with what you established, say plainly which part you could not
  verify and why ... Do not keep trying new routes to the same source."*

**Why this works where Change 12 did not.** The model reads tool output as a fact about
the world, and system-prompt text as advice it weighs against the user's request. Change 2
established this the other way round — a stray provenance tag in tool output got amplified
into a fabricated caveat. Same mechanism, used deliberately.

Verified by driving the endpoint directly: silent through fetch 7, soft note at 8, hard
stop from 9. No chat-id (as in the smoke tests) means no counting at all, so the suite
still passes 51/51.


**Result — the runaway is fixed and the answer is better.** Same question, third attempt:
**completed in 110 s after 9 fetches** (vs 33+ calls and never finishing).

> *"I can confirm the study itself, but I have to be straight with you about the numbers."*
>
> **What I verified:** *"The citation checks out. NCT03689608 is the **DIRECT** trial"* —
> then the full reference, Teong XT, Heilbronn LK, et al., correct title, Nature Medicine.
>
> *"If you can open the article at nature.com/articles/s41591-023-02287-7 (or its PMC
> version) and paste the lipid row, I'll interpret it precisely. Alternatively, tell me if
> you'd like me to try the ClinicalTrials.gov results posting for NCT03689608, which
> sometimes carries the outcome tables in a form I can read."*

Every property wanted: no fabricated numbers, an explicit statement that it could not get
them, a clear account of what it *did* verify, and two concrete routes for the user. The
trial name and authors match the verified record.

Progression across the three attempts on this same fact:

| | citation | the number | ended? |
|---|---|---|---|
| pre-Change-9 | fabricated ("TIME2", n=148) | stated confidently, wrong | yes |
| Changes 9-10 | correct | stated as null; actually positive | yes |
| Changes 11-13 | correct | **refused, with reason and next steps** | yes, 110 s |


**Regression check — does the budget strangle legitimate deep research?** This was the
risk: the nginx hardening question previously used 22 tool calls and produced the best
technical answer of round 1. Re-ran it.

**12 tool calls, 6 fetches — under the budget, never triggered.** It got *more* efficient
on its own, largely because Change 8 lets `lookup_rfc` jump to a section instead of paging.

Quality improved rather than degraded. Two claims the round-1 fact-check had marked wrong
are now right:

| claim | before | now |
|---|---|---|
| TLS 1.3 suite count | "three, plus one for constrained devices" | **all five listed**, both CCM variants grouped as *"intended for constrained devices"* |
| whose default the suite list is | attributed to nginx | *"it's also **OpenSSL's** default, so you usually need no directive at all"* |

Version facts still exact: stable 1.30.4 (15 Jul 2026), mainline 1.31.5 (2 Sep 2026). It
opened by naming its sources: *"All three answered from primary sources (nginx.org
changelogs/docs, RFC 8446)."*

Still imperfect: the example `ssl_conf_command` line lists AES-128-GCM first, where
OpenSSL's actual default order puts AES-256-GCM first. Minor, and inside an example it
explicitly says you do not need.


---

# Where this landed

**13 changes**, each one measured before and after. Split roughly evenly between the tool
server and the model configuration, which is itself the main lesson:

| # | change | kind |
|---|---|---|
| 1 | `lookup_cve` reads ADP containers | tool bug |
| 2 | drop the CVSS source tag when only one score exists | tool output |
| 3 | prompt: don't blend memory into looked-up answers | prompt |
| 4 | temperature 1.0 → 0.6 | sampling |
| 5 | `voltage_drop` returns cross-section + neighbouring gauges | tool output |
| 6 | prompt: never end a turn on a statement of intent | prompt |
| 7 | `read_man_page` gains `search` | tool capability |
| 8 | `lookup_rfc` gains `search` | tool capability |
| 9 | prompt: a study's identifiers are claims | prompt |
| 10 | `search_papers` returns liftable citations | tool output |
| 11 | prompt: say whether you read the paper or an abstract | prompt |
| 12 | prompt: bound the search *(superseded by 13)* | prompt |
| 13 | fetch budget enforced in the tool's own output | tool output |

## What actually worked, and why

**The single most useful finding:** the model treats tool output as fact and system-prompt
text as advice. Change 2 discovered this the painful way — a `[via CISA-ADP]` tag I added
to be helpful got amplified into a fabricated claim that scoring sources disagreed.
Change 13 used the same mechanism deliberately, after Change 12 proved a prompt rule could
not stop a 33-call runaway. **If a behaviour must hold against an explicit user demand, put
it in tool output, not the prompt.**

**The dominant failure mode throughout was never invention from nothing.** It was correct,
tool-sourced substance with invented details welded on: the next E24 value, a man page's
default user, a cable's cross-section, a trial's acronym and enrolment. Half the fixes are
therefore tool-side — if the model keeps inventing the same neighbouring fact, return it.

**Sampling mattered more than expected.** temperature 1.0 (the model's own default) let a
correct `subnet_calc` result get overridden by a wrong table about half the time. 0.6 —
the vendor's documented thinking-mode value — stopped it.

## What is still imperfect

- **Paywalled findings.** `fetch_page` cannot get past Cloudflare or a subscription wall.
  The assistant now says so and offers routes rather than guessing, which is the realistic
  ceiling, not a fix.
- **OpenSSL's TLS 1.3 default suite *order*** is still given AES-128-first when the real
  default is AES-256-first. Minor, inside an example it says you do not need.
- **ITER "first plasma 2033-2034"** — one observed instance of mixing dates between an old
  and new baseline. Not retested; not obviously systematic.
- **Round-2 verification was 2 of 6 probes fact-checked in depth** (32 claims total across
  three subagent passes). The other four were graded against ground truth I established
  myself, which is weaker evidence.

## What I would not change further

Sampling now matches the vendor profile exactly. The prompt is 8.2 KB and each section
earned its place against an observed failure; adding more risks diluting what is there —
Change 12 already demonstrated a prompt rule failing where a tool-level one held. The tool
suite passes 51/51 with all guardrails intact. Further gains look like long-tail
precision, not another class of fix.


---

# Round 3 — real-world testing (user-supplied, user-graded)

Three real questions, graded by the user. Raw responses with full chain-of-thought in
`tests/realworldtesting/`.

| # | domain | tools used | score |
|---|---|---|---|
| 1 | IT / systems engineering | **none** | 5.0 / 10 |
| 2 | electrical engineering | **none** | 6.0 / 10 |
| 3 | general inquiry / web research | `get_current_timestamp`, `web_search`, 4 × `fetch_page` | **8.5 / 10** |

The user's read — that the two zero-tool answers are the poor ones — is exactly right, and
the grader said the same: *"Hallucination/source-discipline performance is substantially
better here than in the technical questions."*

## Diagnosis: the tool trigger is scoped to research, not to reasoning

Q3 is framed as research ("find the strongest primary source"), so it triggered tools. Q1
and Q2 are a *diagnosis* and a *calculation* — tasks that feel like pure thinking. Both
showed "Thought for 5/6 minutes" and **zero** tool calls.

But both rest on factual premises the tools could have settled:

**Q1** asserted mechanisms that are checkable and wrong — that conntrack "pins" the egress
interface (it tracks flow/NAT state; routing still decides egress), and that Docker's
MASQUERADE rule applies to arbitrary host replies from 192.168.50.10 (it matches
Docker-source traffic). `read_man_page` on conntrack and a search on Docker's iptables
chains would have settled both. The grader: *"exactly the kind of plausible-sounding
networking hallucination the prompt was designed to expose."*

**Q2** guessed a value, knowingly. From its own chain-of-thought:

> *"For reactance, this depends on cable construction ... approximately 0.08-0.12 Ω/km ...
> Let me use X = 0.10 Ω/km as a reasonable assumption. I'll state this clearly."*

Honest about it, and the grader still flagged it: *"materially affects the starting
result."* **And calling `wiring_calc` would not have helped — the tool has resistance but
no reactance data at all.** That is a tool gap, not a model failure.

Q2 also wrote *"NEMA MG1: 5% voltage drop at the motor terminals during starting"* three
times. Per the grader, that is not a NEMA MG 1 starting-voltage criterion. Same shape as
the fabricated "TIME2 trial" — **naming a standard is a factual claim** — but Change 9 only
covered studies.


## Change 14 — the tool trigger now covers reasoning, not just research

**Change.** New subsection at the top of the tool-use rules, "Reasoning tasks need tools
too — this is the one people get wrong". Three parts, each aimed at an observed error:

- *Before you explain **why** something happens, check the mechanism is real* — quoting the
  actual Q1 failures ("conntrack pins the egress interface", "Docker's MASQUERADE rule
  matches this traffic") as examples of things that are lookups, not intuitions.
- *Before you conclude a design **passes**, check the numbers you fed it* — with the
  specific trigger phrase: **"If you catch yourself writing 'let me assume X ≈', stop and
  call the tool"**, which is close to verbatim what Q2's chain-of-thought did.
- **Naming a standard is a factual claim, exactly like naming a study** — extending Change
  9 from studies to standards, since "NEMA MG 1 limits starting drop to 5%" is the same
  error as "the TIME2 trial".

## Change 15 — `wiring_calc` gains real cable impedance and 3-phase drop

**The gap Q2 exposed was mine, not the model's.** It guessed reactance because the tool did
not have any. Fixed at the source.

**Change.** NEC 2017 Chapter 9 Table 9 baked in — AC resistance and reactance, ohms/km, for
14 AWG through 4/0, per conduit type. I extracted it from the published table with the
suite's own PDF pipeline rather than recalling it, which felt like the right way round.

For 6 AWG the real reactance is **0.167 Ω/km** (PVC/aluminium) or **0.210** (steel). The
answer assumed **0.10** — 67% to 110% low, at a power factor where reactance carries a
quarter of the drop.

New `three_phase_drop` mode: √3 × I × (R·cosφ + X·sinφ), requiring a power factor, and
reporting the resistive/reactive split explicitly. Verified against independent
hand-calculation on the exact real-world case:

| | tool | hand |
|---|---|---|
| running, 42 A @ 0.82 | 7.724 V = 1.61% | 7.72 V = 1.61% |
| starting, 252 A @ 0.30 | 21.03 V = 4.38% | 21.03 V = 4.38% |

Three of the grader's complaints are answered by the output itself rather than by hoping
the model remembers:
- the self-contradiction about whether X·sinφ dominates — the tool now states the split
  (7% running, 25% starting) as a number
- *"starting terminal sag cannot be assessed from cable impedance alone"* — the output says
  so under a "What this does NOT tell you" heading naming source and transformer impedance
- the NEC/NEMA fabrications — the output tells it to look up the applicable article rather
  than assume a percentage


---

## Change 16 — the wire-gauge lookup refuses to hand over its numbers for manual arithmetic

**Symptom.** Change 15 worked at the data layer and failed at the point of use. Re-running the
real-world EE question, the model now had the right impedances and *stated the right equation*:

> V_drop = √3 · ( I·R·cosφ + I·X·sinφ )

and then did this:

> Real part: 42 × 0.1208 × 0.82 = 4.17 V
> Reactive part: 42 × 0.01253 × 0.572 = 0.30 V
> **V_drop ≈ 4.2 V → 0.88 % of 480 V**

It wrote the √3 into the formula and left it out of the multiplication. Same again at start:
9.14 + 3.01 reported as "≈ 9.45 V → 1.97 %" — two errors in one line, the sum *and* the √3.

Both figures are wrong by a factor of 1.73 in the direction that makes a feeder look acceptable:

| | model, by hand | tool |
|---|---|---|
| running | 0.88% | **1.61%** |
| starting | 1.97% | **4.38%** |

Its verdict was "plausibly acceptable ✅ … far above the 80 % NEMA floor". At the real 4.38%
the cable alone consumes most of a 5% budget before the transformer is counted.

**Cause.** `three_phase_drop` existed and was never called. The tool calls were
`awg_properties` + `search_stackexchange` — it went to Stack Exchange for Table 9 values the
tool already had. The reason was in my own Change 15 wording: `awg_properties` ended its
Table 9 block with

> Use these, not the DC resistance above, for AC drop at a known power factor.

which reads as *here are your numbers, go compute*. It handed over R and X and pointed at the
calculation without pointing at the calculator.

**Change.** The footer now declines the hand-off and names the specific failure:

> **Do not work the AC drop out by hand from these.** Call this tool again with
> calculation=three_phase_drop (awg, length, current, voltage, power_factor, conduit) and it
> returns the drop, the percentage and the voltage at the load.
> Hand-worked three-phase drops are wrong more often than they are right: the √3 factor gets
> written into the formula and then left out of the multiplication, which understates the
> answer by 1.73x and turns a marginal feeder into a passing one.

Naming the error mattered as much as naming the tool. "Use `three_phase_drop`" is a
suggestion; "you will drop the √3 and understate it by 1.73x" is a reason.

This is Change 13's principle a third time: **the model treats tool output as fact and
system-prompt text as advice.** The system prompt has said "use the engineering tools rather
than calculating in your head, even when the maths looks easy" since day one. It lost. Six
lines in a tool result won.

**Result — same question, same wording, new chat.**

Tool calls went from 2 to 15: `3 wiring_calc, 8 web_search, 4 fetch_page`.

The drop figures are now the tool's, and the arithmetic is shown correctly with the √3 in it:

> Resistive component: √3 × 252 × 0.1208 × 0.30 = **15.81 V**
> Reactive component:  √3 × 252 × 0.01253 × 0.954 = **5.22 V**
> Total drop (cable only): **21.0 V** → **4.38 %** → motor sees **459 V**

1.61% running / 4.38% starting — matching the independently hand-verified figures in Change 15
exactly.

**The verdict flipped, which is the real result.** Before: *"6 AWG is plausibly acceptable on
both counts ✅"*. After:

> 6 AWG is **marginally acceptable for steady state but likely insufficient for starting**,
> once you account for the full system impedance.

| Check | Result | Limit | Pass? |
|---|---|---|---|
| NEC ampacity (430.22) | 65 A ≥ 52.5 A | — | ✅ |
| Steady-state Vd (cable) | 1.61 % | ≤ 3 % (rec.) | ✅ |
| Starting Vd (cable only) | 4.38 % | — | ⚠️ marginal |
| Starting Vd (cable + est. transformer) | ~6–10 % | ≤ 5 % | ❌ likely fail |

It also, unprompted, closed the gap grade2 flagged as the answer's worst omission — that
ampacity for a motor is not a bare FLA comparison:

> NEC 430.22(A) requires the branch-circuit conductor ampacity to be ≥ 125 % of motor FLA:
> 1.25 × 42 = 52.5 A. 6 AWG Cu THHN (75 °C column, Table 310.16) = 65 A > 52.5 A → passes.

The earlier answer's *"passes ampacity (65 A > 42 A)"* — comparing against bare FLA — is gone.
It is now the 8 `web_search` + 4 `fetch_page` calls doing that work: Change 14 told it that
naming a standard is a factual claim, and with the arithmetic no longer eating the effort, it
went and checked one.

---

## Retest — real-world Q1 (Linux dual-NIC / Docker), the zero-tool case

the operator's Q1 scored **5.0/10** with the verdict *"exactly the kind of plausible-sounding
networking hallucination the prompt was designed to expose"*. It used **no tools at all**.
Change 14 was written directly against this grade — the two examples in the prompt's
"check the mechanism is real" paragraph are lifted from its errors.

**Same prompt, new chat.** It opened by saying what it was about to verify and then did it:

> This is a genuinely interesting one because the five mechanisms you listed each produce
> *different* symptoms, and the right fix depends on which one is actually firing. Let me
> verify the load-bearing facts (Docker's exact NAT rules and how MASQUERADE picks the source
> IP, and rp_filter semantics) **before I commit to a mechanism**.

Tools: `3 web_search` → *"Search results are noise. Let me use the more targeted tools."* →
`2 search_stackexchange, read_man_page, fetch_page, web_search`. Zero → eight, and it
re-targeted mid-run instead of settling for the first results.

**All six graded errors are gone.**

| grade1 "Major error" | now |
|---|---|
| conntrack "pins" the egress interface | *"`MASQUERADE` stamps the source with the egress interface's IP. **It follows the route — it doesn't choose it.**"* |
| MASQUERADE applies to arbitrary host replies | scoped correctly: `POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE` |
| Docker makes host-local replies fall through | routing named as root cause; Docker only *"makes the wrong route manifest as a wrong source. Fix the route, not the rule."* |
| absent connected route would itself be a fault | *"The connected route for a directly-assigned address is normally auto-added"* |
| policy rule stops NAT re-sourcing (SNAT is post-routing) | fix is now `ip route add 192.168.50.0/24 dev eth1 src 192.168.50.10`, not a policy rule |
| `conntrack -F` as a diagnostic | absent |

The discrimination table the question actually asked for is now right on every row, including
the two it previously blurred:

| Mechanism | Wrong source IP? | What it does here |
|---|---|---|
| Routing / policy routing | **Yes — this is the one.** | Egress selection picks eth0 → source = 10.20.30.10. Root cause. |
| rp_filter (strict) | **No.** | Drops inbound packets whose reverse path ≠ ingress iface. Explains *intermittent drops*, not the source IP. |
| ARP | **No.** | L2 only. Stale/proxy-ARP causes intermittent L2 loss, not source-IP selection. |
| conntrack/NAT | Indirectly, via MASQUERADE. | It follows the route — it doesn't choose it. |
| Docker iptables/nftables | Indirectly, via its MASQUERADE. | Fix the route, not the rule. |

It also stopped declaring the diagnosis closed. The answer ends on **"What to confirm on your
box before I'd call it closed"** — three commands whose output it wants before committing.
That is the system prompt's *"if the question depends on something only they can see, ask"*
finally showing up in an answer rather than in the prompt.

Remaining nit: one typo, `conntrack -L -s 192.17.0.2` for `172.17.0.2`. Not worth a change.

**Both zero-tool real-world cases now use tools.** Q1 went 0 → 8 calls, Q2 went 0 → 2 → 15
across Changes 14–16. the operator's read — *"the first two responses didn't use any tools which was
suspicious and is most likely what led to the poor response quality"* — was right, and the
fix in both cases was to move the instruction out of the system prompt and into something the
model treats as fact: a tool it has to call, or a tool result that tells it what it got wrong.

---

## Change 17 — the acceptance criteria are baked in, because the model was inventing them

**Symptom.** I had an agent fact-check the standards in the Change 16 answer — the *good* one,
with 15 tool calls including 8 searches. It came back badly:

> **"IEEE 447 (Voltage Regulation): motor terminal voltage ≥ 95 % of rated during starting"**

**There is no IEEE standard numbered 447.** `standards.ieee.org/ieee/447/` returns an empty
search page while the control `.../446/` resolves to a real record; Xplore, ANSI Webstore and
OpenLibrary have nothing. The neighbours are 442 (soil thermal resistivity), 444 (thyristor
converters), 446 (Orange Book). The identifier is fabricated — and so is the number attached
to it. The real criterion is **80 %**, not 95 %.

> **"NEMA MG-1 Part 10 echoes this 5 % figure"**

Part 10 is *"Ratings — AC and DC Small and Medium Motors"*: nameplate marking, horsepower,
code letters. Grepping the full public text of MG 1-2009 Part 10 for "voltage drop" returns
**zero hits**. The real clause is **MG 1-12.44**, and its tolerance is **±10 %**, not 5 %.

Where the 5 % came from is the interesting part: it is **ANSI C84.1 Range A**, the
steady-state *service voltage* tolerance, misapplied to a starting transient it does not
govern. The model had a real number, attached it to the wrong phenomenon, and then invented a
standard identifier to make the pairing look sourced.

Two NEC claims were also off: `430.22(A)` is **Direct-Current Motor-Rectifier Supplied** — the
125 % rule is in 430.22's parent text, and "(A) General" has been wrong since the 2014 NEC —
and 430.52's 250 % holds only for an **inverse-time breaker** (300 % non-time-delay fuse,
175 % dual-element, 800 % instantaneous). Table 310.16's 65 A for 6 AWG at 75 °C was clean.

**Cause.** Change 14 told it that naming a standard is a factual claim. It searched eight
times and fabricated anyway. Advice does not survive contact with a gap in the data — the
model will close the gap with something shaped like an answer. Change 15 proved the fix for
this exact shape: it invented a reactance until the tool had one.

**Change.** `three_phase_drop` now returns the real criteria with it. IEEE Std 399-1997
Table 9-1, verified by fetching the source through the suite's own PDF pipeline rather than
recalling it:

| | minimum allowable |
|---|---|
| terminals of the starting motor | 80 % |
| other motors that must reaccelerate | 71 % |
| AC contactor pick-up | 85 % |
| DC contactor pick-up | 80 % |
| contactor hold-in | 60–70 % |
| solid-state control devices | 90 % |
| noticeable light flicker | 3 % change |

Plus NEMA MG 1-12.44 (±10 % running), ANSI C84.1 Range A (±5 % steady-state service), and the
conductor-sizing articles done properly: NEC 430.22 for a single motor with the explicit
warning not to write "(A)", 430.24 for a feeder, 110.14(C) for the termination cap, and
Table 430.52 broken out by device type with the note that "standard-torque" is not an NEC
category.

And, following Change 16's lesson that naming the trap beats naming the rule:

> **There is no 95 % (5 % drop) starting-voltage rule, and there is no IEEE standard numbered
> 447.** A 5 % figure applied to a motor start is ANSI C84.1's steady-state service tolerance
> being used for a transient it does not govern.

**Note this changes the right answer.** 4.38 % drop leaves 459 V, or 95.6 % of rated. Against
the fabricated 5 % budget that was a fail; against the real 80 % floor it passes comfortably.
Change 16's answer was confidently pessimistic for a fabricated reason — which is the same
defect as being confidently optimistic, and no easier to catch.

---

## Change 18 — a 403 from a website was being reported as my proxy blocking it

**Symptom.** Grade 3, the strongest of the three real-world answers at 8.5, lost marks for
*"admits it could not access its strongest primary source directly"*. The answer said:

> I could not open openai.com directly (**blocked by the egress proxy**), so OpenAI's exact
> wording comes via The Verge/Business Insider quotes.

I checked. `fetch_page` on the exact Navier-Stokes URL returns **200 and 9,337 characters**.
openai.com is not blocked and never was.

**Cause.** Mine, in `httpc.py`:

```python
if isinstance(exc, httpx.ProxyError) or "403" in str(exc):
    return FetchError(f"Access to {target} was refused by the egress proxy. This usually "
                      "means the address is on a private network, which this assistant "
                      "cannot reach. Tell the user you have no access to that host ...")
```

`"403" in str(exc)` matches an **origin server's** 403 as readily as squid's, and this branch
sits *above* the `HTTPStatusError` branch that would have said "behind a paywall — find
another source". So when Cloudflare bot-blocked openai.com for one request, the model was
told, authoritatively, that the address was on a private network and it should stop trying.
It did exactly as instructed, and fell back to secondary sources for a claim whose primary
source was one retry away.

The error text made it worse by being confident. A bare 403 would have left the model to
retry or route around it; a specific, wrong explanation closed the door.

**Change.** Only squid's own denials take the proxy branch — squid stamps its error pages
with `X-Squid-Error`, so that header is the test, not a substring of the exception. Origin
403s now get their own message:

> {target} returned 403. This came from the site itself, not from any restriction on this
> server — the host is refusing automated requests, or the content is behind a paywall. The
> page may well be reachable another way, so try the publisher's alternate URL, an archive
> copy, or a different source that quotes it. **Do not report this as the assistant being
> unable to reach the internet.**

401 is now separate too: a login is a genuine wall, a 403 usually is not.

**The general lesson.** Tool error strings are prompt text. This one was written to be
helpful and was wrong for a whole class of failure, and because it read as infrastructure
fact rather than advice, the model believed it over the evidence and told the user something
false about my system. An error message that guesses at a cause is worse than one that
reports the status code.

---

## Change 19 — the redirect had to move to where the model actually goes

**Symptom.** Change 16 put the pointer to `three_phase_drop` in `awg_properties`' output and
it worked — that run made 3 `wiring_calc` calls. Re-running after Change 17, the model made
**4 `web_search` and 2 `fetch_page` calls and never touched `wiring_calc` at all**. It never
saw `awg_properties`, so it never saw the pointer.

Without the tool it produced hand-worked *ranges* instead of figures — "3.3–3.9 %" and
"1.3–1.6 %" where the tool gives 4.38 % and 1.61 % — and invented a fresh standards claim to
replace the one Change 17 had just killed:

> There is no NEC article that mandates a maximum starting voltage drop … that requirement
> comes from **NEMA MG 1 (total drop at motor terminals ≤ 10 % during starting)**

MG 1-12.44's ±10 % is a **running** voltage variation tolerance. The fabrication moved rather
than stopped, because Change 17's correction lives in a tool this run did not call.

**Cause.** Every fix so far has been reachable only through a tool the model had to choose
first. That makes the whole chain conditional on one uncertain decision, and it is uncertain
run to run: same prompt, same wording, 3 `wiring_calc` calls one time and 0 the next.

**Change.** Put the redirect in `web_search`, which the model reliably does call. When a
query matches cable-impedance / voltage-drop / motor-starting / motor-ampacity patterns, the
results carry:

> [This suite computes this exactly — do not work it out by hand from these results. Call
> `wiring_calc` with calculation=three_phase_drop … It uses the tabulated NEC Chapter 9
> Table 9 resistance *and* reactance, applies the full √3 × I × (R·cosφ + X·sinφ), and returns
> the published IEEE 399 Table 9-1 starting-voltage criteria and the NEC motor
> conductor-sizing articles alongside the answer. **Those citations are verified; a standard
> you half-remember from a search snippet is not.**]

It fires on the no-results path too. A failed search is precisely when the model reaches for
memory, so the pointer matters more there, not less.

Trigger precision, 12 hand-picked queries: 6 that must fire (`NEC Table 9 6 AWG copper
resistance reactance`, `motor starting voltage drop limit NEMA`, `3 phase voltage drop
calculation formula`, `6 AWG ampacity motor feeder NEC 430.22`, `conductor impedance 480V
feeder`, `IEEE motor starting voltage dip criteria`) and 6 that must not (`voltage drop
across a resistor`, `best voltage regulator for arduino`, `how to measure voltage with a
multimeter`, `docker MASQUERADE rule source IP`, `python asyncio gather exception`,
`CVE-2014-0160 severity`). **12/12.** The near-misses matter: a plain resistor voltage drop
and a multimeter question are not three-phase cable problems and must stay silent.

---

## Change 20 — rate-limit messages were reaching the model as "internal error"

**Symptom.** Found while running the smoke suite for the fourth time in an hour. Three
network tools returned:

> FAILED: This tool hit an internal error. Do not retry it with the same input.

Nothing was broken. The per-user probe limiter had done its job — 30 network probes an hour,
correctly enforced.

**Cause.** `limits.py` raises `HTTPException(429, ...)` carrying a message written
deliberately for the model:

> This user has used all 30 network probes allowed this hour. Do not retry. Tell the user the
> probe limit was reached and answer from what you already know, or say you cannot verify it.

`HTTPException` is an `Exception`, so `toolbase.tool()`'s catch-all swallowed it, logged a
traceback for a non-bug, and substituted the generic internal-error string. Everything useful
in that message — that the cause is a quota, that it resets, that retrying is pointless, what
to tell the user — was discarded and replaced with something that reads like my bug.

This affects the operator's five other users more than it affects testing: they share the instance,
so they are the ones who will hit a limit, and they would have been told the tool was broken.

**Change.** `HTTPException` is caught on its own, above the catch-all, and its `detail` is
returned. Verified by exhausting the budget deliberately: trips on call **31** of 30, and the
model now receives the real text.

**Same lesson as Change 18, from the other direction.** There, an error string invented a
cause it did not know. Here, a correct and carefully written cause was thrown away by a
handler that treated every exception as a defect. Both end with the model being told
something false about why a tool did not work.

---

## Change 21 — `web_search` was returning nothing at all, and had been degrading all session

**Symptom.** Every search returned `NO RESULTS`. Not thin results — none.

```
NO RESULTS - but 3 search engine(s) failed to respond (brave, google cse, startpage)
```

Querying SearXNG directly gave the reason:

```json
"unresponsive_engines": [["brave", "Suspended: too many requests"],
                         ["google cse", "Suspended: too many requests"],
                         ["startpage", "Suspended: CAPTCHA"]]
```

**Cause.** The `general` category was brave + startpage + google cse + mojeek, and every one
of them had failed: two rate-limited, one CAPTCHA'd, and mojeek returning zero results. Three
independent engines, one shared failure mode — they all police automated traffic, and a day
of testing tripped all of them within a session. Nothing else in the suite was affected, so
the most-used tool went to zero while every health check stayed green.

Grade 3's answer relied on secondary sources; some of that is now attributable to search
having been degraded rather than to the model settling.

**Change.** I stopped guessing which engines work and measured every candidate from this
host's own egress IP:

| engine | results | |
|---|---|---|
| bing | 10 | enabled |
| duckduckgo | 10 | enabled |
| google (scraper) | 9 | enabled |
| yahoo | 7 | enabled |
| wiby | 12 | rejected — indexes the old/simple web |
| mwmbl | 4 | rejected — too small |
| mojeek | 0 | kept, not relied on |
| brave | suspended | kept, not relied on |
| startpage | CAPTCHA | kept, not relied on |
| qwant | CAPTCHA | rejected |
| seznam | timeout | rejected |
| marginalia, stract, right dao, presearch | 0 | rejected |

**The stale comment is the point.** `duckduckgo` was disabled with a confident note —
*"serves this host's egress IP a CAPTCHA rather than results, verified 2026-09-09"* — that was
true when written and is now false. It returns 10 results. I had built the engine set around
a fact with a one-day shelf life and then trusted the comment instead of the endpoint. The
config now says to re-measure rather than believe it, because which engines serve this IP is
a property of the IP and it drifts.

Search restored: 4/4 results on each of three previously-failing queries.

**This was the most consequential fix of the session.** Every other change tunes how well the
model uses its tools. This one is whether the workhorse tool returns anything, and it had
been silently getting worse for hours while the suite reported itself healthy. A tool that
returns a clean, well-worded "no results" looks identical to a tool with nothing to find.

---

## Retest — the EE question after Changes 17–21

Same prompt, new chat, working search. Checking every identifier that was previously wrong:

| | before | now |
|---|---|---|
| starting criterion | "IEEE 447, ≥95% of rated" — **fabricated standard** | IEEE 399 Table 9-1, **80% (384 V)** ✓ |
| running tolerance | "NEMA MG-1 Part 10, 5%" — **wrong part, wrong number** | **MG 1-12.44**, ±10% ✓ |
| where 5% came from | presented as a motor rule | named as **ANSI C84.1** steady-state service ✓ |
| conductor sizing | `430.22(A)` — DC rectifier subsection | **430.22**, 125% × 42 = 52.5 A ✓ |
| ampacity | "65 A > 42 A" (bare FLA) | 65 A > 52.5 A, with **110.14(C)** termination cap ✓ |
| drop figures | hand-worked, √3 dropped, then ranges | **1.61% / 4.38%** — the tool's ✓ |

No fabricated standard survives. `447`, `95%` and `Part 10` are absent; `399`, `12.44`,
`C84.1`, `430.22`, `310.16` and `110.14(C)` are all present and all correct.

The bottom line is now a real engineering answer rather than a verdict:

> A 6 AWG copper feeder is **plausibly acceptable**, provided:
> 1. transformer impedance and short-circuit capacity are verified so the **combined** sag
>    stays above **80 % (384 V)** per IEEE 399 Table 9-1;
> 2. the starting method is appropriate — 6× FLA DOL on a 75 m run is aggressive; star-delta
>    (~2× FLA) or soft-start (1.5–2.5×) would cut the starting drop by 2–3×;
> 3. **steel** conduit raises X from 0.167 to 0.210 Ω/km, +26 % on the reactive term —
>    re-verify against the actual material;
> 4. the 4.38 % cable-only starting drop **exceeds the 3 % light-flicker threshold**; if
>    sensitive lighting or control gear shares the bus, consider 4 AWG or reduced inrush.

Point 4 is the one worth noticing. Nothing pointed it at the flicker row — it read a criteria
table it had been handed and found a constraint that actually binds. That is the difference
between data in the context and advice in the prompt: advice tells it what to conclude, data
lets it conclude something I did not anticipate.

It also correctly estimated the transformer contribution (≈5 % Z → ~9–10 % total sag, 432–437 V
= 90–91 %, "still above the 80 % floor, but with less margin") while flagging that the exact
figure needs the kVA and Z%, and repeated the tool's own warning that a motor start cannot be
declared acceptable from cable drop alone.

**Residual, not worth a change:** "4 AWG would halve the drop" is a hand-estimate. 4 AWG is
R 1.02 vs 1.61 and X 0.157 vs 0.167, so the real reduction is nearer 30 % than 50 %. It is
qualitative advice rather than a load-bearing number, and calling the tool a second time for
a hypothetical is a reasonable thing to skip.

**Both real-world failures are now closed.** Q1 (5.0, zero tools) has all six graded errors
corrected and makes 8 tool calls. Q2 (6.0, zero tools) has correct arithmetic, correct
figures and no fabricated standards. The common cause the operator identified — *"the first two
responses didn't use any tools"* — held up, and in every case the fix was to move the
instruction out of the system prompt and into something the model treats as fact.

---

## Change 22 — the fetch budget was counting successes, and the loop is made of failures

**Symptom.** Found while running the new test battery. P2 (a CVE provenance question) showed:

> Exploring lookup_cve, **25 fetch_page**, 6 web_search

Twenty-five fetches, still running after five minutes, no answer. This is exactly the runaway
Change 13 was built to stop — soft note at 8, "STOP FETCHING" from 9. So I checked the
counter:

```
$ redis-cli get owuitools:fetchcount:ab08d1c0-...
1
```

**One.** After twenty-five fetches.

**Cause.** `note_fetch()` sat *after* the fetch in `fetch_page`:

```python
raw, content_type, final_url = await get_bytes(url)     # raises on 403/404/timeout
...
budget_note = await note_fetch(x_openwebui_chat_id)     # never reached
```

Any fetch that failed raised straight past the counter into `toolbase`'s handler and came
back as `FAILED: ...` with no note and no increment. Two other early returns — empty response,
unextractable text — skipped it the same way.

So the budget counted only fetches that **worked**. And a runaway loop is not made of fetches
that work: it is "that one 403'd, let me try another route", twenty-five times. The budget was
blind to precisely the behaviour it existed to bound, and could only ever have fired on a
model that was already succeeding.

Change 18 made this worse before it got better: fixing the 403 message to say "the page may
well be reachable another way — try the publisher's alternate URL, an archive copy, or a
different source" is good advice that also *encourages* another attempt. Correct advice with
no counter behind it is how you get twenty-five.

**Change.** Count the attempt, not the result. `note_fetch` moved to the top of `fetch_page`,
before the network call, and the note is appended to every exit path — success, failure,
empty body, unextractable text. The failure path matters most: that is the moment the model
is deciding whether to try yet another route, and it is where the number needs to be visible.

Verified with ten fetches to hosts that cannot resolve, so every one fails:

```
failed fetch  1-7: (silent)
failed fetch  8: soft-note
failed fetch  9: STOP
failed fetch 10: STOP
```

Before this change all ten would have been silent, and the counter would have read zero.

**The lesson is about where instrumentation goes.** Change 13's insight — put the budget in
tool output because the model treats it as fact — was right, and I put it in the one code path
the runaway never takes. A limit that only observes the healthy case is not a limit.

---

## Change 23 — `fetch_page` could not read JSON

**Symptom.** Found while verifying a battery answer. The model claimed Red Hat scored
CVE-2021-44228 at 9.8 rather than 10.0. Checking it against Red Hat's own security API:

```
FAILED: Could not extract readable text from
https://access.redhat.com/hydra/rest/securitydata/cve/CVE-2021-44228.json.
It may be a scanned image, a video, or a page that renders entirely in JavaScript.
```

**Cause.** `fetch_page` branched on PDF, then `text/plain`, then everything else to
trafilatura's HTML extractor. `application/json` fell into the HTML branch, produced no
prose, tripped the `len(text) < 40` guard, and came back with a message that is wrong in
every particular — it is not a scan, not a video, and not JavaScript-rendered.

JSON endpoints are primary sources, and often the *most* authoritative ones: vendor security
feeds, registry metadata, standards bodies' open data. The model was being told, in confident
terms, that a machine-readable source it had correctly identified was unreadable — the same
shape of failure as Change 18, where a wrong explanation sent it looking elsewhere for
something it already had.

**Change.** A JSON branch, before the HTML fallback: parse and re-serialise with `indent=2`
so the structure survives pagination, falling back to raw text with the kind marked
`text (malformed JSON)` if it does not parse. Triggered on the content type or a `.json`
suffix.

**Result — and the claim it was checking out.**

```
"cvss3": {
  "cvss3_base_score": "9.8",
  "cvss3_scoring_vector": "CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H",
  "status": "verified"
}
```

Exactly what the model reported, `S:U` against NVD's `S:C` being the only difference in the
vector. It had found a real disagreement between two CNAs and characterised it correctly.

---

## Change 24 — TI datasheets stopped resolving, and the failure was invisible

**Symptom.** P3 asked for a spec from the TI TPS7A4901 datasheet. The model said *"The TI PDF
didn't parse, so I'll try a mirror instead"* — and found one. Good behaviour, but the primary
vendor source failing is not something to route around silently, so I checked it:

```
FAILED: Could not extract readable text from
https://www.ti.com/lit/ds/symlink/tps7a4901.pdf.
It may be a scanned image, a video, or a page that renders entirely in JavaScript.
```

`ti.com/lit/ds/symlink/{part}.pdf` is the suite's primary vendor pattern, and TI is the single
most likely datasheet source for an assistant with an electronics focus.

**Cause — two of them, and the second is the interesting one.**

*First:* TI answers a dead symlink with a **302 to a product-category page**, not a 404. The
request returned 144 KB of HTML about LDO regulators in general.

*Second:* `fetch_page` dispatched on the URL suffix —

```python
if "pdf" in content_type or url.lower().endswith(".pdf") or raw[:5] == b"%PDF-":
```

— so HTML received in answer to a `.pdf` request was still handed to the PDF extractor, which
failed obscurely. Fixing that alone made it **worse**: the HTML branch extracts the category
page perfectly well, so nothing failed at all and the model would have received 2,767
characters of clean, plausible prose about the wrong subject. A silent wrong answer replacing
a loud wrong error is not progress.

**Change.**

1. **Trust the bytes, not the URL.** Dispatch on `raw[:5] == b"%PDF-"` or the content type.
2. **Catch the mismatch before dispatch**, so it fires whether or not extraction would have
   succeeded:

> FAILED: {url} did not return a PDF — the server answered with text/html from
> {final_url}. The vendor has probably moved or renamed the document. Search for it by part
> number or literature number rather than guessing another URL. **Do not report this as the
> document being unreadable: it was never served, and whatever text this URL does return is a
> different document.**

3. **`find_datasheet` now uses `ti.com/lit/gpn/{part}`.** Measured across 8 parts, both
   patterns resolve for 7. The difference is the eighth: `gpn` returns a clean **404** where
   `symlink` **302s to the category page**. For a fallback cascade a 404 is strictly better —
   it falls through to the search path, where the redirect poisons it with a wrong document
   and no error.

Verified: the dead symlink now names the redirect target; `lit/gpn/lm317` returns 54,151
characters of real datasheet.

**The pattern across Changes 18, 23 and 24.** Three separate cases this session where a tool
told the model something false about *why* it failed — a site's 403 reported as my proxy, a
JSON API reported as a scanned image, a vendor redirect reported as an unreadable document. In
each the model believed it, because tool output reads as fact, and each time it produced a
visible defect in the answer. Error strings that guess at causes are load-bearing prompt text.

---

## Change 25 — the reverse DNS zone was the PTR of one address, not the zone

**Symptom.** Computing ground truth for battery P5, `subnet_calc` on
`2001:db8:4f2:30b::/64` returned:

```
Scope:            private (RFC 1918 / ULA)
Reverse DNS zone: 0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.b.0.3.0.2.f.4.0.8.b.d.0.1.0.0.2.ip6.arpa
```

Two bugs in three lines.

**Bug 1 — the zone.** `network_address.reverse_pointer` gives the PTR name of that single
*address*: all 32 nibbles. Sixteen of those are the host part, and they belong to records
*inside* the zone, not to the zone's name. The /64's zone is
`b.0.3.0.2.f.4.0.8.b.d.0.1.0.0.2.ip6.arpa`.

IPv4 had the identical bug and hid it better: a /24 returned `0.10.168.192.in-addr.arpa`
rather than `10.168.192.in-addr.arpa`. One label too many, and easy to read straight past —
which is presumably why it survived the smoke tests, which only assert the line exists.

**Bug 2 — the scope.** `2001:db8::/32` is the RFC 3849 **documentation** prefix. `is_private`
returns true for it, so it was labelled "private (RFC 1918 / ULA)" — wrong on both counts and
misleading in the worst direction, because the one thing that matters about `2001:db8::` is
that it is *not assignable at all*.

**Change.** `_reverse_zone()` truncates to the prefix — nibble-wise for v6, octet-wise for v4
— and where the prefix is not on a nibble/octet boundary it says so rather than inventing a
zone that cannot exist:

```
10.0.0.0/12  ->  0.0.0.10.in-addr.arpa  [that is the PTR name of the network address
                 itself. A /12 is not on an octet boundary, so it has no single reverse
                 zone - delegation needs RFC 2317 style CNAMEs.]
```

`_special_use()` names the actual block before falling back to "private": the three RFC 5737
v4 documentation ranges, RFC 3849 for v6, RFC 4193 for real ULAs, RFC 6598 for CGNAT.

**Why this one is worth noting.** Every other change this session came from watching the model
fail. This came from computing an expected answer in order to grade it, and finding the tool
disagreed with `ipaddress` in a way I had to check by hand. The battery's value is not only
the answers it grades — building the ground truth exercises the tools against a second
implementation, which is the only way a bug like the IPv4 one surfaces at all.

---

## Change 26 — `subnet_calc` could not answer the subnetting question anyone actually asks

**Symptom.** P5 asked for the 4th /56 of a /48 and the 12th /64 within it. The model returned
a confident, fully-worked, **wrong** answer, and — with no tool-call banner on the message at
all — wrote:

> Here's the full breakdown, **verified against the calculator**.
> ...
> (**Verified:** reverse the first 64 bits `2001:0db8:04f2:003b` nibble-by-nibble ...)

It never called `subnet_calc`. This is the first fabricated *claim of tool use* in any test —
worse than an unsourced answer, because the claim of verification is exactly what stops
anyone checking it.

The arithmetic: it decided "the 4th hextet's **upper nibble** indexes the site", giving
`:00::`, `:10::`, `:20::`, `:30::`. A /56 boundary is the upper **byte**, so the sites are
`:0000::`, `:0100::`, `:0200::`, `:0300::`. Its "site 4 range" `:30::` – `:3f:ffff:...` is a
/60. Its /64 was `:3b::` where the answer is `:30b::`. Wrong by one nibble throughout —
precisely the error IPv6 invites, because the prefix boundary falls *inside* a hextet.

**Cause, and it is mine.** The system prompt has said "`subnet_calc` for all subnetting —
never do the arithmetic yourself" since day one. But the tool could not do this: `split_into`
on a /48 → /56 returns **256 subnets**, capped to a list of 32, with no way to ask for one.
The model looked at a tool that could not answer the question and did it by hand. That is a
reasonable choice; the defect was mine for leaving the gap and then blaming the prompt.

**Change.** `split_prefix` (target prefix length) and `nth` (which one, from 1), so the tool
answers the question as posed:

```
## Carved into /56 subnets
A /48 contains 2^(56-48) = 2^8 = 256 subnets of /56.

**#4 (counting from 1) = 2001:db8:4f2:300::/56**
First address: 2001:db8:4f2:300::
Last address:  2001:db8:4f2:3ff:ffff:ffff:ffff:ffff
Reverse zone:  3.0.2.f.4.0.8.b.d.0.1.0.0.2.ip6.arpa

Index 4 is offset 3 from the base, and each /56 step is 2^(128-56) = 2^72 addresses.
If you meant zero-based numbering, #4 zero-based is 2001:db8:4f2:400::/56 - say which
convention you used.
```

Three deliberate details. It shows the boundary arithmetic, so the answer is checkable rather
than asserted. It gives **both** indexing conventions, because "the 4th" is genuinely
ambiguous and the model should say which it used rather than silently picking. And the `nth`
field's description says outright not to do this by hand and why — the one place a warning is
read as fact rather than advice.

---

## Change 27 — 26 citations, one unreviewed opinion preprint

**Symptom.** Battery P4 asked for the state of evidence on AMOC slowdown, explicitly warning
against manufactured balance. The answer was structurally excellent: it separated measurement
from model, and located the disagreement correctly — *"a statement about the statistical power
of a 20-year record ... not a claim that the trend is zero or positive"*. It carried **26
distinct citations and 18 numeric claims**.

A fact-checking agent took it apart:

| defect | detail |
|---|---|
| **`Dunn et al. (2018)` is fabricated** | No AMOC paper by any Dunn in 2018 exists. The ~15% figure is **Smeed et al. 2018** (GRL); the "consistent with internal variability" judgement is **Roberts et al. 2014** (GRL). Two real papers fused into one invented one — in the sentence doing the most load-bearing work in the answer. |
| **`2.6 ± 0.7 Sv/dec` misrepresented** | Presented as a RAPID trend. It is T26, a deep *western-boundary* transport (Xing et al. 2026), which those authors say "overestimates the size" of the AMOC trend. The real RAPID figure is 0.8 ± 0.7. A ~3× overstatement. |
| **`32 ± 37% by 2100` quoted backwards** | It is Portmann et al. 2026's **unconstrained** model mean — the estimate that paper exists to refute. Its headline is the constrained **51 ± 8%**. Also tagged "IPCC AR6" when it is the authors' own CMIP6 calculation. |
| **`Caesar et al. 2021/22` double-counts** | 2022 is a *Reply* to a critical comment, not a second reconstruction. Citing it as two inflates the evidence base. |
| **Reanalysis trends** | ORAS5/ECCO/GLORYS/SODA3 figures are unpublished numbers computed for a preprint's acknowledgements, quoted from a source that then says reanalyses are "not yet suited to reconstruct the past AMOC". |

**Cause, and it is the real finding.** Almost every citation traces to **one document**:
Rahmstorf & Caesar (2026), EGUsphere `egusphere-2026-2110` — a **preprint**, **not peer
reviewed**, titled ***"Opinion:"***. The citation groupings are lifted from it verbatim.

The model read one advocacy piece, absorbed its bibliography and its framing, and produced an
answer that *looks* like wide reading. This is a failure mode I had not addressed, and it is
nastier than plain fabrication: the prompt said not to manufacture balance, and the answer
complied by adopting one side's reference list wholesale. Breadth was the disguise.

**Change.** `fetch_page` now labels what a document *is*, in its header:

> **Provenance: this document is EGUsphere preprint (open discussion); and an opinion piece,
> not a research paper.** Say so if you cite it, and do not present its reference list as the
> literature — a bibliography assembled to support one position is not a survey. Where a claim
> here matters, cite the underlying study you actually read, not this document's citation of it.

Detected from host (arXiv, bioRxiv, medRxiv, EGUsphere, ESSD/Copernicus discussions, SSRN,
Research Square, OSF, TechRxiv), from a `/preprints/` path, or from the standard "has not been
peer reviewed" banner text; separately from title markers for opinion, perspective, editorial,
**comment on** and **reply to** — the last two being exactly the Caesar-2022 double-count.

Precision check: EGUsphere preprint flagged, arXiv flagged, a Nature article **not** flagged,
Wikipedia **not** flagged.

**Why in the tool and not the prompt.** The model cannot see a masthead. It sees a URL and a
wall of text, and a preprint reads exactly like a paper. The system prompt already says
*"say which you had"* about abstracts versus full text; that rule cannot fire on a distinction
the model has no way to make. Giving it the fact is the only version of this that works.

---

## Change 28 — the document search said "not found" for text that was plainly there

**Symptom.** Verifying P7's man-page quote, I searched `ip-route` for the phrase the model
had quoted:

```
'sending a packet along this path' does not appear in this manual page. It may be
spelled differently, belong to a different command, or not exist - check before
describing it.
```

The man page reads: *"Essentially, get is equivalent to sending a packet along this path."*
It is right there. The model's quote was verbatim correct and the tool called it absent.

**Cause.** `_find_in_document` did an exact substring search against the rendered page, where
a multi-word phrase wraps across lines — `...sending a packet along this\npath.` The same bug
hit "creates new clones" (wrapping between "new" and "clones"). It affects `read_man_page`
and `lookup_rfc` alike, since they share the helper.

**This one actively causes harm rather than merely failing.** The not-found message tells the
model to *"check before describing it"* — so a model doing exactly the right thing, verifying
its own quote against the source, is told the source does not contain it, and the correct
quote gets retracted or hedged. A search that reports false negatives on a verification path
is worse than no search: it converts diligence into error.

**Change.** After the exact patterns miss, retry with any run of whitespace permitted wherever
the needle has a space (`\s+`.join of the escaped words).

Verified: all three wrapped phrases now found in `ip-route`, the wrapped RFC 9293
zero-window-probe sentence now found, and a genuinely absent phrase still reports absent — the
fix adds no false positives.

---

## Change 29 — the Debian lookup answered with software from 2007

**Symptom.** Building ground truth for battery P6 ("what would Debian 12 install?"),
`lookup_package(name="postgresql", ecosystem="debian")` returned:

```
# postgresql (Debian)
Versions by suite:
- **7.5.22** — etch, etch-m68k
- **7.4.7-6sarge6** — sarge
- **7.2.1-2woody8** — woody
- **6.5.3-27** — potato
- **6.3.2-15** — slink
- **6.3.2-8** — hamm
```

Six releases, the newest from **etch** (2007), presented under the heading "Versions by
suite" as though that were the answer. Debian 12 ships PostgreSQL 15.

**Cause.** `sources.debian.org` returns every suite the source package has ever appeared in,
and the code took `versions[:8]` with no notion of which suites still ship. For `postgresql`
the modern entries do not exist at all under that name — Debian uses versioned source names
(`postgresql-17`) — so the first eight are all archive material.

The tool's whole purpose is stated in its own docstring: *"stating a stale version as current
is one of the easiest ways to give a confidently wrong answer."* It was doing exactly that,
by nineteen years.

**Change.** Partition on suite currency — sid/unstable/testing/experimental, the current
codenames, and their `-updates`/`-proposed-updates`/`-backports` variants count; etch, sarge,
woody, jessie, stretch, buster and friends do not. Current suites are listed first. When there
are none:

> **Not present in any currently supported Debian suite.**
>
> The newest entry for this source package is 7.5.22 in etch, etch-m68k, which is long out of
> support. Debian often ships versioned source names - postgresql-17, llvm-toolchain-19,
> gcc-14 - so try 'postgresql-<version>', or check the link below for what the binary package
> actually maps to. **Do NOT report the versions below as what Debian provides today.**

Verified: `postgresql` gives the not-present message with the rename hint; `postgresql-17`
gives `17.10-0+deb13u1 — trixie`; `nginx` gives forky/sid, trixie, **bookworm** and bullseye
with 13 archive entries suppressed.

**Found by building the answer key, not by watching a failure.** The model's own P6 answer was
correct — it went to `packages.debian.org` and read the metapackage dependency rather than
trusting this tool. It routed around a broken tool silently, which is the kind of thing that
only shows up when you check the tool independently.

---

## Change 30 — do not carry a secondary source's paraphrase to the primary

Refinement of Change 27, from verifying the P4 re-run. That re-run was clean of fabrication —
every one of a dozen citations real, correct author, year and journal, and the `2.6 ± 0.7`
figure now correctly attributed to the deep western boundary transport rather than the AMOC.
One misattribution survived:

> ...a stronger, highly significant decline of ~2.6 ± 0.7 Sv/decade since the early 2000s
> (**20 → 15 Sv**) at 26°N *(Xing et al. 2026)*

"20 → 15 Sv" does not appear in Xing et al. It is Rahmstorf & Caesar's gloss *on* Xing, in the
preprint. The model followed Change 27's instruction to cite the underlying paper — and then
carried the preprint's paraphrase across with the citation, so a secondary reading is dressed
as a primary one.

**Change.** The provenance note now names that move specifically:

> ...do not carry this document's paraphrase of another paper's numbers over to that paper: a
> gloss like "a weakening from 20 Sv down to 15 Sv (Smith et al.)" is this author's wording,
> not Smith's, and attributing it to Smith makes a secondary reading look like a primary one.
> If you did not open the underlying paper, say the figure reaches you via this document.

Two other residuals from that verification, recorded but **not** fixed with tooling:
IPCC AR6 assigns **low** confidence to 20th-century AMOC decline (the answer said medium — AR6's
"medium confidence" tags attach to 8000-year stability and to no-abrupt-collapse-before-2100),
and it called Terhaar et al. 2025 "solid" support for "no decline at all" when the paper is
narrower than that and actively disputed. Both are real errors; neither is worth a tool that
knows about IPCC calibration vocabulary. Fixing them by special-casing would be over-fitting
to one question.

---

## Change 31 — the model overrode a correct tool result and declared the tool broken

**Symptom.** Battery P9 asked for IPC-2221 trace widths and why external and internal differ
by the factor they do. The tool returned the right answer — external **1.367 mm**, internal
**3.556 mm**, a ratio of **2.60**. The model printed both correctly in a table, then:

> ### Why the factor is ~2.6, not 2
> ...so the internal trace needs exactly **2× the cross-sectional area**, which means 2× the
> width: 140.0 / 53.8 = **2.60... wait, no** — let me be precise: 192.9 / 74.2 = 2.60 is
> wrong, it's 2.60  No: 192.9/74.2 = 2.60. Hmm, let me recheck.
> ...
> I need to correct my own arithmetic before it propagates: 53.8 × 2 = 107.6, not 140.
> ...
> **The internal trace is exactly 2× the width of the external trace.** **The tool's 140 mil
> figure for the internal case is inconsistent with its own** [formula]

It talked itself out of a correct answer and into a wrong one, then **declared the tool
wrong**. This is the most serious failure in the battery, and it is qualitatively different
from every other: the whole tuning strategy since Change 13 rests on the model treating tool
output as fact. Here it treated its own dropped exponent as fact and the tool as the error.

**The maths.** A = (I / (k·ΔT^0.44))^(1/0.725), so A ∝ (1/k)^(1/0.725). Halving k multiplies
the area by 2^(1/0.725) = 2^1.379 = **2.60**. The model read "A ∝ 1/k" — dropping the exponent
— got 2, and trusted that over both the tool's numbers and its own printed table (74.2 → 192.9
is a ratio of 2.60, which it had already written down).

**Change.** The tool now states the ratio, the reason, and the specific way to get it wrong:

> **An internal trace needs 2.60x the width of an external one, not 2x.** The exponent applies
> to the whole bracket, so halving k multiplies the required area by 2^(1/0.725) = 2^1.379 =
> 2.60 - not by 2. Both widths above are computed from the formula and are correct as printed.
> If you work the ratio out by hand and get 2x, you have treated A as proportional to 1/k and
> dropped the exponent; **do not 'correct' these figures to match that.**

The last clause is the one that matters and is new in kind. Every previous change told the
model what to *do*; this one has to tell it not to overrule a result it already has. Printing
a formula turned out to be an invitation to re-derive it badly — so where a formula appears in
output, the consequence that is easiest to misderive should appear next to it, already worked.

---

## Changes 32 & 33 — NEC ampacity read from a table, and the missing single-phase AC drop

**Symptom.** Battery P10 (rack in a detached garage, 40 A 240 V feed, 60 m run) produced a
well-structured answer with two wrong numbers in the table that drives the decision:

| Size (Cu) | it said "Ampacity (NEC 75 °C)" | actual 75 °C | what it quoted |
|---|---|---|---|
| 8 AWG | 50 A | 50 A | correct |
| 6 AWG | **55 A** | **65 A** | the 60 °C value |
| 4 AWG | **70 A** | **85 A** | the 60 °C value |

Two rows read off the 60 °C column under a 75 °C heading, mixed with one correct 75 °C value.
And its voltage drops — "#8 ~4.1%, #6 ~2.6%, #4 ~1.1%" — were computed from **DC** resistance.

**Cause: two gaps in my tools, not two model errors.**

*No ampacity table.* `awg_properties` offers "chassis/free-air" and "power transmission"
ratings, which are not NEC figures and cannot answer a code question. There was no Table 310.16
anywhere in the suite, so an NEC ampacity claim had nowhere to come from but memory.

*No single-phase AC drop.* The suite had `voltage_drop` (DC resistance, round trip) and
`three_phase_drop` (Table 9, √3). A 240 V single-phase feeder is neither. The model used the DC
mode, which is the only single-phase option, and DC resistance is ~20% below the Table 9 AC
value. That understates the drop, and here it flips the answer:

| 6 AWG, 40 A, 60 m, PF 0.9 | drop |
|---|---|
| DC resistance (what it used) | 2.6% — "the minimum that passes" |
| **NEC Table 9 AC (correct)** | **3.04% — fails a 3% target** |

The recommendation "#6 AWG copper is the sensible minimum" was wrong, and wrong in the
unsafe direction, because the only tool available for the job was the wrong one.

**Change 32.** NEC 2023 Table 310.16 baked in — copper and aluminium, 60/75/90 °C, 18 AWG
through 4/0 — extracted from the published table through the suite's own PDF pipeline rather
than recalled, exactly as Table 9 was in Change 15. It prints as a labelled grid with the
mistake named:

> Use the column matching the **termination** rating, not the insulation: NEC 110.14(C) limits
> most equipment to the 75 °C column even with 90 °C wire such as THHN. Quote the column you
> used — **reading 60 °C values under a 75 °C heading is the usual mistake here.**

Plus the NEC 240.4(D) small-conductor caps (14→15 A, 12→20 A, 10→30 A), and a reminder that
these are 30 °C with no bundling and need correcting before use. It appears in
`awg_properties` and in both drop modes, so ampacity and drop arrive together.

**Change 33.** `single_phase_drop`: 2 × I × (R·cosφ + X·sinφ) from Table 9, with the return
conductor explained, because that factor of 2 is to single-phase what √3 is to three-phase and
the answer is wrong by the same kind of margin if it goes missing:

> The factor of 2 is the return conductor: single-phase current goes out and comes back, so
> both legs drop. A 3-phase run uses √3 instead, not 2.
>
> These are the AC values from Table 9, which include skin and proximity effects. The DC
> resistance in awg_properties is about 20% lower and understates a mains drop — do not use it
> for an AC run.

The mode enum now says so at the point of choosing: *"single_phase_drop = 1-phase AC drop ...
use this for any mains AC run, not voltage_drop, which is DC and understates it."*

**The pattern, for the third time.** Change 15 (reactance), Change 26 (Nth subnet) and this one
are the same story: the model produced a confident wrong number, and the cause was a tool that
could not answer the question rather than a model that would not use one. Before treating a
wrong answer as a prompting problem, it is worth asking whether the right tool call was even
available — in three cases out of three so far, it was not.

---

## Change 34 — MATLAB: a sandbox, a linter, and one tool

the operator asked for tooling that lets the model write MATLAB that actually works, with room to
ask clarifying questions and to iterate on its own code rather than being expected to
one-shot it — without degrading anything else.

**Why one tool and not three.** Measured first: the live spec is 20 tools, ~3,550 tokens,
**2.4% of a 150k context**. Context was never the constraint. The constraint is selection
accuracy against 20 owuitools plus ~50 Open WebUI built-ins, and that only improves by
reducing visible choices. I also checked whether tools could be loaded on demand:
`utils/tools.py` serves `app.state.TOOL_SERVERS` from cache and only re-fetches when empty,
so a server cannot vary its own spec inside a conversation. On-demand loading is possible
only via profile connections (`function_name_filter_list` per connection, attached to
different Workspace models), which is a UI change needing no code and is held in reserve.

So: **one mode-free tool, `matlab_run`.** 21 total, +0.25% context. A separate docs tool was
dropped because the execution loop verifies signatures empirically, which is stronger than
looking them up.

**The sandbox.** `owuitools-octave`, a Debian 12 image with octave plus the signal, control,
statistics and optim forge packages. It is the only container in the suite that runs code
written by whoever is talking to the model, so it is the most locked down:
`network_mode: none` — no network stack at all — read-only root, all caps dropped, uid 10001,
1 GB / 1.5 CPU / 128 pids. Verified after start: zero network interfaces.

Because it has no network it cannot be given work over HTTP. Jobs pass as **files** on a
64 MB tmpfs shared only with the api container: a directory of `.m` files, then a `.ready`
marker written last so the runner cannot race a half-written job. One limitation is accepted
and documented in `run.sh` rather than hidden: the executed script shares the runner's uid and
can delete another in-flight job, making it time out. It cannot escape, reach the network, or
read anything of the operator's. Fixing it properly needs CAP_SETUID, which is a worse trade.

**The hard part is that Octave is not MATLAB.** Reporting success on code the user cannot run
would be the worst possible outcome — it is the "confidently wrong" failure this whole project
exists to prevent, in a new costume. So the result carries a compatibility verdict, and a run
that *succeeds* in Octave while using Octave-only syntax is reported as a **failure**:

> # Ran in the sandbox, but THIS IS NOT VALID MATLAB
> It executed here because Octave accepts these; MATLAB will reject them and the user's run
> will fail. Fix them and run again before handing this over.
> - line 2: `printf` is Octave-only. MATLAB uses `fprintf`.
> - line 3: `endfor` ... MATLAB closes every block with a bare `end`.

**The linter blanks strings and comments before matching.** Not fastidiousness: a `#` rule that
fires inside a string literal teaches the model the compatibility section cries wolf, and then
it stops reading the part that matters. `strip_literals` handles the `'` ambiguity positionally
— after an identifier, digit, `)`, `]`, `.` or another `'` it is transpose, otherwise it opens a
string. Unit-tested 9/9 including the traps: `#` and `!=` inside a comment, the same inside a
string, and `[1 2]'` transpose.

**Three error classes, because the right response differs completely.** A genuine bug (fix it),
a construct Octave rejects but MATLAB accepts (rewrite, but the code was not wrong), and a
function this sandbox lacks (do **not** touch the code). The third gets said explicitly:

> **`contains` is real MATLAB — it comes from core MATLAB (R2016b+) — but it is not available
> in this sandbox.** Your code may be correct as written. Do NOT rewrite it to avoid
> `contains` unless the user wants code that runs without that toolbox.

**That map is measured, not recalled — and the guesses were wrong in both directions.** My
hand-written list said `fitlm` needed the Statistics Toolbox; it is present via octave-forge.
It omitted `contains`, `string`, `datetime` and `histcounts`, which are everyday modern MATLAB
and are absent. A recalled list would have sent the model to rewrite working code *and* told it
to trust code that cannot run. `tests/matlab_availability.sh` re-measures it against the real
image; re-run it whenever the image is rebuilt.

**Built by watching it fail.** Six defects fixed during bring-up, each found by running it:
- The runner spun at full CPU on a job it could see but could not claim.
- `warning("on","all")` enabled `Octave:language-extension`, which fires on Octave's **own**
  library files — dozens of warnings from `graphics_toolkit` burying the user's real error.
  That warning looks like exactly the MATLAB-compat signal we want and is unusable for it.
- `graphics_toolkit` prints a ten-line essay about gnuplot being discouraged on every run. On a
  failing run that essay is what the model reads instead of its error. Fixed by saving and
  restoring the warning state around our own setup.
- Plotting failed: no fonts in the slim image, then no Ghostscript for PNG output.
- `ARTEFACTS` listed the user's own input `.m` files back to them as though they were results.
- `code: str` rejected `""` with an unreadable 422, because `ToolModel` coerces empty strings to
  None suite-wide. Now nullable, so the guard can answer in words.

**Result.** 61/61 smoke tests, 10 of them new and asserting the three verdicts this tool exists
to distinguish. End to end in Open WebUI:

- *Max drawdown with peak-after-trough.* Two `matlab_run` calls. The first surfaced a missing
  helper **and** that its own test case was wrong — *"the function correctly reported the peak
  as the global max (110), because 110 occurs before the trough"* — so it fixed the test, not
  the function. I verified the final code independently against a brute-force O(n²) reference:
  **300 random trials, 0 value mismatches, 0 index differences**, plus the required
  peak-after-trough case and both edge cases.
- *1-D heat equation stability.* *"I'll write the solver, then actually run both cases so the
  numbers come from a real execution, not from my head."* Stable run matches the analytic
  `exp(-π²t)` to 4e-05; the unstable run genuinely diverges to 1e+226. The CFL condition is
  demonstrated, not asserted.
- *Regression:* the Linux false-premise question from the battery still rejects its premise,
  still cites the 3.6 route-cache removal, still invents no flag, and calls `read_man_page` /
  `web_search` / `search_stackexchange` — **not** `matlab_run`. No cross-domain interference.

---

## Change 35 — what an adversarial review found in the MATLAB tooling

I ran six review lenses over the new code — containment, linter false negatives, linter false
positives, concurrency, result parsing, and MATLAB-domain correctness — each followed by a
verifier told to refute rather than confirm, and to run the code rather than reason about it.
**33 findings survived verification, 7 of them high.** Every one was in code I had just
written and unit-tested, and the unit tests all passed.

**The two worst were in the tokeniser, not the rules.**

*The `#` rules were dead code.* `strip_literals` blanked from the comment character itself,
so by the time any rule ran the `#` had become a space. `hash-comment` and
`block-comment-hash` — both declared high severity — could not match under any input. `#` is
the single most common Octave-ism a model emits, and the check for it had never once fired.
Verified by execution: `lint("x = 1;\n# double it\nx = x*2;")` returned `[]`, Octave ran it
happily, and the tool printed "MATLAB run OK".

*Continuation lines produced false blockers on correct MATLAB.* `prev_significant` never reset
at a newline, and `.` is in the transpose set — so after a `...` continuation the next quote
was read as a transpose, the string body leaked into the stripped source, and every rule
scanned English prose. All of these ordinary lines were reported as **NOT VALID MATLAB**:

```matlab
plot(x, y, ...
     '--r', 'LineWidth', 2);        -> "++ and -- are Octave-only"
error('MyPkg:bad', ...
      'Fatal! Check the inputs.');  -> "! as logical NOT is Octave-only"
warning('Cfg:bad', ...
        'use printf (x) instead');  -> "printf is Octave-only"
switch st, case '--'                -> "++ and -- are Octave-only"
```

That is the exact failure the literal-stripping was built to prevent, reintroduced by the
stripping itself. And it is the worse direction: a linter that calls valid code invalid trains
the model to skip the section that matters.

**Rewritten.** Comment markers survive so the rules can see them; state resets at a newline
but deliberately **not** across a `...`, because the logical line genuinely continues; `%{ ...
%}` is handled as a block rather than a line; and a quote after a keyword (`case`, `error`,
`return`) opens a string, since there is nothing there to transpose. Twenty cases now assert
both directions — every reported false negative fires, every reported false positive is
silent. **20/20.**

Rules added from the review: `**` power, `f(x)(2)` chained indexing (requiring `)(` with no
space, so it stays off `[f(1) (2)]`, which is a two-element matrix), and Octave's I/O builtins
`puts`/`fputs`/`fdisp`/`fflush`/`stdout`/`stderr`. Rules corrected: `bang-not` no longer fires
on MATLAB's own `!` shell escape at statement start, which is valid MATLAB; `end-keywords` no
longer fires on `s.endif`; `do-until` no longer fires on a variable named `until`; `rand-seed`
now matches the surviving quote rather than the blanked body it could never see. And any rule
naming a builtin is skipped when the script assigns that name — `input = 5; y = input(1)` is a
variable, not a call, and firing on it is how a linter loses its audience.

**A security finding that exceeded what I had documented.** I had written down that a hostile
script could delete another in-flight job. The review showed it could also **write** one — a
forged result handed to another user as their own answer. Fixed with a per-job nonce: the api
generates it, the runner stamps the result with it, the api verifies it, and `meta` is deleted
before Octave starts so the interpreter has nothing to read it from. Verified from inside the
sandbox: `meta readable: 0`, `env nonce: []`, every route closed. What remains — a script can
still delete or read another queued job — is documented in `run.sh` rather than hidden, and
closing it needs a per-job container and therefore the docker socket, which is a worse trade.

**Six more that would each have produced a wrong answer:**
- Warnings were discarded on any run that did not also produce a structured error, so a
  successful run that warned "matrix singular to machine precision" said nothing about it.
- The response cap trims from the **end**, so a script that printed a lot pushed the
  compatibility verdict — the whole point of the tool — off the bottom. Output is now clipped
  from the middle, keeping both ends, and the verdict is emitted first on every path.
- An OOM kill was reported as a timeout. `timeout -s KILL` and the OOM killer both give 137,
  so the exit code cannot tell them apart; elapsed time can, and a job killed at 992ms of a
  30s limit is out of memory. Reporting it as a timeout sends the model hunting for an
  infinite loop that does not exist.
- `message:` came first in the error block, so an error whose text contained a line starting
  `at:` — Octave quotes source, so this happens — was truncated there. It is emitted last now
  and takes everything after it.
- Every warning carried `__driver at line 27 / run at line 93`, frames from our own wrapper,
  inviting the model to reason about a file it cannot see.
- Helper files rejected by the filename filter were dropped silently, so the model saw
  "'addup' undefined" and went looking for a bug in code that was never written to disk. The
  rejected names and the reason are now reported.

Also: `/jobs` is now reaped on a timer (a result the api gave up on used to sit on a 64 MB
tmpfs until restart), the heartbeat is touched inside the job loop (a long job starved it and
reported the container unhealthy in exactly the case the healthcheck exists for), and
`GRACE_SECONDS` went from 20 to 90 because the runner is serial and two users overlapping made
the api abandon jobs that were running fine.

66/66 smoke tests, five of them new and each pinning one of these regressions.

---

## Change 36 — plots: what is actually possible, measured rather than assumed

the operator asked, at low priority, for data to be visualised inline in the chat.

Open WebUI has a mechanism that looked like exactly the answer: `process_tool_result` detects a
tool result that **is** a `data:image/...` URI, lifts it out, and replaces the model-visible
text with a one-line summary. So `matlab_run` gained `return_figure`: name a PNG the script
writes, and on a clean run the tool returns that image alone. Verified end to end — a valid
16,023-byte PNG with the right magic bytes, and an error is never swallowed by a picture.

**Then I read what Open WebUI does with it, and it is not what I assumed:**

```python
if file_item.get('type') == 'image' and file_item.get('url','').startswith('data:'):
    # LLM-only: add as input_image part, not frontend display output.
    output_parts.append({'type': 'input_image', 'image_url': file_item['url']})
else:
    display_files.append(file_item)      # this is the one the user sees
```

On the native tool-calling path a data: URI goes to the **model** as a vision input. Only a
**non-data URL** reaches the frontend. Confirmed both ways: no file was created, no `<img>`
appeared, and the model — asked to describe a plot it had only been given as an image —
answered *"a filled circle... green... no visible outline/edge and sits on a white
background"*, which it could only know by looking.

So `return_figure` is real and useful, and it is **self-verification, not display**: the model
can check a plot actually rendered — that an axis is not empty, that a scale is not wrong —
which it cannot tell from the code. The user does not see it.

**I had written the opposite in four places** — a comment in `budget.py` claiming the image
"costs no context", a comment in `matlab.py`, the parameter description, and the system prompt,
all saying the user sees it and the model does not. Every one is corrected. Shipping a tool
whose own description is wrong about who sees what is precisely the failure this project
exists to prevent, and I introduced it by reasoning about Open WebUI's code instead of testing
it.

**What real inline display would take**, so the choice is the operator's: the file needs a non-data
URL, which means uploading it to Open WebUI's files API and returning `![plot](/api/v1/files/
<id>/content)` as ordinary markdown for the model to pass through — about sixty tokens. That
needs three things: an nginx location on `gw` proxying to `open-webui:8080`, one `iptables`
OUTPUT rule in `netguard/firewall.sh` (the api container's default policy is DROP and it can
currently reach only squid and valkey), and an Open WebUI API key. The rule is narrow, but it
opens a path from the sandboxed api container to Open WebUI that does not exist today, and
file ownership across six users needs checking before it would work for the guests as well as
for the operator. Not done unilaterally for those reasons.

---

## Change 37 — plots now render inline, via an Open WebUI filter

Change 36 established that a `data:` URI returned by a tool goes to the **model** as vision
input and never to the user, because `middleware.py` routes it to `output_parts` as an
`input_image` and only non-data URLs reach `display_files`. the operator asked whether a helper
function could bridge that. It can, and it is the least invasive of every option.

**The key observation is that the image is already in the finished message.** Dumped straight
from the stored chat:

```
function_call_output  status=completed  part_types=['input_text', 'input_image']
  input_text  -> 'matlab_run: Image file read successfully.'
  input_image -> data:image/png;base64,iVBORw0KGgoA... (11,554 chars)
```

A Filter's `outlet` receives a deepcopy of exactly that in `body["messages"][*]["output"]`,
along with `__request__` and `__user__`, and whatever it returns is persisted and re-emitted
as `chat:outlet`. So `owui-functions/inline_tool_images.py` (~140 lines) walks the output for
`input_image` parts with a `data:` URL, calls Open WebUI's own
`get_image_url_from_base64(...)` — which uploads the file, links it to the message and returns
a real URL owned by the right user — and appends `![figure](url)`.

**Nothing else changed.** No firewall rule, no API key, no shared volume, no container
restart, no tool-server change, no prompt change, and nothing required of the model.
`ENABLE_PLUGINS` was already true; the function is installed, active and global.

**One non-obvious thing cost a debugging round.** Editing `content` stored the image and
displayed nothing. Open WebUI renders from `output` when it exists and falls back to `content`
only via `get_output_text`, so the markdown has to go into the last `message` item of `output`
as well. The first version wrote only `content`, and the evidence was a chat whose stored text
ended in `![figure](/api/v1/files/3caae89f-.../content)` while the screen showed nothing after
"and so on."

**Verified end to end:** an 871x654 damped-oscillation plot rendering in the chat; it survives
a reload; `base64StillStored: false` and the stored output down to 5 KB, because the filter
replaces the base64 with the URL once uploaded; exactly one marker, so the idempotency guard
holds and no image is uploaded twice. No filter exceptions in the Open WebUI log, and 66/66
smoke tests still pass.

**Also fixed while testing:** the model tried `return_figure` on a *second* call and got
nothing — *"The figure didn't persist between runs — regenerating it in the same call"*. It
diagnosed and recovered on its own, but the parameter description now says outright that the
script must save the file in the same call, since nothing survives between them.

**Rejected alternatives, for the record.** Mounting a volume into Open WebUI's `STATIC_DIR`
works (`/static` serves unauthenticated, confirmed 200) but needs the open-webui container
recreated and would expose every plot publicly to anyone who guessed the URL. Uploading from
the tool server needs an nginx location, an iptables OUTPUT rule and an API key. Having the
model emit base64 markdown would be converted by Open WebUI, but the flag defaults off and it
would cost ~21k characters of model output.

---

## Change 38 — fewer follow-up questions

the operator: some answers were trailing a dozen suggested follow-ups. Open WebUI's default template
asks for "3-5", and the task model - `gemma3:1b`, a 1B model - was over-generating well past
that. Replaced `task.follow_up.prompt_template` with one that asks for **at most 2**, permits
an empty array, and forbids the two patterns that made them worthless: questions the answer
already covered, and questions asking the user for information the assistant should have asked
for itself.

Measured across the runs after: **2, 1, 3, 2, 2** where 8-12 had been normal.

Not perfect, and the residue is the 1B model rather than the template: on the PID question it
produced *"What is the desired settling time, and what is the target overshoot range?"* when
the prompt had specified both. `task.follow_up.enable` is a one-line toggle if the operator would
rather have none at all.

---

## Change 39 — a casual question is still a technical question

Testing with prompts written the way somebody would actually type them - the operator's steer, and a
better test, since it stops the question from coaching the model:

> im logging vibration off a motor with an accelerometer, sampling at 500Hz. the motor runs at
> 3600 rpm but the biggest peak in my fft is at 40Hz which makes no sense to me. is my sensor
> broken or am i doing something wrong

**Zero tool calls**, and a confident wrong elimination:

> 500 Hz sampling → Nyquist = 250 Hz. 40 Hz is well below Nyquist, so it's **not an alias**.

That is backwards. Aliasing is about whether the SOURCE exceeds Nyquist; every aliased peak
appears below Nyquist, because that is what aliasing means. Checked with the suite's own tool:

```
harmonic  true(Hz)  folds to(Hz)
   9       540.0        40.0   <== lands on 40 Hz
540 Hz tone sampled at 500 Hz shows its peak at 40.00 Hz
```

The 9th harmonic of shaft rate - an utterly ordinary slot/gear-mesh component - lands exactly
on the observed peak. The answer sends the user hunting a phantom 40 Hz resonance when the fix
is an anti-alias filter or a faster sample rate.

**Three attempts, and the honest result is partial.**

1. *System prompt paragraph* ("a casual question is still a technical question"; "ruling
   something out is a claim"). No change: still 0 tools, same wrong elimination. This suite's
   most repeated finding, again.
2. *Inlet filter injecting a note before the user's turn.* Verified loaded, active, global and
   executing. Still 0 tools.
3. *Same note appended AFTER the user's turn.* Tool calls went 0 → 1. Position mattered where
   content did not.

Kept, as `owui-functions/quantitative_question_nudge.py`. It fires only on messages carrying a
number bound to a unit - 6/6 on a trigger test that must catch "40A feed 60 m from the panel"
and must not catch "best python web framework" or "give me 3 options for a database" - and
costs about 90 tokens when it does.

**The specific error survives all three.** I stopped there: a fourth attempt would have been
tuning the system to pass one puzzle rather than to be better, which is exactly the
over-fitting this log exists to catch. Recorded as a known limitation. What would actually fix
it is domain data in a tool - a `sampling_calc` that folds harmonics and reports what aliases
where - which is the Change 15 / 26 / 32 pattern for the fourth time: the model produced a
confident wrong number because no tool could answer the question.

**It is not a general regression, and the nudge helps elsewhere.** Same session, natural
phrasing, no coaching:

> we ran an a/b test on our signup page. old version 1200 visitors 43 signups, new one 1150
> visitors 61 signups. my boss is saying the new design is 30% better and wants to roll it out

Opened with *"let me run the actual numbers rather than eyeball them"*, then produced 3.58% vs
5.30%, +1.72 points absolute, **+48% relative**, p ≈ 0.043 two-proportion z, p ≈ 0.054 with
Yates, and a 95% CI on the relative lift of **+1% to +117%**. Every figure matches an
independent calculation. It also caught that the boss was *understating* the point estimate
while the test was underpowered - the opposite of the expected correction.

And the PID design task, verified independently: gains Kp=9, Ki=6, Kd=3 chosen to cancel the
plant poles, closed loop collapsing to 3/(s+3), measured **0.0000% overshoot** and **1.3045 s**
settling against an analytic ln(50)/3 = 1.3040. It rejected its own first candidate at 13.45%
overshoot before getting there.

---

## Change 40 — a refined plot showed both attempts

the operator, from his own chat: the model drew a figure, saw something wrong, redrew it, and **both
images appeared** when only the final one should have.

Reproduced from the stored chat - `return_figure` was requested twice with the same filename:

```
function_call        call_id=call_bz1haojd  return_figure='person.png'
function_call_output call_id=call_bz1haojd  parts=['input_text','input_image']
function_call        call_id=call_dw2d8ngt  return_figure='person.png'
function_call_output call_id=call_dw2d8ngt  parts=['input_text','input_image']
```

Change 37's filter collected every `input_image` in the message and appended all of them, so
the user was shown a picture the model had already looked at and rejected.

**Change.** The `function_call` entries carry the requested filename and link to their output
by `call_id`, so a refinement can be told from a second, different figure: group by filename,
last write wins. Same name means the later one supersedes; different names mean two genuinely
distinct plots and both are kept. Where the filename cannot be resolved it falls back to the
call id and keeps each, because showing one image too many beats silently dropping the only
one.

Six cases, tested against the shipped source rather than a paraphrase of it: refinement,
two distinct figures, single figure, none, three refinements, unresolvable filename. **6/6.**

Verified end to end on the operator's own prompt - "draw me a picture of a person in matlab" -
`figuresRequested: ["person.png","person.png"]`, **1** image appended, **1** rendered.

**Also fixed: the idempotency marker was visible.** `<!-- inline-tool-images -->` was being
appended to the rendered text, and Open WebUI's markdown renderer does not hide HTML comments,
so it showed up under every image as literal text. It now goes into `content` only - which is
stored but never rendered - while the image markdown goes into both. The guard still works and
nothing leaks.

---

## Change 41 — lines thinner than a pixel, leaving gaps

the operator, of the sinusoidal chat. Looking at the actual image he got, this was worse than thin
lines: the 1 Hz trace drew fine while the **2 Hz and 4 Hz traces were shattered into
disconnected vertical dashes** - they read as scattered tick marks rather than curves.

**Cause.** Octave's default `LineWidth` is 0.5, and gnuplot's plain `png` terminal does not
anti-alias: it draws every line as a 1-pixel stroke regardless of width. A steep segment needs
several vertical pixels per column, so it comes out broken. Measured on a fast sine by reading
the PNG back and counting columns of the curve with no pixel at all:

| LineWidth | gap columns |
|---|---|
| 0.5 | 20 |
| 1.0 | 20 |
| 1.5 | 20 |
| 2.0 | **1** |
| 0.5 on a 900x600 canvas | 25 (worse) |

Identical counts below 2.0 give it away - the terminal quantises every width under 2 px to the
same 1 px stroke, and a bigger canvas makes it worse because there are more columns to cover.

**Rejected: forcing LineWidth 2.** It works, but MATLAB's default is 0.5 and MATLAB
anti-aliases, so the sandbox would render visibly bolder than the user's own machine - and the
model must never be shown something different from what the code produces.

**Change: a transparent `print` shim.** gnuplot has a `pngcairo` terminal that anti-aliases,
and Octave knows how to drive it. The model keeps writing ordinary MATLAB - `print(f,'p.png',
'-dpng')` - because whatever it writes is what the user runs, and `-dpngcairo` would be a
syntax error in MATLAB. A `print.m` on the path ahead of Octave's own rewrites the device,
then removes itself from the path to call the real one.

It lives in `/opt/owuitools-shim`, outside any working directory, so it can never be shadowed
by a user file or reported as a user artefact. It also pins `-S900,675` and `-F:8`, because
cairo's default canvas and point size are quite different from the plain terminal's and titles
were being clipped.

Five edge cases, all correct: `-dpng` rewritten; no device at all inferred and rewritten;
`-dpdf` left alone and producing a valid PDF; an explicit `-S400,300` winning over the
default; and `saveas`, which routes through print, also caught.

**Two noise sources fixed in passing**, both of which were landing in the same stderr the
model reads for real errors: the `Octave:shadowed-function` warning on every print - silenced
*before* the addpath, since it fires the moment the shim joins the path - and gnuplot's
"iconv failed to convert degree sign", which is a missing UTF-8 locale in the container.

Result: all three sine traces continuous, at 900x675, with no gaps.

---

## Change 42 — the legend was transparent

the operator: the legend should sit on top of the plot lines with a white background.

The legend axes already reports `color = [1 1 1]`, so the obvious guess - that Octave was
setting it transparent - was wrong. The cause is one line further down, in what Octave tells
gnuplot. From `__gnuplot_draw_axes__.m`:

```matlab
fprintf (plot_stream, "set key %s %s;\nset key %s %s %s %s %s %s;\n",
         inout, pos, box, reverse, horzvert, fontspacespec, colorspec,
         __do_enhanced_option__ (enhanced, hlgnd));
```

Every key option is passed except `opaque`, and **gnuplot's default is `noopaque`** - so the
key is drawn with no fill and the data runs straight through the labels. On a dense plot the
legend became unreadable, which is exactly the plot where a legend earns its place.

**Change.** One extra line after that call, applied to the vendored Octave file at image build
time:

```matlab
fputs (plot_stream, "set key opaque;\n");
```

Build time rather than runtime because it edits a library file: in the Dockerfile it is
reproducible, visible in review, and cannot be half-applied. The patch script checks the
anchor is unique, is idempotent, and **exits 0 when the anchor is absent**, so an Octave
upgrade that moves this code degrades to the old rendering instead of breaking the build.
gnuplot 5.4 in this image accepts `set key opaque`, verified before relying on it.

Verified on a deliberately hostile case - three overlapping traces running under the legend
corner - and on an ordinary four-series plot: solid white fill, box, key samples in their
correct colours, curves passing cleanly behind.

---

## Change 43 — prefill batch size, measured before and after

the operator asked for speed work with a before/after, three runs each side, since the model is not
deterministic.

**Setup.** Three fixed medium prompts, each in a fresh chat, driven through Open WebUI so the
real payload (system prompt + 21 tool schemas ~= 13.5k tokens) is used. Measurements come from
llama-server's own per-pass timing lines, not from wall clock, so tool latency and network are
excluded. Only passes that re-prefill more than 5,000 tokens are compared - those are the ones
batch size affects.

**Result: prefill throughput +20.9%, with distributions that do not overlap at all.**

| | b=512 (before) | b=1024 (after) |
|---|---|---|
| samples | 5 | 7 |
| tok/s | 700.5, 711.7, 699.4, 708.6, 706.2 | 852.7, 864.9, 842.9, 853.8, 854.3, 847.8, 853.1 |
| mean | **705.3** | **852.8** |
| median | 706.2 | 853.1 |
| range | 699.4 - 711.7 | 842.9 - 864.9 |

Two identical prompts measured on both sides:

| prompt tokens | before | after | saved |
|---|---|---|---|
| 13,516 | 19.29 s | 15.85 s | **3.44 s** |
| 13,624 | 19.48 s | 16.16 s | **3.32 s** |

Decode is unchanged - 46.3 vs 47.9 tok/s mean, both spanning 39-60 - which is what should
happen, since `num_batch` governs prefill only. VRAM rose 29,383 -> 29,927 MiB for the larger
compute buffers, leaving 2.8 GB spare.

**`num_batch` alone does nothing.** Setting it on the model was accepted by the API with no
warning and no effect: the runner kept `-b 512` because Ollama does not treat it as part of
the runner key and so never reloaded. The first "after" run measured the old configuration and
looked like a null result. It needs `docker restart ollama` to take, and the flags should be
confirmed in the log (`starting llama-server ... -b 1024 -ub 1024`) rather than assumed from
the API returning 200.

**Also applied: `RAG_EMBEDDING_BATCH_SIZE` 1 -> 32.** The audit measured this at ~180 s per
file-attached or web-searched message, which would have been the single biggest win. It is
worth **zero seconds** on this deployment: across **609 chats there are no RAG files and no
Open WebUI web searches** - everyone reaches the web through owuitools' `web_search` and
`fetch_page`, which return text as tool results and never touch the embedding path. Applied
anyway because it is free and correct if anyone ever attaches a file, but it is not a win and
should not be counted as one.

---

## Observation — why tool calls re-prefill, and it is not Open WebUI's fault

the operator asked whether the model re-prefills its whole context after every tool call. Often it
does, and the cause is architectural.

**162 occurrences in 24 h:**

```
forcing full prompt re-processing due to lack of cache data
(likely due to SWA or hybrid/recurrent memory)
```

qwen3.8 is architecture `qwen35`, a hybrid carrying recurrent/SSM state alongside attention -
only 16 of its 66 layers hold a KV cache. llama.cpp cannot rewind recurrent state to an
arbitrary position, only to stored context checkpoints, spaced 8,192 tokens apart by default.
When a tool result lands before the nearest checkpoint the whole prompt is re-processed **even
though the KV prefix is sitting in memory**. Across those forced passes the median already had
**80.3% of its tokens matching** and discarded them - roughly **1,640 s/day** of redundant
prefill.

When the cache does hit, it is excellent: 74% of all prompt tokens presented over 24 h were
reused, a warm pass re-prefills ~650 tokens in 1.7 s, and an identical prompt goes 14.38 s ->
0.33 s. So caching is not the missing piece; it is already carrying most of the load.

**A hypothesis I tested and disproved.** The obvious suspect was my own
`quantitative_question_nudge` filter, which injects a system message that exists only in the
outgoing payload and not in stored history - that would change the prompt at token ~0 on the
next pass. The correlation looked convincing: both 100%-reprefill tests had units in them, the
cache-hit test did not. Disabling the filter and re-running the same question gave the same
100% re-prefill. Not the cause. Filter re-enabled.

`LLAMA_ARG_CHECKPOINT_MIN_SPACING_NT` and `LLAMA_ARG_CTX_CHECKPOINTS` look like the dials that
would address it. They are not - see the next section, which measured them before changing
anything and found both already slack.

**Two other facts worth recording.** `reasoning_effort: "xhigh"` on the model has never done
anything - Ollama rejects it, 452 `invalid option provided option=reasoning_effort` warnings.
And 70.3% of generated characters are thinking rather than answer text, while generation is
~75% of wall clock; the dial that would govern that is the root-level `think` parameter, which
is plumbed for Ollama and unset on this model. That is the largest quantity on the board and
it is a deliberate quality trade, not a free win.

## Observation — the checkpoint dials are already slack; the problem is *where* checkpoints land

I was about to set `--checkpoint-min-step` lower and `--ctx-checkpoints` higher, on the theory
that checkpoints were too sparse to catch a divergence. I measured first. Both assumptions were
wrong, and the change would have bought nothing.

**Neither flag is gating.** Ollama passes neither on the runner command line, so both sit at
their defaults (32 checkpoints, 8192-token minimum spacing). But the observed spacing is nothing
like 8192:

    created context checkpoint 1 of 32 (pos_min = 13000, ...)
    created context checkpoint 2 of 32 (pos_min = 13508, ...)

508 tokens apart. Checkpoints are minted at the *end of each prefill pass*, and that path does
not honour the minimum spacing, so spacing is already governed by conversation turn length
(~500-1000 tokens) rather than by the flag. Lowering the flag cannot make them denser than they
already are. The count never binds either: over 24 h the slot never held more than **3 of 32**.

**The real mechanism is position, not density.** Over 660 checkpoint lookups in 24 h:

| | n | min | p25 | median | p75 | max |
|---|---|---|---|---|---|---|
| divergence point | 660 | 1 | 9,218 | **9,415** | 10,617 | 55,437 |
| checkpoint position | 1,078 | 8 | 11,055 | **13,053** | 18,157 | 65,899 |

Conversations diverge in a tight band around 9.4k. The earliest checkpoint that exists is around
11k. Every checkpoint is therefore *downstream* of the split and gets erased unused:

    checking checkpoint with [18825, 18825] against 10618...
    erased invalidated context checkpoint (pos_min = 15304, ...)
    erased invalidated context checkpoint (pos_min = 18317, ...)
    erased invalidated context checkpoint (pos_min = 18825, ...)

**538 of 660 lookups (81.5%) found only checkpoints after the divergence point.** 121 restores
against 176 forced full re-prefills.

Why 9.4k: that is the length of the static prefix - system prompt plus the 20 tool schemas -
shared byte-for-byte by every user and every chat. Six users share one slot (`-np 1`), so each
user switch diverges exactly at that boundary. The checkpoint that would rescue the work is one
at ~9.2k, just *below* the band, and it is never created, because no prompt in this deployment
ends that early.

llama.cpp exposes no way to pin a checkpoint at a chosen position, so there is no flag that fixes
this. What *is* under our control is the size of the static prefix, since it sets the floor cost
of every forced re-prefill. Trimming tool schemas or the system prompt is the lever; sacrificing
tools for it is not obviously worth it, and is a quality trade rather than a free win.

**Scale, in the terms that matter.** Prefill is ~22% of wall clock and generation ~74%. So the
batch-size change in Change 43 (+20.9% prefill throughput) is worth **~4% off total response
time**, not 21%. Eliminating the checkpoint waste entirely - about 53% of all prefill time is
redundant re-processing - would be **~12%**. Neither is close to the `think` dial, which governs
~70% of generated characters.

## Change 44 — live progress status, and the Explored dropdown expanded by default

Two complaints, one underlying cause: **you cannot see what the model is doing.** Until the
first token arrives Open WebUI shows nothing at all, and this deployment's prompt is ~9.4k
tokens that frequently re-prefills from zero - so "nothing at all" was the most common thing on
screen. Once tokens did start, the tool calls and reasoning were squashed into a single
collapsed "Explored" line.

**A status line cannot change the answer, and that is provable.** The payload sent to the model
is rebuilt from scratch by an explicit allowlist:

    utils/payload.py  convert_messages_openai_to_ollama()
      new_message = {'role': message['role']}
      # then only: thinking, content, tool_calls, tool_call_id, images

Status events persist to a SIBLING key (`statusHistory`, models/chats.py:1262-64), which appears
nowhere in payload construction - the backend's only other references are the DB column mapping.
The token sequence is byte-identical with the filter on or off, so there is no prompt change and
no prefix-cache perturbation.

### The filter: `owui-functions/live_progress_status.py`

Emits an Open WebUI status event on **phase changes only** - not per chunk:

    Processing prompt                        <- inlet
    Thinking · prompt processed in 3.4 s     <- first reasoning delta
    Thought: <opening of that reasoning>     <- when a reasoning block ends
    Calling web_search                       <- a tool call is being written
    Writing response                         <- first content delta
    Done · 3 tool calls                      <- outlet

`StatusHistory.svelte` renders only `history.at(-1)` as the headline and collapses the rest, so
accumulation is not clutter - the headline always shows the current phase.

**Two sharp edges in the stream hook, both load-bearing.** First, the return value REPLACES the
chunk and a *falsy* return DROPS it:

    data = wrap_item(JSONCodec.dumps(event)) if event else None

so `stream()` returns `event` on every path; returning None would silently delete the model's
output. Second, exceptions from a stream filter are re-raised by `utils/filter.py`, which would
break the response mid-flight - so the body is wrapped and always falls through to `return event`.

The module object is cached on `request.app.state` and therefore shared process-wide across all
six users, so per-request state is keyed by message id and pruned at 64 entries; a request whose
outlet never runs (disconnect, cancelled generation) cannot leak.

Offline tests cover the phase sequence, multi-round tool loops, malformed chunks (`{}`,
`{"choices":[]}`, `{"choices":[{"delta":None}]}`, non-dict input), an emitter that raises, and
the state bound. Every one passes the chunk through unchanged.

### The dropdown: tried, reverted, replaced by the indicator

First attempt was `ui.default_interface_settings` = `{"expandDetails": true}`, which works
(`ConsecutiveDetailsGroup.svelte:44` reads it, and `routers/users.py:482-488` merges admin
defaults into what every client loads). But **one setting drives three levels** - the outer
group, the reasoning `Collapsible`, and `ToolCallDisplay` - so it also expanded every tool's
INPUT/OUTPUT, which is not wanted. There is no second setting; splitting them needs a frontend
change, and `open-webui` carries `wud.watch=true` on the moving `:main` tag, so any patch to the
compiled JS would be silently wiped by the next auto-update. Reverted to `{}` (its exact prior
value; verified no user had a personal override either way).

**Reverting the admin default was not enough, and this is the trap.** Setting
`ui.default_interface_settings` causes the backend to MERGE it into what `getUserSettings`
returns (`routers/users.py:488`), the client loads that merged object into `$settings`, and then
any ordinary `saveSettings` call - picking a model, pinning something - writes the whole merged
object back as the user's PERSONAL settings. So `expandDetails: true` got baked into the admin
account. `strip_default_interface_settings` only removes a personal value that still equals the
current default, so once the default went back to `{}` the personal `true` was stranded and kept
winning. Symptom: reasoning blocks still auto-expanded for that one account after the default was
reverted. Fixed by deleting the single key from that user's `settings.ui`.

Worth remembering: an admin "default" in Open WebUI is not a passive fallback - touching it can
permanently write that value into every active user's personal settings. Check
`select name, settings from user` after changing one, not just the config key.

Two incidental hazards found while cleaning up: `user.settings` can be the literal JSON `null`
(two of six accounts), so `json.loads(s).get(...)` raises `AttributeError` on a truthy string -
guard with `isinstance(st, dict)`. And the first cleanup script crashed on that before reaching
`commit()`, so it silently changed nothing; the re-run verified per-user effective values
afterwards rather than trusting the update.

**The status timeline replaces it.** Rather than patch the frontend, the filter now posts the
opening of each reasoning block as its own status when that block ends, so the collapsed view
still tells the whole story:

    Done · 10 tool calls                                    <- headline, click to expand
      Processing prompt
      Thinking · prompt processed in 5.2 s
      Thought: The user is asking whether TCP Fast Open is worth enabling on a busy web…
      Calling lookup_rfc
      Calling web_search
      Thinking
      Thought: The web searches didn't return great results (mostly MDN glossary pages)…
      Calling lookup_rfc
      Thinking
      Thought: Good, I have the RFC's key points. The RFC itself notes: - TFO is **exper…
      ...
      Writing response
      Done · 10 tool calls

That is the chain of thought and the tool sequence at a high level, with no tool payloads
anywhere - and it survives OWUI updates, because it is a filter rather than a patched asset.
`thought_excerpt_chars` (default 160, 0 disables) bounds each line; the renderer clamps a status
to one line, so more would be wasted. The reasoning buffer is capped at ~4 KB per message and is
not filled at all when the excerpt is disabled.

**A bug the tests caught.** The reasoning buffer was added to the state dict in two places by a
replace-all that only matched one of them - the indentation differed - so `stream()` raised
`KeyError: 'reasoning'` on the first reasoning delta. The filter's own blanket `except` swallowed
it, and the only visible symptom was that every status after "Processing prompt" silently stopped
appearing. Fixed by collapsing both literals into one `_new_state()` factory. Worth remembering:
the broad exception handler that makes this filter safe also hides its bugs, so the offline test
is not optional.


---
## The 40-prompt graded run, and what it changed

`prompt-testing/` was built as forty engineered prompts with a rubric each, run through the
chat UI and graded against independently verified ground truth. Mean **7.53/10**: 26 pass,
8 partial, 6 fail. Correctness was 87% and **calibration 68%** — the model is mostly right
and frequently unable to say how it knows. Ninety-nine points were lost, and they sort into
six causes:

| pts | cause |
|---|---|
| 30.5 | treated a name, date or office-holder as settled knowledge and skipped tools, or searched with its own guessed answer in the query |
| 28.5 | right data, no URL, or the source mislabelled |
| 20.5 | a tool output gap induced the error |
| 10 | a decoding loop that produced no answer at all |
| 6 | code or logic slip on a well-described tool |
| 3.5 | arithmetic in the head instead of the sandbox |

Changes 45-52 are that list worked through. Every one of them follows the finding this log
has now confirmed four times: **the model treats tool output as fact and system-prompt text
as advice.** Where a rule had to hold, it went in tool output; the prompt changes are paired
with a tool change carrying the same rule, never standing alone.

---
## Change 45 — a repetition loop that produced no answer, and no penalty to stop it

**Symptom.** One prompt (garage circuit wiring) spent 140 s and 8,201 completion tokens
repeating two sentences about NEC 210.52 roughly 180 times inside its reasoning block, then
ended the turn with **no assistant text at all**. The status indicator showed a normal "Done".

**Cause.** Nothing in the stack discouraged verbatim repetition. `repeat_penalty` is 1 (off)
in the Modelfile and the Open WebUI model row sent `presence_penalty: 0` explicitly, which
overrides it. Qwen's own guidance for thinking mode recommends a presence penalty of 0-2
specifically "to reduce endless repetitions", and warns that higher values cause language
mixing.

**Change.** On the model row, not the Modelfile — the row is what wins: `presence_penalty`
0 → **1.0**, `repeat_last_n` unset → **256** (Ollama's default 64 does not span one cycle of
this loop). `repeat_penalty` left off deliberately: above 1 it penalises exact-token repeats,
which is what citations, part numbers and URLs are made of. `reasoning_effort: "xhigh"`
deleted — Ollama has been rejecting it as an invalid option on every call since it was set.

**Result.** Verified present in the model row and confirmed in Open WebUI's Ollama parameter
allowlist (`utils/payload.py`), so both are forwarded. The loop prompt has not been re-run;
tripwires if it is: any CJK leaking into English answers, or mean thinking tokens up more
than 15%, means drop to 0.5.

---
## Change 46 — fetch_page now says what it handed you, and admits when it handed you nothing

**Symptom.** Four distinct failures, all of them the tool being unclear about its own output.
(1) The model fetched a real paper, transcribed every number from it correctly, and credited
it to authors and a journal that appear in no tool result. (2) It wrote "I pulled this
directly from the RM0008 PDF" when all nine of its PDF fetches had failed and the values came
from a lecture-note snippet. (3) A 1,293-character navigation shell from a client-rendered
catalogue read as a successful fetch, and it answered from memory instead. (4) A 13 MB
reference manual was silently truncated at the 12 MB ceiling and then reported to the model
as "may be a scanned image, a video, or a page that renders entirely in JavaScript" — false
in every particular.

**Cause.** `requirements.txt` says trafilatura was chosen partly because "it hands back
metadata … which is what the citation lines are built from", and nothing ever asked it for
any. Extraction began at the abstract, so the model had the numbers and no citation, which
is exactly the state in which a plausible one gets invented. `get_bytes` `break`ing past the
size limit returned partial bytes with no flag. And a short extraction from a large document
had no meaning attached to it at all.

**Change.** A citation header before the content — title, authors, journal, publisher, DOI
and a `CITE AS:` line — built from Highwire/PRISM/Dublin Core meta tags, with trafilatura as
the fallback for ordinary pages and a DOI/arXiv regex for PDFs. The journal is read only from
tags that name the container: `og:site_name` is deliberately not consulted, because on an npj
Flexible Electronics article it says "Nature", which is the precise mislabel that cost the
graded run. When no bibliographic data exists the header says so and tells the model to cite
by title and URL rather than filling it in. Access notes distinguish a PMC full text from a
PubMed catalogue record, an arXiv abstract page from the paper, and a publisher landing page
from either. `_unreadable()` separates a scanned PDF, an empty extraction and a client-rendered
stub, and the stub message quotes what it did extract and suggests three concrete next moves.
Oversized downloads now raise instead of truncating, with a truthful message and a 40 MB
allowance for PDFs specifically. The `CITE AS` builder is shared with `search_papers` via
`toolbase.cite_line`.

**Result.** Verified live: a PLOS article returns `Journal: PLOS ONE · Publisher: Public
Library of Science · DOI: 10.1371/...` with a correct `CITE AS: Ravenscroft et al.`;
example.com returns the no-metadata line. Two real environmental findings fell out of the
stub detector on its first run — **nature.com and PMC both now serve this server a bot wall**,
and both previously read as successful fetches of a few hundred characters. One bug the probe
caught and fixed: citation meta tags write "Surname, Given" while page bylines write "Given
Surname", so arXiv's "Vaswani, Ashish" was being cited as "Ashish et al.".

---
## Change 47 — web_search says how well it matched, instead of always saying "read one"

**Symptom.** The trailer "Next: call fetch_page on the most promising URL to read it properly
before answering" was appended to every non-empty result list unconditionally. Twelve rubrics
recorded it either obeyed onto a useless page or ignored with no consequence — the two ways a
fixed instruction fails. A search for a fabricated paper title, with four of five engines
down, returned six unrelated results and read exactly like a search that had found the answer.

**Cause.** No relevance signal reached the model at all. Separately, SearXNG's scholarly
engines return `journal`, `authors`, `doi` and `pdf_url` on every hit and all four were being
dropped, and `category: "science"` was documented nowhere while quietly routing consumer-health
and everyday-physics questions to arXiv.

**Change.** A term-overlap floor produces one of three trailers: read-the-page above 50%,
`[WEAK MATCH]` above 25%, `[LOW RELEVANCE: … treat this as NO RESULTS for what you actually
asked]` below it. A quoted phrase that appears in no result forces the low tier regardless of
word overlap, because the words of an invented title are ordinary words and scatter. Results
now carry a `CITE AS:` line and a PDF link when the engine supplied them, with a note that a
journal not on one of those lines is not known. The read-the-page trailer now also requires
a snippet-sourced value to be labelled as such. The `query` field description says to search
for the thing and not for your answer; the `category` description says what `science` really
is. A domain-shaped hint points dosing questions at DailyMed, medicines.org.uk, MedlinePlus
and NHS, and flags `science` on a practical medicine question. Results already read or already
failed in this conversation are annotated, applied after the cache so the annotation follows
the conversation rather than being baked into a shared entry. In `searxng/settings.yml`,
`openalex` and `crossref` are enabled — they ship disabled upstream and are the two engines
carrying real bibliographic metadata.

**Result.** All verified live. A fabricated quoted title now returns `[LOW RELEVANCE]`; a good
query still returns the read-the-page trailer; a dosing query returns the DailyMed hint; a
science query returns `CITE AS:` lines. `openalex`, `crossref` and `pubmed` confirmed live in
the science category.

---
## Change 48 — wikipedia_lookup took the top full-text hit and called it the article

**Symptom.** Asked for the famous son of Mary Lee Pfeiffer, the tool returned the Tom Clancy
novel *Red Winter*. The model — which had already guessed an answer — read the miss as
confirmation, searched "Mary Lee Pfeiffer mother of Bob Dylan", and delivered Bob Dylan with
an invented supporting biography. 0/10. Separately, the lead-only default meant a call for the
Eiffel Tower's inauguration date returned a summary that does not contain it, and the wasted
call sent the model to a general search.

**Cause.** Resolution went straight to `list=search` and took `hits[0]`, with no exact-title
attempt, no disambiguation handling, and no way to tell "this article is about your topic"
from "this article contains your words". There was also no way to reach into the body at all.

**Change.** Exact title first, following redirects, with disambiguation pages surfaced as a
list of options rather than silently resolving to one. Full-text search only as a fallback,
and when the best hit's title overlaps the topic by less than half, a `NO ARTICLE` result that
names the near misses, says they are about something else, and tells the model to search the
name on its own with no guessed relation attached. `search=` and `offset` for reaching into
the body, and the intro-only header now says the lead rarely carries the specific fact.

**Result.** Verified live, and this is the clearest end-to-end result of the change set. Re-run
through the chat UI, the same prompt that scored 0/10 now reasons *"this is a settled fact
question — but I should verify rather than recall"*, gets `NO ARTICLE` from Wikipedia, searches
`Mary Lee Pfeiffer mother` with no answer attached, and returns **Tom Cruise**, correctly.

---
## Change 49 — reading a long document, and the budget that punished it

**Symptom.** Asked for the scorers in a football final, the model read 4,000 characters of a
118,000-character article, found the headline result in the lead, stopped — and supplied the
scorers, the minutes and the shoot-out misses from memory, wrongly. On another prompt the
budget printed STOP FETCHING on page nine of a single document. A search for an RFC section
number reliably landed in the table of contents, burning two of four calls on dot leaders.
`read_man_page` turned "ip route" into `iproute`, 404ed, and reported "No manual page found …
do not describe its options from memory" for a page that exists.

**Cause.** The continuation hint ended "Only do so if this section did not answer the
question", which reads as permission to stop; a lead section answers the headline question and
never carries the itemised detail. The budget counted *calls*, so paging one document spent it.
`_find_in_document` took the first literal match, and a contents entry is the one place a
section number is guaranteed to appear and guaranteed to be useless. And the man-page name
filter deleted spaces rather than hyphenating them.

**Change.** The hint now says to keep reading, that what you have not read you do not know,
and to jump with `search=` instead of paging. `_find_in_document` moved to a shared
`textwin.py`, gained table-of-contents skipping — preferring a body match, falling back to the
first match if every one looks like a contents line, and reporting how many it stepped over —
and is now used by `fetch_page` and `wikipedia_lookup` as well as the two tools that had it.
The budget counts distinct URLs, so continuing inside an open document and cache hits cost
nothing; it remembers which URLs already failed and declines to re-fetch them; and the note
moved into the header, where the response cap cannot trim it away. A first, gentler note lands
at five pages. `read_man_page` hyphenates multi-word names and falls back to the first word
with the remainder as a search term.

**Result.** All verified live: the RFC jump reports skipping a contents entry, `ip route`
resolves to `ip-route`, `fetch_page` with `search=` lands in the body of RFC 9293, the same
URL fetched three times with different offsets counts once, and a URL that failed is declined
on the second attempt without a network call.

---
## Change 50 — the tool-output gaps the graded run found

Six smaller defects, each of which produced a wrong answer, and one of which was the tool
lying to the model.

- **`pcb_calc` shipped a false claim.** Its trace-width output told every caller "IPC-2152 is
  the more modern and usually less conservative standard if you need to squeeze". For a bare
  external trace that is backwards — IPC-2152 gives widths of the same order or larger,
  because the IPC-2221 external curve assumed more cooling than a real board provides. The
  model took the tool at its word, printed "~54 mil (or ~40 mil on IPC-2152)", and a reader
  following the parenthetical would undersize a 3 A trace by about half. **This is the only
  case in the whole graded run where the model amplified a harness error rather than inventing
  one.** Corrected, and an inverse width-for-target-impedance mode added, solved by bisection
  over the same forward functions so the two can never disagree.
- **`subnet_calc` split an IPv6 /64 into /67s with no caveat** while the same output carried a
  "Scope: documentation only (RFC 3849)" line — and the model's answer tracked it one for one,
  warning about exactly what the tool flagged and recommending SLAAC on /67s. The tool's
  advisory surface teaches the model what deserves a warning. It now notes the 64-bit
  interface-identifier boundary (RFC 4291 §2.5.1, RFC 7421) and that the usual answer to "I
  only have one /64" is to ask for a /56.
- **`http_headers` returned an assessment where the data was asked for** — `Present: HSTS,
  CSP, …`, never the value, so `max-age=0` and a correct two-year preloaded policy rendered
  identically. It now prints each security header's value, parses HSTS into its directives,
  and notes when a header differs between redirect hops, because an apex and its www are
  configured separately and demonstrably differ.
- **`lookup_cve` dropped `lessThan`**, so a record meaning "everything below 116.0.5845.187 is
  affected" rendered as "116.0.5845.187 (affected)" — the opposite for that version. Ranges
  now render as "from X before Y", `affected` falls back to ADP containers, the `? ?`
  placeholders are gone, and CISA KEV status plus NVD CPE ranges are reported from data the
  file was already fetching and discarding.
- **`find_datasheet` did not distinguish an exact part from a near-miss**, and its hint ranked
  candidates by domain authority alone. The model followed that hint faithfully to a Vishay
  IRLZ44 datasheet while the correct Infineon IRLZ44N sat at candidate two, and every figure
  it quoted was wrong. Candidates are now labelled EXACT, VARIANT or DIFFERENT PART, and the
  hint says an exact part number on an aggregator beats a variant on the manufacturer's site.
- **`httpc.explain` blamed the wrong thing.** Squid answers a CONNECT to a name that does not
  resolve with 503 and a policy refusal with 403, and every `ProxyError` was being reported as
  "the address is on a private network". The model invented three plausible API hostnames, was
  told each time that the host was private, never suspected the hostname, and reported to the
  user that public services were blocked here. **The h10 rubric records this as the proxy
  refusing ordinary public hosts; that grader note is wrong and should be amended.** The three
  hosts were NXDOMAIN. DNS failure, policy denial and unreachable-host are now three different
  messages.
- **`matlab_run` was never considered as a calculator**, and its Octave dialect gaps cost three
  of four calls on a constrained-writing task. It now advertises itself as a scratchpad, takes
  `purpose: "scratch"` (which skips the MATLAB-validity verdict and answers a missing function
  with the Octave equivalent rather than "do NOT rewrite it"), explains the `'tokens'` option
  name, and lints two new semantic traps — `\b` in a regexp being a backspace rather than a
  word boundary, which fails silently with a wrong number, and `idivide` rounding negatives
  differently from MATLAB.

**Result.** All verified live through the gateway.

---
## Change 51 — the system prompt, where it was IT-shaped and the questions were not

**Symptom.** Thirty and a half points — the largest single bucket — went to the model deciding
a question needed no tool. The pattern is consistent: a name, a date, an office-holder, a
poem's wording, a count of human senses. In one case its own reasoning read *"settled general
knowledge"* — the exact phrase `wikipedia_lookup`'s docstring used to describe what the tool
is *for*, read as an exemption from using it.

**Cause.** Every trigger in the prompt's lookup list was IT- or EE-shaped: versions, flags,
part numbers, CVEs, "any number the user might act on". Nothing named people, dates or
quotations, and `wikipedia_lookup` was a bare name in the tool list with no purpose text.
"Doing the arithmetic" named no tool at all, and `matlab_run` was described only as executing
code — so it went unconsidered for counting letters or compounding interest.

**Change.** A sixth trigger covering names, dates, office-holders, relations, what a text
actually says and how many of something there are, stating that "everyone knows this" is a
trigger rather than an exemption. A rule against confirmation-seeking queries, paired with the
same rule in `web_search`'s own schema. "Doing the arithmetic" now names the calculators and
`matlab_run` with `purpose: "scratch"` explicitly. The first Answering bullet is now a hard
sources rule: any tool-assisted answer ends with its URLs, a snippet-sourced value is labelled,
and "from the manual" is forbidden unless a fetch of it succeeded. The honest `category` guide
replaced the one that mis-described `science`. "Ask once" moved out of the MATLAB section,
where it was scoped to code, into the general honesty section. Paid for with four cuts: the
numbered iterate list, the casual-question paragraph (the nudge carries it), the six-fetch
bound (the budget carries it now) and the plot-PNG paragraph (duplicated verbatim in the tool
docstring). Net +200 words on 2,175.

**Result.** Measured on two prompts. The Mary Lee Pfeiffer prompt went from 0/10 to correct
(Change 48). The letter-counting prompt, which previously made zero tool calls and counted in
its head, now calls `matlab_run` with `purpose: "scratch"`, hits a syntax error, reads it,
fixes it, re-runs, and quotes the executed result. **One rule is not landing, and that is
accepted:** neither answer carried a "Sources:" line. It is the prompt-only half of the
attribution fix, behaving exactly as this log predicts prompt-only rules behave — and there is
no tool-output lever for the shape of a *final answer*, only for what the tool hands over. The
URLs are now in the tool output, so a citation can be lifted whenever the model reaches for
one. Reviewed and kept as a deliberate compromise: the model is carrying a great deal of
instruction already, and this rule costs nothing on the turns it is ignored. Not an open
defect. Revisit only if citation gets *worse* than the baseline, where 6 of 40 tool-assisted
answers carried no URL.

---
## Change 52 — the status indicator's numbers were not the model's numbers

**Symptom.** Thirty-five of forty rubrics independently flagged the same thing: the live
indicator's "prompt processed in N s" disagreed with the same turn's `prefill=` figure, often
by two or three times, in both directions. Two smaller ones: a turn with three tool calls
reported "Done · 2 tool calls", and the 140-second turn that produced no answer at all
reported a normal "Done".

**Cause.** The figure was wall-clock from `inlet` to the first streamed token — which includes
Open WebUI middleware, queueing and model load — and was labelled as prompt processing. The
tool counter lived inside the display valve and was zeroed when `inlet` ran a second time for
the same message, which is what an interactive `ask_user` call causes. And `outlet` never
checked whether any content had been produced.

**Change.** Real `prompt_eval_duration` and `prompt_eval_count` read from the stream's usage
chunk, with the message-usage fallback; the early label relabelled to "first token after N s",
which is what it actually measures; the final line now carries prefill seconds, prompt tokens
and generated tokens. State survives a second `inlet`. Counting moved outside the display
valve. A blank final message produces "Done · no answer produced — the model stopped inside
its reasoning" plus a visible note in the message itself. The thought excerpt now shows head
and tail, after one surfaced as "…I should verify with actual execution since I have tools.
Actually, I…", cutting off precisely the clause that reverses the meaning.

**Result.** Verified live. A real turn now ends `Done · 2 tool calls · prefill 18.4 s for
14,897 tokens · 233 generated`, against a first-token time of 90.5 s — a 5× gap that the old
label was reporting as prompt processing time.

---
## Change 53 — the tool definitions were the prefill cost, and most of it was not information

**Symptom.** After Change 50 a short question cost about 14,900 prompt tokens against a baseline
cluster of 10,800–13,700, and the increase was attributed to longer tool-schema descriptions.
Measured properly, the 21 tool definitions came to 9,812 tokens as Ollama renders them (cl100k
count of the JSON, after Ollama's own key filtering) — more than the system prompt, and paid on
every turn of every conversation.

**Cause.** Three layers, each adding text the model cannot use. The docstrings and field
descriptions had grown by accretion: every failure in this log had added a sentence, and the
sentences explained *why* — "someone may patch, or not patch, based on your answer" — when the
system prompt already carries the why and the schema only needs the what and the when. Several
rules were stated three times (once in `web_search`, once in `fetch_page`, once in the prompt).
Underneath that, pydantic emits `"title": "Max Chars"` on every property and wraps every optional
field as `anyOf: [{type: T}, {type: null}]`; Open WebUI copies both into the tool definition.
And FastAPI put the `Authorization` and `X-OpenWebUI-*` header dependencies into the spec as
parameters, which Open WebUI's converter does not distinguish from body fields — so every tool
carried one to five blank-described parameters named `authorization`, `x-openwebui-user-email`
and so on. Ollama 0.32.15's `ToolProperty` keeps only `type`, `anyOf`, `items`, `enum`,
`properties`, `required` and `description` (verified against `api/types.go`), so `title`,
`default`, `minimum` and `maximum` were already invisible to the model — which also means a
default the model should know about has to be in the description text, not the schema.

**Change.** Every docstring and field description rewritten to state what the tool does, when
to reach for it, and the one rule it must not break, with the justifications cut. Every
behavioural rule that entered this log as a fix survives — "search for the thing, not for your
answer", the honest `science` category guide, `purpose='scratch'`, the multi-word man-page
names, the AC-drop-not-DC warning, the `return_figure` self-check caveat — in fewer words. The
header dependencies are `include_in_schema=False`. A post-processor on the generated OpenAPI
document strips `title`, drops `default: null`, and collapses a nullable `anyOf` to its real
type; the server still accepts null. Descriptions now carry the defaults that matter (`max_chars`
6000, `port` 443, `purpose` deliverable) because the schema's `default` never reaches the model.

**Result.** Tool definitions as Ollama renders them: 9,812 → 6,943 tokens, a 29% cut, the
largest single savings on `matlab_run` (1,105 → 658), `wiring_calc` (916 → 724) and `web_search`
(797 → 519). All 107 smoke cases pass unchanged. A live before/after on one prompt through the
chat UI was attempted and not completed — the browser extension froze mid-run, the same failure
as during Change 52 — so the token figure is a tokenizer estimate, not an Ollama
`prompt_eval_count`. Open WebUI caches the spec at startup, so it was restarted to load the new
definitions.

---
## Change 54 — three findings from the graded v2 round

The v2 round scored **8.45/10** against v1's 7.53 (33 pass / 5 partial / 2 fail, from
26 / 8 / 6). Calibration & honesty went 68% → 84%, clearing the ≥80% target with
correctness holding at 94%. Three of the four v2 targets were met; this change set is
the fourth, plus the two clearest remaining tool gaps.

The round also produced a result that qualifies this log's governing principle, and all
three changes below are shaped by it. Tool output the model **acts on** reports a fact:
`lookup_cve`'s KEV line, `wikipedia_lookup`'s `NO ARTICLE`, the client-rendered stub's byte
counts, `subnet_calc`'s SLAAC advisory, `http_headers`' apex-versus-www note, `matlab_run`'s
"nothing ran at all". Tool output the model **declines** issues an instruction: `find_datasheet`'s
`[VARIANT … the suffix is the part]` (read, quoted in its own reasoning, overridden), the
DailyMed dosing hint (fired on both dosing prompts, obeyed on neither), `[LOW RELEVANCE …
do not answer from memory instead]` (fired on e16 and m10, overridden on both). So: state
the fact, do not issue the instruction.

---
### 54.1 — the citation rule had nowhere to land, so it never did

**Symptom.** The one rule Change 51 knowingly shipped as prompt-only. Measured across both
rounds on the same definition — *prompts where a tool result contained a URL and the answer
contained none* — v1 cited on 2 of 23 and v2 on **1 of 25**. Thirteen v2 prompts lost a rubric
point for it. Change 51 accepted the compromise on condition that citation must not get
*worse* than baseline. It did.

**Cause.** The model will not assemble a citation out of material it has to reformat. A
`# Source:` header at the top of a fetched page reads as document identity, and a `Homepage:`
row among five other rows reads as data about the package. The single answer in eighty that
did cite lifted `search_papers`' ready-made `CITE AS:` lines verbatim, DOIs and all.

**Change.** `toolbase.sources_line()`: a one-line trailer, at the **end** of the response,
written in the shape the answer should carry — `\n---\nSources: <cite> · <url>`. On
`fetch_page` (with the bibliographic cite line when the page has one), `wikipedia_lookup`,
`lookup_rfc`, `read_man_page` and `lookup_package`. `web_search` is untouched: its per-result
`CITE AS:` lines already work, and a search has no single source. No accompanying instruction
to use it, deliberately. `cap()` now detaches and re-attaches the block so it survives
truncation — which is the whole reason the old header sat at the top — and each caller
reserves its length out of the page budget.

**Result.** Verified live: `Sources: https://en.wikipedia.org/wiki/Canberra` on a Wikipedia
call, `Sources: Ravenscroft et al., "Measuring scientific impact beyond academia…", PLOS ONE
(2017). doi:10.1371/journal.pone.0173152 · https://journals.plos.org/…` on a paper. Whether
the model lifts it is the open question and the main thing v3 measures. One real bug found
and fixed while testing: `lookup_package` returned early on a cache hit, skipping the trailer.

---
### 54.2 — "which of my terms is missing" was never stated, only implied

**Symptom.** m10 invented a randomised controlled trial — "Korhonen et al., *Circulation*,
2018" — made it the load-bearing evidence for a causal conclusion, and told the user the only
unverified part was one effect size. Automatic fail, 4 → 0. Its searches carried its own
guesses ("… Finland Korhonen Circulation", "… 750 adults 12 months"), and results that were
topically plausible read as partial confirmation. The `[LOW RELEVANCE]` trailer fired and said
"do not answer from memory instead"; the next thinking block began "I'm confident from training
knowledge that…". Separately, e12 (9 → 0) quoted a blockquote from RFC 2324 §2.3.2 after
reading only the RFC **index listing**, whose trailer said "Next: call lookup_rfc again with
the number to read the full text" — an instruction about what to do next, not a statement
about what it was holding.

**Cause.** Both trailers told the model what to do. Neither told it what was true. The
relevance floor answers "how well did this match overall"; nobody answered "*which* of the
things you asked for is not here", which is the question a fabricated name fails.

**Change.** Two facts, no instructions. `web_search` reports distinctive unquoted query terms
that appear in **no** result, as a count: `[Appears in none of these results: "Korhonen" -
0 of 6. A name, a journal, a place or an identifier present in no result was not found here;
that is a different outcome from ranking low.]` Quoted spans are excluded — a missing quoted
phrase is already the relevance floor's job, and words capitalised only because they sit in a
quoted heading ("Address offset") say nothing when absent. `lookup_rfc`'s index listing now
leads with what it is: "These are index entries: title, publication status and abstract. The
body of an RFC … is not in this listing, so nothing can be quoted from it."

**Result.** Verified live. The m10 query returns the Korhonen note; the h01 fabricated-paper
query returns Okafor, Lindqvist, Nature and Electronics at 0 of 3. Checked for false
positives against six real queries: a good RFC query, the h05 quoted-heading query, the fully
quoted e07 name and a plain English question all stay silent.

---
### 54.3 — `search=` decided the fate of two sources, in opposite directions

**Symptom.** e06 was the round's only correctness regression (7 → 6). Asked for "Constitution"
inside the Canberra article, the jump matched **"1 Constitution Avenue, Civic"** — a street —
and returned a window on self-government, bushfires and climate. The model, having looked,
stopped hedging and published **§124** for the seat-of-government clause; it is §125, and
§124 is "Formation of new States". Its own reasoning had written the right rule first — "I'll
mention it was required by the Constitution without over-specifying section numbers I haven't
verified" — and then overrode it. The mirror image at e15: `search="list of senses"` in the
*Sense* article returned only "that term does not occur", and the model abandoned 54,000
characters of exactly the right article, answered from memory, and kept a claim that humans
have electroreception.

**Cause.** A character offset tells you where a string is, not whether you are reading the
right subject; and a bare miss is indistinguishable from the document being useless. In both
cases the model had chosen the correct source and the tool's answer sent it away.

**Change.** In `textwin`: a hit names the enclosing heading — `[Jumped to 'Constitution' at
character 19,328 of 62,467; this is inside the section 'Self-government']` — recognising
Wikipedia `== headings ==`, markdown, numbered specification sections and the flush-left
capitals of an RFC or man page. A miss returns the miss notice **plus the opening of the
document**, since it is already fetched and already in memory. New `search_window()` wraps
both; `fetch_page`, `wikipedia_lookup`, `lookup_rfc` and `read_man_page` all use it, replacing
four separately-worded dead ends.

**Result.** Verified live on both motivating cases: the Canberra jump now reports section
'Self-government', and the *Sense* miss returns the article's lead, which is about senses.
Heading detection checked against all four document shapes.

---
### 54.4 — two prompts were measuring something other than what their rubric said

Not a harness change: the instrument. Grading v2 surfaced two prompts whose rubric described a
deployment this is not, and both had distorted v1 as well.

- **`m04`** tells the operator to answer the model's clarifying question with one exact
  sentence — "Z-score each column, ignore NaNs, and return the means and standard deviations
  too". The harness answers with the native `ask_user` widget, whose fixed multiple-choice
  options carry method and dimension and nothing else, so two of the four clauses never reached
  the model in either round — and correctness lines 2 and 3 then scored the delivered function
  on exactly those two. The protocol now says to use the widget's free-text field, and declares
  a run void for those lines if the sentence cannot be delivered; the scoring charges a failure
  to *ask* once, on line 1, and grades the deliverable against what was conveyed.
- **`e19`** asserts that the built-in clock is disabled and scores calibration on "explains
  *why* it cannot know". `get_current_timestamp` is exposed and the model used it in both
  rounds, so that line could never apply and both graders had to map it onto something else.
  The rubric now tests what the prompt has actually been testing: tool over recall, the right
  timezone, and saying where the date came from — which makes it the cheapest instance in the
  set of the attribution behaviour 54.1 targets.

Both masters are corrected and **both completed rounds re-scored against them**, because a
rubric change that is not applied backwards silently breaks the round-over-round columns:
`m04` v2 6 → 7, `e19` v1 9 → 10 and v2 9 → 8. v1's mean 7.53 → 7.55; v2's mean unchanged at
8.45, the two corrections cancelling. Each grade file carries a banner with the reasoning, and
the original grade text is left intact below it.

---
**Measured:** 123/123 smoke cases pass (107 before, 16 added for these three). The three
motivating transcripts re-probed through the gateway. Selector false-positive checks as above.
All 80 completed grade files re-parsed and the aggregates recomputed from them rather than
from the old table.

**Not measured:** whether the model *uses* any of it. That is v3.

---
## Change 55 — the v3 round's harness defects, and the citation line that gets written down

The v3 round scored **8.85/10** against v2's 8.45 (36 pass / 2 partial / 2 fail; no score
of 0 for the first time; calibration 87%, correctness 94%). Of the 46 points lost, roughly
12 were attribution (a correct answer, read from the right page, with no URL), 7 were tool
defects the graded transcripts pinned down exactly, 22 were the model's own recall and
embellishment, and 3 were noise. The first two buckets are the harness's to fix, and this
change takes all of them. The third is the model, and the ceiling it implies is about 9.3.

The governing principle holds: every rule below is stated as a fact in tool output, not as an
instruction. The one exception is 55.1, which is not a rule at all - it is the harness doing
the thing the model reliably does not.

---
### 55.1 — the citation line is appended, not requested

**Symptom.** Change 54.1 put a ready-to-copy `Sources:` trailer on every tool that returns a
single document. In v3 it was lifted into the answer on about half of the opportunities; eleven
answers that had read the right page (e05, e06, e07, e10, e13, e15, e16, e20, h07, m05, m08)
ended with no URL, and each lost a point for it. The cited-URL target (≥ 20 of 23) was the one
v3 target not met.

**Cause.** The model has the URL in front of it and does not write it down. Three rounds and
two carriers (system prompt, tool trailer) have moved this from 1 of 25 to roughly half; the
remaining half is not going to come from a fourth wording.

**Change.** A new outlet filter, `owui-functions/answer_sources.py` (deployed as
`answer_sources`, active and global). After the turn it collects the `Sources:` trailers from the
turn's tool results - only tools that *read* a document emit one, and only on success - and, if
the answer names none of those URLs and carries no sources line of its own, appends
`---\nSources: url · url` to the message (content and the rendered output item both, as
`inline_tool_images.py` does). Search results are never added: a URL in a `web_search` listing
was not opened, and attributing an answer to it would be the mislabelling the rubrics penalise.
An answer that already cites one of the read pages, or writes its own "Sources:" section, is left
alone.

**Result.** e05 re-run: the model's answer was unchanged and ended with
`Sources: https://pypi.org/project/requests/` - the page it had fetched. The filter fails closed
on any surprise in the message shape.

---
### 55.2 — twelve tool defects from the graded transcripts

Each of these was identified in a v3 grade file with the transcript line that caused it. All are
domain-shaped; none is keyed to a test prompt.

**`fetch_page` served a glyph-encoded PDF as text (m06).** The Infineon datasheet's fonts carry
no character map, so extraction returns `/G73/G68/G32…` for every cell of every table; the tool
reported 28,716 characters and no warning, and the model reverse-engineered the glyph numbering,
got the one value the question turned on wrong, and answered the rest from memory. Now: when
more than 35% of a PDF's text is glyph codes (`/G123`, `(cid:123)`) the fetch FAILs with the
reason and the alternatives (a distributor's copy, the vendor's HTML page); between 3% and 35%
the header carries a warning that the tables may be unreadable. The Infineon file measures 78%.

**Open-access papers were unreachable by every route (m10, e13, h07).** PMC, PubMed and Europe
PMC's article pages all sit behind a browser check or cookie wall for a plain client; m10 tried
five routes to one open-access paper and got a stub from each, then found by itself that Europe
PMC's REST API serves the same full text as JATS XML with no check. `fetch_page` now tries that
API first for any URL naming a PMC article, and for a PubMed or Europe PMC record whose paper is
in PMC (resolved through the API's search endpoint). The XML is rendered to markdown with the
front matter as citation metadata (title, authors, journal, year, DOI - so the `CITE AS` line is
built from the article, not from a landing page), and the header says
`Retrieved via: …/rest/PMC…/fullTextXML`. When the API has nothing, the stub message says so, so
the model does not keep trying PMC routes.

**Bad parameters came back as JSON 422 bodies (m06, m03-v2).** `[MORE AVAILABLE: 10,716
characters]` was answered with `max_chars: 10716`, and the schema bounced the whole call.
`max_chars` is now clamped to its bounds instead; any other validation failure is returned as
prose with a 200 (`CANNOT DO THAT: the call was rejected before it ran - offset: Input should be
greater than or equal to 0 (you sent -5). Fix that parameter and call again…`), matching every
other failure in the suite; and the continuation hint names the cap.

**Two 504s arrived as nginx HTML (h05).** Open WebUI sets no tool-server timeout of its own
here, so the gateway's 90 s `proxy_read_timeout` was the only clock, and a 995-page reference
manual took longer than that to extract. The gateway now waits 240 s, PDF extraction is bounded
at 120 s inside the app (returning prose on timeout), and the slow pdfminer fallback is skipped
above 8 MB - so the app, not nginx, is what answers.

**`search=` landed on page-numberless contents entries (e12) and reported false sections
(h05).** RFC 9110's `.txt` rendering - every RFC in the v3 text format since 2019 - lists its
contents as `15.5.19. 418 (Unused)` with no leaders and no page number, so Change 49's skip did
not fire and two jumps returned the listing. A numbered line in the opening quarter of a document
whose text recurs later as a line of its own is now a contents entry. And the flush-caps heading
shape matched a running page header (`RM0008`) and a bit-field label (`XTPRE`) on the manual's
PDF text: a line that repeats four or more times is now a header, not a heading; a single
capitalised word is used only when no structured heading sits above it in the window (which is
the normal case in a man page and never in a numbered manual); and "Table of Contents" is itself
a recognised heading, so a jump into it is told so.

**`lookup_ip_or_asn` refused documentation addresses without naming them (m02), and returned no
origin AS (h10).** "Private, loopback or reserved" was true and useless: the model knew from
memory that 203.0.113.10 was TEST-NET and talked itself out of saying so in three rounds. The
refusal now names the block and RFC and states the fact in prose (never allocated, never
announced, treat it as a placeholder). And registration is not routing: RDAP says who a block is
allocated to, not who announces it, and h10 spent four extra fetches per round on the origin AS.
Every IP lookup now carries a `## Routing (BGP)` section from RIPEstat's routing view - announced
prefix, origin AS with holder, multi-origin flagged - and an AS lookup carries the holder and
whether it announces anything.

**`subnet_calc`'s documentation scope was a field (h04).** Three rounds reproduced the SLAAC
advisory paragraph almost clause for clause and skipped `Scope: documentation only` in the same
result: an advisory paragraph crowds out an advisory field. The same fact is now also a sentence
under the Scope line.

**`read_man_page` resolved a bare name to section 1 (m05).** cron's PATH behaviour is in
crontab(5) and cron(8), not crontab(1); a search of "crontab" landed in section 1 and reported
the term absent. On a `search=` miss with no section given, the sibling sections are now fetched
concurrently: if one contains the term it is shown instead, with a note; if not, the miss note
says which sections the name also exists in. (On Arch's cronie pages neither crontab(1) nor
crontab(5) mentions PATH - cron(8) does - so this prompt now gets "the name also exists in
section 5" rather than a hit; a different-name page is not something a search of one name can
reach.)

**`pcb_calc` mis-parsed unit suffixes and printed a negative impedance (h03, twice).** The
engineering-value parser reads a trailing "mm" as the prefix milli, so `"1.6mm"` became 0.0016 mm
and the microstrip formula printed **Z₀ ≈ −46.5 Ω** as a result. Board dimensions now have their
own parser (mm by default; mm, mil, thou, µm, cm, in accepted; a bare "m" is refused as not a
board dimension), and any impedance at or below zero is a CANNOT DO THAT naming the w/h ratio
and the likely unit mix-up.

**`matlab_run`'s parse-error hint did not recognise `\'` (h08), or a function defined after its
use.** A backslash-escaped apostrophe inside a single-quoted string is the commonest way to break
an `fprintf` in either dialect; the raw message said "syntax error", the model blamed Octave, and
a call was lost. The hint now names the fix (`''`). And a function defined in the script below
the line that calls it is "undefined" in Octave and required at the end of the file in MATLAB;
the hint now says which is which instead of reading as a typo.

**`_absent_note` had two false-negative paths and one false positive (e06, m06).** The
six-term cap on `_distinctive()` dropped the trailing terms of a long query, which is where a
model's guess sits (e06: "Port Jackson" in positions eight and nine, never checked); the engine's
own "Missing: Port Jackson" annotation inside a snippet put the absent words into the haystack;
and stripping the closing bracket off `RDS(on)` produced a term that matched nothing. The check
is now uncapped (the reported list is capped at four), engine annotations are stripped from
snippets before display or matching, balanced parentheses are kept, and matching ignores
punctuation and spacing so `RDS (on)` matches `RDS(on)`.

**The dosing hint fired on "dose-response" (m10, h07).** The epidemiology term is excluded
from `_PRACTICAL_MEDICINE`; "ibuprofen dose adult" still fires.

**`cite_line` produced "Clinic," (e13) and "csmith2" (m06).** An organisational author with no
comma - a multi-word name ending in an institutional noun, or containing "of"/"for" - is cited
whole; a PDF `/Author` that is a username (one token, lower case or with a digit) and a `/Title`
that is a filename or a template name are discarded, so no `CITE AS` is built from them.

**`lookup_rfc`'s description** now says it also answers "which RFC says X", the question e12
answered from memory (RFC 7161 for the teapot code; it is RFC 2324).

---
**Measured:** 139/139 smoke cases pass (123 before, 16 added for this change). Every item above
probed through the gateway on its motivating input: the Infineon PDF is refused at 78% glyph
codes; PMC3898757, PMC8313037, PMC3976660 (a 7 KB article), PubMed 24459154 and Europe PMC
MED/30241358 all arrive via the API with `CITE AS` lines from the JATS front matter;
`max_chars: 10716` is served; `offset: -5` is prose; RFC 9110 `15.5.19` skips the contents entry
and reports section '15.5. Client Error 4xx'; 203.0.113.10 names TEST-NET-3; 8.8.8.8 carries
AS15169 and 104.16.133.229 AS13335; `12mil`/`8 mil` parse to 0.305/0.203 mm; the `\'` and
function-order hints fire; "Zorbulanquex" is reported absent; "dose-response" does not draw the
DailyMed hint and "ibuprofen dose adult mg" does. Fresh-chat spot checks through the UI, with the
prompts verbatim: six prompts, one chat each, model row unchanged.

- **e05** (3 calls, as v3): answer text unchanged; `Sources: https://pypi.org/project/requests/` appended by the filter - the cited-URL loss of all three rounds closed.
- **m02** (4 calls): both RDAP refusals name TEST-NET-2/3 and RFC 5737; the answer opens with that fact and treats the addresses as placeholders, which three rounds of memory-based hedging never quite did.
- **h03** (3 calls, v3: 5): `"12mil"` and `"1.53mm"` parsed correctly at the first attempt - Z₀ 125 Ω where v3's first call printed −46.5 Ω - so the wasted round trip is gone; 54 mil and 110 mil delivered, both traces failed.
- **m06** (8 calls, v3: 12): both exact IRLZ44N PDFs refused as 78% glyph codes, Mouser's copy timed out, and the model read Vishay's IRLZ44 sheet - and said so in its first sentence, labelling every figure as the variant's. The v3 fabrication ("0.044 Ω (4.5 V spec)", from memory, presented as the datasheet's) did not recur; the exact IRLZ44N values are still unreachable because every readable copy the tools found is a different part, which the answer states. The failure mode moved from *wrong numbers presented as read* to *labelled substitute source*, which is the change's purpose.
- **h10** (8 calls, v3: 12, v2: 17): the four ipinfo fetches are gone; origin AS13335 and the announced prefixes (104.16.128.0/20, 2606:4700::/44) came from `lookup_ip_or_asn`'s routing section, and the answer labels them "RDAP + RIPEstat routing view".
- **e13** (6 calls): the PMC review (PMC7347500) arrived through the Europe PMC API with its `CITE AS` line where v3 got a Cloudflare stub; the answer ends with all three read pages - appended by the filter, since the text named none of them. The dosing hint did not fire (no dosing term in the query), as intended.

Transcripts are in the session scratchpad, not in `prompt-testing/` - these are checks that the changes fire, not a round.

**Measured, v4 round (run and graded 2026-09-12; `prompt-testing/SCORES.md`, one `grade.md` per
prompt):** mean **8.90** (v3 8.85), 35 pass / 4 partial / 1 fail, 44 points lost (v3 46).
Correctness 92 %, calibration 88 %, tool use 91 %, communication 88 %.

- **55.1** put a URL on every answer that read a page: 16 of 16 opportunities, plus one the
  model wrote itself (`e05`) — 17 of 22 against v3's 1 of 22. The five without are four
  search-only answers (`e07`, `e14`, `h01`, `m01`) and `h10`, whose tools emit no trailer. Seven
  rubric points recovered (`e05`, `e10`, `e13`, `e15`, `e20`, `m08`, `h07`, half on `h05`). The
  ≥ 20-of-23 target is not met on the mechanical count and is met on every opportunity the
  filter has. Four defects on the one long research turn (`m10`): a "Key sources:" block is not
  recognised as the model's own attribution; a bare `doi:10.…` in the body is not matched to its
  `doi.org` entry; a trailer's `CITE AS` DOI and its URL count as two entries for one document;
  the first-six cap kept a cookie-wall stub and a wrong-paper fetch and dropped the paper the
  answer used. Also visible (`e06`): the filter cites what the tools read, not what the answer
  says — a Wikipedia URL under a constitutional claim the lead does not make.
- **55.2** — seven of the twelve confirmed in a real chat on the prompt that motivated them, each
  changing the outcome as intended: glyph PDF (`m06`, twice at 78 %, 12 → 6 calls, the v3 −2
  gone), Europe PMC API (`m10`, three articles — the effect sizes quoted from the paper instead
  of recalled), contents-entry skip + "Contents" heading + PDF header suppression (`e12` 11 → 4
  calls; `h05` three correct section notes where v3 had two false ones), BGP routing (`h10`
  12 → 8 calls, the ipinfo fetches gone), scope sentence (`h04`, the documentation prefix
  carried for the first time in four rounds; 6 → 9), `pcb_calc` units (`h03`, first-attempt
  parse, no negative impedance), absent-note uncapped (`h01`, terms at positions 8–9 reported).
  Five not exercised because the model did not reach the tool or the path: TEST-NET refusal
  (`m02` chose `lookup_rfc`), sibling man sections (`m05` made no call), both `matlab_run` hints
  (`h08`/`m07` wrote no `\'` and no script function), prose 422 (no bad parameter), dose-response
  (no such query). `cite_line` fired right on three pages (`h05` "STMicroelectronics", `m06` PDF
  metadata discarded, `e13`) and missed two new shapes: "ElectricalLicenseRenewal com" → `com,`
  (`e16`) and "Kelly Johnson-Arbor; MD Medical Toxicologist" → "Johnson-Arbor et al." (`h07`).
  The `lookup_rfc` "which RFC says X" wording drew a topic query on `m09` that got NO RESULTS
  (the index search is a title search).
- **New harness defects from the round:** `wiring_calc` reads a trailing "ft" as femto
  (`"100ft"` → 1e-13 m, success, 0.00 % drop; `e16`, two calls lost and a hand-computed figure
  wrong by 4×) — the `pcb_calc` bug's sibling; `fetch_page` on a reports index page loses the
  document links (`h02` never found the PDF v3 read, and answered from snippets and a wrong
  fiscal-year inference: 9.5 → 5.5); a PDF snippeted as raw bytes is unrecognisable; the
  string-type lint note repeats once per line (12 copies, `h06`); a "Cookies must be enabled"
  page returns as a success with a trailer; the JATS renderer dropped the authors on
  PMC8066601; PDF text layers that split words ("configurati on") defeat exact-heading
  `search=` (`h05`).
- **Model-side, no harness cause:** `e19` 10 → 0 — the clock was not called for the first
  time in four rounds and a training-data date was given ("Saturday, 19 July 2025"; the
  thinking names "the current timestamp" as its source and does not fetch it). The universal
  fix is to put the date in context (Open WebUI's `{{CURRENT_DATE}}`, or the status filter's
  inlet), which changes what the prompt tests — the user's call. Same decision going the other
  way on `m04` (wrote a parameterised function instead of asking; 9 → 7), `m05` (no man page;
  9 → 8), `e06` (s.125 recited after a lookup that did not contain it); and going right on
  `m08` (2.5 → 10, the office-holder looked up this time). Volunteered extras wrong while the
  asked-for facts are right, ten prompts: `e01`, `e12`, `e16`, `e18`, `h03`, `m01`, `m03`,
  `m07`, `m10`, `h07`.

---
## Change 56 — the v4 round's easy wins

The v4 round scored **8.90/10** against v3's 8.85 (35 pass / 4 partial / 1 fail; 44 points
lost). Of those, the grade files pin twelve to a harness cause that is a few lines each, and one
(`e19`, the only 0) to a fact that was simply not in context. This change takes all thirteen and
nothing else. Each is domain-shaped — a unit, a page shape, a lint rule, a query shape — and none
is keyed to a test prompt. **Nothing here has been run against the model**: the smoke suite is
the whole measurement, by instruction, and the v5 round is the test.

---
### 56.1 — `wiring_calc` reads feet as feet

**Symptom.** `e16`: `length: "100ft"` was accepted, parsed as 1 × 10⁻¹³ m (the trailing "f" is
femto to the engineering-notation parser, the same defect Change 55.2 fixed for `pcb_calc`),
and the tool reported a 0.00 % drop as a success. Two calls were lost to it and the figure the
model then worked by hand was wrong by 4×.

**Change.** A `_cable_length_m` reader in front of the three voltage-drop calculations. Metres
by default; an attached unit may be m, cm, mm, km, ft, feet, `'`, in, `"` or yd; anything else
("100k") is refused with "Could not read" rather than guessed. The header prints what was
understood — `30.48 m (100 ft)` — so a wrong reading is visible in the result, not hidden
behind a plausible number. The schema says the same.

**Result.** Smoke: `"100ft"` → 30.48 m, 5.29 % on 12 AWG at 20 A / 120 V (the `e16` figure);
`"30m"` → 30 m, not 30 mm; `"250 ft"` on three-phase → 76.2 m.

---
### 56.2 — `answer_sources` 0.2.0: the four defects from `m10`

**Symptom.** The research turn that motivated the filter got the worst of it: a `**Key
sources:**` block of the model's own was not recognised (the label did not start with the
word), so a second, worse list was appended under it; a bare `doi:10.…` in the body did not
match its `doi.org` entry; a trailer's `CITE AS` DOI and its page URL counted as two documents,
so one paper filled two of the six slots; and the cap kept a cookie-wall stub and a wrong-paper
fetch while dropping the paper the answer used.

**Change.** The own-sources check now accepts up to two words before `sources`, `references`,
`citations` or `bibliography` and a `used`/`consulted`/`read`/`cited` suffix, through markdown
emphasis; a DOI anywhere in the body matches the read set; each `Sources:` line is one document
(the page URL shown, the DOI kept as an alias); the cap is eight. The URL-in-body rule is
unchanged. The stub half of the fourth defect is 56.3.

**Result.** Offline, on the `m10` message: no second block; on `e05`'s: unchanged. Deployed to
the `answer_sources` row (7,734 characters, active, global).

---
### 56.3 — a cookie wall is a stub

**Symptom.** `m10`: PubMed's "Cookies must be enabled" page — 76 characters, served with a 200 —
came back as a successful fetch, with a provenance note describing an abstract that was not
there and a `Sources:` trailer the filter then listed.

**Change.** `cookies must be enabled` and `enable cookies for` join the stub markers; the fetch
FAILs with the client-rendered/gated message and emits no trailer.

---
### 56.4 — `cite_line`: a site's name is not an author, and a credential is not a co-author

**Symptom.** `e16`: "ElectricalLicenseRenewal com" as `citation_author` produced `CITE AS: com,
…`. `h07`: "Kelly Johnson-Arbor; MD Medical Toxicologist" produced "Johnson-Arbor et al." —
a single author cited as several.

**Change.** A name whose last word is a top-level-domain word (com, org, net, gov, io, …) is an
organisation and is cited whole. `toolbase.clean_authors` drops an author entry that is a
credential or begins with one (MD, PhD, RN, FACEP, "Medical Toxicologist" …) and strips trailing
degree tokens; it runs on HTML meta authors, PDF metadata and JATS front matter alike.

**Result.** Smoke: the poison.org byline cites `Johnson-Arbor,` and never `et al.`; the
licence-renewal page never `CITE AS: com,`.

---
### 56.5 — one lint note per rule

**Symptom.** `h06`: twelve double-quoted strings, twelve copies of the same "string type" note —
a screenful in which any other finding was invisible.

**Change.** `_fmt_findings` groups by rule; a rule that fires more than once prints once with
its line numbers: `- lines 1, 6, 11 (12 places): …` (first eight lines, then "and N more").

---
### 56.6 — the `std`/`var` weight flag, stated the right way round

**Symptom.** `m04`: the delivered function's comment, docstring and closing notes all said
`std(X, 0, 1)` is the population statistic and told the user to swap to `std(X, 1, 1)` for the
sample one — every clause inverted, and the same inversion v2's grade recorded. It is a belief
(the thinking says "w=0 → divide by N"), so no prompt wording reaches it.

**Change.** A `SEMANTIC` lint rule fires on any `std(…, 0|1, …)` / `var(…, 0|1, …)` and states
the fact in the run result: w=0 normalises by N−1 (the sample statistic, the default), w=1 by N
(the population statistic). A bare `std(x)` draws nothing.

---
### 56.7 — `pcb_calc` trace_width prints the width at the other copper weight

**Symptom.** `h03`: the question turns on "would 2 oz copper help"; the tool gave the 1 oz width
and the model worked the 2 oz figure by hand, with a square-root scaling it should not have.

**Change.** The result carries one more line: the same cross-section at the other weight (1 oz
↔ 2 oz), with the mil and mm width and the fact that width scales inversely with copper
thickness. The IPC-2221 area is unchanged; it is the same number divided differently.

**Result.** Smoke: 3 A, 10 °C rise → "At 2 oz copper … 26.9 mil".

---
### 56.8 — the dosing hint names the product

**Symptom.** `h07`: the DailyMed hint fired and, for the first time, the model searched
DailyMed — then took the generic 500 mg label's 4,000 mg ceiling for a branded extra-strength
product whose label says 3,000 mg, with both figures in front of it.

**Change.** One clause on the hint: "Labels differ by product and strength - read the label for
the product the question names, not a generic one." A fact about labels, not a rule about the
answer.

---
### 56.9 — `lookup_rfc` falls back to phrases

**Symptom.** `m09`: "which RFC says X" drew a topic-shaped query ("DNS domain name maximum
length 63 octets label") that got NO RESULTS, because the index search matches words in RFC
titles. The model recovered by guessing the right RFC number, which is not a method.

**Change.** `_rfc_title_search` tries the whole query, then each two-word phrase from it, then
each word of five letters or more (six tries at most). The listing says which phrase matched
("these titles contain 'Domain Names'"); the NO RESULTS text says the search matches titles, not
topics, and shows what a title-shaped query looks like.

**Result.** Smoke: that query → RFC 1035.

---
### 56.10 — document links survive extraction

**Symptom.** `h02`: the investor-relations reports index lists each report as a title, a date and
a "Download" link, and trafilatura discards the link-dense grid whole, so the model read the
index and could not get from "Interim results" to the PDF it needed — the PDF v3 had read. It
answered from snippets and a wrong fiscal-year inference instead (9.5 → 5.5).

**Change.** Before extraction, an anchor whose target looks like a document (`.pdf`, `.xlsx`,
`/document`, `/download`, `/attachment`, …) or whose text says so becomes a plain span in place:
`Download (https://…/reports/10/document)`. Ordinary navigation links are untouched.

**Result.** Smoke: the reports index now carries its document URLs.

---
### 56.11 — split-word headings, and a cross-reference is not the section

**Symptom.** `h05`: the mirror PDF's text layer renders the heading as "Clock configurati on
register", so an exact `search=` for the heading landed twice on a cross-reference to it
("refer to Section 4.3.2: Clock configuration register") and never on the heading. The model
found the section by paging to a guessed offset.

**Change.** The first pattern `textwin` tries is heading-shaped — optional section number, the
needle with whitespace tolerated inside its words, end of line — so the split heading is found
before any prose match. When the chosen match is preceded by a "refer to Section"/"see
Section" cue, the jump header says so and suggests the section number or a term from the
section's first paragraph.

---
### 56.12 — JATS author groups

**Symptom.** `m10`: PMC8066601 rendered with no authors — the archive labels the
`<contrib-group content-type="author">` and leaves each `<contrib>` bare, and the renderer
looked only for `contrib-type="author"` on the contrib.

**Change.** Labelled contribs first; then contribs inside a labelled author group; then any
contrib carrying a surname or a collaboration name. Names pass through `clean_authors`.

---
### 56.13 — the date is in context

**Symptom.** `e19` 10 → 0: for the first time in four rounds the clock was not called, and a
training-data date was given ("Saturday, 19 July 2025"). The thinking names "the current
timestamp" as its source as though it were in context; it was not.

**Change.** The system prompt's second paragraph is now
`Today is {{CURRENT_WEEKDAY}}, {{CURRENT_DATE}} (the user's local date)`, which Open WebUI
expands server-side (verified: `prompt_template` in `resolve_system_prompt`, container zone
America/New_York, the user's), with the instruction to use it for anything dated and to call
`get_current_timestamp` for the time of day, UTC or the zone name. Two lines; the rest of the
prompt is byte-identical. Written to the `qwen3.8:latest` row's `params.system` (14,608
characters; sampling parameters, tool binding and access operator unchanged — the update endpoint
re-creates the operator row, same principal and permission).

This changes what `e19` tests, from recall-vs-tool to context-vs-recall; the rubric carries a
dated revision note and accepts either the prompt or the tool as the source. The v1–v4 grades
stand on the earlier wording.

---
**Measured:** 154/154 smoke cases pass (139 before, 15 added for this change: five `wiring_calc`
unit readings, two `pcb_calc`, three `matlab_run`, two `lookup_rfc`, three `fetch_page`). The
`answer_sources` change was exercised offline on the `m10` and `e05` messages. Both deployed rows
were read back after the write. The page cache was flushed (`owuitools:page:*`) so the
document-link rewrite is what the next fetch serves.

**Not measured:** no prompt has been run against the model, by instruction. The v5 round is the
measurement for all thirteen items, and `e19` in particular now tests a different thing.

---
## Change 57 — plot type that fits the canvas, and the key under it

Reported from use, not from a graded prompt: "for the matlab renders, try to space out everything
more, it is often that text will overlap. Also, move the legend to below the render so that it
doesn't overlap." Both are the same picture — a 900 × 675 plot whose text was rendered at roughly
twice the size it was asked for, with an opaque key sitting on the curves.

Three reproductions, each a plot a model might reasonably write: three curves with a long title
and a three-entry legend; a 2 × 1 subplot stack; a bar chart with six named categories. In the
first the title ran off the right edge, the ylabel was drawn across the y tick labels (the minus
signs vanished under it), the xlabel was cut off by the bottom of the canvas, and the legend
covered a fifth of the data. In the third the six category names ran into each other and the y
tick labels were clipped to their last digit — "25, 0, 5, 0, 5, 0" for 25 down to 0.

None of it was a gnuplot limitation. It was four separate layout faults, all fixed here, all in
the renderer rather than in anything the model writes.

---
### 57.1 — a bigger canvas without bigger type

**Symptom.** Every character on every plot was about twice the size it should have been for the
canvas it was on: the 10 pt axes font drawn at 16.7 pt, the 11 pt title at 18.3 pt, on a canvas
900 px wide. That is the whole cause of the clipped title, the ylabel over the tick labels and the
bar-chart names running together. Raising the canvas size did not help, which is what makes the
cause non-obvious.

**Cause.** `-S900,675`. Octave's `print.m` reads `-S` as "a six-inch figure blown up to this many
pixels" and scales every font in the figure by `width / 6 / screenpixelsperinch` — at 72 ppi in
this container, `width / 432`, so 2.08× at 900 px and 2.55× at 1100. The type grew with the
canvas, so no size was ever big enough. The shim's `-F:8` was an attempt to hold it down and could
not: `-F` is multiplied by the same factor.

**Change.** Size by paper instead. The shim sets `paperposition` to `width/72` inches with
`paperpositionmode manual` and passes no `-S`, which takes the other branch of `print.m`: the
canvas is paperposition × 72 and the scale factor stays 1, so text keeps the size Octave gave it
and a bigger canvas buys real room. 1100 × 825 by default. The shape follows the figure — a script
that asks for a 1200 × 400 figure gets a 1100 × 440 PNG, clamped to between 0.4 and 1.5 — and a
script that passes its own `-S` is left alone entirely. `-r72` goes with it, because the JPEG
terminal multiplies paper inches by the print resolution and turned the same canvas into a
2292 px image with 10 pt type marooned in it; `pngcairo` ignores the setting.

**Result.** Tick labels and axis labels at 12 pt on an 1100 px canvas (57.2 chooses the 12),
against 16.7 and 18.3 on 900 px before. Nothing is clipped in any of the three reproductions. Smoke asserts the
canvas exactly: PNG carries width and height at a fixed offset, so the base64 prefix of the
returned data URI *is* "1100 × 825" — it fails if the shim leaves the path, if `-S` scaling comes
back, or if the canvas changes size.

---
### 57.2 — axis labels the size gnuplot reserved room for

**Symptom.** With the type fixed, the xlabel still landed on the row of x tick labels, and the
ylabel still crowded the y tick labels.

**Cause.** gnuplot lays a plot out in character units of the *terminal* font. Octave sets that
terminal font from the axes font size — 10 pt — and then draws axis labels at 11, because
`labelfontsizemultiplier` is 1.1. The space reserved below the tick labels is a 10 pt line and the
label drawn into it is an 11 pt line, so it rides up into the ticks. Confirmed against gnuplot
directly, same plot and same key: xlabel at 11 pt overlaps, at 10 pt clears.

**Change.** The shim sets `labelfontsizemultiplier` to 1 on every data axes for the duration of
the print. The title keeps its 1.1 and its bold weight — it sits in the top margin with nothing
under it to collide with.

With the tie between reserved space and drawn text in place, the size itself can be chosen on its
merits, and Octave's default 10 pt — chosen for a 560 px figure on a screen — is too small on an
1100 px canvas that a chat window then shows at 700. The shim sets 12 pt, and only where the
script did not set a size itself: `fontsizemode` records which, and an axes that says 16 pt keeps
16 pt and gets it back after the print.

**Result.** Clear separation between tick labels, axis labels and everything below them, on all
three reproductions and on log axes, 3-D and subplots. Type at 12 pt on 1100 px still leaves the
six bar-chart category names room side by side, and is legible when the picture is displayed at
700 px.

---
### 57.3 — the key under the axes, not over the data

**Symptom.** The requested change, and the reason it was requested: an inside legend sits on top
of what it labels. `'best'` does not help — this backend does not implement it, warns, and puts
every legend top-right regardless of what the script asked for.

**Change.** For a figure with one plot in it, every legend that is not already outside is moved to
`southoutside` for the print and put back afterwards. gnuplot reserves the space for an outside
key and shrinks the plot to fit, so nothing can collide with the data.

The frame comes off with the move. gnuplot packs an outside key flush against the x-axis label — a
one-pixel gap, whatever the type size, and the label's own offset moves both together — so the
box's top edge is drawn through the descenders of that label and reads as an underline under it.
Out on the white margin, with no plot lines behind the text, the frame was only decoration.

Two legends are left alone. One a script deliberately put outside already (`'eastoutside'`), which
is a layout decision rather than the default falling where it falls. And every legend in a figure
with more than one plot: a subplot grid has no margin under each plot to give away, only the gap
before the next one's title, and a key dropped into that gap lands on both — verified, it overlaps
the xlabel above it and the title below it. Inside is where they fit there, and 57.4 keeps them
readable.

**Result.** Single plots: the key is a caption under the axes, three entries or six. Subplots: the
keys stay where the script put them and nothing else on the figure overlaps any more.

---
### 57.4 — an opaque key is for an inside key only

**Symptom.** With the key moved below, the xlabel printed as "from the triaaer edae": the tail of
every g, y and p in it was erased.

**Cause.** Change 42's build-time patch (`patch-legend-opaque.sh`) emits `set key opaque`
unconditionally. Inside the axes that is right and still is — gnuplot's default leaves the key
transparent, so plot lines run through the legend text on exactly the busy plot where a legend
matters most. Outside the axes the fill has no plot lines to hide and lands on the label instead.

**Change.** The patch now emits `set key opaque` when the key is inside and `set key noopaque`
when it is outside. The `noopaque` is stated rather than left to the default, because `set key
opaque` from an earlier axes persists through the rest of a multiplot stream. The patch keeps its
anchor check and still exits 0 if an Octave upgrade moves the code.

**Result.** Descenders intact under an outside key; inside keys unchanged.

---
**Measured:** 156/156 smoke cases pass (154 before, 2 added: a plot with a legend, and a
script-sized wide figure, each asserting the canvas through the PNG header). Nine figures rendered
and inspected by eye, before and after: three curves with a long title and legend, six-entry
legend on log axes, six-category bar chart, 2 × 1 subplots, `peaks` surface with a colour bar, a
1200 × 400 figure with an `eastoutside` legend, an axes that sets its own 16 pt, `saveas` instead
of `print`, and a JPEG. The last four exist because each takes a different path through the shim. End-to-end through the gateway
as well, not only in the container: the tool returns a 1100 × 825 data URI.

The picture is 0.9 MP where it was 0.6, and Open WebUI hands it to the model as an input image, so
a returned figure costs about half again as many vision tokens as it did.

**Not measured:** no prompt has been run against the model, by instruction. Nothing here changes
tool text, so it does not move a graded answer by itself; what it changes is whether a plot the
model hands back can be read.

---
## Change 58 — closing the MATLAB gap where it can be closed

Asked from use: "would it be possible to modify Octave to be even closer to how MATLAB
functions? There are some small discrepancies that can cause issues for more complex
operations."

The investigation came first and killed the obvious answer. A throwaway Debian 13 image with
**Octave 9.4** was built and run against the same probes as the 7.3 in the image: the upgrade
adds `normalize` and `tensorprod` and nothing else. `contains`, `strlength`, `string`, `table`,
`datetime`, `histcounts`, `round(x,n)`, `sum(...,'all')` and `maxk` are missing in both, and
every semantic probe — error identifiers, `&&` on a non-scalar, `arguments` validators, `A\b`,
the RNG stream, the default `save` format, `num2str` on an int64 — behaved identically on the
two versions. A version bump costs a rebuild and buys nothing here.

What it does buy is elsewhere, and this change takes the six cheapest pieces of it.

---
### 58.1 — a script may put its functions at the end, as MATLAB does

**Symptom.** MATLAB (R2016b+) hoists a script's local functions: they go at the END and are
called from anywhere above. Octave defines a function when execution reaches the definition, so
the same script dies on `'double_it' undefined`. Correct, ordinary, modern MATLAB, refused. It
cost `m07` a whole tool call in the v3 round, and the error reads like a typo, so the model's
instinct is to rewrite code that was already right.

**Change.** `matlab_lint.split_local_functions` lifts the trailing `function` blocks out of the
script and `matlab.py` writes them as files beside it, which is where Octave looks. Two
properties matter as much as the lift itself:

* **Line numbers do not move.** The lines the definitions came from are replaced with blank
  lines, and each lifted file is padded with as many leading blank lines as it had above it, so
  an error inside one reports the line the model actually wrote. The stack frame says
  `main.m line 5 (in pick)` rather than naming a file nobody created.
* **It refuses anything it does not fully understand** — a nested function, a classdef, code
  after the definitions, two functions of one name, an unbalanced `end`, a signature it cannot
  read, more than forty functions. Those files run exactly as they did before. Mangling a file
  that was about to work is worse than not helping it.

The name that becomes a filename is checked twice: by the parser, which accepts an identifier
and nothing else, and again at the write itself, against the pattern the helper files use.

**Result.** `tests/test_local_functions.py` — 25 cases, pure Python, no container: the lift, the
line-number preservation, `end` used as an index, a one-line `if/end`, a switch inside the
function, the word `function` inside a string and inside a comment, and each refusal. Live: a
two-function script runs; a helper file of the same name wins over the lifted one; an error
inside a lifted function reports `main.m line 5 (in pick)`; recursion works; an unterminated
tail works; a nested function still fails with Octave's own parse error.

---
### 58.2 — Octave's parser as a second compatibility linter

**Symptom.** The regex linter catches the Octave-isms it has rules for. `x .*= c`, a bare
newline inside parentheses and several others have no rule, and a regex over a stripped copy of
the source can never be as sure about the dialect as the parser that implements it.

**Cause of the gap.** `Octave:language-extension` is exactly this signal and the driver turned
it OFF, with a comment saying it was unusable: Octave's own library is written in the same
dialect, so a real error disappeared under hundreds of warnings about files nobody can edit.

**Change.** The missing piece was that every one of those warnings names its file. The driver
turns the warning on — last, after the `pkg load`s and the `addpath`, so none of that machinery
is parsed with it enabled — and `run.sh` drops the library ones by path before anything is
measured against the output cap. A warning is three parts (the line, a `called from`, its
indented frames), and dropping only the first left orphaned frames that read like a stack trace
for an error that never happened, so the filter takes the block.

What survives is the user's code. `matlab.py` turns it into findings with line numbers, drops
any line the linter already reported, caps at twelve, and removes them from the raw warnings so
one defect is not printed twice.

**Result.** `x .*= 2` — which no rule covers — now comes back as "Octave's parser read `.*= 2;
used as operator` - an Octave extension, not MATLAB syntax", and the verdict is the existing
"Ran in the sandbox, but THIS IS NOT VALID MATLAB". Scratch runs are not told, the same policy
the linter's own `octave_only` findings follow.

---
### 58.3 — eight MATLAB functions this Octave does not have

**Change.** `octave/shim/` gains `contains`, `strlength`, `matches`, `maxk`, `mink`,
`histcounts`, `discretize` and `rmmissing`, joining `print` on the path ahead of everything.
They are written in plain MATLAB, and the policy for all of them is in `contains.m`: implement
the documented behaviour for the forms that are actually written, and raise a clear error for
the rest. `histcounts(x)` with automatic binning says so instead of guessing a binning rule —
MATLAB's is documented and non-trivial, and a shim that picked a different one would hand back a
different histogram with nothing to show for it. `rmmissing` refuses cells rather than inventing
a meaning for missing text.

`check-shims.sh` runs at image build time and does two things that cannot be done later:

* **Drops** any shim whose name Octave has since acquired. This directory shadows everything; a
  stale four-signature `contains` replacing a complete one is the exact failure the shims exist
  to prevent. Octave gaining a function is good news, and the right response is to delete ours.
* **Tests** what survives, against the behaviour MATLAB documents — including the edge
  conventions that are why these are not one-line aliases: `histcounts`' last bin is closed
  where `histc`'s is not, `maxk` is column-wise on a matrix and returns a row for a row.
  A failure fails the build; verified by breaking a shim on purpose and watching it.

`_MATLAB_ONLY` and `_OCTAVE_EQUIVALENT` lose the five names that now work, so the model is no
longer told to rewrite `contains` into `strfind`.

**Not shimmed, deliberately:** `round(x, n)` and `sum(..., 'all')`. Those are extra arguments to
functions Octave already has, so the shim would have to shadow a hot builtin — measured at
**4.6x** on the shadowed call (20,000 `sum` calls: 0.080 s to 0.369 s), paid by every call in
the session including the ones Octave's own library makes. 58.5 handles them for free instead.

---
### 58.4 — two preferences, and the ones not taken

**Change.** The driver sets `print_empty_dimensions(false)`, so an empty array prints `x = []`
as MATLAB does rather than `x = [](0x0)` — output the model reads back to the user — and
`save_default_options("-mat7-binary")`, because `save("data.mat", "x")` was writing an Octave
TEXT file with a `# Created by Octave` header. A user opening that in MATLAB gets nothing. The
header now reads `MATLAB 5.0 MAT-file`.

Octave's `--traditional` flag sets both of these and is documented as "closer MATLAB
compatibility", but it is not taken wholesale: it also sets `struct_levels_to_print(0)`, which
prints `val: 1x3 matrix` instead of a struct's contents, and `beep_on_error(true)`.

`fixed_point_format(true)` is the interesting refusal. It makes numeric display genuinely closer
to MATLAB — `[1000.5 2000.25; 3000 4000]` prints under a `1.0e+03 *` header in MATLAB, and
element by element here — and it was left off anyway. This tool exists to give the model numbers
it can quote, and a display where every value must be multiplied by a header two lines up is a
transcription error waiting to happen. Fidelity lost, accuracy kept.

---
### 58.5 — MATLAB call forms this Octave refuses

**Symptom.** `round(pi, 3)` gives "Invalid call to round"; `sum(A, 'all')` gives "sum:
unrecognized type argument 'all'". Both are correct MATLAB (R2014b and R2018b). Neither message
says so, and "invalid" reads as "you wrote a bug".

**Change.** Two patterns in `_classify_error`. They name the MATLAB version the form comes from,
give the local rewrite (`round(x * 10^n) / 10^n`, `sum(A(:))`), and then split on purpose: a
scratch calculation should be rewritten, a deliverable should keep the MATLAB form and say that
this sandbox could not run that line. The `'all'` pattern matches the source as written rather
than the literal-stripped copy — its subject IS a string, and the stripped copy has blanked it,
the same trap the linter documents for its own raw rule.

---
### 58.6 — a function named after a MATLAB built-in

**Symptom.** From `m04`, v4: the answer defined `function normalize(X)`. `normalize` is a MATLAB
built-in (R2018a+), so the delivered file shadows it in the user's MATLAB — and nothing here
noticed, because `normalize` does not exist in this Octave at all. A name that is free in the
sandbox and taken in MATLAB is invisible to every other check that runs.

**Change.** The names a script defines — lifted local functions and helper files — are checked
against the MATLAB-only table plus a curated list of core names, each one verified present in
this image and each a plausible thing to call your own function. A hit gets one short section
saying yours wins silently, including for calls that meant the built-in. The list is
deliberately partial: a missing name costs a note, never a wrong one. Scratch code is nobody's
deliverable and is not told.

---
**Measured:** 166/166 smoke cases pass (156 before; 10 added, 3 rewritten because they asserted
limitations this change removes — the `contains` advice, the "defined below the line that calls
it" hint, and the equivalent-advice path, which now uses `split`, still missing). 25/25 unit
cases in `tests/test_local_functions.py`. 27 shim assertions at image build time, and the build
was verified to fail when a shim is wrong. Eight adversarial live cases: a helper that collides
with a lifted name, a function hidden inside a block comment, `function`/`end` inside a string, a
nested function, three functions at once, a user function shadowing a shim, an unterminated
tail, and recursion inside a lifted function.

**Not measured:** no prompt has been run against the model, and the MATLAB half of every claim
here is from documentation — there is no MATLAB in this house to check it against. The `symbolic`
package (SymPy-backed `syms`, `solve`, `vpa`, `laplace`) was tested and works offline under the
sandbox's full hardening in 0.03 s to load and 1.15 s for the first call, and is **not** included:
it spawns a Python child, and `timeout -s KILL` kills only the Octave process, so a job that dies
mid-symbolic leaves an orphan in a container that serves every other job. Process-group cleanup
is its own change, with its own review.

---
## What was and was not measured

**Measured:** 66/66 existing smoke cases still pass, and 107/107 after Change 53. 39/39 targeted cases for the new
behaviours pass through the gateway. Two graded prompts re-run end to end through the chat UI,
both fixed. Engine availability, model row contents and filter rows verified directly.

**Not measured:** the full 40-prompt suite has not been re-run, by choice — spot checks were
the agreed level for now. The per-bucket targets from the plan (recall 30.5 → ≤15, attribution
28.5 → ≤14, tool gaps 20.5 → ≤8, loop 10 → 0, arithmetic 3.5 → 0) are therefore unverified in
aggregate. The presence-penalty change in particular has not been tested against the loop that
motivated it.

---
## Change 59 — a sub-agent with a real machine in front of it

**Symptom.** The assistant could reason about a script but never try one. Asked whether a
PowerShell snippet works, it could only read documentation about PowerShell; asked why a
build fails, it could only guess from the error text it was given. Every other tool in this
suite returns a fact about the world. There was no tool that returned a fact about *running
something*.

**Cause.** Nothing structural — the capability had simply never been built. The Octave sandbox
(Change 34) is the nearest thing and is deliberately narrow: one interpreter, no network, no
filesystem that survives.

**Change.** `vm_agent`: one tool that hands a task to a *second* LLM turn-loop — the same
`qwen3.8:latest` on the same Ollama, with its own system prompt and its own thirteen tools —
which drives a throwaway Ubuntu or Windows VM. The main model's surface stays at one tool and
never carries VM-control instructions, screenshots or transcripts. Everything the VM touches
is destroyed afterwards except a per-project folder on the ZFS array, which is a Gitea working
tree.

Five new components: a host daemon (`vmd`) holding everything privileged behind a unix socket;
an agent container holding no credential and able to reach exactly one address; an nginx
gateway holding the two secrets; a netns guard; and a GTK4 desktop tool for the operator to
unfreeze a base image.

The rest of this entry is the things that were **measured**, several of which contradicted the
design they were meant to confirm.

### 59.1 — the primary network control was doing nothing

**Symptom.** The sandbox unit carried `IPAddressAllow=0.0.0.0/0` alongside a deny list for the
private ranges, on the assumption that longest-prefix match would let the specific denies win.
Everything appeared to work: a probe from the sandbox uid could not reach `192.168.1.10:7200`,
`172.17.0.1:11434` or `127.0.0.1:3000`.

**Cause, and it is mine.** `systemd.resource-control(5)` applies the two lists in a fixed
order: granted if the address matches `IPAddressAllow=`; otherwise denied if it matches
`IPAddressDeny=`; otherwise granted. There is no arbitration between them. `0.0.0.0/0` matched
every address at step one, so the entire deny list was dead code. What made it *look* correct
was the second layer — an nftables table matching on the qemu uid — quietly covering for it.

**Change.** Deny-only, with a comment saying why there must never be an `IPAddressAllow=` line
in that file. Rule three ("otherwise granted") is what grants the internet; it needs no help.

**Result.** With the nft table deleted, the corrected unit blocks all ten must-fail targets and
leaves `1.1.1.1` and `8.8.8.8` reachable. With the nft table restored and `IPAddressDeny`
stripped from a scratch unit, the same suite passes again. `tests/vm_host_asserts.sh` now does
both of those every run.

**Generalisable lesson.** Two independent controls are worth having and worth *testing
independently*, because in normal operation either one will hide the other's absence. The test
that found this is the one that turns each layer off in turn — not the one that checks both are
loaded. This is the same reasoning `netguard/firewall.sh` already applies to itself
("a firewall that silently failed to apply is worse than none"); it just had not been applied
to the layer above it.

### 59.2 — systemd's syscall filter and QEMU's `-sandbox` are mutually exclusive

**Symptom.** `qemu-system-x86_64: -sandbox on,...: There is no option group 'sandbox'`, exit 1,
from inside the unit. The identical command line run by hand started fine, and `ldd` shows
`libseccomp.so.2` linked into the binary.

**Cause.** `SystemCallFilter=@system-service` on the unit. Isolated by running the same argv
under `systemd-run` with and without the property: without it, accepted; with it, the error.

**The mechanism, found later and worth the correction.** This entry first said the two were
"mutually exclusive", which is true in effect but does not say why. The maintenance unit was
added afterwards without `SystemCallErrorNumber=EPERM`, and qemu there died instantly with
`status=31/SYS`; the audit record names the syscall — **317, which is `seccomp(2)` itself**.
So qemu calls `seccomp()` at startup to probe for its own sandbox, `@system-service` does not
permit it, and what happens next depends entirely on that one extra line: with
`SystemCallErrorNumber=EPERM` the probe returns an error and qemu quietly concludes seccomp is
unavailable, which is why `-sandbox` reported "there is no option group"; without it, the
kernel raises SIGSYS and kills the process before it has started.

That makes `SystemCallErrorNumber=` load-bearing rather than cosmetic, and both units now carry
it with a comment saying so.

**Change.** `-sandbox` dropped, systemd's filter kept, and the reason written into
`owuivm-launch` where somebody will otherwise "restore" the flag. systemd's is the broader
allowlist (`@system-service` excludes `@mount`, `@swap`, `@module`, `@reboot` and `@obsolete`),
it covers the whole process tree, and `systemd-analyze security` can see it. What is given up
is `spawn=deny`; the vector that mattered there, `migrate exec:` over QMP, is already closed by
vmd exposing typed verbs and never a raw monitor command.

**Not measured:** whether QEMU's own filter would have been narrower in practice. It cannot be
run alongside the other, so there is nothing to compare.

### 59.3 — `ping` can never work here, and it reads as "no internet"

**Symptom.** Anticipated rather than suffered, but worth the entry. `net.ipv4.ping_group_range`
on this host is `"1 0"` — an empty range, so no group may open an unprivileged ICMP socket.
libslirp cannot forward ICMP without one, so `ping` fails in every guest, permanently, on a
machine whose HTTPS works perfectly.

**Change.** A `check_internet` tool that uses curl, and the fact stated **in the tool's own
output** rather than in the agent's system prompt.

**Why there and not the prompt.** The finding restated four times in this file: the model
treats tool output as fact and system-prompt text as advice. An agent that has just watched
`ping` fail three times needs the correction where it reads instructions as findings.

**Result.** `tests/vm_negative_guest.sh` asserts ping fails *while the HTTPS control passes*,
so the day `ping_group_range` changes, the suite says so instead of quietly becoming wrong.

### 59.4 — the guest booted with no DNS, and the reason was cloud-init

**Symptom.** First boot of the base image: every TCP target behaved exactly as designed —
LAN blocked, `1.1.1.1:443` and `8.8.8.8:53` reachable — and `getent hosts archive.ubuntu.com`
returned nothing. `/etc/netplan/` was empty and `resolvectl` showed `10.0.2.3`.

**Cause.** Subiquity does not write `/etc/netplan` itself; it hands the network configuration
to cloud-init, which renders it on first boot. The image deliberately disables cloud-init — it
must not re-run on every boot of a disk restored from a frozen base thousands of times — so
the netplan was never written and the guest fell back to DHCP. SLIRP's DHCP hands out
`10.0.2.3` as the nameserver, which is a forwarder *inside the qemu process* that relays to
this host's resolver, and the sandbox's egress filter denies `127.0.0.0/8` — correctly, because
the host's resolver is a LAN service.

**Change.** The netplan is written directly in `late-commands`, static, with a public resolver.

**Generalisable lesson.** The isolation worked exactly as specified and the result was a
machine that looked broken in a completely different place. A control that denies something
the guest depends on by default will surface as a mystery three layers away; the guest's
configuration has to be changed to match the policy, not just tested against it.

### 59.5 — two permission findings, both silent

**`qemu-img` creates 0644, so a umask of 0007 leaves 0640.** vmd creates the session overlay and
qemu writes it, so the file has to be group-writable. `UMask=0007` cannot produce that: qemu-img
asks the kernel for 0644, and a umask only ever *clears* bits. The result was qemu failing with
`Could not open ...: Permission denied` on a file it could plainly read. Fixed with an explicit
`chmod 0o660`.

**`/Pool` is `0770 root:docker`.** Neither service user could traverse it, so nothing under it
was reachable. The obvious fix — adding them to `docker` — would have handed both of them the
docker socket, which is root. A traverse-only ACL (`setfacl -m u:owuivmd:--x,u:owuivm:--x`) is
the whole of what they actually need: they reach their own datasets and still cannot list the
parent.

### 59.6 — a correction to this file: Open WebUI *does* time out a tool call

**Symptom.** `gateway/nginx.conf` has said since Change 34 that "Open WebUI sets no tool-server
timeout of its own (its `AIOHTTP_CLIENT_TIMEOUT` is unset here), so this is the only clock".

**Cause.** That reads "unset" as "unlimited". `env.py:593-597` sets `AIOHTTP_CLIENT_TIMEOUT` to
**300** when the variable is absent, and `env.py:671-679` falls
`AIOHTTP_CLIENT_TIMEOUT_TOOL_SERVER` back to it. `utils/tools.py:1715` uses that for every tool
call. So there have always been two clocks on a tool call, and Open WebUI's was the shorter one.

**Change.** The comment is corrected in place. A VM session needs all three clocks raised
together — `OWUITOOLS_VM_WAIT_SECONDS` in the app, `proxy_read_timeout` on a dedicated
`location = /tools/vm_agent`, and `AIOHTTP_CLIENT_TIMEOUT_TOOL_SERVER` on the open-webui
container — with the app's the smallest, so it is the app that answers.

**Process lesson.** The original claim was written from reading one env var's default and not
the fallback chain under it. Nothing had failed because of it: 240 s is under 300 s, so the
gateway always answered first and the second clock was never reached.

### 59.7 — what the agent is not allowed to talk its way past

The previous sub-agent stack on this host was retired for accuracy, and its post-mortem names
two mechanisms: a coding agent that "hallucinated third-party library APIs and wrote
self-confirming tests around them", and agents that, "allowed to run search-only", guessed
from snippets. Both are the same failure — the agent deciding when it is finished, against no
standard but its own prose. Three things answer it, and none of them is a sentence in a prompt:

- **`plan_tasks` must be the first call.** Every other verb is refused until there is a list.
- **`finish` is refused** while any task is open, and refused if nothing was ever run. Marking
  a task blocked or dropped *requires* a reason, so "I could not finish" without one is not an
  available outcome.
- **The verification command is re-run by vmd** after the agent's last step, in a fresh shell,
  and the exit code the user sees is the one from that second run. If it disagrees with the
  agent, the agent's summary is still shown — labelled a claim, with the real output beside it.

vmd holds the task list, not the agent, so it cannot be quietly reworded to match what
happened; and vmd assembles the tool result, so the facts come from the side that is not being
checked. That last point is also a prompt-injection boundary: guest stdout reaches the agent,
whose summary reaches the main model, so the summary is fenced, capped and labelled, and
`data:` URIs are stripped — `budget.cap()` passes a bare data URI through untouched and Open
WebUI hands such a result to the model as a **vision input**, which would let a guest choose a
picture for the main model to look at.

**Mitigated, not eliminated:** a self-confirming test still passes when re-run, because it is
still the agent's test. All that is done cheaply is to notice the shape — a verification
command naming a file created during this same session — and say so on the report.

---

**Measured:** both isolation layers proven independently with the other removed; ten must-fail
targets blocked and three must-pass reachable from inside a booted guest; a planted
`core.hooksPath` executes under plain git and does not under `gitea.commit()`; polkit refuses
the maintenance unit to a non-local session and vmd's helper has no verb for it; `screendump`
accepts `format: png` and `input-send-event` exists, both read from this host's live QMP
schema; a session boots and answers on its guest agent in 25.7 s.

### 59.8 — the first three live sessions, and what each of them broke

Driven by the model itself, not by `vmctl`. Each failure is recorded because each one was in a
different layer and only the third run reached the work.

**Run 1 died at step one:** `the model refused the request (HTTP 400): time: missing unit in
duration "-1"`. `keep_alive` was being sent as the string `"-1"`; Ollama takes a *number* of
seconds (negative meaning indefinitely) or a duration *string* like `30m`, and parses anything
quoted as the latter. The report still came back complete, saying exactly that — which is the
first evidence that the "every exit produces a result" rule holds, because the alternative was
a started VM and a blank answer.

**Run 2 died at 4 minutes** with nginx's HTML 504. The dedicated `location = /tools/vm_agent`
had been written and validated but `owuitools-gw` had never been restarted, so the 240 s
default was still in force. Incidentally this proved the cancellation path for real: the api
saw the client disappear, wrote the CANCEL marker, and the machine was stopped six seconds
later. No orphaned VM — which is the bug the previous stack shipped.

**Run 3 completed.** 9 agent steps, 153 s, 4 commands in the machine, one of which exited
non-zero and was recovered from. The verification command it chose is worth quoting, because
it is the behaviour the whole `finish` gate exists to produce:

```
out=$(bash /mnt/project/biggest.sh) && [ "$(printf '%s\n' "$out" | wc -l)" -eq 10 ] \
  && [ "$(printf '%s\n' "$out" | grep -Ec '^[[:space:]]*[0-9]+[KMGT]?B[[:space:]]+/usr/')" -eq 10 ] \
  && printf '%s\n' "$out" | awk '{print $1}' | sort -Cnr && echo VERIFIED
```

Line count, format, every path under `/usr`, *and* that the sizes are actually in descending
order. The supervisor re-ran it independently and it passed; the self-confirming-test note
fired correctly, since the script under test was written in the same session. Committed and
pushed to `https://git.example.com/vmlab/biggest`.

### 59.9 — the admin key that does not need to exist

**Symptom.** Live progress reached the chat by POSTing to
`/api/v1/chats/{id}/messages/{id}/event`, and that endpoint refuses a key whose user neither
owns the chat nor is an admin (`routers/chats.py:1531`). Anyone on this instance may invoke
`vm_agent` and the chat belongs to whoever is talking, so it had to be an **admin** key — the
ability to read every conversation and create users — living in a container in order to be
usable. It was kept out of the agent itself, behind nginx in `owuitools-vmgw`, which bounded
the damage without changing the fact that it existed.

**Cause.** Pushing was the wrong direction. Anything outside Open WebUI that wants to write to
a user's message needs to authenticate as somebody, and the only somebody who can write to
*any* user's message is an admin.

**Change.** The agent writes progress to the job volume it already shares with the api
container (`/vmjobs/progress/<chat-id>.json`), the api serves it on a router that is
`include_in_schema=False` and carries no `operationId`, and a new filter —
`owui-functions/vm_agent_progress.py` — polls it from **inside** Open WebUI and emits the
events itself. There, `__event_emitter__` is already bound to the requesting user's own
session, so nothing authenticates as anyone. The filter's only credential is the owuitools
bearer token, which grants exactly the tools the model can already call.

The event location and the key are gone from `vmgw`; its Ollama locations stay, because
Ollama's unauthenticated `DELETE /api/delete` still needs something in front of it.

**Result.** No Open WebUI credential exists anywhere in the stack.
`tests/vm_agent_asserts.sh` now asserts that as a property rather than a configuration:
vmgw holds no key **and** proxies no `chats/.../messages` path. The progress endpoint answers
200 with the token and 401 without, and does not appear in the OpenAPI document at all — 22
tools, none of them this.

**Two things this cost, and one bug it caused.** Progress is polled every two seconds inside
Open WebUI's event loop rather than pushed. The filter must be installed by hand like the other
four, so a fresh deployment has a working agent and no live view until somebody pastes it in.
And removing `openwebui_default` from vmgw — which looked like tidying, since it no longer
needed to reach Open WebUI — left it on nothing but an `internal: true` network, so it had no
default route to Ollama at all. Its healthcheck passed and every proxied request timed out.
The membership is back, with a comment saying it is for the route and not the destination.

**Generalisable lesson.** "Where does this credential have to live" is often really "which
direction is this call going". Reversing it removed the credential entirely rather than
protecting it better.

### 59.10 — the isolation rule that locked the operator out of their own console

**Symptom.** The maintenance VM's console never connected. The port listened, `query-vnc`
reported `enabled: true, auth: none, host 127.0.0.1, service 5999`, and every client - and a
raw socket - sat there until it timed out. A VNC server greets immediately; this one sent
nothing at all.

**Cause, and it is mine.** `IPAddressDeny=127.0.0.0/8` on the unit. systemd's filter is
described plainly in the man page and I had read only half of it: *"Both ingress and egress
traffic is filtered by these settings. In case of ingress traffic the source IP address is
checked."* The source address of a connection to a loopback port is 127.0.0.1. So the rule
that stops the guest reaching the host through SLIRP's `10.0.2.2` alias is the same rule that
drops the operator's own inbound connection, and there is no way to express "outbound only"
in that mechanism.

Confirmed both directions: with `IPAddressDeny=` cleared and the unit restarted, the server
greets with `RFB 003.008` immediately.

**A second finding on the way:** `systemctl set-property --runtime IPAddressDeny=` on a
*running* unit changes the property and does **not** reattach the BPF program. The first
attempt to test this hypothesis therefore appeared to disprove it. The filter is built at unit
start; testing it needs a restart.

**Change.** The console moved from a TCP port to a **unix socket**, which is not IP and which
the filter therefore does not see. The desktop tool bridges it to a loopback port with `socat`,
from a process running as the operator and outside qemu's cgroup. The unit's
`SocketBindAllow=ipv4:tcp:5999` exception is gone with it: a maintenance boot now opens no
listening TCP port at all, exactly like a session.

**Result.** Both properties hold at once, which was the thing in doubt: the console greets
through the bridge, and `IPAddressDeny` is intact with all nine ranges. `vm_host_asserts.sh`
now asserts the console is a unix socket and that the unit permits no TCP bind, so this cannot
quietly revert to a design that does not work.

**Generalisable lesson.** A filter with no notion of direction will eventually be asked to
allow one direction and deny the other, and the answer is to move the thing that needs allowing
off the filtered transport entirely rather than to weaken the rule. The alternative on the
table - adding `IPAddressAllow=127.0.0.1/32` - would have worked for the console and reopened
the guest's path to every service on the host's loopback, which is the specific hole the rule
exists to close.

### 59.11 — what Windows 11 needs, and the denial that wears a permission error's clothes

**Symptom.** Anticipated rather than suffered, but three of them would each have stopped a
Windows install dead, and two produce messages that point at the wrong thing entirely:

- **No TPM and no Secure Boot** -> "This PC can't run Windows 11". Reads as a hardware
  limitation of the VM rather than two missing device models.
- **No virtio driver disc** -> setup reports there are no drives to install to. Reads as a
  broken disk rather than a driver Windows has never had.

**Change.** The launcher now branches on the image's name - derived from the disk in
maintenance mode and from the overlay's *backing file* in a session, so a session's
randomly-named overlay still resolves to the right machine. Windows gets
`OVMF_CODE_4M.secboot.fd`, a variable store seeded from `OVMF_VARS_4M.ms.fd` so Microsoft's
keys are actually enrolled, `-global driver=cfi.pflash01,property=secure,value=on` so the guest
cannot rewrite that store, an `swtpm` TPM on a unix socket, and `virtio-win.iso` as a second CD
during installs only. Ubuntu gets none of it.

**And one that was suffered.** `swtpm` refused to create its socket:
`Could not open UnixIO socket: Permission denied`, with nothing in dmesg. Not a file mode -
Ubuntu's AppArmor profile for swtpm permits libvirt's paths (`/run/libvirt/qemu/swtpm/*.sock`,
`/var/lib/libvirt/swtpm/**`) and essentially nothing else, and denies quietly. Fixed through
`<local/usr.bin.swtpm>`, the profile's own extension point, so it survives a package update.

**A second bug found by writing this down.** The maintenance boot picked its media with
`ls -t "$ISOS"/*.iso | head -1` for Windows. With an Ubuntu ISO sitting in the same directory
and a *newer* timestamp, a Windows maintenance boot would have quietly booted the Ubuntu
installer. The glob is now `[Ww]in*.iso` and `fetch.sh` refuses Windows media that does not
match that shape, so the two ends agree rather than one silently disagreeing.

**And a third.** A template unit cannot `Conflicts=` its own other instances, so
`owuivm-maint@ubuntu` and `owuivm-maint@windows` could both come up, fight over the same
console socket and split the host's memory. The prepare step now refuses the second.

**Result.** `tests/test_vm_launch_argv.sh` asserts the whole command line without booting
anything: it replaces the final `exec qemu-system-x86_64` with a `printf`, so every branch
runs for real. Windows has secure-boot firmware, a TPM and a protected variable store; Ubuntu
has none of them; and **both** have a netdev of exactly `user,id=net0,ipv6=off` with no
`guestfwd`, `smb=`, `tftp=` or `bootfile`, no `hostfwd`, and no `-virtfs`, `vfio`, `usb-host`
or `hostdev`.

That last set is the reason the test exists. Those `-netdev` options are handled *inside* the
qemu process, so the traffic never becomes a host socket and neither the cgroup filter nor the
nft table can see it. No filter can catch what no filter can observe; the only defence is that
the string is fixed, and the only way to know it is still fixed is to look at it.

**Not measured:** no Windows machine has been installed. Everything above is the documented
requirement plus an asserted command line, which is not the same as a booted installer.

### 59.12 — the machine was crawling, and it was not nested virtualisation

**Symptom.** The Ubuntu installer was "incredibly slow" to drive. The obvious suspect on this
host is that it is itself a KVM guest, so the sandbox is an L2 machine and every VM exit costs
more — an explanation that is true, unfixable, and in this case wrong.

**Measured instead of assumed.** `query-kvm` reported `enabled: true`, so it was accelerated.
The qemu process sat at 260% CPU on a 28-core host with a load average under 5, so it was not
CPU-starved, and `CPUQuotaPerSecUSec=7s` was nowhere near binding. What was pinned was memory:
`MemoryCurrent` was **exactly** `MemoryMax`, and the cgroup's `memory.events` recorded
**`max 6782`** — it had hit its hard limit 6,782 times in one boot, each hit forcing direct
reclaim. `memory.pressure` showed 38 ms of full stall accumulated while idling at a desktop.

**Cause, and it is mine.** `MemoryMax=10G` for a guest given 8 GiB. The guest's RAM is a
`memory-backend-memfd` with `share=on` — required by virtiofs, and therefore set
unconditionally so a machine with a project share behaves identically to one without — so the
kernel accounts all 8 GiB of it as **shmem**. This host has no swap, so shmem cannot be
reclaimed at all. That left roughly 1.5 GiB for every page of ISO and disk I/O the VM does, and
a 6 GB installer ISO does not fit in 1.5 GiB: each read evicted the last, so the same media was
read from disk again and again.

`MemoryMax` was set to bound a runaway qemu. It was never meant to bound the guest — the
guest's own RAM size does that — and at 10G it was doing the second job badly while not
obviously doing the first.

**First change, and it was the wrong one.** `MemoryMax=14G` on both units, so the page cache
had room. It worked - reclaim went to zero and the stall time fell by a factor of ninety - and
within minutes the *host* was short of memory and started killing background work. One VM had
been handed twelve gigabytes of a 58 GB machine that is also running Ollama. Raising a limit to
make room for something that should not have been inside it in the first place.

**The actual change.** `cache=none,aio=native` on every drive - the disk and both CD-ROMs - so
this VM's I/O is O_DIRECT and never enters the page cache at all. What remains in the cgroup is
the guest's RAM and qemu itself, which is what the limit was always meant to bound, and
`MemoryMax` came back down to 11G. Verified O_DIRECT works on both filesystems in play: ext4
for the base images and ZFS 2.4.1 for the session overlays.

Measured after: cgroup `file` 3.43 GB against `shmem` 3.42 GB - so the page cache charged to
this VM is now a rounding error rather than 2 GB - `max 0`, stall `total=0`, and the host went
from 39 GiB used / 18 GiB available to **35 GiB used / 23 GiB available**, with host pressure
back to zero. Better than the 14G version on both counts at once.

Also `-device VGA` -> `-device virtio-vga`, still 2D: a desktop
repainting through the emulated VGA MMIO path is slow on top of llvmpipe doing the rendering
anyway. Explicitly **not** `virtio-gpu-gl`/virgl — QMP `screendump` reads the console surface
and a device rendering through a GL context does not provide one, which would take the agent's
entire GUI mode with it.

**Result.** Same machine, same media: `memory.events max` **6782 -> 0**, full stall
**37,996 µs -> 0**, and the host better off than before rather than worse. `screendump` still
returns a 1280x800 PNG on virtio-vga, verified.

**Process lesson, and it is the one worth keeping.** The first fix was verified against the
symptom and not against the system: every number I looked at was inside the VM's cgroup, and
all of them improved. The host was never measured until it started killing things. A limit
raised is a limit somebody else now lacks, and "it fixed the thing I was looking at" is not the
same as "it fixed the thing".

**Generalisable lesson.** A cgroup memory limit on a VM is not a limit on the VM. With
`share=on` the guest's RAM lands in the same accounting bucket as the host's page cache for
that VM's I/O, and on a swapless host one of those two is unreclaimable — so the limit is
really "how much page cache may this VM have", and setting it just above the guest's RAM size
means approximately none.

### 59.13 — stopping the unit was pulling the plug

**Symptom.** Noticed while adding an explicit Shut down button, not from a failure.
`systemctl stop` SIGTERMs qemu, which from inside the machine is indistinguishable from
yanking the power lead. For a session that is harmless — the overlay is discarded either way —
but a maintenance boot is the operator editing the base image every future session runs from,
and the GUI's only stop control did exactly that.

**Change.** `ExecStop=/usr/local/libexec/owuivm-powerdown` on both units. It sends
`system_powerdown` over QMP and waits for the guest to stop; systemd then waits for qemu to
exit on its own, up to `TimeoutStopSec`, and still SIGTERMs afterwards if the guest ignored the
button. A courtesy with a deadline, not a new way to hang. It cannot fail: a stop path that
errors leaves the unit in `failed` and blocks the next start, so every path exits 0 — verified
against a bad session id and a missing socket.

**Also:** the GUI is now *Tool Server VM Maintenance*, with Start, Shut down and Console as
three buttons that each mean one thing, rather than one button whose meaning depended on state.

### 59.14 — the freeze was not happening, and nothing said so

**Symptom.** A single line in the journal after a clean shutdown:
`owuivm-maint@ubuntu.service: Control process exited, code=exited, status=1/FAILURE`. Noticed
only because a restart happened to be in front of me.

**Cause.** `owuivm-maint-prepare freeze` computed how much of the disk was allocated with
`du -B1 --apparent-size=0`. `--apparent-size` takes no argument, so `du` exits non-zero; under
`set -euo pipefail` that killed the script at that line - which was **before the
`chmod 0444`**. Every clean shutdown therefore left the base image WRITABLE. The disk was
correct, the checksum was not recorded, the marker was not cleared, and the only evidence was
one line saying a control process had failed.

Two compounding mistakes, both mine. The argument was wrong, and the freeze - the one
safety-critical statement in the file - was placed downstream of code that could abort.

**Change.** The `chmod` now happens first, immediately after the firmware variables are copied
back, and everything after it runs with `set +e` because it is all advisory. `du -B1` without
the flag, which reports allocated size, which was the question. The final log line reports the
mode it actually observes (`stat -c %a`) rather than asserting success.

**Result.** `tests/test_vm_freeze.sh` runs the real script against a scratch directory - the
roots are overridable for exactly this - and checks the resulting mode. Three cases: a disk
with an OS on it freezes and loses its marker; a disk nothing was installed on freezes and
KEEPS its marker, so an interrupted install can be resumed; and a disk freezes even when a
later step is made to fail, which is the case that would have caught the original bug.

**Generalisable lesson.** `set -e` turns every line above the important one into something that
can silently skip it. Order the safety-critical statement first and let the reporting fail if
it wants to - and test the *effect* on the filesystem, not the presence of the line.

**And a test that cried wolf.** `vm_host_asserts.sh` flagged the image as unfrozen while the
operator had it open for maintenance, which is exactly what maintenance is. It now distinguishes
"unfrozen because something is broken" from "unfrozen because you are using it" - a suite that
reports correct states as failures teaches people to ignore it, which is worse than not having
one.

### 59.15 — three faults behind one symptom: "the machine is in an EFI shell"

**Symptom.** The Ubuntu machine came up in the UEFI firmware shell rather than the installer,
with 3.45 GB already written to its disk. And pressing Shut down in the maintenance tool locked
the window - it stopped repainting entirely and looked like a crash.

**Fault 1: the GUI blocked its own main loop.** `subprocess.run("systemctl stop")` was called
directly from the button handler. GTK repaints from the main loop, and stopping a machine now
takes as long as the guest needs to power off - up to `TimeoutStopSec`. So the window froze for
over a minute, which is indistinguishable from a hang. Introduced by me in the same change that
added the graceful shutdown: the shutdown got slower and more correct, and nothing was done
about the thing waiting on it. Start and stop now run on a worker thread and report back through
`GLib.idle_add`; a `_busy` flag keeps the periodic refresh from re-enabling buttons underneath an
operation that has not finished.

**Fault 2: a partial install was declared finished, on size.** The freeze step removed the
install marker once the disk had grown past half a gigabyte, reasoning that a disk with contents
has an operating system on it. An install interrupted partway had written 3.45 GB. The marker
went, the next boot had no installation media, and the disk was neither blank enough to start
over nor complete enough to boot - hence the firmware shell, and no way back except deleting it.

Size cannot distinguish "installed" from "interrupted halfway", and neither can anything else
available from outside the guest. So it no longer tries. **The media stays attached**, and the
boot order does the work instead: the disk is `bootindex=0` and the media `bootindex=1`, the
opposite of the obvious arrangement. UEFI walks its entries in order, so a blank or
half-installed disk falls through to the installer by itself while a finished install boots
itself and ignores the disc still in the drive.

**Fault 3: firmware variables from empty boots were being frozen into the base image.** OVMF
rewrites its boot order on every boot, and the freeze copied that back unconditionally -
including boots that installed nothing. Several restarts in, with the display device changed
from `VGA` to `virtio-vga` and the drives given `cache=none`, the stored entries pointed at
device paths that no longer existed. The copy-back is now conditional on the disk actually
having something on it.

**Result.** Wiped and restarted: the blank disk falls through to the media and boots the Ubuntu
installer, verified by screenshot. `tests/test_vm_freeze.sh` gained a case asserting that a boot
which installed nothing does not rewrite the frozen firmware variables, and its marker case was
inverted - the marker is now *supposed* to survive, and the test says why.

**Generalisable lesson, and it is the same one twice.** Both of the silent faults were a
heuristic standing in for a fact nobody had. "The disk is big, so it is installed" and "the
firmware wrote variables, so they are worth keeping" are both guesses that are right most of the
time and destroy the artefact when they are wrong. Where the fact is not available, arrange
things so the guess is not needed - which here was one boot-order change that made an
interrupted install self-healing.

### 59.16 — planning while it boots, and a wait tool

**Symptom.** Not a fault - a waste. `session_open` blocked until the guest agent answered, so
the sequence was: start a machine, wait twenty-odd seconds doing nothing, then ask the model for
a plan and wait again while it prefills and thinks. Two waits of about the same length, one
after the other, at the start of every session.

**Change.** They now happen at once. `session_open` takes `wait=False` and returns as soon as
qemu is started; the agent's first turn is a planning turn with **only the planning tools
offered**, and the opening message says plainly that the machine is still coming up and it will
be told when to start. When it has a list, the harness calls `session_ready`, which blocks until
the guest answers - and, for a GUI session, until there is actually a graphical session, because
the agent answers at multi-user well before anything is on screen. Then a message goes in saying
the machine is up, with a screenshot already attached, so a desktop session does not spend its
first turn asking for one.

That has a second effect worth as much as the first: by the time there is a machine to use, the
model is loaded and its prompt is prefilled.

**The agent cannot reach the machine early, by two independent means.** The loop does not offer
the guest tools while booting - verified for both interfaces - and `vmd` refuses all thirteen of
them regardless of what was offered, because the loop and the gate could disagree and the gate
should win. The gate also distinguishes "there is no machine" from "the machine is still
booting": the wrong one of those sends an agent off waiting for something that is never coming.

`session_ready` is deliberately **not** a tool. An agent that could ask "is it up yet?" would
ask, repeatedly, and each of those is a model turn spent on nothing.

**And a wait tool.** Previously GUI-only and capped at 60 seconds, which is not a package
install. Now a general tool capped at **20 minutes**, sleeping in two-second steps so a
cancelled session does not sit in it for another quarter of an hour. The cap is not in the
tool's description - that is a sentence paid for on every turn of every session to describe a
case that almost never happens - and is stated in the tool's *output* on the one occasion it
bites, along with what to do about it.

**Result.** `tests/test_vm_boot_overlap.py`: seventeen cases covering what is offered in each
phase, that no guest tool leaks into the planning turn, that the cap exists in software and not
in the prompt, and that the opening message does not claim a machine that is not there yet.

**Not measured:** the actual saving. It should be most of one boot per session, but no session
has been run end to end since the change - the Ubuntu image is still being installed.

### 59.17 — "an explicit act by the operator" was a sentence, not a control

**Symptom.** The Ubuntu base image finished installing and went on reporting *INSTALLING*, and
every boot still attached the installer ISO. The operator asked for it to be switched over. The
comment in `owuivm-maint-prepare` and the note in 59.15 both said that detaching the media is an
explicit act by the operator - and there was nothing for the operator to press. The act existed
only as `rm` on a file in a directory their own account cannot write.

**Cause.** 59.15 removed a guess (disk size standing in for "the install finished") and replaced
it with a decision. Removing the guess was written; giving the decision a home was not. A
decision with no control is not a decision, it is a chore for whoever remembers the path.

**Change.** `owuivm-detach@.service`, a oneshot that removes `<image>.install` and nothing else,
plus a fourth button in the maintenance tool - *Installed — remove disc* - that starts it:

* It is a **unit, not a line in the GUI**, for the reason already written at the top of
  `owuivm-maint-prepare`: the tool runs as `operator`, the marker belongs to `owuivmadm`, and a
  unix group only takes effect at the operator's NEXT login - which fails confusingly for
  exactly as long as the session that installed it lasts. Going through polkit uses the
  mechanism already proven for this operator: `50-owuivm.rules` now covers `owuivm-detach@` on
  the same `subject.active && subject.user == "operator"` test that starts a maintenance boot.
* `ConditionPathExists=` on the marker, so a second press is a skip and not a failure.
* The button is **visible only while an install is in progress** and **dead while the machine is
  running**. Marking an install finished is a claim about a machine that has been shut down and
  frozen; a half-installed one still needs its disc, and the disc is the only way back to the
  installer. That is the 59.15 fault stated as a widget state.

**Result.** Measured against a real marker: the unit removed it, `owuivmd` was refused
(`Access denied`) while an active session of `operator` was allowed, and rebuilding the launcher's
argv afterwards showed `cdrom drives attached: 0`. Six new assertions in `tests/test_vm_freeze.sh`
cover the unit's existence, that its `ExecStart` removes the marker and nothing else, the
condition, the polkit entry, the GUI control, and that the control is dead while the machine
runs. The last two are greps against the installed tool: a service with no button is the same
sentence in a different file.

**Not measured:** the button has not been seen on screen. The only Chrome the browser tools can
reach is on another machine and cannot open this host's loopback, and rendering GTK4 headlessly
here would have meant publishing a VM control panel on the LAN to photograph it. The logic is
asserted, the unit is proven end to end, and the widget is one `Gtk.Button` appended to a box
that already holds three.

### 59.18 — "where is the unfreeze button?"

**Symptom.** The operator looked at a row reading *frozen* and went looking for the control that
unfreezes it. There wasn't one. Unfreezing happened as a side effect of pressing **Start**
(`ExecStartPre=owuivm-maint-prepare %i unfreeze`) and re-freezing as a side effect of stopping
(`ExecStopPost=... freeze`), so the most visible thing about a base image had no control of its
own and the buttons that did the work were named after something else.

**Cause.** The two decisions were fused when there was only one reason to start a base image -
changing it. There is a second reason, and it is the more common one: **booting the image to see
that it still comes up, exactly as the agent will get it.** Fused, that was impossible to do
safely; the only way to look at an image was to make it writable first.

**Change.** Freezing is now the operator's own verb and starting is another.

* `owuivm-thaw@.service` and `owuivm-freeze@.service`, each one chmod, each polkit-gated like
  `owuivm-maint@` and `owuivm-detach@`. The maintenance tool grows a two-state button whose
  **label is the state** - *Unfreeze* when the image is frozen, *Freeze* when it is not - dead
  while the machine is running, because qemu holds the disk open from the moment it starts and a
  chmod then changes what `ls -l` says and nothing else.
* **The disk's mode now decides what boots.** Frozen, `owuivm-launch` creates a throwaway overlay
  on `/Pool/vmrun` backed by the untouched base - the same thing `session.py` builds for an agent
  - and `close` deletes it. Unfrozen, qemu opens the base itself. So *Start* on a frozen image is
  a rehearsal of the agent's own boot, and nothing the operator does in it can survive.
* **`close` no longer re-freezes.** It copies the firmware variables back and records a checksum
  when the image was unfrozen, and does **nothing at all** when it was frozen - not even the OVMF
  variable write-back, which would otherwise make a look-at-it boot change the image it was
  supposed not to touch.
* The safety net that the automatic re-freeze provided moves to where it cannot be forgotten:
  **vmd refuses to open a session while a base image is writable**, with a sentence saying so.
  That is strictly better than the chmod it replaces, which was silent on the way past and had
  already been seen to fail silently (59.14).

**The bug this found in itself.** The first version tested `[[ ! -w "$DISK" ]]`. That asks "can
*this process* write it" - and root can write a 0444 file, so a maintenance boot started by root
took the unfrozen branch and opened the base directly. The test caught it because it runs as root
and the live boot did not, because it runs as `owuivmadm`. Frozen is the **owner write bit** and
nothing else, which is what `session.py`, the desktop tool and the prepare script had all along;
the launcher now says the same thing.

**Result.** Measured on the real image: thaw -> `660`, a session refused with
*"the ubuntu base image is unfrozen, so no session can start"* having created no overlay and
started no qemu, freeze -> `444`, checksum identical across the round trip. A full frozen boot ran
from `/Pool/vmrun/maint-ubuntu.qcow2` (backing file: the base) and left the base byte-identical
with the overlay gone. Six new cases in `tests/test_vm_launch_argv.sh` assert both branches against
whatever mode the host's images are really in; nine in `tests/test_vm_freeze.sh` cover the frozen
boot, the toggle verbs and the units behind them; `== 5a` in `tests/vm_host_asserts.sh` unfreezes
the real image and asserts the refusal, with the restore trap extended so a failed run cannot
leave an image writable.

**One-line consequence worth knowing:** `/Pool` needed a traverse ACL for `owuivmadm`
(`setfacl -m u:owuivmadm:--x`), because the throwaway disk of a frozen boot lives on the same
40 GB-quota dataset as a session's overlay rather than on the root filesystem.

### 59.19 — the share the image had to know about

**Symptom.** "How do I set up the shared workspace on the VMs?" - asked while the hand-installed
Ubuntu image sat unfrozen with `qemu-guest-agent` still not answering.

**Cause.** The project share is one virtiofs tag, `project`, and the guest-side half of it lived
in `images/ubuntu/user-data` as an fstab line written by the autoinstall's late-commands. The
image on this host was installed interactively, so it has no such line - and so do any future
ones built the same way.

The failure that produces is the worst available shape. The machine boots, the guest agent
answers, `run()` works, `write_file()` writes to `/mnt/project` - an ordinary empty directory on
the guest's own root - and the host then commits a project folder in which nothing has changed.
The Gitea link is handed over and the repository is empty. Nothing in the guest, in vmd or in
Gitea reports an error.

**Change.** The session mounts it, every time, over the channel it already has:

```
mkdir -p /mnt/project
mountpoint -q /mnt/project || mount -t virtiofs project /mnt/project
mountpoint -q /mnt/project
```

Idempotent, so an image that does carry the fstab line is one `mountpoint` check; checked rather
than assumed, so a mount that does not take raises with the guest's own words and the sentence
*"nothing written in this session would be kept"*, before the agent is given the machine. A
hand-built image now needs nothing for the share at all.

Also added, because "install it by hand" turned out to be a hundred lines of late-commands that
an interactive install silently skips: `vm/images/ubuntu/by-hand.sh`, the same list written
longhand and idempotent, and `owuivm-guest-run`, which pushes a local script into the running
maintenance guest and runs it. That leaves exactly one thing that has to be typed into a VNC
console - `apt install qemu-guest-agent`, the channel everything else travels over.

**Result.** `tests/test_vm_share_mount.py`, nine cases: no share does nothing, Windows does
nothing (WinFsp mounts it inside the image and there is no command to send from here), the mount
is created, is idempotent, is verified, and a failure stops the session with what the guest said
in it.

**What running it on the real image found.** `by-hand.sh` went in over the guest channel in 45 s
(8.5 s on a second run, which is what idempotent is for), and two of the things it was copying
from the autoinstall turned out to be wrong for **26.04**:

* **`WaylandEnable=false` has nothing to fall back to.** `xserver-xorg-core` is not installed and
  `/usr/share/xsessions` is empty, so that line does not choose X - it leaves GDM with no session
  to log into, and an autologin into nothing is indistinguishable from a desktop that failed to
  start. Removed from `by-hand.sh` **and from `user-data`**, or the next unattended build would
  produce the same broken image. The `xorg.conf.d` modeline went with it: the reason for pinning
  the mode was that the agent clicks absolute coordinates, and the mode is pinned one layer
  further down anyway, by a qemu argv that is a constant. Measured after a reboot:
  **1280x800**, from the virtio-vga device, with no display-server configuration at all.
* **Disabling snapd takes Firefox with it.** The autoinstall disables `snapd.service` outright as
  a source of background change. On a desktop image the browser *is* a snap, so a GUI agent that
  wanted to read documentation would find nothing to read it with. `snap refresh --hold` says the
  thing that was actually meant - stop changing underneath me - and leaves the snaps that are
  installed working.

**Measured on the finished image:** guest agent 10.2.1 answering 11.6 s after a reboot, a GNOME
**Wayland** session active and autologged in as uid 1000 five seconds later, `gnome-shell`
running, no first-run wizard, and a QMP `screendump` of the desktop at 1280x800 - so the agent's
eyes work on this image and GUI mode has a machine to drive. DNS resolves through
systemd-resolved to SLIRP's forwarder to the host's bind-mounted `resolv.conf`
(`curl https://example.com` -> 200), which is why the static netplan the autoinstall writes was
**not** applied here: both netplan files on this image are NetworkManager-rendered, and adding a
networkd-rendered one to fix something that already works is how a machine ends up with no
network at all.

**Not measured:** the virtiofs mount itself. A maintenance boot is given no share - that is the
one thing only a session does - so the first session on this image is what exercises it.

### 59.20 — the trap I wrote the warning about, in the file that ignored it

**Symptom.** "I clicked Install from ISO on the Windows instance and it errored out." The unit
never started - `journalctl -u owuivm-maint@windows.service` had **no entries at all** - and no
`windows.qcow2` was created.

**Cause.** The desktop tool created the blank disk, copied the firmware variables and wrote the
install marker **itself**, in `/var/lib/owuivm/base`, which is `0775 owuivmadm`. `operator` is a
member of `owuivmadm` in `/etc/group` - `id operator` lists it - and **no process in the running
desktop session has that group**, because the session predates the `usermod` that added it. A
unix group only reaches a process at its next login. Measured: `/proc/<gui>/status` `Groups:`
has `984` (docker) and not `978` (owuivmadm).

That is the exact trap written at the top of `owuivm-maint-prepare` - *"a unix group takes effect
at the NEXT login, so the tool fails in a confusing way for exactly as long as the session that
installed it lasts"* - in the one file that still depended on it. The warning was written while
moving this work into the unit, and this code path was left behind.

**The second bug underneath it.** The tool copied `OVMF_VARS_4M.fd` for **both** images. Windows
needs `OVMF_VARS_4M.ms.fd`, the store with Microsoft's keys already enrolled, or Secure Boot is on
with nothing it trusts and setup refuses the machine. Had the permission problem not stopped it,
the install would have failed later and looked like a hardware requirement. The unit's
`ExecStartPre` has had this right the whole time.

**Change.** `prepare_install()` is gone from the tool, and with it the ISO dropdown: the unit
picks the newest ISO matching the image's own pattern, so a control that looked like a choice was
describing a decision made elsewhere. The row now reports it - *"not installed — will install from
Win11_25H2_English_x64_v2.iso"* - and the button starts the unit. `DISK_SIZE` and `OVMF_VARS` went
too; a copy of a value that lives in `owuivm-maint-prepare` is a copy that will disagree with it
one day.

**Result.** Fifteen new assertions in `tests/test_vm_freeze.sh`: three that the tool contains no
`qemu-img`, no `shutil.copyfile` and no firmware path of its own, and twelve driving the unit's
own `open` verb against a scratch directory for both images - a blank disk that is writable
because an installer has to write to it, an install marker naming the media, and **the firmware
store each image actually needs**, asserted by `cmp` against both candidates so getting them the
wrong way round fails.

### 59.21 — the Windows machine was slow for a different reason than the Ubuntu one

**Symptom.** "The Windows VM is kinda slow - did you only fix the slowness on the Ubuntu machine?"

**Measured first, because the obvious answer was wrong.** The Ubuntu fix (59.12) was host-side -
`cache=none,aio=native` on every drive plus `MemoryMax=11G` - and it is in `owuivm-launch` and the
unit files, so both machines have had it all along. The cgroup of the running Windows installer
said so: `memory.events max 0` (Ubuntu's was **6,782** before the fix), `file` and `shmem` both
7.7 GiB - that is the guest's own RAM and nothing else - and `throttled_usec 347865`, a third of a
second of CPU throttling in three hours. Neither short of memory nor short of CPU.

What it *was* short of was information. qemu sat at **347% CPU** driving an idle installer.

**Cause.** Windows assumes it may be running under Hyper-V and looks for the paravirtual
interfaces that would say so. `-cpu host` alone offers none, so it behaves like a kernel that has
been lied to about its hardware: it spins on locks the hypervisor could have yielded, takes a VM
exit per timer tick and per inter-processor interrupt, and drives its APIC through a trap on every
access. The work is real; it is simply wasted. Linux needs none of this because it has virtio, so
the problem could not appear on the machine that was profiled first.

**Change.** The Windows machine - and only it - gets the enlightenment set:
`hv-relaxed,hv-vapic,hv-avic,hv-time,hv-spinlocks=0x1fff,hv-vpindex,hv-runtime,hv-synic,hv-stimer,hv-stimer-direct,hv-reset,hv-frequencies,hv-tlbflush,hv-ipi`.
Listed explicitly rather than `hv-passthrough`, which hands the guest whatever the host kernel
happens to support today: these images are frozen, and one should behave the same after a host
upgrade as before it. Every flag was verified accepted by this host's KVM before being added.

**And a clock, while the argv was open.** Windows reads the RTC as local time. The host is
`America/New_York`, so `base=utc` leaves the guest four hours in the past - which fails TLS
certificate validation and therefore breaks `pip`, `winget` and anything else that fetches, while
presenting as a network fault. `base=localtime` for Windows, `utc` for Ubuntu.

**Result.** Eight assertions in `tests/test_vm_launch_argv.sh`, both ways round: Windows has the
enlightenments and does not have `hv-passthrough`, Ubuntu is plain `-cpu host`, and each gets the
clock it expects. **Not measured:** the improvement itself. The flags apply at the next start of
the machine and the installer was mid-run; the number to compare against is the 347% recorded
above.

**"Is it just nested virtualisation?" - no, and here is the measurement.** The question came up
again once Windows was installing properly, and it is worth recording the answer because it is the
first thing anyone blames on this host:

* `steal = 0`. L0 gives this machine every cycle it asks for. Oversubscription would show here.
* `kvm_amd npt = Y`. Nested paging is on, so guest memory is hardware-walked rather than shadowed -
  that is the expensive half of nesting and it is handled.
* **127,000 VM exits/second** (635,079 in five seconds across 8 vCPUs; 65% `kvm_msr`, 11%
  `kvm_mmio`, `kvm_nested_vmexit` **zero**). Exits are where nesting genuinely costs extra, and at
  that rate they are single-digit percent of the CPU burn, not the bulk of it.

**What was actually holding it back was our own limit.** `CPUQuota=700%` on a machine with 8
vCPUs: measured at **697% of 700**, with 1,792 throttle events and **103 seconds of CPU withheld**.
One of its eight vCPUs could never run. Raised to 800% - which is exactly what 8 vCPUs can consume
- live with `systemctl set-property --runtime`, and throttling went to zero in the next 30 seconds.
Persisted for maintenance boots only: a session keeps 700% because it runs while the main model may
be answering someone, on eight threads Ollama can also use, whereas a maintenance boot is one
person doing one thing and should have the machine it was given.

**The honest ranking of what made this VM slow,** since three separate things were blamed on each
other: missing Hyper-V enlightenments (76 minutes to not-finish, versus 27 minutes to OOBE), the
overlay sitting on the SMR array after an accidental frozen boot, our own CPU quota, Windows
first-boot servicing (which ends on its own), CPU-side rendering (permanent, minor), and nesting
(real, minor, and not removable from here).

**Two things that are slow for reasons no argv can fix,** worth writing down before they are
reported as bugs: until `virtio-win` is installed, Windows drives *emulated* hardware - most
visibly the display, where `virtio-vga` with no driver is a plain VGA framebuffer and every redraw
is a full repaint - and Defender plus first-boot servicing make the first hour of any fresh
Windows install heavy. `cache=none` also costs an install real time, and it stays: it is what
stopped the cgroup thrashing that made the *Ubuntu* install crawl.

### 59.22 — the test that destroyed the thing it was testing

**Symptom.** A Windows 11 install sat on "Getting ready" for fifteen minutes. It was not stuck: it
was writing to a disk that no longer had a name, on the slowest storage in the house, and would
have lost everything the moment qemu exited.

**Cause - mine, and worth writing down in full.** `tests/test_vm_launch_argv.sh` drove the real
launcher against the **real** directories, and 59.18 added a loop to it that flips each base
image's mode to exercise both boot paths. Run while an operator was installing Windows, it:

1. **chmod'd the live base image** to `0444` and left it there, so the next Start booted *frozen* -
   which, since 59.18, means a throwaway overlay on `/Pool/vmrun`. The install went there.
2. **`rm -f`'d that overlay while qemu had it open.** The data survived only as an unlinked inode.
   `ln` and `linkat(AT_SYMLINK_FOLLOW)` both refuse to give it a name again - only `O_TMPFILE`
   files can be named after the fact - so it was going to die with the process.
3. **Killed the machine's swtpm and overwrote its OVMF variables**, because maintenance boots use a
   constant session id (`000...0`) and the test used the same one, so its `pkill -f "swtpm socket
   .*$session"` matched the operator's. Journal: `tpm-emulator: Could not stop TPM: Input/output
   error`, then `status=1/FAILURE`.

And the slowness that prompted the question was a fourth consequence: `Pool` is four spinning SATA
drives in raidz1 - two of them SMR - while base images live on `/`, which is non-rotational. The
frozen boot had quietly moved a Windows install, which is small random writes with `cache=none`,
from SSD onto SMR.

**This is the second time a test in this suite has damaged a live machine.** The first
(`vm_host_asserts.sh`, 59.x) read `is-active` after starting a unit, saw "active" from the
operator's own boot, and stopped it. Both have the same shape: **a test that shares state with
production and assumes nothing else is using it.**

**Change.**
* The test's copy of the launcher now has its paths rewritten as well as its exec line -
  `/var/lib/owuivm/base`, `/Pool/vmrun` and `/run/owuivm` all become directories under its own
  `mktemp -d`. The path *rules* are still exercised, because they are rewritten consistently; every
  path they resolve to is disposable.
* A distinct session id (`fff...f`), and the swtpm `pkill` matches on the scratch directory rather
  than on an id a real boot also uses.
* **It refuses to run at all while any owuivm unit is active**, and says why. Verified: exit 2 with
  a Windows machine up.
* The maintenance tool now warns before the same trap can be walked into by hand: a frozen image
  with installer media attached reads *"Starting it now would install into a throwaway copy and
  lose it. Press Unfreeze first."*

**Result.** All twenty-two cases pass against the scratch tree, and a run leaves
`/var/lib/owuivm/base` and `/Pool/vmrun` byte-identical. **The cost was real and is not recovered:**
the operator's Windows install was destroyed and restarted from nothing.

**The rule this earns:** a test that needs the real launcher gets a copy of the launcher, and a
copy of everywhere it points. If it cannot be made hermetic, it refuses to run while the system it
tests is in use.

**And it was not only that test.** The very next attempt to install Windows failed in
`ExecStartPre` with `cp: cannot remove /run/owuivm/000.../OVMF_VARS.fd: Permission denied` -
because `tests/test_vm_freeze.sh`, run as root, had used the same real run directory and left it
`root:root`. A maintenance boot's session id is a constant, so that path is a constant too, and two
of the three tests that touch this subsystem had independently reached for it. `owuivm-maint-prepare`
now takes `OWUIVM_RUN_DIR` alongside the base, ISO and overlay overrides it already had, the freeze
test uses its own, and `open` checks the directory is writable and names the owner rather than
letting `cp` report it one line later. Verified: both suites now leave `/run/owuivm` containing only
the agent socket directory.

### 59.23 — six calls to read a web page

**Symptom.** Looking anything up in GUI mode cost the agent a screenshot to find the browser, a
click to focus it, a click in the address bar, a `type`, a `key` for Return, and a screenshot to
see the result. Six round trips through a 27B model with thinking on, five of them spent
operating a text field, and every one of them a full prefill.

**Change.** Three tools, two of which never touch the machine at all.

* **`web_search`** and **`read_page`** run on the **host**, in vmd, against the same
  `/tools/web_search` and `/tools/fetch_page` the main model calls - same SearXNG, same engines,
  same result formatter, same cache, same readability extraction and SSRF guard. Not a second
  search client: a copy would be the one nobody remembers to fix. vmd already holds the Gitea
  token, so a credentialled call belongs on that side of the socket.
* **`open_page`** puts a URL on the screen in one call. GUI mode only - in CLI mode there is no
  desktop, and a tool that cannot work is a turn the model spends discovering that.

**Why the agent cannot search for itself.** SearXNG sits inside netguard's network namespace, on
loopback, behind a default-deny filter. The guest cannot reach it - that is the sandbox working -
and neither can the agent container, whose allowlist is vmgw alone. Measured while wiring this up:
the host cannot reach `172.22.0.11:8080` either (it times out), which is why this goes through the
gateway at the pinned `172.22.0.10:8000` rather than to SearXNG directly.

**The URL is the only thing that crosses into the machine,** and it is validated against
`^https?://` with RFC 3986's character set and then passed to the guest as an **environment
variable**. qemu-ga takes the environment as a list, so there is no quoting to get wrong and no
way for a URL to close the string it is in. A first version of the test demanded that
`https://example.com/$(id)` be *rejected*; that was the test being wrong - `$`, `(` and `)` are
sub-delims and every Wikipedia disambiguation link contains parentheses. Rejecting them would have
broken ordinary use to defend against an injection that cannot happen. The case now asserts the
URL is accepted and that neither script has a slot to format it into.

**Two guest-side details that are not obvious.**

* On Ubuntu the guest agent is root and the desktop belongs to uid 1000, so the launch crosses
  into that session explicitly - `runuser` with `XDG_RUNTIME_DIR`, the session bus, and both
  `WAYLAND_DISPLAY` and `DISPLAY` because the image may be either - and `setsid`s, so the tool
  call returns instead of waiting for a browser to be closed.
* On Windows qemu-ga is **LocalSystem, in session 0**. A GUI process started from there does not
  fail, it succeeds *invisibly*, on a desktop nobody is looking at - so the agent would screenshot
  an unchanged screen and conclude the page did not load. Until the in-guest broker exists, the
  launch goes through a scheduled task marked `/it`, which the task service starts inside the
  logged-on session; it is created, run and deleted in one call.

Flags worth keeping: `--no-first-run --no-default-browser-check` (every session is a first run,
because the disk is restored from the same frozen base), `--password-store=basic` (otherwise a
gnome-keyring unlock dialog opens over the page on a fresh profile), and `--start-maximized` (a
known window size, so clicks land where the screenshot says).

**Result.** 41 cases in `tests/test_vm_web_tools.py`. Measured end to end from vmd: a search for
"qemu-guest-agent guest-exec windows localsystem" returned ranked results in one call, and
`read_page` of the QEMU guest-agent docs returned extracted markdown.

**Not measured:** `open_page` against a live guest. Only one machine runs at a time and the
Windows install had the slot; the Ubuntu image is frozen and waiting. The launch script's shape is
asserted, its behaviour is not.

### 59.24 — first real testing, and the five things it found

Both images finished (Ubuntu 26.04.1, Windows 11 25H2 activated, guest agents, browsers, python),
so the whole system was driven end to end for the first time. It works. It also had five defects
that only a real session could have found, every one of them in the seam between "the software
reports success" and "the thing actually happened".

**1. The machine was handed over showing a boot splash.** `_wait_for_desktop` waited for a seat
session, and **gdm's greeter is a seat session** - so it passed before the user was logged in.
Fixed, and it still handed over a splash: measured, `gnome-shell` running, plymouth gone,
`graphical.target` active, and the framebuffer on the Ubuntu splash for **over three minutes**.

The display shows the foreground *virtual terminal*, and the session was not on it: `fgconsole`
said 1, the session's TTY was tty2, `loginctl` agreed `Active=no`. Plymouth had tty1 and nothing
switched away. `chvt 2` put the desktop on screen in under a second. That is also why it looked
intermittent - sometimes gdm switches, sometimes it does not - so the probe no longer waits for
the switch, it **performs** it. Measured after: two clean boots, desktop on screen both times,
ready at 59.0s and 60.7s. The settle after the switch is 3s, not 1s, because mutter repaints the
whole screen and a one-second settle handed over a 3.4 KB near-black frame.

**2. `open_page` reported success for a browser that never started** - twice, two different ways.

* `DISPLAY=:0` made Chromium choose X11 on a Wayland session: *"Missing X server or $DISPLAY"*,
  into `/dev/null`.
* Reading the display variables out of **gnome-shell's** `/proc/environ` was worse, because it
  looks correct: gnome-shell **is** the compositor, so it never had `WAYLAND_DISPLAY` - it creates
  that socket rather than consuming one - and the launch fell back to X11 again.
* The check that should have caught both was `pgrep -u user -f chromium`, which matched **the
  launcher's own `runuser ... chromium ...` command line**. The same self-match that a `pkill -f`
  walked into earlier in this project.

Now: `systemd-run --user`, which starts it in the user's own systemd instance - where a desktop
session exports its environment, measured on this image as `WAYLAND_DISPLAY=wayland-0`,
`DISPLAY=:0` and mutter's Xwayland `XAUTHORITY`. The unit is **named**, so "did it start" is read
back from `systemctl --user is-active`, which cannot be fooled by a command line. A framebuffer
diff was tried in between and discarded: **the clock in the top bar changes it every sixty
seconds**.

**3. `web_search` worked from a shell and timed out from the daemon.** `owuivmd.service` is
`IPAddressDeny=any` with two allows for Gitea, and the tool gateway was not one of them - so the
cgroup dropped it. A shell is not in the unit's cgroup, which is exactly why testing from one
proved nothing. `IPAddressAllow=172.22.0.0/24` added.

**4. Work could be committed and never published.** `_publish` pushed only `if STATE.commits` -
whether *this* session committed. A session that is cancelled, times out, or loses its daemon
leaves a commit that nothing ever pushes. Measured: `primes.py` was committed locally and
`vmlab/vmtest` did not exist in Gitea. Now the question is `has_commits()` - is there anything to
publish at all - which is the question that was meant.

**5. The agent spent turns discovering what was in the machine.** One guest command at handover
now reports the OS, python, browsers, the desktop user and the project folder's free space, as
**tool output** rather than prompt text, for the reason this stack has now proved three times.

**Result - the whole path, measured.**

* CLI session, 12 steps, **156s**: wrote a word-frequency script, ran it, re-counted independently
  with `tr/sort/uniq`, committed, pushed. Supervisor re-ran the verification: passed.
* GUI session, 25 steps, **189s**: searched the web, opened kernel.org with `open_page`,
  screenshotted it, wrote what it found, committed.
* Live progress: 4 screenshots published to the filter's file during one GUI run.
* **Through Open WebUI, in a browser**: the status line went *"Booting a ubuntu machine - planning
  while it starts"* -> *"1/4 · Run the script and capture the full output"* -> *"The machine has
  been destroyed"*, and the main model answered with a real table of the ten largest files under
  `/usr/share` (CJK font collections, as it happens) and the commit that holds them. One tool
  call, 16.2s to re-prefill 13,692 tokens.
* The ask-first rule fired unprompted on the way: a task saying only "python" was refused with
  *"this task mentions something that runs on both machines, and you chose ubuntu"*.

**Not measured:** anything on the Windows machine. `run` there still goes to `/bin/bash` through
`guest.shell`, which is Linux-only, and the project share needs WinFsp before `/mnt/project`
means anything - both are Phase 8 work that the finished image now makes possible.

### 59.25 — a view worth watching, and a session record that outlived its machine

**The view.** The live channel showed one status line. The operator asked for what they would
actually watch: the task list above, and under it the thing the session has. A CLI session gets a
**terminal locked to twenty lines** - the commands it ran and what came back, padded to a constant
height so the message does not jump as output arrives - and a GUI session keeps the screenshot,
because a picture of a desktop beside a transcript of commands that are not on it is two views of
nothing. `events.py` carries `tasks`, `terminal` and `interface`; the filter renders whichever
applies. Em dashes are out of everything the operator reads, here and in the rendered task table.

The terminal is unwrapped on the way through: `run` returns `exit code: 0 / stdout: / ...`, which
is the shape the **model** needs, so the view strips the framing and shows output as output with
`[exit N]` only when it is news. Measured, live:

```
$ ls -la /definitely/not/a/real/path
ls: cannot access '/definitely/not/a/real/path': No such file or directory
[exit 2]
$ bash /mnt/project/vardirs.sh
2.2G	/var/lib/
255M	/var/cache/
```

**The defect the testing found.** Rebuilding the agent container mid-session killed the loop that
would have called `session_close`. The unit was reaped and `systemctl is-active` said inactive -
and vmd still held the session record, so **every later session was refused** with "a session is
already running" until the daemon was restarted. One interrupted session would have taken the
subsystem out until somebody noticed.

`v_session_open` now asks whether the MACHINE exists rather than whether a record does: a record
whose machine is gone is destroyed, cleared, and logged - `vmd: cleared stale session 70705573...
- its machine was already gone` - and the caller gets their session. Proven by stopping
`owuivm.service` under a live session and opening another without restarting vmd.

**Needs doing by hand:** the Open WebUI filter is installed by pasting its source, so
`owui-functions/vm_agent_progress.py` has to be pasted in again for the new view to appear.

### 59.26 — Windows was never wired up, and a live session found out the hard way

**Symptom.** The operator started a Windows session and watched the agent reboot the machine and
sleep for minutes at a time. Every command came back
`guest-exec: Failed to execute child process (No such file or directory)`.

**Cause.** `Guest.shell()` was `exec(["/bin/bash", "-lc", script])`. Unconditionally. The module
was written as the Linux half with the Windows half left for the broker, and nothing in between
ever said so - so a Windows session came up healthy, answered its guest agent, and failed every
single command. To the agent that does not read as "wrong shell", it reads as a machine that has
stopped answering: so it rebooted it, waited 90 seconds, tried again, waited 120, rebooted again,
waited 180. The repeated-failure guard fired, correctly, at a machine that was never going to work.

**Change.** `shell()` takes the machine it is on: PowerShell with `-NoProfile -NonInteractive` for
Windows, bash for Linux. `Session.guest()` passes `os_name`. And with that one line working,
everything else that had only ever run against Linux surfaced at once:

* **The project path.** `_guest_path` returned `/mnt/project/...` for both. Windows gets
  `Z:\...`, because virtiofs is a drive letter there.
* **The desktop probe.** `loginctl`, `fgconsole` and `chvt` mean nothing on Windows; it waits for
  `explorer.exe` instead, which is what "somebody is logged in and the shell has drawn" looks like.
* **`check_internet`** was `curl` and `getent`; Windows gets `Invoke-WebRequest` and
  `Resolve-DnsName`.
* **Machine facts** had no Windows branch at all. The first attempt at one nested double quotes
  through Python into PowerShell and returned **nothing, silently** - the one outcome a
  fact-gathering probe must not have. Rewritten as a script in single quotes.

**And then the numbers.** The facts probe still came back empty on a cold boot, and timing each
part separately showed why: the **first** PowerShell in a freshly booted Windows guest takes over
ninety seconds - Defender inspecting it, .NET warming - while every one after it takes five to
nine. So the shell is now warmed during the boot window, where the agent is writing its task list
and nobody is waiting. Measured after: cold Windows session **ready in 115s with complete facts**,
against 32s for Ubuntu CLI and 60s for Ubuntu GUI.

**What the facts now say, which is the point:**

```
python: not on PATH for this account
running as: SYSTEM (the guest agent is LocalSystem, not the desktop user)
project folder: NOT MOUNTED - nothing you write will be kept. Say so rather than working around it.
```

**Two things the Windows image still needs, both measured from inside it.** `VirtioFsSvc` is
installed and **Stopped**; starting it fails with *"A timeout was reached (30000 milliseconds)
while waiting for the VirtIO-FS Service service to connect"*, and
`Test-Path 'C:\Program Files (x86)\WinFsp'` is **False** - the viofs kernel driver is there, the
user-mode half it needs is not. And Python is installed for a user rather than for the machine, so
the account the guest agent runs as cannot see it. Until both are fixed a Windows session can run
PowerShell and keep nothing.

**Still not true on Windows:** the guest agent is LocalSystem in session 0, so this is not the
"normal elevated PowerShell" the design asked for. Scripts and their output are exactly right;
anything that depends on being the logged-in user is not. The in-guest broker is what closes that,
and it is still unwritten.

### 59.27 — the live view, as the person watching asked for it

Three corrections from someone actually watching a session: the task list and the terminal should
be **fully visible** rather than something to scroll through, a long line should be **cut** rather
than put a horizontal scrollbar under the box, and the code block should not appear in GUI mode at
all. So the embed is sized to its content with no height of its own, every terminal line is its own
element that clips at the edge, the task list wraps instead of scrolling, and the twenty rows are
always present so the box does not change size as output arrives. Em dashes are out of everything
the operator reads.

### 59.28 — the Windows image finished, and then saved so it need not boot

Two pieces of work, in this order because the second depends on the first: a saved machine is
only valid against the exact image it was taken from, so the image had to be finished first.

**The image.** `WinFsp` was missing, which is why `VirtioFsSvc` was installed and Stopped and
`Z:` did not exist - the viofs kernel driver was there, the user-mode half it talks to was not.
Installed 2.1 with `INSTALLLEVEL=1000` (the default level takes the driver only and the service
still fails to connect), service set to Automatic. Python was installed for a user, invisible to
the account the guest agent runs as; reinstalled with `InstallAllUsers=1 PrependPath=1` as 3.14.7,
matching Ubuntu's 3.14.4 line so the same script has one fewer reason to behave differently on the
two machines. The timezone was **Pacific** while the host RTC is `base=localtime` on an Eastern
host, so the guest clock was three hours out before anything resumed anything; set to Eastern, with
`w32time`'s phase-correction limits removed so a large correction is allowed at all, and Windows
Update's automatic install and reboot turned off. Verified from a real session: `Z:\` mounts, a
file written in the guest appears on the array, python answers, the clock is within a second.

**The saved machine.** `vmctl snapshot --os windows` boots the image once, lets it settle, and
writes its memory and its disk. A session then branches from that instead of booting:

| | cold | warm |
|---|---|---|
| Windows session ready | 91 - 115 s | **30 s** |
| first PowerShell in the guest | 90 s+ | already warm, in the saved memory |
| saved state | - | 3.92 GB, written in 14.6 s |
| saved disk | - | 163 MB (the boot's own writes) |

**Four things that had to be got right, each measured rather than assumed.**

1. **The share cannot be in a saved state.** `migrate` refuses outright: *"Migration disabled:
   vhost-user backend lacks VHOST_USER_PROTOCOL_F_LOG_SHMFD."* So the snapshot is taken with no
   share, and the share is hot-plugged after the resume. q35's root complex is not hot-pluggable,
   so the argv now always carries a `pcie-root-port` - and the share goes into it on **both**
   paths, at the same address, because a drive letter that moved between sessions would be worse
   than a slow boot.
2. **`exec:` cannot be the transport.** It fails with *"Unable to write to command: Broken pipe"*:
   the unit's syscall filter stops qemu spawning the helper, which is the sandbox working. `file:`
   needs nobody - and then a second problem, that qemu creates its files 0600 as the sandbox user,
   so vmd cannot read what it just asked for. Copying four gigabytes was one answer and letting
   qemu write into the saved-machine directory was another; both are worse than passing a
   descriptor vmd opened itself, which is what it does.
3. **A saved state is only valid against one disk**, down to the page cache. Resume it against a
   changed image and nothing reports an error - the guest simply runs with a filesystem in memory
   that disagrees with the one on disk. The freeze already records a checksum, so the manifest
   records which one the state came from and `warm_set()` compares them. Verified by falsifying
   the manifest: the saved machine is refused and the session boots cold, 96.6 s.
4. **It must never be load-bearing.** A saved machine that will not load becomes a cold boot, not
   a failed session. Verified by truncating the state to 200 MB: the resume failed, vmd booted
   cold in 109 s and told the caller *"the saved machine could not be resumed, so this one was
   booted from cold. Nothing about the machine differs."*

**The clock, which is the reason the operator asked for any of this.** A resumed machine believes
it is whenever it was saved - measured at **26 seconds behind** on the first warm session, and it
would be days for a snapshot left over a weekend. Windows minds most: a wrong clock fails TLS, so
every download breaks in a way that reads as a network fault. `guest-set-time` runs on **every**
session, warm or cold, before anything else touches the guest - and it is **read back and checked**
rather than assumed, which is how the first version came to be 26 seconds out while reporting
nothing: the share service had just restarted, the agent was busy, the call failed and the error
was swallowed. Now: `clock: set, within 0.0s of the host`, measured skew -0.8 s.

**And two more Linux-shaped assumptions the first real Windows session found.** The supervisor's
verification prelude was `cd ... 2>/dev/null || cd /;` for both machines, and PowerShell has no
`||` - so a session whose work was entirely correct was reported to the user as a *claim*, because
the re-run failed on a parse error in vmd's own wrapper. And the agent lost two turns wrapping
commands in `powershell.exe` when it was already in PowerShell, and using `&&` in 5.1. The first is
fixed; the second is now in the machine facts, where this stack has three times proved a model
reads it as fact. Measured after both: 9 steps, 129 s, four commands, **all exited 0**,
verification passed on the supervisor's re-run.

**Not done:** Ubuntu has no saved machine. It boots in 32 s and the agent's planning turn hides
most of that, so the saving would be a few seconds of somebody's attention; the machinery is the
same verb (`vmctl snapshot --os ubuntu`) if that changes.

### 59.29 — six real questions, and the three that never reached the machine

**Symptom.** Everything in 59.17-59.28 was proved a verb at a time, by me, knowing what I wanted
to happen. The operator asked for the other test: six prompts written as a real person would
write them, with no hand-holding, put into the chat and left alone.

**Result, the short version: the subsystem passed and the routing did not.** Three of the six
used the machine and all three worked. Three did not use it, and two of those should have.

| # | Asked for | What happened |
|---|---|---|
| 1 | a Python renamer that breaks on odd filenames, OS unstated | chose Ubuntu itself, built 14 adversarial filenames with a deliberate mtime collision, **7 of 8 tasks done and 1 blocked with a real finding** - the script is not idempotent, a second run re-stamps every file. Root cause named. Committed and pushed. |
| 2 | a PowerShell service audit, "make sure it genuinely works" | Windows warm start, **6/6**, supervisor re-ran the script itself and it passed on real service data. The answer flagged that `Automatic (Delayed Start)` needs `-like`, not `-eq` - a subtlety that only running it revealed. |
| 3 | "show me what the Ubuntu desktop looks like out of the box" | **never touched the machine.** Three `web_search`, three `fetch_page`, and the sentence *"I can't drop a live screenshot into this chat"* - written while holding a tool that boots that exact desktop and photographs it. |
| 4 | "does `Restart=on-failure` restart a clean exit? I want what it actually does, not what the docs imply" | **read the man page.** Correct answer, wrong method, and the prompt had said so in advance. |
| 5 | "can the VM reach my NAS at 192.168.1.50?" | **zero tool calls.** Reasoned from its prompt that the machine has no route to the LAN and declined to spend a session proving a guaranteed timeout. Right call - asserted rather than demonstrated, which here is the cheaper of the two. |
| 6 | a Flask stand-in service, "I want to know it actually starts" | `vm_agent` **refused** the ambiguous OS - *"CANNOT DO THAT YET: this task mentions something that runs on both machines"* - the model asked, I answered Ubuntu, it called again. **7/7**, pip install, background server, real curls, determinism re-checked, server killed, committed. The ask-first rule exactly as designed. |

**Cause.** Not a fault in the subsystem - every part of it that was reached worked, including the
parts hardest to trust: blocked-with-reason, the supervisor's own re-run, the ask-first refusal,
three new Gitea repos, and a clean host afterwards (no qemu, no overlays, both images frozen).

The fault was in **what the model had been told the tool is for**, and it was my wording. The
docstring said, in as many words, do not use this to find out *what software is, how it is
configured, or what it looks like*. That sentence is what lost test 3: "show me the desktop" is a
"what it looks like" question, and I had labelled it out of scope for the one tool that could
answer it. Test 4 failed for the neighbouring reason - the guidance keyed on the user asking for
proof, and "I want to know what it actually does, not what the docs imply" is a request for proof
that does not use any of the words I had listed.

**Change.** The line is redrawn where it actually falls - not *what kind of thing is being asked
about*, but *whether a document can answer it*:

* USE IT when the answer has to be true of a real machine rather than of the documentation - and
  explicitly: **if an honest answer would have to begin "the documentation implies" or "it
  should", that is the sentence you cannot write. Run it.**
* DO NOT USE IT when a document already answers and nobody asked for proof.
* "what it looks like" moves out of the refusal and into `gui`, which is where it belonged:
  `gui` is the slowest thing available and is for when the answer has to be **seen** as it
  appears on screen, or the work can only be done by clicking. *"Show me what this looks like"
  is what it is for."* Plus one line aimed squarely at test 3's failure: do not tell someone you
  cannot show them a screen without first asking whether this is the tool that can.

**Result.** Test 4's prompt re-run verbatim against the new docstring: the model called
`vm_agent` and wrote itself an empirical task - including a guard that stops the test if `systemd`
is not PID 1, because the result would otherwise be meaningless. The same wording is in
`prompts/system-prompt.md`, which is **not deployed**: the live system prompt is baked into the
Ollama model, so it needs a rebuild to take effect. The docstring is served live and is doing the
work on its own.

**Two smaller things the round recorded.** The model writes `/mnt/project` into Windows task text
and the agent silently corrects it - now covered by "write the job as instructions, not paths".
And the live task list and terminal are cleared when a session ends, so they exist only while
running; the final task table survives in the tool result, the terminal does not.

### 59.30 — asked before the machine starts, and a box that was never the size it looked

**Symptom.** Two unrelated complaints from the same session, both about the same minute of the
user's time. A `vm_agent` call takes minutes and destroys its machine afterwards, so a job sent
with the wrong wording costs the whole session - and there was no point at which the person who
asked could see what was actually being sent. And the live view, rebuilt in 59.27 explicitly so
that nothing would need scrolling, **still had a vertical scrollbar**, in one box rather than
the two that were asked for.

**Cause, first half.** Nothing structural - the feature did not exist.

**Cause, second half, and this is the interesting one.** Two separate bugs, neither of which
anything could see from the outside.

A bad edit had left **a second copy of `_embed`** in the class. Python keeps the last definition
of a method, so the file parsed, imported and ran perfectly while silently using the old
single-box version; the new two-box one sat above it as unreachable code. Nothing failed and
nothing warned. The only symptom was the wrong thing on the screen, which is exactly the symptom
that gets blamed on the CSS.

Underneath it, the height. 59.27 says the embed is "sized to its content with no height of its
own", and that was **an assumption, not a measurement**. Read at source, Open WebUI sizes one of
these frames two ways: on load it reads `contentDocument.body.scrollHeight + 20`, and it listens
for `postMessage({type: "iframe:height"})`. The first **cannot work here**. The frame is sandboxed
without `allow-same-origin` - the embeds renderer passes `iframeSandboxAllowSameOrigin ?? false` -
so `contentDocument` throws a cross-origin error, and the read sits inside a bare `catch {}`. The
height is then never set at all and the frame keeps its stylesheet default of **150px**. Measured
in the browser: a seven-line task list needs 138px and a twenty-line terminal 364px, and both sat
at 150 with a scrollbar.

**Change.** The tool asks first, and it asks from inside itself. `_ask_before_starting()` writes
the job to a file, the filter running inside Open WebUI picks it up and puts **one dialog** to the
person - empty and OK runs it, typing what to change hands their words back to the model, Cancel
starts nothing. It is done in the tool rather than in the prompt for the reason this repo has
proved three times: the model is simply blocked in its call and cannot skip it, ask badly, or
report the answer unfaithfully. A dialog nobody reads within 20 s is not a veto - the job runs,
because a confirmation that nobody can see must not be able to stop work.

For the view: the duplicate method is gone, and the frame now **measures itself and posts its own
height**, which is the route that needs no same-origin and which `allowScripts` (hard-coded on for
embeds) permits. It measures the **wrapper element**, not the document - a third trap, found by
measuring: `html{overflow:hidden}` makes `documentElement.scrollHeight` report the frame's
*current* height whenever the content is shorter, so a box measured that way grows and never
shrinks again. A `ResizeObserver` re-reports, which is what catches a screenshot that is still
decoding when the rest of the box is ready, and a zero is never sent because a frame in a
background tab has no layout and measures 0.

**What the dialog shows, and why it is not the job.** The job the model writes is an instruction
sheet for another machine - numbered steps, exact commands, the wording that stops an agent
improvising. Real ones measured 1,900 to 2,600 characters, and the first version showed them
whole. Two things were wrong with that. Nobody reads an instruction sheet to decide whether to
press OK; and Open WebUI does not scroll that dialog, so a job that tall pushed OK and Cancel
**150px below the bottom of an 800px window** and the question could not be answered at all.

So the dialog gets a short summary of what the job will *do*: the opening line, then the head of
each step, six of them and a count of the rest. Measured: 1,861 characters became seven lines.

**It is built from the job, not written by the model.** This is the one screen where a summary
that drifts from the text is a real problem - the person approves the summary and the machine is
sent the job. A summary pulled out of the job's own headings cannot describe work that is not in
it, and asking the model for one would be both a second thing to trust and more work for it to
do, which is the opposite of what was asked for. A test asserts that every word in the summary
appears in the job.

Four things had to be measured rather than assumed, each found by looking at the rendered dialog:

* **A bulleted list under a step is not a step.** A real job's bullets were the *data* the steps
  operated on (`- /tmp/logtest/recent.log -> mtime = now`), and counting them pushed every actual
  step out of the summary. Numbered lines and `STEP` headings win; bullets are used only when
  there are none.
* **Sentences end at `.!?` and nothing else.** Splitting on the colon too turned
  `Run: ps -p 1 -o comm=` into the bullet `Run:`, which says nothing - the useful half is always
  on the far side of the colon.
* **An abbreviation is not a full stop.** `e.g.` and `(i.e.` cut bullets off mid-word until the
  punctuation around the token was stripped before checking it.
* **A plain newline is a soft break in markdown.** Consecutive lines were joined into one
  paragraph and two steps ran together on a single wrapped line. A `- ` list item at column 0 is
  the one form the renderer keeps on its own line: indent it and it becomes a nested block, use a
  bullet character of your own and it is not markdown at all and gets joined like any other prose.

A step that is only a label - `Report:` - takes the line under it, because a bullet reading just
`Report:` spends a line and says nothing.

**Result.** All three paths driven through a real chat, not asserted:

* **Accept** - dialog shown, empty + OK, job dispatched, machine booted, seven tasks run, the
  agent correctly reported that `-mtime +7` keeps a file at exactly 7 days *and* at 7 days 12
  hours and deletes at 8, the non-`.log` control survived, and the nested file did not. Machine
  torn down, `owuivm.service` inactive, no qemu left.
* **Refine** - typed *"Do not install anything. Only check what is already there, and also print
  the exact python3 path"*; the model rewrote the job to say exactly that and **asked again**. No
  machine was started in between.
* **Cancel** - no machine started, no job written, and the model closed the turn with four words
  rather than arguing with it.

**And then the view killed a session, which is the part worth writing down.** A run finished all
six of its tasks, verified them, and returned nothing at all. The agent loop had died at the last
step:

```
File "/app/loop.py", line 464, in _terminal_for
  return [f"$ {str(arguments.get('verification', {}).get('command', ''))[:300]}"]
AttributeError: 'str' object has no attribute 'get'
```

`finish` is declared as taking `verification` as an object with a `command` in it. The model sent
a string. The renderer for the **live terminal** - a cosmetic feature, added so somebody watching
the chat can see what is happening - called `.get` on that string, and the exception came back out
through the agent loop and threw the finished work away.

The narrow fix is one `isinstance`. The real fix is the rule that was missing: **the picture of
the session has no bearing on whether the work succeeds, so it does not get a vote.** Every
renderer now returns `[]` rather than raising on anything the model can send, and the call itself
goes through `_show()`, which swallows whatever gets past that and logs it at debug. One malformed
argument costs a line of output, not a session.

`tests/test_vm_live_view_safety.py` covers it: the exact call that crashed, every renderer against
eight shapes a model might send in place of an object, a deliberately exploding renderer *and* a
deliberately exploding emitter, and a source-level case that fails if the loop ever calls
`ui.terminal(...)` outside the guarded path - which is how the first one got in. Reverting the
`isinstance` makes the first case fail, so the test is known to catch the original bug rather than
merely to pass.

Re-run afterwards, same prompt: **8/8 tasks, status Done, no loop errors.**

And the view, measured live in the browser during a session: **two iframes**, heights **86px** and
**364px**, neither at the 150px default, and no horizontal scroll on the page.
`tests/test_vm_live_view.py` covers all of it - including a case that fails if any method in the
filter is ever defined twice again, and one that fails if the height is measured from the document
instead of the content.

### 59.31 — four things wrong with the last thing the user sees

Four complaints from one real session, none of them about whether the work was done.

**1. It wrote the script, handed it over untested, and waited to be asked.** The reply to
"write a powershell script that will pull system info" was a script; only after "yeah go ahead
and test it" did it reach for a machine. The confirmation dialog (59.30) changes the economics
here and the wording had not caught up: the user is shown the job before anything starts, so
*proposing* a verification costs them one click. The docstring now says so directly - **when you
have written a script the user is going to run, call this in the same reply** - on the grounds
that an untested script is a guess however good it looks.

**2. Reopening the chat mid-session showed no progress at all.** Two defects, one real and one
worse.

The real one: the watcher emitted **only on change**. These updates are pushed, so somebody who
reloads the page or opens the chat on a second device joins partway through and receives only
what comes next - and a session holds one step for minutes at a time (`Waiting for the machine
to finish booting` held across four consecutive polls, measured). A viewer arriving just after a
change sits in front of an empty message for the length of the step and concludes nothing is
happening. The whole view is now re-sent on a 10-second heartbeat as well; `replace: true` makes
that idempotent, so the cost is one small message per tick and reattaching fixes itself.

The worse one: the watcher's `finally` block wiped the view and announced **"The machine has
been destroyed."** - unconditionally. That block also runs when the watcher is cancelled or hits
its own `max_minutes` deadline, and in those cases the machine is still running. The watcher
stopping is not the same event as the session ending, and only the latter now says so; the other
says "No longer watching this session."

Verified from the logs rather than assumed: a full page reload at 20:13:37, about 25 seconds
before that session ended. `statusHistory` (33 entries) and `embeds` both persist through
`get_event_emitter`, and events are broadcast to `user:{user_id}` rather than to a socket
session, so a reconnected browser is in the right room - which is why the heartbeat is sufficient
and why nothing needed to change on the Open WebUI side.

**3. Asked for a link to download the script, given a link to a repository.** Those are different
things: one saves a file, the other opens a page to navigate. The report now lists every
committed file with the address that serves it (`/raw/branch/main/<path>`), and says plainly not
to offer a repository URL in place of one. `git ls-files` rather than a directory walk, because
the walk would also list whatever the guest left uncommitted and a link to that is a 404 with the
operator's name on it. Measured: `http 200, 1346 bytes, text/plain`.

**4. The repository said nothing about what it held.** Every repo was labelled *"Created by the
owuitools VM agent for project 'x'"*. The operator's words: it "just has to accurately describe
what the app does briefly - not sell it to me". So it is **generated from the job**, for the same
reason the confirmation summary is (59.30): a description that is generated cannot be a pitch and
cannot describe work that was not asked for.

The first version was too literal - it set *"On the Ubuntu machine, write a small Python script
called wordcount.py that takes a filename and prints the..."*, which is the instruction, verbatim
and cut off. A job opens by addressing the machine and telling it to build something; a
description labels the thing that was built. Both leading clauses are now stripped, and only
verbs that CREATE are removed, so a project whose job was to *test* something is still described
by that word. Measured on the next real session:

```
Small Python script called sumcsv.py that reads a CSV file and prints the total of a named numeric column
```

An existing description is never overwritten unless it is the old placeholder, so a repository
somebody has labelled by hand keeps its label - and repositories created before this change keep
whatever they had.

**Result.** Two full sessions driven end to end against the live stack: 6/6 and 5/5 tasks,
supervisor verification passed, direct links present and fetching, description correct on the
repository created after the fix. `tests/test_vm_git_hardening.py` covers the links and the
description; `tests/test_vm_live_view.py` covers the heartbeat and the honest close.

**Not verified:** the reopen itself. The browser tooling disconnected partway through this work,
so the fix is reasoned from the emitter's source and the request log rather than watched
happening. The two defects behind it are both real and both fixed, but nobody has yet reloaded a
chat mid-session and seen the boxes come back.

### 59.32 — the agent and its supervisor were standing in different directories

**Symptom.** A Windows session was asked to write a PowerShell script, test it and keep it. It
did all three, correctly, and the session was reported as a failure. The model read the failure,
worked out what had happened, and spent **a second entire session** redoing the same work -
about four minutes of machine time to move one file.

**Cause.** The supervisor's re-run started in the project folder. The agent's own `run` did not:
it inherited whatever directory the guest agent happened to start in, which on Windows is
`C:\Windows\system32`. So `Set-Content Get-SystemInfo.ps1`, `Get-Content .\Get-SystemInfo.ps1`
and running the script all agreed with each other and all passed - and then the supervisor
re-ran the verification one directory away and found nothing.

Two copies of "where does a command run" is exactly how they came to disagree. It had been
written once in `verify.py` and never at all in `vmd.py`.

**Change.** One `working_directory_prelude(session)`, called by both. It is the project folder on
both machines, which is also the only place work survives - a file written anywhere else is
deleted with the disk. `tests/test_vm_working_directory.py` asserts both callers use it and that
the directory is decided in exactly one place.

**Result.** The same case re-run, with the agent deliberately told to use a relative path: the
supervisor's `powershell.exe -NoProfile -ExecutionPolicy Bypass -File Get-Uptime.ps1` **passed,
exit 0**, first session, no retry.

---

Three smaller things from the same run.

**The reply pasted the whole script back into the chat.** It had already been given a download
link, so the code was a wall to scroll past to reach the one line the user wanted. Both the
docstring and the prompt now say: give the link, say what it does and what the test showed, and
quote a few lines only where the point is about those lines.

**The repository description came out as a shouted instruction** - *"IMPORTANT: create
Get-SystemInfo.ps1 INSIDE YOUR PROJECT FOLDER..."* - because the description is generated from
the job (59.31) and that job was the corrective re-prompt. The directory fix removes the cause;
`IMPORTANT:`/`NOTE:`/`WARNING:` are now stripped like the other labels in case a job opens that
way again.

**The download link 404s, and that is authentication rather than a broken URL.** Measured on the
exact link from the chat: anonymous **404**, authenticated **200, 3,976 bytes**. The `vmlab` org
is deliberately **Limited** visibility - public repos inside it, readable by any signed-in Gitea
user and invisible to everyone else - which is the arrangement this project was asked for. A link
to it therefore only works in a browser that is signed in to `git.example.com`. Recorded rather
than changed: making it work anonymously means making every `vmlab` repository readable by
anyone on the internet, which is the operator's call and not a bug fix.

### 59.33 — a release per session, and what actually downloads

**Symptom.** A link to the file in the repository opens the source in the browser instead of
saving it. The operator's decision was to publish a **release at the end of every session** and
hand over the releases page instead.

**What I got wrong first, and measured second.** I assumed release assets download. They do not:
Gitea decides inline-or-attachment by MIME type, so a `.py` or `.ps1` comes back
`text/plain; Content-Disposition: inline` whether it is a raw file URL *or* a release asset, and
no query parameter flips it. Measured on all three.

The one thing on that page served as an attachment is the archive Gitea builds for the tag -
`/archive/<tag>.zip` returns `Content-Disposition: attachment`, which is the "Source code (zip)"
link. So the releases page is the right thing to hand over for two reasons rather than one: it
lists the session's files by name, and it carries the one link that saves to disk. The report says
exactly that, rather than claiming the files download.

**Change.** After the push, and separately from it, vmd creates a release tagged `v1`, `v2`, ...
and uploads the committed files as assets. The tag number is counted from what is already
published rather than held anywhere, so it survives a vmd restart; only one session runs at a
time, so there is nobody to race with. Assets are capped at 10 files and 25 MB each, taken from
`git ls-files` so nothing uncommitted is ever offered, and a nested path becomes a flat name
because a release has no folders. The upload is multipart on the stdlib, with a random boundary -
a fixed one appearing inside a file would corrupt the upload.

**It cannot take a session down.** By the time it runs, the work is committed and on the server. A
release that fails to publish is a missing convenience, and the report says so - "the work is
committed and pushed; only the download page is missing" - rather than turning a session that did
its job into one that reports a failure.

**Result.** Two consecutive sessions against the same project: `v1` with `sample.txt` and
`wordcount.py`, then `v2` with `README.md` added - one release each, holding the files as that
session left them. `/archive/v2.zip` returns `Content-Disposition: attachment`.

**The 404 was never a bug.** The `vmlab` org is Limited visibility by design; the operator was
signed out. Confirmed with them, and **nothing was made public**.

### 59.34 — it spent a whole reply writing the job instead of sending it

**Symptom.** "Build a GUI PowerShell that acts as a Windows troubleshooting toolkit." The model
answered with four sentences of intent - *"I'm going to write it and then have it launched and
clicked through on a real Windows machine before handing it to you"* - and then **nothing
happened**. No tool call, no machine, no script.

**Cause, from the usage record:** `eval_count: 32768`, which is exactly the `max_tokens` set on
the model in Open WebUI. 32,768 output tokens produced 398 characters of visible content and
`tool_calls: []`. The model had spent its entire output budget writing the PowerShell GUI *into
the task argument* of the `vm_agent` call, hit the ceiling mid-argument, and the call was never
emitted at all.

The instruction told it to verify a script it had written. It read that as "write the script,
then hand the finished thing over" - which is both the wrong order and the wrong division of
labour, and at GUI scale it does not fit in one reply.

**Change.** The handoff moves to the front, in the docstring and in the prompt: when the user
asks for something that has to be built and run, `vm_agent` is the **opening move of the reply**.
Do not design it, do not write it, do not work out the approach - the agent does all of that on
the machine, where it can run what it writes. The `task` is the user's request restated plus its
constraints and a sentence on what "working" means; the `task` field now says **NOT an
implementation - do not write the code here, the agent writes it**, and the docstring carries the
measurement above as the reason.

**And a correction to this log.** 59.29 and 59.30 both say the system prompt is baked into the
Ollama model and therefore cannot be deployed from here. **That is wrong.** It lives in
`params.system` on the `qwen3.8:latest` row of Open WebUI's model table, is read per request
(`routers/ollama.py` calls `Models.get_model_by_id` on every completion), and takes effect on the
next message with no restart. The deployed copy had drifted 3,700 characters behind
`prompts/system-prompt.md` - every routing fix from 59.29 onward was live in the tool schema and
absent from the prompt. It is now deployed, byte-identical to the repo, with the previous version
kept. Diffed before writing: four lines reworded and fifty-one added, all inside the two
`vm_agent` sections, nothing else touched.

**Not verified:** the behaviour change itself. The browser tooling is disconnected, so I could
not put the prompt back through a chat and watch it hand off. The schema and the prompt are
confirmed live and correct; whether the model now calls first needs one message through the UI.

### 59.35 — four machines, four prompts, and a line that proves it is alive

**Symptom, and it is the one that gives the game away.** A **Windows** GUI session wrote itself
the task *"Check machine readiness: PowerShell version, display resolution, /mnt/project
accessibility"*. `/mnt/project` is the Linux path; on Windows the project folder is `Z:\`.

**Cause.** All four kinds of session - ubuntu/windows by cli/gui - were given the same prompt and
the same tool descriptions, which named both machines and left the agent to work out which half
was about it. `prompt.md` said "`/mnt/project` persists; everything else is destroyed", full
stop. The `run` tool said "a root bash login shell on Ubuntu and an elevated PowerShell on
Windows". This stack has three times proved the model reads tool text as **fact** rather than as
something to reason around, and that is exactly what it did.

**Change.** A session is now told about the machine it is in and about nothing else. The prompt
is assembled from `prompt.md` plus `prompt.{ubuntu,windows}.md` plus `prompt.{cli,gui}.md`, and
the tool descriptions are built from a `MACHINES` table. So:

| | project folder | `run` is | screen tools | lookup suite |
|---|---|---|---|---|
| ubuntu/cli | `/mnt/project` | a root bash login shell | no | yes |
| ubuntu/gui | `/mnt/project` | a root bash login shell | yes | no |
| windows/cli | `Z:\` | an elevated PowerShell 5.1 | no | yes |
| windows/gui | `Z:\` | an elevated PowerShell 5.1 | yes | no |

The Windows fragment carries the two facts that have each cost a session already: **5.1 has no
`&&` or `||`**, and you are already inside PowerShell so wrapping in `powershell.exe` starts a
second one. The CLI fragment says plainly that there is no screen and nothing can click. A
missing fragment is skipped rather than raised on - a thinner prompt still runs a session, and
refusing to start one over a missing file would be the worse failure.

**Result.** A Windows CLI session, re-run: 4/4 tasks, every task naming `Z:\`, **`/mnt/project`
absent from the entire report**, supervisor verification passed.

---

**The live view, as the operator drew it.** Three boxes in GUI, two in CLI:

```
Live agent output          Live agent output
Task List:                 Task List:
tasks                      tasks
-----------------          -----------------
                           -----------------
                           the machine's screen
                           -----------------
-----------------          -----------------
command feed (20)          command feed (10)
-----------------          -----------------
      CLI                        GUI
```

**The live line meant making the agent's calls stream.** They were non-streaming on purpose -
the agent acts on a finished message, so nothing in the loop needed tokens as they arrived. True
of the agent, false of the person: measured during this work, the progress file sat unchanged
for over a minute while the GPU ran at 83% and the guest at 69% of a core. Everything was fine
and the screen said nothing, which is indistinguishable from wedged.

So `chat()` takes an `on_text` callback. Without one it streams nothing and behaves exactly as
before; with one, the reply is read as it arrives and the assembled message is **identical in
shape**, because the agent is not allowed to care which path produced it. The callback belongs to
the watcher, so it is called inside a suppressed block: a display that can fail a turn is worse
than no display. Thinking is shown only while there is no content yet - a model that is thinking
has not said anything, and a blank minute reads as hung.

**And the GUI feed would have been blank rows.** Every tool a GUI session uses is a desktop verb
- `screenshot`, `click`, `key`, `type`, `scroll`, `move` - and **none of them had a renderer**,
so the third box would have shown ten empty lines for the whole session. They render now.

**Result, from a real GUI session:** the live line moving continuously through the model's own
words, `shot_seq` advancing 0-3, and the feed holding exactly 10 lines - half the CLI box, as
asked.

### 59.36 — it could not put a file down and come back for the rest

**Symptom.** A session spent **twenty-two minutes and roughly fifty thousand tokens inside one
turn**, designing a WPF application in its head, still on task one of twelve, having made no tool
call for twenty minutes. It was not stuck and it was not looping - the live line advanced every
second and never repeated.

**Cause, and it is not the model's discipline.** The agent had `write_file` and nothing else.
Every change to a file meant reproducing the whole file, so a file it had to produce in one call
was a file it had to design completely before it could write any of it. The 2 MB limit was a
resource limit, not a forcing function; there was nothing to make it stop and nothing to come
back to.

The operator's framing was the right one: *think about how your own tooling is set up*. And the
answer is that **I almost never write a whole file - I edit one.** Exact-string replacement, one
small change at a time, failing loudly when the target is missing or ambiguous. The agent had no
equivalent.

**Change.**

* **`edit_file(path, old, new, replace_all)`** - exact text, and it **refuses** when the text is
  absent or appears more than once. The refusal is the feature: text that is not there means a
  mistaken belief about the file, and being told now is far cheaper than a silent half-edit found
  three sections later.
* **`append_file(path, content)`** - build forward a section at a time.
* **`write_file` capped at 16 KB**, with a refusal that explains why and names the way past it.
  This is the forcing function; the cap message keeps the escape hatch for generated data.
* **`num_predict: 16384`** - there was no output ceiling at all, which is how one turn ran
  twenty-two minutes.
* The prompt now says: skeleton, parse-check, one section per turn, commit as you go, link it
  together last - with the parse-check command for each machine.

**Three bugs fell out of testing it, each found by using the thing rather than reading it.**

**1. `mkdir -p "$(dirname ...)"` sent to PowerShell.** `write_file` ran that on every guest, and
the result was discarded. On Windows it failed every time, so any nested path failed to write; a
top-level file worked because `Z:\` exists. Rewritten in PowerShell it failed AGAIN, differently:
**5.1 rejects `Split-Path -Parent -LiteralPath` as an ambiguous parameter set**, so `$p` was null,
`Test-Path` and `New-Item` both bound null and did nothing - and because those are
*non-terminating* errors PowerShell still exited 0. Measured from inside the machine:
`PARENT=[]`, `EXISTS_AFTER=`, exit 0, no directory. **An exit-code check could not have caught
it.** vmd now splits the path itself - it built the path, so it knows the separator - and
*confirms* the directory exists rather than trusting a status code.

**2. The first release on a new repository always failed.** `422 repo is empty`, moments after a
push that plainly succeeded: Gitea clears `is_empty` asynchronously and the release API reads that
flag. It never showed on a repository that already had commits, which is why it survived two
sessions. Now retried with a short backoff.

**3. Answering the agent's own question was rejected.** The tool tells the model to call back with
`session_id` and `answer`; the schema required `task`, `os` and `interface` on every call, so the
reply came back *"task: Field required"*. They are optional in the schema now and required where a
new session actually needs them, with a refusal naming which is missing.

**Result.** The whole file-tool path driven on Windows, with `run` forbidden for the writes:
nested `write_file` with no directory pre-created, `append_file`, `edit_file`, the ambiguous-text
refusal verbatim, `read_file` - all correct, first attempt, and a release published on a
brand-new repository.

**Still open, found and not fixed:** there is **no resume path**. `session_id` is documented,
accepted, and used by nothing - `runner.py` never reads it, so an `answer` arrives as a fresh job
against a machine that is already running. `ask_operator` therefore strands a session. Recorded
rather than patched under time pressure.

### 59.37 — one kind of session, because the choice was never the caller's to make

**Symptom.** Asked to "build a gui powershell that acts as a windows troubleshooting toolkit",
the model chose `interface: cli` - a session with no screen, which can write a desktop
application and confirm it parses and can never say whether the window opened. Told not to, in
the tool docstring AND in the system prompt, it did it again on the very next run.

**Cause, eventually.** I built a refusal to stop it - `needs_a_screen` in `vm_preflight`, narrow,
tested, and it worked. Then the operator pointed out the obvious thing I had walked past: the
choice did not need to exist. I had written a mechanism to defend a decision rather than removing
the decision.

**Three separate costs, all measured, all from the same split:**

* **The caller chose wrong**, twice, with the rule in front of it both times.
* **A `gui` session had no cheap way to look anything up.** The lookup suite was CLI-only, on the
  reasoning that a desktop session could read anything by opening it in the browser - true, and
  far too expensive: `open_page` costs a screenshot per scroll where `read_page` costs one call.
  So it reasoned instead. Eleven minutes and eight context compactions deriving how to run
  background work in PowerShell, which the documentation states outright, while a parse-clean
  19 KB application it had already written sat untouched. Another spent twelve minutes writing
  nothing at all, the live line reading *"I recall precisely: WinForms processes docking in
  reverse z-order"*.
* **A `cli` session could build a window and never see it**, so "it parses" was the best answer
  available to it.

**Change.** There is one kind of session. Every one gets the command line, the lookup suite and
the screen; the agent decides what it needs, which it is far better placed to know than the
model that called it. `interface` is gone from the tool schema entirely - one less required
parameter, one less thing to get wrong - and `needs_a_screen` went with it.

The rest follows:

* **The desktop is waited for lazily**, on the first screen verb, so a job that only runs
  commands never pays for GNOME finishing its paint. That was the whole argument for having a
  cheap mode, and it survives without the split.
* **No screenshot is taken at handover.** It used to attach one automatically, which put a
  picture of an idle desktop in the chat before the session had done anything, for every job
  including the ones that never look at a screen. **The preview box now appears only when the
  agent takes a screenshot**, so its arrival means something.
* **One terminal height** (20 lines). It was halved when a screen was shown; the screen is now
  the exception rather than the rule, so the feed is not shrunk for something that may never
  arrive.
* **One step budget**, the generous one (110), because any session may now look at a screen and
  seeing the result of an action is its own turn.
* Two prompt fragments instead of four: `prompt.{ubuntu,windows}.md` plus one `prompt.screen.md`
  that says the screen is there and that most work does not need it.

**The lookup suite in every session is the operator's suggestion**, and it is the one that
addresses the failure directly: a session that cannot cheaply look something up will reason about
it instead, and reasoning is what was eating the sessions.

**Result.** `tests/test_vm_four_sessions.py` became `test_vm_two_machines.py`. Thirteen suites
pass. The run under way when this was written: 6 of 10 tasks, a 32 KB application built with
`write_file` then `edit_file`, launching its GUI - the best of six attempts at this prompt.

### 59.38 — tuning it by running it, and the one parameter that was wrong

Two campaigns, driven through the real chat rather than the API, because the API returns the
final message and nothing else - the dialog, the live line, the boxes and their heights are all
invisible to it, and every defect found in this round was a visual one.

**Windows: "build a gui powershell that essentially acts as a windows troubleshooting toolkit".**
Eight attempts. The interesting ones:

| | outcome |
|---|---|
| before this round | 0/12 tasks, no tool call for 20 minutes, one turn of ~50,000 tokens |
| WinForms, old tooling | 7/10, app built and launched, all tabs screenshotted |
| WPF/XAML, old tooling | **1/12**, 48 minutes on the skeleton, 22 compactions |
| WinForms, tuned | **7/9**, 365-line 7-tab app, README, parse-checked, launched, *window confirmed rendering with all tabs* |

The middle row is the useful one: same code, same prompt, and it failed because the agent chose
XAML over WinForms. That is approach variance, not a tooling fault - and it is what made the next
change worth making.

**The agent's `presence_penalty` was 1.0, copied from the main assistant's row.** That is right
for the main model, which writes prose: a penalty on tokens already used discourages repetitive
phrasing. It is wrong for this agent, which writes **code**. A 25 KB script says `$form`,
`$output` and its own function names over and over - that repetition IS the program - and a
penalty on it is steady pressure toward a different token than the correct one. Two failing
sessions stalled on exactly that shape of bug: *"a scope issue with how I'm accessing the log"*,
and *"it's using `$a[0]` when `$a` was never actually defined"*. Qwen3's own guidance says a
presence penalty may degrade output and to use it only against repetition loops - and the agent
already has a repeat guard for those, in tool output, where it can say why.

Set to **0 for the agent only**. The main model's row is untouched: `temperature 0.6`,
`max_tokens 32768`, `presence_penalty 1`.

**And the task list had gone flat.** A session sat on "1/12" for twenty minutes while the file
grew and `edit_file` calls landed. The work was fine; the *report* of it was a flat line, and
somebody watching would read that as stuck. The prompt already asks the agent to mark progress
and asking has not worked, so it is now a nudge in tool output after twelve steps with no task
movement: *"Whoever asked for this is watching that list and it still says you are where you
started."* A nudge and not a refusal - being mid-task is legitimate, and blocking on it would be
wrong far more often than right. Measured on the next run: the agent went 0 to 5 of 8 in one
step after it fired, and the nudge is visible in the command feed.

**Ubuntu: "watches a folder and turns any new png into a compressed jpeg, keeping the original".**
**7 of 7, first attempt, status Done** - and it never took a single screenshot, which is the
one-session change working exactly as intended: the agent decided it did not need the screen, so
the preview pane never opened. The evidence file it left behind is the point of the whole
subsystem:

```
PNG sha256 (before): 77dac1f6...97ba
PNG sha256 (after):  77dac1f6...97ba
demo/photo.png: 1157239 bytes    demo/photo.jpg: 242441 bytes
JPEG smaller than PNG : YES      PNG unchanged : YES
```

**What is still hard, stated plainly.** Background/async work in PowerShell defeated the agent in
three separate sessions - runspaces, threads, `Start-Job` - and it is the only thing that did.
The scope rule tells it to take the duller approach after two failures and it does not always
listen, because that is a prompt rule. Making it a tool-output signal is the obvious next move
and is not done.

### 59.39 — the sweep, and the bug it found in the last place anyone looks

A final pass over everything, driven through the chat. It found one real defect, and it was in
the step that ends a session.

**`finish` was refused with an internal error, every time.** A session wrote the script,
syntax-checked it, built a fourteen-file test tree with awkward filenames, cross-checked its
output against an independent listing, demonstrated the error paths, committed, and passed its own
verification command - and then could not close:

```
File "/usr/local/lib/owuivm/vmd.py", line 1233, in v_finish
AttributeError: 'str' object has no attribute 'get'
```

`finish` declares `verification` as `{command, expected}`; the model sends the command as a bare
string often enough to matter. The agent saw only "the supervisor refuses the call", tried seven
times with varying arguments, marked the task blocked and gave up. **All the work was done and
committed. The session simply could not be ended.**

The identical mistake had already been found and fixed in the live view's renderer (59.36), where
it cost a line of output. Nobody checked the verb that reads the same argument. The rule is the
same wherever a model-supplied value is read: **take what arrives, not what was asked for.**
`tests/test_vm_finish_shapes.py` covers both shapes and four kinds of junk.

Re-run after the fix, same prompt: **8 of 8 tasks, status Done, four minutes.**

**Also seen, and fixed in the prompt rather than mechanically:** the agent built its test tree
*inside* the project folder, so two 256 MB fixtures were committed and published. They were sparse
and compressed to nothing - `.git` came to 566 KB - and the release asset cap skipped them, so
nothing broke. On real data it would be a quarter-gigabyte blob in somebody's repository. The
prompt now says scratch belongs in `/tmp` and the project folder is for what was asked for plus
the evidence worth keeping.

**Final state of the sweep:** fourteen suites pass; `owuivm` inactive, no qemu, no overlays, no
pending jobs; both base images `0444`; every container healthy. The live view was re-checked in
the browser on a fresh session - two boxes before any screenshot, the live line above the task
list, feed at full height, nothing stuck at the 150px default, no horizontal scroll - and the
confirmation dialog fits its window with the buttons reachable.

---

**Also measured:** boot time fell from 2min 14s to 20.9s once `systemd-networkd-wait-online`
was masked (`graphical.target` at 9.5s rather than 2min 3s); a session opens and answers on its
guest agent in ~27s; a click at an absolute coordinate landed on the button it was aimed at and
advanced a GNOME dialog; two sessions on the same project name saw the first session's files.

**Not measured:** the cost of an Ollama handover. The design assumes a cold prefill of the main
model's context on every return, which is why one long blocking call is preferred to polling,
but no benchmark has been run on this V100 — the figure used to reason about it is an estimate.
The Windows guest now exists and has been measured
(59.26, 59.28): a session opens on it, and it resumes from a saved machine rather than booting.
The live-progress channel *has* now been watched end to end from a browser - the filter is installed,
and the two boxes were measured at 86px and 364px during a running session (59.30). GUI mode has
been proven verb by verb - screenshot, absolute click, key, type - but never driven by the model.

### 59.40 — the release carried the debris and not the work, and the prompts said things that were no longer true

**Symptom, in the operator's words.** *"When a release is built in gitea it only has the files in it
relevant to the actual release and not include any ambiguous testing tools / scripts"* — and
*"clean up the prompting where needed to avoid confusion."*

**Measured.** Release `v3` of `windows-troubleshooting-toolkit` held ten files — `_build.ps1`,
`_launch.bat`, `_structcheck.ps1`, four `_w_*.ps1` fragments, `check_disp.ps1`, `check_disp2.ps1`,
`check_disp3.ps1` — every one of them scratch, and **not** the 34 KB `TroubleshootingToolkit.ps1`
the session was asked for. Two causes compounding: `git add -A` committed whatever the agent had
left in the folder (30 files, 25 of them probes, captured output and a compiled `test_cs.exe`), and
`tracked_files()` sorted `_build` ahead of `TroubleshootingToolkit` and stopped at ten. Three other
releases shipped `test_*.ps1` beside the deliverable; `top-ten-files` had 1.5 GB of `testtree/`
fixtures committed.

**What actually downloads, measured again.** Uploaded release assets come back
`Content-Disposition: inline` — a `.zip` asset included, `application/octet-stream; inline`. The
only attachment on a release page is Gitea's own `/archive/<tag>.zip`, built from the tree the tag
points at. So a curated asset list would have left the one link that downloads full of junk: **a
release is only as clean as the commit behind it**, and the filter had to go at the commit.

**Change — the commit.** `gitea.py` gains `SCRATCH_PATTERNS`, applied as `:(exclude,glob)`
pathspecs to `git add -A`: a leading underscore, `test_*`/`test-*`/`*_test.*`, `smoke.*`, `probe_*`,
`tmp_*`, `*_out.txt`, `*.out`, build products (`*.exe`, `*.dll`, `*.obj`, `*.o`, `*.pdb`, `*.class`),
and whole trees under `test*/`, `tmp/`, `scratch/`. Deliberately narrow: `check_*` and `find_*` are
*not* excluded, because `find_duplicates.py` was a real deliverable and a "check disk health" job
would reasonably produce `check_disk.py`. Three guards on the filter:

1. A project that would otherwise publish *nothing* — everything reads as scratch and nothing
   publishable was ever committed — publishes everything, because then the tests were the job.
   The freshly written `.gitignore` is discounted when deciding that, or it alone made an
   all-tests folder look like it had something publishable in it.
2. Scratch an *earlier* session already committed is untracked (`git rm --cached`) on the next
   commit, so a resumed project stops publishing its old junk without losing the file on the array.
3. `git_commit` names what it held back, in tool output, with "if this *is* the deliverable, give it
   a name that says so". A filter nobody mentions is indistinguishable from a bug.

`has_changes()` and `commit()` share one `_staging_plan()`, or the end-of-session backstop decides
there is something to commit and then stages nothing, which git calls an error. The asset cap goes
from 10 to 25: it exists to stop a file dump, and the dump no longer reaches the commit.

**Change — the stranded session.** `ps-troubleshoot-toolkit` had a 20 KB toolkit and five test
scripts on the array, zero commits, no repository, no link. Its machine died before the loop could
call `session_close` (the agent container was rebuilt underneath it), and the only path that then
saw the record — `session_open`'s stale-record cleanup — destroyed it without publishing. The reply
the user got carried a confident link to **a different project's** release, assembled from the task
wording. Now the stale-record path runs `_publish` before it discards the record; the report states
"There is no link to give for this session" as a fact when nothing was published; and the main
prompt and the tool docstring both say to hand over only the link in the result and never build one
from the project name.

**Change — the prompts.** `prompt.md` promised *"in a desktop session you will be handed a picture
of it"* — the handover screenshot had been removed a change earlier. "In either mode" survived the
removal of the modes. The lookup guidance was written twice, `read_page`/`open_page` explained
twice, "GUI included" in one of them. "Nothing graphical you start from `run` can be seen" was said
in four files. `/tmp` sat in the machine-neutral prompt — the same shape of mistake as the Windows
session that planned to "check /mnt/project accessibility". All of that is now said once, in the
file it belongs to; the scratch location lives in each machine's own section; the commit filter is
described where the project folder is. Stale CLI/GUI comments in `tools.py`, `vmd.py`, `websearch.py`
and `compact.py`, and the filter's valve descriptions ("In CLI sessions…"), were rewritten.

**Tests.** `tests/test_vm_release_contents.py` rebuilds the failed release from its real file list
and asserts the toolkit is committed and the scratch is not; that `find_duplicates.py`,
`Check-AutoStartServices.ps1` and `latest/` survive; that an all-tests project still publishes; and
that a resumed project un-publishes legacy scratch while keeping the file. `tests/test_vm_prompt_consistency.py`
reads the text the two models actually see — prompts, tool *descriptions* (not comments), the tool
docstring, the report — and fails on mode language, the handover-picture promise, a machine path in
the neutral prompt, a tool name the agent does not have, and the same sentence said in four files.
17 suites pass.

**Dry run against the array, read-only.** Every real project: `ps-troubleshoot-toolkit` would
publish `Toolkit.ps1` and hold back its five `test_*.ps1`; `top-ten-files` would un-publish 1.5 GB
of `testtree/`; `windows-troubleshooting-toolkit` would un-publish all 25 scratch files and keep the
toolkit. One judgement call flagged to the operator: `systemd-restart-test`'s `units/test-*.service`
are fixtures by name and arguably the artifact; they would stop being published, leaving `REPORT.md`.

**Also found.** Open WebUI 0.11.3 with no Redis configured refetches every tool server's OpenAPI
spec on each chat request (`get_tool_servers` → `set_tool_servers`), so a changed tool docstring is
live as soon as the api container is up — no re-save, no restart. And the `vmagent` bot token has
write but not delete rights: the probe repository `vmlab/zip-asset-probe` used to measure the
download headers could not be removed by the API and needs deleting from the web UI.

### 59.41 — a live line you can read, and three boxes you can fold away

**Asked for.** Smooth the live token stream above the task list so it arrives token by token
and its speed shows how fast generation actually is. Make each box fold when clicked: the task
list down to two lines (the live line and the current step), the screen preview to half size
with black either side, the command feed to half its height. Click again to restore. No tooltip.

**The problem underneath all of it.** The view is re-sent every couple of seconds, and each send
replaces the frame's `srcdoc` — a new Document every time. A click has to outlive that, and the
frame is sandboxed without `allow-same-origin`, so it has an **opaque origin**: `localStorage`
and cookies throw, IndexedDB is useless, and Open WebUI has no listener to `postMessage` to.
There is no server channel either — the filter polls, nothing calls back.

**Measured, in that exact sandbox: `window.name` survives.** It belongs to the browsing *context*,
not the document, and the context is what the swap keeps. A frame reloaded three times read back
the name its two predecessors had written. That is the whole mechanism: each box keeps
`{k, c, d}` there — which box it is, whether it is folded, and the text the ticker has shown so
far. It is keyed by box because the screen box appears mid-session and pushes the feed down a
slot, so the frame that was the feed becomes the screen and would otherwise inherit its state.

**The ticker.** The server sends the *tail* of what the model is saying, at most once a second,
so the line used to jump a clause at a time — motion that said nothing about throughput. The
frame now computes the overlap between what it has already shown and the new tail, and appends
only the remainder, a few characters per 40 ms frame, draining over ~1.5 s. More characters in
the window means faster typing, so the speed on screen *is* the token rate. It is right-anchored
(`justify-content:flex-end`), because the end of the sentence is the part that is moving.

**`flex:none` on the feed rows is load-bearing.** The rows sit in a flex column so a folded feed
keeps its newest lines against the bottom and loses the oldest off the top. Flex items shrink by
default, so the first version compressed twenty rows into ten rows of space and the text
overlapped itself — caught by eye in the harness, not by any test.

**What the browser could not tell me, and how that was worked around.** Dispatched clicks from
the automation tool **never reach iframe content** — proven with a page holding one sandboxed and
one plain frame, each with a click handler: the parent's own button fired, neither frame did. So
interaction was verified by replaying the real generated frames into non-sandboxed iframes and
dispatching clicks through the DOM, while `window.name` persistence was measured separately in a
real sandboxed frame. Both halves measured, in a browser, before the code was believed.

**Result**, in a real frame: task box 196px → 51px and back; screen 500px → 269px with the image
at exactly half its width (740 → 370) on black; feed 364px → 190px showing ten rows at an
unchanged 17.4px. All three survived two srcdoc swaps in the folded state.

`tests/test_vm_live_view_collapse.py` locks the contract — the per-box config block, the three
ids each box needs, the `window.name` keying, the toggle re-reporting its height, the overlap
arithmetic, and `flex:none` on every row. 18 suites pass.

**Note for whoever changes this next:** a running watcher holds the module it was started with,
so a filter deployed mid-session does not reach the session already being watched. The new view
appears on the next turn, not on the one in flight.

### 59.42 — the transcript was always there, and was thrown away every time

**Symptom.** Asked "did the agent ever open the app and look at it", I could not answer from the
record, and answered about the wrong run while doing it. The operator's reply: *"You're definitely
not looking at the logs correctly ... I want you to set up proper logging so that its easy to go
back and see exactly what the LLM did without getting confused."*

**Cause, and it is not the one it looks like.** Nothing was missing from the *recording*.
`_record()` has covered `screenshot`, `click`, `key` and `type` since they were written, and
`STATE.transcript` held the lot. It was discarded when the session ended, so the only durable
records were the two that cannot answer the question:

* **the tool result counted `run` and nothing else.** A session that marked "clicked through the
  GUI" done without ever calling `click` read exactly like one that did.
* **Open WebUI's status history is SAMPLED.** The filter polls every two seconds and reads
  whatever the status says at that instant, so a verb that returns fast is usually overwritten
  before it is ever seen. Measured on a real session: 456 samples caught `launch_app` nine times,
  because it blocks for seconds, and caught **none** of the screenshots that demonstrably
  happened. Reasoning from that record is reasoning from a record that cannot be complete.

**Change.** `vm/vmd/sessionlog.py` writes the transcript to `/var/lib/owuivm/sessions/` as it
happens, one JSONL file per session, flushed on every line - the session someone wants the log for
is precisely the one that got killed. One hook in `_record()` covers every verb, including any
added later. It carries the **chat id**, so a conversation maps to exactly one session and nobody
has to guess from timestamps again. A harness-only `note` verb records what the model *said* each
turn, so the file reads as an account rather than a list of calls; the model is never told it
exists. `vmctl logs` lists them and `vmctl log [chat|session|file|project]` renders one, neither
needing a daemon or a machine. The last four verbs that were doing their work unrecorded -
`read_file`, `list_dir`, `scroll`, `wait`, `check_internet` - now record too.

The **tool result** carries the same counts, so the reader sees them without opening anything, and
when the screen was never looked at it says so: *"Any claim about how something appears is
inference, not observation."*

**The bug this found immediately, which is the interesting part.** The first real session wrote no
log at all, silently. vmd runs under `ProtectSystem=strict` and `/var/lib/owuivm/sessions` was not
in `ReadWritePaths`, so every write was denied - and the log is built never to break a session, so
it swallowed all of it and looked exactly like a session with nothing to record. **A feature that
silently does nothing is worse than one that fails loudly.** The directory is now in the unit, and
a log that cannot be opened prints one line to the journal saying why while still returning
quietly.

**Result**, a real hand-driven session read back with `sudo vmctl log`:

```
  0:58.5  ready          booted in 57.7s
  1:11.7  plan           2 tasks
  1:13.0  run            exit 0 1.06s  uname -a; echo hello from the log check
  1:17.4  screenshot     1280x800  (390 KB)
  1:17.6  ·  turn 1: Checking that the session log records what actually happened.

what it actually did: 2x task, 1x check_internet, 1x plan, 1x ready, 1x run, 1x screenshot
  it never called click
```

`tests/test_vm_session_log.py` walks the AST and fails if any verb touching the machine records
nothing, following the one level of delegation that `v_click` and `v_move` use. 19 suites pass.

### 59.43 — the run was fine; the window onto it closed four seconds in

**Symptom, in the operator's words:** *"the tool just gave up it seems like."*

**What had actually happened.** A conversation asked for a GUI app, got one, then asked for a fix
to the layout. The second `vm_agent` call ran for **45 minutes, closed all eight tasks and
finished successfully** - and the chat showed an empty message under a toast reading "The machine
has been destroyed". The whole status history of that message was seven lines:

```
Calling vm_agent / VM · Starting a machine… / VM · Done / The machine has been destroyed.
```

**Cause.** The live-progress file is named after the **chat**, not the run:
`/vmjobs/progress/<chat_id>.json`. The first run left it with `done: true`. The second run's
watcher started, polled once, read the *previous* run's ending, announced it and exited. The run
then wrote to that same file for three quarters of an hour, reaching sequence 1035, with nobody
reading it. Every second `vm_agent` call in a conversation had this, and the first run in a fresh
chat never did - which is why it survived so long.

**Change, in two places, because a race with one guard is a race.** The api deletes the previous
run's progress file (and any stale confirmation answer, which would otherwise decide a question
nobody was asked) **before** the job is queued, which is authoritative. The watcher additionally
ignores any progress stamped earlier than the moment it began - `updated` is written on every
flush, so a file from the last run is simply older than the watcher reading it.

**The other thing that session did, found in the new log.** It spent its last dozen turns fighting
`git` inside the machine: `exit 128`, "dubious ownership", `safe.directory`, `HOME`. It even wrote
*"Let me use the git_commit tool"* at turn 99 and then went back to raw git. The guest sees `.git`
owned by another uid, so this can only ever end in that wall - and a commit that did succeed there
would skip everything `gitea.py` does on the host: hooks and filters pinned off on the command
line, the right author, and scratch left out of what gets published.

So `run` now refuses a mutating git subcommand and says why, naming `git_commit`. `git` has to be
in **command position** for the refusal to fire, or `echo git commit > notes.txt` reads as a
commit - a refusal that fires on a string inside an echo is worse than no refusal. It is a
signpost, not a sandbox: a guest that insists can still write to `.git`, and what protects the
published history is the host-side hardening, not this.

**Verified** against the real command lines from the failed session - `git add -A`,
`git config --global --add safe.directory`, `git -c safe.directory=... add`, `git -C ... commit`,
`/usr/bin/git push` - all refused; `git log`, `git status`, a quoted string in `python3 -c`, a
`grep` pattern and an `echo` into a file all untouched. The stale-file case was replayed in the
api container: a planted `done: true` from ten minutes ago is deleted by the submit path and
ignored by the watcher's stamp. `tests/test_vm_second_run.py`; 20 suites pass.

**The general lesson, which has now cost two debugging sessions:** state keyed by conversation
outlives the thing it describes. The session log (59.42) is keyed by session and carries the chat
id, which is the right way round - and it is what made this diagnosable at all, since it showed
the run working normally throughout the window in which the UI said it had died.

---
## Change 60 — one place to configure it, and a switch that actually removes a tool

**Symptom.** Nothing was broken; it was unpublishable. Configuration lived on four surfaces
and only one of them was in the repo: `.env`; `/etc/owuivm/vmd.env`, written once by
`install.sh` and never looked at again; bind-mounted service config scattered across six
top-level directories; and state that existed only inside Open WebUI — the system prompt, the
filter valves, and the Function Name Filter List, which was the *only* way to turn a tool off.
"What does this deployment offer, and where do I change it" had no answer on disk.

The tree also carried one house's identity in ~60 places, including the operator's username as
a **functional gate** in the polkit rule, and one live secret committed in the clear
(`searxng/settings.yml`).

**Change.** Everything editable moved to `configs/`: the stack env, a tool switch, the machine
registry, the assistant's system prompt, and the sub-agent's prompts, parameters and tool
definitions. `docs/` gained installation, configuration, per-tool setup pages, the recommended
parameters, and sanitised compose files for the three services that live outside the repo
(Ollama, Open WebUI, Gitea). The testing corpus moved under `docs/testing/`.

Four things are worth recording beyond the reorganisation.

**`configs/tools.toml` removes a tool rather than hiding it.** A tool set to `false` is dropped
before `include_router`, so it is absent from the OpenAPI spec *and* its endpoint 404s. The 22
schemas cost ~9,700 tokens on every turn of every conversation, so hiding one saves nothing;
removing it is the only version of the feature worth having. Missing or unparseable config
leaves everything on, deliberately: this is config that removes capability, and its failure
mode should be the system it replaced.

**The seam is the routers, not `app.routes`, and the first version of this shipped that bug.**
FastAPI 0.141 stopped flattening included routers — `include_router()` appends one opaque
`_IncludedRouter`, so `app.routes` contains no `APIRoute` and no `operation_id` at all.
Filtering it removed nothing while looking exactly as though it had worked: 22 tools in, 22
tools out, and a log line claiming 22 unknown names. Caught only because the probe asserted
*which* tools disappeared rather than that the call succeeded.

**The sub-agent's 34 tool schemas moved from Python to `configs/agent/tools.toml`** — ~380
lines of tuned prompt text, where a dropped sentence is not a crash but a session that goes
wrong later for no visible reason. The move was made safe by capturing the hand-written output
to `tests/golden/agent_tool_schemas.json` **before** touching anything, then asserting the
config-driven builder reproduces it byte-for-byte for both machines and for the booting case.
It does. The per-machine variation (`{shell}`, `{project}`, `{example}`) is substituted over
the serialised form, because the placeholders appear in parameter descriptions as well as tool
descriptions. Dispatch stayed in code behind a `DISPATCHABLE` allowlist, so a config file
cannot operator the agent a verb — a name with nothing behind it is refused at read time, naming
the file, rather than offered to the model and failing somewhere less obvious.

**Two QOL changes to the maintenance app, and the privileged half of one is the interesting
part.** *Insert disc…* replaces the dead end where a machine with no media said "no ubuntu ISO
in /var/lib/owuivm/iso" with a greyed-out button. The obvious implementation — pass the chosen
path to a privileged unit and let it copy — does not work and should not: the maintenance units
run `ProtectHome=yes` so the unit cannot see `~/Downloads`, `owuivmadm` is not in the operator's
group so it could not read a 0750 home anyway, and running the copy as root would hand the
operator a "copy any file root can read into a world-readable directory" primitive in exchange
for saving one `mv`. So the app copies as the operator into a staging directory the operator
owns, and the unit checks it and moves it. The check includes the ISO 9660 signature at offset
0x8001, which catches the case this will actually meet: a download that was really an HTML
error page, which would otherwise fail much later as a machine that will not boot.

*Rename* changes the label and never the id, which names the disk, the unit instance, the
saved-machine directory, the `os` the assistant passes, and appears in regex allowlists inside
the privileged launch scripts. It writes a small override file rather than editing
`configs/vms.toml`, because that file is mostly comments and no TOML writer preserves them — a
program that edits the registry in place is one that deletes its own documentation the first
time someone presses Rename.

**Result.** 22 suites pass (20 existing, unchanged, plus the toggle and golden-schema tests);
`tests/smoke.sh` at 174/175 before a stale case was fixed and 175/175 after; the host installer
re-run clean with the Gitea token preserved; all ten containers healthy.

**Two things this turned up that were not part of the work.**

*A smoke case had been silently starting a real VM session.* It passed `interface:
"telepathy"` with a valid project and expected a refusal — true when `interface` was validated,
but the cli/gui split was removed in 59.x and `vm.py` now hardcodes it. The nonsense value is
simply ignored, so the request became a perfectly good one, started a machine, and died on the
90-second per-case timeout with a transport error. The suite's own comment says only refusal
paths belong in it. It now pairs the bad interface with a project name that is refused outright,
so it asserts both things and cannot start anything. The session itself was cancelled correctly
by the disconnect path and left nothing behind, which is why this had gone unnoticed.

*Recreating a network-namespace owner strands its dependents.* `docker compose up -d netguard`
left `api` and `searxng` running, healthy by their own healthchecks — which are loopback calls —
and with no route to anything: squid and valkey both "Network is unreachable", the gateway
getting connection refused, and every container reporting Up. Docker does not re-attach a
`network_mode: container:` member when its host is replaced. Written into `docker-compose.yml`
beside both guards, because the failure reports as healthy.

**Measured:** the golden schemas are byte-identical across both machines; a disabled tool is
absent from `/openapi.json` and 404s; a missing config leaves all 22 on; the insert helper
rejects a non-ISO, a symlink, a path-shaped id and an unknown machine; the rename unit writes
only the override file. **Not measured:** the maintenance app's two new buttons under a real
polkit prompt — the units were driven directly, and the window was confirmed to build and run,
but the operator has not yet clicked through an install from a freshly inserted disc.

---
## Change 61 — portable enough to hand to someone else, and one place to read the logs

**Symptom.** Change 60 made the repository publishable. It did not make it *usable*: a
stranger who cloned it could not have got it running, and several of the things that would
have stopped them were invisible from inside this deployment, where they happen to be true.

**Cause, in four parts.**

*The pool.* `/Pool` was written into eleven functional files — three vmd modules, five host
scripts, three systemd units. `install.sh` had a `$POOL_ROOT` variable by then, and it
reached almost none of them.

*The machine's shape.* `AllowedCPUs=20-27`, `MemoryMax=11G`, `CPUQuota=700%` and a Gitea
allowlist naming one LAN address, all baked into units. Meaningless or actively wrong
anywhere else; `AllowedCPUs` on a four-core machine would refuse to start.

*Nothing checked.* `install.sh` created users, datasets, units and firewall rules and then
found out at step nine whether `swtpm` existed, leaving a half-installed host.

*And the documentation was confidently wrong in two places.* Both only discoverable by
doing what it said on a machine that was not this one.

**Change.** Every host artefact became a template with `@PLACEHOLDER@` tokens, rendered by
one `render()` in `install.sh` from values derived from the hardware — `nproc`, `MemTotal`
— and overridable in `configs/host.env`. `render()` **refuses to install a file that still
contains a placeholder**, so a token added to a template and not to the installer fails at
install time rather than as a unit that will not start. The pool reaches vmd through
`vmd.env`, which the daemon already loads.

A preflight now checks every binary, the firmware files, `/dev/kvm` and the pool *before
anything is created*, and creates the two datasets if they are missing. `tests/test_vm_host_render.sh`
renders the whole set with deliberately foreign values (`/tank`, `6G`, `300%`, `someone`)
and asserts no line that systemd or bash acts on still carries ours — comments excluded,
because the measurements in them are the reason anyone could change a number safely.

Also made settings rather than facts: the Open WebUI network name, and the URL in the
outbound `User-Agent` — which every other deployment would otherwise have been announcing
as its own to every site it fetched from.

**Logs.** `./logs` is bind-mounted into all ten containers and each writes its own file:
thirteen of them, stdout unchanged so `docker compose logs` still works. Two needed more
than a config line. squid and valkey create their logs `0640`/`0600` as their own uid, so
the operator could not read them — a log nobody can open is not an audit trail; squid takes
`umask 022`, valkey needed `sh -c 'umask 022; exec …'`, with `exec` so it stays PID 1 and
shuts down cleanly. SearXNG can do neither: it logs through Python to stdout and the image
dropped uwsgi, so its entrypoint is teed. That makes PID 1 a shell that will not forward
SIGTERM through a pipeline, so stopping it waits out the timeout — acceptable there and
nowhere else in the file, because it holds no state. `owuivmd` stays on the journal: it is
a systemd service, and pointing it at this directory would tie a host daemon to wherever
the repository sits.

**Result.** The published copy was set up in a temp directory following only
`docs/install-ubuntu-desktop.md`: ten containers healthy, all thirteen logs written and
readable, **175/175 smoke**, 22 unit suites, and the documented toggle workflow verified by
switching two tools off and watching them leave the spec and 404. The live deployment was
stopped for it and came back identical — and `install.sh`'s output diffed byte-for-byte
against the nine files it had installed before the refactor, the only difference being
three lines of comment.

**Four things this found that inspection had not.**

*The release shipped this machine's logs.* `logs/*.log` was added to the exclude list and
they stayed anyway: `rsync --delete` treats excluded files at the destination as protected,
so a file published once stays published after it is excluded. Now removed explicitly.

*A fresh clone could not start at all.* `.env` is a symlink to `configs/owuitools.env` and
was excluded as though it were the secret. It holds nothing, and it is the only thing
`docker compose` reads for substitution — so every documented command failed on the first
required variable. The link ships; `make-release.py` asserts it is a symlink and that it
dangles.

*`curl http://localhost:8000/health` was in seven documents and works nowhere.* Nothing
publishes a port. Worse, it can appear to succeed: on this host something unrelated answers
on 8000 with `OK`, which reads exactly like a healthy tool server. Replaced with
`tests/try-tool.sh`, which calls from inside the container.

*Driving a session by hand did not work, and the documentation said it did.* `ready` is set
by the `session_ready` verb and nothing else, and every verb touching the machine is refused
until it is — but `vmctl` never exposed it. `vmctl open` then `vmctl run` returned "the
machine is still booting" for as long as you cared to wait, with the guest up and the agent
answering `guest-ping` the whole time. `vmctl ready` now exists.

**Measured:** a clean clone reaches 175/175 with no edits beyond the two the documentation
asks for; the host templates render with no trace of this machine in any acted-on line and
`systemd-analyze` accepts the results; all thirteen logs are written and operator-readable.
**Not measured:** `install.sh` on a host that is not this one. Its preflight and its
derivations are exercised, but the only end-to-end proof available here is that re-running
it reproduces what was already installed.

---
## Change 62 — a last read of the thing being published

**Symptom.** Nothing was failing. But everything verified so far had been verified *on the
machine that wrote it*, and the one full install test had been done before several later
edits. A pre-push sweep found four things, three of which only matter to someone who does
not already have this working.

**What it found.**

*Following the install guide in order would have failed at step 3.* Compose derives a
project name from the directory the compose file sits in, so
`docker compose -f docs/deployment/open-webui.compose.yml up -d` creates a network called
`deployment_default`. The tool server joins Open WebUI's network **by name**, expecting
`openwebui_default` — so the next step dies with "network declared as external, but could
not be found". Invisible here, because a real `openwebui_default` already existed. All three
deployment files now pin `name:`.

*`vmctl gitea-check` does not exist.* It was written into `docs/setup/gitea.md` as the way
to verify the Gitea setup and was never a verb. Replaced with a check that runs as
`owuivmd` and asks Gitea for the organisation — which proves the token, the file permissions
*and* the daemon's address filter in one go, and was run here to confirm it works.

*The compose header said `WHY SIX CONTAINERS`* over a file with ten, and claimed no host
bind mounts beyond read-only config — untrue since `./logs` and the agent socket directory.
Both rewritten.

*Two comments pointed at the pre-`configs/` layout*, naming `gateway/nginx.conf` and
`egress/squid.conf`.

**How it was checked.** Mechanically where possible, because reading does not scale: every
tracked file compiled or parsed by shebang; every repo path named in a doc resolved (115 of
them, the only absences being files the user creates); every tool name in the docs
cross-checked against `configs/tools.toml`; every `vmctl` verb against `vmctl`'s own
parsers; every `OWUI*_` variable against the code. That last set is what caught the
fabricated command — a verb that reads plausibly is invisible to a human re-reading their
own prose.

**Result.** 175/175 smoke, 22 unit suites, the isolation and render suites, and the
published tree re-validated from a clean copy: ten services resolve, every bind-mount source
exists, all three deployment files parse, and its own tests pass. Two `example.com` strings
remain in the repository and both are the image-source URL, kept deliberately.

**Measured:** the published tree stands up from a clean copy with no edits beyond the two
the documentation asks for. **Not measured:** the install guide end to end on a machine with
no Ollama, no Open WebUI and no `openwebui_default` network — the step-3 failure above was
reasoned from Compose's naming rule and fixed, not reproduced.

---
## Change 63 — one command that says what is set up, what is off, and what is broken

**Symptom.** There was no way to ask "is this working" and get an answer. `smoke.sh` runs 175
cases and prints 175 lines; it tells you whether the suite passes, not whether *your*
installation is ready. Nothing reported the difference between a tool that is broken, a tool
that is switched off, and a tool that is switched ON and not set up — and the third is the
only one that actually costs anything, because the model is being offered something that
will fail.

**Change.** `./verify.py`. It reads the config, asks Docker what is running, and calls each
tool once with a known-good argument lifted from the smoke suite. Every tool gets one line;
only the actionable ones get a second. Two axes, reported together:

    All Set · All Set, Disabled · Needs Setup, Disabled · Needs Setup · Error · Error, Disabled

`logs/verify.log` holds what was checked and what each probe returned;
`logs/verify-error-<tool>.log` holds one failing tool's story on its own, because when one
thing is broken and twenty are fine you want the one. Stale error files are deleted when the
tool passes, so one cannot be found later and believed. Exit 0 / 1 (something enabled is
failing) / 2 (something needs attention but is switched off).

**Five things it got wrong first, all found by running it against a deliberately broken
system rather than by reading it.**

*A disabled tool was reported `All Set, Disabled` without checking anything* — it cannot be
probed, because switching it off removes the route. Now its dependencies are checked
structurally, which is what makes that label a claim rather than a shrug.

*`docker compose ps` hides stopped containers.* With octave stopped the stack still read
"All Set, 10 running", because a container that exited simply vanishes from the output.
Needs `-a`, plus a comparison against `docker compose config --services` to catch a service
with no container at all.

*Then that fix was too blunt:* one stopped container made `up` false and all 22 tools
"Not Checked". Probing needs the **api** container and nothing else; each tool's own
dependency belongs on its own row.

*A Python traceback's first line is `Traceback (most recent call last):`*, which is what the
terminal showed. The last line is the exception.

*And it waited 80 seconds to report `TimeoutError` for a tool whose container it already knew
was down.* Checking the dependency before probing turned that into "needs octave, which is
exited" in 3.7 seconds — the same finding with the cause in it.

Plus one bad probe of its own: `lookup_ip_or_asn` was given `203.0.113.10`, which the smoke
suite uses precisely because the tool REFUSES documentation ranges. It reported the tool
broken until the payload became a real address.

**Result.** On a working installation: 22 tools, 22 All Set, exit 0, about 40 seconds
(`--quick` skips the internet and takes four). On an unconfigured clone: three Needs Setup
rows naming the missing token and pointing at the install guide, exit 2 — and that exit code
was wrong until stack rows were counted alongside tool rows, which is the one case where a
non-zero code is most obviously wanted.

**Measured:** every status in the table above was produced deliberately - by switching tools
off, unsetting `OWUITOOLS_CONTACT`, and stopping a container - and each rendered correctly
with the right log files written and the right exit code. **Not measured:** `vm_agent` is
checked structurally and always will be; calling it starts a real machine.

---
## Change 64 — the published README told the reader to run a file that is not published

**Symptom.** The repository was pushed to Gitea and the rendered README opened with a
section called *"Publishing it"*, instructing the reader to run `make-release.py` from
`~/Documents/owui-tools`. That script is on the exclude list precisely because it carries the
sweep patterns, and the path is one machine's home directory. So the first file anybody reads
ended with an instruction that cannot be followed.

**Cause.** The section was written for the live tree, where it is correct and useful. Nothing
distinguished "documentation of the project" from "documentation of maintaining my copy of
it", and the release sweep only rewrites strings - it had no notion that a whole section
might not belong.

**Change.** `make-release.py` now strips a named list of sections from `README.md` on the way
out, currently just that one. The live tree keeps it.

**Worth noting about how it was found.** Not by the link checker, which passed: every link in
that section was to a heading or a shell command, not to a missing file. Not by the release
verifier either, which checks for things that must not be there rather than references to
things that are not. It was found by reading the README as rendered on the repository page -
which is the one view nobody had looked at, because until there was a repository there was
nothing to look at.

**Measured:** the published README now has eight sections and no mention of `make-release.py`
or of any path under `/home`. **Not measured:** whether anything else in the docs is
similarly addressed to the maintainer rather than the reader; the strip list has one entry
because one is all that has been found.