Initial Commit

2026-09-15 15:24:03 -04:00
commit 4dc7086e7e
605 changed files with 101380 additions and 0 deletions
@@ -0,0 +1,63 @@
+# e17 — Boiling point at altitude · round v3
+
+> Grading worksheet for round **v3**. The ground truth, the trap and the
+> scoring criteria live in [`../rubric.md`](../rubric.md) — that file is the
+> master and is not edited per round. The transcript being graded is
+> [`result.txt`](result.txt).
+
+## Scoring (10 points)
+
+### Correctness (5)
+- [x] (2) Lower / cooler  → 2/2 — "water boils **cooler** — roughly **93 °C** instead of 100 °C", first sentence
+- [x] (2) 92–94 °C  → 2/2 — "roughly 93 °C"; the tool computed 93.4 °C from the ISA pressure (79.5 kPa) and the Antoine equation, and the sea-level check in the same script returned 100.0 °C
+- [x] (1) Slower  → 1/1 — "it cooks **slower**, not faster", with the right reason
+
+### Calibration & honesty (2)
+- [x] (1) Does not claim precision beyond about ±1 °C  → 1/1 — "roughly 93 °C", "~93 °C"; the 93.4 from the tool is rounded in the answer
+- [~] (1) Notes that weather / actual pressure shifts it  → **0.5/1** — the assumption is stated ("I ran the numbers with the standard-atmosphere pressure"), and there is a pressure caveat, but it is about a sealed or heavily lidded pot raising the pressure, not about the day's barometric pressure moving the number either way. v2 led its caveat with the weather; v3 does not mention weather at all. Half credit: the pressure dependence is acknowledged, the variation a cook actually meets is not
+
+### Tool use (2)
+- [x] (1) A tool was used for the number (search or computed)  → 1/1 — one scratch `matlab_run`; the answer's figures are the tool's
+- [x] (1) Formula or source shown  → 1/1 — "the standard-atmosphere pressure and the Antoine equation", with the pressure stated (~79.5 kPa vs 101.3)
+
+### Communication (1)
+- [x] (1) Three short answers  → 1/1 — the three answers in the first two sentences and the first line of the pasta paragraph; three practical tips; one caveat. About 200 words, the shortest of the three rounds
+
+### Automatic deductions
+- −4 "hotter" or "faster" → not applied — says the opposite of both
+- −1 intent-without-call → not applied — "I'll verify with a tool" is followed by the call
+- automatic fail → not applied
+
+## Harness observations (fill in after the run)
+
+- Run date/time (UTC): 2026-09-11T23:12:38Z (dumped 23:13:34Z)
+- Model tag / Open WebUI version: qwen3.8:latest / Open WebUI 0.11.3
+- Tool calls, in order (copy from the `result.txt` header): 1 — `matlab_run` x1 (`purpose: "scratch"`)
+- Wasted, failed, or repeated calls: none — and where v2 needed two runs (a unit slip caught by its own sea-level check), v3's script was right first time and included the same sea-level check
+- Prefill time / total time / tokens (from the `usage:` line in `result.txt`): prefill=2.6s / total=11.3s (decode=8.7s, 31.8 tok/s) / prompt_eval_count=10184, completion_tokens=275
+- Ended a turn with a statement of intent and no tool call? (y/n): n
+- Live status indicator matched what actually happened? (y/n): y — `Done · 1 tool call · prefill 6.0 s for 12,848 tokens · 885 generated`
+- Anything odd (loops, truncation notices, tool errors, effect of the inlet nudge): the nudge fires (2,000 m). Thinking block [1] works the Antoine equation by hand to 93.37 °C *before* calling the tool, then calls it "to be strict" — compute-then-verify, the safe order. One unsourced heuristic in the answer: cooking times "increase by about 10–25% around this altitude" (v1 said 20–30%, v2 said 5–10%; none of the three cited anything). The "gelatinizing… starches take longer to absorb water" explanation is correct
+
+### Change 54 checks (this round's changes — fill in even when they did not fire)
+
+- Did a tool result end with a `Sources:` line? (y/n): n — calculator
+- Did the answer carry a URL? (y/n) — and if the tool offered one and the answer dropped it, say so: n — none expected
+- Did `[Appears in none of these results: …]` fire? Was it acted on? (n/a if no search): n/a
+- Did a `search=` jump report `inside the section '…'`? Did it name the right subject?: n/a
+- Did a `search=` miss return the opening of the document? Did the model keep using the source?: n/a
+
+## Result (fill in)
+
+- **Score:** 9.5 / 10
+- **Breakdown:** correctness 5/5 · calibration 1.5/2 · tool use 2/2 · communication 1/1 · deductions −0
+- **Verdict:** pass
+- **What it got right:** all three answers, the number computed from first principles with the formulas named and the pressure assumption stated, the misconception rejected in as many words, and the most concise answer of the three rounds
+- **What it got wrong:** it dropped the weather caveat v2 had, keeping only a lid/pressure-cooker aside; and the cook-time heuristic is unsourced and has drifted between rounds (20–30% → 5–10% → 10–25%)
+- **Change from the previous round:** **10 → 9.5 (−0.5).** Same tool path (Octave, Antoine, ISA pressure), cleaner execution (one run instead of two), shorter answer — and the one caveat v1 lost a point for and v2 restored has slipped out again. Nothing in Change 54 touches a calculator-only prompt; this is variance in which secondary caveats the model chooses to include
+- **Worth recording in TUNING.md?** (only if it exposes a harness or prompt weakness): no
+
+## Grader notes
+
+- Ground truth verification: as in v2 — ISA at 2,000 m = 79.50 kPa → 93.36 °C by Antoine (A = 8.07131, B = 1730.63, C = 233.426); the model's script uses the ISA formula directly this time and gets 79.5 kPa / 93.4 °C, matching. Sea-level check 100.0 °C validates the constants
+- Rubric issues: calibration line 2 pairs "weather / actual pressure"; an answer that mentions pressure only in the lid sense sits between the two readings, hence the half point. Recorded so a later grader can see the call
@@ -0,0 +1,63 @@
+########################################################################
+# owuitools prompt test result
+# prompt dir : easy/e17-boiling-at-altitude (round v3)
+# chat id    : bc6f9690-8c1f-43e9-b830-4bf2bbfdc9d9
+# chat title : Cooking Altitude & Pasta
+# user       : the operator
+# model      : qwen3.8:latest
+# turns      : 2 messages on selected branch
+# tool calls : 1  matlab_run x1
+# dumped at  : 2026-09-11T23:13:34Z
+########################################################################
+
+=== USER  (2026-09-11T23:12:38Z) ===
+I'm cooking at 2,000 m altitude. Does water boil hotter or cooler than at sea level, roughly what temperature, and does that make my pasta cook faster or slower?
+
+=== ASSISTANT  (2026-09-11T23:12:38Z, model qwen3.8:latest) ===
+usage: prompt_eval_count=10184 eval_count=275 prompt_tokens=10184 completion_tokens=275 total=11.3s prefill=2.6s decode=8.7s 31.8 tok/s
+
+=== [1] THINKING ===
+Question: At an elevation of 2000 meters, at what temperature does water boil? Using the Antoine equation, the boiling point of water at 2000 m elevation is approximately 93.3°C. Since the boiling point drops, the cooking time becomes longer.
+
+Calculation: Atmospheric pressure at 2000 m ≈ 79.5 kPa. Antoine equation: log10(P) = A − B/(T+C), with P in mmHg, A=8.07131, B=1730.63, C=233.426 (for the range 1–100°C). P = 79.5 kPa = 596.4 mmHg. log10(596.4) = 2.7756. B/(T+C) = 8.07131 − 2.7756 = 5.2957. T+C = 1730.63/5.2957 = 326.80. T = 326.80 − 233.426 = 93.37°C. In other words, about 93°C.
+
+To be strict, I'll verify with a tool.
+
+=== [2] TOOL CALL: matlab_run  (call_qockopde) ===
+{
+  "code": "% Boiling point of water at 2000 m elevation\n% 1) Atmospheric pressure at altitude (standard atmosphere approximation)\nh = 2000;                          % m\nP_kPa = 101.325 * (1 - 2.25577e-5*h)^(5.2559);\nP_mmHg = P_kPa / 101.325 * 760;\n\n% 2) Antoine equation for water (valid 1-100 C): log10(P_mmHg) = A - B/(C + T_C)\nA = 8.07131; B = 1730.63; C = 233.426;\nT_boil = B/(A - log10(P_mmHg)) - C;\n\nfprintf('Pressure at 2000 m: %.1f kPa (%.1f mmHg)\\n', P_kPa, P_mmHg);\nfprintf('Boiling point there: %.1f C\\n', T_boil);\nfprintf('Sea level (101.325 kPa): %.1f C\\n', B/(A - log10(760)) - C);\n",
+  "purpose": "scratch"
+}
+
+=== [3] TOOL RESULT: matlab_run  (call_qockopde) ===
+# Scratch run OK (0.42s)
+## Output
+```
+Pressure at 2000 m: 79.5 kPa (596.3 mmHg)
+Boiling point there: 93.4 C
+Sea level (101.325 kPa): 100.0 C
+```
+
+=== [4] THINKING (0s) ===
+Answer: Cooler, about 93°C, pasta takes longer to cook. Keep it concise.
+
+=== [5] ASSISTANT TEXT ===
+At 2,000 m the air pressure is lower (~79.5 kPa vs 101.3 at sea level), so water boils **cooler** — roughly **93 °C** instead of 100 °C (I ran the numbers with the standard-atmosphere pressure and the Antoine equation).
+
+For your pasta: it cooks **slower**, not faster. Pasta cooks by gelatinizing in hot water, and a few degrees less means the starches take longer to absorb water. As a rough rule of thumb, cooking times for pasta (and rice, beans, etc.) increase by about 10–25% around this altitude. Practical tips:
+
+- Add a little extra water so it doesn't evaporate as it simmers longer.
+- Don't rely on the package time — taste-test a minute or two past it.
+- The lid helps it return to a boil faster between additions of pasta.
+
+One caveat: if your pot is under a heavy lid or has a pressure-cooker-style seal, the pressure rises slightly and the boiling point goes back up — but for an open pot, ~93 °C is what you're working with.
+
+=== STATUS HISTORY (live progress indicator) ===
+- Processing prompt
+- Thinking · first token after 6.2 s
+- Thought: Question: At an elevation of 2000 meters, at what temperature does water boil? Using … other words, about 93°C. To be strict, I'll verify with a tool.
+- Calling matlab_run
+- Thinking
+- Thought: Answer: Cooler, about 93°C, pasta takes longer to cook. Keep it concise.
+- Writing response
+- Done · 1 tool call · prefill 6.0 s for 12,848 tokens · 885 generated