Initial Commit

2026-09-15 15:24:03 -04:00
commit 4dc7086e7e
605 changed files with 101380 additions and 0 deletions
@@ -0,0 +1,61 @@
+# e17 — Boiling point at altitude · round v4
+
+> Grading worksheet for round **v4**. The ground truth, the trap and the
+> scoring criteria live in [`../rubric.md`](../rubric.md) — that file is the
+> master and is not edited per round. The transcript being graded is
+> [`result.txt`](result.txt).
+
+## Scoring (10 points)
+
+### Correctness (5)
+- [x] (2) Lower / cooler  → 2/2 — "**Cooler, not hotter.**", first words
+- [x] (2) 92–94 °C  → 2/2 — "**Roughly 93 °C** (about 200 °F)"; the tool computed 93.4 °C from ISA pressure and the Antoine equation
+- [x] (1) Slower  → 1/1 — "**Your pasta cooks slower.**", with the right reason (the water cannot exceed its local boiling point; gelatinisation and hydration are slower at 93 °C)
+
+### Calibration & honesty (2)
+- [x] (1) Does not claim precision beyond about ±1 °C  → 1/1 — "Roughly 93 °C"; the tool's 93.4 is rounded
+- [ ] (1) Notes that weather / actual pressure shifts it  → **0/1** — absent. The answer states the ISA pressure (~79.5 kPa) as a fixed fact and never says the day's barometric pressure moves the number; the lid/pressure-cooker aside that earned v3 a half-point is not here either. Scored as v1 was (0/1 for the same omission)
+
+### Tool use (2)
+- [x] (1) A tool was used for the number (search or computed)  → 1/1 — one scratch `matlab_run`; the answer's figures are the tool's
+- [ ] (1) Formula or source shown  → **0.5/1** — the answer shows the intermediate ("air pressure there is ~79.5 kPa instead of 101 kPa") and the physical principle ("Water boils at whatever temperature its vapor pressure matches the surrounding air pressure"), but names neither a formula nor a source. v2 and v3 both named the Antoine equation and the standard atmosphere in the answer and scored 1/1; v1 named a table and its rows. Half credit for the intermediate and the principle
+
+### Communication (1)
+- [x] (1) Three short answers  → 1/1 — the three answers are the first three bold sentences. What follows (two numbered mechanisms, four tips, a rule of thumb) is the longest tail of the four rounds (663 completion tokens against v3's 275) and is not what the line scores
+
+### Automatic deductions
+- −4 "hotter" or "faster" → not applied — says the opposite of both
+- −1 intent-without-call → not applied — "Let me run the calculation" is followed by the call
+- automatic fail → not applied
+
+## Harness observations (fill in after the run)
+
+- Run date/time (UTC): 2026-09-12T04:18:58Z (dumped 04:20:10Z)
+- Model tag / Open WebUI version: qwen3.8:latest / Open WebUI 0.11.3
+- Tool calls, in order (copy from the `result.txt` header): 1 — `matlab_run` x1 (`purpose: "scratch"`)
+- Wasted, failed, or repeated calls: none — right first time, with the sea-level sanity line
+- Prefill time / total time / tokens (from the `usage:` line in `result.txt`): prefill=2.5s / total=22.8s (decode=20.2s, 32.9 tok/s) / prompt_eval_count=10904, completion_tokens=663
+- Ended a turn with a statement of intent and no tool call? (y/n): n
+- Live status indicator matched what actually happened? (y/n): y — `Done · 1 tool call · prefill 5.1 s for 12,932 tokens · 1,905 generated`
+- Anything odd (loops, truncation notices, tool errors, effect of the inlet nudge): the nudge fires (2,000 m). Thinking block [1] works the ISA formula and the Antoine equation by hand to 93.35 °C before calling the tool "to avoid decimal slip, per the instructions" — compute-then-verify, as v3. The formulas are named and worked in thinking and then left out of the answer, which is where the half-point went. Two unscored loose claims in the tail: "it reaches a boil *earlier*" (true in temperature, confusing in time — a pot at altitude does not boil sooner on the clock in any useful sense), and the cook-time heuristic has drifted again (v1 20–30%, v2 5–10%, v3 10–25%, v4 15–25%; none sourced). The salt remark (a few tenths of a degree) is correct
+
+### Change 55 checks (this round's changes — fill in even when they did not fire)
+
+- Did a tool result carry a `Sources:` trailer? (y/n): n — calculator
+- Did the answer end with a URL — written by the model, or appended by the `answer_sources` filter? (model / filter / none): none — none expected
+- Any of the 55.2 tool fixes exercised? Which, and did it fire correctly?: none — the Octave ran clean
+
+## Result (fill in)
+
+- **Score:** 8.5 / 10
+- **Breakdown:** correctness 5/5 · calibration 1/2 · tool use 1.5/2 · communication 1/1 · deductions −0
+- **Verdict:** pass
+- **What it got right:** all three answers, the number computed from first principles and verified in the tool, the misconception rejected in the first two words, three short answers up front
+- **What it got wrong:** no weather/actual-pressure caveat at all (v3 had half of one); the formula and source that the thinking used are not named in the answer; and the tail is twice the length it needs, with a drifting unsourced cook-time percentage
+- **Change from the previous round:** **9.5 → 8.5 (−1).** Same tool path, same number, same correctness; the losses are both in what the answer chose to say. v3's grade called the missing weather caveat "variance in which secondary caveats the model chooses to include", and v4 confirms that: the caveat has now been present (v2), half-present (v3) and absent (v1, v4) with no harness change that touches it. The formula-naming loss is new — the answer grew in tips and shrank in method. Change 55 has no surface on a calculator-only prompt
+- **Worth recording in TUNING.md?** (only if it exposes a harness or prompt weakness): no — model-side variance on secondary content. The one thing worth a line is the pattern across the round of answers growing longer tails (e04, e13, e17) with the extra material unsourced
+
+## Grader notes
+
+- Ground truth verification: ISA at 2,000 m = 79.50 kPa → 93.36 °C by Antoine (A = 8.07131, B = 1730.63, C = 233.426); the model's script computes both from the same constants and prints 79.5 kPa / 93.4 °C / 200.1 °F. Correct. Salt: 10 g/L NaCl raises the boiling point by ~0.17 °C — "a few tenths" is right
+- Rubric issues: calibration line 2 pairs "weather / actual pressure"; v3 sat between the readings and got half, v4 has neither and gets none. The tool-use line "formula or source shown" was read as requiring one of the two to be *named* in the answer; an intermediate value alone is half. Recorded so a later grader can see both calls
@@ -0,0 +1,117 @@
+########################################################################
+# owuitools prompt test result
+# prompt dir : easy/e17-boiling-at-altitude (round v4)
+# chat id    : 56ce08b3-9481-40bd-8d28-db7f1839a68f
+# chat title : Altitude Cooking & Pasta
+# user       : the operator
+# model      : qwen3.8:latest
+# turns      : 2 messages on selected branch
+# tool calls : 1  matlab_run x1
+# dumped at  : 2026-09-12T04:20:10Z
+########################################################################
+
+=== USER  (2026-09-12T04:18:58Z) ===
+I'm cooking at 2,000 m altitude. Does water boil hotter or cooler than at sea level, roughly what temperature, and does that make my pasta cook faster or slower?
+
+=== ASSISTANT  (2026-09-12T04:18:58Z, model qwen3.8:latest) ===
+usage: prompt_eval_count=10904 eval_count=663 prompt_tokens=10904 completion_tokens=663 total=22.8s prefill=2.5s decode=20.2s 32.9 tok/s
+
+=== [1] THINKING ===
+The user is asking about boiling point at 2000m altitude. This is a physics question I can calculate. The boiling point of water depends on atmospheric pressure. At 2000m, pressure is lower, so water boils at a lower temperature.
+
+Let me think about this properly. The standard atmosphere model gives pressure as a function of altitude. Then I use the Clausius-Clapeyron relation or Antoine equation to find the boiling point.
+
+Standard atmosphere: at 2000m, pressure is roughly 79.5 kPa (about 784 mmHg). Let me compute more precisely.
+
+Using the international standard atmosphere (below 11km):
+P = P0 * (1 - L*h/(T0))^5.2561
+where P0 = 101325 Pa, L = 0.0065 K/m, T0 = 288.15 K, h = 2000 m
+
+1 - L*h/T0 = 1 - 0.0065*2000/288.15 = 1 - 13/288.15 = 1 - 0.045115 = 0.954885
+0.954885^5.2561 = ?
+
+ln(0.954885) = -0.04617
+* 5.2561 = -0.24268
+exp(-0.24268) = 0.7845
+
+So P = 101325 * 0.7845 = 79,490 Pa ≈ 79.5 kPa
+
+Now the boiling point. Use Antoine equation for water:
+log10(P_mmHg) = A - B/(C + T_C)
+A = 8.07131, B = 1730.63, C = 233.426 (valid 1-100°C)
+
+P in mmHg = 79490 / 133.322 = 596.2 mmHg
+log10(596.2) = 2.7754
+2.7754 = 8.07131 - 1730.63/(233.426 + T)
+1730.63/(233.426 + T) = 8.07131 - 2.7754 = 5.2959
+233.426 + T = 1730.63/5.2959 = 326.78
+T = 326.78 - 233.426 = 93.35°C
+
+So roughly 93°C at 2000m.
+
+Let me verify with a tool. I should use matlab_run scratch to be precise, and maybe do a quick web check on the standard value. Actually, this is well-established physics. Let me just compute it carefully with matlab_run scratch.
+
+Actually the rule of thumb: boiling point drops about 1°C per ~285-300 m of altitude near sea level, but that's not linear. A common figure: at 2000m (about 6500 ft), water boils at about 93°C (199°F). That matches my calc.
+
+Pasta cooks slower because lower temperature means slower cooking (starch gelatinization and protein denaturation happen faster at higher temps). The pasta still floats when done, so the float test works, but it takes longer.
+
+Let me run the calculation to be precise and cite it. I'll do a matlab scratch run.
+
+I don't strictly need web search — this is textbook physics. But let me verify the number with a tool to avoid decimal slip, per the instructions.
+
+=== [2] TOOL CALL: matlab_run  (call_ayhowvb3) ===
+{
+  "purpose": "scratch",
+  "code": "% Boiling point of water at 2000 m altitude\n% 1) International Standard Atmosphere pressure (valid below 11 km)\nP0 = 101325;          % Pa\nL  = 0.0065;          % K/m\nT0 = 288.15;          % K\nh  = 2000;            % m\nP  = P0*(1 - L*h/T0)^5.2561;\nfprintf('Pressure at 2000 m: %.1f kPa (%.1f mmHg)\\n', P/1000, P/133.322);\n\n% 2) Antoine equation for water (1-100 C): log10(P_mmHg)=A-B/(C+T)\nA=8.07131; B=1730.63; C=233.426;\nPmm = P/133.322;\nTb  = B/(A - log10(Pmm)) - C;\nfprintf('Boiling point at 2000 m: %.1f C (%.1f F)\\n', Tb, Tb*9/5+32);\n\n% For reference, sea level\nfprintf('Sea-level boiling point: ~100 C\\n');"
+}
+
+=== [3] TOOL RESULT: matlab_run  (call_ayhowvb3) ===
+# Scratch run OK (0.43s)
+## Output
+```
+Pressure at 2000 m: 79.5 kPa (596.3 mmHg)
+Boiling point at 2000 m: 93.4 C (200.1 F)
+Sea-level boiling point: ~100 C
+```
+
+=== [4] THINKING (5s) ===
+The calculation confirms ~93°C. Now let me answer clearly and concisely.
+
+Key points:
+- Water boils COOLER at altitude, not hotter. Lower pressure → lower boiling point.
+- At 2000m: about 93°C (roughly 93–94°C).
+- Pasta cooks SLOWER, because the water is cooler during cooking, and the cooking reactions (starch gelatinization, hydration) proceed more slowly at lower temperature.
+- Practical tips: allow extra cook time, don't add salt to raise boiling point (it barely changes it), use the package time as a starting point and taste-test.
+
+One nuance people get wrong: adding salt doesn't meaningfully raise the boiling point (only by a fraction of a degree). I'll mention that briefly since it's a common misconception tied to this exact situation.
+
+Let me write a clean answer.
+
+=== [5] ASSISTANT TEXT ===
+**Cooler, not hotter.** Water boils at whatever temperature its vapor pressure matches the surrounding air pressure — and air pressure drops with altitude. So at 2,000 m it reaches a boil *earlier* (at a lower temperature), not later.
+
+**Roughly 93 °C** (about 200 °F) at 2,000 m, versus 100 °C at sea level. That works out because the air pressure there is ~79.5 kPa instead of 101 kPa. The drop isn't linear, but "≈1 °C per 285 m" is a decent rule of thumb in this range.
+
+**Your pasta cooks slower.** Two things are both working against you:
+
+1. **Lower cooking temperature.** Once boiling, the water can't get any hotter than its local boiling point, so your pasta is being cooked at ~93 °C instead of 100 °C. Starch gelatinization and hydration run more slowly at that temperature.
+2. **The "extra time" effect compounds.** Because the whole pot sits a few degrees cooler the entire time, the difference is a real percentage of cook time, not a rounding error — expect to add a few minutes over the package's sea-level time.
+
+Practical tips for altitude cooking:
+
+- **Start from the package time and taste-test** rather than trusting it — it's written for sea level.
+- **Don't rely on salt to fix it.** Adding a normal amount of salt raises the boiling point by only a fraction of a degree (a few tenths), far too little to matter. It seasons the pasta; it doesn't speed it up.
+- **Use more water / a bigger pot** if you can — it helps the water return to a boil faster between additions, which matters more the bigger the batch.
+- The **"it floats when it's done"** trick still works — buoyancy isn't temperature-dependent — but it'll take longer to get there.
+
+Rule of thumb: add roughly 15–25% extra time to your usual pasta cook time at 2,000 m, and check a minute or two before the package says.
+
+=== STATUS HISTORY (live progress indicator) ===
+- Processing prompt
+- Thinking · first token after 6.8 s
+- Thought: The user is asking about boiling point at 2000m altitude. This is a physics question I … number with a tool to avoid decimal slip, per the instructions.
+- Calling matlab_run
+- Thinking
+- Thought: The calculation confirms ~93°C. Now let me answer clearly and concisely. Key points: - … tied to this exact situation. Let me write a clean answer.
+- Writing response
+- Done · 1 tool call · prefill 5.1 s for 12,932 tokens · 1,905 generated