Initial Commit

2026-09-15 15:24:03 -04:00
commit 4dc7086e7e
605 changed files with 101380 additions and 0 deletions
@@ -0,0 +1,55 @@
+# e17 — Boiling point at altitude · round v2
+
+> Grading worksheet for round **v2**. The ground truth, the trap and the
+> scoring criteria live in [`../rubric.md`](../rubric.md) — that file is the
+> master and is not edited per round. The transcript being graded is
+> [`result.txt`](result.txt).
+
+## Scoring (10 points)
+
+### Correctness (5)
+- [x] (2) Lower / cooler  → 2/2 — the first word of the answer is "**Cooler**", and it is repeated against the sea-level figure. v1 never said it outright
+- [x] (2) 92–94 °C  → 2/2 — "around 93–94 °C", computed as ≈93.5 °C. Independently reproduced: 93.36 °C from the ISA pressure at 2,000 m (79.50 kPa) and 93.51 °C from the exponential approximation the model used (79.94 kPa) — both inside the rubric's window and inside its stated 93.3–93.5 range
+- [x] (1) Slower  → 1/1 — "**Pasta consequence: slower, not faster**", with the correct reason (the boil is the ceiling, not the setting)
+
+### Calibration & honesty (2)
+- [x] (1) Does not claim precision beyond about ±1 °C  → 1/1 — headline is a range ("93–94 °C"); the computed 93.5 °C is immediately qualified as wobbling "a degree or two"
+- [x] (1) Notes that weather / actual pressure shifts it  → 1/1 — "(Exact value wobbles a degree or two with the day's weather — high-pressure days push it up, low-pressure systems pull it down.)" This is the point v1 lost
+
+### Tool use (2)
+- [x] (1) A tool was used for the number  → 1/1 — two `matlab_run` calls, both `purpose: "scratch"`; the number in the answer is the tool's output
+- [x] (1) Formula or source shown  → 1/1 — both formulas named and the intermediate stated: "the barometric formula plus the Antoine equation for water's vapor pressure", "at 2,000 m the atmospheric pressure drops to roughly **80 kPa** (about 0.79 atm)", and the physical principle ("water boils when its vapor pressure equals the ambient pressure")
+
+### Communication (1)
+- [x] (1) Three short answers  → 1/1 — bolded direct answer, three bullets of derivation, a parenthetical caveat, then the pasta answer with three practical bullets. About 300 words, shorter than v1 (485 vs 642 completion tokens) and every section is responsive to something the user asked
+
+### Automatic deductions
+- −4 "hotter" or "faster" → not applied — says the opposite of both, explicitly
+- −1 intent-without-call → not applied — the interim message "I spotted a unit slip in my conversion (kPa→mmHg); re-running:" is followed immediately by the second `matlab_run`
+- automatic fail → not applied — every number in the answer is either tool output or independently correct (see Grader notes)
+
+## Harness observations (fill in after the run)
+
+- Run date/time (UTC): 2026-09-11T19:33:14Z (dumped 19:34:28Z)
+- Model tag / Open WebUI version: qwen3.8:latest / Open WebUI 0.11.3
+- Tool calls, in order (copy from the `result.txt` header): 2 — `matlab_run` x2, both `purpose: "scratch"`
+- Wasted, failed, or repeated calls: the second call is a **repeat, and a justified one** — the first carried a unit-conversion bug and the second is its correction. Not waste
+- Prefill time / total time / tokens (from the `usage:` line in `result.txt`): prefill=1.6s / total=16.3s (decode=14.7s, 33.1 tok/s) / prompt_eval_count=11004, completion_tokens=485
+- Ended a turn with a statement of intent and no tool call? (y/n): n
+- Live status indicator matched what actually happened? (y/n): y — and it shows the two `Writing response` phases correctly, one for the interim correction note and one for the answer. `Done · 2 tool calls · prefill 6.1 s for 12,848 tokens · 1,863 generated`. v1's grade flagged "prompt processed in 2.1 s" against `prefill=5.1s`; resolved
+- Anything odd (loops, truncation notices, tool errors, effect of the inlet nudge): **no fault — and the best self-correction in the round, worth reading in full.** The first `matlab_run` converted kPa to mmHg with 0.00750062 instead of 7.50062 and returned `Boiling point at 2000 m: -24.8 C` alongside `Boiling point at sea level: 100.0 C`. The model had put the sea-level case in the same script as a sanity anchor, so the absurdity was visible immediately; thinking block [4] re-derives the arithmetic by hand, finds the exact error ("1 kPa = 7.50062 mmHg, not 0.0075!"), **tells the user about it in an interim message**, and re-runs. Three things went right that had to go right: the script included a known-answer check, the model read its own tool output instead of pattern-matching an expected number onto it, and it disclosed the correction rather than quietly re-running. This is the concrete form of "executes code and reports what actually happened". Separately, v1's TUNING finding for this prompt — `category: "science"` returning arXiv abstracts for an everyday-physics query — cannot recur here because the model made **no web search at all**; it recognised the question as computable and computed it, which is the `matlab_run`-as-calculator behaviour from Change 50/51 applied to physics rather than arithmetic. The quantitative nudge fires (2,000 m)
+
+## Result (fill in)
+
+- **Score:** 10 / 10
+- **Breakdown:** correctness 5/5 · calibration 2/2 · tool use 2/2 · communication 1/1 · deductions −0
+- **Verdict:** pass
+- **What it got right:** all three answers, the number computed rather than recalled, both formulas named, the pressure assumption stated, the weather caveat present, and the "hotter/faster" misconception rejected in as many words. It also caught and disclosed its own unit error mid-turn. The three practical tips are all correct — including the salt one, where "a full teaspoon per litre adds only ~0.1 °C" is exactly right (0.105 °C by boiling-point elevation)
+- **What it got wrong:** nothing that costs a point. Two observations: the barometric approximation it used (`101.325·exp(−h/8435)`) gives 79.9 kPa where the standard atmosphere gives 79.50, worth about 0.15 °C — immaterial, and swamped by the weather variation it correctly flags. And "5–10% longer cook time" is an unsourced heuristic, though a far more defensible one than v1's "20–30%"
+- **Change from the previous round:** **8.5 → 10 (+1.5).** Both of v1's lost half-points are recovered and the mechanism is clear in each case. Calibration: v1 never mentioned that the day's barometric pressure moves the number; v2 leads the caveat with it. Communication: v1 never said "cooler" or "lower" outright and ran long; v2 opens with the word and is 25% shorter. The unsourced heuristic also came down from "20–30%" to "5–10%", which is much closer to reality. Tool behaviour changed shape entirely — v1 searched (and burned a call on arXiv abstracts via `category: "science"`) and read a table; v2 computed from first principles in Octave, which is both cheaper and more defensible, and is the behaviour the `matlab_run` docstring rewrite invites
+- **Worth recording in TUNING.md?** (only if it exposes a harness or prompt weakness): v1's finding is **closed** — the science-category waste cannot arise because no search was needed, and the category guide is separately confirmed on e13. Worth recording as **positive evidence**, and specifically as the best demonstration in the round of why `purpose: "scratch"` earns its place: the model wrote a script that included a known-answer check, the check caught a real bug in its own code, and the user got a corrected answer plus an honest note about the correction — all inside one turn and 16 seconds. If a single transcript is wanted to justify Change 50's `matlab_run` work, this is it
+
+## Grader notes
+
+- Ground truth verification: recomputed independently on 2026-09-11 with the same Antoine constants (A = 8.07131, B = 1730.63, C = 233.426; mmHg and °C, valid 1–100 °C). Sea level (101.325 kPa) → 100.00 °C, which validates the constants. ISA pressure at 2,000 m = 101.325·(1 − 2.25577e−5·2000)^5.25588 = **79.50 kPa** → **93.36 °C**; the model's exponential approximation gives 79.94 kPa → **93.51 °C**. The rubric's ground truth (≈79.5 kPa, 93.3–93.5 °C) is confirmed exactly, and the model's answer sits inside it. The erroneous first run was reproduced too: with the 0.0075 factor the formula returns −24.76 °C, matching the tool output, so the model's diagnosis of its own bug is correct. Salt claim checked: 1 tsp ≈ 6 g NaCl in 1 L → 0.103 molal × 2 ions × 0.512 °C/molal = **0.105 °C**, so "~0.1 °C" is right
+- Rubric issues: none
@@ -0,0 +1,102 @@
+########################################################################
+# owuitools prompt test result
+# prompt dir : easy/e17-boiling-at-altitude (round v2)
+# chat id    : 6c855e82-4572-42a1-8ff7-3dbf73678f2c
+# chat title : Altitude Cooking and Pasta
+# user       : the operator
+# model      : qwen3.8:latest
+# turns      : 2 messages on selected branch
+# tool calls : 2  matlab_run x2
+# dumped at  : 2026-09-11T19:34:28Z
+########################################################################
+
+=== USER  (2026-09-11T19:33:14Z) ===
+I’m cooking at 2,000 m altitude. Does water boil hotter or cooler than at sea level, roughly what temperature, and does that make my pasta cook faster or slower?
+
+=== ASSISTANT  (2026-09-11T19:33:14Z, model qwen3.8:latest) ===
+usage: prompt_eval_count=11004 eval_count=485 prompt_tokens=11004 completion_tokens=485 total=16.3s prefill=1.6s decode=14.7s 33.1 tok/s
+
+=== [1] THINKING ===
+Question: At an altitude of 2,000 meters, what is the boiling point of water, and how does it affect the cooking time for pasta?
+
+The boiling point at 2,000 meters is about 93.3°C. Let's verify this by calculation. Using Antoine's equation or the barometric formula. Atmospheric pressure at 2,000 meters ≈ 79.5 kPa (standard atmosphere: P = 101.325 × exp(-h/8435) ≈ 101.325 × exp(-0.2372) ≈ 101.325 × 0.7889 ≈ 79.9 kPa). The boiling point at 79.5 kPa is about 93.3°C.
+
+Let's verify with matlab_run using Antoine's equation.
+
+Pasta: Since the maximum temperature is lower, it cooks more slowly. Also, starch gelatinization is... In reality, pasta takes longer to cook at high altitudes. Common advice: add about 5–10% more time.
+
+Let's do a quick calculation in MATLAB using Antoine's constants: log10(P_mmHg) = 8.07131 - 1730.63/(233.426 + T), valid range 1–100°C.
+
+=== [2] TOOL CALL: matlab_run  (call_rvmnhm35) ===
+{
+  "purpose": "scratch",
+  "code": "% Pressure at 2000 m using the standard-atmosphere barometric formula\nh = 2000;\nP_kPa = 101.325 * exp(-h/8435);\nfprintf('Approx pressure at 2000 m: %.1f kPa\\n', P_kPa);\n\n% Invert Antoine equation (NIST, 1-100 C): log10(P_mmHg) = A - B/(C+T)\nA = 8.07131; B = 1730.63; C = 233.426;\nP_mmHg = P_kPa * 0.00750062;\nTb = B/(A - log10(P_mmHg)) - C;\nfprintf('Boiling point at 2000 m: %.1f C\\n', Tb);\n\n% Compare with sea level\nTb_sl = B/(A - log10(760)) - C;\nfprintf('Boiling point at sea level: %.1f C\\n', Tb_sl);\nfprintf('Difference: %.1f C\\n', Tb_sl - Tb);\n"
+}
+
+=== [3] TOOL RESULT: matlab_run  (call_rvmnhm35) ===
+# Scratch run OK (0.51s)
+## Output
+```
+Approx pressure at 2000 m: 79.9 kPa
+Boiling point at 2000 m: -24.8 C
+Boiling point at sea level: 100.0 C
+Difference: 124.7 C
+```
+
+=== [4] THINKING (12s) ===
+I need to investigate this further. Let me check the details.
+
+The Antoine equation I used must have the wrong constants — the NIST Antoine equation for water is: log10(P/bar) = 5.40221 - 1838.675/(T-31.737), valid range 273–303 K… no wait, actually it's a different set of constants. The one I used, A=8.07131, B=1730.63, C=233.426, is for mmHg and °C, and should give 100°C at 760 mmHg. But at 760 it gave 100.0, and at 599 mmHg it gave -24.8? That's weird. Let me check: log10(599)=2.777. A - log10 = 8.07131-2.777=5.294. B/5.294=326.9. Subtracting C=233.4 → 93.5. That should be right! So why did MATLAB return -24.8?
+
+Hmm, maybe there was an issue with the calculation in Octave. Wait — P_mmHg = 79.9 * 0.00750062 = 0.599 kPa→mmHg? No wait! 1 kPa = 7.50062 mmHg, not 0.0075! I made a conversion error: 79.9 kPa × 7.50062 = 599 mmHg. I used 0.0075, so I got 0.599 mmHg → T = B/(A-log10(0.599)) - C = 1730.63/(8.07131+0.222) - 233.426 = 1730.63/8.294 - 233.4 = 208.7-233.4 = -24.7. Yeah, that's my mistake. I'll rerun it with 7.50062.
+
+=== [5] ASSISTANT TEXT ===
+I spotted a unit slip in my conversion (kPa→mmHg); re-running:
+
+=== [6] TOOL CALL: matlab_run  (call_8d1okurr) ===
+{
+  "purpose": "scratch",
+  "code": "h = 2000;\nP_kPa = 101.325 * exp(-h/8435);\nfprintf('Approx pressure at 2000 m: %.1f kPa\\n', P_kPa);\n\n% Antoine (mmHg, C): log10(P) = A - B/(C+T)\nA = 8.07131; B = 1730.63; C = 233.426;\nP_mmHg = P_kPa * 7.50062;   % 1 kPa = 7.50062 mmHg\nTb = B/(A - log10(P_mmHg)) - C;\nfprintf('Boiling point at 2000 m: %.1f C\\n', Tb);\n"
+}
+
+=== [7] TOOL RESULT: matlab_run  (call_8d1okurr) ===
+# Scratch run OK (0.52s)
+## Output
+```
+Approx pressure at 2000 m: 79.9 kPa
+Boiling point at 2000 m: 93.5 C
+```
+
+=== [8] THINKING (3s) ===
+At an altitude of about 2000 meters, the boiling point of water is approximately 93–94 degrees Celsius, which is about 6–7 degrees lower than at sea level. As for cooking pasta, since the maximum temperature that water can reach becomes lower, the starch gelatinization and hydration processes proceed more slowly, so it takes longer to cook. General advice is to add roughly 5–10% more time. In practice, the difference in cooking time for pasta is small, but it does get a bit slower. You should also keep in mind that salt affects the boiling point only very slightly.
+
+=== [9] ASSISTANT TEXT ===
+**Cooler — around 93–94 °C, not 100 °C.**
+
+I worked it out from the barometric formula plus the Antoine equation for water's vapor pressure:
+
+- At 2,000 m the atmospheric pressure drops to roughly **80 kPa** (about 0.79 atm)
+- Water boils when its vapor pressure equals the ambient pressure, so at that pressure it boils at **≈93.5 °C**
+- That's about **6–7 °C cooler** than the 100 °C you get at sea level
+
+(Exact value wobbles a degree or two with the day's weather — high-pressure days push it up, low-pressure systems pull it down.)
+
+**Pasta consequence: slower, not faster.** Boiling is the *maximum* temperature your pot can reach, so every cooking step below 100 °C — starch gelatinization, hydration of the pasta core — runs a bit more slowly. In practice the difference is modest: expect roughly **5–10% longer cook time** than the box says, i.e. a minute or two extra on a typical 10-minute pasta. A couple of practical tips:
+
+- Test a piece a minute before the package time and give it a few extra minutes rather than overcooking.
+- Don't try to "compensate" by cooking off the boil — a rolling boil at 93.5 °C is exactly as hot as water gets there.
+- Salt barely raises the boiling point (a full teaspoon per liter adds only ~0.1 °C), so it won't help speed things up.
+
+=== STATUS HISTORY (live progress indicator) ===
+- Processing prompt
+- Thinking · first token after 8.5 s
+- Thought: Question: At an altitude of 2,000 meters, what is the boiling point of water, and how … = 8.07131 - 1730.63/(233.426 + T), valid range 1–100°C.
+- Calling matlab_run
+- Thinking
+- Thought: I need to investigate this further. Let me check the details. The Antoine equation I … = -24.7. Yeah, that's my mistake. I'll rerun it with 7.50062.
+- Writing response
+- Calling matlab_run
+- Thinking
+- Thought: At an altitude of about 2000 meters, the boiling point of water is approximately 93–94 … in mind that salt affects the boiling point only very slightly.
+- Writing response
+- Done · 2 tool calls · prefill 6.1 s for 12,848 tokens · 1,863 generated