Initial Commit

2026-09-15 15:24:03 -04:00
commit 4dc7086e7e
605 changed files with 101380 additions and 0 deletions
@@ -0,0 +1,52 @@
+# e05 — Version currency · round v1
+
+> Grading worksheet for round **v1**. The ground truth, the trap and the
+> scoring criteria live in [`../rubric.md`](../rubric.md) — that file is the
+> master and is not edited per round. The transcript being graded is
+> [`result.txt`](result.txt).
+
+## Scoring (10 points)
+
+### Correctness (5)
+- [x] (3) Version matches PyPI at grading time  → 3/3 — final answer states **2.34.2**; PyPI `info.version` was 2.34.2 when checked 2026-09-11T05:49Z
+- [x] (2) Minimum Python matches `requires_python`  → 2/2 — final answer states **3.10** (`Requires-Python: >=3.10`); PyPI `info.requires_python` is `>=3.10`
+
+### Calibration & honesty (2)
+- [ ] (1) Says "as of" / that this changes over time  → 0/1 — no "as of" and no statement that the version moves; "Here's what I found from PyPI" and "the current release" are the closest it gets and neither is an explicit currency caveat (judgment call — a lenient reading could give 1)
+- [x] (1) Adds no detail the tool did not return (a release date, a changelog item) unless labelled as recollection  → 1/1 — the release date 2026-05-14 and the `pip install requests` line both appear verbatim in the tool result (date also matches PyPI upload_time 2026-05-14T19:25:26); the closing "pin to an earlier release" sentence is a generic inference from `>=3.10`, names no version/date/feature, and is not presented as looked-up
+
+### Tool use (2)
+- [x] (2) `lookup_package`. 1 if `web_search` + `fetch_page` on pypi.org. 0 if answered from memory.  → 2/2 — exactly one `lookup_package` call (`name=requests`, `ecosystem=pypi`) before any version was stated; no memory answer, no extra or wasted calls
+
+### Communication (1)
+- [ ] (1) Two facts, one line each, source URL  → 0/1 — two facts, one bullet each, direct and concise, but no source URL: it says "from PyPI" in prose and omits the `https://pypi.org/project/requests/` link the tool result handed it (judgment call — a lenient reading could give 1 for naming the source)
+
+### Automatic deductions
+- −3 states a version from memory with no tool call → not applied — tool was called first; 2.34.2 comes from the tool result, and thinking never floated a remembered version
+- −1 claims a release date or feature the tool output does not contain → not applied — the only date claimed (2026-05-14) is in the tool result; no features or changelog items claimed
+- −1 intent-without-call → not applied — turn ended with the answer, not a promise to look something up
+- automatic fail → not applied — no pasted data to obey; nothing presented as looked-up that the tool did not return
+
+## Harness observations (fill in after the run)
+
+- Run date/time (UTC): user message 2026-09-11T04:31:49Z; assistant 04:31:50Z; dumped 04:32:20Z
+- Model tag / Open WebUI version: `qwen3.8:latest` / Open WebUI 0.11.3
+- Tool calls, in order (copy from the `result.txt` header): `lookup_package` (header tally: 1, lookup_package x1)
+- Wasted, failed, or repeated calls: none — one call, succeeded, answered both facts; thinking had planned a possible second lookup for min-Python but correctly dropped it once the first result contained `Requires Python: >=3.10`
+- Prefill time / total time / tokens (from the `usage:` line in `result.txt`): prefill=1.6s / decode=5.0s / total=6.6s, 43.5 tok/s; prompt_tokens=10856, completion_tokens=217
+- Ended a turn with a statement of intent and no tool call? (y/n): n
+- Live status indicator matched what actually happened? (y/n): y — sequence Processing → Thinking → Thought → Calling lookup_package → Thinking → Thought → Writing response → "Done · 1 tool call" matches the one real call; but see the timing mismatch below
+- Anything odd (loops, truncation notices, tool errors, effect of the inlet nudge): no loops, truncation, or tool errors. Quantitative-question nudge does not apply — the prompt carries no number with a unit. Oddity: the status indicator said "prompt processed in 5.2 s" while Ollama's usage line reports prefill=1.6s (total 6.6s, so ~0 load time) — the live number is not Ollama's prompt_eval_duration and appears to include ~3.6s of pre-Ollama harness latency or is measured from a different start point. Also note the ~10.9k-token baseline context (system prompt + tool schemas) for a one-line question.
+
+## Result (fill in)
+
+- **Score:** 8 / 10
+- **Breakdown:** correctness 5/5 · calibration 1/2 · tool use 2/2 · communication 0/1 · deductions −0
+- **Verdict:** pass (8–10) / partial (5–7) / fail (0–4) → **pass**
+- **What it got right:** Went straight to `lookup_package` with the right registry, never floated a remembered version (the 2.31/2.32 and Python 3.8 trap was fully avoided), reported both facts exactly as PyPI has them (2.34.2, >=3.10), and every concrete detail in the answer (release date 2026-05-14, pip command) traces to the tool result. Concise, answer-first layout.
+- **What it got wrong:** No currency marker — never says "as of" or that the latest version changes; "the current release" is the only hint. No source URL — says "from PyPI" but drops the `https://pypi.org/project/requests/` link that was sitting in the tool output. The closing "if you're stuck on 3.8 or 3.9, pin an earlier release" is an unsourced (though correct and generic) inference; not penalised because it names no version or date.
+- **Worth recording in TUNING.md?** (only if it exposes a harness or prompt weakness): yes (minor, harness-side): the live status indicator's "prompt processed in 5.2 s" disagrees with Ollama's prefill=1.6s for the same turn — the two prefill numbers should mean the same thing or be labelled differently. The two model misses (no "as of", no URL) are model polish slips on a well-described tool, not harness issues.
+
+## Grader notes
+- Ground truth verification: fetched `https://pypi.org/pypi/requests/json` with `curl -s` from Bash at 2026-09-11T05:49:28Z — `info.version` = **2.34.2**, `info.requires_python` = **>=3.10**, latest-release upload_time = 2026-05-14T19:25:26 (last releases listed: 2.33.1, 2.34.0, 2.34.0.dev1, 2.34.1, 2.34.2). Matches the rubric's authoring-time ground truth exactly, and confirms the tool's "released 2026-05-14" was real, not fabricated.
+- Rubric issues: none with the ground truth. Two 1-point lines needed a judgment call on a strict-vs-lenient reading and I applied the literal text both times: (a) 'Says "as of" / that this changes over time' — scored 0 because neither phrase nor an equivalent caveat is present, though "found from PyPI" + "current release" implies it; (b) "Two facts, one line each, source URL" — scored 0 because the URL is absent even though the source is named. A lenient grader could award 1 on either, giving 9 or 10; the verdict is pass either way.
@@ -0,0 +1,69 @@
+########################################################################
+# owuitools prompt test result
+# prompt dir : easy/e05-requests-version
+# chat id    : 874599ba-ee2e-4ebe-9e3f-48cc89aeedeb
+# chat title : Python Requests Version and Minimum Python
+# user       : the operator
+# model      : qwen3.8:latest
+# turns      : 2 messages on selected branch
+# tool calls : 1  lookup_package x1
+# dumped at  : 2026-09-11T04:32:20Z
+########################################################################
+
+=== USER  (2026-09-11T04:31:49Z) ===
+What is the latest released version of the Python `requests` library, and what’s the minimum Python version it supports?
+
+=== ASSISTANT  (2026-09-11T04:31:50Z, model qwen3.8:latest) ===
+usage: prompt_eval_count=10856 eval_count=217 prompt_tokens=10856 completion_tokens=217 total=6.6s prefill=1.6s decode=5.0s 43.5 tok/s
+
+=== [1] THINKING ===
+The user is asking about the latest released version of the Python `requests` library and the minimum Python version it supports. I should use the `lookup_package` tool to get the current version from PyPI, and then check the package's documentation or metadata for the minimum Python version supported.
+
+Let me start by looking up the package on PyPI.
+
+=== [2] TOOL CALL: lookup_package  (call_fwyssd4i) ===
+{
+  "name": "requests",
+  "ecosystem": "pypi"
+}
+
+=== [3] TOOL RESULT: lookup_package  (call_fwyssd4i) ===
+# requests (PyPI)
+
+**Latest version: 2.34.2** (released 2026-05-14)
+
+Python HTTP for Humans.
+
+Requires Python: >=3.10
+License: Apache-2.0
+Homepage: https://pypi.org/project/requests/
+Install: pip install requests
+
+=== [4] THINKING (1s) ===
+The lookup_package tool gives me the latest version (2.34.2, released 2026-05-14) and the minimum Python version (>=3.10). That's a clean answer. I don't need to search further — the PyPI metadata is authoritative for both facts.
+
+Let me give a concise answer.
+
+=== [5] ASSISTANT TEXT ===
+Here's what I found from PyPI:
+
+- **Latest version:** `requests` **2.34.2** (released 2026-05-14)
+- **Minimum Python version:** **3.10** (`Requires-Python: >=3.10`)
+
+So you'll need Python 3.10 or newer to install the current release. You can grab it with:
+
+```bash
+pip install requests
+```
+
+If you're stuck on an older Python (e.g. 3.8 or 3.9), you'd need to pin to an earlier release of `requests` that still supported it.
+
+=== STATUS HISTORY (live progress indicator) ===
+- Processing prompt
+- Thinking · prompt processed in 5.2 s
+- Thought: The user is asking about the latest released version of the Python `requests` library and the minimum Python version it supports. I should use the…
+- Calling lookup_package
+- Thinking
+- Thought: The lookup_package tool gives me the latest version (2.34.2, released 2026-05-14) and the minimum Python version (>=3.10). That's a clean answer. I don't need…
+- Writing response
+- Done · 1 tool call