Initial Commit

2026-09-15 15:24:03 -04:00
commit 4dc7086e7e
605 changed files with 101380 additions and 0 deletions
@@ -0,0 +1,54 @@
+# e02 — IPv4 subnet arithmetic · round v1
+
+> Grading worksheet for round **v1**. The ground truth, the trap and the
+> scoring criteria live in [`../rubric.md`](../rubric.md) — that file is the
+> master and is not edited per round. The transcript being graded is
+> [`result.txt`](result.txt).
+
+## Scoring (10 points)
+
+### Correctness (5)
+- [x] (1) Network address correct  → 1/1 — answer gives 10.42.7.128 (verified with python3 ipaddress)
+- [x] (1) Broadcast correct  → 1/1 — answer gives 10.42.7.159
+- [x] (2) Usable range correct at both ends  → 2/2 — 10.42.7.129 – 10.42.7.158, both ends match
+- [x] (1) 30 usable hosts  → 1/1 — "Usable hosts | 30", and the prose repeats "30 usable hosts"
+
+### Calibration & honesty (2)
+- [x] (1) Mask or prefix restated correctly (no contradiction between /27 and the mask it names)  → 1/1 — header line says "/27 (netmask 255.255.255.224)"; prose "32 total addresses (2⁵)" is consistent with /27
+- [x] (1) No hedging language on a deterministic result  → 1/1 — every value stated flatly; no "should be", "approximately", or "I think"
+
+### Tool use (2)
+- [x] (2) `subnet_calc` was called. 0 if computed mentally, even if every number is right — the rule exists because the mental version is wrong often enough.  → 2/2 — first thought was "Let me use the subnet_calc tool"; one call with `{"network": "10.42.7.130/27"}`, answer built from the tool result, no mental arithmetic before the call
+
+### Communication (1)
+- [x] (1) Four values, clearly labelled, first thing in the answer  → 1/1 — one-line header naming the subnet, then a 4-row labelled table with exactly the four requested values; the explanatory paragraph comes after
+
+### Automatic deductions
+- −2 any wrong address  → not applied — all four addresses correct
+- −1 "32 hosts"  → not applied — model says "32 total addresses … = 30 usable hosts", which is the correct distinction, not the slip
+- −1 intent-without-call  → not applied — the only turn ends with the answer, tool was actually called
+- automatic fail  → not applied — no pasted data, nothing presented as looked-up that was not in tool output (the RFC 1918 remark comes from the tool's "Scope: private (RFC 1918 / ULA)" line)
+
+## Harness observations (fill in after the run)
+
+- Run date/time (UTC): 2026-09-11T04:30:00Z (dumped 04:30:27Z)
+- Model tag / Open WebUI version: qwen3.8:latest / Open WebUI 0.11.3
+- Tool calls, in order (copy from the `result.txt` header): subnet_calc (1 call total: `subnet_calc x1`)
+- Wasted, failed, or repeated calls: none — single call, returned cleanly, used directly
+- Prefill time / total time / tokens (from the `usage:` line in `result.txt`): prefill=1.6s / decode=4.3s / total=5.9s; prompt_tokens=10974, completion_tokens=228, 53.2 tok/s
+- Ended a turn with a statement of intent and no tool call? (y/n): n
+- Live status indicator matched what actually happened? (y/n): y — Processing prompt → Thinking → Thought (quoted) → Calling subnet_calc → Thinking → Thought (quoted) → Writing response → "Done · 1 tool call"; sequence and count match the transcript exactly
+- Anything odd (loops, truncation notices, tool errors, effect of the inlet nudge): nothing odd. Quantitative-question nudge does NOT fire on this prompt (checked the `_MEASUREMENT` regex in `owui-functions/quantitative_question_nudge.py` against prompt.txt: no match — "/27" is not a number+unit), so the tool call was driven by the system prompt's subnetting rule alone, which is the behaviour this prompt probes. Minor: status line says "prompt processed in 2.1 s" while `usage:` says prefill=1.6s — wall-clock vs Ollama prompt_eval_duration; not a mismatch of events, just two different clocks. Tool output included an unsolicited reverse-DNS/RFC 2317 note that the model correctly left out of the answer.
+
+## Result (fill in)
+
+- **Score:** 10 / 10
+- **Breakdown:** correctness 5/5 · calibration 2/2 · tool use 2/2 · communication 1/1 · deductions −0
+- **Verdict:** pass
+- **What it got right:** Everything. Called `subnet_calc` immediately without attempting the arithmetic in its head, and reported network 10.42.7.128, broadcast 10.42.7.159, usable range 10.42.7.129 – 10.42.7.158, 30 usable hosts, netmask 255.255.255.224 — all verified. Table-first layout, concise, no hedging, and the closing note ("32 total addresses … minus the network and broadcast = 30 usable hosts"; 10.42.7.130 is a valid host, RFC 1918 private) is correct and sourced from the tool output.
+- **What it got wrong:** nothing material.
+- **Worth recording in TUNING.md?** (only if it exposes a harness or prompt weakness): no — clean run; the system-prompt subnetting rule, the `subnet_calc` schema, and the status indicator all behaved as designed.
+
+## Grader notes
+- Ground truth verification: reproduced with `python3 -c "import ipaddress; ipaddress.ip_network('10.42.7.130/27', strict=False)"` on 2026-09-11 05:49 UTC — network 10.42.7.128/27, mask 255.255.255.224, broadcast 10.42.7.159, hosts 10.42.7.129–10.42.7.158, 30 usable, 32 total, 10.42.7.130 is a host address, block is private. Rubric ground truth is correct as written.
+- Rubric issues: none. The rubric does not say whether the quantitative nudge applies; I checked the filter regex directly and recorded that it does not fire on this prompt.
@@ -0,0 +1,63 @@
+########################################################################
+# owuitools prompt test result
+# prompt dir : easy/e02-subnet-calc
+# chat id    : 9f90c4b1-c741-416a-89a0-16c26c529dc5
+# chat title : Network Configuration
+# user       : the operator
+# model      : qwen3.8:latest
+# turns      : 2 messages on selected branch
+# tool calls : 1  subnet_calc x1
+# dumped at  : 2026-09-11T04:30:27Z
+########################################################################
+
+=== USER  (2026-09-11T04:30:00Z) ===
+What are the network address, broadcast address, usable host range and number of usable hosts for 10.42.7.130/27?
+
+=== ASSISTANT  (2026-09-11T04:30:00Z, model qwen3.8:latest) ===
+usage: prompt_eval_count=10974 eval_count=228 prompt_tokens=10974 completion_tokens=228 total=5.9s prefill=1.6s decode=4.3s 53.2 tok/s
+
+=== [1] THINKING ===
+The user is asking about subnet details for 10.42.7.130/27. Let me use the subnet_calc tool.
+
+=== [2] TOOL CALL: subnet_calc  (call_v4neb7ix) ===
+{
+  "network": "10.42.7.130/27"
+}
+
+=== [3] TOOL RESULT: subnet_calc  (call_v4neb7ix) ===
+# 10.42.7.128/27
+
+Network address:  10.42.7.128
+Broadcast:        10.42.7.159
+Netmask:          255.255.255.224  (/27)
+Wildcard mask:    0.0.0.31
+Total addresses:  32
+Usable hosts:     30
+Usable range:     10.42.7.129 - 10.42.7.158
+Scope:            private (RFC 1918 / ULA)
+Reverse DNS zone: 128.7.42.10.in-addr.arpa  [that is the PTR name of the network address itself. A /27 is not on an octet boundary, so it has no single reverse zone - delegation needs RFC 2317 style CNAMEs.]
+
+=== [4] THINKING (0s) ===
+The tool has already computed everything. Let me present it clearly.
+
+=== [5] ASSISTANT TEXT ===
+For **10.42.7.130/27** (netmask 255.255.255.224):
+
+| Item | Value |
+|------|-------|
+| **Network address** | 10.42.7.128 |
+| **Broadcast address** | 10.42.7.159 |
+| **Usable host range** | 10.42.7.129 – 10.42.7.158 |
+| **Usable hosts** | 30 |
+
+A /27 gives 32 total addresses (2⁵), minus the network and broadcast addresses = **30 usable hosts**. The address 10.42.7.130 falls within this block, so it's a valid host address (not the network or broadcast). It's in the RFC 1918 private range.
+
+=== STATUS HISTORY (live progress indicator) ===
+- Processing prompt
+- Thinking · prompt processed in 2.1 s
+- Thought: The user is asking about subnet details for 10.42.7.130/27. Let me use the subnet_calc tool.
+- Calling subnet_calc
+- Thinking
+- Thought: The tool has already computed everything. Let me present it clearly.
+- Writing response
+- Done · 1 tool call