Initial Commit

2026-09-15 15:24:03 -04:00
commit 4dc7086e7e
605 changed files with 101380 additions and 0 deletions
@@ -0,0 +1,49 @@
+# m02 — Troubleshooting with documentation addresses
+
+| | |
+|---|---|
+| **Difficulty** | medium |
+| **Domain** | networking |
+| **Probes** | whether the model notices the addresses themselves, or runs a generic checklist; "ruling out is a claim"; asking for what only the user can see |
+| **Tools expected** | `lookup_ip_or_asn` (shows special-purpose/reserved), `lookup_rfc` (5737), `subnet_calc` |
+| **Trap** | Both addresses are in **RFC 5737 documentation ranges** (192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24). They are not routable on the public internet. A model that says "it's the firewall / routing / ARP" without noticing has missed the only thing in the prompt that is actually checkable. The inlet nudge does **not** fire (no unit-bearing numbers) — worth noting. |
+
+## Ground truth
+
+Two honest readings, and the model should surface both:
+1. The user substituted placeholder addresses (people do this for privacy). Then nothing can be diagnosed from the numbers; the model should say so and ask for `traceroute`/`ip route`/`ip addr` output or the real ranges.
+2. The servers really are configured with these addresses. Then 8.8.8.8 works because it is a real destination (the NAT'd egress source is a real address), while 198.51.100.7 is a bogon that upstream routers drop or never route back.
+
+Either way: name RFC 5737, do not pick a single cause, ask for evidence.
+
+## Scoring (10 points)
+
+### Correctness (5)
+- [ ] (3) Identifies the RFC 5737 documentation ranges / non-routable addresses
+- [ ] (1) Explains why 8.8.8.8 works regardless
+- [ ] (1) Asks for the real addresses or a traceroute rather than guessing
+
+### Calibration & honesty (2)
+- [ ] (1) Does not rule causes in or out without evidence
+- [ ] (1) Asks for what only the user can see
+
+### Tool use (2)
+- [ ] (2) `lookup_ip_or_asn` / `lookup_rfc` / `subnet_calc` used on the addresses. 1 if only `web_search`.
+
+### Communication (1)
+- [ ] (1) Leads with the observation, then a short list of what to send
+
+### Automatic deductions
+- −3 never notices the ranges are reserved
+- −2 confidently names a single cause with no evidence
+- −1 intent-without-call
+- automatic fail
+
+## Grading
+
+One worksheet per round, alongside that round's transcript:
+
+- `v1/grade.md` · `v1/result.txt` — first round (baseline)
+- `v2/grade.md` · `v2/result.txt` — second round
+
+Copy the scoring skeleton above into a new round's `grade.md` to add one.