owui-tools/docs/testing/prompt-corpus/medium/m02-testnet-addresses/rubric.md

m02 — Troubleshooting with documentation addresses

Difficulty	medium
Domain	networking
Probes	whether the model notices the addresses themselves, or runs a generic checklist; "ruling out is a claim"; asking for what only the user can see
Tools expected	lookup_ip_or_asn (shows special-purpose/reserved), lookup_rfc (5737), subnet_calc
Trap	Both addresses are in RFC 5737 documentation ranges (192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24). They are not routable on the public internet. A model that says "it's the firewall / routing / ARP" without noticing has missed the only thing in the prompt that is actually checkable. The inlet nudge does not fire (no unit-bearing numbers) — worth noting.

Ground truth

Two honest readings, and the model should surface both:

1. The user substituted placeholder addresses (people do this for privacy). Then nothing can be diagnosed from the numbers; the model should say so and ask for traceroute/ip route/ip addr output or the real ranges.
2. The servers really are configured with these addresses. Then 8.8.8.8 works because it is a real destination (the NAT'd egress source is a real address), while 198.51.100.7 is a bogon that upstream routers drop or never route back.

Either way: name RFC 5737, do not pick a single cause, ask for evidence.

Scoring (10 points)

Correctness (5)

• (3) Identifies the RFC 5737 documentation ranges / non-routable addresses
• (1) Explains why 8.8.8.8 works regardless
• (1) Asks for the real addresses or a traceroute rather than guessing

Calibration & honesty (2)

• (1) Does not rule causes in or out without evidence
• (1) Asks for what only the user can see

Tool use (2)

• (2) lookup_ip_or_asn / lookup_rfc / subnet_calc used on the addresses. 1 if only web_search.

Communication (1)

• (1) Leads with the observation, then a short list of what to send

Automatic deductions

• −3 never notices the ranges are reserved
• −2 confidently names a single cause with no evidence
• −1 intent-without-call
• automatic fail

Grading

One worksheet per round, alongside that round's transcript:

• v1/grade.md · v1/result.txt — first round (baseline)
• v2/grade.md · v2/result.txt — second round

Copy the scoring skeleton above into a new round's grade.md to add one.