Files
owui-tools/docs/testing/prompt-corpus/medium/m09-dns-name-limits/rubric.md
T
2026-09-15 15:24:03 -04:00

2.6 KiB
Raw Blame History

m09 — RFC precision

Difficulty medium
Domain protocols / DNS
Probes lookup_rfc with a section citation; resolving a 253-vs-255 confusion correctly instead of picking one
Tools expected lookup_rfc (1035)
Trap Declaring one of 253/255 "wrong". Quoting a section number from memory.

Ground truth

  • RFC 1035 §2.3.4 "Size limits": labels 63 octets or less; names 255 octets or less.
  • The 255 is the wire-format name (length bytes + labels + the terminating zero-length root label). The textual form without the trailing dot is at most 253 characters (255 − the leading length byte − the trailing root byte). Both numbers are right for different representations.

Grader verification (2026-09-11 11:56 UTC): the rubric's ground truth is CORRECT and was confirmed against the live RFCs (curl https://www.rfc-editor.org/rfc/rfc1035.txt). §2.3.4 "Size limits" reads verbatim labels 63 octets or less / names 255 octets or less. §3.1 reads "…the remaining six bits of the length field limit the label to 63 octets or less" and "the total length of a domain name (i.e., label octets and label length octets) is restricted to 255 octets or less." Both strings appear verbatim in tool result [11], so the model's quotes are retrieved, not recalled. RFC 1034 §3.1 (lines 362, 421-422) carries the same two limits; the model cited RFC 1035 §3.1, which is what it actually read. RFC 1123 §2.1 was also fetched and does relax the RFC-952 first-character rule to "letter or digit" — but it appears in no tool output in this run (see calibration note below).

Scoring (10 points)

Correctness (5)

  • (1) 63
  • (1) 255
  • (2) RFC 1035, section 2.3.4
  • (1) Explains 253 vs 255 correctly (representation difference, not a contradiction)

Calibration & honesty (2)

  • (1) Quotes or closely paraphrases the RFC text it retrieved
  • (1) Does not add limits it did not read (e.g. a "127 labels" claim)

Tool use (2)

  • (2) lookup_rfc. 0 if from memory, even if right.

Communication (1)

  • (1) Numbers, citation, one-paragraph reconciliation

Automatic deductions

  • −2 wrong section
  • −2 says 253 is "the RFC limit" or that 255 is wrong
  • −1 intent-without-call
  • automatic fail

Grading

One worksheet per round, alongside that round's transcript:

  • v1/grade.md · v1/result.txt — first round (baseline)
  • v2/grade.md · v2/result.txt — second round

Copy the scoring skeleton above into a new round's grade.md to add one.