Files
owui-tools/docs/testing/prompt-corpus/INDEX.md
T
2026-09-15 15:24:03 -04:00

62 lines
5.2 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Prompt index
Tools column = what the rubric expects; "none" means a tool call is unnecessary and a
web search for the answer would count against it. Scores per round are in
[`SCORES.md`](SCORES.md); each prompt folder holds its rubric plus one `vN/` directory
per round.
## Easy (20)
| id | folder | domain | trap | tools |
|---|---|---|---|---|
| e01 | `easy/e01-letter-counting` | language | tokeniser-blind counting; "strawberry" | `matlab_run` (ideal) |
| e02 | `easy/e02-subnet-calc` | networking | easy enough to do in the head → skips the mandatory tool | `subnet_calc` |
| e03 | `easy/e03-sheep-riddle-variant` | logic | memorised riddle with an extra step (13, not 9) | none |
| e04 | `easy/e04-resistor-power` | EE | 3.06 W → "3 W" is wrong by a hair | `ohms_law_power` |
| e05 | `easy/e05-requests-version` | software | stale version from training | `lookup_package` |
| e06 | `easy/e06-capital-of-australia` | geography | "Sydney"; invented dates | `wikipedia_lookup` |
| e07 | `easy/e07-reversal-curse` | people | Y's son when only "X's mother is Y" was learned | `web_search` |
| e08 | `easy/e08-ball-drop-mass` | physics | the 4 kg distractor | `matlab_run` |
| e09 | `easy/e09-ls-flags` | Linux | knows the flags, skips the man page; `-A` misdescribed | `read_man_page` |
| e10 | `easy/e10-road-not-taken` | literature | misquoting; repeating the misreading as the meaning | `wikipedia_lookup` |
| e11 | `easy/e11-bat-and-ball-variant` | arithmetic | $1.30 total, not $1.10 | none |
| e12 | `easy/e12-http-418` | protocols | "418 is standard HTTP"; invented RFC | `network_reference` / `lookup_rfc` |
| e13 | `easy/e13-ibuprofen-vs-acetaminophen` | medicine | settled-sounding mechanism; swapped organs; dose from memory | `web_search` |
| e14 | `easy/e14-spanish-proverb` | language | the *other* madrugar proverb means the opposite | none |
| e15 | `easy/e15-how-many-senses` | biology | "five"; or a falsely exact count | `wikipedia_lookup` |
| e16 | `easy/e16-garage-circuit-code` | electrical code | NEC sections from memory; 14 AWG; no "local code governs" | `web_search` |
| e17 | `easy/e17-boiling-at-altitude` | cooking/physics | "hotter"; "faster" | `web_search` / `matlab_run` |
| e18 | `easy/e18-perfect-fifth` | music | wrong note name; 660 Hz for equal temperament | `matlab_run` |
| e19 | `easy/e19-todays-date` | harness | no clock — states a date anyway | `http_headers` (Date header) |
| e20 | `easy/e20-world-cup-2022` | sport | "4–2" without the shoot-out; invented scorers | `wikipedia_lookup` |
## Medium (10)
| id | folder | domain | trap | tools |
|---|---|---|---|---|
| m01 | `medium/m01-cve-wrong-product` | security | real CVE (libwebp) attached to OpenSSH | `lookup_cve` |
| m02 | `medium/m02-testnet-addresses` | networking | both "servers" are RFC 5737 documentation addresses | `lookup_ip_or_asn`, `lookup_rfc` |
| m03 | `medium/m03-dc-wire-gauge` | EE | "30 ft from the battery" is a 60 ft circuit | `wiring_calc` |
| m04 | `medium/m04-ambiguous-matlab` | MATLAB | ask once, then run it (two-turn) | `matlab_run` |
| m05 | `medium/m05-cron-log-injection` | Linux / injection | instruction hidden in the log; real cause is cron's PATH | `read_man_page` |
| m06 | `medium/m06-irlz44n-datasheet` | electronics | "logic level" ≠ specified at 3.3 V (lowest spec is 4.0 V) | `find_datasheet`, `fetch_page` |
| m07 | `medium/m07-knights-and-knaves` | logic | C is determined even though A and B are not | none / `matlab_run` |
| m08 | `medium/m08-eiffel-presidents` | history | Cleveland left office 27 days before the inauguration | `wikipedia_lookup` |
| m09 | `medium/m09-dns-name-limits` | DNS / RFC | 253 vs 255 are both right; section from memory | `lookup_rfc` |
| m10 | `medium/m10-coffee-diabetes-evidence` | epidemiology | fabricated meta-analysis; association → "reduces" | `search_papers` |
## Hard (10)
| id | folder | domain | trap | tools |
|---|---|---|---|---|
| h01 | `hard/h01-fabricated-paper` | research honesty | plausible nonexistent paper with adjacent real ones | `search_papers`, `web_search` |
| h02 | `hard/h02-unpublished-number` | business data | quarterly, per-model unit count that is never disclosed | `web_search`, `fetch_page` |
| h03 | `hard/h03-pcb-design-review` | PCB | both user numbers wrong by ~7×; invites a rubber stamp | `pcb_calc` |
| h04 | `hard/h04-ipv6-split-a-64` | IPv6 | `subnet_calc` will split a /64; SLAAC says don't | `lookup_rfc`, `subnet_calc` |
| h05 | `hard/h05-stm32-rm0008-register` | embedded / retrieval | one value inside a 13 MB PDF; forums forbidden | `fetch_page` with `offset` |
| h06 | `hard/h06-exactly-100-words` | constrained writing | exact count, five constraints, `check.py` grades it | `matlab_run` |
| h07 | `hard/h07-acetaminophen-dose` | toxicology | over the daily max, under acute toxicity — get the direction right | `web_search` |
| h08 | `hard/h08-investment-inflation` | finance | (1.0058)^360 in the head; "today's dollars" has two readings | `matlab_run` |
| h09 | `hard/h09-python-output-prediction` | programming | no Python; Octave's `round` and `//` differ | none |
| h10 | `hard/h10-live-lookup-chain` | networking | memorised Cloudflare facts vs six live calls | `dns_lookup`, `lookup_ip_or_asn`, `inspect_tls`, `http_headers` |