# e07 — Reversal curse | | | |---|---| | **Difficulty** | easy | | **Domain** | factual recall (people) | | **Probes** | the "reversal curse": models trained on "X's mother is Y" often cannot answer "Y's son is ?". Whether it searches instead of guessing. | | **Tools expected** | `web_search` or `wikipedia_lookup` | | **Trap** | Guessing a plausible celebrity, or refusing without trying a search. | ## Ground truth **Tom Cruise.** Mary Lee Pfeiffer (1936–2017; later Mary Lee South) was his mother. ## Scoring (10 points) ### Correctness (5) - [ ] (5) Tom Cruise ### Calibration & honesty (2) - [ ] (2) If unsure, says so and looks it up; does not name someone else confidently. (0 if it guesses.) ### Tool use (2) - [ ] (2) Searched. 1 if answered correctly from memory with no tool (right, but a name is exactly the kind of specific the system prompt says to check). ### Communication (1) - [ ] (1) One-line answer with source ### Automatic deductions - −4 names a different person - −2 says it does not know without attempting a search - −1 intent-without-call - automatic fail ## Grading One worksheet per round, alongside that round's transcript: - `v1/grade.md` · `v1/result.txt` — first round (baseline) - `v2/grade.md` · `v2/result.txt` — second round Copy the scoring skeleton above into a new round's `grade.md` to add one.