Grant/owui-tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial Commit

Browse Source
This commit is contained in:
Grant
2026-09-15 15:24:03 -04:00
commit 4dc7086e7e
605 changed files with 101380 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/testing/prompt-corpus/easy/e18-perfect-fifth/v3/grade.md
+64
View File
@@ -0,0 +1,64 @@
# e18 — Music theory with a calculation · round v3
> Grading worksheet for round **v3**. The ground truth, the trap and the
> scoring criteria live in [`../rubric.md`](../rubric.md) — that file is the
> master and is not edited per round. The transcript being graded is
> [`result.txt`](result.txt).
## Scoring (10 points)
### Correctness (5)
- [x] (1) 7 → 1/1 — "A perfect fifth spans **7 semitones**"
- [x] (1) E5 → 1/1 — "the note is **E5**"; neither A5 nor E4 is entertained
- [x] (2) 659.2–659.3 Hz → 2/2 — **659.255 Hz**, from the tool's 659.2551
- [x] (1) 660 Hz for just intonation → 1/1 — **660.000 Hz** from the 3:2 ratio, correctly identified as the just value
### Calibration & honesty (2)
- [x] (1) States the tuning assumption (A4 = 440, equal temperament) explicitly → 1/1 — "440 × 2^(7/12)" under a "12-tone equal temperament" heading; "A just perfect fifth is the simple ratio **3:2**"
- [x] (1) Cents difference, if given, is right (~2 cents) — or not given → 1/1 — "0.745 Hz higher — roughly 2 cents sharper"; the tool computed 1.955 cents. Direction ("wider") correct
### Tool use (2)
- [x] (2) `matlab_run` for the frequency. 1 if hand-calculated with the formula shown. 0 if just asserted. → 2/2 — one scratch call computing both frequencies, the Hz difference and the cents difference; every figure in the answer is from it
### Communication (1)
- [x] (1) Four short answers → 1/1 — four bold headings in the order asked, each answered in one line, then one sentence on the difference
### Automatic deductions
- −2 wrong note name → not applied — E5
- −2 equal-temperament frequency off by more than 1 Hz → not applied — 659.255 vs 659.2551
- −1 intent-without-call → not applied — "I'll verify with a tool" is followed by the call
- automatic fail → not applied
## Harness observations (fill in after the run)
- Run date/time (UTC): 2026-09-11T23:13:27Z (dumped 23:14:08Z)
- Model tag / Open WebUI version: qwen3.8:latest / Open WebUI 0.11.3
- Tool calls, in order (copy from the `result.txt` header): 1 — `matlab_run` x1 (`purpose: "scratch"`)
- Wasted, failed, or repeated calls: none
- Prefill time / total time / tokens (from the `usage:` line in `result.txt`): prefill=2.3s / total=8.2s (decode=5.8s, 38.4 tok/s) / prompt_eval_count=9860, completion_tokens=223
- Ended a turn with a statement of intent and no tool call? (y/n): n
- Live status indicator matched what actually happened? (y/n): y — `Done · 1 tool call · prefill 5.1 s for 12,857 tokens · 495 generated`
- Anything odd (loops, truncation notices, tool errors, effect of the inlet nudge): nothing. The nudge fires (440 Hz). Thinking pre-computes all four answers correctly, then verifies. One odd phrase in the answer — "(this is the familiar 'E5' reference pitch)" — is filler, not an error. Shorter than v2 (223 vs 438 completion tokens): the 700-vs-701.955-cent explanation is gone, replaced by one sentence on why a just fifth sounds "wider… with no beating"
### Change 54 checks (this round's changes — fill in even when they did not fire)
- Did a tool result end with a `Sources:` line? (y/n): n — calculator
- Did the answer carry a URL? (y/n) — and if the tool offered one and the answer dropped it, say so: n — none expected
- Did `[Appears in none of these results: …]` fire? Was it acted on? (n/a if no search): n/a
- Did a `search=` jump report `inside the section '…'`? Did it name the right subject?: n/a
- Did a `search=` miss return the opening of the document? Did the model keep using the source?: n/a
## Result (fill in)
- **Score:** 10 / 10
- **Breakdown:** correctness 5/5 · calibration 2/2 · tool use 2/2 · communication 1/1 · deductions −0
- **Verdict:** pass
- **What it got right:** all four answers, both traps avoided, every number from the tool including the cents difference, assumptions stated, four short answers
- **What it got wrong:** nothing
- **Change from the previous round:** **10 → 10 (0).** Same tool call shape (frequencies plus cents in one script), same answers, about half the length. The pattern v2 noted — a secondary quantity put into the scratch script stops being wrong in prose — holds for the third round on the cents figure
- **Worth recording in TUNING.md?** (only if it exposes a harness or prompt weakness): no
## Grader notes
- Ground truth verification: 440 × 2^(7/12) = 659.2551 Hz; 440 × 1.5 = 660 Hz; 1200·log₂(660/659.2551) = 1.955 cents. Matches rubric, tool and answer
- Rubric issues: none
docs/testing/prompt-corpus/easy/e18-perfect-fifth/v3/result.txt
+64
View File
@@ -0,0 +1,64 @@
########################################################################
# owuitools prompt test result
# prompt dir : easy/e18-perfect-fifth (round v3)
# chat id : f9bacbb0-5c15-49ec-a874-65e14b457302
# chat title : Semitone Interval and Frequency
# user : the operator
# model : qwen3.8:latest
# turns : 2 messages on selected branch
# tool calls : 1 matlab_run x1
# dumped at : 2026-09-11T23:14:08Z
########################################################################
=== USER (2026-09-11T23:13:27Z) ===
How many semitones are in a perfect fifth? What note is a perfect fifth above A4 (440 Hz), and what's its frequency in 12-tone equal temperament? How would that differ in just intonation?
=== ASSISTANT (2026-09-11T23:13:27Z, model qwen3.8:latest) ===
usage: prompt_eval_count=9860 eval_count=223 prompt_tokens=9860 completion_tokens=223 total=8.2s prefill=2.3s decode=5.8s 38.4 tok/s
=== [1] THINKING ===
The perfect fifth is 7 semitones. A perfect fifth above A4 (440 Hz) is E5. In 12-TET, E5 = 440 × 2^(7/12) ≈ 659.255 Hz. In just intonation, the perfect fifth is a ratio of 3:2, so E5 = 440 × 3/2 = 660 Hz.
I'll verify with a tool.
=== [2] TOOL CALL: matlab_run (call_4wla2u7k) ===
{
"code": "f0 = 440;\nf_tet = f0 * 2^(7/12);\nf_just = f0 * 3/2;\nfprintf('12-TET (x2^(7/12)): %.4f Hz\\n', f_tet);\nfprintf('Just (x3/2): %.4f Hz\\n', f_just);\nfprintf('Difference: %.4f Hz (%.3f cents)\\n', f_just - f_tet, 1200*log2(f_just/f_tet));",
"purpose": "scratch"
}
=== [3] TOOL RESULT: matlab_run (call_4wla2u7k) ===
# Scratch run OK (0.43s)
## Output
```
12-TET (x2^(7/12)): 659.2551 Hz
Just (x3/2): 660.0000 Hz
Difference: 0.7449 Hz (1.955 cents)
```
=== [4] THINKING (0s) ===
I'm ready to answer the question.
=== [5] ASSISTANT TEXT ===
**Semitones:** A perfect fifth spans **7 semitones**.
**Note above A4:** A perfect fifth above A is **E**, so the note is **E5**.
**Frequency in 12-tone equal temperament:**
E5 = 440 × 2^(7/12) ≈ **659.255 Hz** (this is the familiar "E5" reference pitch).
**In just intonation:**
A just perfect fifth is the simple ratio **3:2**, so:
E5 = 440 × 3/2 = **660.000 Hz** exactly.
**The difference:** Just intonation's E5 is about **0.745 Hz higher** than the equal-tempered one — roughly **2 cents** sharper, which is why a pure 3:2 fifth sounds slightly wider (and sweeter, with no beating) than the tempered fifth that lets you modulate freely between keys.
=== STATUS HISTORY (live progress indicator) ===
- Processing prompt
- Thinking · first token after 5.3 s
- Thought: The perfect fifth is 7 semitones. A perfect fifth above A4 (440 Hz) is E5. In 12-TET, … of 3:2, so E5 = 440 × 3/2 = 660 Hz. I'll verify with a tool.
- Calling matlab_run
- Thinking
- Thought: I'm ready to answer the question.
- Writing response
- Done · 1 tool call · prefill 5.1 s for 12,857 tokens · 495 generated