Grant/owui-tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial Commit

Browse Source
This commit is contained in:
Grant
2026-09-15 15:24:03 -04:00
commit 4dc7086e7e
605 changed files with 101380 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/advanced-params.md
+141
View File
@@ -0,0 +1,141 @@
# Recommended parameters
Two models are configured here and they are tuned for different jobs. The assistant writes
prose and chooses tools; the VM sub-agent writes code and reads error messages. Numbers that
suit one do not suit the other.
Everything below was arrived at by measurement against the corpus in
[testing/](testing/README.md); the reasoning is in [TUNING.md](TUNING.md).
---
## The assistant
Set in Open WebUI: **Workspace → Models → your model → Edit → Advanced Params**.
The full set from a working deployment, not just the interesting ones — the defaults
Open WebUI leaves blank are as much a part of this as the values that were changed.
| Parameter | Value | Why |
|---|---|---|
| `num_ctx` | **163840** | Tool results are not truncated by Open WebUI — whatever a tool returns lands in the context verbatim. A research answer with three fetched pages is large. |
| `max_tokens` | **32768** | Room for a long answer without letting one run away. |
| `temperature` | 0.6 | |
| `top_p` | 0.95 | |
| `top_k` | 20 | |
| `min_p` | 0 | |
| `presence_penalty` | **1** | The opposite of the sub-agent's 0, and deliberately so. This model writes prose, where repetition is the failure mode; the sub-agent writes code, where reusing an identifier is correct. Do not copy one to the other. |
| `repeat_last_n` | 256 | |
| `num_batch` | 1024 | |
| `keep_alive` | `-1` | Never unload between turns. Here it is a string in Open WebUI's own field; the sub-agent sends it to Ollama directly and must send an **integer**. |
| **Function Calling** | **Default** (`native`), not Legacy | Legacy mode reformats tool definitions in a way that loses the request-body schema, which is where the enums and defaults live. |
### Per-model capabilities
On the same edit page, below the parameters. What matters for this project:
| Capability | Set to | Why |
|---|---|---|
| `builtin_tools` | on | Required for the tool connection to be offered at all |
| `citations` | on | The tools return citation lines meant to be lifted verbatim |
| `status_updates` | on | Needed by `live_progress_status` and by the VM sub-agent's live view |
| `vision`, `file_upload`, `file_context` | on | Ordinary; `vision` is also what makes a returned plot legible to the model |
| `code_interpreter`, `terminal` | **off** | `matlab_run` and `vm_agent` already provide execution, inside sandboxes this project controls. Leaving these on gives the model a second, unsandboxed route to the same thing. |
| `web_search` | your call | See below |
**Open WebUI has its own web search, and it overlaps with `web_search`.** If both are on the
model can reach the same capability two ways, with different output formats and different
citation behaviour, and which one it picks is not predictable. Pick one. The tool goes
through your SearXNG and squid and returns results shaped for the citation rules in the
system prompt; Open WebUI's built-in goes out on its own terms.
### System prompt
Paste [`../configs/assistant/system-prompt.md`](../configs/assistant/system-prompt.md) into
the model's System Prompt field.
**This matters more than any of the numbers.** The tool descriptions alone do not produce
the behaviour: using tools instead of recalling, checking rather than asserting, and saying
"I don't know". Those come from the prompt.
Note that it is a *file in the repo*, not something the server serves — nothing loads it at
runtime. If you change the live prompt, change the file too, or the two drift and the file
stops being a record of anything.
### `num_ctx` must match what Ollama actually loaded
Ollama allocates the KV cache per loaded model. Asking for a different `num_ctx` than the
resident one makes it **reload the model** — on a 27B Q4_K_M that is about 18 GB across the
bus, in the middle of a conversation. The value in Open WebUI, the value in
`OWUITOOLS_AGENT_NUM_CTX`, and the value the model is actually loaded with must all be the
same number.
Check what is loaded with `curl -s http://172.17.0.1:11434/api/ps | jq`.
---
## The VM sub-agent
Set in `configs/agent/params.env`. It runs on the **same Ollama and the same model**, with
its own sampling.
| Setting | Default | Why |
|---|---|---|
| `OWUITOOLS_AGENT_MODEL` | `qwen3.8:latest` | Same model as the assistant, so there is one thing loaded |
| `OWUITOOLS_AGENT_NUM_CTX` | `163840` | Must equal the resident value — see above |
| `OWUITOOLS_AGENT_KEEP_ALIVE` | `-1` | Never unload. Must be an **integer**: the string `"-1"` is rejected with `time: missing unit in duration` |
| `OWUITOOLS_AGENT_TEMPERATURE` | `0.6` | |
| `OWUITOOLS_AGENT_TOP_P` | `0.95` | |
| `OWUITOOLS_AGENT_TOP_K` | `20` | |
| `OWUITOOLS_AGENT_PRESENCE_PENALTY` | `0.0` | **Deliberately zero.** A penalty pushes the model away from tokens it has already used, and code reuses identifiers constantly. A non-zero value here produced subtly wrong variable names. |
| `OWUITOOLS_AGENT_NUM_PREDICT` | `16384` | One turn's output ceiling |
| `OWUITOOLS_AGENT_THINK` | `true` | Also accepts `low`/`medium`/`high`/`max` |
Every option is sent on every request, rather than inherited from the Modelfile. That is on
purpose: a Modelfile carrying an unnoticed `presence_penalty` is exactly the failure above,
and it is invisible from the calling side.
### Session bounds
| Setting | Default | What it bounds |
|---|---|---|
| `OWUITOOLS_VM_MAX_STEPS` | `110` | Tool calls in one session. The backstop for an agent going in circles. |
| `OWUITOOLS_VM_SESSION_SECONDS` | `7200` | Wall-clock. The backstop for everything else. |
| `OWUITOOLS_VM_TASK_DRIFT` | `12` | Steps without touching the task list before it is told it is drifting |
| `OWUITOOLS_VM_WAIT_SECONDS` | `7000` | How long one `vm_agent` call may block |
---
## The three clocks
A `vm_agent` call is a single HTTP request that stays open for as long as the session runs.
Three separate timeouts can end it, and **the turn dies at the shortest one** — usually with
the work already done and only the reporting left, which is the most confusing possible
failure.
| Layer | Where | Value |
|---|---|---|
| The app | `OWUITOOLS_VM_WAIT_SECONDS` in `configs/owuitools.env` | **7000** |
| The gateway | `location = /tools/vm_agent` in `configs/services/gateway.nginx.conf` | **7200s** |
| Open WebUI | `AIOHTTP_CLIENT_TIMEOUT_TOOL_SERVER` on the container | **7200** |
Keep the app's the **smallest**, so it is the app that answers — with a real result — rather
than a proxy cutting the connection and leaving the model with a transport error.
**Open WebUI's is not unlimited when unset.** It falls back to `AIOHTTP_CLIENT_TIMEOUT`,
which defaults to **300 seconds**. This is the single most common reason a working VM
session reports as a failure.
If you front Open WebUI with your own reverse proxy, that is a fourth clock. Raise it too.
---
## Context cost
The 22 tool schemas cost roughly **9,700 tokens**, paid on every turn of every conversation
whether or not a tool is used.
If you do not need some of them, turn them off in
[`../configs/tools.toml`](../configs/tools.toml) — that removes them from the spec entirely
rather than hiding them, so you stop paying for them. The electronics group is the cheapest
to drop if it is not your field. See [tools.md](tools.md).