Recommended parameters

Two models are configured here and they are tuned for different jobs. The assistant writes prose and chooses tools; the VM sub-agent writes code and reads error messages. Numbers that suit one do not suit the other.

Everything below was arrived at by measurement against the corpus in testing/; the reasoning is in TUNING.md.

The assistant

Set in Open WebUI: Workspace → Models → your model → Edit → Advanced Params.

The full set from a working deployment, not just the interesting ones — the defaults Open WebUI leaves blank are as much a part of this as the values that were changed.

Parameter	Value	Why
`num_ctx`	163840	Tool results are not truncated by Open WebUI — whatever a tool returns lands in the context verbatim. A research answer with three fetched pages is large.
`max_tokens`	32768	Room for a long answer without letting one run away.
`temperature`	0.6
`top_p`	0.95
`top_k`	20
`min_p`	0
`presence_penalty`	1	The opposite of the sub-agent's 0, and deliberately so. This model writes prose, where repetition is the failure mode; the sub-agent writes code, where reusing an identifier is correct. Do not copy one to the other.
`repeat_last_n`	256
`num_batch`	1024
`keep_alive`	`-1`	Never unload between turns. Here it is a string in Open WebUI's own field; the sub-agent sends it to Ollama directly and must send an integer.
Function Calling	Default (`native`), not Legacy	Legacy mode reformats tool definitions in a way that loses the request-body schema, which is where the enums and defaults live.

Per-model capabilities

On the same edit page, below the parameters. What matters for this project:

Capability	Set to	Why
`builtin_tools`	on	Required for the tool connection to be offered at all
`citations`	on	The tools return citation lines meant to be lifted verbatim
`status_updates`	on	Needed by `live_progress_status` and by the VM sub-agent's live view
`vision`, `file_upload`, `file_context`	on	Ordinary; `vision` is also what makes a returned plot legible to the model
`code_interpreter`, `terminal`	off	`matlab_run` and `vm_agent` already provide execution, inside sandboxes this project controls. Leaving these on gives the model a second, unsandboxed route to the same thing.
`web_search`	your call	See below

Open WebUI has its own web search, and it overlaps with web_search. If both are on the model can reach the same capability two ways, with different output formats and different citation behaviour, and which one it picks is not predictable. Pick one. The tool goes through your SearXNG and squid and returns results shaped for the citation rules in the system prompt; Open WebUI's built-in goes out on its own terms.

System prompt

Paste ../configs/assistant/system-prompt.md into the model's System Prompt field.

This matters more than any of the numbers. The tool descriptions alone do not produce the behaviour: using tools instead of recalling, checking rather than asserting, and saying "I don't know". Those come from the prompt.

Note that it is a file in the repo, not something the server serves — nothing loads it at runtime. If you change the live prompt, change the file too, or the two drift and the file stops being a record of anything.

`num_ctx` must match what Ollama actually loaded

Ollama allocates the KV cache per loaded model. Asking for a different num_ctx than the resident one makes it reload the model — on a 27B Q4_K_M that is about 18 GB across the bus, in the middle of a conversation. The value in Open WebUI, the value in OWUITOOLS_AGENT_NUM_CTX, and the value the model is actually loaded with must all be the same number.

Check what is loaded with curl -s http://172.17.0.1:11434/api/ps | jq.

The VM sub-agent

Set in configs/agent/params.env. It runs on the same Ollama and the same model, with its own sampling.

Setting	Default	Why
`OWUITOOLS_AGENT_MODEL`	`qwen3.8:latest`	Same model as the assistant, so there is one thing loaded
`OWUITOOLS_AGENT_NUM_CTX`	`163840`	Must equal the resident value — see above
`OWUITOOLS_AGENT_KEEP_ALIVE`	`-1`	Never unload. Must be an integer: the string `"-1"` is rejected with `time: missing unit in duration`
`OWUITOOLS_AGENT_TEMPERATURE`	`0.6`
`OWUITOOLS_AGENT_TOP_P`	`0.95`
`OWUITOOLS_AGENT_TOP_K`	`20`
`OWUITOOLS_AGENT_PRESENCE_PENALTY`	`0.0`	Deliberately zero. A penalty pushes the model away from tokens it has already used, and code reuses identifiers constantly. A non-zero value here produced subtly wrong variable names.
`OWUITOOLS_AGENT_NUM_PREDICT`	`16384`	One turn's output ceiling
`OWUITOOLS_AGENT_THINK`	`true`	Also accepts `low`/`medium`/`high`/`max`

Every option is sent on every request, rather than inherited from the Modelfile. That is on purpose: a Modelfile carrying an unnoticed presence_penalty is exactly the failure above, and it is invisible from the calling side.

Session bounds

Setting	Default	What it bounds
`OWUITOOLS_VM_MAX_STEPS`	`110`	Tool calls in one session. The backstop for an agent going in circles.
`OWUITOOLS_VM_SESSION_SECONDS`	`7200`	Wall-clock. The backstop for everything else.
`OWUITOOLS_VM_TASK_DRIFT`	`12`	Steps without touching the task list before it is told it is drifting
`OWUITOOLS_VM_WAIT_SECONDS`	`7000`	How long one `vm_agent` call may block

The three clocks

A vm_agent call is a single HTTP request that stays open for as long as the session runs. Three separate timeouts can end it, and the turn dies at the shortest one — usually with the work already done and only the reporting left, which is the most confusing possible failure.

Layer	Where	Value
The app	`OWUITOOLS_VM_WAIT_SECONDS` in `configs/owuitools.env`	7000
The gateway	`location = /tools/vm_agent` in `configs/services/gateway.nginx.conf`	7200s
Open WebUI	`AIOHTTP_CLIENT_TIMEOUT_TOOL_SERVER` on the container	7200

Keep the app's the smallest, so it is the app that answers — with a real result — rather than a proxy cutting the connection and leaving the model with a transport error.

Open WebUI's is not unlimited when unset. It falls back to AIOHTTP_CLIENT_TIMEOUT, which defaults to 300 seconds. This is the single most common reason a working VM session reports as a failure.

If you front Open WebUI with your own reverse proxy, that is a fourth clock. Raise it too.

Context cost

The 22 tool schemas cost roughly 9,700 tokens, paid on every turn of every conversation whether or not a tool is used.

If you do not need some of them, turn them off in ../configs/tools.toml — that removes them from the spec entirely rather than hiding them, so you stop paying for them. The electronics group is the cheapest to drop if it is not your field. See tools.md.

7.1 KiB Raw Permalink Blame History