129 lines
5.5 KiB
Markdown
129 lines
5.5 KiB
Markdown
# OWUI Tools
|
|
|
|
A tool server for [Open WebUI](https://github.com/open-webui/open-webui), and a VM
|
|
sub-agent that can test before claiming that everything works.
|
|
|
|
A local assistant can only reason. It cannot *check*. This tool server allows the LLM to
|
|
ground its answer in fact: **22 tools** for looking things up and working things out, and one more
|
|
that hands a task to a second model driving a **real, throwaway virtual machine** which
|
|
can write the script, run it, read the actual errors, fix them, and loop until it works.
|
|
|
|
Everything runs on your own hardware. Nothing leaves the machine except the web requests a
|
|
tool makes on your behalf.
|
|
|
|
> It speaks **OpenAPI**, which is what Open WebUI consumes natively.
|
|
|
|
```sh
|
|
cp configs/owuitools.env.example configs/owuitools.env
|
|
sed -i "s|^OWUITOOLS_AUTH_TOKEN=.*|OWUITOOLS_AUTH_TOKEN=$(openssl rand -hex 32)|" \
|
|
configs/owuitools.env
|
|
docker compose up -d --build
|
|
```
|
|
|
|
Then register it in Open WebUI and paste the system prompt -
|
|
**[docs/install-ubuntu-desktop.md](docs/install-ubuntu-desktop.md)** walks through the whole
|
|
setup process on a clean machine.
|
|
|
|
## What it gives the assistant
|
|
|
|
| Group | Tools |
|
|
|---|---|
|
|
| **Web and research** | `web_search`, `fetch_page`, `wikipedia_lookup`, `search_stackexchange`, `search_papers` |
|
|
| **IT and reference** | `lookup_rfc`, `lookup_cve`, `read_man_page`, `lookup_package` |
|
|
| **Networking** | `subnet_calc`, `dns_lookup`, `inspect_tls`, `http_headers`, `lookup_ip_or_asn`, `network_reference` |
|
|
| **Electronics** | `ohms_law_power`, `component_calc`, `pcb_calc`, `wiring_calc`, `find_datasheet` |
|
|
| **Execution** | `matlab_run` (Octave, sandboxed), `vm_agent` (a real machine) |
|
|
|
|
Most need no setup at all. Turn any of them off in
|
|
[`configs/tools.toml`](configs/tools.toml) - that removes them from the spec rather than
|
|
hiding them, so you stop paying their schema tokens on every turn. Full list and what each
|
|
needs: **[docs/tools.md](docs/tools.md)**.
|
|
|
|
## Configuring it
|
|
|
|
Everything you need to edit is in **[`configs/`](configs/)** - the stack's settings,
|
|
tool switches, the assistant's system prompt, and the sub-agent's prompts, parameters and
|
|
tool definitions. Nothing else in the tree needs changing to run this.
|
|
|
|
See **[docs/configuration.md](docs/configuration.md)**.
|
|
|
|
## Isolation
|
|
|
|
The tool server fetches URLs chosen by a language model. It assumes the model may be
|
|
persuaded to request something it should not.
|
|
|
|
- The app container has **no default route**. Its network namespace only permits traffic
|
|
to squid and valkey, plus inbound requests from its gateway.
|
|
- **squid blocks private address space**.
|
|
- The filter **verifies its rules** before passing its healthcheck, so downstream services
|
|
do not rely on a filter that failed to apply.
|
|
- Every container drops all capabilities, runs read-only where possible, and has memory
|
|
and process limits.
|
|
|
|
The VM sub-agent adds a second boundary: the guest has internet access but no route to your
|
|
network. This is enforced primarily by a systemd address filter. A firewall alone fails
|
|
open after `nft flush ruleset`. `tests/vm_host_asserts.sh` tests the isolation by disabling
|
|
each layer in turn and running the same probe against the remaining layers.
|
|
|
|
Details, including what the isolation does **not** prevent:
|
|
**[docs/vm-agent/overview.md](docs/vm-agent/overview.md)**.
|
|
|
|
## The VM sub-agent
|
|
|
|
`vm_agent` starts a throwaway Ubuntu or Windows machine and hands it to a second model with
|
|
about thirty tools - a shell, a file editor, a screen it can view and click on, and a lookup
|
|
suite. It returns a report with facts assembled by the trusted side.
|
|
|
|
The machine is destroyed afterwards. The project folder survives on a dataset with a
|
|
quota and can be published to Gitea as a release.
|
|
|
|
Setup takes more work than the rest of the stack. Follow
|
|
**[docs/setup/vm-agent.md](docs/setup/vm-agent.md)**.
|
|
|
|
## Documentation
|
|
|
|
- **[docs/](docs/README.md)** - installation, configuration, tool details, recommended
|
|
parameters and filters.
|
|
- **[docs/operating.md](docs/operating.md)** - logs, rate limits, caching, safe restarts
|
|
and troubleshooting.
|
|
- **[logs/](logs/README.md)** - each component writes its own log file here, giving you
|
|
one place to check instead of running nine `docker compose logs` commands.
|
|
- **[docs/deployment/](docs/deployment/README.md)** - compose files for Ollama, Open WebUI
|
|
and Gitea, the three services that run outside this repo.
|
|
- **[docs/TUNING.md](docs/TUNING.md)** - the change log. Each entry records one change,
|
|
what it was meant to fix and the measured result. It is long, but the most useful
|
|
reference if you plan to modify the project.
|
|
- **[docs/testing/](docs/testing/README.md)** - the test corpus used to measure tuning:
|
|
forty graded prompts with rubrics, transcripts and scores.
|
|
|
|
## Requirements
|
|
|
|
Docker, an Ollama instance with a model, and Open WebUI.
|
|
|
|
The optional VM sub-agent also needs KVM, a ZFS pool and about 50 GB for base images.
|
|
The tool removes itself when the VM stack is absent.
|
|
|
|
Developed against Open WebUI 0.11.3 and `qwen3.8:latest` with a 160k context window.
|
|
|
|
## Checking it works
|
|
|
|
```sh
|
|
./verify.py # what is configured, disabled or broken
|
|
```
|
|
|
|
This prints one line per tool, writes a full log to `logs/verify.log`, and creates a
|
|
separate `logs/verify-error-<tool>.log` for each tool that fails. See
|
|
[docs/verifying.md](docs/verifying.md).
|
|
|
|
## Running the tests
|
|
|
|
```sh
|
|
./tests/smoke.sh # every tool through the real gateway
|
|
|
|
# Unit tests, no container needed
|
|
for test_file in tests/test_*.py; do
|
|
python3 "$test_file"
|
|
done
|
|
|
|
sudo ./tests/vm_host_asserts.sh # VM isolation tests
|
|
``` |