Grant/owui-tools
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Initial Commit

Browse Source
This commit is contained in:
Grant
2026-09-15 15:24:03 -04:00
commit 4dc7086e7e
605 changed files with 101380 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/deployment/README.md
+68
View File
@@ -0,0 +1,68 @@
# The services outside this repo
`docker-compose.yml` in the project root brings up everything the tools need *inside* the
security boundary: the gateway, the guarded namespace, squid, SearXNG, valkey, the Octave
sandbox, and the VM sub-agent's own containers. You do not need anything here for those.
These three run outside it and have to exist first.
| File | Service | Needed by | Optional? |
|---|---|---|---|
| `ollama.compose.yml` | Ollama | everything | No — this is the model |
| `open-webui.compose.yml` | Open WebUI | everything | No — this is the front end |
| `gitea.compose.yml` | Gitea | `vm_agent` only | Yes — without it a session still runs and keeps its files, it just returns no link |
Each file is the working configuration from a real deployment with the secrets and domain
names replaced. Read the header of each before running it; the parts that matter to
owuitools are called out there, and in two places getting them wrong produces a failure
that does not look like a configuration problem.
## Project names are pinned, on purpose
Each file sets its own `name:`. Compose otherwise derives a project name from the directory
the file sits in, so running these as `docker compose -f docs/deployment/...` would produce
a project called `deployment` and a network called `deployment_default`.
That matters because the tool server joins Open WebUI's network **by name**, expecting
`openwebui_default`. Without the pin, following the install guide in order fails at the next
step with *"network declared as external, but could not be found"*. If you already run Open
WebUI under a different project name, set `OWUITOOLS_OWUI_NETWORK` in
`configs/owuitools.env` rather than renaming your stack.
## Order
1. **Ollama**, and pull your model. Everything else waits on it.
2. **Open WebUI**, pointed at Ollama. Check you can chat before going further — if that
does not work, nothing downstream will, and the tool server's failures will be blamed
for it.
3. **owuitools** itself: `cp configs/owuitools.env.example configs/owuitools.env`, generate
a token, `docker compose up -d --build`. See [../install-ubuntu-desktop.md](../install-ubuntu-desktop.md).
4. **Gitea**, only if you want the VM sub-agent to publish. See [../setup/gitea.md](../setup/gitea.md).
## The two settings that cause confusing failures
**`ENABLE_FORWARD_USER_INFO_HEADERS: "True"`** on Open WebUI. Without it the tool server
never learns which user or which chat a call belongs to. The per-user rate limit has
nothing to key on, and the VM sub-agent's live view logs *"live progress disabled: no
usable chat id was forwarded"* and shows nothing — which reads as a broken feature rather
than a missing header.
**`AIOHTTP_CLIENT_TIMEOUT_TOOL_SERVER: "7200"`**. Unset does **not** mean unlimited: Open
WebUI falls back to `AIOHTTP_CLIENT_TIMEOUT`, which defaults to 300 seconds. A `vm_agent`
session runs for minutes to hours, so at the default it is cut off at five minutes — often
with the work already done and only the reporting left. See
[../advanced-params.md](../advanced-params.md) for the three clocks that have to agree.
## Ollama is unauthenticated, and that shapes the design
Ollama has no authentication at all and serves `DELETE /api/delete`, `POST /api/pull` and
`POST /api/create`. Anything that can reach port 11434 can delete your model.
Nothing in this project talks to it directly. The VM sub-agent — the component that runs a
second model on behalf of whoever is chatting — reaches it through `owuitools-vmgw`, an
nginx that exposes exactly `/ollama/api/chat` and `/ollama/api/ps` as *exact-match*
locations and returns 404 for everything else. Its container can reach that one address and
nothing else, enforced by a default-deny iptables policy in `vmguard/firewall.sh`.
That is also why the compose file binds Ollama to the docker bridge address rather than
`0.0.0.0`. If you change it, change it knowingly.
docs/deployment/gitea.compose.yml
+77
View File
@@ -0,0 +1,77 @@
# =====================================================================
# Gitea - where the VM sub-agent publishes what it built.
#
# ONLY vm_agent NEEDS THIS. Every other tool works without it, and so
# does vm_agent itself: without Gitea a session still runs, still keeps
# its files in the project folder on the array, and simply returns no
# link. If you are not using the VM sub-agent, skip this file entirely.
#
# WHAT THE SUB-AGENT NEEDS FROM IT
# * an organisation to create repositories in (OWUITOOLS_GITEA_ORG)
# * a bot account with write access to it (OWUITOOLS_GITEA_BOT)
# * an access token for that account (OWUITOOLS_GITEA_TOKEN)
#
# All three go in /etc/owuivm/vmd.env, from configs/vmd.env. The token
# NEVER enters the guest or the agent container: vmd clones, commits and
# pushes on the host and supplies the credential at push time through
# GIT_ASKPASS. See docs/setup/gitea.md.
#
# WHY TWO URLs
# vmd pushes to the LAN address and gives the user the public one. The
# daemon's systemd unit has an address allowlist with exactly this
# destination on it, and a public name behind a CDN would not match.
# =====================================================================
# Pinned so the project and its volume have the same names wherever this
# file is run from - see the note in open-webui.compose.yml.
name: gitea
services:
gitea:
image: gitea/gitea:1
container_name: gitea
restart: unless-stopped
environment:
USER_UID: 1000
USER_GID: 1000
GITEA__server__PROTOCOL: http
GITEA__server__HTTP_ADDR: 0.0.0.0
GITEA__server__HTTP_PORT: 3000
GITEA__server__DOMAIN: git.example.com
GITEA__server__ROOT_URL: https://git.example.com/
GITEA__server__LOCAL_ROOT_URL: http://localhost:3000/
# The agent creates repositories through the API as its bot user.
# Registration stays closed; the bot is made by hand, once.
GITEA__service__DISABLE_REGISTRATION: "true"
GITEA__server__DISABLE_SSH: "false"
GITEA__server__START_SSH_SERVER: "false"
GITEA__server__SSH_USER: git
GITEA__server__SSH_DOMAIN: git.example.com
GITEA__server__SSH_PORT: 7222
# Only if you front this with a reverse proxy. Naming the proxy
# rather than "*" is what stops a client forging its own source IP.
GITEA__security__REVERSE_PROXY_LIMIT: 1
GITEA__security__REVERSE_PROXY_TRUSTED_PROXIES: "192.168.1.0/24"
GITEA__repository__ROOT: /data/git/repositories
ports:
# Bind to the LAN address rather than 0.0.0.0 if this machine has
# one; plain "7200:3000" is fine otherwise. Whatever you choose
# here is what OWUITOOLS_GITEA_URL must say, and what
# owuivmd.service's IPAddressAllow must permit.
- "7200:3000"
- "7222:22"
volumes:
- /srv/gitea/repositories:/data/git/repositories
- gitea-data:/data
- /etc/timezone:/etc/timezone:ro
- /etc/localtime:/etc/localtime:ro
volumes:
gitea-data:
docs/deployment/ollama.compose.yml
+242
View File
@@ -0,0 +1,242 @@
# =====================================================================
# Ollama, plus a watchdog that restarts it when the GPU goes away.
#
# This is the model host. Both the assistant and the VM sub-agent talk to
# one Ollama with one model loaded, which is the whole reason the
# sub-agent's session is a single long block rather than many short ones:
# a handover evicts the other model's prefix cache and forces a cold
# re-prefill of its entire context.
#
# WHAT TO CHANGE
# * the port binding, if 172.17.0.1 is not your docker bridge
# * mem_limit, to suit the machine
# * the ntfy block at the bottom, or delete it - alerting is optional
#
# WHAT NOT TO CHANGE
# OLLAMA_NUM_PARALLEL and OLLAMA_MAX_LOADED_MODELS are both 1 on
# purpose. The assistant runs at a 160k context, and the KV cache costs
# roughly 85 KB of VRAM per token on a 27B Q4_K_M model - about 13.4 GB
# on top of 16.5 GB of weights. A second slot of that size does not fit
# in 32 GB, and letting Ollama try means it unloads and reloads the
# model mid-conversation.
#
# WHY IT BINDS TO THE DOCKER BRIDGE AND NOT 0.0.0.0
# Ollama has NO AUTHENTICATION and serves DELETE /api/delete,
# POST /api/pull and POST /api/create. Anything that can reach it can
# delete your model. Nothing in this project talks to it directly: the
# sub-agent reaches it through owuitools-vmgw, an nginx that exposes
# exactly /api/chat and /api/ps and 404s everything else.
# =====================================================================
# Pinned so the project, its volume and its network have the same names
# wherever this file is run from - see the note in open-webui.compose.yml.
name: ollama
services:
ollama:
image: ollama/ollama:latest
container_name: ollama
restart: unless-stopped
ports:
# The docker bridge address, not 0.0.0.0. See the note above.
- "172.17.0.1:11434:11434"
volumes:
- ollama:/root/.ollama
environment:
OLLAMA_MAX_LOADED_MODELS: "1"
OLLAMA_NUM_PARALLEL: "1"
NVIDIA_VISIBLE_DEVICES: "all"
NVIDIA_DRIVER_CAPABILITIES: "compute,utility"
mem_limit: 24g
memswap_limit: 24g
gpus: all
deploy:
resources:
limits:
memory: 24g
reservations:
devices:
- driver: nvidia
count: 1
capabilities: [gpu]
# -------------------------------------------------------------------
# The watchdog. Optional, and worth having.
#
# It exists because of a failure mode that does not look like a
# failure: the container keeps running and answering, and silently
# falls back to CPU, so every reply still arrives and takes twenty
# times as long. Three probes catch it - NVML through nvidia-smi, an
# HTTP GET that catches a hung server, and a grep of the logs for the
# CPU-fallback strings.
#
# Restarts are rate-limited, and alerts are throttled to one a day, so
# a persistent fault does not become a persistent notification.
# -------------------------------------------------------------------
ollama-gpu-watchdog:
container_name: ollama-gpu-watchdog
depends_on:
- ollama
image: docker:cli
restart: unless-stopped
init: true
mem_limit: 128m
volumes:
- /var/run/docker.sock:/var/run/docker.sock
healthcheck:
test: ["CMD-SHELL", "test $$(( $$(date +%s) - $$(stat -c %Y /tmp/heartbeat) )) -lt 300"]
interval: 60s
timeout: 5s
retries: 3
start_period: 30s
entrypoint: ["/bin/sh", "-c"]
command:
- |
set -u
: "$${TARGET:=ollama}"
: "$${INSTANCE:=$$TARGET}"
: "$${CHECK_INTERVAL:=30}"
: "$${RESTART_COOLDOWN:=600}"
: "$${STARTUP_GRACE:=60}"
: "$${GPU_FAIL_LIMIT:=2}"
: "$${API_FAIL_LIMIT:=3}"
: "$${LOG_WINDOW:=60s}"
: "$${PROBE_TIMEOUT:=20}"
: "$${GPU_CHECK_INTERVAL:=600}"
: "$${GPU_RETRY_INTERVAL:=60}"
: "$${GPU_PROBE_OFFSET:=0}"
: "$${API_URL:=http://$$TARGET:11434/api/tags}"
: "$${NOTIFY_COOLDOWN:=86400}"
: "$${NTFY_URL:=}"
: "$${NTFY_TOPIC:=}"
: "$${NTFY_TOKEN:=}"
log() { echo "$$(date -u +%Y-%m-%dT%H:%M:%SZ) [$$INSTANCE] $$*"; }
LAST_RESTART=0
LAST_NOTIFY=0
GPU_FAILS=0
API_FAILS=0
notify() {
[ -n "$$NTFY_URL" ] && [ -n "$$NTFY_TOKEN" ] || return 0
now="$$(date +%s)"
if [ "$$((now - LAST_NOTIFY))" -lt "$$NOTIFY_COOLDOWN" ]; then
log "ntfy suppressed (24h throttle): $$*"
return 0
fi
if wget -q -T 10 -O /dev/null \
--header "Authorization: Bearer $$NTFY_TOKEN" \
--post-data "$$(date -u +%Y-%m-%dT%H:%M:%SZ) $$*" \
"$$NTFY_URL/$$NTFY_TOPIC?title=Ollama+watchdog:+$$INSTANCE&priority=high&tags=rotating_light"; then
LAST_NOTIFY="$$(date +%s)"
log "ntfy alert sent: $$*"
else
log "ERROR: ntfy publish failed (will retry on next event)"
fi
}
restart_target() {
reason="$$1"
notify "$$reason - watchdog intervening on $$INSTANCE"
now="$$(date +%s)"
if [ "$$((now - LAST_RESTART))" -lt "$$RESTART_COOLDOWN" ]; then
log "restart wanted ($$reason) but in cooldown"
return 0
fi
log "RESTARTING $$TARGET: $$reason"
if timeout 180 docker restart "$$TARGET" >/dev/null 2>&1; then
LAST_RESTART="$$(date +%s)"
GPU_FAILS=0
API_FAILS=0
log "restart issued OK; grace sleep $${STARTUP_GRACE}s"
sleep "$$STARTUP_GRACE"
else
log "ERROR: docker restart $$TARGET failed/timed out"
notify "docker restart of $$INSTANCE FAILED after: $$reason"
fi
}
log "watchdog up"
NEXT_GPU_PROBE="$$(( $$(date +%s) + GPU_PROBE_OFFSET ))"
while true; do
touch /tmp/heartbeat
state="$$(timeout 10 docker inspect -f '{{.State.Status}}' "$$TARGET" 2>/dev/null || echo unknown)"
if [ "$$state" != "running" ]; then
log "target state=$$state - waiting (docker restart policy owns crash recovery)"
sleep "$$CHECK_INTERVAL"
continue
fi
# 1. GPU / NVML, only every GPU_CHECK_INTERVAL: without a
# persistence client every nvidia-smi is a full driver
# init/teardown. After a failure, re-probe sooner.
now="$$(date +%s)"
if [ "$$now" -ge "$$NEXT_GPU_PROBE" ]; then
if timeout "$$PROBE_TIMEOUT" docker exec "$$TARGET" nvidia-smi >/dev/null 2>&1; then
[ "$$GPU_FAILS" -gt 0 ] && log "GPU probe recovered"
GPU_FAILS=0
NEXT_GPU_PROBE="$$((now + GPU_CHECK_INTERVAL))"
else
GPU_FAILS="$$((GPU_FAILS + 1))"
log "GPU probe FAILED ($$GPU_FAILS/$$GPU_FAIL_LIMIT)"
NEXT_GPU_PROBE="$$((now + GPU_RETRY_INTERVAL))"
if [ "$$GPU_FAILS" -ge "$$GPU_FAIL_LIMIT" ]; then
restart_target "container lost GPU (NVML dead)"
NEXT_GPU_PROBE="$$(( $$(date +%s) + GPU_RETRY_INTERVAL ))"
fi
fi
fi
# 2. the API itself - catches a hung server with no docker exec
if timeout "$$PROBE_TIMEOUT" wget -q -T "$$PROBE_TIMEOUT" -O /dev/null "$$API_URL"; then
[ "$$API_FAILS" -gt 0 ] && log "API probe recovered"
API_FAILS=0
else
API_FAILS="$$((API_FAILS + 1))"
log "API probe FAILED ($$API_FAILS/$$API_FAIL_LIMIT)"
if [ "$$API_FAILS" -ge "$$API_FAIL_LIMIT" ]; then
restart_target "API unresponsive"
fi
fi
# 3. the silent one: still answering, but on the CPU
if timeout 30 docker logs --since "$$LOG_WINDOW" "$$TARGET" 2>&1 | grep -qE \
'msg="inference compute".*id=cpu|library=cpu|total_vram="0 B"|no compatible GPUs|failed to discover.*GPU|CUDA.*error|CUDA.*failure|unable to load CUDA'; then
restart_target "GPU fallback / CUDA error in logs"
fi
sleep "$$CHECK_INTERVAL"
done
environment:
TARGET: "ollama"
INSTANCE: "ollama"
CHECK_INTERVAL: "30"
RESTART_COOLDOWN: "600"
STARTUP_GRACE: "60"
GPU_FAIL_LIMIT: "2"
API_FAIL_LIMIT: "3"
LOG_WINDOW: "60s"
PROBE_TIMEOUT: "20"
GPU_CHECK_INTERVAL: "600"
GPU_RETRY_INTERVAL: "60"
GPU_PROBE_OFFSET: "0"
NOTIFY_COOLDOWN: "86400"
# Alerting is optional: leave these empty and the watchdog still
# restarts, it just does not tell anyone. Fill them in to get a
# push when it intervenes. Use a write-only token.
NTFY_URL: ""
NTFY_TOPIC: ""
NTFY_TOKEN: ""
volumes:
ollama:
docs/deployment/open-webui.compose.yml
+120
View File
@@ -0,0 +1,120 @@
# =====================================================================
# Open WebUI - the front end, and the thing that calls the tool server.
#
# TWO SETTINGS HERE ARE LOAD-BEARING FOR owuitools. Everything else is
# ordinary Open WebUI configuration you can change freely.
#
# ENABLE_FORWARD_USER_INFO_HEADERS
# Sends X-OpenWebUI-User-Id and X-OpenWebUI-Chat-Id to the tool
# server. Without them the per-user probe budget has nothing to key
# on, and the VM sub-agent's live progress view has no chat to post
# into - it degrades to "live progress disabled: no usable chat id
# was forwarded", which looks like a broken feature.
#
# AIOHTTP_CLIENT_TIMEOUT_TOOL_SERVER
# Open WebUI's own clock on a tool call. IT IS NOT UNLIMITED WHEN
# UNSET: env.py falls back to AIOHTTP_CLIENT_TIMEOUT, which
# defaults to 300. A vm_agent session drives a real machine for
# minutes to hours, and was being cut off at five minutes with the
# work already finished.
#
# There are THREE clocks on that call and they must agree:
# OWUITOOLS_VM_WAIT_SECONDS (configs/owuitools.env, 7000)
# proxy_read_timeout (configs/services/gateway.nginx.conf,
# 7200s, on /tools/vm_agent only)
# this one (7200)
# Keep the app's the smallest, so IT answers rather than a proxy
# cutting the connection with no result.
#
# If you run this through Portainer, enter these in Portainer's own stack
# editor. A redeploy regenerates stack.env and discards edits made to the
# file on disk.
#
# The OIDC block at the bottom is entirely optional - delete it and use
# the built-in login form. It is included because it is what this project
# was developed against.
# =====================================================================
# The project name is pinned, and that matters more than it looks.
#
# Compose derives a project name from the directory the compose file is
# in, so running this as `docker compose -f docs/deployment/...` would
# create a network called `deployment_default`. The tool server's own
# compose joins Open WebUI's network BY NAME, expecting
# `openwebui_default` - so without this line, following the install guide
# in order fails at the next step with "network declared as external, but
# could not be found".
#
# If you already run Open WebUI under a different project name, set
# OWUITOOLS_OWUI_NETWORK in configs/owuitools.env instead of changing
# this.
name: openwebui
services:
open-webui:
image: ghcr.io/open-webui/open-webui:main
container_name: open-webui
restart: unless-stopped
init: true
ports:
- "3000:8080"
environment:
# --- required by owuitools ---------------------------------------
ENABLE_FORWARD_USER_INFO_HEADERS: "True"
AIOHTTP_CLIENT_TIMEOUT_TOOL_SERVER: "7200"
# --- ordinary configuration --------------------------------------
TZ: "UTC"
OLLAMA_BASE_URL: "http://host.docker.internal:11434"
# Generate one: `openssl rand -hex 32`. Changing it logs everyone out.
WEBUI_SECRET_KEY: "CHANGE_ME"
WEBUI_URL: "https://ai.example.com"
ENABLE_PERSISTENT_CONFIG: "true"
ENABLE_API_KEYS: "true"
ENABLE_EVALUATION_ARENA_MODELS: "False"
# --- optional: OIDC through an identity provider ------------------
# Delete this whole block to use the built-in login form instead.
# ENABLE_OAUTH_PERSISTENT_CONFIG=false keeps these under the
# compose file's control rather than letting the first boot persist
# them into the database and then ignore every later change.
ENABLE_OAUTH_PERSISTENT_CONFIG: "false"
ENABLE_OAUTH_SIGNUP: "true"
ENABLE_LOGIN_FORM: "true"
OAUTH_MERGE_ACCOUNTS_BY_EMAIL: "true"
OAUTH_PROVIDER_NAME: "authentik"
OAUTH_CLIENT_ID: ${OAUTH_CLIENT_ID:-}
OAUTH_CLIENT_SECRET: ${OAUTH_CLIENT_SECRET:-}
OPENID_PROVIDER_URL: "https://auth.example.com/application/o/open-webui/.well-known/openid-configuration"
OPENID_REDIRECT_URI: "https://ai.example.com/oauth/oidc/callback"
OAUTH_SCOPES: "openid email profile"
ENABLE_OAUTH_ID_TOKEN_COOKIE: "false"
ENABLE_OAUTH_ROLE_MANAGEMENT: "true"
OAUTH_ROLES_CLAIM: "roles"
OAUTH_ALLOWED_ROLES: "user,admin"
OAUTH_ADMIN_ROLES: "admin"
volumes:
- /srv/openwebui/data:/app/backend/data
extra_hosts:
- "host.docker.internal:host-gateway"
healthcheck:
test:
[
"CMD-SHELL",
"python -c \"import urllib.request; urllib.request.urlopen('http://127.0.0.1:8080/health', timeout=5)\" || exit 1"
]
interval: 30s
timeout: 10s
retries: 5
start_period: 40s
logging:
driver: local
options:
max-size: "10m"
max-file: "5"