ubuntu-post-install/services/ai-stack.md

Roles

• Open WebUI (chat, research, light coding) — local Ollama + any cloud providers in one model dropdown; wired to your code via the RAG + MCP servers and Gitea.
• PaintPlus (separate paintplus service) — the front end for all image work (inpaint / upscale / generate). Point its AI_PROVIDER at a cloud API, or at this stack's local comfyui / invokeai for local image-gen.
• Gitea + GitHub sync — bash gitea-github-sync.sh mirrors repos both ways (pull GitHub → local git, or push local → GitHub).
• RAG / MCP / Kiwix — retrieve just the relevant context so you feed the model less text (saves tokens), for both local and cloud models.
• Web search uses DuckDuckGo (no SearXNG in this build).

GPU switcher (small local GPU only)

One small GPU can't run local chat and local image-gen at once. Swap it:

~/docker/ai-stack/gpu-mode.sh images   # before generating locally in PaintPlus
~/docker/ai-stack/gpu-mode.sh llm       # back to local chat in Open WebUI
~/docker/ai-stack/gpu-mode.sh status    # see which is active

Cloud models work anytime and need no swap.

Service URLs

Service	URL	Auth
Open WebUI	http://localhost:3000	built-in (first visit = admin)
InvokeAI	http://localhost:9090	none
ComfyUI	http://localhost:8188	none
Kiwix	http://localhost:8181	none
Gitea	http://localhost:3001	built-in
Portainer	https://localhost:9443	built-in

Manage the stack

cd ~/docker/ai-stack
bash start.sh          # pull latest images + docker compose up -d
bash stop.sh           # docker compose down
bash status.sh         # GPU / container / RAG health
bash pull-models.sh    # pull Ollama models (run once after first install)

Also a systemd unit: sudo systemctl {start,stop,status} local-ai

Vision models (image understanding)

None of the tier-selected chat/code models above can read an image. pull-models.sh offers one optional vision model at the end — pick it there, or pull one manually any time:

docker exec ollama ollama pull moondream   # or llava:7b / qwen2.5vl:7b / llama3.2-vision:11b

Model	Size	Notes
moondream	~1.7 GB	By Moondream AI — tiny, built for CPU-only or weak/old-GPU hardware. Best default if you don't have a real GPU.
llava:7b	~4.7 GB	General-purpose vision, moderate resources.
qwen2.5vl:7b	~6 GB	Stronger accuracy, needs more RAM/VRAM.
llama3.2-vision:11b	~7.9 GB	Meta's vision model — heaviest of these four.

Point any OpenAI-compatible app's vision/image-import feature at this stack's Ollama endpoint with the pulled model. For Mealie's "import recipe from photo" specifically — checked against docs.mealie.io directly, since Mealie moved this off env vars at some point and old OPENAI_* env var guidance for it is now stale: it's configured live in the UI, not .env — Group Settings → AI Providers in Mealie itself, not this stack's .env or docker-compose.yml. Add a provider with:

• base_url: http://host.docker.internal:11434/v1 (Ollama publishes on the host at 0.0.0.0:11434, and Mealie is a separate compose project not sharing a network with this stack, so it has to be reached over the host the same way Caddy reaches bridge-mode services — see Mealie's own compose: add extra_hosts: ["host.docker.internal:host-gateway"] to its mealie: service if that hostname doesn't already resolve there. The host's real LAN IP works too with no compose edit, just less stable across DHCP renewals.)
• api_key: any non-empty placeholder — required by Mealie's form, ignored by Ollama.
• model: the vision model just pulled (e.g. moondream).

Then mark that provider as the one used for image recognition (a separate toggle from the general default-provider setting) — that's what actually turns on the photo-import feature. No Mealie container restart needed, it applies live. See Open WebUI → Settings → Connections if you'd rather confirm the local base URL/model name there first.

Cloud LLM providers (Open WebUI)

Open WebUI uses an OpenAI-compatible connection list. The local RAG server is the first entry; any cloud providers added at install follow it. Two semicolon-separated lists in .env, matched by position (RAG must stay first):

# ~/docker/ai-stack/.env
OPENAI_API_BASE_URLS=http://rag-server:8001/v1;https://api.groq.com/openai/v1
OPENAI_API_KEYS=local-rag;gsk_xxx
cd ~/docker/ai-stack && docker compose up -d open-webui   # apply

Provider	Base URL	Key
Groq	https://api.groq.com/openai/v1	https://console.groq.com/keys
DeepInfra	https://api.deepinfra.com/v1/openai	https://deepinfra.com/dash/api_keys
OpenAI	https://api.openai.com/v1	https://platform.openai.com/api-keys
OpenRouter	https://openrouter.ai/api/v1	https://openrouter.ai/keys

Alternatively, add them at runtime in Open WebUI → Settings → Admin → Connections (no file edits, survives image upgrades).

Update

Re-run the ai-stack installer (refreshes vendored source, keeps your .env), then bash ~/docker/ai-stack/start.sh. Or in place: cd ~/docker/ai-stack && bash local-ai-setup.sh --force.

Caddy

Open WebUI is reverse-proxied as open-webui:8080 on caddy_net (or your configured Caddy network name; attached with docker network connect after start). Other services are LAN-only by default — add Caddy site blocks for them if you want remote access.