Local AI stack — image generation and LLM chat on a single NVIDIA GPU machine. Portal at localai.mydomain.com switches between stacks to share VRAM.

What this does

InvokeAI — image generation with canvas inpainting (images.mydomain.com)
Ollama + Open WebUI — local LLM chat with web search (chat.mydomain.com)
SearXNG — local search engine for Open WebUI (no API key needed, or use Brave Search API)
ai-portal — landing page that swaps GPU between image and chat stacks

Machine requirements

Ubuntu 24.04 LTS
NVIDIA GPU (tested on Quadro RTX 3000 6GB)
Docker + NVIDIA Container Toolkit (see setup below)
homelab-auth running on Caddy machine (for SSO)

Directory structure

homelab-ai/
├── ai-image-gen/
│   ├── docker-compose.yml     ← InvokeAI
│   └── archive.sh             ← saves Docker images for offline restore
├── ai-llm/
│   └── docker-compose.yml     ← Ollama + Open WebUI + SearXNG
├── ai-portal/
│   ├── docker-compose.yml
│   ├── Dockerfile
│   ├── app.py                 ← Flask portal + GPU swap logic
│   └── templates/
│       ├── portal.html
│       └── switching.html
└── README.md

First time setup

1. Install Docker

curl -fsSL https://get.docker.com | sh
sudo usermod -aG docker $USER
newgrp docker

2. Install NVIDIA driver

sudo apt update && sudo apt install -y ubuntu-drivers-common
sudo ubuntu-drivers install
sudo reboot

3. Install NVIDIA Container Toolkit

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey \
  | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-ct-keyring.gpg

curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list \
  | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-ct-keyring.gpg] https://#g' \
  | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt update && sudo apt install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

4. Verify GPU visible in Docker

docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi

Must show your GPU. If not, check driver installation.

5. Install linux-modules-extra (required for WiFi and some GPU features)

sudo apt install -y linux-modules-extra-$(uname -r)

6. Create data directories

mkdir -p ~/docker/ai-image-gen/data/invokeai
mkdir -p ~/docker/ai-llm/data/ollama
mkdir -p ~/docker/ai-llm/data/open-webui
mkdir -p ~/docker/ai-llm/data/searxng
mkdir -p ~/docker/ai-portal/templates

7. Create .env files

echo "TZ=America/New_York" | tee ~/docker/ai-llm/.env ~/docker/ai-image-gen/.env ~/docker/ai-portal/.env

7. Copy files from repo

Copy each directory from this repo to ~/docker/ on the machine. Replace mydomain.com with your real domain in:

ai-portal/app.py
ai-portal/templates/switching.html
ai-portal/templates/portal.html

8. Start the portal (runs permanently)

cd ~/docker/ai-portal && docker compose up -d

9. Start whichever stack you want first

# For LLM/chat
cd ~/docker/ai-llm && docker compose up -d

# For image generation
cd ~/docker/ai-image-gen && docker compose up -d

After that, use the portal at localai.mydomain.com to switch between them.

Models

Pull once after first starting the LLM stack:

docker exec ai-llm-ollama-1 ollama pull qwen2.5-coder:7b   # coding
docker exec ai-llm-ollama-1 ollama pull qwen2.5:7b          # paper writing
docker exec ai-llm-ollama-1 ollama pull llava:7b            # vision/screenshots

First model for InvokeAI — go to images.mydomain.com → Model Manager → search sd-1.5-inpainting → Install.

InvokeAI Usage Guide

Installing your first model

Open https://images.mydomain.com
Click Model Manager (left sidebar)
Click HuggingFace
Search: runwayml/stable-diffusion-inpainting
Click Install — downloads automatically

Text to Image (like ChatGPT image generation)

Click the canvas icon (left sidebar)
Make sure model is selected at top (e.g. sd-1.5-inpainting)
Type your prompt in the prompt box
Click Invoke
Results appear in the gallery (right side)

Key settings to learn:

Steps — higher = more refined, slower (20-30 is good)
CFG Scale — how closely to follow the prompt (7-9 is good)
Seed — lock it to reproduce the exact same image

Inpainting (fixing details in an existing image)

Open an image from the gallery or upload one
Click Send to Canvas
Select the Brush tool
Paint a mask over the area you want to change (face, hand, background)
Type what you want in that area in the prompt
Adjust Denoising Strength — lower (0.3-0.5) = subtle, higher (0.7+) = more creative
Click Invoke
The masked area is regenerated, rest of image stays intact

Tips:

Use Inpaint at Full Resolution for face/hand details
Keep denoising low (0.3-0.4) for seamless blending
Mask blur of 4-8px softens the edge between inpainted and original

Using an image as a base (img2img)

Upload or select an image from gallery
Click Send to Canvas
Change mode to Image to Image
Type your prompt describing what you want
Adjust Denoising Strength:
- Low (0.3) = keeps original, subtle changes
- High (0.8) = major transformation
Click Invoke

Consistent character images (for product lines)

To maintain a consistent character across many images:

Method 1 — Seed locking:

Generate until you get the character you want
Note the seed number
Lock the seed — same seed = same character base every time
Change only the prompt details (pose, background, clothing)

Method 2 — Reference image + low denoising:

Use your established character image as the base
Set denoising to 0.3-0.4
Change prompt to describe the new scene/pose
Character features stay consistent, context changes

Method 3 — LoRA models:

Train or download a LoRA for your specific character
Load the LoRA in Model Manager
Add the LoRA trigger word to your prompt
Most consistent method for commercial product lines
LoRAs available at civitai.com

Workflow for best results

Generate several variations (change seed each time)
Pick the best base image
Lock the seed
Inpaint problem areas (faces, hands, backgrounds)
Use img2img for scene variations with same character
Upscale final image for print quality

SearXNG's wikidata engine fails on startup due to external blocks. Disable it:

# Find the line number
sudo grep -n "name: wikidata" ~/docker/ai-llm/data/searxng/settings.yml

# Add inactive: true after the engine line (replace 771 with actual line number)
sudo sed -i '771a\    inactive: true' ~/docker/ai-llm/data/searxng/settings.yml

# Verify
sudo sed -n '769,774p' ~/docker/ai-llm/data/searxng/settings.yml

# Restart
docker compose -f ~/docker/ai-llm/docker-compose.yml restart searxng

In Open WebUI Admin Panel → Settings → Web Search:

Engine: Brave
API Key: get free key at api.search.brave.com (free $5/month credits, set spending cap to Free)
Results Count: 3

GPU swap behavior

The portal stops/starts individual containers to free VRAM:

Switching to Images: stops Ollama, starts InvokeAI
Switching to Chat: stops InvokeAI, starts Ollama
Open WebUI stays up always (uses no VRAM)
Authelia sessions stay intact (networks never go down)

Archiving images for offline restore

Run after first successful boot:

chmod +x ~/docker/ai-image-gen/archive.sh
cd ~/docker/ai-image-gen && ./archive.sh

Store the archive/ directory and data/ on external drive. To restore on a new machine: ./archive/restore.sh then docker compose up -d.

Updating

Only update when you have a specific reason. Run from the relevant stack directory:

./update.sh

Notes

Models survive container restarts — they live in data/ volumes on the host
VRAM is 6GB — only one GPU-heavy stack can run at a time
InvokeAI takes ~10 seconds after start before the UI is ready
Ollama automatically unloads models from VRAM after 5 minutes of inactivity
WiFi requires linux-modules-extra — if WiFi is missing after kernel update, reinstall it