ai-6gb-gpu/README.md

# ai-6gb-gpu

Local AI stack — image generation and LLM chat on a single NVIDIA GPU machine.
Portal at `localai.mydomain.com` switches between stacks to share VRAM.

## What this does

- **InvokeAI** — image generation with canvas inpainting (`images.mydomain.com`)
- **Ollama + Open WebUI** — local LLM chat with web search (`chat.mydomain.com`)
- **SearXNG** — local search engine for Open WebUI (no API key needed, or use Brave Search API)
- **ai-portal** — landing page that swaps GPU between image and chat stacks

## Machine requirements

- Ubuntu 24.04 LTS
- NVIDIA GPU (tested on Quadro RTX 3000 6GB)
- Docker + NVIDIA Container Toolkit (see setup below)
- homelab-auth running on Caddy machine (for SSO)

## Directory structure

```
homelab-ai/
├── ai-image-gen/
│   ├── docker-compose.yml     ← InvokeAI
│   └── archive.sh             ← saves Docker images for offline restore
├── ai-llm/
│   └── docker-compose.yml     ← Ollama + Open WebUI + SearXNG
├── ai-portal/
│   ├── docker-compose.yml
│   ├── Dockerfile
│   ├── app.py                 ← Flask portal + GPU swap logic
│   └── templates/
│       ├── portal.html
│       └── switching.html
└── README.md
```

## First time setup

### 1. Install Docker
```bash
curl -fsSL https://get.docker.com | sh
sudo usermod -aG docker $USER
newgrp docker
```

### 2. Install NVIDIA driver
```bash
sudo apt update && sudo apt install -y ubuntu-drivers-common
sudo ubuntu-drivers install
sudo reboot
```

### 3. Install NVIDIA Container Toolkit
```bash
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey \
  | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-ct-keyring.gpg

curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list \
  | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-ct-keyring.gpg] https://#g' \
  | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

sudo apt update && sudo apt install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
```

### 4. Verify GPU visible in Docker
```bash
docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi
```
Must show your GPU. If not, check driver installation.

### 5. Install linux-modules-extra (required for WiFi and some GPU features)
```bash
sudo apt install -y linux-modules-extra-$(uname -r)
```

### 6. Create data directories
```bash
mkdir -p ~/docker/ai-image-gen/data/invokeai
mkdir -p ~/docker/ai-llm/data/ollama
mkdir -p ~/docker/ai-llm/data/open-webui
mkdir -p ~/docker/ai-llm/data/searxng
mkdir -p ~/docker/ai-portal/templates
```

### 7. Create .env files
```bash
echo "TZ=America/New_York" | tee ~/docker/ai-llm/.env ~/docker/ai-image-gen/.env ~/docker/ai-portal/.env
```

### 7. Copy files from repo
Copy each directory from this repo to `~/docker/` on the machine.
Replace `mydomain.com` with your real domain in:
- `ai-portal/app.py`
- `ai-portal/templates/switching.html`
- `ai-portal/templates/portal.html`

### 8. Start the portal (runs permanently)
```bash
cd ~/docker/ai-portal && docker compose up -d
```

### 9. Start whichever stack you want first
```bash
# For LLM/chat
cd ~/docker/ai-llm && docker compose up -d

# For image generation
cd ~/docker/ai-image-gen && docker compose up -d
```

After that, use the portal at `localai.mydomain.com` to switch between them.

## Models

Pull once after first starting the LLM stack:
```bash
docker exec ai-llm-ollama-1 ollama pull qwen2.5-coder:7b   # coding
docker exec ai-llm-ollama-1 ollama pull qwen2.5:7b          # paper writing
docker exec ai-llm-ollama-1 ollama pull llava:7b            # vision/screenshots
```

First model for InvokeAI — go to `images.mydomain.com` → Model Manager → search `sd-1.5-inpainting` → Install.

## InvokeAI Usage Guide

### Installing your first model
1. Open `https://images.mydomain.com`
2. Click **Model Manager** (left sidebar)
3. Click **HuggingFace**
4. Search: `runwayml/stable-diffusion-inpainting`
5. Click Install — downloads automatically

### Text to Image (like ChatGPT image generation)
1. Click the **canvas icon** (left sidebar)
2. Make sure model is selected at top (e.g. `sd-1.5-inpainting`)
3. Type your prompt in the prompt box
4. Click **Invoke**
5. Results appear in the gallery (right side)

Key settings to learn:
- **Steps** — higher = more refined, slower (20-30 is good)
- **CFG Scale** — how closely to follow the prompt (7-9 is good)
- **Seed** — lock it to reproduce the exact same image

### Inpainting (fixing details in an existing image)
1. Open an image from the gallery or upload one
2. Click **Send to Canvas**
3. Select the **Brush** tool
4. Paint a mask over the area you want to change (face, hand, background)
5. Type what you want in that area in the prompt
6. Adjust **Denoising Strength** — lower (0.3-0.5) = subtle, higher (0.7+) = more creative
7. Click **Invoke**
8. The masked area is regenerated, rest of image stays intact

Tips:
- Use **Inpaint at Full Resolution** for face/hand details
- Keep denoising low (0.3-0.4) for seamless blending
- Mask blur of 4-8px softens the edge between inpainted and original

### Using an image as a base (img2img)
1. Upload or select an image from gallery
2. Click **Send to Canvas**
3. Change mode to **Image to Image**
4. Type your prompt describing what you want
5. Adjust **Denoising Strength**:
   - Low (0.3) = keeps original, subtle changes
   - High (0.8) = major transformation
6. Click **Invoke**

### Consistent character images (for product lines)
To maintain a consistent character across many images:

**Method 1 — Seed locking:**
- Generate until you get the character you want
- Note the seed number
- Lock the seed — same seed = same character base every time
- Change only the prompt details (pose, background, clothing)

**Method 2 — Reference image + low denoising:**
- Use your established character image as the base
- Set denoising to 0.3-0.4
- Change prompt to describe the new scene/pose
- Character features stay consistent, context changes

**Method 3 — LoRA models:**
- Train or download a LoRA for your specific character
- Load the LoRA in Model Manager
- Add the LoRA trigger word to your prompt
- Most consistent method for commercial product lines
- LoRAs available at civitai.com

### Workflow for best results
1. Generate several variations (change seed each time)
2. Pick the best base image
3. Lock the seed
4. Inpaint problem areas (faces, hands, backgrounds)
5. Use img2img for scene variations with same character
6. Upscale final image for print quality

SearXNG's wikidata engine fails on startup due to external blocks. Disable it:

```bash
# Find the line number
sudo grep -n "name: wikidata" ~/docker/ai-llm/data/searxng/settings.yml

# Add inactive: true after the engine line (replace 771 with actual line number)
sudo sed -i '771a\    inactive: true' ~/docker/ai-llm/data/searxng/settings.yml

# Verify
sudo sed -n '769,774p' ~/docker/ai-llm/data/searxng/settings.yml

# Restart
docker compose -f ~/docker/ai-llm/docker-compose.yml restart searxng
```
In Open WebUI Admin Panel → Settings → Web Search:
- Engine: Brave
- API Key: get free key at `api.search.brave.com` (free $5/month credits, set spending cap to Free)
- Results Count: 3

## GPU swap behavior
The portal stops/starts individual containers to free VRAM:
- Switching to Images: stops Ollama, starts InvokeAI
- Switching to Chat: stops InvokeAI, starts Ollama
- Open WebUI stays up always (uses no VRAM)
- Authelia sessions stay intact (networks never go down)

## Archiving images for offline restore
Run after first successful boot:
```bash
chmod +x ~/docker/ai-image-gen/archive.sh
cd ~/docker/ai-image-gen && ./archive.sh
```
Store the `archive/` directory and `data/` on external drive.
To restore on a new machine: `./archive/restore.sh` then `docker compose up -d`.

## Updating
Only update when you have a specific reason. Run from the relevant stack directory:
```bash
./update.sh
```

## Notes
- Models survive container restarts — they live in `data/` volumes on the host
- VRAM is 6GB — only one GPU-heavy stack can run at a time
- InvokeAI takes ~10 seconds after start before the UI is ready
- Ollama automatically unloads models from VRAM after 5 minutes of inactivity
- WiFi requires `linux-modules-extra` — if WiFi is missing after kernel update, reinstall it