Update README.md

2026-06-01 09:10:01 -04:00
parent 405016e425
commit 685150edb5
1 changed files with 156 additions and 1 deletions
@@ -1 +1,156 @@
-# ai-6gb-gpu
+# ai-6gb-gpu
+
+Local AI stack — image generation and LLM chat on a single NVIDIA GPU machine.
+Portal at `localai.mydomain.com` switches between stacks to share VRAM.
+
+## What this does
+
+- **InvokeAI** — image generation with canvas inpainting (`images.mydomain.com`)
+- **Ollama + Open WebUI** — local LLM chat with web search (`chat.mydomain.com`)
+- **SearXNG** — local search engine for Open WebUI (no API key needed, or use Brave Search API)
+- **ai-portal** — landing page that swaps GPU between image and chat stacks
+
+## Machine requirements
+
+- Ubuntu 24.04 LTS
+- NVIDIA GPU (tested on Quadro RTX 3000 6GB)
+- Docker + NVIDIA Container Toolkit (see setup below)
+- homelab-auth running on Caddy machine (for SSO)
+
+## Directory structure
+
+```
+~/docker
+├── ai-image-gen/
+│   ├── docker-compose.yml     ← InvokeAI
+│   └── archive.sh             ← saves Docker images for offline restore
+├── ai-llm/
+│   └── docker-compose.yml     ← Ollama + Open WebUI + SearXNG
+├── ai-portal/
+│   ├── docker-compose.yml
+│   ├── Dockerfile
+│   ├── app.py                 ← Flask portal + GPU swap logic
+│   └── templates/
+│       ├── portal.html
+│       └── switching.html
+└── README.md
+```
+
+## First time setup
+
+### 1. Install Docker
+```bash
+curl -fsSL https://get.docker.com | sh
+sudo usermod -aG docker $USER
+newgrp docker
+```
+
+### 2. Install NVIDIA driver
+```bash
+sudo apt update && sudo apt install -y ubuntu-drivers-common
+sudo ubuntu-drivers install
+sudo reboot
+```
+
+### 3. Install NVIDIA Container Toolkit
+```bash
+curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey \
+  | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-ct-keyring.gpg
+
+curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list \
+  | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-ct-keyring.gpg] https://#g' \
+  | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
+
+sudo apt update && sudo apt install -y nvidia-container-toolkit
+sudo nvidia-ctk runtime configure --runtime=docker
+sudo systemctl restart docker
+```
+
+### 4. Verify GPU visible in Docker
+```bash
+docker run --rm --gpus all nvidia/cuda:12.4.1-base-ubuntu22.04 nvidia-smi
+```
+Must show your GPU. If not, check driver installation.
+
+### 5. Install linux-modules-extra (required for WiFi and some GPU features)
+```bash
+sudo apt install -y linux-modules-extra-$(uname -r)
+```
+
+### 6. Create data directories
+```bash
+mkdir -p ~/docker/ai-image-gen/data/invokeai
+mkdir -p ~/docker/ai-llm/data/ollama
+mkdir -p ~/docker/ai-llm/data/open-webui
+mkdir -p ~/docker/ai-llm/data/searxng
+mkdir -p ~/docker/ai-portal
+```
+
+### 7. Copy files from repo
+Copy each directory from this repo to `~/docker/` on the machine.
+Replace `mydomain.com` with your real domain in:
+- `ai-portal/app.py`
+- `ai-portal/templates/switching.html`
+- `ai-portal/templates/portal.html`
+
+### 8. Start the portal (runs permanently)
+```bash
+cd ~/docker/ai-portal && docker compose up -d
+```
+
+### 9. Start whichever stack you want first
+```bash
+# For LLM/chat
+cd ~/docker/ai-llm && docker compose up -d
+
+# For image generation
+cd ~/docker/ai-image-gen && docker compose up -d
+```
+
+After that, use the portal at `localai.mydomain.com` to switch between them.
+
+## Models
+
+Pull once after first starting the LLM stack:
+```bash
+docker exec ai-llm-ollama-1 ollama pull qwen2.5-coder:7b   # coding
+docker exec ai-llm-ollama-1 ollama pull qwen2.5:7b          # paper writing
+docker exec ai-llm-ollama-1 ollama pull llava:7b            # vision/screenshots
+```
+
+First model for InvokeAI — go to `images.mydomain.com` → Model Manager → search `sd-1.5-inpainting` → Install.
+
+## Open WebUI web search setup
+In Open WebUI Admin Panel → Settings → Web Search:
+- Engine: Brave
+- API Key: get free key at `api.search.brave.com` (free $5/month credits, set spending cap to Free)
+- Results Count: 3
+
+## GPU swap behavior
+The portal stops/starts individual containers to free VRAM:
+- Switching to Images: stops Ollama, starts InvokeAI
+- Switching to Chat: stops InvokeAI, starts Ollama
+- Open WebUI stays up always (uses no VRAM)
+- Authelia sessions stay intact (networks never go down)
+
+## Archiving images for offline restore
+Run after first successful boot:
+```bash
+chmod +x ~/docker/ai-image-gen/archive.sh
+cd ~/docker/ai-image-gen && ./archive.sh
+```
+Store the `archive/` directory and `data/` on external drive.
+To restore on a new machine: `./archive/restore.sh` then `docker compose up -d`.
+
+## Updating
+Only update when you have a specific reason. Run from the relevant stack directory:
+```bash
+./update.sh
+```
+
+## Notes
+- Models survive container restarts — they live in `data/` volumes on the host
+- VRAM is 6GB — only one GPU-heavy stack can run at a time
+- InvokeAI takes ~10 seconds after start before the UI is ready
+- Ollama automatically unloads models from VRAM after 5 minutes of inactivity
+- WiFi requires `linux-modules-extra` — if WiFi is missing after kernel update, reinstall it