Local AI Stack

A fully offline, self-hosted AI environment for Ubuntu 24.04. Runs on any NVIDIA GPU (or CPU-only).

Services: Ollama · Open WebUI · RAG · MCP · ChromaDB · SearXNG · Kiwix · Gitea · InvokeAI · Portainer

---

Quick Start

git clone <this-repo>
cd local-ai
./laptop_full_setup.sh

That's it. The script installs Docker, NVIDIA drivers (if needed), generates all config, starts the stack, and optionally pulls models.

---

Service URLs

After setup, all services are available on your LAN:

Service	URL	Purpose
Open WebUI	http://<ip>:3000	Chat interface (Ollama + RAG)
InvokeAI	http://<ip>:9090	Image generation
SearXNG	http://<ip>:8888	Private web search
Kiwix	http://<ip>:8181	Offline Wikipedia / docs
Gitea	http://<ip>:3001	Self-hosted Git
RAG Health	http://<ip>:8001/health	RAG server status
MCP SSE	http://<ip>:8002/sse	MCP endpoint for Claude Code
Portainer	https://<ip>:9443	Docker management UI

---

Day-to-Day Commands

All generated into ~/docker/ai-stack/ by the setup script:

bash ~/docker/ai-stack/start.sh       # pull latest images + docker compose up -d
bash ~/docker/ai-stack/stop.sh        # docker compose down
bash ~/docker/ai-stack/status.sh      # GPU / container / RAG health
bash ~/docker/ai-stack/pull-models.sh # pull Ollama models (run once after first install)

The stack also registers as a systemd service that starts on boot:

sudo systemctl start local-ai
sudo systemctl stop local-ai
sudo systemctl status local-ai

---

Script Reference

Script	Lines	What it does
laptop_full_setup.sh	620	Main setup. Installs Docker + NVIDIA toolkit, creates ~/docker/ai-stack/, writes docker-compose.yml, starts stack, registers systemd service.
local-ai-setup.sh	837	Alternative setup script. Same as above but also auto-detects VRAM and selects models accordingly (14B for ≥14GB VRAM, 7B for CPU). Use this instead of laptop_full_setup.sh if you want VRAM-aware model selection.
ubuntu-post-install.sh	8,889	Full Ubuntu 24.04 post-install (dev tools, fonts, apps, tweaks). Run once on a fresh OS install. Independent of the AI stack.
configure-storage.sh	239	Storage/mount configuration helper. Run separately if you have a secondary drive for AI data.
kiwix_download.sh	198	Downloads ZIM files (Wikipedia, Stack Overflow, etc.) for offline use. Run separately — files are large.

Which setup script should I use?

• laptop_full_setup.sh — fixed model selection (qwen2.5:14b / qwen2.5-coder:7b), simpler
• local-ai-setup.sh — detects your VRAM at runtime and picks appropriate models, also embeds server.py and mcp_server.py directly (doesn't need repo files copied separately)

Both scripts are idempotent — safe to re-run for updates. Config files are kept on re-run unless you pass --force.

---

Generated File Layout

~/docker/ai-stack/
├── docker-compose.yml      # generated by setup script
├── .env                    # API tokens — edit this, never overwritten
├── server.py               # RAG server (copied from repo)
├── mcp_server.py           # MCP server (copied from repo)
├── requirements.txt        # RAG Python deps
├── mcp_requirements.txt    # MCP Python deps
├── start.sh                # start the stack
├── stop.sh                 # stop the stack
├── status.sh               # GPU + container + RAG health
├── pull-models.sh          # pull Ollama models
├── Caddyfile.example       # reverse proxy config template
├── papers/                 # drop PDFs here for RAG indexing
├── repos/                  # git repos indexed by RAG
├── workspace/              # MCP working directory
├── index/                  # ChromaDB vector store (persistent)
├── kiwix/                  # ZIM files for Kiwix
├── gitea/                  # Gitea data
├── invokeai-outputs/       # InvokeAI generated images
└── logs/

---

First Run Checklist

1. Run setup:

   ./laptop_full_setup.sh
   

2. Pull models (prompted at end of setup, or run manually):

   bash ~/docker/ai-stack/pull-models.sh
   

   Downloads ~15-30GB. Takes 10-40 min depending on connection.

3. Add API tokens (optional — for Gitea/GitHub MCP tools):

   nano ~/docker/ai-stack/.env
   

4. Connect Claude Code to MCP:

   claude mcp add local http://<your-ip>:8002/sse
   

5. Download ZIMs for offline docs (optional, large):

   ./kiwix_download.sh
   

---

Updating

Re-run the setup script — it detects an existing install and skips prereqs:

./laptop_full_setup.sh
# or
./laptop_full_setup.sh --force   # also overwrites config files

---

GPU / Model Tiers (local-ai-setup.sh)

VRAM	Chat model	Code model	Context
≥ 14 GB	qwen2.5:14b	qwen2.5-coder:14b	32k
8–14 GB	qwen2.5:14b	qwen2.5-coder:7b	16k
4–8 GB	qwen2.5:7b	qwen2.5-coder:7b	8k
CPU	qwen2.5:7b	qwen2.5-coder:7b	4k

Embed model is always nomic-embed-text (required for RAG).