Add README with full setup instructions and script reference
Documents workflow, service URLs, day-to-day commands, generated file layout, first-run checklist, GPU/model tiers, and what each script does. https://claude.ai/code/session_012gDnantBmFTWZGCiKyjazx
This commit is contained in:
@@ -0,0 +1,155 @@
|
|||||||
|
# Local AI Stack
|
||||||
|
|
||||||
|
A fully offline, self-hosted AI environment for Ubuntu 24.04. Runs on any NVIDIA GPU (or CPU-only).
|
||||||
|
|
||||||
|
**Services:** Ollama · Open WebUI · RAG · MCP · ChromaDB · SearXNG · Kiwix · Gitea · InvokeAI · Portainer
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Quick Start
|
||||||
|
|
||||||
|
```bash
|
||||||
|
git clone <this-repo>
|
||||||
|
cd local-ai
|
||||||
|
./laptop_full_setup.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
That's it. The script installs Docker, NVIDIA drivers (if needed), generates all config, starts the stack, and optionally pulls models.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Service URLs
|
||||||
|
|
||||||
|
After setup, all services are available on your LAN:
|
||||||
|
|
||||||
|
| Service | URL | Purpose |
|
||||||
|
|------------|----------------------------|--------------------------------|
|
||||||
|
| Open WebUI | `http://<ip>:3000` | Chat interface (Ollama + RAG) |
|
||||||
|
| InvokeAI | `http://<ip>:9090` | Image generation |
|
||||||
|
| SearXNG | `http://<ip>:8888` | Private web search |
|
||||||
|
| Kiwix | `http://<ip>:8181` | Offline Wikipedia / docs |
|
||||||
|
| Gitea | `http://<ip>:3001` | Self-hosted Git |
|
||||||
|
| RAG Health | `http://<ip>:8001/health` | RAG server status |
|
||||||
|
| MCP SSE | `http://<ip>:8002/sse` | MCP endpoint for Claude Code |
|
||||||
|
| Portainer | `https://<ip>:9443` | Docker management UI |
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Day-to-Day Commands
|
||||||
|
|
||||||
|
All generated into `~/docker/ai-stack/` by the setup script:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash ~/docker/ai-stack/start.sh # pull latest images + docker compose up -d
|
||||||
|
bash ~/docker/ai-stack/stop.sh # docker compose down
|
||||||
|
bash ~/docker/ai-stack/status.sh # GPU / container / RAG health
|
||||||
|
bash ~/docker/ai-stack/pull-models.sh # pull Ollama models (run once after first install)
|
||||||
|
```
|
||||||
|
|
||||||
|
The stack also registers as a **systemd service** that starts on boot:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo systemctl start local-ai
|
||||||
|
sudo systemctl stop local-ai
|
||||||
|
sudo systemctl status local-ai
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Script Reference
|
||||||
|
|
||||||
|
| Script | Lines | What it does |
|
||||||
|
|---|---|---|
|
||||||
|
| `laptop_full_setup.sh` | 620 | **Main setup.** Installs Docker + NVIDIA toolkit, creates `~/docker/ai-stack/`, writes `docker-compose.yml`, starts stack, registers systemd service. |
|
||||||
|
| `local-ai-setup.sh` | 837 | Alternative setup script. Same as above but also **auto-detects VRAM** and selects models accordingly (14B for ≥14GB VRAM, 7B for CPU). Use this instead of `laptop_full_setup.sh` if you want VRAM-aware model selection. |
|
||||||
|
| `ubuntu-post-install.sh` | 8,889 | Full Ubuntu 24.04 post-install (dev tools, fonts, apps, tweaks). Run once on a fresh OS install. Independent of the AI stack. |
|
||||||
|
| `configure-storage.sh` | 239 | Storage/mount configuration helper. Run separately if you have a secondary drive for AI data. |
|
||||||
|
| `kiwix_download.sh` | 198 | Downloads ZIM files (Wikipedia, Stack Overflow, etc.) for offline use. Run separately — files are large. |
|
||||||
|
|
||||||
|
### Which setup script should I use?
|
||||||
|
|
||||||
|
- `laptop_full_setup.sh` — fixed model selection (`qwen2.5:14b` / `qwen2.5-coder:7b`), simpler
|
||||||
|
- `local-ai-setup.sh` — detects your VRAM at runtime and picks appropriate models, also embeds `server.py` and `mcp_server.py` directly (doesn't need repo files copied separately)
|
||||||
|
|
||||||
|
Both scripts are **idempotent** — safe to re-run for updates. Config files are kept on re-run unless you pass `--force`.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Generated File Layout
|
||||||
|
|
||||||
|
```
|
||||||
|
~/docker/ai-stack/
|
||||||
|
├── docker-compose.yml # generated by setup script
|
||||||
|
├── .env # API tokens — edit this, never overwritten
|
||||||
|
├── server.py # RAG server (copied from repo)
|
||||||
|
├── mcp_server.py # MCP server (copied from repo)
|
||||||
|
├── requirements.txt # RAG Python deps
|
||||||
|
├── mcp_requirements.txt # MCP Python deps
|
||||||
|
├── start.sh # start the stack
|
||||||
|
├── stop.sh # stop the stack
|
||||||
|
├── status.sh # GPU + container + RAG health
|
||||||
|
├── pull-models.sh # pull Ollama models
|
||||||
|
├── Caddyfile.example # reverse proxy config template
|
||||||
|
├── papers/ # drop PDFs here for RAG indexing
|
||||||
|
├── repos/ # git repos indexed by RAG
|
||||||
|
├── workspace/ # MCP working directory
|
||||||
|
├── index/ # ChromaDB vector store (persistent)
|
||||||
|
├── kiwix/ # ZIM files for Kiwix
|
||||||
|
├── gitea/ # Gitea data
|
||||||
|
├── invokeai-outputs/ # InvokeAI generated images
|
||||||
|
└── logs/
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## First Run Checklist
|
||||||
|
|
||||||
|
1. **Run setup:**
|
||||||
|
```bash
|
||||||
|
./laptop_full_setup.sh
|
||||||
|
```
|
||||||
|
2. **Pull models** (prompted at end of setup, or run manually):
|
||||||
|
```bash
|
||||||
|
bash ~/docker/ai-stack/pull-models.sh
|
||||||
|
```
|
||||||
|
Downloads ~15-30GB. Takes 10-40 min depending on connection.
|
||||||
|
|
||||||
|
3. **Add API tokens** (optional — for Gitea/GitHub MCP tools):
|
||||||
|
```bash
|
||||||
|
nano ~/docker/ai-stack/.env
|
||||||
|
```
|
||||||
|
|
||||||
|
4. **Connect Claude Code to MCP:**
|
||||||
|
```bash
|
||||||
|
claude mcp add local http://<your-ip>:8002/sse
|
||||||
|
```
|
||||||
|
|
||||||
|
5. **Download ZIMs** for offline docs (optional, large):
|
||||||
|
```bash
|
||||||
|
./kiwix_download.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## Updating
|
||||||
|
|
||||||
|
Re-run the setup script — it detects an existing install and skips prereqs:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./laptop_full_setup.sh
|
||||||
|
# or
|
||||||
|
./laptop_full_setup.sh --force # also overwrites config files
|
||||||
|
```
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
|
## GPU / Model Tiers (`local-ai-setup.sh`)
|
||||||
|
|
||||||
|
| VRAM | Chat model | Code model | Context |
|
||||||
|
|---------|-----------------|-----------------------|---------|
|
||||||
|
| ≥ 14 GB | qwen2.5:14b | qwen2.5-coder:14b | 32k |
|
||||||
|
| 8–14 GB | qwen2.5:14b | qwen2.5-coder:7b | 16k |
|
||||||
|
| 4–8 GB | qwen2.5:7b | qwen2.5-coder:7b | 8k |
|
||||||
|
| CPU | qwen2.5:7b | qwen2.5-coder:7b | 4k |
|
||||||
|
|
||||||
|
Embed model is always `nomic-embed-text` (required for RAG).
|
||||||
Reference in New Issue
Block a user