Add README with full setup instructions and script reference
Documents workflow, service URLs, day-to-day commands, generated file layout, first-run checklist, GPU/model tiers, and what each script does. https://claude.ai/code/session_012gDnantBmFTWZGCiKyjazx
This commit is contained in:
Claude
@@ -0,0 +1,155 @@
|
||||
# Local AI Stack
|
||||
|
||||
A fully offline, self-hosted AI environment for Ubuntu 24.04. Runs on any NVIDIA GPU (or CPU-only).
|
||||
|
||||
**Services:** Ollama · Open WebUI · RAG · MCP · ChromaDB · SearXNG · Kiwix · Gitea · InvokeAI · Portainer
|
||||
|
||||
---
|
||||
|
||||
## Quick Start
|
||||
|
||||
```bash
|
||||
git clone <this-repo>
|
||||
cd local-ai
|
||||
./laptop_full_setup.sh
|
||||
```
|
||||
|
||||
That's it. The script installs Docker, NVIDIA drivers (if needed), generates all config, starts the stack, and optionally pulls models.
|
||||
|
||||
---
|
||||
|
||||
## Service URLs
|
||||
|
||||
After setup, all services are available on your LAN:
|
||||
|
||||
| Service | URL | Purpose |
|
||||
|------------|----------------------------|--------------------------------|
|
||||
| Open WebUI | `http://<ip>:3000` | Chat interface (Ollama + RAG) |
|
||||
| InvokeAI | `http://<ip>:9090` | Image generation |
|
||||
| SearXNG | `http://<ip>:8888` | Private web search |
|
||||
| Kiwix | `http://<ip>:8181` | Offline Wikipedia / docs |
|
||||
| Gitea | `http://<ip>:3001` | Self-hosted Git |
|
||||
| RAG Health | `http://<ip>:8001/health` | RAG server status |
|
||||
| MCP SSE | `http://<ip>:8002/sse` | MCP endpoint for Claude Code |
|
||||
| Portainer | `https://<ip>:9443` | Docker management UI |
|
||||
|
||||
---
|
||||
|
||||
## Day-to-Day Commands
|
||||
|
||||
All generated into `~/docker/ai-stack/` by the setup script:
|
||||
|
||||
```bash
|
||||
bash ~/docker/ai-stack/start.sh # pull latest images + docker compose up -d
|
||||
bash ~/docker/ai-stack/stop.sh # docker compose down
|
||||
bash ~/docker/ai-stack/status.sh # GPU / container / RAG health
|
||||
bash ~/docker/ai-stack/pull-models.sh # pull Ollama models (run once after first install)
|
||||
```
|
||||
|
||||
The stack also registers as a **systemd service** that starts on boot:
|
||||
|
||||
```bash
|
||||
sudo systemctl start local-ai
|
||||
sudo systemctl stop local-ai
|
||||
sudo systemctl status local-ai
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Script Reference
|
||||
|
||||
| Script | Lines | What it does |
|
||||
|---|---|---|
|
||||
| `laptop_full_setup.sh` | 620 | **Main setup.** Installs Docker + NVIDIA toolkit, creates `~/docker/ai-stack/`, writes `docker-compose.yml`, starts stack, registers systemd service. |
|
||||
| `local-ai-setup.sh` | 837 | Alternative setup script. Same as above but also **auto-detects VRAM** and selects models accordingly (14B for ≥14GB VRAM, 7B for CPU). Use this instead of `laptop_full_setup.sh` if you want VRAM-aware model selection. |
|
||||
| `ubuntu-post-install.sh` | 8,889 | Full Ubuntu 24.04 post-install (dev tools, fonts, apps, tweaks). Run once on a fresh OS install. Independent of the AI stack. |
|
||||
| `configure-storage.sh` | 239 | Storage/mount configuration helper. Run separately if you have a secondary drive for AI data. |
|
||||
| `kiwix_download.sh` | 198 | Downloads ZIM files (Wikipedia, Stack Overflow, etc.) for offline use. Run separately — files are large. |
|
||||
|
||||
### Which setup script should I use?
|
||||
|
||||
- `laptop_full_setup.sh` — fixed model selection (`qwen2.5:14b` / `qwen2.5-coder:7b`), simpler
|
||||
- `local-ai-setup.sh` — detects your VRAM at runtime and picks appropriate models, also embeds `server.py` and `mcp_server.py` directly (doesn't need repo files copied separately)
|
||||
|
||||
Both scripts are **idempotent** — safe to re-run for updates. Config files are kept on re-run unless you pass `--force`.
|
||||
|
||||
---
|
||||
|
||||
## Generated File Layout
|
||||
|
||||
```
|
||||
~/docker/ai-stack/
|
||||
├── docker-compose.yml # generated by setup script
|
||||
├── .env # API tokens — edit this, never overwritten
|
||||
├── server.py # RAG server (copied from repo)
|
||||
├── mcp_server.py # MCP server (copied from repo)
|
||||
├── requirements.txt # RAG Python deps
|
||||
├── mcp_requirements.txt # MCP Python deps
|
||||
├── start.sh # start the stack
|
||||
├── stop.sh # stop the stack
|
||||
├── status.sh # GPU + container + RAG health
|
||||
├── pull-models.sh # pull Ollama models
|
||||
├── Caddyfile.example # reverse proxy config template
|
||||
├── papers/ # drop PDFs here for RAG indexing
|
||||
├── repos/ # git repos indexed by RAG
|
||||
├── workspace/ # MCP working directory
|
||||
├── index/ # ChromaDB vector store (persistent)
|
||||
├── kiwix/ # ZIM files for Kiwix
|
||||
├── gitea/ # Gitea data
|
||||
├── invokeai-outputs/ # InvokeAI generated images
|
||||
└── logs/
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## First Run Checklist
|
||||
|
||||
1. **Run setup:**
|
||||
```bash
|
||||
./laptop_full_setup.sh
|
||||
```
|
||||
2. **Pull models** (prompted at end of setup, or run manually):
|
||||
```bash
|
||||
bash ~/docker/ai-stack/pull-models.sh
|
||||
```
|
||||
Downloads ~15-30GB. Takes 10-40 min depending on connection.
|
||||
|
||||
3. **Add API tokens** (optional — for Gitea/GitHub MCP tools):
|
||||
```bash
|
||||
nano ~/docker/ai-stack/.env
|
||||
```
|
||||
|
||||
4. **Connect Claude Code to MCP:**
|
||||
```bash
|
||||
claude mcp add local http://<your-ip>:8002/sse
|
||||
```
|
||||
|
||||
5. **Download ZIMs** for offline docs (optional, large):
|
||||
```bash
|
||||
./kiwix_download.sh
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Updating
|
||||
|
||||
Re-run the setup script — it detects an existing install and skips prereqs:
|
||||
|
||||
```bash
|
||||
./laptop_full_setup.sh
|
||||
# or
|
||||
./laptop_full_setup.sh --force # also overwrites config files
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## GPU / Model Tiers (`local-ai-setup.sh`)
|
||||
|
||||
| VRAM | Chat model | Code model | Context |
|
||||
|---------|-----------------|-----------------------|---------|
|
||||
| ≥ 14 GB | qwen2.5:14b | qwen2.5-coder:14b | 32k |
|
||||
| 8–14 GB | qwen2.5:14b | qwen2.5-coder:7b | 16k |
|
||||
| 4–8 GB | qwen2.5:7b | qwen2.5-coder:7b | 8k |
|
||||
| CPU | qwen2.5:7b | qwen2.5-coder:7b | 4k |
|
||||
|
||||
Embed model is always `nomic-embed-text` (required for RAG).
|
||||
Reference in New Issue
Block a user