diff --git a/README.md b/README.md index c6c382b..1723882 100644 --- a/README.md +++ b/README.md @@ -135,6 +135,118 @@ Both scripts are **idempotent** — safe to re-run for updates. Config files are --- +## Querying Your Codebase from Open WebUI + +The stack includes a **code-aware RAG server** that sits between Open WebUI and Ollama. When you chat in Open WebUI, the RAG server automatically retrieves relevant code from your indexed repos and injects it into the prompt — so the model answers with your actual code as context. + +This is NOT the same as Open WebUI's built-in Knowledge Collections. This is a separate, always-on layer that understands code structure. + +### How it works (architecture) + +``` +You type a question in Open WebUI + ↓ +Open WebUI → RAG Server (port 8001) /v1/chat/completions + ↓ +RAG Server: searches ChromaDB for relevant code chunks + (AST-parsed Python functions, regex-split JS/Go/Rust, etc.) + ↓ +RAG Server: prepends code snippets to your prompt: + "### auth.py:authenticate + def authenticate(user, password): ..." + ↓ +RAG Server → Ollama: generates response WITH your code as context + ↓ +Response sent back to Open WebUI +``` + +Open WebUI is already configured to route through the RAG server via: +``` +OPENAI_API_BASE_URL=http://rag-server:8001/v1 +``` + +### Step 1: Index your code + +**Option A: Drop repos in the repos directory** +```bash +cd ~/docker/ai-stack/repos +git clone http://localhost:3001/your-user/your-repo.git +# Restart RAG server to trigger indexing: +docker restart rag-server +``` + +**Option B: Use the ingest API** +```bash +# From Gitea: +curl -X POST http://localhost:8001/ingest/repo \ + -H 'Content-Type: application/json' \ + -d '{"url": "http://gitea:3000/user/repo", "name": "my-repo"}' + +# From GitHub: +curl -X POST http://localhost:8001/ingest/repo \ + -H 'Content-Type: application/json' \ + -d '{"url": "https://github.com/user/repo", "name": "my-repo", "branch": "main"}' +``` + +**Option C: Auto-index on push (Gitea webhook)** +1. In Gitea → your repo → **Settings** → **Webhooks** → **Add Webhook** → **Gitea** +2. Target URL: `http://rag-server:8001/webhook/gitea` +3. Trigger: Push events +4. Now every `git push` to Gitea auto-reindexes that repo + +### Step 2: Chat about your code + +Just ask questions in Open WebUI. The RAG server automatically retrieves relevant code: + +- "What does the authenticate function do?" +- "How is the database connection configured?" +- "Show me all the API endpoints" +- "What tests exist for the user model?" + +The model sees the actual code snippets and responds based on them — not hallucinating. + +### What gets indexed + +**Supported file types:** +`.py`, `.js`, `.ts`, `.tsx`, `.jsx`, `.go`, `.rs`, `.java`, `.c`, `.cpp`, `.h`, `.hpp`, `.cs`, `.rb`, `.sh`, `.yaml`, `.yml`, `.toml`, `.sql`, `.md` + +**Smart chunking:** +- **Python:** AST-parsed — each function and class is its own searchable chunk +- **Other languages:** Regex-split on `function`, `class`, `const`, `func`, `impl`, etc. +- **Fallback:** Sliding window (1200 chars, 200 char overlap) + +**Skipped directories:** +`node_modules`, `.git`, `__pycache__`, `dist`, `build`, `.venv`, `venv`, `vendor`, `target`, `bin`, `obj` + +### Checking RAG status + +```bash +curl http://localhost:8001/health +``` + +Returns document counts per collection (code, papers) and overall status. + +### RAG server vs Knowledge Collections vs Memories + +These are **three separate systems** in the stack. Understanding the difference matters: + +| | **RAG Server** | **Knowledge Collections** | **Memories** | +|---|---|---|---| +| **What** | Code-aware retrieval layer | Open WebUI's built-in document RAG | Short facts about the user | +| **Content** | Git repos (auto-indexed) | PDFs, text files you upload | Extracted from conversations | +| **Chunking** | AST/regex (code-aware) | Generic text chunking | Single sentences | +| **Activation** | Always on (every chat) | Per-chat (`#` tag) | Global (every chat) | +| **Best for** | "What does this function do?" | Project docs, research notes | "Remember I prefer Python" | +| **Context cost** | ~1500-2500 tokens (top 6 chunks) | ~1500-2500 tokens (top K chunks) | ~200 tokens | + +**For code questions:** The RAG server handles this automatically — no setup needed beyond indexing your repos. + +**For project docs/notes:** Use Knowledge Collections — type `#` to scope per-chat. + +**For personal preferences:** Use Memories (sparingly — they're global). + +--- + ## Image Generation from Open WebUI Open WebUI can generate images inline in chat conversations using ComfyUI as the backend. When configured, you can ask any model to "generate an image of..." and it will call ComfyUI to create the image. @@ -450,13 +562,14 @@ If your knowledge collections aren't returning good results, tune these in **Adm | **Context window** | 4K-32K (model dependent) | 200K | | **Cross-session memory** | Manual handoffs or Auto Memory | Automatic (CLAUDE.md, project memory) | | **Project scoping** | Knowledge collections (manual) | Built-in (each project has its own context) | -| **Code awareness** | None — you paste code in | Reads your entire codebase | +| **Code awareness** | RAG server auto-retrieves relevant chunks from indexed repos | Reads your entire codebase directly | | **Continuation** | New chat + `#collection` handoff | "Let's finish X" just works | +| **Code editing** | Can't edit files (chat only) | Reads, writes, runs code directly | | **Cost** | Free (your electricity) | API usage fees | | **Privacy** | 100% local | Cloud-based | | **Offline** | Works without internet | Requires internet | -Local AI requires more manual workflow management. The tradeoff is privacy, cost, and offline capability. For code-heavy work, Claude Code is dramatically better. For private research, document Q&A, and learning — local models with knowledge collections work well once you get the handoff workflow down. +Local AI requires more manual workflow management but has real code awareness via the RAG server. For code-heavy editing and multi-step tasks, Claude Code is dramatically better. For private code Q&A, document research, and learning — the local stack with RAG + knowledge collections is solid. ### Channels (beta feature)