Add memory, knowledge, function install, and model sizing docs

- Manual function/action installation without community signup
- Auto Memory setup with Ollama configuration
- Memory vs Knowledge collections comparison (context impact)
- Context window consumption analysis (memories are ~200 tokens fixed,
  conversation history is the real context hog)
- Project-scoped memory workarounds (Knowledge collections recommended)
- System prompt fix for models outputting code instead of natural language
- VRAM reality check table (model file size != inference VRAM needed)
- Qwen 9B does NOT fit in 6GB VRAM despite setup script claiming so

https://claude.ai/code/session_01PtYTPherSJaxDEVPgF6Nxu

README.md

@@ -220,12 +220,190 @@ If you prefer AUTOMATIC1111 over ComfyUI:
 | `AUTOMATIC1111_BASE_URL` | — | AUTOMATIC1111 API URL |
 | `AUTOMATIC1111_API_AUTH` | — | Auth credentials (`user:pass`) |
 
-### Recommended Open WebUI Functions
+### Installing Functions & Actions (without community signup)
 
-Install these from **Admin → Functions → ＋ → Import From Link** or search in Discover:
+The Open WebUI community hub (`openwebui.com`) requires a free account to download functions. If you get parse errors pasting URLs or don't want to sign up, you can install functions manually by pasting their source code directly.
 
-- **Auto Memory** — Automatically stores relevant info as persistent memories across chats
+#### Manual installation (no account needed)
-- **Generate Image** — Adds a "Generate Image" action button to messages for quick re-generation
+
+1. Find the function's source on GitHub — most are in the [open-webui/functions](https://github.com/open-webui/functions) repo or linked from community pages
+2. Copy the **raw Python source code** (the entire `.py` file)
+3. In Open WebUI → **Workspace** → **Functions** → click **＋** (Create)
+4. Paste the code into the editor
+5. Give it a name and save
+6. Enable it under **Workspace** → **Functions** — toggle it on globally or per-model
+
+For **Actions** (like Generate Image), the process is the same but go to **Workspace** → **Functions** and set the function type to **Action** in the metadata.
+
+#### Recommended functions
+
+| Function | Type | What it does |
+|----------|------|-------------|
+| **Auto Memory** | Filter | Automatically extracts and stores facts from conversations as persistent memories |
+| **Generate Image** | Action | Adds a "Generate Image" button to messages for quick image generation |
+
+### Auto Memory setup
+
+Auto Memory is a filter function that runs after each message exchange, uses an LLM call to extract noteworthy facts, and stores them as user memories in Open WebUI's built-in memory system.
+
+#### Step 1: Install the function
+
+Use the manual method above. The source code is at:
+`https://github.com/open-webui/functions` — search for `auto_memory` or `adaptive_memory`
+
+#### Step 2: Configure the function settings
+
+After installing, click the **gear icon** on the function to configure:
+
+| Setting | Value for local stack | Notes |
+|---------|----------------------|-------|
+| `openai_api_url` | `http://ollama:11434/v1` | Ollama's OpenAI-compatible endpoint |
+| `model` | Your chat model (e.g. `qwen2.5:14b`) | Used for memory extraction LLM calls |
+| `api_key` | `ollama` | Any non-empty string — Ollama ignores auth |
+| `context_window_n` | `4` (default) | Number of related memories injected per message |
+| `similarity_score_filter` | Leave default | Cosine similarity threshold for memory retrieval |
+| `messages_to_consider` | Leave default | How many recent messages to analyze |
+| `allow_modify_user_memories` | `false` | **Keep off** — lets LLM delete/edit memories via API if enabled |
+
+#### Step 3: Enable memories per user
+
+Each user must enable the memory feature individually:
+1. Click your **profile icon** → **Settings** → **Personalization**
+2. Toggle **Memory** to **ON**
+
+Without this, the function installs but does nothing.
+
+### Generate Image action setup
+
+1. Install via manual method above (search for `generate_image` in the functions repo)
+2. Configure the function settings:
+   - Uses your existing Open WebUI image generation settings (ComfyUI/A1111)
+   - No additional API configuration needed if image generation already works
+3. After installing, a **"Generate Image"** button appears on AI messages
+4. Clicking it sends the message content as an image generation prompt
+
+### Memory system: how it works and context impact
+
+Open WebUI has two separate systems for persistent knowledge. Understanding the difference is critical for managing your context window.
+
+#### Built-in Memories vs Knowledge Collections
+
+| | **Memories** | **Knowledge** |
+|---|---|---|
+| **What** | Short facts extracted from chats | Document collections (PDFs, text files) |
+| **How stored** | Text + vector embeddings in DB | Chunked documents + embeddings in ChromaDB |
+| **How injected** | Prepended to system prompt every message | Retrieved via RAG only when relevant chunks match |
+| **Scope** | **Global per user** — injected into ALL chats | **Per-chat** — you choose which collection with `#` |
+| **Context cost** | Always consumed, every message | Only consumed when you tag a collection |
+| **Control** | All or nothing (on/off globally) | Fine-grained (pick per conversation) |
+
+#### Context window consumption by memories
+
+Each memory is roughly 20–80 tokens (a sentence or two). With the default `context_window_n=4` setting in Auto Memory, 4 related memories are injected per message.
+
+**Estimated context consumption per message:**
+
+| Stored memories | Injected per msg | Tokens consumed | % of 8K context | % of 32K context |
+|----------------|-----------------|-----------------|-----------------|------------------|
+| 10 | ~4 | ~200 | 2.5% | 0.6% |
+| 50 | ~4 | ~200 | 2.5% | 0.6% |
+| 200 | ~4 | ~200 | 2.5% | 0.6% |
+
+The key insight: **only the top N similar memories are injected** (default 4), not all of them. So having 200 memories doesn't consume more context than having 10 — the retrieval system picks the most relevant ones.
+
+However, **the memories are injected on every single message** in every chat. This is the overhead cost.
+
+#### Context consumed by chat history (the bigger problem)
+
+The real context pressure comes from **conversation history**, not memories. Here's what actually fills your context window:
+
+| Messages in chat | ~Tokens used | % of 8K | % of 32K |
+|-----------------|-------------|---------|----------|
+| 5 exchanges (10 msgs) | ~2,000–4,000 | 25–50% | 6–12% |
+| 20 exchanges (40 msgs) | ~8,000–16,000 | **100%+ (truncated)** | 25–50% |
+| 100 exchanges (200 msgs) | ~40,000–80,000 | **way over** | **100%+ (truncated)** |
+
+With a small model running 8K context, you'll hit the limit after ~10-20 exchanges. The model starts dropping earlier messages. Memories add a small fixed overhead (~200 tokens) on top of this.
+
+#### Across multiple separate chats
+
+**Good news:** separate chats do NOT share context windows. Each chat starts fresh. The only cross-chat cost is the ~200 tokens of memories injected into each new chat's system prompt.
+
+So 5, 20, or 100 separate chats don't accumulate — each one independently uses the context window. Memories are the only thing that carries over.
+
+### Project-scoped memory (avoiding global memory pollution)
+
+**Open WebUI does NOT have native project-scoped memory.** Memories are global per user — every fact extracted from any chat gets injected into every other chat.
+
+This is a known limitation. Here are workarounds:
+
+#### Option 1: Use Knowledge Collections instead of Memories (recommended)
+
+Knowledge collections give you the scoping you want:
+
+1. **Create a collection:** Workspace → Knowledge → Create Collection (e.g. "GPU Research Project")
+2. **Add documents:** Upload PDFs, text files, or paste notes into the collection
+3. **Use per-chat:** In any chat, type `#` and select your collection — only that chat gets the context
+4. **One-off chats stay clean:** Don't tag a collection, and no project context is injected
+
+This is the closest thing to "projects" in Open WebUI. You can have separate knowledge collections for separate projects, and only pull them in when relevant.
+
+#### Option 2: Disable Auto Memory, use manual memories
+
+1. Turn off the Auto Memory function
+2. Manually add memories via **Profile → Settings → Personalization → Memories**
+3. Keep only universally useful facts (your name, preferences, etc.)
+4. Use Knowledge collections for project-specific context
+
+#### Option 3: Periodically clear memories
+
+**Profile → Settings → Personalization → Memories → Clear All** — nuclear option, but keeps things clean between projects.
+
+### Channels (beta feature)
+
+Channels are persistent chat rooms (like Slack/Discord channels) with multi-model support. They do NOT scope memories differently — memories are still global per user. Channels are useful for team collaboration, not memory isolation.
+
+### Fixing models that output code instead of natural language
+
+If your model (especially smaller ones like Qwen 3.5) responds with Python code blocks instead of plain English answers (as shown in the screenshot), this is a common behavior with code-optimized models.
+
+#### Why it happens
+
+- Small models like `qwen3.5:4b` are heavily optimized for coding tasks
+- They interpret ambiguous questions as "write me a program" instead of "answer my question"
+- The model's training data skews toward code generation at smaller parameter counts
+- Thinking/reasoning mode (if enabled) amplifies this tendency
+
+#### Fix: Set a system prompt in Open WebUI
+
+1. Go to **Admin** → **Settings** → **Interface** → **Default System Prompt** (for all chats)
+
+   Or per-model: **Workspace** → **Models** → select model → **System Prompt**
+
+2. Use a system prompt like:
+
+```
+You are a helpful assistant. Answer questions in clear, natural language.
+Explain concepts conversationally. Only include code if the user explicitly
+asks for code or a script. When discussing technical topics, use plain
+English explanations with examples, not programs.
+```
+
+3. For the specific GPU pricing question in the screenshot, the model should have responded with a comparison table in text, not a Python script. A good system prompt prevents this.
+
+#### Alternative: Use a chat-optimized model
+
+Some models are better at conversational responses:
+
+| Model | Size | Better for chat? |
+|-------|------|-----------------|
+| `qwen2.5:14b` | 14B | Yes — more balanced |
+| `qwen2.5:7b` | 7B | Yes — good general chat |
+| `llama3.1:8b` | 8B | Yes — conversational |
+| `qwen2.5-coder:7b` | 7B | No — code-focused |
+| `qwen3.5:4b` | 4B | No — too small, code-biased |
+
+Avoid using `-coder` variants or very small models (≤4B) for general chat. Use them only when you actually want code.
 
 ### VRAM considerations
 
@@ -295,3 +473,25 @@ Re-run the setup script — it detects an existing install and skips prereqs:
 | CPU     | qwen2.5:7b      | qwen2.5-coder:7b      | 4k      |
 
 Embed model is always `nomic-embed-text` (required for RAG).
+
+### VRAM reality check
+
+Ollama will **always try to run** any model — it silently offloads layers to CPU when VRAM is insufficient. The model still works but gets significantly slower. The setup script's "fully in VRAM" label can be misleading.
+
+**Actual VRAM needed for common models (Q4_K_M quantization):**
+
+| Model | Download size | VRAM for inference | Fits in 6GB? | Fits in 8GB? |
+|-------|-------------|-------------------|-------------|-------------|
+| `qwen3.5:4b` | ~2.5 GB | ~3.5–4 GB | Yes | Yes |
+| `qwen2.5:7b` | ~4.4 GB | ~5.5–6 GB | Tight | Yes |
+| `qwen3.5:9b` | ~5.5 GB | ~6.5–7 GB | **No — partial CPU offload** | Tight |
+| `qwen2.5:14b` | ~8.7 GB | ~10–11 GB | No | No |
+| `qwen3.5-35b-a3b` (MoE) | ~20 GB | ~3.5 GB active | Yes (only 3B active) | Yes |
+
+**Why the file size != VRAM needed:** Inference requires additional memory for KV cache, attention buffers, and CUDA overhead. Expect ~1-2 GB more than the model file size.
+
+**Signs of CPU offload (model too big for your VRAM):**
+- Tokens per second drops from 20-40 to 2-8
+- `nvidia-smi` shows VRAM maxed out
+- CPU usage spikes during generation
+- First token takes much longer than usual