Merge pull request #26 from outis1one/claude/gpu-setup-research-c78YT
Claude/gpu setup research c78 yt
This commit is contained in:
Outis
@@ -67,6 +67,9 @@ sudo systemctl status local-ai
|
||||
| `configure-storage.sh` | 239 | Storage/mount configuration helper. Run separately if you have a secondary drive for AI data. |
|
||||
| `kiwix_download.sh` | 198 | Downloads ZIM files (Wikipedia, Stack Overflow, etc.) for offline use. Run separately — files are large. |
|
||||
| `invokeai-import-lora.sh` | 85 | Copies a LoRA `.safetensors` file into InvokeAI's Docker model volume. |
|
||||
| `comfyui-import-lora.sh` | 97 | Copies a LoRA into ComfyUI and prints workflow setup instructions. |
|
||||
| `comfyui-install-ipadapter.sh` | 185 | Installs IP-Adapter nodes + models into ComfyUI for reference-image workflows (same face, different settings). |
|
||||
| `setup-image-models.sh` | 200 | **GPU-aware** image model installer. Detects VRAM, offers appropriate SD/SDXL/Flux models, installs into InvokeAI and/or ComfyUI. |
|
||||
|
||||
### Which setup script should I use?
|
||||
|
||||
@@ -135,6 +138,118 @@ Both scripts are **idempotent** — safe to re-run for updates. Config files are
|
||||
|
||||
---
|
||||
|
||||
## Querying Your Codebase from Open WebUI
|
||||
|
||||
The stack includes a **code-aware RAG server** that sits between Open WebUI and Ollama. When you chat in Open WebUI, the RAG server automatically retrieves relevant code from your indexed repos and injects it into the prompt — so the model answers with your actual code as context.
|
||||
|
||||
This is NOT the same as Open WebUI's built-in Knowledge Collections. This is a separate, always-on layer that understands code structure.
|
||||
|
||||
### How it works (architecture)
|
||||
|
||||
```
|
||||
You type a question in Open WebUI
|
||||
↓
|
||||
Open WebUI → RAG Server (port 8001) /v1/chat/completions
|
||||
↓
|
||||
RAG Server: searches ChromaDB for relevant code chunks
|
||||
(AST-parsed Python functions, regex-split JS/Go/Rust, etc.)
|
||||
↓
|
||||
RAG Server: prepends code snippets to your prompt:
|
||||
"### auth.py:authenticate
|
||||
def authenticate(user, password): ..."
|
||||
↓
|
||||
RAG Server → Ollama: generates response WITH your code as context
|
||||
↓
|
||||
Response sent back to Open WebUI
|
||||
```
|
||||
|
||||
Open WebUI is already configured to route through the RAG server via:
|
||||
```
|
||||
OPENAI_API_BASE_URL=http://rag-server:8001/v1
|
||||
```
|
||||
|
||||
### Step 1: Index your code
|
||||
|
||||
**Option A: Drop repos in the repos directory**
|
||||
```bash
|
||||
cd ~/docker/ai-stack/repos
|
||||
git clone http://localhost:3001/your-user/your-repo.git
|
||||
# Restart RAG server to trigger indexing:
|
||||
docker restart rag-server
|
||||
```
|
||||
|
||||
**Option B: Use the ingest API**
|
||||
```bash
|
||||
# From Gitea:
|
||||
curl -X POST http://localhost:8001/ingest/repo \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"url": "http://gitea:3000/user/repo", "name": "my-repo"}'
|
||||
|
||||
# From GitHub:
|
||||
curl -X POST http://localhost:8001/ingest/repo \
|
||||
-H 'Content-Type: application/json' \
|
||||
-d '{"url": "https://github.com/user/repo", "name": "my-repo", "branch": "main"}'
|
||||
```
|
||||
|
||||
**Option C: Auto-index on push (Gitea webhook)**
|
||||
1. In Gitea → your repo → **Settings** → **Webhooks** → **Add Webhook** → **Gitea**
|
||||
2. Target URL: `http://rag-server:8001/webhook/gitea`
|
||||
3. Trigger: Push events
|
||||
4. Now every `git push` to Gitea auto-reindexes that repo
|
||||
|
||||
### Step 2: Chat about your code
|
||||
|
||||
Just ask questions in Open WebUI. The RAG server automatically retrieves relevant code:
|
||||
|
||||
- "What does the authenticate function do?"
|
||||
- "How is the database connection configured?"
|
||||
- "Show me all the API endpoints"
|
||||
- "What tests exist for the user model?"
|
||||
|
||||
The model sees the actual code snippets and responds based on them — not hallucinating.
|
||||
|
||||
### What gets indexed
|
||||
|
||||
**Supported file types:**
|
||||
`.py`, `.js`, `.ts`, `.tsx`, `.jsx`, `.go`, `.rs`, `.java`, `.c`, `.cpp`, `.h`, `.hpp`, `.cs`, `.rb`, `.sh`, `.yaml`, `.yml`, `.toml`, `.sql`, `.md`
|
||||
|
||||
**Smart chunking:**
|
||||
- **Python:** AST-parsed — each function and class is its own searchable chunk
|
||||
- **Other languages:** Regex-split on `function`, `class`, `const`, `func`, `impl`, etc.
|
||||
- **Fallback:** Sliding window (1200 chars, 200 char overlap)
|
||||
|
||||
**Skipped directories:**
|
||||
`node_modules`, `.git`, `__pycache__`, `dist`, `build`, `.venv`, `venv`, `vendor`, `target`, `bin`, `obj`
|
||||
|
||||
### Checking RAG status
|
||||
|
||||
```bash
|
||||
curl http://localhost:8001/health
|
||||
```
|
||||
|
||||
Returns document counts per collection (code, papers) and overall status.
|
||||
|
||||
### RAG server vs Knowledge Collections vs Memories
|
||||
|
||||
These are **three separate systems** in the stack. Understanding the difference matters:
|
||||
|
||||
| | **RAG Server** | **Knowledge Collections** | **Memories** |
|
||||
|---|---|---|---|
|
||||
| **What** | Code-aware retrieval layer | Open WebUI's built-in document RAG | Short facts about the user |
|
||||
| **Content** | Git repos (auto-indexed) | PDFs, text files you upload | Extracted from conversations |
|
||||
| **Chunking** | AST/regex (code-aware) | Generic text chunking | Single sentences |
|
||||
| **Activation** | Always on (every chat) | Per-chat (`#` tag) | Global (every chat) |
|
||||
| **Best for** | "What does this function do?" | Project docs, research notes | "Remember I prefer Python" |
|
||||
| **Context cost** | ~1500-2500 tokens (top 6 chunks) | ~1500-2500 tokens (top K chunks) | ~200 tokens |
|
||||
|
||||
**For code questions:** The RAG server handles this automatically — no setup needed beyond indexing your repos.
|
||||
|
||||
**For project docs/notes:** Use Knowledge Collections — type `#` to scope per-chat.
|
||||
|
||||
**For personal preferences:** Use Memories (sparingly — they're global).
|
||||
|
||||
---
|
||||
|
||||
## Image Generation from Open WebUI
|
||||
|
||||
Open WebUI can generate images inline in chat conversations using ComfyUI as the backend. When configured, you can ask any model to "generate an image of..." and it will call ComfyUI to create the image.
|
||||
@@ -191,6 +306,134 @@ In any Open WebUI chat, type something like:
|
||||
|
||||
The model will detect the image generation request and pass it to ComfyUI.
|
||||
|
||||
### Using LoRAs with Open WebUI (workflow-per-style)
|
||||
|
||||
LoRAs let you apply trained styles (anime, photorealistic, specific characters, etc.) to image generation. The trick: **bake the LoRA into a ComfyUI workflow, export it, and import into Open WebUI.** After that, you iterate from chat without touching ComfyUI.
|
||||
|
||||
#### Step 1: Import your LoRA file
|
||||
|
||||
```bash
|
||||
./comfyui-import-lora.sh ~/Downloads/my-style-lora.safetensors "Anime Style"
|
||||
```
|
||||
|
||||
Or manually:
|
||||
```bash
|
||||
docker cp ~/Downloads/my-style-lora.safetensors comfyui:/opt/ComfyUI/models/loras/
|
||||
```
|
||||
|
||||
#### Step 2: Build a workflow with the LoRA (one-time)
|
||||
|
||||
1. Open ComfyUI at `http://<ip>:8188`
|
||||
2. Load the default text-to-image workflow
|
||||
3. Add a **Load LoRA** node: right-click → Add Node → loaders → Load LoRA
|
||||
4. Wire it between the checkpoint and the rest of the pipeline:
|
||||
```
|
||||
[Load Checkpoint] → MODEL → [Load LoRA] → MODEL → [KSampler]
|
||||
→ CLIP → → CLIP → [CLIP Text Encode]
|
||||
```
|
||||
5. Select your LoRA file, set `strength_model` and `strength_clip` (start 0.7–0.85)
|
||||
6. Test it — click **Queue Prompt** and verify it works
|
||||
7. Enable **Dev Mode** (gear icon) → click **Save (API Format)**
|
||||
|
||||
#### Step 3: Import into Open WebUI
|
||||
|
||||
1. Open WebUI → **Admin** → **Settings** → **Images**
|
||||
2. Click **Import Workflow** → upload the `workflow_api.json`
|
||||
3. Map the prompt node (usually CLIPTextEncode)
|
||||
4. Save
|
||||
|
||||
Now every "Generate an image of..." in chat uses your LoRA automatically.
|
||||
|
||||
#### Combining multiple LoRAs in one image
|
||||
|
||||
You can chain multiple Load LoRA nodes to blend styles:
|
||||
|
||||
```
|
||||
[Load Checkpoint] → [Load LoRA 1] → [Load LoRA 2] → [KSampler]
|
||||
(anime, 0.7) (lighting, 0.5)
|
||||
```
|
||||
|
||||
Each LoRA has its own `strength_model` / `strength_clip` sliders. Export the combined workflow as usual — both LoRAs are baked in and always applied.
|
||||
|
||||
#### Managing multiple LoRA styles
|
||||
|
||||
Each LoRA combo needs its own exported workflow. Practical approach:
|
||||
|
||||
| Workflow file | LoRAs | Use case |
|
||||
|---------------|-------|----------|
|
||||
| `anime-only.json` | Anime @ 0.8 | Anime style |
|
||||
| `anime-cinematic.json` | Anime @ 0.7 + Cinematic @ 0.5 | Anime + dramatic lighting |
|
||||
| `photorealistic.json` | Photorealistic @ 0.9 | Photo look |
|
||||
| `base-model.json` | None | No LoRA, base checkpoint only |
|
||||
|
||||
Switch between them in **Admin → Settings → Images → Import Workflow**. The active workflow applies to all image generation requests.
|
||||
|
||||
> **Important:** There are no keywords or trigger words to activate LoRAs from chat. LoRAs are hardwired in the workflow JSON — whatever prompt you type, the LoRA(s) always apply. To change styles, swap the workflow in OWUI settings.
|
||||
|
||||
#### LoRA compatibility
|
||||
|
||||
| LoRA trained on | Must use checkpoint |
|
||||
|----------------|-------------------|
|
||||
| SD 1.5 | Any SD 1.5 model (e.g. `v1-5-pruned-emaonly.safetensors`) |
|
||||
| SDXL | Any SDXL model (e.g. `sd_xl_base_1.0.safetensors`) |
|
||||
| Flux | Flux checkpoint |
|
||||
|
||||
Mismatched architectures will produce errors or garbage output.
|
||||
|
||||
### IP-Adapter: same face, different settings (reference image workflows)
|
||||
|
||||
IP-Adapter lets you upload a reference photo and generate new images that preserve the face/subject while changing everything else — age, emotion, setting, art style. This is different from LoRAs (which bake in a trained style). IP-Adapter works from a single image with no training needed.
|
||||
|
||||
#### What you can do
|
||||
|
||||
| Task | How | Example prompt |
|
||||
|------|-----|----------------|
|
||||
| Different setting | Reference face + new scene | "same person, sitting in a cafe in Paris" |
|
||||
| Age up/down | Reference face + age prompt | "same person as an elderly man" or "same person as a child" |
|
||||
| Emotions | Reference face + emotion | "same person, laughing" or "same person, crying" |
|
||||
| Art style | Reference face + style | "same person, oil painting, renaissance style" |
|
||||
| Pose change | Reference face + pose | "same person, looking over shoulder, dramatic lighting" |
|
||||
|
||||
#### Quick install
|
||||
|
||||
```bash
|
||||
./comfyui-install-ipadapter.sh # SDXL models (~2.5GB)
|
||||
./comfyui-install-ipadapter.sh --faceid # + FaceID for stronger face lock
|
||||
./comfyui-install-ipadapter.sh --all # SDXL + SD 1.5 models
|
||||
```
|
||||
|
||||
Then restart ComfyUI: `docker restart comfyui`
|
||||
|
||||
#### Basic workflow in ComfyUI
|
||||
|
||||
1. Open `http://<ip>:8188`
|
||||
2. Add nodes: right-click → Add Node → ipadapter
|
||||
3. Wire up:
|
||||
```
|
||||
[Load Checkpoint] → MODEL → [IPAdapter Apply] → [KSampler] → [VAE Decode] → [Save Image]
|
||||
→ CLIP → [CLIP Text Encode] → positive →
|
||||
[Load Image (your photo)] → [IPAdapter Unified Loader] → ipadapter →
|
||||
```
|
||||
4. **IPAdapter Unified Loader**: set preset to `PLUS FACE (portrait)`
|
||||
5. **IPAdapter Apply**: set weight 0.7–1.0 (higher = more faithful to reference)
|
||||
6. Text prompt: describe the new scene/emotion/age
|
||||
7. Click **Queue Prompt**
|
||||
|
||||
#### Adjusting face fidelity
|
||||
|
||||
| Weight | Effect |
|
||||
|--------|--------|
|
||||
| 0.5–0.6 | Loose reference — inspired by the face but not a match |
|
||||
| 0.7–0.8 | Good balance — recognizable face, creative freedom in scene |
|
||||
| 0.9–1.0 | Strong lock — very close to reference face |
|
||||
| FaceID preset | Strongest — uses face detection for identity lock |
|
||||
|
||||
#### Important: ComfyUI only (not from Open WebUI chat)
|
||||
|
||||
IP-Adapter workflows require uploading a reference image to ComfyUI. Open WebUI's image generation integration only sends text prompts — it can't attach reference images. For this use case, work directly in ComfyUI at `http://<ip>:8188`.
|
||||
|
||||
For text-only image generation from Open WebUI chat, use [LoRA workflows](#using-loras-with-open-webui-workflow-per-style) instead.
|
||||
|
||||
### Setup: AUTOMATIC1111 (alternative)
|
||||
|
||||
If you prefer AUTOMATIC1111 over ComfyUI:
|
||||
@@ -220,12 +463,313 @@ If you prefer AUTOMATIC1111 over ComfyUI:
|
||||
| `AUTOMATIC1111_BASE_URL` | — | AUTOMATIC1111 API URL |
|
||||
| `AUTOMATIC1111_API_AUTH` | — | Auth credentials (`user:pass`) |
|
||||
|
||||
### Recommended Open WebUI Functions
|
||||
### Installing Functions & Actions (without community signup)
|
||||
|
||||
Install these from **Admin → Functions → ＋ → Import From Link** or search in Discover:
|
||||
The Open WebUI community hub (`openwebui.com`) requires a free account to download functions. If you get parse errors pasting URLs or don't want to sign up, you can install functions manually by pasting their source code directly.
|
||||
|
||||
- **Auto Memory** — Automatically stores relevant info as persistent memories across chats
|
||||
- **Generate Image** — Adds a "Generate Image" action button to messages for quick re-generation
|
||||
#### Manual installation (no account needed)
|
||||
|
||||
1. Find the function's source on GitHub — most are in the [open-webui/functions](https://github.com/open-webui/functions) repo or linked from community pages
|
||||
2. Copy the **raw Python source code** (the entire `.py` file)
|
||||
3. In Open WebUI → **Workspace** → **Functions** → click **＋** (Create)
|
||||
4. Paste the code into the editor
|
||||
5. Give it a name and save
|
||||
6. Enable it under **Workspace** → **Functions** — toggle it on globally or per-model
|
||||
|
||||
For **Actions** (like Generate Image), the process is the same but go to **Workspace** → **Functions** and set the function type to **Action** in the metadata.
|
||||
|
||||
#### Recommended functions
|
||||
|
||||
| Function | Type | What it does |
|
||||
|----------|------|-------------|
|
||||
| **Auto Memory** | Filter | Automatically extracts and stores facts from conversations as persistent memories |
|
||||
| **Generate Image** | Action | Adds a "Generate Image" button to messages for quick image generation |
|
||||
|
||||
### Auto Memory setup
|
||||
|
||||
Auto Memory is a filter function that runs after each message exchange, uses an LLM call to extract noteworthy facts, and stores them as user memories in Open WebUI's built-in memory system.
|
||||
|
||||
#### Step 1: Install the function
|
||||
|
||||
Use the manual method above. The source code is at:
|
||||
`https://github.com/open-webui/functions` — search for `auto_memory` or `adaptive_memory`
|
||||
|
||||
#### Step 2: Configure the function settings
|
||||
|
||||
After installing, click the **gear icon** on the function to configure:
|
||||
|
||||
| Setting | Value for local stack | Notes |
|
||||
|---------|----------------------|-------|
|
||||
| `openai_api_url` | `http://ollama:11434/v1` | Ollama's OpenAI-compatible endpoint |
|
||||
| `model` | Your chat model (e.g. `qwen2.5:14b`) | Used for memory extraction LLM calls |
|
||||
| `api_key` | `ollama` | Any non-empty string — Ollama ignores auth |
|
||||
| `context_window_n` | `4` (default) | Number of related memories injected per message |
|
||||
| `similarity_score_filter` | Leave default | Cosine similarity threshold for memory retrieval |
|
||||
| `messages_to_consider` | Leave default | How many recent messages to analyze |
|
||||
| `allow_modify_user_memories` | `false` | **Keep off** — lets LLM delete/edit memories via API if enabled |
|
||||
|
||||
#### Step 3: Enable memories per user
|
||||
|
||||
Each user must enable the memory feature individually:
|
||||
1. Click your **profile icon** → **Settings** → **Personalization**
|
||||
2. Toggle **Memory** to **ON**
|
||||
|
||||
Without this, the function installs but does nothing.
|
||||
|
||||
### Generate Image action setup
|
||||
|
||||
1. Install via manual method above (search for `generate_image` in the functions repo)
|
||||
2. Configure the function settings:
|
||||
- Uses your existing Open WebUI image generation settings (ComfyUI/A1111)
|
||||
- No additional API configuration needed if image generation already works
|
||||
3. After installing, a **"Generate Image"** button appears on AI messages
|
||||
4. Clicking it sends the message content as an image generation prompt
|
||||
|
||||
### Memory system: how it works and context impact
|
||||
|
||||
Open WebUI has two separate systems for persistent knowledge. Understanding the difference is critical for managing your context window.
|
||||
|
||||
#### Built-in Memories vs Knowledge Collections
|
||||
|
||||
| | **Memories** | **Knowledge** |
|
||||
|---|---|---|
|
||||
| **What** | Short facts extracted from chats | Document collections (PDFs, text files) |
|
||||
| **How stored** | Text + vector embeddings in DB | Chunked documents + embeddings in ChromaDB |
|
||||
| **How injected** | Prepended to system prompt every message | Retrieved via RAG only when relevant chunks match |
|
||||
| **Scope** | **Global per user** — injected into ALL chats | **Per-chat** — you choose which collection with `#` |
|
||||
| **Context cost** | Always consumed, every message | Only consumed when you tag a collection |
|
||||
| **Control** | All or nothing (on/off globally) | Fine-grained (pick per conversation) |
|
||||
|
||||
#### Context window consumption by memories
|
||||
|
||||
Each memory is roughly 20–80 tokens (a sentence or two). With the default `context_window_n=4` setting in Auto Memory, 4 related memories are injected per message.
|
||||
|
||||
**Estimated context consumption per message:**
|
||||
|
||||
| Stored memories | Injected per msg | Tokens consumed | % of 8K context | % of 32K context |
|
||||
|----------------|-----------------|-----------------|-----------------|------------------|
|
||||
| 10 | ~4 | ~200 | 2.5% | 0.6% |
|
||||
| 50 | ~4 | ~200 | 2.5% | 0.6% |
|
||||
| 200 | ~4 | ~200 | 2.5% | 0.6% |
|
||||
|
||||
The key insight: **only the top N similar memories are injected** (default 4), not all of them. So having 200 memories doesn't consume more context than having 10 — the retrieval system picks the most relevant ones.
|
||||
|
||||
However, **the memories are injected on every single message** in every chat. This is the overhead cost.
|
||||
|
||||
#### Context consumed by chat history (the bigger problem)
|
||||
|
||||
The real context pressure comes from **conversation history**, not memories. Here's what actually fills your context window:
|
||||
|
||||
| Messages in chat | ~Tokens used | % of 8K | % of 32K |
|
||||
|-----------------|-------------|---------|----------|
|
||||
| 5 exchanges (10 msgs) | ~2,000–4,000 | 25–50% | 6–12% |
|
||||
| 20 exchanges (40 msgs) | ~8,000–16,000 | **100%+ (truncated)** | 25–50% |
|
||||
| 100 exchanges (200 msgs) | ~40,000–80,000 | **way over** | **100%+ (truncated)** |
|
||||
|
||||
With a small model running 8K context, you'll hit the limit after ~10-20 exchanges. The model starts dropping earlier messages. Memories add a small fixed overhead (~200 tokens) on top of this.
|
||||
|
||||
#### Across multiple separate chats
|
||||
|
||||
**Good news:** separate chats do NOT share context windows. Each chat starts fresh. The only cross-chat cost is the ~200 tokens of memories injected into each new chat's system prompt.
|
||||
|
||||
So 5, 20, or 100 separate chats don't accumulate — each one independently uses the context window. Memories are the only thing that carries over.
|
||||
|
||||
### Project-scoped memory (avoiding global memory pollution)
|
||||
|
||||
**Open WebUI does NOT have native project-scoped memory.** Memories are global per user — every fact extracted from any chat gets injected into every other chat.
|
||||
|
||||
This is a known limitation. Here are workarounds:
|
||||
|
||||
#### Option 1: Use Knowledge Collections instead of Memories (recommended)
|
||||
|
||||
Knowledge collections give you the scoping you want:
|
||||
|
||||
1. **Create a collection:** Workspace → Knowledge → Create Collection (e.g. "GPU Research Project")
|
||||
2. **Add documents:** Upload PDFs, text files, or paste notes into the collection
|
||||
3. **Use per-chat:** In any chat, type `#` and select your collection — only that chat gets the context
|
||||
4. **One-off chats stay clean:** Don't tag a collection, and no project context is injected
|
||||
|
||||
This is the closest thing to "projects" in Open WebUI. You can have separate knowledge collections for separate projects, and only pull them in when relevant.
|
||||
|
||||
#### Option 2: Disable Auto Memory, use manual memories
|
||||
|
||||
1. Turn off the Auto Memory function
|
||||
2. Manually add memories via **Profile → Settings → Personalization → Memories**
|
||||
3. Keep only universally useful facts (your name, preferences, etc.)
|
||||
4. Use Knowledge collections for project-specific context
|
||||
|
||||
#### Option 3: Periodically clear memories
|
||||
|
||||
**Profile → Settings → Personalization → Memories → Clear All** — nuclear option, but keeps things clean between projects.
|
||||
|
||||
### Working with Knowledge Collections (small context window survival guide)
|
||||
|
||||
With small local models (4B-9B, 4K-8K context), you'll hit the context window limit fast — often after 10-20 exchanges. Here's how to work effectively despite that.
|
||||
|
||||
#### How Knowledge Collections actually work
|
||||
|
||||
When you type `#` in chat and select a collection, Open WebUI does **RAG retrieval** — it finds the most relevant chunks from the collection, not the whole thing. This is efficient and doesn't blow up your context window.
|
||||
|
||||
- Default chunk size: ~500 tokens
|
||||
- Typically 3-5 relevant chunks are retrieved per query (~1500-2500 tokens)
|
||||
- The chunks are injected as context alongside your message
|
||||
|
||||
#### Creating Knowledge Collections from chat conversations
|
||||
|
||||
There is **no built-in "export chat to knowledge" button** — this is a requested but unimplemented feature. Here's the practical workflow:
|
||||
|
||||
**The handoff method (like the ChatGPT dark ages, but structured):**
|
||||
|
||||
1. When you're ~60-70% through your context window (you'll feel the model getting fuzzy), ask:
|
||||
```
|
||||
Summarize everything we've discussed and decided so far. Include:
|
||||
- Key decisions made
|
||||
- Current state of the work
|
||||
- What still needs to be done
|
||||
- Any important details or constraints
|
||||
Format as a structured document I can use to continue this conversation.
|
||||
```
|
||||
2. **Copy the summary output**
|
||||
3. Go to **Workspace → Knowledge → Create Collection** (e.g. "GPU Build - Session 1")
|
||||
4. Click **Add Content** → paste the summary as a text file (`.txt` or `.md`)
|
||||
5. Start a **new chat**, type `#` and select your collection, then continue where you left off
|
||||
|
||||
**Ongoing project workflow:**
|
||||
|
||||
```
|
||||
Chat 1: Research phase
|
||||
→ Ask for summary at end
|
||||
→ Save summary to "Project X" knowledge collection
|
||||
|
||||
Chat 2: Type # → select "Project X" → continue
|
||||
→ Model gets relevant context via RAG
|
||||
→ Ask for updated summary at end
|
||||
→ Add updated summary to collection (replace or append)
|
||||
|
||||
Chat 3: Type # → select "Project X" → continue
|
||||
→ Repeat...
|
||||
```
|
||||
|
||||
Each chat starts fresh with full context window available, but can pull in relevant history from previous sessions via RAG.
|
||||
|
||||
#### Useful community functions for context management
|
||||
|
||||
Install these manually (Workspace → Functions → Create, paste the code):
|
||||
|
||||
| Function | What it does |
|
||||
|----------|-------------|
|
||||
| **Checkpoint Summarization Filter** | Auto-summarizes conversation history when context gets long |
|
||||
| **Chat Context Clipper** | Keeps only the last N messages, preserves system prompt and first message |
|
||||
| **Context Length Filter** | Hard limits on turns (default 25) and tokens (default 10,000) |
|
||||
|
||||
The **Checkpoint Summarization Filter** is the closest to automatic handoff — it summarizes older messages so the model can keep going without losing track.
|
||||
|
||||
#### How Open WebUI handles context overflow
|
||||
|
||||
By default, Open WebUI **truncates** old messages (drops them silently) — it does NOT auto-summarize. The model just loses access to earlier conversation. This is why you suddenly feel like the model "forgot" what you were talking about.
|
||||
|
||||
You can control this with:
|
||||
- **Context Length Filter** function: set max turns and token limits explicitly
|
||||
- **Chat Context Clipper**: keeps latest N messages, always preserves system prompt + first message pair
|
||||
- Or do manual handoffs before you hit the limit
|
||||
|
||||
#### RAG tuning for better knowledge retrieval
|
||||
|
||||
If your knowledge collections aren't returning good results, tune these in **Admin → Settings → Documents**:
|
||||
|
||||
| Setting | Default | Recommendation |
|
||||
|---------|---------|---------------|
|
||||
| Chunk Size | 500 | 300 for factual docs, 800 for narrative/code |
|
||||
| Chunk Overlap | 100 | 50-100 (higher = better continuity, more tokens) |
|
||||
| Top K | 4 | 3-5 (more = more context consumed) |
|
||||
| Relevance Threshold | 0.0 | 0.3-0.5 (filters out low-quality matches) |
|
||||
|
||||
**Important:** Chunk size cannot exceed your embedding model's token limit. `nomic-embed-text` supports up to 8192 tokens, so you have plenty of headroom.
|
||||
|
||||
#### The honest comparison: Local AI vs Claude Code
|
||||
|
||||
| | **Local AI (Open WebUI + Ollama)** | **Claude Code** |
|
||||
|---|---|---|
|
||||
| **Context window** | 4K-32K (model dependent) | 200K |
|
||||
| **Cross-session memory** | Manual handoffs or Auto Memory | Automatic (CLAUDE.md, project memory) |
|
||||
| **Project scoping** | Knowledge collections (manual) | Built-in (each project has its own context) |
|
||||
| **Code awareness** | RAG server auto-retrieves relevant chunks from indexed repos | Reads your entire codebase directly |
|
||||
| **Continuation** | New chat + `#collection` handoff | "Let's finish X" just works |
|
||||
| **Code editing** | Can't edit files (chat only) | Reads, writes, runs code directly |
|
||||
| **Cost** | Free (your electricity) | API usage fees |
|
||||
| **Privacy** | 100% local | Cloud-based |
|
||||
| **Offline** | Works without internet | Requires internet |
|
||||
|
||||
Local AI requires more manual workflow management but has real code awareness via the RAG server. For code-heavy editing and multi-step tasks, Claude Code is dramatically better. For private code Q&A, document research, and learning — the local stack with RAG + knowledge collections is solid.
|
||||
|
||||
### Channels (beta feature)
|
||||
|
||||
Channels are persistent chat rooms (like Slack/Discord channels) with multi-model support. They do NOT scope memories differently — memories are still global per user. Channels are useful for team collaboration, not memory isolation.
|
||||
|
||||
### Realistic expectations by model size
|
||||
|
||||
Not all models can do all tasks. Here's what to actually expect:
|
||||
|
||||
| Task | 4B (qwen3.5:4b) | 9B (qwen3.5:9b) | 35B MoE (qwen3.5-35b-a3b) | 14B+ dense |
|
||||
|------|:---:|:---:|:---:|:---:|
|
||||
| Answer simple questions | OK | Good | Good | Good |
|
||||
| Explain existing code (with RAG) | OK | Good | Good | Good |
|
||||
| Fix a simple bug (typo, off-by-one) | Maybe | Usually | Usually | Yes |
|
||||
| Write a small utility function | Shaky | OK | Good | Good |
|
||||
| Fix logic error across 2-3 functions | No | Maybe | Usually | Usually |
|
||||
| Write a new feature (multiple files) | No | Shaky | Maybe | Maybe |
|
||||
| Refactor with style consistency | No | No | Sometimes | Sometimes |
|
||||
| Summarize a conversation for handoff | OK | Good | Good | Good |
|
||||
|
||||
**The 35B MoE model (`qwen3.5-35b-a3b`) is the sweet spot for small GPUs.** It was trained as a 35B model but only activates 3B parameters per token. This means it has the *knowledge* of a 35B model with the VRAM footprint closer to a 4B. On a 6GB card it may fit (VRAM usage varies with context length and KV cache settings).
|
||||
|
||||
**Bottom line for a 6GB GPU:**
|
||||
- Use `qwen3.5:4b` for quick chat, explanations, and summarization
|
||||
- Try `qwen3.5-35b-a3b` for code tasks — if it fits, it will be significantly better than 4B
|
||||
- Use Claude Code for anything that requires reading/writing multiple files or complex reasoning
|
||||
- The RAG server helps a lot — even a 4B model gives useful answers when it has the right code chunks in context
|
||||
|
||||
### Fixing models that output code instead of natural language
|
||||
|
||||
If your model (especially smaller ones like Qwen 3.5) responds with Python code blocks instead of plain English answers (as shown in the screenshot), this is a common behavior with code-optimized models.
|
||||
|
||||
#### Why it happens
|
||||
|
||||
- Small models like `qwen3.5:4b` are heavily optimized for coding tasks
|
||||
- They interpret ambiguous questions as "write me a program" instead of "answer my question"
|
||||
- The model's training data skews toward code generation at smaller parameter counts
|
||||
- Thinking/reasoning mode (if enabled) amplifies this tendency
|
||||
|
||||
#### Fix: Set a system prompt in Open WebUI
|
||||
|
||||
1. Go to **Admin** → **Settings** → **Interface** → **Default System Prompt** (for all chats)
|
||||
|
||||
Or per-model: **Workspace** → **Models** → select model → **System Prompt**
|
||||
|
||||
2. Use a system prompt like:
|
||||
|
||||
```
|
||||
You are a helpful assistant. Answer questions in clear, natural language.
|
||||
Explain concepts conversationally. Only include code if the user explicitly
|
||||
asks for code or a script. When discussing technical topics, use plain
|
||||
English explanations with examples, not programs.
|
||||
```
|
||||
|
||||
3. For the specific GPU pricing question in the screenshot, the model should have responded with a comparison table in text, not a Python script. A good system prompt prevents this.
|
||||
|
||||
#### Alternative: Use a chat-optimized model
|
||||
|
||||
Some models are better at conversational responses:
|
||||
|
||||
| Model | Size | Better for chat? |
|
||||
|-------|------|-----------------|
|
||||
| `qwen2.5:14b` | 14B | Yes — more balanced |
|
||||
| `qwen2.5:7b` | 7B | Yes — good general chat |
|
||||
| `llama3.1:8b` | 8B | Yes — conversational |
|
||||
| `qwen2.5-coder:7b` | 7B | No — code-focused |
|
||||
| `qwen3.5:4b` | 4B | No — too small, code-biased |
|
||||
|
||||
Avoid using `-coder` variants or very small models (≤4B) for general chat. Use them only when you actually want code.
|
||||
|
||||
### VRAM considerations
|
||||
|
||||
@@ -241,35 +785,127 @@ ComfyUI models typically need 4–8GB VRAM (SD 1.5: ~4GB, SDXL: ~7GB, Flux: ~12G
|
||||
|
||||
---
|
||||
|
||||
## Using LoRA Models in InvokeAI
|
||||
## InvokeAI vs ComfyUI — which to use when
|
||||
|
||||
LoRA (Low-Rank Adaptation) files let you customize image generation with fine-tuned styles or characters. If you trained a LoRA on RunPod or elsewhere, here's how to use it.
|
||||
Both are installed by the setup script. Here's when to use each:
|
||||
|
||||
### Import a LoRA file
|
||||
| Task | InvokeAI (`:9090`) | ComfyUI (`:8188`) |
|
||||
|------|:---:|:---:|
|
||||
| Friendly UI, sliders, drag-and-drop | Yes | No (node editor) |
|
||||
| Use a LoRA you trained on RunPod | Yes — just import + select | Yes — but wire LoRA Loader node |
|
||||
| Same face, different settings (img2img) | Yes — Image to Image tab | Yes — IP-Adapter nodes |
|
||||
| Age/emotion/scene changes | Yes — img2img + prompt | Yes — IP-Adapter + prompt |
|
||||
| Chat-integrated image gen (from Open WebUI) | No (no OWUI API) | Yes (workflow export) |
|
||||
| Maximum flexibility / custom pipelines | No | Yes |
|
||||
| Learning curve | Low | High |
|
||||
|
||||
**Bottom line:** Use InvokeAI for the interactive "play with images" workflow. Use ComfyUI only when you need Open WebUI chat integration or advanced node pipelines.
|
||||
|
||||
## Using InvokeAI for image iteration
|
||||
|
||||
InvokeAI is the easier path for what you want — take a reference image and generate variations with different settings, ages, emotions, art styles.
|
||||
|
||||
### Step 1: Install a base model (required first)
|
||||
|
||||
Before LoRAs or img2img will work, InvokeAI needs a base Stable Diffusion model.
|
||||
|
||||
```bash
|
||||
# Copy your LoRA into the InvokeAI Docker volume:
|
||||
./invokeai-import-lora.sh ~/Downloads/my-lora.safetensors
|
||||
# Option A: Download SD 1.5 (~4GB VRAM needed) via the container
|
||||
docker exec invokeai invokeai-model-install --add stabilityai/stable-diffusion-v1-5
|
||||
|
||||
# Optionally give it a display name:
|
||||
./invokeai-import-lora.sh ~/Downloads/my-lora.safetensors "My Custom Style"
|
||||
# Option B: Download SDXL (~7GB VRAM needed)
|
||||
docker exec invokeai invokeai-model-install --add stabilityai/stable-diffusion-xl-base-1.0
|
||||
```
|
||||
|
||||
### Use the LoRA in InvokeAI
|
||||
Or use the UI: open `http://<ip>:9090` → **Model Manager** (cube icon) → **Starter Models** tab → install SD 1.5 or SDXL.
|
||||
|
||||
1. Open InvokeAI at `http://<ip>:9090`
|
||||
2. Go to **Model Manager** (cube icon, left sidebar) and click **Scan for Models** / **Sync Models**
|
||||
3. Your LoRA should appear in the model list
|
||||
4. Switch to **Text to Image** tab
|
||||
5. In the left panel, find the **LoRA** section (below the model selector)
|
||||
6. Click **+** to add your LoRA, then adjust the **weight** slider (start at 0.7–0.85)
|
||||
> **6GB GPU note:** SD 1.5 fits comfortably. SDXL is tight — it may work with float16 (already set) but could be slow. Start with SD 1.5.
|
||||
|
||||
### Troubleshooting greyed-out upload buttons
|
||||
### Step 2: Import your RunPod LoRA
|
||||
|
||||
- **No base model installed:** You need a fully downloaded base model (e.g., SD 1.5) before InvokeAI enables LoRA uploads. Use Model Manager to install one first.
|
||||
- **Model not synced:** After copying files, click **Scan for Models** in Model Manager.
|
||||
- **Architecture mismatch:** A LoRA trained on SD 1.5 only works with SD 1.5 base models — not SDXL or SD 2.x.
|
||||
- **Use the import script instead:** The greyed-out UI upload can be bypassed entirely by using `invokeai-import-lora.sh` to copy files directly into the model volume.
|
||||
```bash
|
||||
# Copy the .safetensors file from wherever you downloaded it:
|
||||
./invokeai-import-lora.sh ~/Downloads/my-lora.safetensors "My Character"
|
||||
```
|
||||
|
||||
Then in the InvokeAI UI:
|
||||
1. **Model Manager** (cube icon) → **Scan for Models**
|
||||
2. Your LoRA should appear in the list
|
||||
|
||||
If it doesn't:
|
||||
- Click **Add Model** → **Scan Folder** → enter `/invokeai/models/lora`
|
||||
- Make sure the LoRA architecture matches your base model (SD 1.5 LoRA needs SD 1.5 base)
|
||||
|
||||
### Step 3: Generate with your LoRA (text-to-image)
|
||||
|
||||
1. Go to the **Text to Image** tab
|
||||
2. Select your **base model** (must match the LoRA's training base)
|
||||
3. In the left panel, expand the **LoRA** section (below model selector)
|
||||
4. Click **+** → select your LoRA → set weight to **0.7–0.85**
|
||||
5. Write a prompt: "portrait of [subject], smiling, studio lighting"
|
||||
6. Click **Invoke**
|
||||
|
||||
### Step 4: Iterate on an image (img2img)
|
||||
|
||||
This is where InvokeAI shines for your use case — take an image and riff on it:
|
||||
|
||||
1. Switch to the **Image to Image** tab
|
||||
2. Drag your reference photo onto the canvas (or click to upload)
|
||||
3. Keep your LoRA active (same as above)
|
||||
4. Set the **Denoising Strength** slider:
|
||||
|
||||
| Strength | Effect |
|
||||
|----------|--------|
|
||||
| 0.2–0.3 | Subtle tweaks — mostly keeps the original, minor style changes |
|
||||
| 0.4–0.5 | Moderate changes — recognizable but different mood/lighting |
|
||||
| 0.6–0.7 | Significant changes — same composition, new details/style |
|
||||
| 0.8–1.0 | Major rewrite — loosely inspired by original, mostly new |
|
||||
|
||||
5. Change the prompt to describe what you want different:
|
||||
- **Age up:** "same person, elderly, wrinkles, grey hair, wise expression"
|
||||
- **Age down:** "same person as a young child, bright eyes, playground"
|
||||
- **Emotion:** "same person, laughing joyfully" or "same person, crying, dramatic lighting"
|
||||
- **Setting:** "same person, sitting in a Parisian cafe, afternoon light"
|
||||
- **Art style:** "same person, oil painting, renaissance style, dramatic chiaroscuro"
|
||||
6. Click **Invoke** — iterate by adjusting strength and prompt
|
||||
|
||||
### Step 5: Use the Unified Canvas for inpainting
|
||||
|
||||
This is the "fix this specific thing" workflow — brush over a hand, arm, face, background, whatever, and regenerate just that area while keeping everything else untouched.
|
||||
|
||||
1. Switch to the **Unified Canvas** tab
|
||||
2. Upload or paste your image
|
||||
3. Select the **Mask** brush tool (not the paint brush)
|
||||
4. Brush over **only the area you want to change** — everything else stays locked
|
||||
5. Write a prompt describing what the masked area should become
|
||||
6. Set **Denoising Strength** to 0.6–0.8 (higher = more change)
|
||||
7. Click **Invoke** — only the masked pixels regenerate
|
||||
|
||||
**Common inpainting fixes:**
|
||||
|
||||
| Problem | Mask | Prompt |
|
||||
|---------|------|--------|
|
||||
| Hand in wrong position | Brush over the arm/hand | "natural hand resting at side, relaxed pose" |
|
||||
| Extra fingers | Brush over the hand | "normal human hand, five fingers, anatomically correct" |
|
||||
| Weird eyes | Brush over both eyes | "natural eyes, looking at camera, detailed iris" |
|
||||
| Bad background | Brush over background only | "clean studio backdrop" or "forest trail, golden hour" |
|
||||
| Wrong clothing | Brush over the clothing area | "wearing blue denim jacket, casual style" |
|
||||
| Face swap / aging | Brush over the face | "same person, elderly, wrinkles" or "same person as child" |
|
||||
|
||||
**Tips for better inpainting results:**
|
||||
- **Mask slightly larger** than the problem area — gives the model room to blend edges
|
||||
- **Use soft brush edges** (lower brush hardness) for more natural blending
|
||||
- If the result has visible seams, increase your mask area and try again
|
||||
- **Lower denoising (0.4–0.5)** for subtle fixes, **higher (0.7–0.9)** for major changes
|
||||
- Keep your LoRA active during inpainting — it maintains the trained style/face consistency
|
||||
|
||||
### Troubleshooting
|
||||
|
||||
- **Greyed-out upload/LoRA buttons:** Install a base model first (Step 1). InvokeAI disables most features until a checkpoint is loaded.
|
||||
- **Model not synced:** After copying files via script, click **Scan for Models** in Model Manager.
|
||||
- **Architecture mismatch:** A LoRA trained on SD 1.5 only works with SD 1.5 base models — not SDXL. Check what your RunPod training used.
|
||||
- **Out of VRAM:** Try SD 1.5 instead of SDXL, or reduce image size to 512x512.
|
||||
- **Use the import script:** The greyed-out UI upload can be bypassed entirely by using `invokeai-import-lora.sh` to copy files directly into the model volume.
|
||||
|
||||
---
|
||||
|
||||
@@ -295,3 +931,52 @@ Re-run the setup script — it detects an existing install and skips prereqs:
|
||||
| CPU | qwen2.5:7b | qwen2.5-coder:7b | 4k |
|
||||
|
||||
Embed model is always `nomic-embed-text` (required for RAG).
|
||||
|
||||
### VRAM reality check
|
||||
|
||||
Ollama will **always try to run** any model — it silently offloads layers to CPU when VRAM is insufficient. The model still works but gets significantly slower. The setup script's "fully in VRAM" label can be misleading.
|
||||
|
||||
**Actual VRAM needed for common models (Q4_K_M quantization):**
|
||||
|
||||
| Model | Download size | VRAM for inference | Fits in 6GB? | Fits in 8GB? |
|
||||
|-------|-------------|-------------------|-------------|-------------|
|
||||
| `qwen3.5:4b` | ~2.5 GB | ~3.5–4 GB | Yes | Yes |
|
||||
| `qwen2.5:7b` | ~4.4 GB | ~5.5–6 GB | Tight | Yes |
|
||||
| `qwen3.5:9b` | ~5.5 GB | ~6.5–7 GB | **No — partial CPU offload** | Tight |
|
||||
| `qwen2.5:14b` | ~8.7 GB | ~10–11 GB | No | No |
|
||||
| `qwen3.5-35b-a3b` (MoE) | ~20 GB | ~3.5 GB active | Yes (only 3B active) | Yes |
|
||||
|
||||
**Why the file size != VRAM needed:** Inference requires additional memory for KV cache, attention buffers, and CUDA overhead. Expect ~1-2 GB more than the model file size.
|
||||
|
||||
**Signs of CPU offload (model too big for your VRAM):**
|
||||
- Tokens per second drops from 20-40 to 2-8
|
||||
- `nvidia-smi` shows VRAM maxed out
|
||||
- CPU usage spikes during generation
|
||||
- First token takes much longer than usual
|
||||
|
||||
### Image Generation Model Tiers
|
||||
|
||||
The `setup-image-models.sh` script detects your GPU and offers appropriate models:
|
||||
|
||||
| VRAM | Available Models | Default | Notes |
|
||||
|------|-----------------|---------|-------|
|
||||
| ≥ 24GB | SD 1.5, SDXL, SDXL Turbo, Flux.1-schnell, Flux.1-dev | SDXL | All models, no constraints |
|
||||
| 12–23GB | SD 1.5, SDXL, SDXL Turbo, Flux.1-schnell | SDXL | Flux-dev too tight |
|
||||
| 8–11GB | SD 1.5, SDXL (tight), SDXL Turbo | SD 1.5 | SDXL works at 512px, may be slow |
|
||||
| 4–7GB | SD 1.5 (float16) | SD 1.5 | Only SD 1.5 fits |
|
||||
| < 4GB | none | — | CPU generation not recommended |
|
||||
|
||||
**GPU sharing:** Ollama and image generation share the GPU. Ollama auto-unloads models
|
||||
after its `KEEP_ALIVE` timeout (default 24h), so image gen gets full VRAM when the LLM
|
||||
is idle. For immediate unload: `docker exec ollama ollama stop <model-name>`
|
||||
|
||||
**Multi-GPU scaling:** With dual GPUs (e.g., 2× RTX 5000 = 32GB total), the VRAM
|
||||
is summed for tier selection. Both InvokeAI and ComfyUI will use all available GPUs.
|
||||
|
||||
```bash
|
||||
# Install image models (auto-detects GPU):
|
||||
./setup-image-models.sh
|
||||
|
||||
# Or auto-install the recommended default:
|
||||
./setup-image-models.sh --auto
|
||||
```
|
||||
|
||||
Executable
+97
@@ -0,0 +1,97 @@
|
||||
#!/usr/bin/env bash
|
||||
# Import a LoRA (.safetensors) into ComfyUI's Docker volume
|
||||
# Usage: ./comfyui-import-lora.sh /path/to/my-lora.safetensors
|
||||
set -euo pipefail
|
||||
|
||||
RED='\033[0;31m'; GREEN='\033[0;32m'; YELLOW='\033[1;33m'
|
||||
CYAN='\033[0;36m'; BOLD='\033[1m'; NC='\033[0m'
|
||||
|
||||
if [[ $# -lt 1 ]]; then
|
||||
echo -e "${BOLD}Usage:${NC} $0 <lora-file.safetensors> [display-name]"
|
||||
echo ""
|
||||
echo " Copies a LoRA file into ComfyUI's models/loras/ directory so it"
|
||||
echo " can be used in workflows (including via Open WebUI)."
|
||||
echo ""
|
||||
echo " Examples:"
|
||||
echo " $0 ~/Downloads/my-style-lora.safetensors"
|
||||
echo " $0 ~/Downloads/my-style-lora.safetensors \"Anime Style\""
|
||||
echo ""
|
||||
echo " After importing, create a workflow in ComfyUI that uses this LoRA,"
|
||||
echo " export it as API format, and import into Open WebUI for chat-based"
|
||||
echo " image generation with this LoRA applied automatically."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
LORA_FILE="$1"
|
||||
DISPLAY_NAME="${2:-$(basename "$LORA_FILE" .safetensors)}"
|
||||
|
||||
# Validate file exists
|
||||
if [[ ! -f "$LORA_FILE" ]]; then
|
||||
echo -e "${RED}Error:${NC} File not found: $LORA_FILE"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Validate file extension
|
||||
if [[ "$LORA_FILE" != *.safetensors && "$LORA_FILE" != *.ckpt && "$LORA_FILE" != *.pt ]]; then
|
||||
echo -e "${YELLOW}Warning:${NC} File doesn't have a typical LoRA extension (.safetensors, .ckpt, .pt)"
|
||||
read -rp "Continue anyway? [y/N] " yn
|
||||
[[ "$yn" != [yY]* ]] && exit 1
|
||||
fi
|
||||
|
||||
# Check if ComfyUI container exists
|
||||
if ! docker ps -a --format '{{.Names}}' 2>/dev/null | grep -q '^comfyui$'; then
|
||||
echo -e "${RED}Error:${NC} ComfyUI container not found. Is the AI stack running?"
|
||||
echo " Try: docker compose up -d comfyui"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Check if container is running
|
||||
if ! docker ps --format '{{.Names}}' 2>/dev/null | grep -q '^comfyui$'; then
|
||||
echo -e "${YELLOW}ComfyUI container is stopped. Starting it...${NC}"
|
||||
docker start comfyui
|
||||
sleep 3
|
||||
fi
|
||||
|
||||
FILENAME=$(basename "$LORA_FILE")
|
||||
|
||||
echo -e "${CYAN}[..]${NC} Copying ${BOLD}$FILENAME${NC} into ComfyUI..."
|
||||
|
||||
# Copy the LoRA file into the container's loras directory
|
||||
docker exec comfyui mkdir -p /opt/ComfyUI/models/loras
|
||||
docker cp "$LORA_FILE" "comfyui:/opt/ComfyUI/models/loras/$FILENAME"
|
||||
|
||||
echo -e "${GREEN}[OK]${NC} LoRA '${DISPLAY_NAME}' copied to ComfyUI!"
|
||||
echo ""
|
||||
echo -e "${BOLD}Next: Create a workflow that uses this LoRA${NC}"
|
||||
echo ""
|
||||
echo " 1. Open ComfyUI at http://localhost:8188"
|
||||
echo " 2. Load or create a text-to-image workflow"
|
||||
echo " 3. Add a ${BOLD}LoRA Loader${NC} node:"
|
||||
echo " Right-click canvas → Add Node → loaders → Load LoRA"
|
||||
echo " 4. Wire it between the ${BOLD}checkpoint loader${NC} and the ${BOLD}CLIP/sampler${NC}:"
|
||||
echo ""
|
||||
echo " [Load Checkpoint] → MODEL → [Load LoRA] → MODEL → [KSampler]"
|
||||
echo " → CLIP → → CLIP → [CLIP Text Encode]"
|
||||
echo ""
|
||||
echo " 5. In the LoRA Loader node, select ${BOLD}${FILENAME}${NC}"
|
||||
echo " 6. Set ${BOLD}strength_model${NC} and ${BOLD}strength_clip${NC} (start with 0.7–0.85)"
|
||||
echo ""
|
||||
echo -e "${BOLD}To use from Open WebUI chat (no more ComfyUI interaction needed):${NC}"
|
||||
echo ""
|
||||
echo " 7. Click the ${BOLD}gear icon${NC} → enable ${BOLD}Dev Mode${NC}"
|
||||
echo " 8. Click ${BOLD}Save (API Format)${NC} → saves workflow_api.json"
|
||||
echo " 9. In Open WebUI → Admin → Settings → Images → ${BOLD}Import Workflow${NC}"
|
||||
echo " 10. Upload the workflow_api.json and map the prompt node"
|
||||
echo " 11. Now just chat: \"Generate an image of a forest in ${DISPLAY_NAME} style\""
|
||||
echo ""
|
||||
echo -e "${YELLOW}Tip:${NC} The LoRA must match your base model architecture."
|
||||
echo " SD 1.5 LoRA → SD 1.5 checkpoint. SDXL LoRA → SDXL checkpoint."
|
||||
echo ""
|
||||
echo -e "${YELLOW}Tip:${NC} You can ${BOLD}chain multiple LoRAs${NC} in one workflow:"
|
||||
echo " [Checkpoint] → [Load LoRA 1] → [Load LoRA 2] → [KSampler]"
|
||||
echo " Each has its own strength slider so you can blend styles."
|
||||
echo ""
|
||||
echo -e "${YELLOW}Tip:${NC} Export multiple workflows (one per LoRA combo) and switch"
|
||||
echo " between them in Open WebUI's image settings as needed."
|
||||
echo " LoRAs are baked into the workflow — there's no keyword to"
|
||||
echo " toggle them on/off from chat."
|
||||
Executable
+206
@@ -0,0 +1,206 @@
|
||||
#!/usr/bin/env bash
|
||||
# Install IP-Adapter custom nodes + models into ComfyUI's Docker container
|
||||
# Enables: same face/different settings, age up/down, emotion changes, style transfer
|
||||
#
|
||||
# Usage:
|
||||
# ./comfyui-install-ipadapter.sh — install for SDXL (recommended)
|
||||
# ./comfyui-install-ipadapter.sh --sd15 — install for SD 1.5
|
||||
# ./comfyui-install-ipadapter.sh --all — install both SDXL + SD 1.5
|
||||
# ./comfyui-install-ipadapter.sh --faceid — also install FaceID (better face lock)
|
||||
set -euo pipefail
|
||||
|
||||
RED='\033[0;31m'; GREEN='\033[0;32m'; YELLOW='\033[1;33m'
|
||||
CYAN='\033[0;36m'; BOLD='\033[1m'; NC='\033[0m'
|
||||
info() { echo -e "${CYAN}[..]${NC} $*"; }
|
||||
ok() { echo -e "${GREEN}[OK]${NC} $*"; }
|
||||
warn() { echo -e "${YELLOW}[!!]${NC} $*"; }
|
||||
die() { echo -e "${RED}[XX]${NC} $*" >&2; exit 1; }
|
||||
|
||||
# ── parse args ──────────────────────────────────────────────────────────────
|
||||
INSTALL_SD15=false
|
||||
INSTALL_SDXL=true
|
||||
INSTALL_FACEID=false
|
||||
|
||||
for arg in "$@"; do
|
||||
case "$arg" in
|
||||
--sd15) INSTALL_SD15=true; INSTALL_SDXL=false ;;
|
||||
--all) INSTALL_SD15=true; INSTALL_SDXL=true ;;
|
||||
--faceid) INSTALL_FACEID=true ;;
|
||||
--help|-h)
|
||||
echo -e "${BOLD}Usage:${NC} $0 [--sd15] [--all] [--faceid]"
|
||||
echo ""
|
||||
echo " Installs IP-Adapter custom nodes and models into ComfyUI."
|
||||
echo " Enables reference-image workflows: same face in different"
|
||||
echo " settings, age changes, emotions, style transfer."
|
||||
echo ""
|
||||
echo " Options:"
|
||||
echo " --sd15 Install SD 1.5 models (instead of SDXL)"
|
||||
echo " --all Install both SDXL + SD 1.5 models"
|
||||
echo " --faceid Also install FaceID models (better face lock,"
|
||||
echo " requires insightface — adds ~1GB)"
|
||||
echo ""
|
||||
echo " Default: SDXL models only (~2.5GB download)"
|
||||
exit 0
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
# ── check container ─────────────────────────────────────────────────────────
|
||||
if ! docker ps --format '{{.Names}}' 2>/dev/null | grep -q '^comfyui$'; then
|
||||
if docker ps -a --format '{{.Names}}' 2>/dev/null | grep -q '^comfyui$'; then
|
||||
warn "ComfyUI container is stopped. Starting it..."
|
||||
docker start comfyui
|
||||
sleep 5
|
||||
else
|
||||
die "ComfyUI container not found. Is the AI stack running?"
|
||||
fi
|
||||
fi
|
||||
|
||||
# ── helper: download into container ─────────────────────────────────────────
|
||||
# Usage: dl_model <url> <container_path>
|
||||
dl_model() {
|
||||
local url="$1" dest="$2"
|
||||
local filename; filename=$(basename "$dest")
|
||||
if docker exec comfyui test -f "$dest" 2>/dev/null; then
|
||||
ok "Already exists: $filename"
|
||||
return 0
|
||||
fi
|
||||
local dir; dir=$(dirname "$dest")
|
||||
docker exec comfyui mkdir -p "$dir"
|
||||
info "Downloading $filename..."
|
||||
if ! docker exec comfyui wget -q --show-progress -O "$dest" "$url" 2>&1; then
|
||||
# wget --show-progress not always available
|
||||
docker exec comfyui wget -q -O "$dest" "$url"
|
||||
fi
|
||||
ok "Downloaded $filename"
|
||||
}
|
||||
|
||||
HF_IPA="https://huggingface.co/h94/IP-Adapter/resolve/main"
|
||||
MODELS="/opt/ComfyUI/models"
|
||||
|
||||
# ── Step 1: Install custom nodes ────────────────────────────────────────────
|
||||
echo ""
|
||||
echo -e "${BOLD}━━━ IP-Adapter Installation for ComfyUI ━━━${NC}"
|
||||
echo ""
|
||||
|
||||
info "Installing ComfyUI_IPAdapter_plus custom nodes..."
|
||||
if docker exec comfyui test -d /opt/ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus 2>/dev/null; then
|
||||
info "Custom nodes already installed — pulling updates..."
|
||||
docker exec comfyui bash -c "cd /opt/ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus && git pull -q"
|
||||
ok "Custom nodes updated"
|
||||
else
|
||||
docker exec comfyui bash -c "cd /opt/ComfyUI/custom_nodes && git clone --depth 1 https://github.com/cubiq/ComfyUI_IPAdapter_plus.git"
|
||||
ok "Custom nodes installed"
|
||||
fi
|
||||
|
||||
# Install Python dependencies if requirements.txt exists
|
||||
if docker exec comfyui test -f /opt/ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus/requirements.txt 2>/dev/null; then
|
||||
info "Installing Python dependencies..."
|
||||
docker exec comfyui pip install -q -r /opt/ComfyUI/custom_nodes/ComfyUI_IPAdapter_plus/requirements.txt 2>/dev/null || true
|
||||
fi
|
||||
|
||||
# ── Step 2: CLIP Vision models (required by all variants) ───────────────────
|
||||
echo ""
|
||||
info "Downloading CLIP Vision encoders..."
|
||||
|
||||
dl_model \
|
||||
"https://huggingface.co/h94/IP-Adapter/resolve/main/models/image_encoder/model.safetensors" \
|
||||
"$MODELS/clip_vision/CLIP-ViT-H-14-laion2B-s32B-b79K.safetensors"
|
||||
|
||||
if $INSTALL_SDXL; then
|
||||
dl_model \
|
||||
"https://huggingface.co/h94/IP-Adapter/resolve/main/sdxl_models/image_encoder/model.safetensors" \
|
||||
"$MODELS/clip_vision/CLIP-ViT-bigG-14-laion2B-39B-b160k.safetensors"
|
||||
fi
|
||||
|
||||
# ── Step 3: IP-Adapter models ──────────────────────────────────────────────
|
||||
echo ""
|
||||
docker exec comfyui mkdir -p "$MODELS/ipadapter"
|
||||
|
||||
if $INSTALL_SDXL; then
|
||||
info "Downloading SDXL IP-Adapter models..."
|
||||
dl_model "$HF_IPA/sdxl_models/ip-adapter-plus_sdxl_vit-h.safetensors" \
|
||||
"$MODELS/ipadapter/ip-adapter-plus_sdxl_vit-h.safetensors"
|
||||
dl_model "$HF_IPA/sdxl_models/ip-adapter-plus-face_sdxl_vit-h.safetensors" \
|
||||
"$MODELS/ipadapter/ip-adapter-plus-face_sdxl_vit-h.safetensors"
|
||||
fi
|
||||
|
||||
if $INSTALL_SD15; then
|
||||
info "Downloading SD 1.5 IP-Adapter models..."
|
||||
dl_model "$HF_IPA/models/ip-adapter-plus_sd15.safetensors" \
|
||||
"$MODELS/ipadapter/ip-adapter-plus_sd15.safetensors"
|
||||
dl_model "$HF_IPA/models/ip-adapter-plus-face_sd15.safetensors" \
|
||||
"$MODELS/ipadapter/ip-adapter-plus-face_sd15.safetensors"
|
||||
fi
|
||||
|
||||
# ── Step 4 (optional): FaceID models ───────────────────────────────────────
|
||||
if $INSTALL_FACEID; then
|
||||
echo ""
|
||||
info "Installing FaceID dependencies (insightface, onnxruntime)..."
|
||||
docker exec comfyui pip install -q insightface onnxruntime 2>/dev/null \
|
||||
|| warn "Could not install insightface — FaceID nodes may not work"
|
||||
|
||||
HF_FACEID="https://huggingface.co/h94/IP-Adapter-FaceID/resolve/main"
|
||||
info "Downloading FaceID models..."
|
||||
|
||||
if $INSTALL_SDXL; then
|
||||
dl_model "$HF_FACEID/ip-adapter-faceid-plusv2_sdxl.bin" \
|
||||
"$MODELS/ipadapter/ip-adapter-faceid-plusv2_sdxl.bin"
|
||||
fi
|
||||
if $INSTALL_SD15; then
|
||||
dl_model "$HF_FACEID/ip-adapter-faceid-plusv2_sd15.bin" \
|
||||
"$MODELS/ipadapter/ip-adapter-faceid-plusv2_sd15.bin"
|
||||
fi
|
||||
|
||||
# FaceID LoRAs (required for FaceID models)
|
||||
docker exec comfyui mkdir -p "$MODELS/loras"
|
||||
if $INSTALL_SDXL; then
|
||||
dl_model "$HF_FACEID/ip-adapter-faceid-plusv2_sdxl_lora.safetensors" \
|
||||
"$MODELS/loras/ip-adapter-faceid-plusv2_sdxl_lora.safetensors"
|
||||
fi
|
||||
if $INSTALL_SD15; then
|
||||
dl_model "$HF_FACEID/ip-adapter-faceid-plusv2_sd15_lora.safetensors" \
|
||||
"$MODELS/loras/ip-adapter-faceid-plusv2_sd15_lora.safetensors"
|
||||
fi
|
||||
fi
|
||||
|
||||
# ── Done ────────────────────────────────────────────────────────────────────
|
||||
echo ""
|
||||
echo -e "${GREEN}━━━ Installation complete! ━━━${NC}"
|
||||
echo ""
|
||||
echo -e "${BOLD}Restart ComfyUI to load new nodes:${NC}"
|
||||
echo " docker restart comfyui"
|
||||
echo ""
|
||||
echo -e "${BOLD}Then open ComfyUI at http://localhost:8188 and try it:${NC}"
|
||||
echo ""
|
||||
echo " Basic reference image (style/scene transfer):"
|
||||
echo " ┌──────────────────────────────────────────────────────────────────┐"
|
||||
echo " │ [Load Checkpoint]──→[Load Image]──→[IPAdapter Unified Loader] │"
|
||||
echo " │ │ │ │"
|
||||
echo " │ ├── MODEL ──────────────→ [IPAdapter Apply] → [KSampler]│"
|
||||
echo " │ └── CLIP → [CLIP Text Encode]──→ positive ──→ │"
|
||||
echo " │ \"a castle at sunset\" │"
|
||||
echo " └──────────────────────────────────────────────────────────────────┘"
|
||||
echo ""
|
||||
echo " 1. Add nodes: right-click → Add Node → ipadapter"
|
||||
echo " 2. ${BOLD}IPAdapter Unified Loader${NC}: set preset to '${BOLD}PLUS FACE (portrait)${NC}'"
|
||||
echo " 3. ${BOLD}Load Image${NC}: upload your reference photo"
|
||||
echo " 4. ${BOLD}IPAdapter Apply${NC}: connect model + image, set weight 0.7–1.0"
|
||||
echo " 5. Text prompt controls the new scene: \"elderly, sitting in cafe, smiling\""
|
||||
echo ""
|
||||
echo -e "${BOLD}What each preset does:${NC}"
|
||||
echo " PLUS — general style/scene transfer"
|
||||
echo " PLUS FACE — preserves face likeness (best for your use case)"
|
||||
if $INSTALL_FACEID; then
|
||||
echo " FACEID PLUSV2 — strongest face lock (uses insightface for detection)"
|
||||
fi
|
||||
echo ""
|
||||
echo -e "${BOLD}Example prompts with a reference face:${NC}"
|
||||
echo " • \"same person, elderly, wise expression, studio lighting\""
|
||||
echo " • \"same person as a child, playing in a park, happy\""
|
||||
echo " • \"same person, crying, dramatic lighting, black and white\""
|
||||
echo " • \"same person, oil painting style, renaissance setting\""
|
||||
echo ""
|
||||
echo -e "${YELLOW}Note:${NC} This works directly in ComfyUI. Open WebUI's ComfyUI integration"
|
||||
echo " only sends text prompts — it can't attach a reference image."
|
||||
echo " For chat-based image gen (without reference images), use LoRAs instead."
|
||||
+294
-112
@@ -80,6 +80,44 @@ else
|
||||
CTX=4096; OLLAMA_KV_CACHE="q4_0"; GPU_TIER="CPU only — 4B models (slow)"
|
||||
fi
|
||||
|
||||
# ── Image generation model tiers (VRAM-aware) ────────────────────────────────
|
||||
# Image gen shares GPU with Ollama — Ollama unloads after KEEP_ALIVE timeout,
|
||||
# so image gen gets full VRAM when Ollama is idle.
|
||||
if [[ "$TOTAL_VRAM" -ge 24 ]]; then
|
||||
IMG_MODELS="SD 1.5, SDXL, SDXL Turbo, Flux.1-dev, Flux.1-schnell"
|
||||
IMG_TIER="all models including Flux"
|
||||
IMG_DEFAULT="SDXL"
|
||||
elif [[ "$TOTAL_VRAM" -ge 12 ]]; then
|
||||
IMG_MODELS="SD 1.5, SDXL, SDXL Turbo, Flux.1-schnell (tight)"
|
||||
IMG_TIER="SDXL + Flux-schnell"
|
||||
IMG_DEFAULT="SDXL"
|
||||
elif [[ "$TOTAL_VRAM" -ge 8 ]]; then
|
||||
IMG_MODELS="SD 1.5, SDXL (tight at 512px), SDXL Turbo"
|
||||
IMG_TIER="SD 1.5 comfortable, SDXL possible"
|
||||
IMG_DEFAULT="SD 1.5"
|
||||
elif [[ "$TOTAL_VRAM" -ge 4 ]]; then
|
||||
IMG_MODELS="SD 1.5 (float16)"
|
||||
IMG_TIER="SD 1.5 only"
|
||||
IMG_DEFAULT="SD 1.5"
|
||||
else
|
||||
IMG_MODELS="none (CPU generation extremely slow)"
|
||||
IMG_TIER="CPU only — not recommended"
|
||||
IMG_DEFAULT=""
|
||||
fi
|
||||
|
||||
# ── InvokeAI precision (GPU-aware) ───────────────────────────────────────────
|
||||
# Ampere+ (compute 8.0+) supports bfloat16 natively for better precision.
|
||||
# All modern NVIDIA GPUs support float16. Fall back to auto if unsure.
|
||||
GPU_COMPUTE=$(nvidia-smi --query-gpu=compute_cap --format=csv,noheader 2>/dev/null \
|
||||
| head -1 | tr -d '.' || echo "0")
|
||||
if [[ "$GPU_COMPUTE" -ge 80 ]]; then
|
||||
INVOKEAI_PRECISION="bfloat16" # Ampere+ (RTX 30xx/40xx/50xx, A-series, H100)
|
||||
elif [[ "$GPU_COMPUTE" -ge 60 ]]; then
|
||||
INVOKEAI_PRECISION="float16" # Pascal+ (GTX 10xx, RTX 20xx, Tesla P40/V100)
|
||||
else
|
||||
INVOKEAI_PRECISION="auto" # Let InvokeAI decide
|
||||
fi
|
||||
|
||||
# ── new vs update ─────────────────────────────────────────────────────────────
|
||||
IS_UPDATE=false
|
||||
[[ -f "$BASE/docker-compose.yml" ]] && IS_UPDATE=true
|
||||
@@ -95,6 +133,7 @@ echo ""
|
||||
info "Machine : $(hostname)"
|
||||
info "LAN IP : $LOCAL_IP"
|
||||
info "GPU : ${GPU_NAME} (${VRAM_GB}GB VRAM)"
|
||||
info "Image : $IMG_TIER"
|
||||
$IS_UPDATE && warn "Existing install found. Config files kept unless --force is passed."
|
||||
|
||||
# ── Q1: Top-level — what to run ───────────────────────────────────────────────
|
||||
@@ -217,15 +256,21 @@ fi
|
||||
SSH_IMPORT_IDS=() # list of "gh:username" or "lp:username" entries
|
||||
|
||||
if ! $IS_UPDATE || [[ ! -f "$HOME/.ssh/authorized_keys" ]]; then
|
||||
echo ""
|
||||
echo -e " ${BOLD}[SSH] Import SSH public keys? (for passwordless SSH into this machine)${NC}"
|
||||
echo " Pulls your public keys from GitHub or Launchpad and adds them to"
|
||||
echo " ~/.ssh/authorized_keys using ssh-import-id."
|
||||
echo ""
|
||||
echo " Examples: gh:yourusername lp:yourlaunchpadid"
|
||||
echo " Multiple: gh:alice lp:alice"
|
||||
echo ""
|
||||
read -rp " Usernames (or Enter to skip): " SSH_INPUT
|
||||
if command -v whiptail &>/dev/null; then
|
||||
SSH_INPUT=$(whiptail --title "SSH Key Import" \
|
||||
--inputbox "Import SSH public keys for passwordless SSH into this machine.\n\nExamples: gh:yourusername lp:yourlaunchpadid\nMultiple: gh:alice lp:alice\n\nLeave blank to skip." \
|
||||
14 68 "" 3>&1 1>&2 2>&3) || SSH_INPUT=""
|
||||
else
|
||||
echo ""
|
||||
echo -e " ${BOLD}[SSH] Import SSH public keys? (for passwordless SSH into this machine)${NC}"
|
||||
echo " Pulls your public keys from GitHub or Launchpad and adds them to"
|
||||
echo " ~/.ssh/authorized_keys using ssh-import-id."
|
||||
echo ""
|
||||
echo " Examples: gh:yourusername lp:yourlaunchpadid"
|
||||
echo " Multiple: gh:alice lp:alice"
|
||||
echo ""
|
||||
read -rp " Usernames (or Enter to skip): " SSH_INPUT
|
||||
fi
|
||||
if [[ -n "$SSH_INPUT" ]]; then
|
||||
read -ra SSH_IMPORT_IDS <<< "$SSH_INPUT"
|
||||
fi
|
||||
@@ -236,10 +281,6 @@ OLLAMA_STORAGE="volume" # "volume" = Docker named volume, else a host path
|
||||
OLLAMA_HOST_PATH=""
|
||||
|
||||
if $INSTALL_AI; then
|
||||
echo ""
|
||||
echo -e " ${BOLD}[2/6] Where should Ollama models be stored?${NC}"
|
||||
echo " (Models are large — 5-50GB each. A fast SSD or large HDD is ideal.)"
|
||||
echo ""
|
||||
mapfile -t _MPTS < <(
|
||||
df -h --output=target,avail,fstype 2>/dev/null \
|
||||
| awk 'NR>1 && $2~/[0-9]/ {
|
||||
@@ -248,37 +289,58 @@ if $INSTALL_AI; then
|
||||
if ((unit=="G" && num>=20) || unit=="T") print $0
|
||||
}' | head -10
|
||||
)
|
||||
echo " 0) Docker volume (default — /var/lib/docker/volumes/)"
|
||||
for _i in "${!_MPTS[@]}"; do
|
||||
printf " %d) %s\n" "$((_i+1))" "${_MPTS[$_i]}"
|
||||
done
|
||||
echo ""
|
||||
read -rp " Choice [0]: " STORAGE_CHOICE
|
||||
STORAGE_CHOICE="${STORAGE_CHOICE:-0}"
|
||||
if [[ "$STORAGE_CHOICE" == "0" ]]; then
|
||||
OLLAMA_STORAGE="volume"
|
||||
elif [[ "$STORAGE_CHOICE" =~ ^[0-9]+$ ]] && (( STORAGE_CHOICE >= 1 && STORAGE_CHOICE <= ${#_MPTS[@]} )); then
|
||||
_MP=$(awk '{print $1}' <<< "${_MPTS[$(( STORAGE_CHOICE-1 ))]}")
|
||||
OLLAMA_HOST_PATH="$_MP/ollama-models"
|
||||
OLLAMA_STORAGE="bind"
|
||||
ok "Ollama models → $OLLAMA_HOST_PATH"
|
||||
|
||||
if command -v whiptail &>/dev/null; then
|
||||
WHIP_STORAGE=("volume" "Docker volume (default — /var/lib/docker/volumes/)" ON)
|
||||
for _i in "${!_MPTS[@]}"; do
|
||||
_mp_path=$(awk '{print $1}' <<< "${_MPTS[$_i]}")
|
||||
_mp_avail=$(awk '{print $2}' <<< "${_MPTS[$_i]}")
|
||||
WHIP_STORAGE+=("$_mp_path" "${_mp_path} (${_mp_avail} free)" OFF)
|
||||
done
|
||||
STORAGE_CHOICE=$(whiptail --title "Ollama Model Storage" \
|
||||
--radiolist "Models are large (5-50GB each). Choose a location.\nSPACE = select ENTER = confirm" \
|
||||
$((10 + ${#_MPTS[@]})) 72 $((1 + ${#_MPTS[@]})) \
|
||||
"${WHIP_STORAGE[@]}" 3>&1 1>&2 2>&3) || STORAGE_CHOICE="volume"
|
||||
STORAGE_CHOICE="${STORAGE_CHOICE//\"/}"
|
||||
if [[ "$STORAGE_CHOICE" == "volume" ]]; then
|
||||
OLLAMA_STORAGE="volume"
|
||||
else
|
||||
OLLAMA_HOST_PATH="${STORAGE_CHOICE}/ollama-models"
|
||||
OLLAMA_STORAGE="bind"
|
||||
ok "Ollama models → $OLLAMA_HOST_PATH"
|
||||
fi
|
||||
else
|
||||
# Fallback: treat input as a literal path
|
||||
OLLAMA_HOST_PATH="${STORAGE_CHOICE%/}"
|
||||
[[ -z "$OLLAMA_HOST_PATH" ]] && die "No path entered."
|
||||
OLLAMA_STORAGE="bind"
|
||||
echo ""
|
||||
echo -e " ${BOLD}[2/6] Where should Ollama models be stored?${NC}"
|
||||
echo " (Models are large — 5-50GB each. A fast SSD or large HDD is ideal.)"
|
||||
echo ""
|
||||
echo " 0) Docker volume (default — /var/lib/docker/volumes/)"
|
||||
for _i in "${!_MPTS[@]}"; do
|
||||
printf " %d) %s\n" "$((_i+1))" "${_MPTS[$_i]}"
|
||||
done
|
||||
echo ""
|
||||
read -rp " Choice [0]: " STORAGE_CHOICE
|
||||
STORAGE_CHOICE="${STORAGE_CHOICE:-0}"
|
||||
if [[ "$STORAGE_CHOICE" == "0" ]]; then
|
||||
OLLAMA_STORAGE="volume"
|
||||
elif [[ "$STORAGE_CHOICE" =~ ^[0-9]+$ ]] && (( STORAGE_CHOICE >= 1 && STORAGE_CHOICE <= ${#_MPTS[@]} )); then
|
||||
_MP=$(awk '{print $1}' <<< "${_MPTS[$(( STORAGE_CHOICE-1 ))]}")
|
||||
OLLAMA_HOST_PATH="$_MP/ollama-models"
|
||||
OLLAMA_STORAGE="bind"
|
||||
ok "Ollama models → $OLLAMA_HOST_PATH"
|
||||
else
|
||||
OLLAMA_HOST_PATH="${STORAGE_CHOICE%/}"
|
||||
[[ -z "$OLLAMA_HOST_PATH" ]] && die "No path entered."
|
||||
OLLAMA_STORAGE="bind"
|
||||
fi
|
||||
fi
|
||||
unset _MPTS _MP _i
|
||||
unset _MPTS _MP _i WHIP_STORAGE
|
||||
fi
|
||||
|
||||
# ── Q3: Storage for Kiwix ZIMs ────────────────────────────────────────────────
|
||||
KIWIX_DIR="$BASE/kiwix" # default
|
||||
|
||||
if $SVC_KIWIX; then
|
||||
echo ""
|
||||
echo -e " ${BOLD}[3/6] Where should Kiwix ZIM files be stored?${NC}"
|
||||
echo " (ZIMs are large — Wikipedia alone is ~46GB. Total collection ~130GB.)"
|
||||
echo ""
|
||||
mapfile -t _MPTS < <(
|
||||
df -h --output=target,avail,fstype 2>/dev/null \
|
||||
| awk 'NR>1 && $2~/[0-9]/ {
|
||||
@@ -287,22 +349,44 @@ if $SVC_KIWIX; then
|
||||
if ((unit=="G" && num>=50) || unit=="T") print $0
|
||||
}' | head -10
|
||||
)
|
||||
echo " 0) Default: $KIWIX_DIR"
|
||||
for _i in "${!_MPTS[@]}"; do
|
||||
printf " %d) %s\n" "$((_i+1))" "${_MPTS[$_i]}"
|
||||
done
|
||||
echo ""
|
||||
read -rp " Choice [0]: " KIWIX_CHOICE
|
||||
KIWIX_CHOICE="${KIWIX_CHOICE:-0}"
|
||||
if [[ "$KIWIX_CHOICE" != "0" ]] && [[ "$KIWIX_CHOICE" =~ ^[0-9]+$ ]] && (( KIWIX_CHOICE >= 1 && KIWIX_CHOICE <= ${#_MPTS[@]} )); then
|
||||
_MP=$(awk '{print $1}' <<< "${_MPTS[$(( KIWIX_CHOICE-1 ))]}")
|
||||
KIWIX_DIR="$_MP/kiwix"
|
||||
ok "Kiwix ZIMs → $KIWIX_DIR"
|
||||
elif [[ "$KIWIX_CHOICE" != "0" ]] && [[ -n "$KIWIX_CHOICE" ]]; then
|
||||
# Fallback: treat as a literal path
|
||||
KIWIX_DIR="${KIWIX_CHOICE%/}"
|
||||
|
||||
if command -v whiptail &>/dev/null; then
|
||||
WHIP_KIWIX=("default" "Default: $KIWIX_DIR" ON)
|
||||
for _i in "${!_MPTS[@]}"; do
|
||||
_mp_path=$(awk '{print $1}' <<< "${_MPTS[$_i]}")
|
||||
_mp_avail=$(awk '{print $2}' <<< "${_MPTS[$_i]}")
|
||||
WHIP_KIWIX+=("$_mp_path" "${_mp_path} (${_mp_avail} free)" OFF)
|
||||
done
|
||||
_KIWIX_SEL=$(whiptail --title "Kiwix ZIM Storage" \
|
||||
--radiolist "ZIMs are large — Wikipedia alone ~46GB, total ~130GB.\nSPACE = select ENTER = confirm" \
|
||||
$((10 + ${#_MPTS[@]})) 72 $((1 + ${#_MPTS[@]})) \
|
||||
"${WHIP_KIWIX[@]}" 3>&1 1>&2 2>&3) || _KIWIX_SEL="default"
|
||||
_KIWIX_SEL="${_KIWIX_SEL//\"/}"
|
||||
if [[ "$_KIWIX_SEL" != "default" ]]; then
|
||||
KIWIX_DIR="${_KIWIX_SEL}/kiwix"
|
||||
ok "Kiwix ZIMs → $KIWIX_DIR"
|
||||
fi
|
||||
else
|
||||
echo ""
|
||||
echo -e " ${BOLD}[3/6] Where should Kiwix ZIM files be stored?${NC}"
|
||||
echo " (ZIMs are large — Wikipedia alone is ~46GB. Total collection ~130GB.)"
|
||||
echo ""
|
||||
echo " 0) Default: $KIWIX_DIR"
|
||||
for _i in "${!_MPTS[@]}"; do
|
||||
printf " %d) %s\n" "$((_i+1))" "${_MPTS[$_i]}"
|
||||
done
|
||||
echo ""
|
||||
read -rp " Choice [0]: " KIWIX_CHOICE
|
||||
KIWIX_CHOICE="${KIWIX_CHOICE:-0}"
|
||||
if [[ "$KIWIX_CHOICE" != "0" ]] && [[ "$KIWIX_CHOICE" =~ ^[0-9]+$ ]] && (( KIWIX_CHOICE >= 1 && KIWIX_CHOICE <= ${#_MPTS[@]} )); then
|
||||
_MP=$(awk '{print $1}' <<< "${_MPTS[$(( KIWIX_CHOICE-1 ))]}")
|
||||
KIWIX_DIR="$_MP/kiwix"
|
||||
ok "Kiwix ZIMs → $KIWIX_DIR"
|
||||
elif [[ "$KIWIX_CHOICE" != "0" ]] && [[ -n "$KIWIX_CHOICE" ]]; then
|
||||
KIWIX_DIR="${KIWIX_CHOICE%/}"
|
||||
fi
|
||||
fi
|
||||
unset _MPTS _MP _i _KIWIX_CHOICE
|
||||
unset _MPTS _MP _i _KIWIX_CHOICE WHIP_KIWIX
|
||||
|
||||
# ── Q4: Download ZIMs now? ─────────────────────────────────────────────────
|
||||
# ON if any matching ZIM file already exists in KIWIX_DIR
|
||||
@@ -447,11 +531,16 @@ fi
|
||||
# ── Q4b: Firewall (LAN subnet) ────────────────────────────────────────────────
|
||||
LAN_SUBNET="192.168.1.0/24"
|
||||
if command -v ufw &>/dev/null && { [[ ! -f "$BASE/.ufw-done" ]] || $FORCE; }; then
|
||||
echo ""
|
||||
# Auto-detect likely subnet from current IP
|
||||
AUTO_SUBNET=$(echo "$LOCAL_IP" | awk -F. '{print $1"."$2"."$3".0/24"}')
|
||||
echo -e " ${BOLD}[4b/6] Firewall — allow LAN access to services${NC}"
|
||||
read -rp " LAN subnet [${AUTO_SUBNET}]: " LAN_INPUT
|
||||
if command -v whiptail &>/dev/null; then
|
||||
LAN_INPUT=$(whiptail --title "Firewall — LAN Access" \
|
||||
--inputbox "Allow LAN access to all services.\n\nYour detected subnet:" \
|
||||
10 60 "$AUTO_SUBNET" 3>&1 1>&2 2>&3) || LAN_INPUT=""
|
||||
else
|
||||
echo ""
|
||||
echo -e " ${BOLD}[4b/6] Firewall — allow LAN access to services${NC}"
|
||||
read -rp " LAN subnet [${AUTO_SUBNET}]: " LAN_INPUT
|
||||
fi
|
||||
LAN_SUBNET="${LAN_INPUT:-$AUTO_SUBNET}"
|
||||
[[ "$LAN_SUBNET" =~ /[0-9]+$ ]] || LAN_SUBNET="${LAN_SUBNET}/24"
|
||||
fi
|
||||
@@ -462,35 +551,48 @@ REASON_MODEL=""
|
||||
|
||||
if $INSTALL_AI; then
|
||||
|
||||
# speed estimate based on Q4 model size vs available VRAM
|
||||
# no hard limits — just honest labels so the user can choose
|
||||
# speed estimate: model_file_size + ~1.5GB overhead for KV cache/CUDA vs VRAM
|
||||
# Ollama has no hard limits — silently offloads to CPU when over VRAM
|
||||
speed_label() {
|
||||
local mgb="$1" # approximate Q4 size in GB
|
||||
local mgb="$1" # approximate Q4 file size in GB
|
||||
local needed=$(( mgb + 2 )) # +2GB for KV cache, attention, CUDA overhead
|
||||
if [[ "$VRAM_GB" -eq 0 ]]; then
|
||||
printf "CPU only — very slow"
|
||||
elif (( mgb <= VRAM_GB )); then
|
||||
printf "✓ fast — fully in VRAM"
|
||||
elif (( mgb <= VRAM_GB + 2 )); then
|
||||
printf "~ good — fits with small overhang (~reading speed)"
|
||||
elif (( mgb <= VRAM_GB + 8 )); then
|
||||
printf "✗ slow — partial CPU offload"
|
||||
elif (( needed <= VRAM_GB )); then
|
||||
printf "✓ fast — fits in VRAM (~%dGB needed)" "$needed"
|
||||
elif (( needed <= VRAM_GB + 2 )); then
|
||||
printf "~ tight — ~%dGB needed, may spill to CPU" "$needed"
|
||||
elif (( needed <= VRAM_GB + 8 )); then
|
||||
printf "✗ slow — ~%dGB needed, partial CPU offload" "$needed"
|
||||
else
|
||||
printf "✗ very slow — heavy CPU offload"
|
||||
printf "✗ very slow — ~%dGB needed, heavy CPU offload" "$needed"
|
||||
fi
|
||||
}
|
||||
|
||||
echo ""
|
||||
echo -e " ${BOLD}[5/6] Model selection${NC}"
|
||||
echo " GPU: ${GPU_NAME:-None} (${VRAM_GB}GB VRAM)"
|
||||
echo ""
|
||||
echo " Origin preference:"
|
||||
echo " 1) Western-only — Codestral (Mistral 🇫🇷) · Phi4 (Microsoft 🇺🇸) · Mistral 7B"
|
||||
echo " 2) Performance-first — Qwen 3.5 (Feb 2026, top benchmarks, vision+code)"
|
||||
echo " 3) Mixed — Western for chat, Qwen 3.5 for coding"
|
||||
echo " 4) Custom — enter model names manually"
|
||||
echo ""
|
||||
read -rp " Choice [2]: " MODEL_PREF
|
||||
MODEL_PREF="${MODEL_PREF:-2}"
|
||||
if command -v whiptail &>/dev/null; then
|
||||
MODEL_PREF=$(whiptail --title "Model Selection — ${GPU_NAME:-No GPU} (${VRAM_GB}GB VRAM)" \
|
||||
--radiolist "Choose model origin preference.\nSPACE = select ENTER = confirm" \
|
||||
14 78 4 \
|
||||
"2" "Performance-first — Qwen 3.5 (top benchmarks, vision+code)" ON \
|
||||
"1" "Western-only — Codestral · Phi4 · Mistral 7B" OFF \
|
||||
"3" "Mixed — Western chat + Qwen 3.5 coding" OFF \
|
||||
"4" "Custom — enter model names manually" OFF \
|
||||
3>&1 1>&2 2>&3) || MODEL_PREF="2"
|
||||
MODEL_PREF="${MODEL_PREF//\"/}"
|
||||
else
|
||||
echo ""
|
||||
echo -e " ${BOLD}[5/6] Model selection${NC}"
|
||||
echo " GPU: ${GPU_NAME:-None} (${VRAM_GB}GB VRAM)"
|
||||
echo ""
|
||||
echo " Origin preference:"
|
||||
echo " 1) Western-only — Codestral (Mistral) · Phi4 (Microsoft) · Mistral 7B"
|
||||
echo " 2) Performance-first — Qwen 3.5 (Feb 2026, top benchmarks, vision+code)"
|
||||
echo " 3) Mixed — Western for chat, Qwen 3.5 for coding"
|
||||
echo " 4) Custom — enter model names manually"
|
||||
echo ""
|
||||
read -rp " Choice [2]: " MODEL_PREF
|
||||
MODEL_PREF="${MODEL_PREF:-2}"
|
||||
fi
|
||||
|
||||
# Q4_K_M approximate weights in GB: 4B=2.5, 7B=4, 9B=5.5, 14B=9, 22B=13, 27B=17, 35B-MoE=12, 70B=41
|
||||
|
||||
@@ -554,21 +656,59 @@ if $INSTALL_AI; then
|
||||
|
||||
if [[ "$MODEL_PREF" == "4" ]]; then
|
||||
# Custom — free-form entry
|
||||
echo " Current defaults: fast=$FAST_MODEL chat=$CHAT_MODEL code=$CODE_MODEL"
|
||||
echo " Press Enter on any line to keep the default shown."
|
||||
echo ""
|
||||
read -rp " Fast/chat model [$FAST_MODEL]: " _in; FAST_MODEL="${_in:-$FAST_MODEL}"
|
||||
read -rp " Smart chat model [$CHAT_MODEL]: " _in; CHAT_MODEL="${_in:-$CHAT_MODEL}"
|
||||
read -rp " Code model [$CODE_MODEL]: " _in; CODE_MODEL="${_in:-$CODE_MODEL}"
|
||||
read -rp " Reasoning model (Enter to skip): " REASON_MODEL
|
||||
else
|
||||
read -rp " Choose tier [$REC_NUM]: " _TIER_INPUT
|
||||
_TIER_INPUT="${_TIER_INPUT:-$REC_NUM}"
|
||||
# Accept either a number (1-4) or the tier name directly
|
||||
if [[ "$_TIER_INPUT" =~ ^[1-4]$ ]]; then
|
||||
TIER_PICK="${_TIER_NAMES[$((_TIER_INPUT-1))]}"
|
||||
if command -v whiptail &>/dev/null; then
|
||||
_in=$(whiptail --title "Custom Models — Fast/Chat" \
|
||||
--inputbox "Fast/chat model (small, quick responses):" \
|
||||
9 60 "$FAST_MODEL" 3>&1 1>&2 2>&3) && FAST_MODEL="${_in:-$FAST_MODEL}"
|
||||
_in=$(whiptail --title "Custom Models — Smart Chat" \
|
||||
--inputbox "Smart chat model (main model for complex tasks):" \
|
||||
9 60 "$CHAT_MODEL" 3>&1 1>&2 2>&3) && CHAT_MODEL="${_in:-$CHAT_MODEL}"
|
||||
_in=$(whiptail --title "Custom Models — Code" \
|
||||
--inputbox "Code model:" \
|
||||
9 60 "$CODE_MODEL" 3>&1 1>&2 2>&3) && CODE_MODEL="${_in:-$CODE_MODEL}"
|
||||
REASON_MODEL=$(whiptail --title "Custom Models — Reasoning" \
|
||||
--inputbox "Reasoning model (leave blank to skip):" \
|
||||
9 60 "" 3>&1 1>&2 2>&3) || REASON_MODEL=""
|
||||
else
|
||||
TIER_PICK="$_TIER_INPUT"
|
||||
echo " Current defaults: fast=$FAST_MODEL chat=$CHAT_MODEL code=$CODE_MODEL"
|
||||
echo " Press Enter on any line to keep the default shown."
|
||||
echo ""
|
||||
read -rp " Fast/chat model [$FAST_MODEL]: " _in; FAST_MODEL="${_in:-$FAST_MODEL}"
|
||||
read -rp " Smart chat model [$CHAT_MODEL]: " _in; CHAT_MODEL="${_in:-$CHAT_MODEL}"
|
||||
read -rp " Code model [$CODE_MODEL]: " _in; CODE_MODEL="${_in:-$CODE_MODEL}"
|
||||
read -rp " Reasoning model (Enter to skip): " REASON_MODEL
|
||||
fi
|
||||
else
|
||||
if command -v whiptail &>/dev/null; then
|
||||
# Build whiptail radiolist with recommended tier pre-selected
|
||||
WHIP_TIERS=()
|
||||
declare -A _TIER_LABELS
|
||||
case "$MODEL_PREF" in
|
||||
1) _TIER_LABELS=([7B]="mistral:7b + codellama:7b" [14B]="phi4:14b + starcoder2:15b" [22B]="phi4:14b + codestral:22b" [70B]="llama3.3:70b + codestral:22b")
|
||||
_TIER_SPEEDS=([7B]="$(speed_label 4)" [14B]="$(speed_label 9)" [22B]="$(speed_label 13)" [70B]="$(speed_label 41)") ;;
|
||||
2) _TIER_LABELS=([4B]="qwen3.5:4b (chat+code)" [9B]="qwen3.5:9b (chat+code)" [35B]="qwen3.5-35b-a3b (MoE, 3B active)" [27B]="qwen3.5:27b (dense)")
|
||||
_TIER_SPEEDS=([4B]="$(speed_label 2)" [9B]="$(speed_label 5)" [35B]="$(speed_label 12)" [27B]="$(speed_label 17)") ;;
|
||||
3) _TIER_LABELS=([7B]="mistral:7b + qwen3.5:4b" [14B]="phi4:14b + qwen3.5:9b" [35B]="phi4:14b + qwen3.5-35b-a3b" [70B]="llama3.3:70b + qwen3.5-35b-a3b")
|
||||
_TIER_SPEEDS=([7B]="$(speed_label 4)" [14B]="$(speed_label 9)" [35B]="$(speed_label 19)" [70B]="$(speed_label 41)") ;;
|
||||
esac
|
||||
for _tn in "${_TIER_NAMES[@]}"; do
|
||||
_onoff="OFF"; [[ "$_tn" == "$REC_TIER" ]] && _onoff="ON"
|
||||
WHIP_TIERS+=("$_tn" "${_TIER_LABELS[$_tn]} | ${_TIER_SPEEDS[$_tn]}" "$_onoff")
|
||||
done
|
||||
TIER_PICK=$(whiptail --title "Model Size — ${VRAM_GB}GB GPU" \
|
||||
--radiolist "Recommended tier pre-selected based on your GPU.\nSPACE = select ENTER = confirm" \
|
||||
14 90 4 \
|
||||
"${WHIP_TIERS[@]}" 3>&1 1>&2 2>&3) || TIER_PICK="$REC_TIER"
|
||||
TIER_PICK="${TIER_PICK//\"/}"
|
||||
unset WHIP_TIERS _TIER_LABELS _TIER_SPEEDS
|
||||
else
|
||||
read -rp " Choose tier [$REC_NUM]: " _TIER_INPUT
|
||||
_TIER_INPUT="${_TIER_INPUT:-$REC_NUM}"
|
||||
if [[ "$_TIER_INPUT" =~ ^[1-4]$ ]]; then
|
||||
TIER_PICK="${_TIER_NAMES[$((_TIER_INPUT-1))]}"
|
||||
else
|
||||
TIER_PICK="$_TIER_INPUT"
|
||||
fi
|
||||
fi
|
||||
unset _TIER_NAMES _TIER_INPUT REC_NUM
|
||||
|
||||
@@ -594,16 +734,29 @@ if $INSTALL_AI; then
|
||||
esac
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo " Models selected:"
|
||||
printf " %-16s %s\n" "Fast chat:" "$FAST_MODEL"
|
||||
printf " %-16s %s\n" "Smart chat:" "$CHAT_MODEL"
|
||||
printf " %-16s %s\n" "Code:" "$CODE_MODEL"
|
||||
[[ -n "$REASON_MODEL" ]] && printf " %-16s %s\n" "Reasoning:" "$REASON_MODEL"
|
||||
printf " %-16s %s\n" "Embed (RAG):" "$EMBED_MODEL"
|
||||
echo ""
|
||||
read -rp " Download these models now? [Y/n]: " DO_PULL
|
||||
[[ "${DO_PULL,,}" != "n" ]] && PULL_MODELS=true
|
||||
# Show selected models and ask about download
|
||||
_MODEL_SUMMARY="Fast chat: $FAST_MODEL\nSmart chat: $CHAT_MODEL\nCode: $CODE_MODEL"
|
||||
[[ -n "$REASON_MODEL" ]] && _MODEL_SUMMARY+="\nReasoning: $REASON_MODEL"
|
||||
_MODEL_SUMMARY+="\nEmbed (RAG): $EMBED_MODEL"
|
||||
|
||||
if command -v whiptail &>/dev/null; then
|
||||
if whiptail --title "Download Models Now?" \
|
||||
--yesno "Models selected:\n\n$_MODEL_SUMMARY\n\nDownload these models now? (can take 10-40 min)" \
|
||||
14 60 3>&1 1>&2 2>&3; then
|
||||
PULL_MODELS=true
|
||||
fi
|
||||
else
|
||||
echo ""
|
||||
echo " Models selected:"
|
||||
printf " %-16s %s\n" "Fast chat:" "$FAST_MODEL"
|
||||
printf " %-16s %s\n" "Smart chat:" "$CHAT_MODEL"
|
||||
printf " %-16s %s\n" "Code:" "$CODE_MODEL"
|
||||
[[ -n "$REASON_MODEL" ]] && printf " %-16s %s\n" "Reasoning:" "$REASON_MODEL"
|
||||
printf " %-16s %s\n" "Embed (RAG):" "$EMBED_MODEL"
|
||||
echo ""
|
||||
read -rp " Download these models now? [Y/n]: " DO_PULL
|
||||
[[ "${DO_PULL,,}" != "n" ]] && PULL_MODELS=true
|
||||
fi
|
||||
fi
|
||||
|
||||
|
||||
@@ -642,8 +795,14 @@ if [[ "$ZIM_CHOICE" == "2" ]]; then
|
||||
echo " ✓ Download ${_nzim} ZIM(s): $ZIM_PICKS"
|
||||
fi
|
||||
echo ""
|
||||
read -rp " Proceed? [Y/n]: " CONFIRM
|
||||
[[ "${CONFIRM,,}" == "n" ]] && echo "Aborted." && exit 0
|
||||
if command -v whiptail &>/dev/null; then
|
||||
whiptail --title "Ready to Install" \
|
||||
--yesno "Everything above will be installed and configured.\n\nProceed?" \
|
||||
9 50 3>&1 1>&2 2>&3 || { echo "Aborted."; exit 0; }
|
||||
else
|
||||
read -rp " Proceed? [Y/n]: " CONFIRM
|
||||
[[ "${CONFIRM,,}" == "n" ]] && echo "Aborted." && exit 0
|
||||
fi
|
||||
echo ""
|
||||
|
||||
# ── helper: write only if missing (or --force) ────────────────────────────────
|
||||
@@ -1033,7 +1192,7 @@ IMGENV
|
||||
environment:
|
||||
- INVOKEAI_HOST=0.0.0.0
|
||||
- INVOKEAI_PORT=9090
|
||||
- INVOKEAI_PRECISION=float16
|
||||
- INVOKEAI_PRECISION=$INVOKEAI_PRECISION
|
||||
deploy:
|
||||
resources:
|
||||
reservations:
|
||||
@@ -1324,6 +1483,21 @@ elif $INSTALL_AI; then
|
||||
info "Skipping model pull — run later: bash $BASE/pull-models.sh"
|
||||
fi
|
||||
|
||||
# ── Install image generation base model (GPU-aware) ──────────────────────────
|
||||
if $INSTALL_AI && ($SVC_INVOKEAI || $SVC_COMFYUI) && $PULL_MODELS; then
|
||||
section "Image Generation Models"
|
||||
info "GPU: ${TOTAL_VRAM}GB VRAM → $IMG_TIER"
|
||||
if [[ -n "$IMG_DEFAULT" ]]; then
|
||||
info "Auto-installing recommended model: $IMG_DEFAULT"
|
||||
info "Supported on your GPU: $IMG_MODELS"
|
||||
bash "$SCRIPT_DIR/setup-image-models.sh" --auto
|
||||
else
|
||||
warn "Not enough VRAM for image generation models (need at least 4GB)"
|
||||
fi
|
||||
elif $INSTALL_AI && ($SVC_INVOKEAI || $SVC_COMFYUI); then
|
||||
info "Skipping image model install — run later: bash $SCRIPT_DIR/setup-image-models.sh"
|
||||
fi
|
||||
|
||||
# ── Start ZIM downloads (answer was captured upfront) ────────────────────────
|
||||
if $SVC_KIWIX && [[ "$ZIM_CHOICE" != "3" ]]; then
|
||||
section "Starting ZIM Downloads"
|
||||
@@ -1476,23 +1650,31 @@ if $INSTALL_AI; then
|
||||
echo " → Auto-summarizes old messages when context fills up (like Claude)"
|
||||
echo " Auto Memory: Install from Admin → Functions → Discover → search 'Auto Memory'"
|
||||
echo " → Automatically stores relevant info as persistent memories across chats"
|
||||
($SVC_COMFYUI || $SVC_INVOKEAI) && {
|
||||
echo ""
|
||||
echo -e " ${YELLOW}Image Generation — GPU: ${TOTAL_VRAM}GB → $IMG_TIER${NC}"
|
||||
echo " Install base models (detects your GPU automatically):"
|
||||
echo " ./setup-image-models.sh # interactive"
|
||||
echo " ./setup-image-models.sh --auto # install recommended default"
|
||||
echo " Supports: $IMG_MODELS"
|
||||
}
|
||||
$SVC_COMFYUI && {
|
||||
echo ""
|
||||
echo -e " ${YELLOW}Image Generation (ComfyUI → Open WebUI):${NC}"
|
||||
echo " ComfyUI is pre-configured. To complete setup:"
|
||||
echo " 1. Open ComfyUI at http://$LOCAL_IP:8188 and install a model (e.g. SD 1.5, SDXL)"
|
||||
echo -e " ${YELLOW}ComfyUI → Open WebUI (chat-integrated image gen):${NC}"
|
||||
echo " 1. Run ./setup-image-models.sh to install a base model"
|
||||
echo " 2. In ComfyUI: Settings (gear) → enable 'Dev Mode' → Save workflow as 'API Format'"
|
||||
echo " 3. In Open WebUI: Admin → Settings → Images"
|
||||
echo " Engine: ComfyUI | URL: http://comfyui:8188 (already set via env vars)"
|
||||
echo " 4. Import your workflow JSON and map the prompt/output nodes"
|
||||
echo " 5. Ask any model to 'generate an image of...' — it will use ComfyUI"
|
||||
}
|
||||
$SVC_INVOKEAI && ! $SVC_COMFYUI && {
|
||||
$SVC_INVOKEAI && {
|
||||
echo ""
|
||||
echo -e " ${YELLOW}Image Generation (InvokeAI — standalone):${NC}"
|
||||
echo " InvokeAI runs at http://$LOCAL_IP:9090 with its own UI"
|
||||
echo " Note: InvokeAI does NOT integrate with Open WebUI natively"
|
||||
echo " For Open WebUI integration, enable ComfyUI in the setup wizard"
|
||||
echo -e " ${YELLOW}InvokeAI (standalone UI — inpainting, img2img, LoRA):${NC}"
|
||||
echo " InvokeAI runs at http://$LOCAL_IP:9090"
|
||||
echo " For inpainting: Unified Canvas tab → brush over area → describe replacement"
|
||||
echo " Import LoRAs: ./invokeai-import-lora.sh <file.safetensors>"
|
||||
$SVC_COMFYUI || echo " For Open WebUI chat integration, enable ComfyUI in the setup wizard"
|
||||
}
|
||||
fi
|
||||
if $SVC_KIWIX && [[ "$ZIM_CHOICE" == "3" ]]; then
|
||||
|
||||
+78
-5
@@ -54,10 +54,48 @@ else
|
||||
fi
|
||||
EMBED_MODEL="nomic-embed-text"
|
||||
|
||||
# ── Image generation model tiers (VRAM-aware) ────────────────────────────────
|
||||
# These vars are used by setup-image-models.sh and printed in status output.
|
||||
# Image gen shares GPU with Ollama — Ollama unloads after KEEP_ALIVE timeout,
|
||||
# so image gen gets full VRAM when Ollama is idle.
|
||||
if [[ "$TOTAL_VRAM" -ge 24 ]]; then
|
||||
IMG_MODELS="SD 1.5, SDXL, SDXL Turbo, Flux.1-dev, Flux.1-schnell"
|
||||
IMG_TIER="all models including Flux"
|
||||
IMG_DEFAULT="SDXL"
|
||||
elif [[ "$TOTAL_VRAM" -ge 12 ]]; then
|
||||
IMG_MODELS="SD 1.5, SDXL, SDXL Turbo, Flux.1-schnell (tight)"
|
||||
IMG_TIER="SDXL + Flux-schnell"
|
||||
IMG_DEFAULT="SDXL"
|
||||
elif [[ "$TOTAL_VRAM" -ge 8 ]]; then
|
||||
IMG_MODELS="SD 1.5, SDXL (tight at 512px), SDXL Turbo"
|
||||
IMG_TIER="SD 1.5 comfortable, SDXL possible"
|
||||
IMG_DEFAULT="SD 1.5"
|
||||
elif [[ "$TOTAL_VRAM" -ge 4 ]]; then
|
||||
IMG_MODELS="SD 1.5 (float16)"
|
||||
IMG_TIER="SD 1.5 only"
|
||||
IMG_DEFAULT="SD 1.5"
|
||||
else
|
||||
IMG_MODELS="none (CPU generation extremely slow)"
|
||||
IMG_TIER="CPU only — not recommended"
|
||||
IMG_DEFAULT=""
|
||||
fi
|
||||
|
||||
# ── InvokeAI precision (GPU-aware) ───────────────────────────────────────────
|
||||
GPU_COMPUTE=$(nvidia-smi --query-gpu=compute_cap --format=csv,noheader 2>/dev/null \
|
||||
| head -1 | tr -d '.' || echo "0")
|
||||
if [[ "$GPU_COMPUTE" -ge 80 ]]; then
|
||||
INVOKEAI_PRECISION="bfloat16" # Ampere+ (RTX 30xx/40xx/50xx, A-series)
|
||||
elif [[ "$GPU_COMPUTE" -ge 60 ]]; then
|
||||
INVOKEAI_PRECISION="float16" # Pascal+ (GTX 10xx, RTX 20xx, Tesla P40/V100)
|
||||
else
|
||||
INVOKEAI_PRECISION="auto" # Let InvokeAI decide
|
||||
fi
|
||||
|
||||
section "Local AI Stack — $($IS_UPDATE && echo UPDATE || echo NEW INSTALL)"
|
||||
info "Base : $BASE"
|
||||
info "IP : $LOCAL_IP"
|
||||
info "GPU : ${VRAM_GB}GB VRAM → $TIER"
|
||||
info "Image : $IMG_TIER ($IMG_MODELS)"
|
||||
|
||||
|
||||
write_if_new() {
|
||||
@@ -97,7 +135,7 @@ fi
|
||||
|
||||
# ── directories ───────────────────────────────────────────────────────────────
|
||||
section "Directories"
|
||||
for d in papers repos workspace index invokeai-data invokeai-outputs kiwix gitea portainer-data logs; do
|
||||
for d in papers repos workspace index invokeai-data invokeai-outputs comfyui-data comfyui-output kiwix gitea portainer-data logs; do
|
||||
mkdir -p "$BASE/$d"
|
||||
done
|
||||
ok "Ready under $BASE"
|
||||
@@ -575,6 +613,9 @@ services:
|
||||
- WEBUI_AUTH=true
|
||||
- ENABLE_RAG_WEB_SEARCH=true
|
||||
- RAG_WEB_SEARCH_ENGINE=duckduckgo
|
||||
- ENABLE_IMAGE_GENERATION=true
|
||||
- IMAGE_GENERATION_ENGINE=comfyui
|
||||
- COMFYUI_BASE_URL=http://comfyui:8188
|
||||
depends_on:
|
||||
ollama: {condition: service_healthy}
|
||||
|
||||
@@ -699,7 +740,26 @@ services:
|
||||
environment:
|
||||
- INVOKEAI_HOST=0.0.0.0
|
||||
- INVOKEAI_PORT=9090
|
||||
- INVOKEAI_PRECISION=float16
|
||||
- INVOKEAI_PRECISION=$INVOKEAI_PRECISION
|
||||
deploy:
|
||||
resources:
|
||||
reservations:
|
||||
devices:
|
||||
- driver: nvidia
|
||||
count: all
|
||||
capabilities: [gpu]
|
||||
|
||||
comfyui:
|
||||
image: ghcr.io/ai-dock/comfyui:latest
|
||||
container_name: comfyui
|
||||
restart: unless-stopped
|
||||
ports: ["0.0.0.0:8188:8188"]
|
||||
volumes:
|
||||
- comfyui-models:/opt/ComfyUI/models
|
||||
- $BASE/comfyui-output:/opt/ComfyUI/output
|
||||
- $BASE/comfyui-data:/opt/ComfyUI/custom_nodes
|
||||
environment:
|
||||
- CLI_ARGS=--listen 0.0.0.0
|
||||
deploy:
|
||||
resources:
|
||||
reservations:
|
||||
@@ -721,6 +781,7 @@ volumes:
|
||||
ollama-models:
|
||||
open-webui-data:
|
||||
invokeai-models:
|
||||
comfyui-models:
|
||||
COMPOSE
|
||||
ok "docker-compose.yml"
|
||||
|
||||
@@ -732,7 +793,7 @@ if command -v ufw &>/dev/null && [[ ! -f "$BASE/.ufw-done" ]] || $FORCE; then
|
||||
[[ "$LAN" =~ /[0-9]+$ ]] || LAN="${LAN}/24"
|
||||
for pc in "3000:Open WebUI" "11434:Ollama" "8001:RAG" "8002:MCP" \
|
||||
"8000:ChromaDB" "8080:Aider" "8181:Kiwix" \
|
||||
"3001:Gitea" "2222:Gitea SSH" "9090:InvokeAI" \
|
||||
"3001:Gitea" "2222:Gitea SSH" "9090:InvokeAI" "8188:ComfyUI" \
|
||||
"9000:Portainer" "9443:Portainer S"; do
|
||||
sudo ufw allow from "$LAN" to any port "${pc%%:*}" proto tcp comment "${pc##*:}" >/dev/null
|
||||
done
|
||||
@@ -750,7 +811,8 @@ docker compose up -d
|
||||
echo ""
|
||||
echo " Open WebUI → http://$LOCAL_IP:3000"
|
||||
echo " Aider UI → http://$LOCAL_IP:8080 (Claude Code-like editor)"
|
||||
echo " InvokeAI → http://$LOCAL_IP:9090"
|
||||
echo " InvokeAI → http://$LOCAL_IP:9090 (inpainting, img2img)"
|
||||
echo " ComfyUI → http://$LOCAL_IP:8188 (chat-integrated image gen)"
|
||||
echo " Kiwix → http://$LOCAL_IP:8181 (run kiwix_download.sh first)"
|
||||
echo " Gitea → http://$LOCAL_IP:3001"
|
||||
echo " RAG → http://$LOCAL_IP:8001/health"
|
||||
@@ -873,17 +935,28 @@ if ! $IS_UPDATE && ! $NO_PULL; then
|
||||
echo ""
|
||||
read -rp "Pull Ollama models now? (~15-30 min) [Y/n]: " DO_PULL
|
||||
[[ "${DO_PULL,,}" != "n" ]] && bash "$BASE/pull-models.sh"
|
||||
|
||||
# Install image generation base model (GPU-aware)
|
||||
if [[ -n "$IMG_DEFAULT" ]] && [[ -x "$SCRIPT_DIR/setup-image-models.sh" ]]; then
|
||||
echo ""
|
||||
section "Image Generation Models"
|
||||
info "GPU: ${TOTAL_VRAM}GB VRAM → $IMG_TIER"
|
||||
info "Auto-installing recommended model: $IMG_DEFAULT"
|
||||
bash "$SCRIPT_DIR/setup-image-models.sh" --auto
|
||||
fi
|
||||
fi
|
||||
|
||||
# =============================================================================
|
||||
echo ""
|
||||
echo -e "${GREEN}${BOLD}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"
|
||||
echo -e "${GREEN}${BOLD} Done! GPU: ${VRAM_GB}GB → $TIER${NC}"
|
||||
echo -e "${GREEN}${BOLD} Image gen: $IMG_TIER${NC}"
|
||||
echo -e "${GREEN}${BOLD}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"
|
||||
echo ""
|
||||
echo -e " ${CYAN}Open WebUI${NC} → http://$LOCAL_IP:3000"
|
||||
echo -e " ${CYAN}Aider UI${NC} → http://$LOCAL_IP:8080 (browser coding assistant)"
|
||||
echo -e " ${CYAN}InvokeAI${NC} → http://$LOCAL_IP:9090"
|
||||
echo -e " ${CYAN}InvokeAI${NC} → http://$LOCAL_IP:9090 (inpainting, img2img)"
|
||||
echo -e " ${CYAN}ComfyUI${NC} → http://$LOCAL_IP:8188 (chat-integrated image gen)"
|
||||
echo -e " ${CYAN}Kiwix${NC} → http://$LOCAL_IP:8181"
|
||||
echo -e " ${CYAN}Gitea${NC} → http://$LOCAL_IP:3001"
|
||||
echo -e " ${CYAN}RAG${NC} → http://$LOCAL_IP:8001/health"
|
||||
|
||||
Executable
+275
@@ -0,0 +1,275 @@
|
||||
#!/usr/bin/env bash
|
||||
# Detect GPU VRAM and install appropriate Stable Diffusion models for
|
||||
# InvokeAI and/or ComfyUI. Run anytime — safe to re-run.
|
||||
#
|
||||
# Usage: ./setup-image-models.sh [--auto]
|
||||
# --auto Skip prompts, install the recommended default for your GPU
|
||||
set -euo pipefail
|
||||
|
||||
RED='\033[0;31m'; GREEN='\033[0;32m'; YELLOW='\033[1;33m'
|
||||
CYAN='\033[0;36m'; BOLD='\033[1m'; NC='\033[0m'
|
||||
info() { echo -e "${CYAN}[..]${NC} $*"; }
|
||||
ok() { echo -e "${GREEN}[OK]${NC} $*"; }
|
||||
warn() { echo -e "${YELLOW}[!!]${NC} $*"; }
|
||||
|
||||
AUTO=false
|
||||
[[ "${1:-}" == "--auto" ]] && AUTO=true
|
||||
|
||||
# ── Detect GPU ────────────────────────────────────────────────────────────────
|
||||
VRAM_GB=$(nvidia-smi --query-gpu=memory.total --format=csv,noheader,nounits 2>/dev/null \
|
||||
| head -1 | awk '{printf "%d", $1/1024}' 2>/dev/null || echo "0")
|
||||
GPU_COUNT=$(nvidia-smi --query-gpu=name --format=csv,noheader 2>/dev/null | wc -l || echo "0")
|
||||
GPU_NAME=$(nvidia-smi --query-gpu=name --format=csv,noheader 2>/dev/null | head -1 || echo "None")
|
||||
TOTAL_VRAM=$((VRAM_GB * GPU_COUNT))
|
||||
|
||||
echo ""
|
||||
echo -e "${BOLD}━━━ Image Generation Model Setup ━━━${NC}"
|
||||
echo ""
|
||||
info "GPU : $GPU_NAME"
|
||||
[[ "$GPU_COUNT" -gt 1 ]] && info "GPU count : $GPU_COUNT"
|
||||
info "VRAM/card : ${VRAM_GB}GB"
|
||||
info "Total VRAM: ${TOTAL_VRAM}GB"
|
||||
echo ""
|
||||
|
||||
# ── Determine which containers are available ──────────────────────────────────
|
||||
HAS_INVOKEAI=false
|
||||
HAS_COMFYUI=false
|
||||
docker ps -a --format '{{.Names}}' 2>/dev/null | grep -q '^invokeai$' && HAS_INVOKEAI=true
|
||||
docker ps -a --format '{{.Names}}' 2>/dev/null | grep -q '^comfyui$' && HAS_COMFYUI=true
|
||||
|
||||
if ! $HAS_INVOKEAI && ! $HAS_COMFYUI; then
|
||||
echo -e "${RED}Error:${NC} Neither InvokeAI nor ComfyUI containers found."
|
||||
echo " Run the setup script first to deploy the AI stack."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
$HAS_INVOKEAI && info "InvokeAI : found"
|
||||
$HAS_COMFYUI && info "ComfyUI : found"
|
||||
echo ""
|
||||
|
||||
# ── Build model menu based on VRAM ────────────────────────────────────────────
|
||||
# Model VRAM requirements (generation, not just loading):
|
||||
# SD 1.5 ~4GB 512x512 native
|
||||
# SDXL ~7GB 1024x1024 native
|
||||
# SDXL Turbo ~7GB 512x512 (4-step)
|
||||
# Flux.1-schnell ~12GB fast, high quality
|
||||
# Flux.1-dev ~20GB best quality, slow
|
||||
|
||||
declare -a MODEL_IDS=()
|
||||
declare -a MODEL_NAMES=()
|
||||
declare -a MODEL_VRAM=()
|
||||
declare -a MODEL_NOTES=()
|
||||
|
||||
add_model() {
|
||||
MODEL_IDS+=("$1"); MODEL_NAMES+=("$2"); MODEL_VRAM+=("$3"); MODEL_NOTES+=("$4")
|
||||
}
|
||||
|
||||
# Always offer SD 1.5 if any GPU exists
|
||||
if [[ "$TOTAL_VRAM" -ge 4 ]]; then
|
||||
add_model "sd15" "Stable Diffusion 1.5" "4" "512px native, most LoRA compatible, fast"
|
||||
fi
|
||||
|
||||
if [[ "$TOTAL_VRAM" -ge 8 ]]; then
|
||||
add_model "sdxl" "Stable Diffusion XL" "7" "1024px native, better quality, more detail"
|
||||
add_model "sdxl-turbo" "SDXL Turbo" "7" "4-step generation, very fast, good quality"
|
||||
fi
|
||||
|
||||
if [[ "$TOTAL_VRAM" -ge 12 ]]; then
|
||||
add_model "flux-schnell" "Flux.1-schnell" "12" "fast Flux variant, excellent quality"
|
||||
fi
|
||||
|
||||
if [[ "$TOTAL_VRAM" -ge 20 ]]; then
|
||||
add_model "flux-dev" "Flux.1-dev" "20" "best quality, slower, needs lots of VRAM"
|
||||
fi
|
||||
|
||||
if [[ ${#MODEL_IDS[@]} -eq 0 ]]; then
|
||||
warn "No GPU with sufficient VRAM detected (need at least 4GB)."
|
||||
warn "CPU-only image generation is extremely slow and not recommended."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# ── Determine default recommendation ─────────────────────────────────────────
|
||||
if [[ "$TOTAL_VRAM" -ge 20 ]]; then DEFAULT_ID="flux-dev"
|
||||
elif [[ "$TOTAL_VRAM" -ge 12 ]]; then DEFAULT_ID="sdxl"
|
||||
elif [[ "$TOTAL_VRAM" -ge 8 ]]; then DEFAULT_ID="sdxl"
|
||||
elif [[ "$TOTAL_VRAM" -ge 4 ]]; then DEFAULT_ID="sd15"
|
||||
else DEFAULT_ID="sd15"
|
||||
fi
|
||||
|
||||
echo -e "${BOLD}Available models for your ${TOTAL_VRAM}GB GPU:${NC}"
|
||||
echo ""
|
||||
for i in "${!MODEL_IDS[@]}"; do
|
||||
DEFAULT_TAG=""
|
||||
[[ "${MODEL_IDS[$i]}" == "$DEFAULT_ID" ]] && DEFAULT_TAG=" ${GREEN}← recommended${NC}"
|
||||
printf " ${BOLD}%d)${NC} %-25s ~%sGB VRAM %s%b\n" \
|
||||
$((i+1)) "${MODEL_NAMES[$i]}" "${MODEL_VRAM[$i]}" "${MODEL_NOTES[$i]}" "$DEFAULT_TAG"
|
||||
done
|
||||
echo ""
|
||||
|
||||
if $AUTO; then
|
||||
SELECTED="$DEFAULT_ID"
|
||||
info "Auto mode: installing $SELECTED"
|
||||
else
|
||||
echo -e " Enter number(s) separated by spaces, or press Enter for recommended."
|
||||
echo -e " Example: ${BOLD}1 2${NC} to install both SD 1.5 and SDXL"
|
||||
echo ""
|
||||
read -rp " Selection [recommended]: " CHOICE
|
||||
|
||||
if [[ -z "$CHOICE" ]]; then
|
||||
SELECTED="$DEFAULT_ID"
|
||||
else
|
||||
SELECTED=""
|
||||
for num in $CHOICE; do
|
||||
idx=$((num - 1))
|
||||
if [[ $idx -ge 0 && $idx -lt ${#MODEL_IDS[@]} ]]; then
|
||||
SELECTED+=" ${MODEL_IDS[$idx]}"
|
||||
else
|
||||
warn "Invalid selection: $num (skipping)"
|
||||
fi
|
||||
done
|
||||
SELECTED="${SELECTED# }"
|
||||
fi
|
||||
fi
|
||||
|
||||
[[ -z "$SELECTED" ]] && { warn "No models selected."; exit 1; }
|
||||
|
||||
echo ""
|
||||
info "Will install: $SELECTED"
|
||||
echo ""
|
||||
|
||||
# ── HuggingFace model identifiers ────────────────────────────────────────────
|
||||
declare -A HF_MODELS=(
|
||||
[sd15]="stabilityai/stable-diffusion-v1-5"
|
||||
[sdxl]="stabilityai/stable-diffusion-xl-base-1.0"
|
||||
[sdxl-turbo]="stabilityai/sdxl-turbo"
|
||||
[flux-schnell]="black-forest-labs/FLUX.1-schnell"
|
||||
[flux-dev]="black-forest-labs/FLUX.1-dev"
|
||||
)
|
||||
|
||||
declare -A MODEL_SIZES=(
|
||||
[sd15]="~4GB"
|
||||
[sdxl]="~7GB"
|
||||
[sdxl-turbo]="~7GB"
|
||||
[flux-schnell]="~12GB"
|
||||
[flux-dev]="~24GB"
|
||||
)
|
||||
|
||||
# ── Install into InvokeAI ────────────────────────────────────────────────────
|
||||
if $HAS_INVOKEAI; then
|
||||
echo -e "${BOLD}━━━ Installing into InvokeAI ━━━${NC}"
|
||||
|
||||
# Make sure container is running
|
||||
if ! docker ps --format '{{.Names}}' | grep -q '^invokeai$'; then
|
||||
info "Starting InvokeAI container..."
|
||||
docker start invokeai
|
||||
sleep 5
|
||||
fi
|
||||
|
||||
for model_id in $SELECTED; do
|
||||
hf_id="${HF_MODELS[$model_id]:-}"
|
||||
[[ -z "$hf_id" ]] && { warn "Unknown model: $model_id"; continue; }
|
||||
info "Installing $model_id (${MODEL_SIZES[$model_id]}) → ${hf_id}..."
|
||||
info " This may take a while depending on your connection."
|
||||
|
||||
if docker exec invokeai invokeai-model-install --add "$hf_id" 2>&1; then
|
||||
ok "$model_id installed in InvokeAI"
|
||||
else
|
||||
warn "$model_id install failed in InvokeAI — try manually via Model Manager at :9090"
|
||||
fi
|
||||
echo ""
|
||||
done
|
||||
fi
|
||||
|
||||
# ── Install into ComfyUI ─────────────────────────────────────────────────────
|
||||
if $HAS_COMFYUI; then
|
||||
echo -e "${BOLD}━━━ Installing into ComfyUI ━━━${NC}"
|
||||
info "ComfyUI downloads models on first use via its UI."
|
||||
info "To pre-download, use the ComfyUI Manager at http://localhost:8188"
|
||||
echo ""
|
||||
|
||||
# Make sure container is running
|
||||
if ! docker ps --format '{{.Names}}' | grep -q '^comfyui$'; then
|
||||
info "Starting ComfyUI container..."
|
||||
docker start comfyui
|
||||
sleep 5
|
||||
fi
|
||||
|
||||
# For ComfyUI, download checkpoints into the models volume
|
||||
for model_id in $SELECTED; do
|
||||
hf_id="${HF_MODELS[$model_id]:-}"
|
||||
[[ -z "$hf_id" ]] && continue
|
||||
|
||||
# Check if model already exists
|
||||
CKPT_DIR="/opt/ComfyUI/models/checkpoints"
|
||||
if docker exec comfyui ls "$CKPT_DIR" 2>/dev/null | grep -qi "${model_id//-/_}"; then
|
||||
ok "$model_id already present in ComfyUI"
|
||||
continue
|
||||
fi
|
||||
|
||||
info "Downloading $model_id for ComfyUI (${MODEL_SIZES[$model_id]})..."
|
||||
info " Downloading from HuggingFace: $hf_id"
|
||||
|
||||
# Use ComfyUI's built-in download mechanism via python
|
||||
case "$model_id" in
|
||||
sd15)
|
||||
docker exec comfyui bash -c \
|
||||
"cd /opt/ComfyUI && python -c \"
|
||||
from huggingface_hub import hf_hub_download
|
||||
hf_hub_download('$hf_id', 'v1-5-pruned-emaonly.safetensors', local_dir='models/checkpoints')
|
||||
\" 2>&1" && ok "$model_id downloaded for ComfyUI" \
|
||||
|| warn "$model_id download failed — install via ComfyUI Manager UI"
|
||||
;;
|
||||
sdxl)
|
||||
docker exec comfyui bash -c \
|
||||
"cd /opt/ComfyUI && python -c \"
|
||||
from huggingface_hub import hf_hub_download
|
||||
hf_hub_download('$hf_id', 'sd_xl_base_1.0.safetensors', local_dir='models/checkpoints')
|
||||
\" 2>&1" && ok "$model_id downloaded for ComfyUI" \
|
||||
|| warn "$model_id download failed — install via ComfyUI Manager UI"
|
||||
;;
|
||||
*)
|
||||
info "$model_id: use ComfyUI Manager to install (complex model structure)"
|
||||
;;
|
||||
esac
|
||||
echo ""
|
||||
done
|
||||
fi
|
||||
|
||||
# ── Summary ───────────────────────────────────────────────────────────────────
|
||||
echo ""
|
||||
echo -e "${GREEN}${BOLD}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"
|
||||
echo -e "${GREEN}${BOLD} Image model setup complete!${NC}"
|
||||
echo -e "${GREEN}${BOLD}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}"
|
||||
echo ""
|
||||
echo -e " ${BOLD}Your GPU:${NC} $GPU_NAME — ${TOTAL_VRAM}GB VRAM"
|
||||
echo -e " ${BOLD}Installed:${NC} $SELECTED"
|
||||
echo ""
|
||||
$HAS_INVOKEAI && echo -e " ${CYAN}InvokeAI${NC} → http://localhost:9090"
|
||||
$HAS_COMFYUI && echo -e " ${CYAN}ComfyUI${NC} → http://localhost:8188"
|
||||
echo ""
|
||||
|
||||
if $HAS_INVOKEAI; then
|
||||
echo -e " ${YELLOW}InvokeAI quick start:${NC}"
|
||||
echo " 1. Open Model Manager → verify your model appears"
|
||||
echo " 2. Go to Text to Image → select the model"
|
||||
echo " 3. For inpainting: use the Unified Canvas tab"
|
||||
echo " - Upload image → brush over the area to change"
|
||||
echo " - Write what you want in that area → Invoke"
|
||||
echo ""
|
||||
fi
|
||||
|
||||
if $HAS_COMFYUI; then
|
||||
echo -e " ${YELLOW}ComfyUI quick start:${NC}"
|
||||
echo " 1. Open ComfyUI → load a basic txt2img workflow"
|
||||
echo " 2. Select your checkpoint in the Load Checkpoint node"
|
||||
echo " 3. For Open WebUI integration: enable Dev Mode → export API workflow"
|
||||
echo ""
|
||||
fi
|
||||
|
||||
echo -e " ${YELLOW}GPU sharing:${NC} Ollama and image gen share the GPU."
|
||||
echo " Ollama auto-unloads models after 24h idle (KEEP_ALIVE=24h)."
|
||||
echo " For immediate unload before heavy image gen:"
|
||||
echo " docker exec ollama ollama stop <model-name>"
|
||||
echo ""
|
||||
echo -e " ${YELLOW}Import LoRAs:${NC} ./invokeai-import-lora.sh <file.safetensors>"
|
||||
echo ""
|
||||
Reference in New Issue
Block a user