Expand InvokeAI docs: LoRA import from RunPod, img2img iteration, comparison
Major expansion of the InvokeAI section: - Add InvokeAI vs ComfyUI comparison table (when to use each) - Step-by-step: install base model, import RunPod LoRA, text-to-image with LoRA, img2img iteration with denoising strength guide - Denoising strength table explaining what 0.2 vs 0.8 actually does - Example prompts for age up/down, emotion, setting, art style changes - Unified Canvas / inpainting instructions for selective editing - 6GB GPU notes (SD 1.5 fits, SDXL is tight) - Expanded troubleshooting for greyed-out buttons and VRAM issues https://claude.ai/code/session_01PtYTPherSJaxDEVPgF6Nxu
This commit is contained in:
@@ -784,35 +784,109 @@ ComfyUI models typically need 4–8GB VRAM (SD 1.5: ~4GB, SDXL: ~7GB, Flux: ~12G
|
||||
|
||||
---
|
||||
|
||||
## Using LoRA Models in InvokeAI
|
||||
## InvokeAI vs ComfyUI — which to use when
|
||||
|
||||
LoRA (Low-Rank Adaptation) files let you customize image generation with fine-tuned styles or characters. If you trained a LoRA on RunPod or elsewhere, here's how to use it.
|
||||
Both are installed by the setup script. Here's when to use each:
|
||||
|
||||
### Import a LoRA file
|
||||
| Task | InvokeAI (`:9090`) | ComfyUI (`:8188`) |
|
||||
|------|:---:|:---:|
|
||||
| Friendly UI, sliders, drag-and-drop | Yes | No (node editor) |
|
||||
| Use a LoRA you trained on RunPod | Yes — just import + select | Yes — but wire LoRA Loader node |
|
||||
| Same face, different settings (img2img) | Yes — Image to Image tab | Yes — IP-Adapter nodes |
|
||||
| Age/emotion/scene changes | Yes — img2img + prompt | Yes — IP-Adapter + prompt |
|
||||
| Chat-integrated image gen (from Open WebUI) | No (no OWUI API) | Yes (workflow export) |
|
||||
| Maximum flexibility / custom pipelines | No | Yes |
|
||||
| Learning curve | Low | High |
|
||||
|
||||
**Bottom line:** Use InvokeAI for the interactive "play with images" workflow. Use ComfyUI only when you need Open WebUI chat integration or advanced node pipelines.
|
||||
|
||||
## Using InvokeAI for image iteration
|
||||
|
||||
InvokeAI is the easier path for what you want — take a reference image and generate variations with different settings, ages, emotions, art styles.
|
||||
|
||||
### Step 1: Install a base model (required first)
|
||||
|
||||
Before LoRAs or img2img will work, InvokeAI needs a base Stable Diffusion model.
|
||||
|
||||
```bash
|
||||
# Copy your LoRA into the InvokeAI Docker volume:
|
||||
./invokeai-import-lora.sh ~/Downloads/my-lora.safetensors
|
||||
# Option A: Download SD 1.5 (~4GB VRAM needed) via the container
|
||||
docker exec invokeai invokeai-model-install --add stabilityai/stable-diffusion-v1-5
|
||||
|
||||
# Optionally give it a display name:
|
||||
./invokeai-import-lora.sh ~/Downloads/my-lora.safetensors "My Custom Style"
|
||||
# Option B: Download SDXL (~7GB VRAM needed)
|
||||
docker exec invokeai invokeai-model-install --add stabilityai/stable-diffusion-xl-base-1.0
|
||||
```
|
||||
|
||||
### Use the LoRA in InvokeAI
|
||||
Or use the UI: open `http://<ip>:9090` → **Model Manager** (cube icon) → **Starter Models** tab → install SD 1.5 or SDXL.
|
||||
|
||||
1. Open InvokeAI at `http://<ip>:9090`
|
||||
2. Go to **Model Manager** (cube icon, left sidebar) and click **Scan for Models** / **Sync Models**
|
||||
3. Your LoRA should appear in the model list
|
||||
4. Switch to **Text to Image** tab
|
||||
5. In the left panel, find the **LoRA** section (below the model selector)
|
||||
6. Click **+** to add your LoRA, then adjust the **weight** slider (start at 0.7–0.85)
|
||||
> **6GB GPU note:** SD 1.5 fits comfortably. SDXL is tight — it may work with float16 (already set) but could be slow. Start with SD 1.5.
|
||||
|
||||
### Troubleshooting greyed-out upload buttons
|
||||
### Step 2: Import your RunPod LoRA
|
||||
|
||||
- **No base model installed:** You need a fully downloaded base model (e.g., SD 1.5) before InvokeAI enables LoRA uploads. Use Model Manager to install one first.
|
||||
- **Model not synced:** After copying files, click **Scan for Models** in Model Manager.
|
||||
- **Architecture mismatch:** A LoRA trained on SD 1.5 only works with SD 1.5 base models — not SDXL or SD 2.x.
|
||||
- **Use the import script instead:** The greyed-out UI upload can be bypassed entirely by using `invokeai-import-lora.sh` to copy files directly into the model volume.
|
||||
```bash
|
||||
# Copy the .safetensors file from wherever you downloaded it:
|
||||
./invokeai-import-lora.sh ~/Downloads/my-lora.safetensors "My Character"
|
||||
```
|
||||
|
||||
Then in the InvokeAI UI:
|
||||
1. **Model Manager** (cube icon) → **Scan for Models**
|
||||
2. Your LoRA should appear in the list
|
||||
|
||||
If it doesn't:
|
||||
- Click **Add Model** → **Scan Folder** → enter `/invokeai/models/lora`
|
||||
- Make sure the LoRA architecture matches your base model (SD 1.5 LoRA needs SD 1.5 base)
|
||||
|
||||
### Step 3: Generate with your LoRA (text-to-image)
|
||||
|
||||
1. Go to the **Text to Image** tab
|
||||
2. Select your **base model** (must match the LoRA's training base)
|
||||
3. In the left panel, expand the **LoRA** section (below model selector)
|
||||
4. Click **+** → select your LoRA → set weight to **0.7–0.85**
|
||||
5. Write a prompt: "portrait of [subject], smiling, studio lighting"
|
||||
6. Click **Invoke**
|
||||
|
||||
### Step 4: Iterate on an image (img2img)
|
||||
|
||||
This is where InvokeAI shines for your use case — take an image and riff on it:
|
||||
|
||||
1. Switch to the **Image to Image** tab
|
||||
2. Drag your reference photo onto the canvas (or click to upload)
|
||||
3. Keep your LoRA active (same as above)
|
||||
4. Set the **Denoising Strength** slider:
|
||||
|
||||
| Strength | Effect |
|
||||
|----------|--------|
|
||||
| 0.2–0.3 | Subtle tweaks — mostly keeps the original, minor style changes |
|
||||
| 0.4–0.5 | Moderate changes — recognizable but different mood/lighting |
|
||||
| 0.6–0.7 | Significant changes — same composition, new details/style |
|
||||
| 0.8–1.0 | Major rewrite — loosely inspired by original, mostly new |
|
||||
|
||||
5. Change the prompt to describe what you want different:
|
||||
- **Age up:** "same person, elderly, wrinkles, grey hair, wise expression"
|
||||
- **Age down:** "same person as a young child, bright eyes, playground"
|
||||
- **Emotion:** "same person, laughing joyfully" or "same person, crying, dramatic lighting"
|
||||
- **Setting:** "same person, sitting in a Parisian cafe, afternoon light"
|
||||
- **Art style:** "same person, oil painting, renaissance style, dramatic chiaroscuro"
|
||||
6. Click **Invoke** — iterate by adjusting strength and prompt
|
||||
|
||||
### Step 5: Use the Unified Canvas for painting/inpainting
|
||||
|
||||
For more control (paint over specific areas, extend an image):
|
||||
|
||||
1. Switch to the **Unified Canvas** tab
|
||||
2. Upload or paste your image
|
||||
3. Use the **brush tool** to mask areas you want to change
|
||||
4. Write a prompt for just the masked area
|
||||
5. Invoke — only the masked area regenerates
|
||||
|
||||
Example: mask just the background → prompt "tropical beach sunset" → keeps the face, replaces the background.
|
||||
|
||||
### Troubleshooting
|
||||
|
||||
- **Greyed-out upload/LoRA buttons:** Install a base model first (Step 1). InvokeAI disables most features until a checkpoint is loaded.
|
||||
- **Model not synced:** After copying files via script, click **Scan for Models** in Model Manager.
|
||||
- **Architecture mismatch:** A LoRA trained on SD 1.5 only works with SD 1.5 base models — not SDXL. Check what your RunPod training used.
|
||||
- **Out of VRAM:** Try SD 1.5 instead of SDXL, or reduce image size to 512x512.
|
||||
- **Use the import script:** The greyed-out UI upload can be bypassed entirely by using `invokeai-import-lora.sh` to copy files directly into the model volume.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user