Expand InvokeAI docs: LoRA import from RunPod, img2img iteration, comparison

Major expansion of the InvokeAI section:
- Add InvokeAI vs ComfyUI comparison table (when to use each)
- Step-by-step: install base model, import RunPod LoRA, text-to-image
  with LoRA, img2img iteration with denoising strength guide
- Denoising strength table explaining what 0.2 vs 0.8 actually does
- Example prompts for age up/down, emotion, setting, art style changes
- Unified Canvas / inpainting instructions for selective editing
- 6GB GPU notes (SD 1.5 fits, SDXL is tight)
- Expanded troubleshooting for greyed-out buttons and VRAM issues

https://claude.ai/code/session_01PtYTPherSJaxDEVPgF6Nxu
This commit is contained in:
Claude
2026-03-22 21:13:06 +00:00
parent 97561c0219
commit da75be9ae6
+93 -19
View File
@@ -784,35 +784,109 @@ ComfyUI models typically need 48GB VRAM (SD 1.5: ~4GB, SDXL: ~7GB, Flux: ~12G
---
## Using LoRA Models in InvokeAI
## InvokeAI vs ComfyUI — which to use when
LoRA (Low-Rank Adaptation) files let you customize image generation with fine-tuned styles or characters. If you trained a LoRA on RunPod or elsewhere, here's how to use it.
Both are installed by the setup script. Here's when to use each:
### Import a LoRA file
| Task | InvokeAI (`:9090`) | ComfyUI (`:8188`) |
|------|:---:|:---:|
| Friendly UI, sliders, drag-and-drop | Yes | No (node editor) |
| Use a LoRA you trained on RunPod | Yes — just import + select | Yes — but wire LoRA Loader node |
| Same face, different settings (img2img) | Yes — Image to Image tab | Yes — IP-Adapter nodes |
| Age/emotion/scene changes | Yes — img2img + prompt | Yes — IP-Adapter + prompt |
| Chat-integrated image gen (from Open WebUI) | No (no OWUI API) | Yes (workflow export) |
| Maximum flexibility / custom pipelines | No | Yes |
| Learning curve | Low | High |
**Bottom line:** Use InvokeAI for the interactive "play with images" workflow. Use ComfyUI only when you need Open WebUI chat integration or advanced node pipelines.
## Using InvokeAI for image iteration
InvokeAI is the easier path for what you want — take a reference image and generate variations with different settings, ages, emotions, art styles.
### Step 1: Install a base model (required first)
Before LoRAs or img2img will work, InvokeAI needs a base Stable Diffusion model.
```bash
# Copy your LoRA into the InvokeAI Docker volume:
./invokeai-import-lora.sh ~/Downloads/my-lora.safetensors
# Option A: Download SD 1.5 (~4GB VRAM needed) via the container
docker exec invokeai invokeai-model-install --add stabilityai/stable-diffusion-v1-5
# Optionally give it a display name:
./invokeai-import-lora.sh ~/Downloads/my-lora.safetensors "My Custom Style"
# Option B: Download SDXL (~7GB VRAM needed)
docker exec invokeai invokeai-model-install --add stabilityai/stable-diffusion-xl-base-1.0
```
### Use the LoRA in InvokeAI
Or use the UI: open `http://<ip>:9090` → **Model Manager** (cube icon) → **Starter Models** tab → install SD 1.5 or SDXL.
1. Open InvokeAI at `http://<ip>:9090`
2. Go to **Model Manager** (cube icon, left sidebar) and click **Scan for Models** / **Sync Models**
3. Your LoRA should appear in the model list
4. Switch to **Text to Image** tab
5. In the left panel, find the **LoRA** section (below the model selector)
6. Click **+** to add your LoRA, then adjust the **weight** slider (start at 0.70.85)
> **6GB GPU note:** SD 1.5 fits comfortably. SDXL is tight — it may work with float16 (already set) but could be slow. Start with SD 1.5.
### Troubleshooting greyed-out upload buttons
### Step 2: Import your RunPod LoRA
- **No base model installed:** You need a fully downloaded base model (e.g., SD 1.5) before InvokeAI enables LoRA uploads. Use Model Manager to install one first.
- **Model not synced:** After copying files, click **Scan for Models** in Model Manager.
- **Architecture mismatch:** A LoRA trained on SD 1.5 only works with SD 1.5 base models — not SDXL or SD 2.x.
- **Use the import script instead:** The greyed-out UI upload can be bypassed entirely by using `invokeai-import-lora.sh` to copy files directly into the model volume.
```bash
# Copy the .safetensors file from wherever you downloaded it:
./invokeai-import-lora.sh ~/Downloads/my-lora.safetensors "My Character"
```
Then in the InvokeAI UI:
1. **Model Manager** (cube icon) → **Scan for Models**
2. Your LoRA should appear in the list
If it doesn't:
- Click **Add Model** → **Scan Folder** → enter `/invokeai/models/lora`
- Make sure the LoRA architecture matches your base model (SD 1.5 LoRA needs SD 1.5 base)
### Step 3: Generate with your LoRA (text-to-image)
1. Go to the **Text to Image** tab
2. Select your **base model** (must match the LoRA's training base)
3. In the left panel, expand the **LoRA** section (below model selector)
4. Click **+** → select your LoRA → set weight to **0.70.85**
5. Write a prompt: "portrait of [subject], smiling, studio lighting"
6. Click **Invoke**
### Step 4: Iterate on an image (img2img)
This is where InvokeAI shines for your use case — take an image and riff on it:
1. Switch to the **Image to Image** tab
2. Drag your reference photo onto the canvas (or click to upload)
3. Keep your LoRA active (same as above)
4. Set the **Denoising Strength** slider:
| Strength | Effect |
|----------|--------|
| 0.20.3 | Subtle tweaks — mostly keeps the original, minor style changes |
| 0.40.5 | Moderate changes — recognizable but different mood/lighting |
| 0.60.7 | Significant changes — same composition, new details/style |
| 0.81.0 | Major rewrite — loosely inspired by original, mostly new |
5. Change the prompt to describe what you want different:
- **Age up:** "same person, elderly, wrinkles, grey hair, wise expression"
- **Age down:** "same person as a young child, bright eyes, playground"
- **Emotion:** "same person, laughing joyfully" or "same person, crying, dramatic lighting"
- **Setting:** "same person, sitting in a Parisian cafe, afternoon light"
- **Art style:** "same person, oil painting, renaissance style, dramatic chiaroscuro"
6. Click **Invoke** — iterate by adjusting strength and prompt
### Step 5: Use the Unified Canvas for painting/inpainting
For more control (paint over specific areas, extend an image):
1. Switch to the **Unified Canvas** tab
2. Upload or paste your image
3. Use the **brush tool** to mask areas you want to change
4. Write a prompt for just the masked area
5. Invoke — only the masked area regenerates
Example: mask just the background → prompt "tropical beach sunset" → keeps the face, replaces the background.
### Troubleshooting
- **Greyed-out upload/LoRA buttons:** Install a base model first (Step 1). InvokeAI disables most features until a checkpoint is loaded.
- **Model not synced:** After copying files via script, click **Scan for Models** in Model Manager.
- **Architecture mismatch:** A LoRA trained on SD 1.5 only works with SD 1.5 base models — not SDXL. Check what your RunPod training used.
- **Out of VRAM:** Try SD 1.5 instead of SDXL, or reduce image size to 512x512.
- **Use the import script:** The greyed-out UI upload can be bypassed entirely by using `invokeai-import-lora.sh` to copy files directly into the model volume.
---