Add GPU-aware image model detection and setup-image-models.sh

- Both setup scripts now detect VRAM and determine which image gen
  models the GPU can run (SD 1.5 at 4GB, SDXL at 8GB, Flux at 12-20GB)
- New setup-image-models.sh: interactive script that detects GPU,
  shows available models with VRAM requirements, and installs into
  InvokeAI and/or ComfyUI. Supports --auto for unattended install.
- Scales from 4GB cards through dual RTX 5000s to high-end 48GB cards
- README: added image gen VRAM tier table, expanded inpainting docs
  with practical fix recipes (hands, fingers, eyes, backgrounds),
  mask tips, and denoising strength guidance
- Setup end messages now show image gen capabilities and point to
  setup-image-models.sh instead of manual model install instructions

https://claude.ai/code/session_01PtYTPherSJaxDEVPgF6Nxu
This commit is contained in:
Claude
2026-03-22 21:23:28 +00:00
parent da75be9ae6
commit 8cc849290a
4 changed files with 397 additions and 14 deletions
+52 -6
View File
@@ -69,6 +69,7 @@ sudo systemctl status local-ai
| `invokeai-import-lora.sh` | 85 | Copies a LoRA `.safetensors` file into InvokeAI's Docker model volume. |
| `comfyui-import-lora.sh` | 97 | Copies a LoRA into ComfyUI and prints workflow setup instructions. |
| `comfyui-install-ipadapter.sh` | 185 | Installs IP-Adapter nodes + models into ComfyUI for reference-image workflows (same face, different settings). |
| `setup-image-models.sh` | 200 | **GPU-aware** image model installer. Detects VRAM, offers appropriate SD/SDXL/Flux models, installs into InvokeAI and/or ComfyUI. |
### Which setup script should I use?
@@ -868,17 +869,35 @@ This is where InvokeAI shines for your use case — take an image and riff on it
- **Art style:** "same person, oil painting, renaissance style, dramatic chiaroscuro"
6. Click **Invoke** — iterate by adjusting strength and prompt
### Step 5: Use the Unified Canvas for painting/inpainting
### Step 5: Use the Unified Canvas for inpainting
For more control (paint over specific areas, extend an image):
This is the "fix this specific thing" workflow — brush over a hand, arm, face, background, whatever, and regenerate just that area while keeping everything else untouched.
1. Switch to the **Unified Canvas** tab
2. Upload or paste your image
3. Use the **brush tool** to mask areas you want to change
4. Write a prompt for just the masked area
5. Invoke — only the masked area regenerates
3. Select the **Mask** brush tool (not the paint brush)
4. Brush over **only the area you want to change** — everything else stays locked
5. Write a prompt describing what the masked area should become
6. Set **Denoising Strength** to 0.60.8 (higher = more change)
7. Click **Invoke** — only the masked pixels regenerate
Example: mask just the background → prompt "tropical beach sunset" → keeps the face, replaces the background.
**Common inpainting fixes:**
| Problem | Mask | Prompt |
|---------|------|--------|
| Hand in wrong position | Brush over the arm/hand | "natural hand resting at side, relaxed pose" |
| Extra fingers | Brush over the hand | "normal human hand, five fingers, anatomically correct" |
| Weird eyes | Brush over both eyes | "natural eyes, looking at camera, detailed iris" |
| Bad background | Brush over background only | "clean studio backdrop" or "forest trail, golden hour" |
| Wrong clothing | Brush over the clothing area | "wearing blue denim jacket, casual style" |
| Face swap / aging | Brush over the face | "same person, elderly, wrinkles" or "same person as child" |
**Tips for better inpainting results:**
- **Mask slightly larger** than the problem area — gives the model room to blend edges
- **Use soft brush edges** (lower brush hardness) for more natural blending
- If the result has visible seams, increase your mask area and try again
- **Lower denoising (0.40.5)** for subtle fixes, **higher (0.70.9)** for major changes
- Keep your LoRA active during inpainting — it maintains the trained style/face consistency
### Troubleshooting
@@ -934,3 +953,30 @@ Ollama will **always try to run** any model — it silently offloads layers to C
- `nvidia-smi` shows VRAM maxed out
- CPU usage spikes during generation
- First token takes much longer than usual
### Image Generation Model Tiers
The `setup-image-models.sh` script detects your GPU and offers appropriate models:
| VRAM | Available Models | Default | Notes |
|------|-----------------|---------|-------|
| ≥ 24GB | SD 1.5, SDXL, SDXL Turbo, Flux.1-schnell, Flux.1-dev | SDXL | All models, no constraints |
| 1223GB | SD 1.5, SDXL, SDXL Turbo, Flux.1-schnell | SDXL | Flux-dev too tight |
| 811GB | SD 1.5, SDXL (tight), SDXL Turbo | SD 1.5 | SDXL works at 512px, may be slow |
| 47GB | SD 1.5 (float16) | SD 1.5 | Only SD 1.5 fits |
| < 4GB | none | — | CPU generation not recommended |
**GPU sharing:** Ollama and image generation share the GPU. Ollama auto-unloads models
after its `KEEP_ALIVE` timeout (default 24h), so image gen gets full VRAM when the LLM
is idle. For immediate unload: `docker exec ollama ollama stop <model-name>`
**Multi-GPU scaling:** With dual GPUs (e.g., 2× RTX 5000 = 32GB total), the VRAM
is summed for tier selection. Both InvokeAI and ComfyUI will use all available GPUs.
```bash
# Install image models (auto-detects GPU):
./setup-image-models.sh
# Or auto-install the recommended default:
./setup-image-models.sh --auto
```