Add GPU-aware image model detection and setup-image-models.sh
- Both setup scripts now detect VRAM and determine which image gen models the GPU can run (SD 1.5 at 4GB, SDXL at 8GB, Flux at 12-20GB) - New setup-image-models.sh: interactive script that detects GPU, shows available models with VRAM requirements, and installs into InvokeAI and/or ComfyUI. Supports --auto for unattended install. - Scales from 4GB cards through dual RTX 5000s to high-end 48GB cards - README: added image gen VRAM tier table, expanded inpainting docs with practical fix recipes (hands, fingers, eyes, backgrounds), mask tips, and denoising strength guidance - Setup end messages now show image gen capabilities and point to setup-image-models.sh instead of manual model install instructions https://claude.ai/code/session_01PtYTPherSJaxDEVPgF6Nxu
This commit is contained in:
@@ -69,6 +69,7 @@ sudo systemctl status local-ai
|
||||
| `invokeai-import-lora.sh` | 85 | Copies a LoRA `.safetensors` file into InvokeAI's Docker model volume. |
|
||||
| `comfyui-import-lora.sh` | 97 | Copies a LoRA into ComfyUI and prints workflow setup instructions. |
|
||||
| `comfyui-install-ipadapter.sh` | 185 | Installs IP-Adapter nodes + models into ComfyUI for reference-image workflows (same face, different settings). |
|
||||
| `setup-image-models.sh` | 200 | **GPU-aware** image model installer. Detects VRAM, offers appropriate SD/SDXL/Flux models, installs into InvokeAI and/or ComfyUI. |
|
||||
|
||||
### Which setup script should I use?
|
||||
|
||||
@@ -868,17 +869,35 @@ This is where InvokeAI shines for your use case — take an image and riff on it
|
||||
- **Art style:** "same person, oil painting, renaissance style, dramatic chiaroscuro"
|
||||
6. Click **Invoke** — iterate by adjusting strength and prompt
|
||||
|
||||
### Step 5: Use the Unified Canvas for painting/inpainting
|
||||
### Step 5: Use the Unified Canvas for inpainting
|
||||
|
||||
For more control (paint over specific areas, extend an image):
|
||||
This is the "fix this specific thing" workflow — brush over a hand, arm, face, background, whatever, and regenerate just that area while keeping everything else untouched.
|
||||
|
||||
1. Switch to the **Unified Canvas** tab
|
||||
2. Upload or paste your image
|
||||
3. Use the **brush tool** to mask areas you want to change
|
||||
4. Write a prompt for just the masked area
|
||||
5. Invoke — only the masked area regenerates
|
||||
3. Select the **Mask** brush tool (not the paint brush)
|
||||
4. Brush over **only the area you want to change** — everything else stays locked
|
||||
5. Write a prompt describing what the masked area should become
|
||||
6. Set **Denoising Strength** to 0.6–0.8 (higher = more change)
|
||||
7. Click **Invoke** — only the masked pixels regenerate
|
||||
|
||||
Example: mask just the background → prompt "tropical beach sunset" → keeps the face, replaces the background.
|
||||
**Common inpainting fixes:**
|
||||
|
||||
| Problem | Mask | Prompt |
|
||||
|---------|------|--------|
|
||||
| Hand in wrong position | Brush over the arm/hand | "natural hand resting at side, relaxed pose" |
|
||||
| Extra fingers | Brush over the hand | "normal human hand, five fingers, anatomically correct" |
|
||||
| Weird eyes | Brush over both eyes | "natural eyes, looking at camera, detailed iris" |
|
||||
| Bad background | Brush over background only | "clean studio backdrop" or "forest trail, golden hour" |
|
||||
| Wrong clothing | Brush over the clothing area | "wearing blue denim jacket, casual style" |
|
||||
| Face swap / aging | Brush over the face | "same person, elderly, wrinkles" or "same person as child" |
|
||||
|
||||
**Tips for better inpainting results:**
|
||||
- **Mask slightly larger** than the problem area — gives the model room to blend edges
|
||||
- **Use soft brush edges** (lower brush hardness) for more natural blending
|
||||
- If the result has visible seams, increase your mask area and try again
|
||||
- **Lower denoising (0.4–0.5)** for subtle fixes, **higher (0.7–0.9)** for major changes
|
||||
- Keep your LoRA active during inpainting — it maintains the trained style/face consistency
|
||||
|
||||
### Troubleshooting
|
||||
|
||||
@@ -934,3 +953,30 @@ Ollama will **always try to run** any model — it silently offloads layers to C
|
||||
- `nvidia-smi` shows VRAM maxed out
|
||||
- CPU usage spikes during generation
|
||||
- First token takes much longer than usual
|
||||
|
||||
### Image Generation Model Tiers
|
||||
|
||||
The `setup-image-models.sh` script detects your GPU and offers appropriate models:
|
||||
|
||||
| VRAM | Available Models | Default | Notes |
|
||||
|------|-----------------|---------|-------|
|
||||
| ≥ 24GB | SD 1.5, SDXL, SDXL Turbo, Flux.1-schnell, Flux.1-dev | SDXL | All models, no constraints |
|
||||
| 12–23GB | SD 1.5, SDXL, SDXL Turbo, Flux.1-schnell | SDXL | Flux-dev too tight |
|
||||
| 8–11GB | SD 1.5, SDXL (tight), SDXL Turbo | SD 1.5 | SDXL works at 512px, may be slow |
|
||||
| 4–7GB | SD 1.5 (float16) | SD 1.5 | Only SD 1.5 fits |
|
||||
| < 4GB | none | — | CPU generation not recommended |
|
||||
|
||||
**GPU sharing:** Ollama and image generation share the GPU. Ollama auto-unloads models
|
||||
after its `KEEP_ALIVE` timeout (default 24h), so image gen gets full VRAM when the LLM
|
||||
is idle. For immediate unload: `docker exec ollama ollama stop <model-name>`
|
||||
|
||||
**Multi-GPU scaling:** With dual GPUs (e.g., 2× RTX 5000 = 32GB total), the VRAM
|
||||
is summed for tier selection. Both InvokeAI and ComfyUI will use all available GPUs.
|
||||
|
||||
```bash
|
||||
# Install image models (auto-detects GPU):
|
||||
./setup-image-models.sh
|
||||
|
||||
# Or auto-install the recommended default:
|
||||
./setup-image-models.sh --auto
|
||||
```
|
||||
|
||||
Reference in New Issue
Block a user