Add comfyui-import-lora.sh and LoRA workflow docs
- New script: comfyui-import-lora.sh — copies .safetensors into ComfyUI's Docker volume and prints step-by-step instructions for wiring it into a workflow and exporting to Open WebUI - README: Add "Using LoRAs with Open WebUI" section documenting the workflow-per-style pattern, multi-LoRA management, and architecture compatibility table https://claude.ai/code/session_01PtYTPherSJaxDEVPgF6Nxu
This commit is contained in:
Claude
@@ -303,6 +303,64 @@ In any Open WebUI chat, type something like:
|
||||
|
||||
The model will detect the image generation request and pass it to ComfyUI.
|
||||
|
||||
### Using LoRAs with Open WebUI (workflow-per-style)
|
||||
|
||||
LoRAs let you apply trained styles (anime, photorealistic, specific characters, etc.) to image generation. The trick: **bake the LoRA into a ComfyUI workflow, export it, and import into Open WebUI.** After that, you iterate from chat without touching ComfyUI.
|
||||
|
||||
#### Step 1: Import your LoRA file
|
||||
|
||||
```bash
|
||||
./comfyui-import-lora.sh ~/Downloads/my-style-lora.safetensors "Anime Style"
|
||||
```
|
||||
|
||||
Or manually:
|
||||
```bash
|
||||
docker cp ~/Downloads/my-style-lora.safetensors comfyui:/opt/ComfyUI/models/loras/
|
||||
```
|
||||
|
||||
#### Step 2: Build a workflow with the LoRA (one-time)
|
||||
|
||||
1. Open ComfyUI at `http://<ip>:8188`
|
||||
2. Load the default text-to-image workflow
|
||||
3. Add a **Load LoRA** node: right-click → Add Node → loaders → Load LoRA
|
||||
4. Wire it between the checkpoint and the rest of the pipeline:
|
||||
```
|
||||
[Load Checkpoint] → MODEL → [Load LoRA] → MODEL → [KSampler]
|
||||
→ CLIP → → CLIP → [CLIP Text Encode]
|
||||
```
|
||||
5. Select your LoRA file, set `strength_model` and `strength_clip` (start 0.7–0.85)
|
||||
6. Test it — click **Queue Prompt** and verify it works
|
||||
7. Enable **Dev Mode** (gear icon) → click **Save (API Format)**
|
||||
|
||||
#### Step 3: Import into Open WebUI
|
||||
|
||||
1. Open WebUI → **Admin** → **Settings** → **Images**
|
||||
2. Click **Import Workflow** → upload the `workflow_api.json`
|
||||
3. Map the prompt node (usually CLIPTextEncode)
|
||||
4. Save
|
||||
|
||||
Now every "Generate an image of..." in chat uses your LoRA automatically.
|
||||
|
||||
#### Managing multiple LoRA styles
|
||||
|
||||
Each LoRA needs its own exported workflow. Practical approach:
|
||||
|
||||
1. Build a workflow per style (e.g. `anime-lora.json`, `photorealistic-lora.json`)
|
||||
2. Switch between them in **Admin → Settings → Images → Import Workflow**
|
||||
3. The active workflow applies to all image generation requests
|
||||
|
||||
> **Limitation:** You can't switch LoRAs dynamically from chat or adjust LoRA weight per-message. The workflow JSON is fixed. To change styles, swap the workflow in OWUI settings.
|
||||
|
||||
#### LoRA compatibility
|
||||
|
||||
| LoRA trained on | Must use checkpoint |
|
||||
|----------------|-------------------|
|
||||
| SD 1.5 | Any SD 1.5 model (e.g. `v1-5-pruned-emaonly.safetensors`) |
|
||||
| SDXL | Any SDXL model (e.g. `sd_xl_base_1.0.safetensors`) |
|
||||
| Flux | Flux checkpoint |
|
||||
|
||||
Mismatched architectures will produce errors or garbage output.
|
||||
|
||||
### Setup: AUTOMATIC1111 (alternative)
|
||||
|
||||
If you prefer AUTOMATIC1111 over ComfyUI:
|
||||
|
||||
Executable
+91
@@ -0,0 +1,91 @@
|
||||
#!/usr/bin/env bash
|
||||
# Import a LoRA (.safetensors) into ComfyUI's Docker volume
|
||||
# Usage: ./comfyui-import-lora.sh /path/to/my-lora.safetensors
|
||||
set -euo pipefail
|
||||
|
||||
RED='\033[0;31m'; GREEN='\033[0;32m'; YELLOW='\033[1;33m'
|
||||
CYAN='\033[0;36m'; BOLD='\033[1m'; NC='\033[0m'
|
||||
|
||||
if [[ $# -lt 1 ]]; then
|
||||
echo -e "${BOLD}Usage:${NC} $0 <lora-file.safetensors> [display-name]"
|
||||
echo ""
|
||||
echo " Copies a LoRA file into ComfyUI's models/loras/ directory so it"
|
||||
echo " can be used in workflows (including via Open WebUI)."
|
||||
echo ""
|
||||
echo " Examples:"
|
||||
echo " $0 ~/Downloads/my-style-lora.safetensors"
|
||||
echo " $0 ~/Downloads/my-style-lora.safetensors \"Anime Style\""
|
||||
echo ""
|
||||
echo " After importing, create a workflow in ComfyUI that uses this LoRA,"
|
||||
echo " export it as API format, and import into Open WebUI for chat-based"
|
||||
echo " image generation with this LoRA applied automatically."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
LORA_FILE="$1"
|
||||
DISPLAY_NAME="${2:-$(basename "$LORA_FILE" .safetensors)}"
|
||||
|
||||
# Validate file exists
|
||||
if [[ ! -f "$LORA_FILE" ]]; then
|
||||
echo -e "${RED}Error:${NC} File not found: $LORA_FILE"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Validate file extension
|
||||
if [[ "$LORA_FILE" != *.safetensors && "$LORA_FILE" != *.ckpt && "$LORA_FILE" != *.pt ]]; then
|
||||
echo -e "${YELLOW}Warning:${NC} File doesn't have a typical LoRA extension (.safetensors, .ckpt, .pt)"
|
||||
read -rp "Continue anyway? [y/N] " yn
|
||||
[[ "$yn" != [yY]* ]] && exit 1
|
||||
fi
|
||||
|
||||
# Check if ComfyUI container exists
|
||||
if ! docker ps -a --format '{{.Names}}' 2>/dev/null | grep -q '^comfyui$'; then
|
||||
echo -e "${RED}Error:${NC} ComfyUI container not found. Is the AI stack running?"
|
||||
echo " Try: docker compose up -d comfyui"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
# Check if container is running
|
||||
if ! docker ps --format '{{.Names}}' 2>/dev/null | grep -q '^comfyui$'; then
|
||||
echo -e "${YELLOW}ComfyUI container is stopped. Starting it...${NC}"
|
||||
docker start comfyui
|
||||
sleep 3
|
||||
fi
|
||||
|
||||
FILENAME=$(basename "$LORA_FILE")
|
||||
|
||||
echo -e "${CYAN}[..]${NC} Copying ${BOLD}$FILENAME${NC} into ComfyUI..."
|
||||
|
||||
# Copy the LoRA file into the container's loras directory
|
||||
docker exec comfyui mkdir -p /opt/ComfyUI/models/loras
|
||||
docker cp "$LORA_FILE" "comfyui:/opt/ComfyUI/models/loras/$FILENAME"
|
||||
|
||||
echo -e "${GREEN}[OK]${NC} LoRA '${DISPLAY_NAME}' copied to ComfyUI!"
|
||||
echo ""
|
||||
echo -e "${BOLD}Next: Create a workflow that uses this LoRA${NC}"
|
||||
echo ""
|
||||
echo " 1. Open ComfyUI at http://localhost:8188"
|
||||
echo " 2. Load or create a text-to-image workflow"
|
||||
echo " 3. Add a ${BOLD}LoRA Loader${NC} node:"
|
||||
echo " Right-click canvas → Add Node → loaders → Load LoRA"
|
||||
echo " 4. Wire it between the ${BOLD}checkpoint loader${NC} and the ${BOLD}CLIP/sampler${NC}:"
|
||||
echo ""
|
||||
echo " [Load Checkpoint] → MODEL → [Load LoRA] → MODEL → [KSampler]"
|
||||
echo " → CLIP → → CLIP → [CLIP Text Encode]"
|
||||
echo ""
|
||||
echo " 5. In the LoRA Loader node, select ${BOLD}${FILENAME}${NC}"
|
||||
echo " 6. Set ${BOLD}strength_model${NC} and ${BOLD}strength_clip${NC} (start with 0.7–0.85)"
|
||||
echo ""
|
||||
echo -e "${BOLD}To use from Open WebUI chat (no more ComfyUI interaction needed):${NC}"
|
||||
echo ""
|
||||
echo " 7. Click the ${BOLD}gear icon${NC} → enable ${BOLD}Dev Mode${NC}"
|
||||
echo " 8. Click ${BOLD}Save (API Format)${NC} → saves workflow_api.json"
|
||||
echo " 9. In Open WebUI → Admin → Settings → Images → ${BOLD}Import Workflow${NC}"
|
||||
echo " 10. Upload the workflow_api.json and map the prompt node"
|
||||
echo " 11. Now just chat: \"Generate an image of a forest in ${DISPLAY_NAME} style\""
|
||||
echo ""
|
||||
echo -e "${YELLOW}Tip:${NC} The LoRA must match your base model architecture."
|
||||
echo " SD 1.5 LoRA → SD 1.5 checkpoint. SDXL LoRA → SDXL checkpoint."
|
||||
echo ""
|
||||
echo -e "${YELLOW}Tip:${NC} Export multiple workflows (one per LoRA/style) and switch"
|
||||
echo " between them in Open WebUI's image settings as needed."
|
||||
Reference in New Issue
Block a user