Add comfyui-import-lora.sh and LoRA workflow docs
- New script: comfyui-import-lora.sh — copies .safetensors into ComfyUI's Docker volume and prints step-by-step instructions for wiring it into a workflow and exporting to Open WebUI - README: Add "Using LoRAs with Open WebUI" section documenting the workflow-per-style pattern, multi-LoRA management, and architecture compatibility table https://claude.ai/code/session_01PtYTPherSJaxDEVPgF6Nxu
This commit is contained in:
@@ -303,6 +303,64 @@ In any Open WebUI chat, type something like:
|
|||||||
|
|
||||||
The model will detect the image generation request and pass it to ComfyUI.
|
The model will detect the image generation request and pass it to ComfyUI.
|
||||||
|
|
||||||
|
### Using LoRAs with Open WebUI (workflow-per-style)
|
||||||
|
|
||||||
|
LoRAs let you apply trained styles (anime, photorealistic, specific characters, etc.) to image generation. The trick: **bake the LoRA into a ComfyUI workflow, export it, and import into Open WebUI.** After that, you iterate from chat without touching ComfyUI.
|
||||||
|
|
||||||
|
#### Step 1: Import your LoRA file
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./comfyui-import-lora.sh ~/Downloads/my-style-lora.safetensors "Anime Style"
|
||||||
|
```
|
||||||
|
|
||||||
|
Or manually:
|
||||||
|
```bash
|
||||||
|
docker cp ~/Downloads/my-style-lora.safetensors comfyui:/opt/ComfyUI/models/loras/
|
||||||
|
```
|
||||||
|
|
||||||
|
#### Step 2: Build a workflow with the LoRA (one-time)
|
||||||
|
|
||||||
|
1. Open ComfyUI at `http://<ip>:8188`
|
||||||
|
2. Load the default text-to-image workflow
|
||||||
|
3. Add a **Load LoRA** node: right-click → Add Node → loaders → Load LoRA
|
||||||
|
4. Wire it between the checkpoint and the rest of the pipeline:
|
||||||
|
```
|
||||||
|
[Load Checkpoint] → MODEL → [Load LoRA] → MODEL → [KSampler]
|
||||||
|
→ CLIP → → CLIP → [CLIP Text Encode]
|
||||||
|
```
|
||||||
|
5. Select your LoRA file, set `strength_model` and `strength_clip` (start 0.7–0.85)
|
||||||
|
6. Test it — click **Queue Prompt** and verify it works
|
||||||
|
7. Enable **Dev Mode** (gear icon) → click **Save (API Format)**
|
||||||
|
|
||||||
|
#### Step 3: Import into Open WebUI
|
||||||
|
|
||||||
|
1. Open WebUI → **Admin** → **Settings** → **Images**
|
||||||
|
2. Click **Import Workflow** → upload the `workflow_api.json`
|
||||||
|
3. Map the prompt node (usually CLIPTextEncode)
|
||||||
|
4. Save
|
||||||
|
|
||||||
|
Now every "Generate an image of..." in chat uses your LoRA automatically.
|
||||||
|
|
||||||
|
#### Managing multiple LoRA styles
|
||||||
|
|
||||||
|
Each LoRA needs its own exported workflow. Practical approach:
|
||||||
|
|
||||||
|
1. Build a workflow per style (e.g. `anime-lora.json`, `photorealistic-lora.json`)
|
||||||
|
2. Switch between them in **Admin → Settings → Images → Import Workflow**
|
||||||
|
3. The active workflow applies to all image generation requests
|
||||||
|
|
||||||
|
> **Limitation:** You can't switch LoRAs dynamically from chat or adjust LoRA weight per-message. The workflow JSON is fixed. To change styles, swap the workflow in OWUI settings.
|
||||||
|
|
||||||
|
#### LoRA compatibility
|
||||||
|
|
||||||
|
| LoRA trained on | Must use checkpoint |
|
||||||
|
|----------------|-------------------|
|
||||||
|
| SD 1.5 | Any SD 1.5 model (e.g. `v1-5-pruned-emaonly.safetensors`) |
|
||||||
|
| SDXL | Any SDXL model (e.g. `sd_xl_base_1.0.safetensors`) |
|
||||||
|
| Flux | Flux checkpoint |
|
||||||
|
|
||||||
|
Mismatched architectures will produce errors or garbage output.
|
||||||
|
|
||||||
### Setup: AUTOMATIC1111 (alternative)
|
### Setup: AUTOMATIC1111 (alternative)
|
||||||
|
|
||||||
If you prefer AUTOMATIC1111 over ComfyUI:
|
If you prefer AUTOMATIC1111 over ComfyUI:
|
||||||
|
|||||||
Executable
+91
@@ -0,0 +1,91 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# Import a LoRA (.safetensors) into ComfyUI's Docker volume
|
||||||
|
# Usage: ./comfyui-import-lora.sh /path/to/my-lora.safetensors
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
RED='\033[0;31m'; GREEN='\033[0;32m'; YELLOW='\033[1;33m'
|
||||||
|
CYAN='\033[0;36m'; BOLD='\033[1m'; NC='\033[0m'
|
||||||
|
|
||||||
|
if [[ $# -lt 1 ]]; then
|
||||||
|
echo -e "${BOLD}Usage:${NC} $0 <lora-file.safetensors> [display-name]"
|
||||||
|
echo ""
|
||||||
|
echo " Copies a LoRA file into ComfyUI's models/loras/ directory so it"
|
||||||
|
echo " can be used in workflows (including via Open WebUI)."
|
||||||
|
echo ""
|
||||||
|
echo " Examples:"
|
||||||
|
echo " $0 ~/Downloads/my-style-lora.safetensors"
|
||||||
|
echo " $0 ~/Downloads/my-style-lora.safetensors \"Anime Style\""
|
||||||
|
echo ""
|
||||||
|
echo " After importing, create a workflow in ComfyUI that uses this LoRA,"
|
||||||
|
echo " export it as API format, and import into Open WebUI for chat-based"
|
||||||
|
echo " image generation with this LoRA applied automatically."
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
LORA_FILE="$1"
|
||||||
|
DISPLAY_NAME="${2:-$(basename "$LORA_FILE" .safetensors)}"
|
||||||
|
|
||||||
|
# Validate file exists
|
||||||
|
if [[ ! -f "$LORA_FILE" ]]; then
|
||||||
|
echo -e "${RED}Error:${NC} File not found: $LORA_FILE"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Validate file extension
|
||||||
|
if [[ "$LORA_FILE" != *.safetensors && "$LORA_FILE" != *.ckpt && "$LORA_FILE" != *.pt ]]; then
|
||||||
|
echo -e "${YELLOW}Warning:${NC} File doesn't have a typical LoRA extension (.safetensors, .ckpt, .pt)"
|
||||||
|
read -rp "Continue anyway? [y/N] " yn
|
||||||
|
[[ "$yn" != [yY]* ]] && exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Check if ComfyUI container exists
|
||||||
|
if ! docker ps -a --format '{{.Names}}' 2>/dev/null | grep -q '^comfyui$'; then
|
||||||
|
echo -e "${RED}Error:${NC} ComfyUI container not found. Is the AI stack running?"
|
||||||
|
echo " Try: docker compose up -d comfyui"
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Check if container is running
|
||||||
|
if ! docker ps --format '{{.Names}}' 2>/dev/null | grep -q '^comfyui$'; then
|
||||||
|
echo -e "${YELLOW}ComfyUI container is stopped. Starting it...${NC}"
|
||||||
|
docker start comfyui
|
||||||
|
sleep 3
|
||||||
|
fi
|
||||||
|
|
||||||
|
FILENAME=$(basename "$LORA_FILE")
|
||||||
|
|
||||||
|
echo -e "${CYAN}[..]${NC} Copying ${BOLD}$FILENAME${NC} into ComfyUI..."
|
||||||
|
|
||||||
|
# Copy the LoRA file into the container's loras directory
|
||||||
|
docker exec comfyui mkdir -p /opt/ComfyUI/models/loras
|
||||||
|
docker cp "$LORA_FILE" "comfyui:/opt/ComfyUI/models/loras/$FILENAME"
|
||||||
|
|
||||||
|
echo -e "${GREEN}[OK]${NC} LoRA '${DISPLAY_NAME}' copied to ComfyUI!"
|
||||||
|
echo ""
|
||||||
|
echo -e "${BOLD}Next: Create a workflow that uses this LoRA${NC}"
|
||||||
|
echo ""
|
||||||
|
echo " 1. Open ComfyUI at http://localhost:8188"
|
||||||
|
echo " 2. Load or create a text-to-image workflow"
|
||||||
|
echo " 3. Add a ${BOLD}LoRA Loader${NC} node:"
|
||||||
|
echo " Right-click canvas → Add Node → loaders → Load LoRA"
|
||||||
|
echo " 4. Wire it between the ${BOLD}checkpoint loader${NC} and the ${BOLD}CLIP/sampler${NC}:"
|
||||||
|
echo ""
|
||||||
|
echo " [Load Checkpoint] → MODEL → [Load LoRA] → MODEL → [KSampler]"
|
||||||
|
echo " → CLIP → → CLIP → [CLIP Text Encode]"
|
||||||
|
echo ""
|
||||||
|
echo " 5. In the LoRA Loader node, select ${BOLD}${FILENAME}${NC}"
|
||||||
|
echo " 6. Set ${BOLD}strength_model${NC} and ${BOLD}strength_clip${NC} (start with 0.7–0.85)"
|
||||||
|
echo ""
|
||||||
|
echo -e "${BOLD}To use from Open WebUI chat (no more ComfyUI interaction needed):${NC}"
|
||||||
|
echo ""
|
||||||
|
echo " 7. Click the ${BOLD}gear icon${NC} → enable ${BOLD}Dev Mode${NC}"
|
||||||
|
echo " 8. Click ${BOLD}Save (API Format)${NC} → saves workflow_api.json"
|
||||||
|
echo " 9. In Open WebUI → Admin → Settings → Images → ${BOLD}Import Workflow${NC}"
|
||||||
|
echo " 10. Upload the workflow_api.json and map the prompt node"
|
||||||
|
echo " 11. Now just chat: \"Generate an image of a forest in ${DISPLAY_NAME} style\""
|
||||||
|
echo ""
|
||||||
|
echo -e "${YELLOW}Tip:${NC} The LoRA must match your base model architecture."
|
||||||
|
echo " SD 1.5 LoRA → SD 1.5 checkpoint. SDXL LoRA → SDXL checkpoint."
|
||||||
|
echo ""
|
||||||
|
echo -e "${YELLOW}Tip:${NC} Export multiple workflows (one per LoRA/style) and switch"
|
||||||
|
echo " between them in Open WebUI's image settings as needed."
|
||||||
Reference in New Issue
Block a user