diff --git a/README.md b/README.md index fd09543..166192c 100644 --- a/README.md +++ b/README.md @@ -70,6 +70,8 @@ sudo systemctl status local-ai | `comfyui-import-lora.sh` | 97 | Copies a LoRA into ComfyUI and prints workflow setup instructions. | | `comfyui-install-ipadapter.sh` | 185 | Installs IP-Adapter nodes + models into ComfyUI for reference-image workflows (same face, different settings). | | `setup-image-models.sh` | 200 | **GPU-aware** image model installer. Detects VRAM, offers appropriate SD/SDXL/Flux models, installs into InvokeAI and/or ComfyUI. | +| `comfyui-workflow-default.json` | — | Pre-made ComfyUI workflow for SD 1.5 (512x512). Upload directly into Open WebUI. | +| `comfyui-workflow-sdxl.json` | — | Pre-made ComfyUI workflow for SDXL (1024x1024). Upload directly into Open WebUI. | ### Which setup script should I use? @@ -225,12 +227,27 @@ This is how "generate an image of..." works from chat. 3. **Engine:** Select `ComfyUI` 4. **ComfyUI Base URL:** `http://comfyui:8188` - (These should already be filled in from the env vars set during setup) -5. Click the **checkmark/verify** button — should show "Connected" -6. **Import Workflow:** Click and upload your `workflow_api.json` - - If you don't have one yet, see [ComfyUI workflow setup](#setup-comfyui--open-webui-recommended) -7. **Map nodes:** After importing, map the **prompt node** to the CLIPTextEncode - node (or whichever node takes the text prompt) -8. Click **Save** +5. Click the **refresh/verify** button (circular arrow icon) — should show "Connected" + - If you see an error about `localhost:1111`, that's a stale config. Just clear the + URL field and re-enter `http://comfyui:8188`, then save. +6. **ComfyUI API Key:** Leave blank (no auth needed on local network) + - If there's a default `sk-1234`, clear it — ComfyUI doesn't use API keys +7. **ComfyUI Workflow:** Click **Upload** and select one of the pre-made workflows: + - `comfyui-workflow-default.json` — SD 1.5 (512x512, works on 4GB+ VRAM) + - `comfyui-workflow-sdxl.json` — SDXL (1024x1024, needs 8GB+ VRAM) + - These files are in your `~/docker/ai-stack/` directory +8. **ComfyUI Workflow Nodes** — after uploading, fill in these node IDs: + + | Field | Node ID | Why | + |-------|---------|-----| + | **Prompt** → text | `6` | Node 6 is the positive CLIPTextEncode — this is where your chat prompt goes | + | **Model** → ckpt_name | `4` | Node 4 is the CheckpointLoaderSimple — picks which SD model to use | + | **Width** → width | `5` | Node 5 is the EmptyLatentImage — controls output width | + | **Height** → height | `5` | Same node 5 — controls output height | + | **Steps** → steps | `3` | Node 3 is the KSampler — controls generation steps | + | **Seed** → seed | `3` | Same node 3 — controls randomness seed | + +9. Click **Save** > **No workflow yet?** You need to first open ComfyUI at `http://:8188`, > build or load a workflow, enable Dev Mode (gear icon), and click @@ -409,35 +426,50 @@ Open WebUI natively supports these image generation engines: If you selected ComfyUI during setup, the environment variables are already configured. You just need to install a model and set up a workflow. -#### Step 1: Install a Stable Diffusion model in ComfyUI +#### Step 1: Install a Stable Diffusion model ```bash -# Open ComfyUI at http://:8188 -# Use the built-in Model Manager to download a model, or manually: -docker exec comfyui bash -c "cd /opt/ComfyUI/models/checkpoints && \ - wget -q 'https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0/resolve/main/sd_xl_base_1.0.safetensors'" -``` +# GPU-aware — detects your card and installs the right model: +./setup-image-models.sh -Or download any `.safetensors` checkpoint and copy it in: -```bash +# Or manually: docker cp ~/Downloads/my-model.safetensors comfyui:/opt/ComfyUI/models/checkpoints/ ``` -#### Step 2: Create and export a workflow +#### Step 2: Upload a workflow into Open WebUI -1. Open ComfyUI at `http://:8188` -2. Build or load a workflow (the default text-to-image workflow works) -3. Click the **gear icon** → enable **Dev Mode** -4. Click **Save (API Format)** — this downloads `workflow_api.json` +Pre-made workflows are included — no need to build one in ComfyUI: -#### Step 3: Configure Open WebUI +| Workflow file | Model | Resolution | VRAM | +|--------------|-------|-----------|------| +| `comfyui-workflow-default.json` | SD 1.5 | 512x512 | 4GB+ | +| `comfyui-workflow-sdxl.json` | SDXL | 1024x1024 | 8GB+ | -1. Open WebUI → **Admin** → **Settings** → **Images** +1. Go to **Admin** → **Settings** → **Images** 2. Set **Engine** to `ComfyUI` -3. Set **URL** to `http://comfyui:8188` (container networking, already set via env vars) -4. Click **Import Workflow** and upload your `workflow_api.json` -5. Map the **prompt node** (usually the KSampler or CLIPTextEncode node) -6. Save settings +3. Set **ComfyUI Base URL** to `http://comfyui:8188` +4. Click the **refresh icon** — should show connected +5. Clear the **ComfyUI API Key** field (ComfyUI doesn't use API keys) +6. Click **Upload** next to **ComfyUI Workflow** +7. Select `comfyui-workflow-default.json` (or `sdxl` version) +8. Fill in the **ComfyUI Workflow Nodes** — these tell Open WebUI which + node in the workflow handles each parameter: + + | Field | Node ID | + |-------|---------| + | **Prompt** → text | `6` | + | **Model** → ckpt_name | `4` | + | **Width** → width | `5` | + | **Height** → height | `5` | + | **Steps** → steps | `3` | + | **Seed** → seed | `3` | + +9. Click **Save** + +> **Custom workflows:** If you build your own workflow in ComfyUI, enable +> Dev Mode (gear icon) → click **Save (API Format)** → upload that JSON file +> instead. The node IDs will be different — check each node's ID number +> in ComfyUI and enter them in the mapping above. #### Step 4: Generate images in chat diff --git a/comfyui-workflow-default.json b/comfyui-workflow-default.json new file mode 100644 index 0000000..1ac3d08 --- /dev/null +++ b/comfyui-workflow-default.json @@ -0,0 +1,59 @@ +{ + "3": { + "class_type": "KSampler", + "inputs": { + "cfg": 7.5, + "denoise": 1, + "latent_image": ["5", 0], + "model": ["4", 0], + "negative": ["7", 0], + "positive": ["6", 0], + "sampler_name": "euler_ancestral", + "scheduler": "normal", + "seed": 42, + "steps": 30 + } + }, + "4": { + "class_type": "CheckpointLoaderSimple", + "inputs": { + "ckpt_name": "v1-5-pruned-emaonly.safetensors" + } + }, + "5": { + "class_type": "EmptyLatentImage", + "inputs": { + "batch_size": 1, + "height": 512, + "width": 512 + } + }, + "6": { + "class_type": "CLIPTextEncode", + "inputs": { + "clip": ["4", 1], + "text": "a beautiful landscape painting" + } + }, + "7": { + "class_type": "CLIPTextEncode", + "inputs": { + "clip": ["4", 1], + "text": "bad quality, blurry, ugly, deformed" + } + }, + "8": { + "class_type": "VAEDecode", + "inputs": { + "samples": ["3", 0], + "vae": ["4", 2] + } + }, + "9": { + "class_type": "SaveImage", + "inputs": { + "filename_prefix": "ComfyUI", + "images": ["8", 0] + } + } +} diff --git a/comfyui-workflow-sdxl.json b/comfyui-workflow-sdxl.json new file mode 100644 index 0000000..317c0fc --- /dev/null +++ b/comfyui-workflow-sdxl.json @@ -0,0 +1,59 @@ +{ + "3": { + "class_type": "KSampler", + "inputs": { + "cfg": 7, + "denoise": 1, + "latent_image": ["5", 0], + "model": ["4", 0], + "negative": ["7", 0], + "positive": ["6", 0], + "sampler_name": "euler_ancestral", + "scheduler": "normal", + "seed": 42, + "steps": 30 + } + }, + "4": { + "class_type": "CheckpointLoaderSimple", + "inputs": { + "ckpt_name": "sd_xl_base_1.0.safetensors" + } + }, + "5": { + "class_type": "EmptyLatentImage", + "inputs": { + "batch_size": 1, + "height": 1024, + "width": 1024 + } + }, + "6": { + "class_type": "CLIPTextEncode", + "inputs": { + "clip": ["4", 1], + "text": "a beautiful landscape painting" + } + }, + "7": { + "class_type": "CLIPTextEncode", + "inputs": { + "clip": ["4", 1], + "text": "bad quality, blurry, ugly, deformed" + } + }, + "8": { + "class_type": "VAEDecode", + "inputs": { + "samples": ["3", 0], + "vae": ["4", 2] + } + }, + "9": { + "class_type": "SaveImage", + "inputs": { + "filename_prefix": "ComfyUI", + "images": ["8", 0] + } + } +}