Fix build cache, DNS/model download, and AI Edit error handling
Build / pip layer fixes:
- Add BUILDID ARG to Dockerfile.gpu; pass from docker-compose.gpu.yml build args
so pip layers can be force-busted without --no-cache:
BUILDID=$(date +%s) docker compose -f docker-compose.gpu.yml up --build
Model download (DNS-blocked environments):
- Change HF model cache from named volume to ./data/hf_cache bind mount
so models can be pre-downloaded on the host (no rebuild needed)
- Remove now-unused hf_model_cache named volume
- README: add iptables fix + huggingface-cli offline download instructions
Error handling improvements:
- ai_edit_region: catch ConnectError/Errno-3 → return 503 with exact fix commands
- _require_remote: give actionable message when local_gpu provider fails to load
- _build_provider: catch AttributeError (torch.xpu from wrong diffusers) not just ImportError
- local_diffusion.py: fix docstring to reflect <0.29.0 pin
https://claude.ai/code/session_01WVDg7amsy1TTtxvpku7bcM
This commit is contained in:
+5
-2
@@ -42,10 +42,13 @@ RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
# Install Python dependencies — base + GPU extras
|
||||
# BUILDID forces pip layers to re-run when you need fresh packages without a full --no-cache:
|
||||
# BUILDID=$(date +%s) docker compose -f docker-compose.gpu.yml up --build
|
||||
ARG BUILDID=1
|
||||
COPY backend/requirements.txt .
|
||||
COPY backend/requirements.gpu.txt .
|
||||
RUN pip install --no-cache-dir -r requirements.txt
|
||||
RUN pip install --no-cache-dir -r requirements.gpu.txt
|
||||
RUN echo "BUILDID=$BUILDID" && pip install --no-cache-dir -r requirements.txt
|
||||
RUN echo "BUILDID=$BUILDID" && pip install --no-cache-dir -r requirements.gpu.txt
|
||||
|
||||
# Smoke-test rembg (model downloads on first use)
|
||||
RUN python -c "from rembg import remove; print('rembg OK')" \
|
||||
|
||||
@@ -57,6 +57,12 @@ docker compose -f docker-compose.gpu.yml up -d --build
|
||||
docker compose up -d --build
|
||||
```
|
||||
|
||||
If pip packages seem stale after a pull (e.g., wrong diffusers version), force a pip layer rebuild without re-downloading the entire PyTorch base image:
|
||||
|
||||
```bash
|
||||
BUILDID=$(date +%s) docker compose -f docker-compose.gpu.yml up -d --build
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## AI Providers
|
||||
@@ -202,6 +208,33 @@ docker compose -f docker-compose.gpu.yml logs | grep -i sam
|
||||
|
||||
If Docker created `./data/` as root and you can't write there without `sudo`, you can also use root's curl as above — the container reads the file regardless of owner.
|
||||
|
||||
**AI Edit returns "model files not yet downloaded" or "Errno -3 / DNS" error**
|
||||
|
||||
The container's DNS is blocked (common on corporate networks or custom iptables rules), so it can't download SDXL models from HuggingFace. Two options:
|
||||
|
||||
*Option A — fix Docker DNS (recommended, one command):*
|
||||
```bash
|
||||
sudo iptables -I DOCKER-USER -p udp --dport 53 -j ACCEPT
|
||||
docker compose -f docker-compose.gpu.yml restart
|
||||
```
|
||||
|
||||
*Option B — pre-download models on the host (if iptables fix isn't possible):*
|
||||
```bash
|
||||
pip install huggingface-hub
|
||||
|
||||
# Download the inpainting model (~6.5 GB, needed for AI Edit):
|
||||
huggingface-cli download diffusers/stable-diffusion-xl-1.0-inpainting-0.1 \
|
||||
--cache-dir ./data/hf_cache \
|
||||
--exclude "*.msgpack" "flax_*" "tf_*"
|
||||
|
||||
# Download the text-to-image model (~6.5 GB, needed for Text → Image):
|
||||
huggingface-cli download stabilityai/stable-diffusion-xl-base-1.0 \
|
||||
--cache-dir ./data/hf_cache \
|
||||
--exclude "*.msgpack" "flax_*" "tf_*"
|
||||
```
|
||||
|
||||
The models land in `./data/hf_cache/` which is bind-mounted into the container — no rebuild needed. Restart the container and the first AI Edit request loads from local disk.
|
||||
|
||||
**Out of VRAM during generation**
|
||||
- Reduce `LOCAL_GPU_MAX_PIPELINES=1` in `.env` (default 2)
|
||||
- Or override to a smaller model: `HF_MODEL_TXT2IMG=runwayml/stable-diffusion-v1-5`
|
||||
|
||||
@@ -80,8 +80,19 @@ def _encode(data: bytes) -> str:
|
||||
|
||||
def _require_remote(operation: str = None):
|
||||
from app.services.remote_provider import get_remote_provider
|
||||
from app.config import settings
|
||||
provider = get_remote_provider(operation)
|
||||
if provider is None:
|
||||
if (settings.ai_provider or "").lower() == "local_gpu":
|
||||
raise HTTPException(
|
||||
status_code=503,
|
||||
detail=(
|
||||
"local_gpu provider failed to load — diffusers may be incompatible with "
|
||||
"the installed PyTorch version. Check container logs for details. "
|
||||
"If you see 'torch has no attribute xpu', rebuild the container from the "
|
||||
"correct branch so the pinned diffusers<0.29.0 is installed."
|
||||
)
|
||||
)
|
||||
op_hint = f"AI_PROVIDER_{operation.upper()} or " if operation else ""
|
||||
raise HTTPException(
|
||||
status_code=503,
|
||||
@@ -502,12 +513,28 @@ async def ai_edit_region(req: AiEditRegionRequest):
|
||||
Works with local_gpu, InvokeAI, ComfyUI, or OpenAI.
|
||||
"""
|
||||
provider = _require_remote("inpaint")
|
||||
result_bytes = await provider.inpaint(
|
||||
_decode(req.image),
|
||||
_decode(req.mask),
|
||||
req.instruction,
|
||||
{"negative_prompt": req.negative_prompt, "steps": req.steps, "cfg_scale": req.cfg_scale},
|
||||
)
|
||||
try:
|
||||
result_bytes = await provider.inpaint(
|
||||
_decode(req.image),
|
||||
_decode(req.mask),
|
||||
req.instruction,
|
||||
{"negative_prompt": req.negative_prompt, "steps": req.steps, "cfg_scale": req.cfg_scale},
|
||||
)
|
||||
except Exception as exc:
|
||||
import traceback; traceback.print_exc()
|
||||
msg = str(exc)
|
||||
if "Errno -3" in msg or "Name or service not known" in msg or "ConnectError" in msg:
|
||||
raise HTTPException(
|
||||
status_code=503,
|
||||
detail=(
|
||||
"AI model files not yet downloaded — container DNS appears to be blocked. "
|
||||
"Fix: sudo iptables -I DOCKER-USER -p udp --dport 53 -j ACCEPT on the host, "
|
||||
"or pre-download the model: pip install huggingface-hub && "
|
||||
"huggingface-cli download diffusers/stable-diffusion-xl-1.0-inpainting-0.1 "
|
||||
"--cache-dir ./data/hf_cache"
|
||||
)
|
||||
)
|
||||
raise HTTPException(status_code=500, detail=msg)
|
||||
return {"result": _encode(result_bytes)}
|
||||
|
||||
|
||||
|
||||
@@ -11,8 +11,8 @@ Supported model families:
|
||||
sd2x → StableDiffusion2*Pipeline (SD 2.x)
|
||||
sd15 → StableDiffusionPipeline (SD 1.5)
|
||||
|
||||
Requires: diffusers>=0.29.0, transformers, accelerate, safetensors
|
||||
(all in requirements.gpu.txt)
|
||||
Requires: diffusers>=0.28.0,<0.29.0, transformers, accelerate, safetensors
|
||||
(all in requirements.gpu.txt — pinned <0.29.0 for PyTorch 2.1.x compatibility)
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
|
||||
@@ -432,7 +432,8 @@ def _build_provider(name: str) -> Optional[RemoteAIProvider]:
|
||||
try:
|
||||
from app.services.local_diffusion import get_local_diffusion_provider
|
||||
return get_local_diffusion_provider(max_pipelines=settings.local_gpu_max_pipelines)
|
||||
except ImportError:
|
||||
except (ImportError, AttributeError) as exc:
|
||||
print(f"[local_gpu] Cannot load diffusion provider: {exc}")
|
||||
return None
|
||||
|
||||
return None
|
||||
|
||||
+11
-7
@@ -62,14 +62,23 @@ services:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile.gpu
|
||||
args:
|
||||
# Increment BUILDID to force pip layers to re-run without full --no-cache:
|
||||
# BUILDID=$(date +%s) docker compose -f docker-compose.gpu.yml up --build
|
||||
BUILDID: ${BUILDID:-1}
|
||||
container_name: editmaskwithai-gpu
|
||||
ports:
|
||||
- "${PORT:-3080}:8000"
|
||||
volumes:
|
||||
# Persistent project data
|
||||
- ./data:/app/data
|
||||
# HuggingFace model cache — keeps downloaded models across rebuilds (~5-20 GB)
|
||||
- hf_model_cache:/root/.cache/huggingface
|
||||
# HuggingFace model cache — bind mount so models can be pre-downloaded on the host.
|
||||
# If container DNS is blocked, download on the host and the container picks them up:
|
||||
# pip install huggingface-hub
|
||||
# huggingface-cli download diffusers/stable-diffusion-xl-1.0-inpainting-0.1 \
|
||||
# --cache-dir ./data/hf_cache
|
||||
# To free disk space: rm -rf ./data/hf_cache
|
||||
- ./data/hf_cache:/root/.cache/huggingface
|
||||
# Scripts (for exec access)
|
||||
- ./scripts:/scripts
|
||||
environment:
|
||||
@@ -136,8 +145,3 @@ services:
|
||||
- 8.8.4.4
|
||||
|
||||
restart: unless-stopped
|
||||
|
||||
volumes:
|
||||
hf_model_cache:
|
||||
# Survives docker compose down; delete manually to free disk space:
|
||||
# docker volume rm editmaskwithai_hf_model_cache
|
||||
|
||||
Reference in New Issue
Block a user