Claude
8fe8498df2
Adds AI_PROVIDER=local_gpu — a fully self-contained GPU inference engine
using HuggingFace Diffusers that requires zero InvokeAI/ComfyUI setup.
All existing providers (InvokeAI, ComfyUI, OpenAI, Replicate) remain intact
and can be mixed with local GPU via per-operation overrides.
New features:
- GPU auto-detection (CUDA/NVIDIA, MPS/Apple Silicon, CPU fallback)
- VRAM-tiered model selection:
ultra ≥16 GB → SDXL inpaint + SDXL base
high 8-16 GB → SDXL inpaint + SDXL base
medium 4-8 GB → SD 2.x inpaint + SD 2.1
low <4 GB → SD 2.x (small)
- Auto-download model weights to HuggingFace disk cache at startup
(background task; first request loads from local disk, not internet)
- LRU pipeline cache evicts oldest GPU pipeline when VRAM limit reached
- Per-operation model overrides via HF_MODEL_INPAINT / HF_MODEL_TXT2IMG etc.
- Optional HF_TOKEN for gated/private HuggingFace models
New files:
- backend/app/services/gpu_detect.py — GPU detection + tier/model mapping
- backend/app/services/local_diffusion.py — Diffusers provider + LRU cache
- backend/app/routers/gpu_status.py — GET /api/gpu/status, POST /api/gpu/prefetch
- backend/requirements.gpu.txt — Diffusers ecosystem deps (GPU only)
- docker-compose.gpu.yml — NVIDIA GPU compose (one-command startup)
- Dockerfile.gpu — pytorch/pytorch:2.1.0-cuda12.1 base image
- scripts/gpu_setup.py — Startup GPU info logger
Modified:
- backend/app/config.py — local_gpu settings added
- backend/app/services/remote_provider.py — local_gpu registered as provider
- backend/app/routers/ai_tools.py — /api/config exposes GPU tier + caps
- backend/app/main.py — GPU router + background prefetch task
- backend/entrypoint.sh — runs gpu_setup.py at container start
- .env.example — local_gpu documented as first option
Quick start with GPU:
docker compose -f docker-compose.gpu.yml up --build
https://claude.ai/code/session_01WVDg7amsy1TTtxvpku7bcM
76 lines
2.6 KiB
Docker
76 lines
2.6 KiB
Docker
# =============================================================================
|
|
# EditmaskwithAI — GPU Container (NVIDIA CUDA)
|
|
#
|
|
# Usage:
|
|
# docker compose -f docker-compose.gpu.yml up --build
|
|
#
|
|
# Requirements on host:
|
|
# - NVIDIA driver ≥ 525 (for CUDA 12.x)
|
|
# - nvidia-container-toolkit installed and configured
|
|
# - docker compose v2 (or docker-compose with GPU device support)
|
|
#
|
|
# AMD ROCm users: replace the pytorch base image with a ROCm variant, e.g.
|
|
# rocm/pytorch:latest (and remove the nvidia-smi check below)
|
|
# =============================================================================
|
|
|
|
# ── Stage 1: Build miniPaint frontend ────────────────────────────────────────
|
|
FROM node:20-alpine AS frontend-build
|
|
|
|
WORKDIR /frontend
|
|
COPY frontend/package.json frontend/package-lock.json* ./
|
|
RUN npm install
|
|
COPY frontend/ ./
|
|
RUN npm run build
|
|
|
|
# ── Stage 2: PyTorch CUDA runtime ────────────────────────────────────────────
|
|
# pytorch/pytorch already includes torch + torchvision built for CUDA 12.1.
|
|
# Using the runtime (not devel) image keeps the layer lean.
|
|
FROM pytorch/pytorch:2.1.0-cuda12.1-cudnn8-runtime
|
|
|
|
WORKDIR /app
|
|
|
|
# System dependencies
|
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
|
libgl1 \
|
|
libglib2.0-0 \
|
|
libsm6 \
|
|
libxext6 \
|
|
libxrender-dev \
|
|
libgomp1 \
|
|
wget \
|
|
git \
|
|
&& rm -rf /var/lib/apt/lists/*
|
|
|
|
# Install Python dependencies — base + GPU extras
|
|
COPY backend/requirements.txt .
|
|
COPY backend/requirements.gpu.txt .
|
|
RUN pip install --no-cache-dir -r requirements.txt
|
|
RUN pip install --no-cache-dir -r requirements.gpu.txt
|
|
|
|
# Smoke-test rembg (model downloads on first use)
|
|
RUN python -c "from rembg import remove; print('rembg OK')" \
|
|
|| echo "WARNING: rembg unavailable — Remove Background disabled"
|
|
|
|
# Copy backend application
|
|
COPY backend/ .
|
|
|
|
# Entrypoint
|
|
COPY backend/entrypoint.sh /entrypoint.sh
|
|
RUN chmod +x /entrypoint.sh
|
|
|
|
# Scripts (SAM download, DB init, GPU setup, etc.)
|
|
COPY scripts/ /scripts/
|
|
RUN chmod +x /scripts/*.py 2>/dev/null || true
|
|
|
|
# Copy built frontend from Stage 1
|
|
COPY --from=frontend-build /frontend/index.html /app/static/
|
|
COPY --from=frontend-build /frontend/dist /app/static/dist
|
|
COPY --from=frontend-build /frontend/images /app/static/images
|
|
COPY --from=frontend-build /frontend/src/css /app/static/src/css
|
|
|
|
# Persistent data directories
|
|
RUN mkdir -p /app/data/projects /app/data/patches /app/data/models
|
|
|
|
EXPOSE 8000
|
|
ENTRYPOINT ["/entrypoint.sh"]
|