Claude
11772e620c
Fix build cache, DNS/model download, and AI Edit error handling
...
Build / pip layer fixes:
- Add BUILDID ARG to Dockerfile.gpu; pass from docker-compose.gpu.yml build args
so pip layers can be force-busted without --no-cache:
BUILDID=$(date +%s) docker compose -f docker-compose.gpu.yml up --build
Model download (DNS-blocked environments):
- Change HF model cache from named volume to ./data/hf_cache bind mount
so models can be pre-downloaded on the host (no rebuild needed)
- Remove now-unused hf_model_cache named volume
- README: add iptables fix + huggingface-cli offline download instructions
Error handling improvements:
- ai_edit_region: catch ConnectError/Errno-3 → return 503 with exact fix commands
- _require_remote: give actionable message when local_gpu provider fails to load
- _build_provider: catch AttributeError (torch.xpu from wrong diffusers) not just ImportError
- local_diffusion.py: fix docstring to reflect <0.29.0 pin
https://claude.ai/code/session_01WVDg7amsy1TTtxvpku7bcM
2026-06-14 00:14:34 +00:00
Claude
372ab48991
Real per-step progress bars for local GPU inference
...
Backend:
- local_diffusion.py: add _make_step_cb() that writes step/total_steps/
progress into _states on every diffusers callback_on_step_end; wired into
txt2img, inpaint, img2img with TypeError fallback for older diffusers
- ai_tools.py: GET /api/generate/progress SSE endpoint — streams _states
as JSON array every 200ms so clients get live denoising step counts
Frontend:
- progress_overlay.js: add connectProgressSSE(pipeType, baseUrl) /
disconnectProgressSSE() — opens EventSource, maps step/total_steps
to bar percentage (0→85% during denoising, 85→100 for decode/place)
- text_to_image.js: connect SSE before POST, disconnect on done/error
- selection_actions.js: connect SSE for AI edit / asymmetry operations
Result: for local GPU, progress bar shows "Step 12 / 30" with exact fill;
for remote providers and upscale (no step callbacks), shimmer animates.
https://claude.ai/code/session_01WVDg7amsy1TTtxvpku7bcM
2026-06-13 16:48:55 +00:00
Claude
fe4d911a00
Dynamic GPU capability detection: probe CC, VRAM, feature flags, pick best model
...
Replaces fixed tier table with real hardware probing and dynamic model selection.
gpu_detect.py — complete rewrite:
- Reads torch.cuda.get_device_properties + mem_get_info for actual free VRAM
- Detects: fp16 (CC≥6.0), bf16 (CC≥8.0), fp8 (CC≥8.9 Ada/Hopper),
int8 (CC≥7.0), tensor_cores (CC≥7.0), xformers presence
- Pre-Pascal (CC<6.0): effective_vram halved (fp32 weights are 2× larger)
- Subtracts 400MB driver overhead from free VRAM before model selection
- _select_txt2img / _select_inpaint / _select_img2img / _select_upscale:
eff≥20GB → FLUX.1-schnell (no offload)
eff≥10GB → FLUX.1-schnell (model_cpu_offload)
eff≥7.5GB → SDXL
eff≥5.5GB → SDXL + attention_slicing
eff≥3.5GB → SD 2.1
eff≥2.5GB → SD 2.1-base + attention_slicing
eff≥1.7GB → SD 1.5
else → SD 1.5 + sequential_cpu_offload
- ModelSpec carries: model_id, family, memory_opt, native_res, vram_fp16_gb
- Warnings: old CC, pre-Pascal fp32, fp8 upgrade hint, xformers install tip
- Compatibility shim get_model_ids() retained for existing callers
- infer_spec_from_model_id() auto-detects family from HF_MODEL_* overrides
local_diffusion.py — refactored to use ModelSpec:
- Reads spec from GpuCapabilities.recommended[op] instead of tier table
- FLUX.1-schnell: FluxPipeline / FluxImg2ImgPipeline, 4 steps, guidance=0.0
- SD families: family-aware pipeline class selection (sd15/sd2x/sdxl)
- Memory opts applied per ModelSpec.memory_opt field
- xformers attention enabled automatically when xformers detected
gpu_status.py — richer response:
- Exposes all feature flags (fp16/bf16/fp8/int8/tensor_cores/xformers)
- Returns full ModelSpec per operation (model_id, family, memory_opt, native_res)
ai_tools.py — /api/config exposes:
- gpu_vram_total, gpu_vram_free, gpu_cc, gpu_fp16, gpu_bf16, gpu_fp8,
gpu_tensor_cores, gpu_eff_vram, local_gpu_warnings
requirements.gpu.txt:
- diffusers bumped to >=0.29.0 (FLUX pipeline added in 0.29)
- transformers bumped to >=4.40.0
- sentencepiece added (FLUX T5 tokenizer)
scripts/gpu_setup.py:
- Prints full model table at startup (op → model_id, family, memory_opt, res)
- Shows all feature flags in one line
https://claude.ai/code/session_01WVDg7amsy1TTtxvpku7bcM
2026-06-13 15:35:37 +00:00
Claude
46b9066bba
Handle old/low-VRAM GPUs and document nvidia-container-toolkit requirement
...
GPU tier table extended:
ultra ≥16 GB → SDXL (unchanged)
high 8-16 GB → SDXL (unchanged)
medium 4-8 GB → SD 2.x (unchanged)
legacy 2-4 GB → SD 1.5 (~1.7 GB fp16) ← new: GTX 970/1060/RX 580 etc.
minimal <2 GB → SD 1.5 + sequential CPU offload ← new: very old/integrated GPUs
gpu_detect.py:
- Detects CUDA compute capability (CC); fp16 disabled for CC < 6.0 (pre-Pascal)
- GpuInfo gains compute_capability and warnings fields
- _make_warnings() emits human-readable warnings for low VRAM and old CC
- model tier fallback updated from 'low' to 'legacy'
local_diffusion.py:
- minimal/legacy tiers use enable_sequential_cpu_offload() + enable_attention_slicing(1)
- target resolution per tier: ultra/high=1024, medium=768, legacy/minimal=512
- .to(device) skipped when sequential CPU offload is active
gpu_status.py:
- Response now includes compute_capability and warnings
docker-compose.gpu.yml:
- Full nvidia-container-toolkit install instructions in header comment
- nvidia-docker2 (legacy) fallback documented as comment block inline
- AMD ROCm swap-in instructions added
- GPU tier table documented in header
scripts/gpu_setup.py:
- Prints compute capability, fp16 status, tier, and model selection at startup
- Prints per-tier warnings (old CC, low VRAM)
https://claude.ai/code/session_01WVDg7amsy1TTtxvpku7bcM
2026-06-13 15:19:01 +00:00
Claude
8fe8498df2
Add local GPU inference: auto-detect GPU, auto-download best diffusion models
...
Adds AI_PROVIDER=local_gpu — a fully self-contained GPU inference engine
using HuggingFace Diffusers that requires zero InvokeAI/ComfyUI setup.
All existing providers (InvokeAI, ComfyUI, OpenAI, Replicate) remain intact
and can be mixed with local GPU via per-operation overrides.
New features:
- GPU auto-detection (CUDA/NVIDIA, MPS/Apple Silicon, CPU fallback)
- VRAM-tiered model selection:
ultra ≥16 GB → SDXL inpaint + SDXL base
high 8-16 GB → SDXL inpaint + SDXL base
medium 4-8 GB → SD 2.x inpaint + SD 2.1
low <4 GB → SD 2.x (small)
- Auto-download model weights to HuggingFace disk cache at startup
(background task; first request loads from local disk, not internet)
- LRU pipeline cache evicts oldest GPU pipeline when VRAM limit reached
- Per-operation model overrides via HF_MODEL_INPAINT / HF_MODEL_TXT2IMG etc.
- Optional HF_TOKEN for gated/private HuggingFace models
New files:
- backend/app/services/gpu_detect.py — GPU detection + tier/model mapping
- backend/app/services/local_diffusion.py — Diffusers provider + LRU cache
- backend/app/routers/gpu_status.py — GET /api/gpu/status, POST /api/gpu/prefetch
- backend/requirements.gpu.txt — Diffusers ecosystem deps (GPU only)
- docker-compose.gpu.yml — NVIDIA GPU compose (one-command startup)
- Dockerfile.gpu — pytorch/pytorch:2.1.0-cuda12.1 base image
- scripts/gpu_setup.py — Startup GPU info logger
Modified:
- backend/app/config.py — local_gpu settings added
- backend/app/services/remote_provider.py — local_gpu registered as provider
- backend/app/routers/ai_tools.py — /api/config exposes GPU tier + caps
- backend/app/main.py — GPU router + background prefetch task
- backend/entrypoint.sh — runs gpu_setup.py at container start
- .env.example — local_gpu documented as first option
Quick start with GPU:
docker compose -f docker-compose.gpu.yml up --build
https://claude.ai/code/session_01WVDg7amsy1TTtxvpku7bcM
2026-06-13 15:08:41 +00:00
Claude