Commit Graph
10 Commits
Author SHA1 Message Date
Claude 851e255641 Fix SAM symlink crash and make prefetch-models.sh resilient per-step
download_sam_model.py crashed with an uncaught PermissionError when
data/models/ is root-owned (common after a prior Docker run) and a
non-root host user tries to recreate the convenience sam_model.pth
symlink — observed in the wild, and it aborted the whole prefetch run
before U2Net/BEN2/BiRefNet-HR were ever attempted. Worse, the same
unguarded symlink call sat inside the post-download try/except, so a
successful download could get deleted just because the symlink step
failed afterward. Wrapped symlink creation in a shared helper that
warns and continues instead of raising — the real model file already
satisfies entrypoint.sh's checks regardless of the symlink.

prefetch-models.sh now treats SAM, U2Net, and the HuggingFace models as
independent steps (one failing no longer aborts the rest) and prints a
summary of which steps failed, so a single run gives full diagnostic
signal instead of stopping at the first error.
2026-06-18 16:33:17 +00:00
Claude fe4d911a00 Dynamic GPU capability detection: probe CC, VRAM, feature flags, pick best model
Replaces fixed tier table with real hardware probing and dynamic model selection.

gpu_detect.py — complete rewrite:
- Reads torch.cuda.get_device_properties + mem_get_info for actual free VRAM
- Detects: fp16 (CC≥6.0), bf16 (CC≥8.0), fp8 (CC≥8.9 Ada/Hopper),
           int8 (CC≥7.0), tensor_cores (CC≥7.0), xformers presence
- Pre-Pascal (CC<6.0): effective_vram halved (fp32 weights are 2× larger)
- Subtracts 400MB driver overhead from free VRAM before model selection
- _select_txt2img / _select_inpaint / _select_img2img / _select_upscale:
    eff≥20GB  → FLUX.1-schnell (no offload)
    eff≥10GB  → FLUX.1-schnell (model_cpu_offload)
    eff≥7.5GB → SDXL
    eff≥5.5GB → SDXL + attention_slicing
    eff≥3.5GB → SD 2.1
    eff≥2.5GB → SD 2.1-base + attention_slicing
    eff≥1.7GB → SD 1.5
    else      → SD 1.5 + sequential_cpu_offload
- ModelSpec carries: model_id, family, memory_opt, native_res, vram_fp16_gb
- Warnings: old CC, pre-Pascal fp32, fp8 upgrade hint, xformers install tip
- Compatibility shim get_model_ids() retained for existing callers
- infer_spec_from_model_id() auto-detects family from HF_MODEL_* overrides

local_diffusion.py — refactored to use ModelSpec:
- Reads spec from GpuCapabilities.recommended[op] instead of tier table
- FLUX.1-schnell: FluxPipeline / FluxImg2ImgPipeline, 4 steps, guidance=0.0
- SD families: family-aware pipeline class selection (sd15/sd2x/sdxl)
- Memory opts applied per ModelSpec.memory_opt field
- xformers attention enabled automatically when xformers detected

gpu_status.py — richer response:
- Exposes all feature flags (fp16/bf16/fp8/int8/tensor_cores/xformers)
- Returns full ModelSpec per operation (model_id, family, memory_opt, native_res)

ai_tools.py — /api/config exposes:
- gpu_vram_total, gpu_vram_free, gpu_cc, gpu_fp16, gpu_bf16, gpu_fp8,
  gpu_tensor_cores, gpu_eff_vram, local_gpu_warnings

requirements.gpu.txt:
- diffusers bumped to >=0.29.0 (FLUX pipeline added in 0.29)
- transformers bumped to >=4.40.0
- sentencepiece added (FLUX T5 tokenizer)

scripts/gpu_setup.py:
- Prints full model table at startup (op → model_id, family, memory_opt, res)
- Shows all feature flags in one line

https://claude.ai/code/session_01WVDg7amsy1TTtxvpku7bcM
2026-06-13 15:35:37 +00:00
Claude 46b9066bba Handle old/low-VRAM GPUs and document nvidia-container-toolkit requirement
GPU tier table extended:
  ultra   ≥16 GB → SDXL (unchanged)
  high    8-16 GB → SDXL (unchanged)
  medium  4-8 GB → SD 2.x (unchanged)
  legacy  2-4 GB → SD 1.5 (~1.7 GB fp16)  ← new: GTX 970/1060/RX 580 etc.
  minimal <2 GB  → SD 1.5 + sequential CPU offload  ← new: very old/integrated GPUs

gpu_detect.py:
- Detects CUDA compute capability (CC); fp16 disabled for CC < 6.0 (pre-Pascal)
- GpuInfo gains compute_capability and warnings fields
- _make_warnings() emits human-readable warnings for low VRAM and old CC
- model tier fallback updated from 'low' to 'legacy'

local_diffusion.py:
- minimal/legacy tiers use enable_sequential_cpu_offload() + enable_attention_slicing(1)
- target resolution per tier: ultra/high=1024, medium=768, legacy/minimal=512
- .to(device) skipped when sequential CPU offload is active

gpu_status.py:
- Response now includes compute_capability and warnings

docker-compose.gpu.yml:
- Full nvidia-container-toolkit install instructions in header comment
- nvidia-docker2 (legacy) fallback documented as comment block inline
- AMD ROCm swap-in instructions added
- GPU tier table documented in header

scripts/gpu_setup.py:
- Prints compute capability, fp16 status, tier, and model selection at startup
- Prints per-tier warnings (old CC, low VRAM)

https://claude.ai/code/session_01WVDg7amsy1TTtxvpku7bcM
2026-06-13 15:19:01 +00:00
Claude 8fe8498df2 Add local GPU inference: auto-detect GPU, auto-download best diffusion models
Adds AI_PROVIDER=local_gpu — a fully self-contained GPU inference engine
using HuggingFace Diffusers that requires zero InvokeAI/ComfyUI setup.
All existing providers (InvokeAI, ComfyUI, OpenAI, Replicate) remain intact
and can be mixed with local GPU via per-operation overrides.

New features:
- GPU auto-detection (CUDA/NVIDIA, MPS/Apple Silicon, CPU fallback)
- VRAM-tiered model selection:
    ultra ≥16 GB → SDXL inpaint + SDXL base
    high  8-16 GB → SDXL inpaint + SDXL base
    medium 4-8 GB → SD 2.x inpaint + SD 2.1
    low  <4 GB   → SD 2.x (small)
- Auto-download model weights to HuggingFace disk cache at startup
  (background task; first request loads from local disk, not internet)
- LRU pipeline cache evicts oldest GPU pipeline when VRAM limit reached
- Per-operation model overrides via HF_MODEL_INPAINT / HF_MODEL_TXT2IMG etc.
- Optional HF_TOKEN for gated/private HuggingFace models

New files:
- backend/app/services/gpu_detect.py   — GPU detection + tier/model mapping
- backend/app/services/local_diffusion.py — Diffusers provider + LRU cache
- backend/app/routers/gpu_status.py    — GET /api/gpu/status, POST /api/gpu/prefetch
- backend/requirements.gpu.txt         — Diffusers ecosystem deps (GPU only)
- docker-compose.gpu.yml               — NVIDIA GPU compose (one-command startup)
- Dockerfile.gpu                       — pytorch/pytorch:2.1.0-cuda12.1 base image
- scripts/gpu_setup.py                 — Startup GPU info logger

Modified:
- backend/app/config.py                — local_gpu settings added
- backend/app/services/remote_provider.py — local_gpu registered as provider
- backend/app/routers/ai_tools.py      — /api/config exposes GPU tier + caps
- backend/app/main.py                  — GPU router + background prefetch task
- backend/entrypoint.sh                — runs gpu_setup.py at container start
- .env.example                         — local_gpu documented as first option

Quick start with GPU:
  docker compose -f docker-compose.gpu.yml up --build

https://claude.ai/code/session_01WVDg7amsy1TTtxvpku7bcM
2026-06-13 15:08:41 +00:00
Claude cea9ee9d6c Smart upscale: auto-detect hardware and pick best Real-ESRGAN path
Detection priority (probed once, cached):
  1. Real-ESRGAN PyTorch + CUDA GPU   → fastest, best quality
  2. Real-ESRGAN PyTorch + Apple MPS  → fast on Apple Silicon
  3. Real-ESRGAN NCNN Vulkan binary   → fast on any GPU via Vulkan (no CUDA needed)
  4. Real-ESRGAN PyTorch CPU          → works, slow (warned in UI)
  5. Lanczos                          → always available, instant fallback

Backend:
- services/upscale.py: full capability probe (probe_upscale_capabilities),
  implementations for PyTorch (CUDA/MPS/CPU auto-device) and NCNN binary,
  upscale_sync() resolves method with fallback chain,
  async upscale_image() runs in thread pool
- print_tools.py: /api/print/upscale uses new service; method="auto" by default;
  GET /api/print/upscale/available returns full capability map with device info
  and recommended_label; POST /api/print/upscale/refresh-caps busts cache
  without restart (useful after installing NCNN binary into container)

Frontend:
- upscale.js: fetches capability map on first open; builds method selector showing
  only available options; labels recommended method with ★; shows device info
  (CUDA/MPS/CPU/NCNN) in dialog; maps display label back to method key on submit;
  shows actual method used in success toast and undo history entry

Scripts:
- scripts/download_realesrgan.py: downloads NCNN Vulkan binary for current platform
  (Linux/macOS/Windows) to /app/data/models/realesrgan/; makes executable;
  run inside container or locally

https://claude.ai/code/session_01B58MaJCU1R6KwBDJCp8AfN
2026-06-09 18:32:32 +00:00
Claude 549a1a4e82 Fix AI tools and My Library, add U2net background removal
- Fix My Library: Add CSS styling for library browser, items now visible
- Integrate My Library into Shapes tool with tabbed interface
- Improve AI Inpaint: Add transform mode for scaling/sizing selections
- Add helpful guidance explaining inpaint vs transform modes
- Add U2net as alternative background removal (avoids rembg issues)
- Create U2net model definition and download script
- Improve Caddyfile with multiple options and troubleshooting guide

Note: Brush Select (AI Paint) tool was already implemented and working.

https://claude.ai/code/session_01CLedz6CanT9t46KBvng3vz
2026-01-27 22:29:35 +00:00
Claude df4ddc2d8c Automate SAM download and fix database path issues
- Fix database path mismatch: download_sample_eyes.py now uses
  ai_photo_edit.db instead of photoedit.db
- Add init_database.py script to initialize DB before eye import
- Add AUTO_DOWNLOAD_SAM=true environment variable (default: enabled)
- Update entrypoint.sh to:
  1. Initialize database first
  2. Auto-download SAM model (~375MB) on first startup
  3. Then import eyes (now works since DB exists)
- Update path detection to work in both Docker and local environments
2026-01-25 20:28:57 +00:00
Claude 8b96c86a94 Add local SAM model support for offline Smart Select
- Add torch, torchvision, segment-anything to requirements
- Create download_sam_model.py script to fetch SAM checkpoint
- Update tools.py to use local SAM with Replicate API fallback
- Add SAM model check to entrypoint.sh with helpful instructions
- Model persists in /app/data/models via Docker volume mount
2026-01-25 18:15:07 +00:00
Claude 7a86d53c88 Add script to download sample eye images for catalog
Creates scripts/download_sample_eyes.py that:
- Downloads public domain classical sculpture images from Wikimedia
- Creates patch records in database
- Generates thumbnails
- Populates the Eye Catalog with starter content
2026-01-25 15:08:50 +00:00
Claude 6238318f9b Add classic eye image assets and import scripts
- Generate 18 classic eye variants in 3 styles (realistic, anime, cartoon)
  with 6 colors each (blue, green, brown, hazel, grey, amber)
- Add scripts for generating, converting, and importing eyes:
  - generate_classic_eyes_ppm.py: Generate PPM images (no dependencies)
  - download_classic_eyes.py: Full-featured generator with PIL
  - startup_import_eyes.py: Import eyes on backend startup
  - import_saved_eyes.py: Import from saved directory
- Update docker-compose to mount scripts directory
- Include metadata.json for tracking eye properties
2026-01-25 02:41:37 +00:00