Backend:
- sam_service.py: auto-downloads SAM ViT-B (~375 MB) on first use with
progress tracking; loads model to CUDA/MPS/CPU; predict_points() takes
multi-point prompts (include/exclude labels) and returns best mask
- POST /api/segment/point: SAM point-prompt endpoint; returns mask PNG
- GET /api/segment/install-status: poll download progress
- POST /api/segment/install: explicit trigger (also auto on first click)
- main.py: pre-download SAM on startup alongside NCNN
Frontend (ai_edit.js):
- Click mode (default): click object → SAM generates mask instantly
Alt+click → subtract (deselect over-selected area)
Multiple clicks accumulate for multi-object or refinement
- Brush + / Brush − modes: paint to add or erase from SAM mask by hand
- If SAM model is still downloading on first click: inline progress bar,
user retries the click when done
- Unified action bar: Erase | Replace (inline prompt) | Upscale | Expand | Clear
- All modes share the same mask canvas; SAM and brush are fully composited
https://claude.ai/code/session_01B58MaJCU1R6KwBDJCp8AfN
Claude
3746c02d44