- Serve /dist and /src with Cache-Control: no-cache so browsers always
revalidate the webpack bundle (it has a fixed filename with no content
hash, so a stale browser cache was hiding the new Remove Background
model dropdown after the rebuild).
- Enable BEN2's refine_foreground pass by default - it's the model's own
documented option for preserving fine/semi-transparent edge detail
(text borders, light rays) instead of a harder cutout, at a small
speed cost.
- Make install-local-gpu.sh and bring-up-local-gpu.sh pre-create ./data
as the invoking non-root user before Docker ever runs, so its daemon
never gets a chance to auto-create those bind-mount dirs as root.
- Make prefetch-models.sh self-heal a root-owned ./data automatically
(sudo chown) instead of erroring out with a manual fix-it command.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ro4PwQKvSc3CH19LSN21Ht
BEN2 becomes the new default local backend (clean cutouts, strong on
hair/fur edges), with BiRefNet-HR available as a high-res/print
alternate and U2Net kept as the lightweight fallback. Both are
MIT-licensed and download weights from HuggingFace on first use
(cached via the existing hf_cache bind mount), unlike U2Net/SAM which
need an explicit download script.
- config: new BG_REMOVAL_MODEL setting (default "ben2")
- tools.py: remove-background-base64 now tries local backends in
order (request.model override > BG_REMOVAL_MODEL > ben2/u2net),
falling back to rembg's birefnet-general session as a last resort
- requirements.gpu.txt / Dockerfile.gpu: add ben2 + transformers deps
needed for the new backends, with a build-time smoke test for ben2
- frontend: model dropdown in the Remove Background dialog, threaded
through api.js to the new request field
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ro4PwQKvSc3CH19LSN21Ht
- Switch U2Net from onnxruntime to OpenCV DNN to avoid
"cannot enable executable stack" error in Docker
- Add alertify dialog styling to fix white text on white
background issue in popups
- Add offerFloatSelection() to AI Paint that prompts user
after selection to move/scale it (Canva-like workflow)
- Auto-switch to Select tool after floating selection
https://claude.ai/code/session_01CLedz6CanT9t46KBvng3vz
- Fix brush_select isProcessing flag not resetting after first use
(reset in on_leave() when switching tools)
- Add onnxruntime dependency for U2net background removal
- Use full U2net model (176MB) instead of lightweight for better quality
https://claude.ai/code/session_01CLedz6CanT9t46KBvng3vz
- Fix brush_select and smart_select mask scaling to match layer dimensions
- Add "Float Selection" feature to Select tool - when switching to Select
with an active AI selection, offers to copy it to a movable layer
- Add aspect ratio lock toggle (🔗 button) in layer details panel
- When locked, changing width auto-updates height and vice versa
- Add U2net model auto-download - will download lightweight u2netp.onnx (~4MB)
automatically if no model found
- Improve error messages for background removal
- Register on_activate for select tool in config
Workflow: Select object with AI tool → Click Select tool → "Float" selection
→ Move/scale the floated layer freely
https://claude.ai/code/session_01CLedz6CanT9t46KBvng3vz
- Fix My Library: Add CSS styling for library browser, items now visible
- Integrate My Library into Shapes tool with tabbed interface
- Improve AI Inpaint: Add transform mode for scaling/sizing selections
- Add helpful guidance explaining inpaint vs transform modes
- Add U2net as alternative background removal (avoids rembg issues)
- Create U2net model definition and download script
- Improve Caddyfile with multiple options and troubleshooting guide
Note: Brush Select (AI Paint) tool was already implemented and working.
https://claude.ai/code/session_01CLedz6CanT9t46KBvng3vz
- Use onnxruntime>=1.17.0 (fixed executable stack issues, no execstack needed)
- Use rembg>=2.0.70 with BiRefNet model support
- Update all remove-background endpoints to use birefnet-general model
- BiRefNet provides better edge detection and matting quality than u2net
- Falls back to default model if BiRefNet unavailable
Frontend:
- Added "Remove Background (AI)" to Image menu
- Created remove_background.js module with dialog options
- Added removeBackground method to API service
Backend:
- Added /tools/remove-background-base64 endpoint for miniPaint frontend
- Uses rembg library for AI-powered background removal
Features:
- Automatically detects main subject and removes background
- Option to create as new layer or replace current
- Enables transparency mode after removal
- Works with any image layer
Frontend:
- Fix smart_select.js to properly render mask overlay
- Add marching ants border around selection
- Calculate selection bounds from mask
- Trigger re-render after mask is loaded
Backend:
- Fix inpaint endpoint to call edit_image() instead of inpaint()
- The AI providers use edit_image() method, not inpaint()
New features:
- Smart Select tool: Click to select objects using SAM (Segment Anything)
- AI Inpaint tool: Edit selected regions with text prompts
Changes:
- frontend/src/js/tools/smart_select.js: SAM-powered selection tool
- frontend/src/js/tools/ai_inpaint.js: AI inpainting with prompt dialog
- frontend/src/js/services/api.js: API service for backend communication
- frontend/src/js/config.js: Register new tools
- frontend/src/css/layout.css: Tool icon styles
- frontend/images/icons/: SVG icons for new tools
- backend/app/routers/tools.py: New base64 API endpoints
- frontend/Dockerfile: Updated for miniPaint build
- frontend/nginx.conf: Added /api prefix proxy
Frontend changes:
- Wire Smart Select and Color Select to canvas click handlers
- Add externalSelection prop to ImageCanvas for displaying AI-generated selections
- Add zoom controls (mouse wheel + buttons) to ImageCanvas
- Fix layer buttons (New Layer, Delete, Duplicate) with proper handlers
- Lift advancedToolMode state to App.jsx for coordination between components
- Add tool mode indicator overlay on canvas
Backend changes:
- Update smart-select endpoint to return JSON with polygon and bbox data
- Update color-select endpoint to return JSON with polygon and bbox data
- Add _mask_to_polygon helper function using OpenCV contour detection
- Add cv2 and base64 imports to tools.py
API changes:
- smartSelect and colorSelect now return { polygon, bbox, mask_base64 }
- Add torch, torchvision, segment-anything to requirements
- Create download_sam_model.py script to fetch SAM checkpoint
- Update tools.py to use local SAM with Replicate API fallback
- Add SAM model check to entrypoint.sh with helpful instructions
- Model persists in /app/data/models via Docker volume mount
- Implement SAM object selection via Replicate API
- Click on any object to select it with AI precision
- Falls back to flood-fill if Replicate API unavailable
- Update Dockerfile for rembg dependencies
- Add required system libraries (libsm6, libxext6, etc)
- Pre-download rembg model during build
- Create data directories for models and patches
Backend:
- Add /tools router with background removal, smart select, color select
- Add rembg dependency for AI background removal
- Add layer management API (list, flatten)
- Fix transparency preservation in blend_patch (veil collapse fix)
- Preserve alpha channel when reverting/resetting images
Frontend:
- Add AdvancedTools panel with background removal, smart select, color select
- Add Layers panel with drag-to-reorder, visibility toggle, flatten
- Add toolsApi for new backend endpoints
- Make right panel scrollable for additional controls
This adds "Photoshop light" capabilities:
- Remove background and create layer
- Smart object selection (click to select)
- Color selection with tolerance
- Layer system with compositing
Claude