Frontend:
- Fix smart_select.js to properly render mask overlay
- Add marching ants border around selection
- Calculate selection bounds from mask
- Trigger re-render after mask is loaded
Backend:
- Fix inpaint endpoint to call edit_image() instead of inpaint()
- The AI providers use edit_image() method, not inpaint()
- Update main Dockerfile to copy all miniPaint static files:
- index.html
- dist/ (webpack bundle)
- images/ (icons and assets)
- src/css/ (stylesheets)
- Update backend main.py to conditionally mount static directories
- Checks if each directory exists before mounting
- Supports both React (assets/) and miniPaint (dist/, images/, src/) structures
New features:
- Smart Select tool: Click to select objects using SAM (Segment Anything)
- AI Inpaint tool: Edit selected regions with text prompts
Changes:
- frontend/src/js/tools/smart_select.js: SAM-powered selection tool
- frontend/src/js/tools/ai_inpaint.js: AI inpainting with prompt dialog
- frontend/src/js/services/api.js: API service for backend communication
- frontend/src/js/config.js: Register new tools
- frontend/src/css/layout.css: Tool icon styles
- frontend/images/icons/: SVG icons for new tools
- backend/app/routers/tools.py: New base64 API endpoints
- frontend/Dockerfile: Updated for miniPaint build
- frontend/nginx.conf: Added /api prefix proxy
ROOT CAUSE: ImageCanvas referenced props that don't exist:
- activeTool, onSmartSelect, onColorSelect were used but never declared
- selectionMode checked for 'smart'/'color' but App passes advancedToolMode with 'smart-select'/'color-select'
Changes:
- Update mode checking to use advancedToolMode instead of selectionMode for smart/color select
- Replace onSmartSelect/onColorSelect calls with onAdvancedToolClick
- Fix useEffect dependency array to reference actual props
- Add missing state/refs: currentZoom, imageRef, baseScaleRef, isDrawingRef
- Add zoom control buttons to canvas UI
- Fix tool mode indicator to check advancedToolMode
- Add zoom support via prop (scales image around center)
- Fix selection interaction: clicking on selection transforms, clicking elsewhere creates new
- Use refs for state accessed in event handlers (fixes stale closure issue)
- Add smart select mode (clicks call onSmartSelect with image coordinates)
- Add color select mode placeholder
- Add pan mode with cursor feedback
- Add move mode for manipulating selections
- Use dashed blue selection style (more visible)
- forwardRef to expose canvas methods
- Redesign layout: left toolbar, center canvas, right sidebar
- Add vertical tool strip with keyboard shortcuts (V, R, E, F, W, B, G, Z, H)
- Collapsible sidebar panels: Tool Options, AI Edit, Quick Actions, Eyes, Layers, History
- Canvas now fills available space with checkerboard background
- Add zoom controls in status bar
- Add menu bar with New, Save, Reset
- Remove old multi-column layout
- Create unified Dockerfile with multi-stage build (Node + Python)
- FastAPI now serves React static files directly
- Remove frontend service and nginx dependency
- Simplify docker-compose to single service
- All routes work without proxy configuration
Frontend changes:
- Wire Smart Select and Color Select to canvas click handlers
- Add externalSelection prop to ImageCanvas for displaying AI-generated selections
- Add zoom controls (mouse wheel + buttons) to ImageCanvas
- Fix layer buttons (New Layer, Delete, Duplicate) with proper handlers
- Lift advancedToolMode state to App.jsx for coordination between components
- Add tool mode indicator overlay on canvas
Backend changes:
- Update smart-select endpoint to return JSON with polygon and bbox data
- Update color-select endpoint to return JSON with polygon and bbox data
- Add _mask_to_polygon helper function using OpenCV contour detection
- Add cv2 and base64 imports to tools.py
API changes:
- smartSelect and colorSelect now return { polygon, bbox, mask_base64 }
- Fix database path mismatch: download_sample_eyes.py now uses
ai_photo_edit.db instead of photoedit.db
- Add init_database.py script to initialize DB before eye import
- Add AUTO_DOWNLOAD_SAM=true environment variable (default: enabled)
- Update entrypoint.sh to:
1. Initialize database first
2. Auto-download SAM model (~375MB) on first startup
3. Then import eyes (now works since DB exists)
- Update path detection to work in both Docker and local environments
- Add torch, torchvision, segment-anything to requirements
- Create download_sam_model.py script to fetch SAM checkpoint
- Update tools.py to use local SAM with Replicate API fallback
- Add SAM model check to entrypoint.sh with helpful instructions
- Model persists in /app/data/models via Docker volume mount
- Create entrypoint.sh that auto-downloads sample eyes on first run
- Update Dockerfile to use entrypoint script
- Rewrite .env.example with step-by-step setup instructions
- Add detailed troubleshooting section
- Clarify which models work for inpainting vs text-to-image
- Implement SAM object selection via Replicate API
- Click on any object to select it with AI precision
- Falls back to flood-fill if Replicate API unavailable
- Update Dockerfile for rembg dependencies
- Add required system libraries (libsm6, libxext6, etc)
- Pre-download rembg model during build
- Create data directories for models and patches
Backend:
- Add /tools router with background removal, smart select, color select
- Add rembg dependency for AI background removal
- Add layer management API (list, flatten)
- Fix transparency preservation in blend_patch (veil collapse fix)
- Preserve alpha channel when reverting/resetting images
Frontend:
- Add AdvancedTools panel with background removal, smart select, color select
- Add Layers panel with drag-to-reorder, visibility toggle, flatten
- Add toolsApi for new backend endpoints
- Make right panel scrollable for additional controls
This adds "Photoshop light" capabilities:
- Remove background and create layer
- Smart object selection (click to select)
- Color selection with tolerance
- Layer system with compositing
Creates scripts/download_sample_eyes.py that:
- Downloads public domain classical sculpture images from Wikimedia
- Creates patch records in database
- Generates thumbnails
- Populates the Eye Catalog with starter content
Fixes:
- Remove scale limit to allow image to fill canvas
- Fix Lasso tool by using Polygon instead of Polyline
- Remove duplicate Clear Selection button
- Fix Undo/Redo dependencies with useCallback
New Features:
- Eye Catalog UI for browsing and applying saved eyes
- Upload new eyes to catalog
- Apply eyes to selected areas with feathering
- Delete eyes from catalog
- Make canvas use full viewport height for larger image display
- Add download button to export edited images
- Implement Undo/Redo with Ctrl+Z / Ctrl+Y keyboard shortcuts
- Add transform controls for selections (move, resize, rotate)
- Add warn on reload to prevent data loss
- Make project naming optional, auto-generate from filename
Claude