ai_edit.js:
- Extend Base_tools_class and add load() + default_events() so mouse
events actually wire up (was the root cause of tool not working)
- Use get_mouse_info() for coordinate mapping instead of manual
clientX/Y math — consistent with all other miniPaint tools
- Fix _mouseToImage() to use mouse.x/y (already in image coords)
and correct display-scale for overlay brush rendering
- mousedown/mousemove/mouseup now guard on config.TOOL.name
ai_edit.svg: new icon (brush + sparkle star) for the left toolbar
layout.css: add .ai_edit:after CSS rule for the icon
main.js:
- Collapse right-panel Colors section by default (respects saved cookie
so user preference persists)
- Mount compact foreground/background color swatches at the bottom of
the left toolbar; click either square to toggle the full color picker
open/closed; syncs live with config.COLOR every 250ms
https://claude.ai/code/session_01B58MaJCU1R6KwBDJCp8AfN
sklearn was not installed in the container, causing ModuleNotFoundError on
import of ai_tools.py and preventing the server from starting.
Replaced with a self-contained numpy k-means++ implementation:
- k-means++ seeding for better initial centers
- 20-iteration Lloyd's algorithm
- Same output: hex colors sorted by cluster frequency
No new dependencies required.
https://claude.ai/code/session_01B58MaJCU1R6KwBDJCp8AfN
pick_color.js: on hover shows a floating tooltip with:
- 40px color swatch of the sampled pixel
- Hex (#RRGGBB), RGB, and HSL values
- Nearest Pantone name and swatch from the database
- ΔE value (CIE76 color distance in LAB space)
- Quality badge: Excellent (<2) / Good (2-5) / Fair (5-10) / Poor (>10)
- Warning note when ΔE > 10 (color cannot be faithfully printed as Pantone)
Click copies hex to clipboard and sets as active color (existing behavior).
color_utils.js: hexToRgb, rgbToHsl, sRGB→XYZ→CIE LAB conversion (D65),
CIE76 deltaE, nearestPantone() (scans all ~350 entries, returns best match).
Pantone database is pre-converted to LAB at module load for fast scanning.
pantone_colors.js: ~350 representative PMS colors across reds, oranges,
yellows, greens, teals, blues, purples, magentas, browns, grays, black/white.
Hex approximations from open-source Pantone datasets (not official Pantone data).
https://claude.ai/code/session_01B58MaJCU1R6KwBDJCp8AfN
Auto-Enhance (Image menu):
POST /api/enhance — gray-world white balance, CLAHE contrast on L channel,
saturation boost ×1.15 in HSV, unsharp mask; all blended by strength slider
Frontend: strength selector (25/50/75/100%), keep-original option
Extract Color Palette (Image menu):
POST /api/extract-colors — k-means on 150×150 thumbnail, returns N dominant
colors sorted by cluster size. Frontend: floating swatch panel, click=copy
hex, shift+click=set as active color, toggle on/off.
History Panel (Edit menu, Ctrl+H):
Pure frontend — reads app.State.action_history and action_history_index,
renders clickable list of past actions (newest first), click any step to
undo/redo to that point. Auto-refreshes every 800ms while open.
Align to Canvas (Layer menu):
Floating toolbar with 7 alignment buttons: center H, center V, center both,
align left/right/top/bottom edges. Uses Update_layer_action for undo support.
Add Text (Generate menu):
6 styled presets (Heading, Subheading, Body, Caption, Quote, Bold Label)
shown as live-rendered previews in the dialog. Click a preset to insert a
text layer with the correct font/size/weight/color pre-applied.
https://claude.ai/code/session_01B58MaJCU1R6KwBDJCp8AfN
Backend:
- sam_service.py: auto-downloads SAM ViT-B (~375 MB) on first use with
progress tracking; loads model to CUDA/MPS/CPU; predict_points() takes
multi-point prompts (include/exclude labels) and returns best mask
- POST /api/segment/point: SAM point-prompt endpoint; returns mask PNG
- GET /api/segment/install-status: poll download progress
- POST /api/segment/install: explicit trigger (also auto on first click)
- main.py: pre-download SAM on startup alongside NCNN
Frontend (ai_edit.js):
- Click mode (default): click object → SAM generates mask instantly
Alt+click → subtract (deselect over-selected area)
Multiple clicks accumulate for multi-object or refinement
- Brush + / Brush − modes: paint to add or erase from SAM mask by hand
- If SAM model is still downloading on first click: inline progress bar,
user retries the click when done
- Unified action bar: Erase | Replace (inline prompt) | Upscale | Expand | Clear
- All modes share the same mask canvas; SAM and brush are fully composited
https://claude.ai/code/session_01B58MaJCU1R6KwBDJCp8AfN
Before: 4 separate tools (ai_inpaint, ai_lama_erase, ai_smart_inpaint,
ai_replace_selection) with no clear relationship, buried in toolbar.
After: one "AI Edit" brush tool. Paint mask → floating action bar appears:
Erase — LaMa removes masked content, no prompt needed
Replace — inline prompt reveals in-place ("make her smile" / "replace
with a wolf") → calls /api/inpaint/remote
Upscale — delegates to existing upscale module
Expand — delegates to existing outpaint module
Clear — wipe mask and start over
Brush size controlled by tool attributes (5–200px).
Red overlay gives immediate visual feedback while painting.
Floating panel appears as soon as first brush stroke is made.
https://claude.ai/code/session_01B58MaJCU1R6KwBDJCp8AfN
- Add _vulkan_available(): checks /dev/dri/renderD* on Linux, assumes
true on macOS/Windows; set REALESRGAN_NCNN=force to override
- Add _test_ncnn_binary(): test-runs the binary after install and checks
stderr for "no vulkan" — marks skipped if Vulkan init fails at runtime
- ensure_ncnn_installed() now returns early with state=skipped when no
Vulkan detected, avoiding a wasted ~30MB download on CPU-only servers
- Recommend PyTorch CPU when available on headless (AI quality, slow but
works); Lanczos as final fallback
- Frontend: handle state=skipped immediately (no polling needed), show
brief informational toast; show "Headless server" note in dialog
https://claude.ai/code/session_01B58MaJCU1R6KwBDJCp8AfN
- upscale.py: add InstallStatus dataclass + ensure_ncnn_installed() async
function that downloads and extracts the NCNN binary for the current
platform (Linux/macOS/Windows), tracks progress (0-100%), and busts the
caps cache when done
- main.py: trigger ensure_ncnn_installed() as a background task on app
startup when no AI upscaler is detected
- print_tools.py: /upscale/available triggers install task when no AI
upscaler found; new GET /upscale/install-status endpoint for polling
- upscale.js: if no AI upscaler on open, poll install-status showing a
progress bar notification, then refresh caps and proceed when done
https://claude.ai/code/session_01B58MaJCU1R6KwBDJCp8AfN
Detection priority (probed once, cached):
1. Real-ESRGAN PyTorch + CUDA GPU → fastest, best quality
2. Real-ESRGAN PyTorch + Apple MPS → fast on Apple Silicon
3. Real-ESRGAN NCNN Vulkan binary → fast on any GPU via Vulkan (no CUDA needed)
4. Real-ESRGAN PyTorch CPU → works, slow (warned in UI)
5. Lanczos → always available, instant fallback
Backend:
- services/upscale.py: full capability probe (probe_upscale_capabilities),
implementations for PyTorch (CUDA/MPS/CPU auto-device) and NCNN binary,
upscale_sync() resolves method with fallback chain,
async upscale_image() runs in thread pool
- print_tools.py: /api/print/upscale uses new service; method="auto" by default;
GET /api/print/upscale/available returns full capability map with device info
and recommended_label; POST /api/print/upscale/refresh-caps busts cache
without restart (useful after installing NCNN binary into container)
Frontend:
- upscale.js: fetches capability map on first open; builds method selector showing
only available options; labels recommended method with ★; shows device info
(CUDA/MPS/CPU/NCNN) in dialog; maps display label back to method key on submit;
shows actual method used in success toast and undo history entry
Scripts:
- scripts/download_realesrgan.py: downloads NCNN Vulkan binary for current platform
(Linux/macOS/Windows) to /app/data/models/realesrgan/; makes executable;
run inside container or locally
https://claude.ai/code/session_01B58MaJCU1R6KwBDJCp8AfN
Backend — new /api/print/* router:
- POST /api/print/frame-fit: fit image to 4x6/5x7/8x10/11x14/16x20/20x24/24x36
and square sizes (4x4/8x8/12x12) at configurable DPI.
Three modes:
crop — center-crop to aspect ratio, Lanczos scale to print res (no AI)
extend — scale to fill one dimension, AI-inpaint the gap; mirror-fill fallback
smart — auto: extend if gap < 15% of frame dimension, else crop
Auto-detects orientation from image shape; respects explicit portrait/landscape.
- POST /api/print/upscale: Lanczos scale (always) or Real-ESRGAN (if installed)
- GET /api/print/frame-sizes: frame catalogue with pixel dimensions at 300dpi
- GET /api/print/upscale/available: reports whether Real-ESRGAN is installed
Frontend:
- modules/image/frame_fit.js: dialog with frame size, orientation, mode, DPI,
optional extend prompt; shows current image size; result as new layer option
- modules/image/upscale.js: dialog with scale factor (1.5–4×), method selector
(auto-hides AI option if Real-ESRGAN not available); result as new layer option
- config-menu.js: Fit to Frame... and Upscale... added under Image menu
https://claude.ai/code/session_01B58MaJCU1R6KwBDJCp8AfN
Each operation (inpaint, txt2img, img2img, outpaint) can now use a different
provider. Resolution order: per-op override → global AI_PROVIDER default.
Example: txt2img→openai, inpaint→invokeai, everything else→invokeai default.
Backend:
- config.py: add AI_PROVIDER_INPAINT / TXT2IMG / IMG2IMG / OUTPAINT settings
- remote_provider.py: get_remote_provider(operation) resolves override then default;
_build_provider() extracted as shared factory; _OP_FIELD maps op→setting name
- ai_tools.py: each endpoint passes its operation to _require_remote();
GET /api/config runs per-op health checks concurrently, returns operations map
and overrides; POST /api/config accepts and applies per-op override fields
Frontend:
- ai_provider_settings.js: four new selects (inpaint/txt2img/img2img/outpaint);
persists to localStorage and sends per-op fields to POST /api/config
- provider-badge.js: shows override summary (e.g. "invokeai · txt2img→openai")
and per-op health in tooltip
- .env.example: document per-op override env vars with examples
https://claude.ai/code/session_01B58MaJCU1R6KwBDJCp8AfN
Frontend:
- tools/ai_replace_selection.js: use any selection → remote inpaint with prompt
- modules/generate/text_to_image.js: Text → Image dialog (new layer or replace canvas)
- modules/generate/outpaint.js: Expand Canvas in any direction via remote provider
- modules/tools/ai_provider_settings.js: in-app provider config (OpenAI / InvokeAI /
ComfyUI / Replicate); persists to localStorage, pushes to POST /api/config at runtime
- config.js: register ai_replace_selection tool
- config-menu.js: add Generate menu (Text→Image, Outpaint); AI Provider Settings under Tools
- modules/help/about.js: updated credits (LaMa, rembg, SAM, InvokeAI, ComfyUI, OpenAI)
- api/capabilities.js: add refreshCapabilities() for post-save cache invalidation
Backend:
- routers/ai_tools.py: POST /api/config — apply provider settings at runtime
without restart (session-scoped, non-persistent; .env for permanence)
https://claude.ai/code/session_01B58MaJCU1R6KwBDJCp8AfN
- Switch U2Net from onnxruntime to OpenCV DNN to avoid
"cannot enable executable stack" error in Docker
- Add alertify dialog styling to fix white text on white
background issue in popups
- Add offerFloatSelection() to AI Paint that prompts user
after selection to move/scale it (Canva-like workflow)
- Auto-switch to Select tool after floating selection
https://claude.ai/code/session_01CLedz6CanT9t46KBvng3vz
- Fix brush_select isProcessing flag not resetting after first use
(reset in on_leave() when switching tools)
- Add onnxruntime dependency for U2net background removal
- Use full U2net model (176MB) instead of lightweight for better quality
https://claude.ai/code/session_01CLedz6CanT9t46KBvng3vz
- Fix brush_select and smart_select mask scaling to match layer dimensions
- Add "Float Selection" feature to Select tool - when switching to Select
with an active AI selection, offers to copy it to a movable layer
- Add aspect ratio lock toggle (🔗 button) in layer details panel
- When locked, changing width auto-updates height and vice versa
- Add U2net model auto-download - will download lightweight u2netp.onnx (~4MB)
automatically if no model found
- Improve error messages for background removal
- Register on_activate for select tool in config
Workflow: Select object with AI tool → Click Select tool → "Float" selection
→ Move/scale the floated layer freely
https://claude.ai/code/session_01CLedz6CanT9t46KBvng3vz
- Fix My Library: Add CSS styling for library browser, items now visible
- Integrate My Library into Shapes tool with tabbed interface
- Improve AI Inpaint: Add transform mode for scaling/sizing selections
- Add helpful guidance explaining inpaint vs transform modes
- Add U2net as alternative background removal (avoids rembg issues)
- Create U2net model definition and download script
- Improve Caddyfile with multiple options and troubleshooting guide
Note: Brush Select (AI Paint) tool was already implemented and working.
https://claude.ai/code/session_01CLedz6CanT9t46KBvng3vz
scikit-image 0.26+ (required by rembg) needs numpy 2.x compatible
packages. Updated torch from 2.1.2 to 2.4+ and torchvision from
0.16.2 to 0.19+ which officially support numpy 2.x.
https://claude.ai/code/session_01MYpjNQXD1fZE4gCweGU4QQ
rembg>=2.0.70 requires scikit-image>=0.26.0, which conflicted with
the pinned scikit-image==0.22.0. Instead of pinning specific versions,
let pip resolve compatible versions automatically based on rembg's
requirements.
Also removed numpy pin as it may conflict with torch/rembg dependencies
- pip will select a compatible version.
https://claude.ai/code/session_01MYpjNQXD1fZE4gCweGU4QQ
rembg 2.0.70+ requires Pillow>=12.1.0,<13.0.0 which conflicted with
the pinned Pillow==10.2.0. Updated to use the version range that
satisfies rembg while remaining compatible with scikit-image and
torchvision (both have no upper bound on Pillow).
https://claude.ai/code/session_01MYpjNQXD1fZE4gCweGU4QQ
- Use onnxruntime>=1.17.0 (fixed executable stack issues, no execstack needed)
- Use rembg>=2.0.70 with BiRefNet model support
- Update all remove-background endpoints to use birefnet-general model
- BiRefNet provides better edge detection and matting quality than u2net
- Falls back to default model if BiRefNet unavailable
- Use onnxruntime 1.14.1 (older version without executable stack requirement)
- Make rembg pre-download optional (won't fail build if onnxruntime has issues)
- Remove Background will be disabled if rembg can't load
- Save layer or selection to personal library
- Browse library with category grouping and thumbnails
- Insert assets as new layers (double-click or Insert button)
- Delete unwanted assets
- Export library to JSON for backup
- Import library from JSON backup
- Assets stored in browser IndexedDB (persistent)
- Categories: General, Shapes, Borders, Icons, Templates, etc.
- New Image > Selection Effects submenu
- Invert Selection: Invert colors only within selected area
- Adjust Selection: Brightness/contrast/gamma on selection only
- Greyscale Selection: Convert just the selected area to greyscale
- All effects work with any selection tool (Smart Select, Brush Select, etc.)
- Useful for CNC depth maps where specific objects need different treatments
The onnxruntime library requires an executable stack which is blocked
by security restrictions in Docker containers. This adds execstack tool
and uses it to clear the executable stack flag after pip install.
https://claude.ai/code/session_01AGoPJaXqdJnnuxtmSv6NLR
- New tool: Paint/brush over objects to select them
- AI (SAM) detects actual object boundaries from brush strokes
- Collects sample points along brush path, sends to SAM
- Combines multiple masks for multi-object selection
- Shift+brush to add to existing selection
- Full cut/copy/delete support with mask-shaped results
- Visual feedback: brush stroke preview, sample points, marching ants
- Fix cut/copy to preserve mask shape with transparency
- Add missing CSS icons for magic_wand, lasso, ellipse_select tools
- Fix toast messages styling (visible on dark theme)
- Update AI Inpaint to work with all selection tools
- Change tab title to +miniPaint
- Add DNS config to docker-compose for external API access
Frontend:
- Added "Remove Background (AI)" to Image menu
- Created remove_background.js module with dialog options
- Added removeBackground method to API service
Backend:
- Added /tools/remove-background-base64 endpoint for miniPaint frontend
- Uses rembg library for AI-powered background removal
Features:
- Automatically detects main subject and removes background
- Option to create as new layer or replace current
- Enables transparency mode after removal
- Works with any image layer
Claude