PaintPlus/README.md

EditmaskwithAI

A self-hosted, web-based AI photo editor. Paint over any object, describe what you want, and the AI replaces just that region — every pixel outside your selection stays untouched.

Quick Start

GPU machine (recommended — free inference, best quality)

# Prerequisites: Docker + nvidia-container-toolkit
# Install toolkit once (Ubuntu/Debian):
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey \
  | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-ctk.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list \
  | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-ctk.gpg] https://#g' \
  | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update && sudo apt-get install -y nvidia-container-toolkit
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker

# Verify GPU passes through into Docker:
docker run --rm --gpus all nvidia/cuda:12.1.0-base-ubuntu22.04 nvidia-smi

# Clone and run:
git clone https://github.com/outis1one/editmaskwithai
cd editmaskwithai
docker compose -f docker-compose.gpu.yml up --build

Open http://localhost:3080

First startup downloads the AI model for your GPU (5–20 GB, one time). Models are cached in a Docker volume and survive rebuilds.

---

Cloud API (no GPU required)

git clone https://github.com/outis1one/editmaskwithai
cd editmaskwithai
cp .env.example .env
# Edit .env: set AI_PROVIDER and your API key (see .env.example for options)
docker compose up -d --build

Open http://localhost:3080

---

Updates (any machine)

git pull
# GPU:
docker compose -f docker-compose.gpu.yml up -d --build
# or cloud:
docker compose up -d --build

If pip packages seem stale after a pull (e.g., wrong diffusers version), force a pip layer rebuild without re-downloading the entire PyTorch base image:

BUILDID=$(date +%s) docker compose -f docker-compose.gpu.yml up -d --build

---

AI Providers

Provider	Setup	Cost	Quality
local_gpu	GPU machine + nvidia-container-toolkit	Free	Best (SDXL/FLUX auto-selected by VRAM)
openai	OPENAI_API_KEY=sk-...	~$0.02–0.04/image	DALL-E 3
replicate	REPLICATE_API_KEY=r8_...	~$0.002–0.03/image	Multiple models
invokeai	InvokeAI running on another machine	Self-hosted	FLUX/SDXL
comfyui	ComfyUI running on another machine	Self-hosted	Any model

You can also mix: set a default provider in .env and override per-operation in the Image → AI Provider Settings dialog inside the app.

---

GPU Tier Auto-Selection

The app detects your GPU at startup and picks the best model it can run:

Effective VRAM	Model selected	Notes
≥ 24 GB	FLUX.1-schnell	Best quality, 4-step generation
12–24 GB	SDXL	Excellent quality
8–12 GB	SDXL + xformers	Good quality
6–8 GB	SDXL + attention slicing	Good quality, slightly slower
4–6 GB	SDXL + CPU offload	Good quality, slower (GTX 1060 6GB range)
2–4 GB	SD 1.5	Fast, lower detail
< 2 GB	SD 1.5 + CPU offload	Very slow — consider a cloud provider

Override the auto-selected model with HF_MODEL_TXT2IMG, HF_MODEL_INPAINT in .env.

---

What it can do

Selection

• Smart Select (SAM brush) — paint over an object, AI detects its exact boundaries
• Smart Select (click) — click any object, SAM selects it
• Rectangle / Ellipse / Lasso — classic selection tools

After selecting

• AI Edit — describe what to change ("add a scar", "make it look aged")
• Make less symmetrical — AI adds natural organic variation
• Replace with clipboard — paste any image into the selection shape
• Scale by % — make the selected object bigger/smaller, AI fills the gap
• Copy / Cut to layer — non-destructive layer workflow
• Erase — remove the selected region with AI fill

Image tools

• Text → Image — generate from a text description (GPU or cloud)
• Upscale — Real-ESRGAN AI upscaling (genuinely adds detail, not just resize)
• Prepare for Print — one-click: AI upscale to target DPI + fit to frame
• Fit to Frame — resize/crop/AI-extend to standard print sizes
• Expand Canvas (Outpaint) — AI extends the image in any direction
• Remove Background — one-click background removal

Print presets

Frame sizes: 4×6, 5×7, 8×10, 11×14, 16×20, 18×24, 20×24, 24×36 (portrait + landscape)
DPI options: 72, 150, 200, 300 — 200 DPI is fine for 18×24" and larger (viewed from distance)

---

Progress bars

All AI operations show a real-time progress overlay. For local GPU inference, the bar advances step-by-step as the model denoises (e.g. "Step 14 / 30"). For cloud providers and upscale operations, it animates to indicate activity.

---

Logs

# GPU container:
docker compose -f docker-compose.gpu.yml logs -f

# Standard container:
docker compose logs -f

---

File structure

EditmaskwithAI/
├── backend/
│   ├── app/
│   │   ├── routers/         # API endpoints (ai_tools, print_tools, …)
│   │   ├── services/        # gpu_detect, local_diffusion, upscale, …
│   │   └── config.py
│   ├── requirements.txt
│   └── requirements.gpu.txt
├── frontend/
│   └── src/js/
│       ├── tools/           # brush_select (SAM paint), smart_select, …
│       ├── modules/
│       │   ├── generate/    # text_to_image, outpaint
│       │   └── image/       # upscale, frame_fit, print_prepare, …
│       └── libs/
│           └── progress_overlay.js
├── docker-compose.yml           # Cloud / no-GPU
├── docker-compose.gpu.yml       # NVIDIA GPU (recommended)
├── docker-compose.dev.yml       # Dev with hot reload
├── Dockerfile
├── Dockerfile.gpu
└── .env.example

---

Troubleshooting

GPU not detected in Docker

# Check toolkit is installed and Docker restarted:
docker run --rm --gpus all nvidia/cuda:12.1.0-base-ubuntu22.04 nvidia-smi
# If that fails, re-run: sudo nvidia-ctk runtime configure --runtime=docker && sudo systemctl restart docker

Model download stalls or fails

# Check logs for HuggingFace errors:
docker compose -f docker-compose.gpu.yml logs -f | grep -E "local_gpu|Error|Failed"
# If a private/gated model: add HF_TOKEN=hf_... to .env

SAM model fails to download (DNS error / firewall blocking port 53)

If the container can't reach dl.fbaipublicfiles.com (you'll see Errno -3 Name or service not known in the logs), download SAM directly on the host and let the bind mount make it visible to the container — no rebuild needed:

mkdir -p ./data/models
# sudo needed if ./data/ was created by Docker (root-owned):
sudo curl -L -o ./data/models/sam_vit_b_01ec64.pth \
  https://dl.fbaipublicfiles.com/segment_anything/sam_vit_b_01ec64.pth

The file is ~375 MB. Once it exists at ./data/models/sam_vit_b_01ec64.pth, the container picks it up on the next startup (no rebuild required). Verify with:

docker compose -f docker-compose.gpu.yml logs | grep -i sam
# Should show: "SAM model loaded on cuda" (or cpu)

If Docker created ./data/ as root and you can't write there without sudo, you can also use root's curl as above — the container reads the file regardless of owner.

AI Edit returns "model files not yet downloaded" or "Errno -3 / DNS" error

The container's DNS is blocked (common on corporate networks or custom iptables rules), so it can't download SDXL models from HuggingFace. Two options:

Option A — fix Docker DNS (recommended, one command):

sudo iptables -I DOCKER-USER -p udp --dport 53 -j ACCEPT
docker compose -f docker-compose.gpu.yml restart

Option B — pre-download models on the host (if iptables fix isn't possible):

pip install huggingface-hub

# Download the inpainting model (~6.5 GB, needed for AI Edit):
huggingface-cli download diffusers/stable-diffusion-xl-1.0-inpainting-0.1 \
  --cache-dir ./data/hf_cache \
  --exclude "*.msgpack" "flax_*" "tf_*"

# Download the text-to-image model (~6.5 GB, needed for Text → Image):
huggingface-cli download stabilityai/stable-diffusion-xl-base-1.0 \
  --cache-dir ./data/hf_cache \
  --exclude "*.msgpack" "flax_*" "tf_*"

The models land in ./data/hf_cache/ which is bind-mounted into the container — no rebuild needed. Restart the container and the first AI Edit request loads from local disk.

Out of VRAM during generation

• Reduce LOCAL_GPU_MAX_PIPELINES=1 in .env (default 2)
• Or override to a smaller model: HF_MODEL_TXT2IMG=runwayml/stable-diffusion-v1-5

Settings saved locally only

• The in-app AI Provider Settings dialog saves to localStorage for the session
• To make settings permanent: edit .env and rebuild

Check API docs

http://localhost:3080/api/docs