Research and document the $850 budget build: 2x Quadro RTX 5000 (16GB each)
connected via NVLink for 32GB unified VRAM. With Qwen 3.5's Gated Delta
Network architecture (Feb 2026), this setup runs the 35B-A3B MoE model
with full 262K context in ~25GB — the best price-to-capability ratio
for local AI coding available.
Includes:
- Exact NVLink bridge part numbers (RTX 5000 uses unique smaller connector)
- Motherboard/PSU requirements and slot spacing guidance
- llama.cpp and Ollama multi-GPU configuration
- VRAM budget calculations for all Qwen 3.5 model sizes
- Phased build plan (start with 1 card at $400, add second later)
- Updated model table with full Qwen 3.5 family specs
- Cost comparison vs RTX 8000, RTX 3090, Claude Max, and API pricing
https://claude.ai/code/session_01PtYTPherSJaxDEVPgF6Nxu
The $750 listing was a single outlier. Actual market price for
Quadro RTX 8000 Passive 48GB is $2,000-2,900 on eBay. Updated
all recommendations and cost comparisons accordingly.
https://claude.ai/code/session_01PtYTPherSJaxDEVPgF6Nxu
- Added full 48GB GPU market comparison (RTX 8000, A40, A6000, L40, RTX 6000 Ada)
- Quadro RTX 8000 Passive at $750-1,400 is 4-5x cheaper than alternatives
- Added RTX 8000 LLM benchmarks (34 t/s on 30B models at 8K context)
- Explained why 48GB >> 24GB for coding: context window is the bottleneck
- Added 2026 coding model landscape (Qwen3.5 27B, Qwen3-Coder, etc.)
- Revised recommendations: RTX 8000 as primary, dual P40 as budget alt
- Updated config notes for 48GB (32K context, higher quantization options)
- All prices verified from real listings as of March 22, 2026
https://claude.ai/code/session_01PtYTPherSJaxDEVPgF6Nxu
- Corrected all GPU prices to actual eBay/Newegg listings as of March 22, 2026
- Added analysis of 32B coding models (Qwen2.5-Coder 32B, Qwen3.5 27B) on 24GB VRAM
- Added honest comparison of local LLM quality vs Claude Code
- Revised recommendations: dual P40 ($400-500) or P40+T4 ($400-550)
- Added configuration notes for 32B models, dual-GPU, and newer Qwen3 models
- RTX A4000 at $700+ is too expensive for this build
https://claude.ai/code/session_01PtYTPherSJaxDEVPgF6Nxu
Research comparing P40, A2000, T4, and M40 GPUs for LLM inference
and image generation in Dell R720/R730 rack servers. Includes
benchmarks, compatibility notes, pricing, and recommendations.
https://claude.ai/code/session_01PtYTPherSJaxDEVPgF6Nxu
Replace subshell ls expansion (which produces no args when /data is empty)
with a conditional: serve ZIM files if they exist, otherwise sleep infinity
so the container stays up gracefully until ZIM files are added.
https://claude.ai/code/session_012gDnantBmFTWZGCiKyjazx
SearXNG:
- Dropped from both setup scripts and docker-compose (Google/Startpage
block self-hosted instances by IP — not reliable enough to include)
- Removed all interactive safe-search and engine-selection prompts
- Removed Open WebUI RAG web-search env vars (ENABLE_RAG_WEB_SEARCH,
SEARXNG_QUERY_URL, etc.)
- Removed port 8888 from UFW rules, start.sh URLs, done output, and
the Caddyfile template
- Removed searxng/ from .gitignore (directory no longer created)
- configure-searxng-safesearch.sh kept in repo for optional manual use
Kiwix:
- Replace the blocking wait-loop (`until ls *.zim`) with a one-liner
that passes whatever ZIM files exist (or none) directly to kiwix-serve,
so the container starts immediately and shows an empty library page
rather than hanging until ZIMs are downloaded
https://claude.ai/code/session_012gDnantBmFTWZGCiKyjazx
All scripts now default BASE to their own directory (SCRIPT_DIR) instead
of ~/docker/ai-stack, so after a git clone the user never needs to change
folders — docker-compose.yml, .env, settings, and data dirs all live next
to the scripts.
configure-searxng-safesearch.sh and configure-storage.sh do the same for
standalone use (still honoured when called with BASE= from a parent script).
docker-compose.yml (generated by both setup scripts) gains a comment block
at the top listing the everyday docker compose commands:
up -d / down / restart / stop / logs -f / pull / ps
so the file itself is the reference for managing containers.
.gitignore added to exclude generated files (docker-compose.yml, .env,
requirements.txt, helper scripts) and data directories (workspace/, kiwix/,
searxng/, gitea/, etc.) from git tracking.
https://claude.ai/code/session_012gDnantBmFTWZGCiKyjazx
mcp_server.py / local-ai-setup.sh:
FastMCP.get_asgi_app() was removed in mcp 1.x — replace with
sse_app(), which returns the same Starlette ASGI app for SSE
transport. Fixes the crash loop the container was stuck in.
laptop_full_setup.sh Caddyfile:
$(hostname).local requires mDNS (Avahi/Bonjour) to resolve from a
proxy machine, which is often not available on all LAN clients.
Replace every occurrence with $LOCAL_IP (the machine's LAN IP)
so the generated Caddyfile.example works reliably regardless of
mDNS support.
https://claude.ai/code/session_012gDnantBmFTWZGCiKyjazx
Adds invokeai-import-lora.sh to copy LoRA .safetensors files directly
into InvokeAI's Docker model volume, bypassing the greyed-out UI upload
buttons. Also adds README documentation for LoRA usage and troubleshooting.
https://claude.ai/code/session_01RU7NQuTbA8S8NRoWojvhR5
SearXNG's preferences.lock list prevents users from changing a setting
via the UI — it greys out the option. Without this, safe_search in
settings.yml was only a server default that users could still override
in their preferences page.
For moderate/strict, add:
preferences:
lock:
- safesearch
For none, the lock is omitted so users can freely adjust it.
https://claude.ai/code/session_012gDnantBmFTWZGCiKyjazx
set +u was added around the ENGINE_BLOCK for-loops but the reporting
lines after ok "Updated settings.yml" still ran under set -u.
bash < 4.4 throws "unbound variable" for ${#ENABLE_MAP[@]} when the
associative array is empty (no --enable-engines argument was given).
Add set +u before the Disabled/Re-enabled info lines; it already covers
the ${#DISABLE_MAP[@]} reference at the end of the script.
https://claude.ai/code/session_012gDnantBmFTWZGCiKyjazx
Root cause of all 85 engines being disabled:
`3>&1 1>&2 2>&3 2>/dev/null` — the trailing `2>/dev/null` overwrites FD2
with /dev/null AFTER the FD swap, destroying the pipe that whiptail writes
selections to. User selections were silently discarded every time. Remove it.
Also: if whiptail exits 0 but returns empty string (all items unchecked),
fall back to pre-checked defaults rather than disabling everything.
ENABLE_MAP: unbound variable (configure-searxng-safesearch.sh):
bash < 4.4 treats `${!empty_assoc[@]}` as unbound under `set -u`.
Wrap for loops and mark_disabled key-check with set +u / set -u.
Add artic (Art Institute of Chicago) to image engine menus.
Add yandex images (defaults OFF for moderate/strict, same as yandex).
Update _SX_AUTOFF to include "yandex images" so re-enabling it works.
Update configure script engine category lists to match expanded menus.
https://claude.ai/code/session_012gDnantBmFTWZGCiKyjazx
- Radiolist first: Choose / All / Skip (ESC = skip)
- Checklist shows all 18 ZIMs with sizes; pre-checks any already
downloaded ZIMs so re-runs don't re-select what you have
- ZIM_PICKS now stores space-separated names (wikipedia, stackoverflow,
archlinux...) instead of numbers; dispatch updated to match
- Text fallback (no whiptail): same numbered list as before, converts
input numbers to names before dispatch
- Summary shows count and names of selected ZIMs
https://claude.ai/code/session_012gDnantBmFTWZGCiKyjazx
For both setup scripts and configure-searxng-safesearch.sh:
- Five whiptail checklist menus: Web · Images · Videos · News · Science
- Pre-populated ON/OFF based on the chosen safe-search level:
none → all engines ON
moderate/strict → no-safe-search engines (mojeek, yandex, baidu, naver,
invidious, piped, peertube, sepiasearch, flickr,
imgur, deviantart, dailymotion, vimeo) default OFF
- User can toggle any engine independently before confirming
- ESC on any menu restores defaults for that category (keeps them unchanged)
- If user re-enables an auto-disabled engine, --enable-engines is passed
to configure script to override the auto-disable
- Text fallback for non-interactive/headless installs
- Summary shows count of disabled engines instead of raw list
https://claude.ai/code/session_012gDnantBmFTWZGCiKyjazx
Setup (both scripts):
- Q6 asks safe-search level: none / moderate / strict
- Then asks which categories to disable: videos images news science social
- Then asks which engines to disable by name (duckduckgo, bing, etc.)
- Choices flow into SEARXNG_QUERY_URL &safesearch=N in docker-compose.yml
- SearXNG summary line added to laptop_full_setup.sh install plan
configure-searxng-safesearch.sh:
- Full rewrite: --disable-categories, --disable-engines, --enable-engines
- Category maps: videos / images / news / science / social engine lists
- --enable-engines overrides auto-disables (e.g. keep yandex on strict)
- Preserves existing secret key on update
- Creates settings.yml from scratch if missing (safe for setup use)
- Help flag (-h/--help)
https://claude.ai/code/session_012gDnantBmFTWZGCiKyjazx
- New configure-searxng-safesearch.sh: set strict/moderate/none level,
disable engines that can't enforce the chosen level (torrent sites,
mojeek, baidu, yandex, nvidious/piped/peertube video frontends),
updates &safesearch= in SEARXNG_QUERY_URL, and restarts searxng
- Add RAG_WEB_SEARCH_RESULT_COUNT=5 and RAG_WEB_SEARCH_CONCURRENT_REQUESTS=10
to Open WebUI env in both setup scripts
- Add &safesearch=0 to SEARXNG_QUERY_URL so the script can find/replace it
- Sync ENABLE_TOOL_SERVERS=true into laptop_full_setup.sh (was missing)
https://claude.ai/code/session_012gDnantBmFTWZGCiKyjazx
- Add WEBUI_URL to .env template and open-webui compose env so sessions
work correctly when served behind Caddy (or any reverse proxy)
- Fix Caddyfile.example to pass X-Forwarded-Proto/Host headers for Open WebUI
- Wrap kiwix-serve command in a wait loop so the container stays healthy
while ZIM files are still downloading (avoids 502 from Caddy)
https://claude.ai/code/session_012gDnantBmFTWZGCiKyjazx
When Docker containers write config files (e.g. searxng/settings.yml),
those files end up owned by root. write_if_new now falls back to
sudo tee when the destination file exists but is not writable.
https://claude.ai/code/session_012gDnantBmFTWZGCiKyjazx
- The "select which ZIMs" menu now fires during Q4 (upfront wizard),
alongside the yes/select/no choice, not after docker/models start
- Each ZIM entry shows [downloaded] if a matching .zim already exists
in KIWIX_DIR — skips re-downloading automatically via dl_zim check
- Download phase now uses the pre-captured ZIM_PICKS directly,
no interactive prompt mid-install
https://claude.ai/code/session_012gDnantBmFTWZGCiKyjazx
Both [2/6] and [3/6] prompts now show detected mount points as a
numbered list. Type 0 for default/Docker volume, or 1/2/3 to pick a
drive directly. Typing a path manually still works as fallback.
https://claude.ai/code/session_012gDnantBmFTWZGCiKyjazx
The tier table now shows 1–4 numbers so you type 1/2/3/4 instead of
typing "14B" or "32B". Tier names are still accepted as fallback.
Recommended tier is shown as a number (e.g. [2] instead of [14B]).
https://claude.ai/code/session_012gDnantBmFTWZGCiKyjazx
All interactive prompts now use numbered toggles (type a number to
toggle on/off, Enter to confirm) instead of requiring whiptail.
Removed whiptail from apt dependencies in ubuntu-post-install.sh.
https://claude.ai/code/session_012gDnantBmFTWZGCiKyjazx
- Top-level menu now just: Full system setup / AI stack
- AI stack opens a second whiptail checklist (same style as
ubuntu-post-install.sh) where each service is individually toggled:
Ollama (always on, core)
Open WebUI · RAG+ChromaDB · MCP Server · SearXNG
Gitea · InvokeAI · Portainer · Kiwix
- Kiwix moved from top-level into the AI stack service list
- Existing containers auto-detected and pre-ticked ON
- MCP dependency on RAG enforced automatically with a warning
- COMPOSE_SERVICES, Done URLs, and summary all driven by per-service
SVC_* flags so only selected services are started/shown
- Text toggle fallback for systems without whiptail
https://claude.ai/code/session_012gDnantBmFTWZGCiKyjazx
Replace fixed model defaults with an interactive wizard:
Origin preference:
1) Western-only — Codestral (Mistral 🇫🇷) · Phi4 · Mistral 7B
2) Performance-first — Qwen2.5 + Qwen2.5-Coder (top benchmarks)
3) Mixed — Western chat/reasoning, Qwen for coding only
4) Custom — free-form model names
Size tier table — speed estimated for the user's actual VRAM with no
hard limits (Ollama already uses all available VRAM via NUM_GPU=999):
✓ fast — fully in VRAM
~ good — fits with small overhang (~reading speed)
✗ slow — partial CPU offload
✗ very slow — heavy CPU offload
Recommended tier shown as suggestion based on VRAM, user can override.
pull-models.sh now bakes in the chosen model names at install time.
REASON_MODEL replaces hardcoded deepseek-r1:14b; skipped if empty or
same as chat model.
https://claude.ai/code/session_012gDnantBmFTWZGCiKyjazx
- Docker + Docker Compose check now runs on every invocation (not skipped
on updates), installing if missing — ensures the stack can always run
- Added Docker Compose plugin install as fallback if compose is missing
- NVIDIA Container Toolkit check also always runs (not update-gated)
- Added SSH key import wizard question: import public keys from GitHub
(gh:username) and/or Launchpad (lp:username) using ssh-import-id,
so the machine is accessible over SSH immediately after setup
https://claude.ai/code/session_012gDnantBmFTWZGCiKyjazx
Replace the single-choice component menu with a multi-select checklist
(whiptail if available, text-based toggle fallback). Users now pick any
combination of:
[*] Full system setup — runs ubuntu-post-install.sh (Ubuntu apps,
security hardening, backups, Docker services)
[*] AI stack — Ollama, Open WebUI, RAG, MCP, Gitea, InvokeAI
[*] Kiwix — Offline Wikipedia, Stack Overflow, Arch Wiki
This makes laptop_full_setup.sh the single entry point — run it once,
toggle what you need (like Ubuntu Server's tasksel), leave overnight.
https://claude.ai/code/session_012gDnantBmFTWZGCiKyjazx
Claude