Context management for local models with limited windows:
- Context Tracker (community function): shows tokens used vs available,
progress bar, percentage remaining — so you see the cliff coming
- Checkpoint Summarization Filter: auto-summarizes old messages when
context fills up, like Claude's auto-compaction
- Both added as recommended post-install links in setup output
(Open WebUI Functions install with one click from the UI)
6GB GPU tier (Quadro P3300, GTX 1060, etc.):
- Qwen 3.5 4B at Q4_K_M = ~2.5GB weights, leaves 3.5GB for KV cache
- With Q8 KV cache: ~32K usable context on 6GB
- Better than squeezing 9B into nothing — more context > slightly smarter
https://claude.ai/code/session_01PtYTPherSJaxDEVPgF6Nxu
The model can now search before it generates:
MCP tools (available via Open WebUI + Claude Code):
- search_docs(query): searches Kiwix ZIM files (Wikipedia, Stack Overflow,
DevDocs, Arch Wiki) — instant, offline, no rate limits
- read_doc(path): reads full article content from Kiwix results
- web_search(query): DuckDuckGo search, no API key needed
Open WebUI native search:
- ENABLE_RAG_WEB_SEARCH=true + RAG_WEB_SEARCH_ENGINE=duckduckgo
- Switched from SearXNG (not in stack) to DDG (zero config)
Search priority: Kiwix first (offline, fast) → DDG fallback (live web)
All 3 setup scripts updated:
- duckduckgo-search added to mcp_requirements.txt
- KIWIX_URL=http://kiwix:80 added to MCP container env
- curl added to MCP container deps (for sync script)
- Open WebUI DDG search enabled by default
https://claude.ai/code/session_01PtYTPherSJaxDEVPgF6Nxu
ubuntu-post-install.sh now embeds both companion scripts as single-quoted
heredocs written to mktemp files at runtime. No external files, git clone,
or internet access needed for [A] Local AI stack or [Z] Download ZIMs —
the single script is the only file required.
local-ai-setup.sh → embedded at lines 1084-1922 (LOCAL_AI_SETUP_EOF)
kiwix_download.sh → embedded at lines 2022-2221 (KIWIX_DOWNLOAD_EOF)
https://claude.ai/code/session_012gDnantBmFTWZGCiKyjazx
Adds two new top-level menu choices to the installation mode selector:
[A] Local AI stack — finds or clones local-ai-setup.sh, asks which
mounted drive to use for Ollama models and Kiwix ZIM files, runs
the installer as the actual user, then patches docker-compose.yml
for alternate storage if selected.
[Z] Download ZIMs — finds or clones kiwix_download.sh, shows
mounted drives, optionally redirects the download via KIWIX_DIR,
then runs the downloader as the actual user in the background.
Both functions locate their scripts alongside ubuntu-post-install.sh
first, then ~/local-ai/, then fall back to cloning outis1one/local-ai.
https://claude.ai/code/session_012gDnantBmFTWZGCiKyjazx
Claude