ai-stack: offer an optional vision-capable Ollama model, including moondream
None of local-ai-setup.sh's tier-selected models (CHAT_MODEL/CODE_MODEL/
EMBED_MODEL) can read an image — there was no way to get vision support out
of this stack at all before now. Added a numbered pick-list to the
generated pull-models.sh, right after the existing DeepSeek-R1 optional
pull, matching that same read -rp pattern:
1) moondream ~1.7 GB by Moondream AI — tiny, built for
CPU-only or weak/old-GPU hardware
2) llava:7b ~4.7 GB general-purpose vision
3) qwen2.5vl:7b ~6 GB stronger accuracy, more RAM/VRAM
4) llama3.2-vision:11b ~7.9 GB heaviest of the four
moondream is the recommended default — sized for exactly the "6 vCPU, 8GB
RAM, no GPU" case this was asked for, unlike the other three which assume
real GPU/RAM headroom.
Verified by actually running the heredoc that generates pull-models.sh
(with EMBED_MODEL/CHAT_MODEL/CODE_MODEL stood in) and syntax-checking the
resulting output script, not just the source — the outer heredoc is
unquoted so $-escaping mistakes wouldn't show up as a bash -n failure on
local-ai-setup.sh itself, only on what it generates.
services/ai-stack.md gets a matching "Vision models" section (sizes, the
manual pull command, and how to point an app's OPENAI_MODEL at one).
laptop_full_setup.sh's separate, non-interactive pull-models.sh generator
is untouched — it's not invoked anywhere in this repo's own install flow
(only local-ai-setup.sh is, from install_ai-stack()), so it's out of
scope here.
This commit is contained in:
@@ -34,6 +34,7 @@ install_ai-stack() {
|
||||
echo "[DRY-RUN] Would copy vendored source $SRC_DIR -> $AS_DIR"
|
||||
echo "[DRY-RUN] Would optionally collect cloud LLM provider keys (Groq/DeepInfra/OpenAI/OpenRouter)"
|
||||
echo "[DRY-RUN] Would run the app installer local-ai-setup.sh (Docker/NVIDIA toolkit, VRAM-aware models, generates compose/.env, starts stack, registers systemd 'local-ai')"
|
||||
echo "[DRY-RUN] Would offer an optional vision-capable model to pull (moondream/llava/qwen2.5vl/llama3.2-vision) via the generated pull-models.sh"
|
||||
echo "[DRY-RUN] Would wire cloud providers into Open WebUI (OPENAI_API_BASE_URLS) preserving the local RAG connection"
|
||||
echo "[DRY-RUN] Would write gpu-mode.sh and optionally enable the GPU switcher (one small GPU shared by Ollama and InvokeAI/ComfyUI)"
|
||||
echo "[DRY-RUN] Would attach Open WebUI to caddy_net and configure Caddy (open-webui:8080, host port 3000)"
|
||||
|
||||
Reference in New Issue
Block a user