root/ubuntu-post-install
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

ai-stack: offer an optional vision-capable Ollama model, including moondream

Browse Source 
None of local-ai-setup.sh's tier-selected models (CHAT_MODEL/CODE_MODEL/
EMBED_MODEL) can read an image — there was no way to get vision support out
of this stack at all before now. Added a numbered pick-list to the
generated pull-models.sh, right after the existing DeepSeek-R1 optional
pull, matching that same read -rp pattern:

  1) moondream            ~1.7 GB  by Moondream AI — tiny, built for
                                    CPU-only or weak/old-GPU hardware
  2) llava:7b             ~4.7 GB  general-purpose vision
  3) qwen2.5vl:7b         ~6 GB    stronger accuracy, more RAM/VRAM
  4) llama3.2-vision:11b  ~7.9 GB  heaviest of the four

moondream is the recommended default — sized for exactly the "6 vCPU, 8GB
RAM, no GPU" case this was asked for, unlike the other three which assume
real GPU/RAM headroom.

Verified by actually running the heredoc that generates pull-models.sh
(with EMBED_MODEL/CHAT_MODEL/CODE_MODEL stood in) and syntax-checking the
resulting output script, not just the source — the outer heredoc is
unquoted so $-escaping mistakes wouldn't show up as a bash -n failure on
local-ai-setup.sh itself, only on what it generates.

services/ai-stack.md gets a matching "Vision models" section (sizes, the
manual pull command, and how to point an app's OPENAI_MODEL at one).
laptop_full_setup.sh's separate, non-interactive pull-models.sh generator
is untouched — it's not invoked anywhere in this repo's own install flow
(only local-ai-setup.sh is, from install_ai-stack()), so it's out of
scope here.
This commit is contained in:
Claude
2026-08-21 03:09:07 +00:00
parent ebe8ea3245
commit ac76ef5181
3 changed files with 42 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
services/ai-stack.md
+20
View File
@@ -39,6 +39,26 @@ bash pull-models.sh # pull Ollama models (run once after first install)
```
Also a systemd unit: `sudo systemctl {start,stop,status} local-ai`
## Vision models (image understanding)
None of the tier-selected chat/code models above can read an image. `pull-models.sh`
offers one optional vision model at the end — pick it there, or pull one manually
any time:
```bash
docker exec ollama ollama pull moondream # or llava:7b / qwen2.5vl:7b / llama3.2-vision:11b
```
| Model | Size | Notes |
|-------|------|-------|
| `moondream` | ~1.7 GB | By Moondream AI — tiny, built for CPU-only or weak/old-GPU hardware. Best default if you don't have a real GPU. |
| `llava:7b` | ~4.7 GB | General-purpose vision, moderate resources. |
| `qwen2.5vl:7b` | ~6 GB | Stronger accuracy, needs more RAM/VRAM. |
| `llama3.2-vision:11b` | ~7.9 GB | Meta's vision model — heaviest of these four. |
Point any OpenAI-compatible app's vision/image-import feature (e.g. Mealie's
"import recipe from photo") at this stack's Ollama endpoint with the pulled
model as `OPENAI_MODEL` — see Open WebUI → Settings → Connections for the
exact local base URL, or `docker inspect ollama` for the container's address
on `caddy_net`/the compose network.
## Cloud LLM providers (Open WebUI)
Open WebUI uses an OpenAI-compatible connection list. The local RAG server is the
first entry; any cloud providers added at install follow it. Two semicolon-separated