Convert all setup prompts to whiptail, fix VRAM estimates, add model expectations
Setup script changes: - All prompts now use whiptail dialogs with text fallback - Q1b (SSH), Q2 (storage), Q3 (Kiwix), Q4b (firewall), Q5 (models), Q6 (download), final confirm all converted - Model tier selection uses radiolist with recommended tier pre-selected - Custom model entry uses inputbox with current defaults pre-filled - Fix speed_label: now shows actual VRAM needed (file size + 2GB overhead) instead of misleading "fully in VRAM" for models that don't fit - qwen3.5-35b-a3b MoE already in tier list (was there, now with accurate VRAM estimate shown) README changes: - Add "Realistic expectations by model size" table - 35B MoE highlighted as sweet spot for small GPUs https://claude.ai/code/session_01PtYTPherSJaxDEVPgF6Nxu
This commit is contained in:
Claude
@@ -575,6 +575,29 @@ Local AI requires more manual workflow management but has real code awareness vi
|
|||||||
|
|
||||||
Channels are persistent chat rooms (like Slack/Discord channels) with multi-model support. They do NOT scope memories differently — memories are still global per user. Channels are useful for team collaboration, not memory isolation.
|
Channels are persistent chat rooms (like Slack/Discord channels) with multi-model support. They do NOT scope memories differently — memories are still global per user. Channels are useful for team collaboration, not memory isolation.
|
||||||
|
|
||||||
|
### Realistic expectations by model size
|
||||||
|
|
||||||
|
Not all models can do all tasks. Here's what to actually expect:
|
||||||
|
|
||||||
|
| Task | 4B (qwen3.5:4b) | 9B (qwen3.5:9b) | 35B MoE (qwen3.5-35b-a3b) | 14B+ dense |
|
||||||
|
|------|:---:|:---:|:---:|:---:|
|
||||||
|
| Answer simple questions | OK | Good | Good | Good |
|
||||||
|
| Explain existing code (with RAG) | OK | Good | Good | Good |
|
||||||
|
| Fix a simple bug (typo, off-by-one) | Maybe | Usually | Usually | Yes |
|
||||||
|
| Write a small utility function | Shaky | OK | Good | Good |
|
||||||
|
| Fix logic error across 2-3 functions | No | Maybe | Usually | Usually |
|
||||||
|
| Write a new feature (multiple files) | No | Shaky | Maybe | Maybe |
|
||||||
|
| Refactor with style consistency | No | No | Sometimes | Sometimes |
|
||||||
|
| Summarize a conversation for handoff | OK | Good | Good | Good |
|
||||||
|
|
||||||
|
**The 35B MoE model (`qwen3.5-35b-a3b`) is the sweet spot for small GPUs.** It was trained as a 35B model but only activates 3B parameters per token. This means it has the *knowledge* of a 35B model with the VRAM footprint closer to a 4B. On a 6GB card it may fit (VRAM usage varies with context length and KV cache settings).
|
||||||
|
|
||||||
|
**Bottom line for a 6GB GPU:**
|
||||||
|
- Use `qwen3.5:4b` for quick chat, explanations, and summarization
|
||||||
|
- Try `qwen3.5-35b-a3b` for code tasks — if it fits, it will be significantly better than 4B
|
||||||
|
- Use Claude Code for anything that requires reading/writing multiple files or complex reasoning
|
||||||
|
- The RAG server helps a lot — even a 4B model gives useful answers when it has the right code chunks in context
|
||||||
|
|
||||||
### Fixing models that output code instead of natural language
|
### Fixing models that output code instead of natural language
|
||||||
|
|
||||||
If your model (especially smaller ones like Qwen 3.5) responds with Python code blocks instead of plain English answers (as shown in the screenshot), this is a common behavior with code-optimized models.
|
If your model (especially smaller ones like Qwen 3.5) responds with Python code blocks instead of plain English answers (as shown in the screenshot), this is a common behavior with code-optimized models.
|
||||||
|
|||||||
+223
-103
@@ -217,15 +217,21 @@ fi
|
|||||||
SSH_IMPORT_IDS=() # list of "gh:username" or "lp:username" entries
|
SSH_IMPORT_IDS=() # list of "gh:username" or "lp:username" entries
|
||||||
|
|
||||||
if ! $IS_UPDATE || [[ ! -f "$HOME/.ssh/authorized_keys" ]]; then
|
if ! $IS_UPDATE || [[ ! -f "$HOME/.ssh/authorized_keys" ]]; then
|
||||||
echo ""
|
if command -v whiptail &>/dev/null; then
|
||||||
echo -e " ${BOLD}[SSH] Import SSH public keys? (for passwordless SSH into this machine)${NC}"
|
SSH_INPUT=$(whiptail --title "SSH Key Import" \
|
||||||
echo " Pulls your public keys from GitHub or Launchpad and adds them to"
|
--inputbox "Import SSH public keys for passwordless SSH into this machine.\n\nExamples: gh:yourusername lp:yourlaunchpadid\nMultiple: gh:alice lp:alice\n\nLeave blank to skip." \
|
||||||
echo " ~/.ssh/authorized_keys using ssh-import-id."
|
14 68 "" 3>&1 1>&2 2>&3) || SSH_INPUT=""
|
||||||
echo ""
|
else
|
||||||
echo " Examples: gh:yourusername lp:yourlaunchpadid"
|
echo ""
|
||||||
echo " Multiple: gh:alice lp:alice"
|
echo -e " ${BOLD}[SSH] Import SSH public keys? (for passwordless SSH into this machine)${NC}"
|
||||||
echo ""
|
echo " Pulls your public keys from GitHub or Launchpad and adds them to"
|
||||||
read -rp " Usernames (or Enter to skip): " SSH_INPUT
|
echo " ~/.ssh/authorized_keys using ssh-import-id."
|
||||||
|
echo ""
|
||||||
|
echo " Examples: gh:yourusername lp:yourlaunchpadid"
|
||||||
|
echo " Multiple: gh:alice lp:alice"
|
||||||
|
echo ""
|
||||||
|
read -rp " Usernames (or Enter to skip): " SSH_INPUT
|
||||||
|
fi
|
||||||
if [[ -n "$SSH_INPUT" ]]; then
|
if [[ -n "$SSH_INPUT" ]]; then
|
||||||
read -ra SSH_IMPORT_IDS <<< "$SSH_INPUT"
|
read -ra SSH_IMPORT_IDS <<< "$SSH_INPUT"
|
||||||
fi
|
fi
|
||||||
@@ -236,10 +242,6 @@ OLLAMA_STORAGE="volume" # "volume" = Docker named volume, else a host path
|
|||||||
OLLAMA_HOST_PATH=""
|
OLLAMA_HOST_PATH=""
|
||||||
|
|
||||||
if $INSTALL_AI; then
|
if $INSTALL_AI; then
|
||||||
echo ""
|
|
||||||
echo -e " ${BOLD}[2/6] Where should Ollama models be stored?${NC}"
|
|
||||||
echo " (Models are large — 5-50GB each. A fast SSD or large HDD is ideal.)"
|
|
||||||
echo ""
|
|
||||||
mapfile -t _MPTS < <(
|
mapfile -t _MPTS < <(
|
||||||
df -h --output=target,avail,fstype 2>/dev/null \
|
df -h --output=target,avail,fstype 2>/dev/null \
|
||||||
| awk 'NR>1 && $2~/[0-9]/ {
|
| awk 'NR>1 && $2~/[0-9]/ {
|
||||||
@@ -248,37 +250,58 @@ if $INSTALL_AI; then
|
|||||||
if ((unit=="G" && num>=20) || unit=="T") print $0
|
if ((unit=="G" && num>=20) || unit=="T") print $0
|
||||||
}' | head -10
|
}' | head -10
|
||||||
)
|
)
|
||||||
echo " 0) Docker volume (default — /var/lib/docker/volumes/)"
|
|
||||||
for _i in "${!_MPTS[@]}"; do
|
if command -v whiptail &>/dev/null; then
|
||||||
printf " %d) %s\n" "$((_i+1))" "${_MPTS[$_i]}"
|
WHIP_STORAGE=("volume" "Docker volume (default — /var/lib/docker/volumes/)" ON)
|
||||||
done
|
for _i in "${!_MPTS[@]}"; do
|
||||||
echo ""
|
_mp_path=$(awk '{print $1}' <<< "${_MPTS[$_i]}")
|
||||||
read -rp " Choice [0]: " STORAGE_CHOICE
|
_mp_avail=$(awk '{print $2}' <<< "${_MPTS[$_i]}")
|
||||||
STORAGE_CHOICE="${STORAGE_CHOICE:-0}"
|
WHIP_STORAGE+=("$_mp_path" "${_mp_path} (${_mp_avail} free)" OFF)
|
||||||
if [[ "$STORAGE_CHOICE" == "0" ]]; then
|
done
|
||||||
OLLAMA_STORAGE="volume"
|
STORAGE_CHOICE=$(whiptail --title "Ollama Model Storage" \
|
||||||
elif [[ "$STORAGE_CHOICE" =~ ^[0-9]+$ ]] && (( STORAGE_CHOICE >= 1 && STORAGE_CHOICE <= ${#_MPTS[@]} )); then
|
--radiolist "Models are large (5-50GB each). Choose a location.\nSPACE = select ENTER = confirm" \
|
||||||
_MP=$(awk '{print $1}' <<< "${_MPTS[$(( STORAGE_CHOICE-1 ))]}")
|
$((10 + ${#_MPTS[@]})) 72 $((1 + ${#_MPTS[@]})) \
|
||||||
OLLAMA_HOST_PATH="$_MP/ollama-models"
|
"${WHIP_STORAGE[@]}" 3>&1 1>&2 2>&3) || STORAGE_CHOICE="volume"
|
||||||
OLLAMA_STORAGE="bind"
|
STORAGE_CHOICE="${STORAGE_CHOICE//\"/}"
|
||||||
ok "Ollama models → $OLLAMA_HOST_PATH"
|
if [[ "$STORAGE_CHOICE" == "volume" ]]; then
|
||||||
|
OLLAMA_STORAGE="volume"
|
||||||
|
else
|
||||||
|
OLLAMA_HOST_PATH="${STORAGE_CHOICE}/ollama-models"
|
||||||
|
OLLAMA_STORAGE="bind"
|
||||||
|
ok "Ollama models → $OLLAMA_HOST_PATH"
|
||||||
|
fi
|
||||||
else
|
else
|
||||||
# Fallback: treat input as a literal path
|
echo ""
|
||||||
OLLAMA_HOST_PATH="${STORAGE_CHOICE%/}"
|
echo -e " ${BOLD}[2/6] Where should Ollama models be stored?${NC}"
|
||||||
[[ -z "$OLLAMA_HOST_PATH" ]] && die "No path entered."
|
echo " (Models are large — 5-50GB each. A fast SSD or large HDD is ideal.)"
|
||||||
OLLAMA_STORAGE="bind"
|
echo ""
|
||||||
|
echo " 0) Docker volume (default — /var/lib/docker/volumes/)"
|
||||||
|
for _i in "${!_MPTS[@]}"; do
|
||||||
|
printf " %d) %s\n" "$((_i+1))" "${_MPTS[$_i]}"
|
||||||
|
done
|
||||||
|
echo ""
|
||||||
|
read -rp " Choice [0]: " STORAGE_CHOICE
|
||||||
|
STORAGE_CHOICE="${STORAGE_CHOICE:-0}"
|
||||||
|
if [[ "$STORAGE_CHOICE" == "0" ]]; then
|
||||||
|
OLLAMA_STORAGE="volume"
|
||||||
|
elif [[ "$STORAGE_CHOICE" =~ ^[0-9]+$ ]] && (( STORAGE_CHOICE >= 1 && STORAGE_CHOICE <= ${#_MPTS[@]} )); then
|
||||||
|
_MP=$(awk '{print $1}' <<< "${_MPTS[$(( STORAGE_CHOICE-1 ))]}")
|
||||||
|
OLLAMA_HOST_PATH="$_MP/ollama-models"
|
||||||
|
OLLAMA_STORAGE="bind"
|
||||||
|
ok "Ollama models → $OLLAMA_HOST_PATH"
|
||||||
|
else
|
||||||
|
OLLAMA_HOST_PATH="${STORAGE_CHOICE%/}"
|
||||||
|
[[ -z "$OLLAMA_HOST_PATH" ]] && die "No path entered."
|
||||||
|
OLLAMA_STORAGE="bind"
|
||||||
|
fi
|
||||||
fi
|
fi
|
||||||
unset _MPTS _MP _i
|
unset _MPTS _MP _i WHIP_STORAGE
|
||||||
fi
|
fi
|
||||||
|
|
||||||
# ── Q3: Storage for Kiwix ZIMs ────────────────────────────────────────────────
|
# ── Q3: Storage for Kiwix ZIMs ────────────────────────────────────────────────
|
||||||
KIWIX_DIR="$BASE/kiwix" # default
|
KIWIX_DIR="$BASE/kiwix" # default
|
||||||
|
|
||||||
if $SVC_KIWIX; then
|
if $SVC_KIWIX; then
|
||||||
echo ""
|
|
||||||
echo -e " ${BOLD}[3/6] Where should Kiwix ZIM files be stored?${NC}"
|
|
||||||
echo " (ZIMs are large — Wikipedia alone is ~46GB. Total collection ~130GB.)"
|
|
||||||
echo ""
|
|
||||||
mapfile -t _MPTS < <(
|
mapfile -t _MPTS < <(
|
||||||
df -h --output=target,avail,fstype 2>/dev/null \
|
df -h --output=target,avail,fstype 2>/dev/null \
|
||||||
| awk 'NR>1 && $2~/[0-9]/ {
|
| awk 'NR>1 && $2~/[0-9]/ {
|
||||||
@@ -287,22 +310,44 @@ if $SVC_KIWIX; then
|
|||||||
if ((unit=="G" && num>=50) || unit=="T") print $0
|
if ((unit=="G" && num>=50) || unit=="T") print $0
|
||||||
}' | head -10
|
}' | head -10
|
||||||
)
|
)
|
||||||
echo " 0) Default: $KIWIX_DIR"
|
|
||||||
for _i in "${!_MPTS[@]}"; do
|
if command -v whiptail &>/dev/null; then
|
||||||
printf " %d) %s\n" "$((_i+1))" "${_MPTS[$_i]}"
|
WHIP_KIWIX=("default" "Default: $KIWIX_DIR" ON)
|
||||||
done
|
for _i in "${!_MPTS[@]}"; do
|
||||||
echo ""
|
_mp_path=$(awk '{print $1}' <<< "${_MPTS[$_i]}")
|
||||||
read -rp " Choice [0]: " KIWIX_CHOICE
|
_mp_avail=$(awk '{print $2}' <<< "${_MPTS[$_i]}")
|
||||||
KIWIX_CHOICE="${KIWIX_CHOICE:-0}"
|
WHIP_KIWIX+=("$_mp_path" "${_mp_path} (${_mp_avail} free)" OFF)
|
||||||
if [[ "$KIWIX_CHOICE" != "0" ]] && [[ "$KIWIX_CHOICE" =~ ^[0-9]+$ ]] && (( KIWIX_CHOICE >= 1 && KIWIX_CHOICE <= ${#_MPTS[@]} )); then
|
done
|
||||||
_MP=$(awk '{print $1}' <<< "${_MPTS[$(( KIWIX_CHOICE-1 ))]}")
|
_KIWIX_SEL=$(whiptail --title "Kiwix ZIM Storage" \
|
||||||
KIWIX_DIR="$_MP/kiwix"
|
--radiolist "ZIMs are large — Wikipedia alone ~46GB, total ~130GB.\nSPACE = select ENTER = confirm" \
|
||||||
ok "Kiwix ZIMs → $KIWIX_DIR"
|
$((10 + ${#_MPTS[@]})) 72 $((1 + ${#_MPTS[@]})) \
|
||||||
elif [[ "$KIWIX_CHOICE" != "0" ]] && [[ -n "$KIWIX_CHOICE" ]]; then
|
"${WHIP_KIWIX[@]}" 3>&1 1>&2 2>&3) || _KIWIX_SEL="default"
|
||||||
# Fallback: treat as a literal path
|
_KIWIX_SEL="${_KIWIX_SEL//\"/}"
|
||||||
KIWIX_DIR="${KIWIX_CHOICE%/}"
|
if [[ "$_KIWIX_SEL" != "default" ]]; then
|
||||||
|
KIWIX_DIR="${_KIWIX_SEL}/kiwix"
|
||||||
|
ok "Kiwix ZIMs → $KIWIX_DIR"
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
echo ""
|
||||||
|
echo -e " ${BOLD}[3/6] Where should Kiwix ZIM files be stored?${NC}"
|
||||||
|
echo " (ZIMs are large — Wikipedia alone is ~46GB. Total collection ~130GB.)"
|
||||||
|
echo ""
|
||||||
|
echo " 0) Default: $KIWIX_DIR"
|
||||||
|
for _i in "${!_MPTS[@]}"; do
|
||||||
|
printf " %d) %s\n" "$((_i+1))" "${_MPTS[$_i]}"
|
||||||
|
done
|
||||||
|
echo ""
|
||||||
|
read -rp " Choice [0]: " KIWIX_CHOICE
|
||||||
|
KIWIX_CHOICE="${KIWIX_CHOICE:-0}"
|
||||||
|
if [[ "$KIWIX_CHOICE" != "0" ]] && [[ "$KIWIX_CHOICE" =~ ^[0-9]+$ ]] && (( KIWIX_CHOICE >= 1 && KIWIX_CHOICE <= ${#_MPTS[@]} )); then
|
||||||
|
_MP=$(awk '{print $1}' <<< "${_MPTS[$(( KIWIX_CHOICE-1 ))]}")
|
||||||
|
KIWIX_DIR="$_MP/kiwix"
|
||||||
|
ok "Kiwix ZIMs → $KIWIX_DIR"
|
||||||
|
elif [[ "$KIWIX_CHOICE" != "0" ]] && [[ -n "$KIWIX_CHOICE" ]]; then
|
||||||
|
KIWIX_DIR="${KIWIX_CHOICE%/}"
|
||||||
|
fi
|
||||||
fi
|
fi
|
||||||
unset _MPTS _MP _i _KIWIX_CHOICE
|
unset _MPTS _MP _i _KIWIX_CHOICE WHIP_KIWIX
|
||||||
|
|
||||||
# ── Q4: Download ZIMs now? ─────────────────────────────────────────────────
|
# ── Q4: Download ZIMs now? ─────────────────────────────────────────────────
|
||||||
# ON if any matching ZIM file already exists in KIWIX_DIR
|
# ON if any matching ZIM file already exists in KIWIX_DIR
|
||||||
@@ -447,11 +492,16 @@ fi
|
|||||||
# ── Q4b: Firewall (LAN subnet) ────────────────────────────────────────────────
|
# ── Q4b: Firewall (LAN subnet) ────────────────────────────────────────────────
|
||||||
LAN_SUBNET="192.168.1.0/24"
|
LAN_SUBNET="192.168.1.0/24"
|
||||||
if command -v ufw &>/dev/null && { [[ ! -f "$BASE/.ufw-done" ]] || $FORCE; }; then
|
if command -v ufw &>/dev/null && { [[ ! -f "$BASE/.ufw-done" ]] || $FORCE; }; then
|
||||||
echo ""
|
|
||||||
# Auto-detect likely subnet from current IP
|
|
||||||
AUTO_SUBNET=$(echo "$LOCAL_IP" | awk -F. '{print $1"."$2"."$3".0/24"}')
|
AUTO_SUBNET=$(echo "$LOCAL_IP" | awk -F. '{print $1"."$2"."$3".0/24"}')
|
||||||
echo -e " ${BOLD}[4b/6] Firewall — allow LAN access to services${NC}"
|
if command -v whiptail &>/dev/null; then
|
||||||
read -rp " LAN subnet [${AUTO_SUBNET}]: " LAN_INPUT
|
LAN_INPUT=$(whiptail --title "Firewall — LAN Access" \
|
||||||
|
--inputbox "Allow LAN access to all services.\n\nYour detected subnet:" \
|
||||||
|
10 60 "$AUTO_SUBNET" 3>&1 1>&2 2>&3) || LAN_INPUT=""
|
||||||
|
else
|
||||||
|
echo ""
|
||||||
|
echo -e " ${BOLD}[4b/6] Firewall — allow LAN access to services${NC}"
|
||||||
|
read -rp " LAN subnet [${AUTO_SUBNET}]: " LAN_INPUT
|
||||||
|
fi
|
||||||
LAN_SUBNET="${LAN_INPUT:-$AUTO_SUBNET}"
|
LAN_SUBNET="${LAN_INPUT:-$AUTO_SUBNET}"
|
||||||
[[ "$LAN_SUBNET" =~ /[0-9]+$ ]] || LAN_SUBNET="${LAN_SUBNET}/24"
|
[[ "$LAN_SUBNET" =~ /[0-9]+$ ]] || LAN_SUBNET="${LAN_SUBNET}/24"
|
||||||
fi
|
fi
|
||||||
@@ -462,35 +512,48 @@ REASON_MODEL=""
|
|||||||
|
|
||||||
if $INSTALL_AI; then
|
if $INSTALL_AI; then
|
||||||
|
|
||||||
# speed estimate based on Q4 model size vs available VRAM
|
# speed estimate: model_file_size + ~1.5GB overhead for KV cache/CUDA vs VRAM
|
||||||
# no hard limits — just honest labels so the user can choose
|
# Ollama has no hard limits — silently offloads to CPU when over VRAM
|
||||||
speed_label() {
|
speed_label() {
|
||||||
local mgb="$1" # approximate Q4 size in GB
|
local mgb="$1" # approximate Q4 file size in GB
|
||||||
|
local needed=$(( mgb + 2 )) # +2GB for KV cache, attention, CUDA overhead
|
||||||
if [[ "$VRAM_GB" -eq 0 ]]; then
|
if [[ "$VRAM_GB" -eq 0 ]]; then
|
||||||
printf "CPU only — very slow"
|
printf "CPU only — very slow"
|
||||||
elif (( mgb <= VRAM_GB )); then
|
elif (( needed <= VRAM_GB )); then
|
||||||
printf "✓ fast — fully in VRAM"
|
printf "✓ fast — fits in VRAM (~%dGB needed)" "$needed"
|
||||||
elif (( mgb <= VRAM_GB + 2 )); then
|
elif (( needed <= VRAM_GB + 2 )); then
|
||||||
printf "~ good — fits with small overhang (~reading speed)"
|
printf "~ tight — ~%dGB needed, may spill to CPU" "$needed"
|
||||||
elif (( mgb <= VRAM_GB + 8 )); then
|
elif (( needed <= VRAM_GB + 8 )); then
|
||||||
printf "✗ slow — partial CPU offload"
|
printf "✗ slow — ~%dGB needed, partial CPU offload" "$needed"
|
||||||
else
|
else
|
||||||
printf "✗ very slow — heavy CPU offload"
|
printf "✗ very slow — ~%dGB needed, heavy CPU offload" "$needed"
|
||||||
fi
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
echo ""
|
if command -v whiptail &>/dev/null; then
|
||||||
echo -e " ${BOLD}[5/6] Model selection${NC}"
|
MODEL_PREF=$(whiptail --title "Model Selection — ${GPU_NAME:-No GPU} (${VRAM_GB}GB VRAM)" \
|
||||||
echo " GPU: ${GPU_NAME:-None} (${VRAM_GB}GB VRAM)"
|
--radiolist "Choose model origin preference.\nSPACE = select ENTER = confirm" \
|
||||||
echo ""
|
14 78 4 \
|
||||||
echo " Origin preference:"
|
"2" "Performance-first — Qwen 3.5 (top benchmarks, vision+code)" ON \
|
||||||
echo " 1) Western-only — Codestral (Mistral 🇫🇷) · Phi4 (Microsoft 🇺🇸) · Mistral 7B"
|
"1" "Western-only — Codestral · Phi4 · Mistral 7B" OFF \
|
||||||
echo " 2) Performance-first — Qwen 3.5 (Feb 2026, top benchmarks, vision+code)"
|
"3" "Mixed — Western chat + Qwen 3.5 coding" OFF \
|
||||||
echo " 3) Mixed — Western for chat, Qwen 3.5 for coding"
|
"4" "Custom — enter model names manually" OFF \
|
||||||
echo " 4) Custom — enter model names manually"
|
3>&1 1>&2 2>&3) || MODEL_PREF="2"
|
||||||
echo ""
|
MODEL_PREF="${MODEL_PREF//\"/}"
|
||||||
read -rp " Choice [2]: " MODEL_PREF
|
else
|
||||||
MODEL_PREF="${MODEL_PREF:-2}"
|
echo ""
|
||||||
|
echo -e " ${BOLD}[5/6] Model selection${NC}"
|
||||||
|
echo " GPU: ${GPU_NAME:-None} (${VRAM_GB}GB VRAM)"
|
||||||
|
echo ""
|
||||||
|
echo " Origin preference:"
|
||||||
|
echo " 1) Western-only — Codestral (Mistral) · Phi4 (Microsoft) · Mistral 7B"
|
||||||
|
echo " 2) Performance-first — Qwen 3.5 (Feb 2026, top benchmarks, vision+code)"
|
||||||
|
echo " 3) Mixed — Western for chat, Qwen 3.5 for coding"
|
||||||
|
echo " 4) Custom — enter model names manually"
|
||||||
|
echo ""
|
||||||
|
read -rp " Choice [2]: " MODEL_PREF
|
||||||
|
MODEL_PREF="${MODEL_PREF:-2}"
|
||||||
|
fi
|
||||||
|
|
||||||
# Q4_K_M approximate weights in GB: 4B=2.5, 7B=4, 9B=5.5, 14B=9, 22B=13, 27B=17, 35B-MoE=12, 70B=41
|
# Q4_K_M approximate weights in GB: 4B=2.5, 7B=4, 9B=5.5, 14B=9, 22B=13, 27B=17, 35B-MoE=12, 70B=41
|
||||||
|
|
||||||
@@ -554,21 +617,59 @@ if $INSTALL_AI; then
|
|||||||
|
|
||||||
if [[ "$MODEL_PREF" == "4" ]]; then
|
if [[ "$MODEL_PREF" == "4" ]]; then
|
||||||
# Custom — free-form entry
|
# Custom — free-form entry
|
||||||
echo " Current defaults: fast=$FAST_MODEL chat=$CHAT_MODEL code=$CODE_MODEL"
|
if command -v whiptail &>/dev/null; then
|
||||||
echo " Press Enter on any line to keep the default shown."
|
_in=$(whiptail --title "Custom Models — Fast/Chat" \
|
||||||
echo ""
|
--inputbox "Fast/chat model (small, quick responses):" \
|
||||||
read -rp " Fast/chat model [$FAST_MODEL]: " _in; FAST_MODEL="${_in:-$FAST_MODEL}"
|
9 60 "$FAST_MODEL" 3>&1 1>&2 2>&3) && FAST_MODEL="${_in:-$FAST_MODEL}"
|
||||||
read -rp " Smart chat model [$CHAT_MODEL]: " _in; CHAT_MODEL="${_in:-$CHAT_MODEL}"
|
_in=$(whiptail --title "Custom Models — Smart Chat" \
|
||||||
read -rp " Code model [$CODE_MODEL]: " _in; CODE_MODEL="${_in:-$CODE_MODEL}"
|
--inputbox "Smart chat model (main model for complex tasks):" \
|
||||||
read -rp " Reasoning model (Enter to skip): " REASON_MODEL
|
9 60 "$CHAT_MODEL" 3>&1 1>&2 2>&3) && CHAT_MODEL="${_in:-$CHAT_MODEL}"
|
||||||
else
|
_in=$(whiptail --title "Custom Models — Code" \
|
||||||
read -rp " Choose tier [$REC_NUM]: " _TIER_INPUT
|
--inputbox "Code model:" \
|
||||||
_TIER_INPUT="${_TIER_INPUT:-$REC_NUM}"
|
9 60 "$CODE_MODEL" 3>&1 1>&2 2>&3) && CODE_MODEL="${_in:-$CODE_MODEL}"
|
||||||
# Accept either a number (1-4) or the tier name directly
|
REASON_MODEL=$(whiptail --title "Custom Models — Reasoning" \
|
||||||
if [[ "$_TIER_INPUT" =~ ^[1-4]$ ]]; then
|
--inputbox "Reasoning model (leave blank to skip):" \
|
||||||
TIER_PICK="${_TIER_NAMES[$((_TIER_INPUT-1))]}"
|
9 60 "" 3>&1 1>&2 2>&3) || REASON_MODEL=""
|
||||||
else
|
else
|
||||||
TIER_PICK="$_TIER_INPUT"
|
echo " Current defaults: fast=$FAST_MODEL chat=$CHAT_MODEL code=$CODE_MODEL"
|
||||||
|
echo " Press Enter on any line to keep the default shown."
|
||||||
|
echo ""
|
||||||
|
read -rp " Fast/chat model [$FAST_MODEL]: " _in; FAST_MODEL="${_in:-$FAST_MODEL}"
|
||||||
|
read -rp " Smart chat model [$CHAT_MODEL]: " _in; CHAT_MODEL="${_in:-$CHAT_MODEL}"
|
||||||
|
read -rp " Code model [$CODE_MODEL]: " _in; CODE_MODEL="${_in:-$CODE_MODEL}"
|
||||||
|
read -rp " Reasoning model (Enter to skip): " REASON_MODEL
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
if command -v whiptail &>/dev/null; then
|
||||||
|
# Build whiptail radiolist with recommended tier pre-selected
|
||||||
|
WHIP_TIERS=()
|
||||||
|
declare -A _TIER_LABELS
|
||||||
|
case "$MODEL_PREF" in
|
||||||
|
1) _TIER_LABELS=([7B]="mistral:7b + codellama:7b" [14B]="phi4:14b + starcoder2:15b" [22B]="phi4:14b + codestral:22b" [70B]="llama3.3:70b + codestral:22b")
|
||||||
|
_TIER_SPEEDS=([7B]="$(speed_label 4)" [14B]="$(speed_label 9)" [22B]="$(speed_label 13)" [70B]="$(speed_label 41)") ;;
|
||||||
|
2) _TIER_LABELS=([4B]="qwen3.5:4b (chat+code)" [9B]="qwen3.5:9b (chat+code)" [35B]="qwen3.5-35b-a3b (MoE, 3B active)" [27B]="qwen3.5:27b (dense)")
|
||||||
|
_TIER_SPEEDS=([4B]="$(speed_label 2)" [9B]="$(speed_label 5)" [35B]="$(speed_label 12)" [27B]="$(speed_label 17)") ;;
|
||||||
|
3) _TIER_LABELS=([7B]="mistral:7b + qwen3.5:4b" [14B]="phi4:14b + qwen3.5:9b" [35B]="phi4:14b + qwen3.5-35b-a3b" [70B]="llama3.3:70b + qwen3.5-35b-a3b")
|
||||||
|
_TIER_SPEEDS=([7B]="$(speed_label 4)" [14B]="$(speed_label 9)" [35B]="$(speed_label 19)" [70B]="$(speed_label 41)") ;;
|
||||||
|
esac
|
||||||
|
for _tn in "${_TIER_NAMES[@]}"; do
|
||||||
|
_onoff="OFF"; [[ "$_tn" == "$REC_TIER" ]] && _onoff="ON"
|
||||||
|
WHIP_TIERS+=("$_tn" "${_TIER_LABELS[$_tn]} | ${_TIER_SPEEDS[$_tn]}" "$_onoff")
|
||||||
|
done
|
||||||
|
TIER_PICK=$(whiptail --title "Model Size — ${VRAM_GB}GB GPU" \
|
||||||
|
--radiolist "Recommended tier pre-selected based on your GPU.\nSPACE = select ENTER = confirm" \
|
||||||
|
14 90 4 \
|
||||||
|
"${WHIP_TIERS[@]}" 3>&1 1>&2 2>&3) || TIER_PICK="$REC_TIER"
|
||||||
|
TIER_PICK="${TIER_PICK//\"/}"
|
||||||
|
unset WHIP_TIERS _TIER_LABELS _TIER_SPEEDS
|
||||||
|
else
|
||||||
|
read -rp " Choose tier [$REC_NUM]: " _TIER_INPUT
|
||||||
|
_TIER_INPUT="${_TIER_INPUT:-$REC_NUM}"
|
||||||
|
if [[ "$_TIER_INPUT" =~ ^[1-4]$ ]]; then
|
||||||
|
TIER_PICK="${_TIER_NAMES[$((_TIER_INPUT-1))]}"
|
||||||
|
else
|
||||||
|
TIER_PICK="$_TIER_INPUT"
|
||||||
|
fi
|
||||||
fi
|
fi
|
||||||
unset _TIER_NAMES _TIER_INPUT REC_NUM
|
unset _TIER_NAMES _TIER_INPUT REC_NUM
|
||||||
|
|
||||||
@@ -594,16 +695,29 @@ if $INSTALL_AI; then
|
|||||||
esac
|
esac
|
||||||
fi
|
fi
|
||||||
|
|
||||||
echo ""
|
# Show selected models and ask about download
|
||||||
echo " Models selected:"
|
_MODEL_SUMMARY="Fast chat: $FAST_MODEL\nSmart chat: $CHAT_MODEL\nCode: $CODE_MODEL"
|
||||||
printf " %-16s %s\n" "Fast chat:" "$FAST_MODEL"
|
[[ -n "$REASON_MODEL" ]] && _MODEL_SUMMARY+="\nReasoning: $REASON_MODEL"
|
||||||
printf " %-16s %s\n" "Smart chat:" "$CHAT_MODEL"
|
_MODEL_SUMMARY+="\nEmbed (RAG): $EMBED_MODEL"
|
||||||
printf " %-16s %s\n" "Code:" "$CODE_MODEL"
|
|
||||||
[[ -n "$REASON_MODEL" ]] && printf " %-16s %s\n" "Reasoning:" "$REASON_MODEL"
|
if command -v whiptail &>/dev/null; then
|
||||||
printf " %-16s %s\n" "Embed (RAG):" "$EMBED_MODEL"
|
if whiptail --title "Download Models Now?" \
|
||||||
echo ""
|
--yesno "Models selected:\n\n$_MODEL_SUMMARY\n\nDownload these models now? (can take 10-40 min)" \
|
||||||
read -rp " Download these models now? [Y/n]: " DO_PULL
|
14 60 3>&1 1>&2 2>&3; then
|
||||||
[[ "${DO_PULL,,}" != "n" ]] && PULL_MODELS=true
|
PULL_MODELS=true
|
||||||
|
fi
|
||||||
|
else
|
||||||
|
echo ""
|
||||||
|
echo " Models selected:"
|
||||||
|
printf " %-16s %s\n" "Fast chat:" "$FAST_MODEL"
|
||||||
|
printf " %-16s %s\n" "Smart chat:" "$CHAT_MODEL"
|
||||||
|
printf " %-16s %s\n" "Code:" "$CODE_MODEL"
|
||||||
|
[[ -n "$REASON_MODEL" ]] && printf " %-16s %s\n" "Reasoning:" "$REASON_MODEL"
|
||||||
|
printf " %-16s %s\n" "Embed (RAG):" "$EMBED_MODEL"
|
||||||
|
echo ""
|
||||||
|
read -rp " Download these models now? [Y/n]: " DO_PULL
|
||||||
|
[[ "${DO_PULL,,}" != "n" ]] && PULL_MODELS=true
|
||||||
|
fi
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
|
||||||
@@ -642,8 +756,14 @@ if [[ "$ZIM_CHOICE" == "2" ]]; then
|
|||||||
echo " ✓ Download ${_nzim} ZIM(s): $ZIM_PICKS"
|
echo " ✓ Download ${_nzim} ZIM(s): $ZIM_PICKS"
|
||||||
fi
|
fi
|
||||||
echo ""
|
echo ""
|
||||||
read -rp " Proceed? [Y/n]: " CONFIRM
|
if command -v whiptail &>/dev/null; then
|
||||||
[[ "${CONFIRM,,}" == "n" ]] && echo "Aborted." && exit 0
|
whiptail --title "Ready to Install" \
|
||||||
|
--yesno "Everything above will be installed and configured.\n\nProceed?" \
|
||||||
|
9 50 3>&1 1>&2 2>&3 || { echo "Aborted."; exit 0; }
|
||||||
|
else
|
||||||
|
read -rp " Proceed? [Y/n]: " CONFIRM
|
||||||
|
[[ "${CONFIRM,,}" == "n" ]] && echo "Aborted." && exit 0
|
||||||
|
fi
|
||||||
echo ""
|
echo ""
|
||||||
|
|
||||||
# ── helper: write only if missing (or --force) ────────────────────────────────
|
# ── helper: write only if missing (or --force) ────────────────────────────────
|
||||||
|
|||||||
Reference in New Issue
Block a user