diff --git a/docs/gpu-setup-research.md b/docs/gpu-setup-research.md index dd9e7c3..c6a96fd 100644 --- a/docs/gpu-setup-research.md +++ b/docs/gpu-setup-research.md @@ -33,12 +33,22 @@ than traditional transformers. | Model | Type | Active Params | Size at Q4_K_M | Max Context | Quality | Notes | |-------|------|---------------|---------------|-------------|---------|-------| -| **Qwen3.5-35B-A3B** | **MoE** | **3B** | **~12GB** | **262K** | **A-** | Best bang for buck — 35B model, 3B active, fits 262K ctx in 25GB | -| **Qwen3.5-27B** | Dense | 27B | ~17GB | 262K | A- | 72.4% SWE-bench, ties GPT-5 mini | +| **Qwen3.5-35B-A3B** | **MoE** | **3B** | **~12GB** | **262K** | **B+ to A-** | 35B total but only 3B active — quality tracks active params | +| **Qwen3.5-27B** | Dense | 27B | ~17GB | 262K | **A-** | 72.4% SWE-bench, ties GPT-5 mini. The real A- option. | | **Qwen3.5-122B-A10B** | MoE | 10B | ~76GB | 262K | A | Matches GPT-5 mini across the board | | **Qwen3.5-9B** | Dense | 9B | ~6GB | 262K | B+ | Fits on any modern GPU | | **Qwen3.5-4B** | Dense | 4B | ~3GB | 262K | B | Tiny but capable | +**Quality reality check:** MoE models route tokens through only a subset of parameters. +The 35B-A3B activates **3B params per token** — think of it as a smart 7B model, not a 35B. +Quality is closer to B+ for complex coding. The 27B dense model is genuinely A- but needs +17GB weights (leaving less room for context on 32GB). At Q4 quantization there's a further +small quality loss. And 262K is a VRAM ceiling, not a quality guarantee — models degrade +at the edges of their context window. Practical high-quality context is more like 64-128K. + +**No local model approaches Claude Opus on hard problems.** The strategy isn't to replace +Opus — it's to offload the 80% of routine work so your Pro plan limits stop being an issue. + ### Previous Generation (Still Relevant) | Model | Size at Q4_K_M | Quality | Notes | @@ -212,7 +222,7 @@ bridge sitting on top of the cards. A 4U chassis gives the most room. | Model | Arch | Quant | Weights | Context | Total VRAM | Quality | |-------|------|-------|---------|---------|------------|---------| -| **Qwen3.5-35B-A3B** | **MoE (3B active)** | **Q4_K_M** | **~12GB** | **262K** | **~25GB** | **A-** | +| **Qwen3.5-35B-A3B** | **MoE (3B active)** | **Q4_K_M** | **~12GB** | **262K** | **~25GB** | **B+ to A-** | | Qwen3.5-27B | Dense | Q4_K_M | ~17GB | 128K+ | ~25GB | A- | | Qwen3-Coder-Next (80B/3B active) | MoE | Q4 | ~20GB | 128K | ~28GB | A | | Qwen2.5-Coder-14B | Dense | Q4_K_M | ~10GB | 128K | ~22GB | B+ |