Honest quality ratings: 35B-A3B is B+ not A-, add reality checks
The 35B-A3B MoE activates only 3B params per token - quality tracks active params, making it more like a smart 7B than a true 35B. Revised rating to B+ to A-. The 27B dense model is the real A- but needs 17GB weights. Also notes: 262K is a VRAM ceiling not a quality guarantee, Q4 quantization costs something, and context quality degrades at edges. Practical high-quality context is more like 64-128K. https://claude.ai/code/session_01PtYTPherSJaxDEVPgF6Nxu
This commit is contained in:
@@ -33,12 +33,22 @@ than traditional transformers.
|
|||||||
|
|
||||||
| Model | Type | Active Params | Size at Q4_K_M | Max Context | Quality | Notes |
|
| Model | Type | Active Params | Size at Q4_K_M | Max Context | Quality | Notes |
|
||||||
|-------|------|---------------|---------------|-------------|---------|-------|
|
|-------|------|---------------|---------------|-------------|---------|-------|
|
||||||
| **Qwen3.5-35B-A3B** | **MoE** | **3B** | **~12GB** | **262K** | **A-** | Best bang for buck — 35B model, 3B active, fits 262K ctx in 25GB |
|
| **Qwen3.5-35B-A3B** | **MoE** | **3B** | **~12GB** | **262K** | **B+ to A-** | 35B total but only 3B active — quality tracks active params |
|
||||||
| **Qwen3.5-27B** | Dense | 27B | ~17GB | 262K | A- | 72.4% SWE-bench, ties GPT-5 mini |
|
| **Qwen3.5-27B** | Dense | 27B | ~17GB | 262K | **A-** | 72.4% SWE-bench, ties GPT-5 mini. The real A- option. |
|
||||||
| **Qwen3.5-122B-A10B** | MoE | 10B | ~76GB | 262K | A | Matches GPT-5 mini across the board |
|
| **Qwen3.5-122B-A10B** | MoE | 10B | ~76GB | 262K | A | Matches GPT-5 mini across the board |
|
||||||
| **Qwen3.5-9B** | Dense | 9B | ~6GB | 262K | B+ | Fits on any modern GPU |
|
| **Qwen3.5-9B** | Dense | 9B | ~6GB | 262K | B+ | Fits on any modern GPU |
|
||||||
| **Qwen3.5-4B** | Dense | 4B | ~3GB | 262K | B | Tiny but capable |
|
| **Qwen3.5-4B** | Dense | 4B | ~3GB | 262K | B | Tiny but capable |
|
||||||
|
|
||||||
|
**Quality reality check:** MoE models route tokens through only a subset of parameters.
|
||||||
|
The 35B-A3B activates **3B params per token** — think of it as a smart 7B model, not a 35B.
|
||||||
|
Quality is closer to B+ for complex coding. The 27B dense model is genuinely A- but needs
|
||||||
|
17GB weights (leaving less room for context on 32GB). At Q4 quantization there's a further
|
||||||
|
small quality loss. And 262K is a VRAM ceiling, not a quality guarantee — models degrade
|
||||||
|
at the edges of their context window. Practical high-quality context is more like 64-128K.
|
||||||
|
|
||||||
|
**No local model approaches Claude Opus on hard problems.** The strategy isn't to replace
|
||||||
|
Opus — it's to offload the 80% of routine work so your Pro plan limits stop being an issue.
|
||||||
|
|
||||||
### Previous Generation (Still Relevant)
|
### Previous Generation (Still Relevant)
|
||||||
|
|
||||||
| Model | Size at Q4_K_M | Quality | Notes |
|
| Model | Size at Q4_K_M | Quality | Notes |
|
||||||
@@ -212,7 +222,7 @@ bridge sitting on top of the cards. A 4U chassis gives the most room.
|
|||||||
|
|
||||||
| Model | Arch | Quant | Weights | Context | Total VRAM | Quality |
|
| Model | Arch | Quant | Weights | Context | Total VRAM | Quality |
|
||||||
|-------|------|-------|---------|---------|------------|---------|
|
|-------|------|-------|---------|---------|------------|---------|
|
||||||
| **Qwen3.5-35B-A3B** | **MoE (3B active)** | **Q4_K_M** | **~12GB** | **262K** | **~25GB** | **A-** |
|
| **Qwen3.5-35B-A3B** | **MoE (3B active)** | **Q4_K_M** | **~12GB** | **262K** | **~25GB** | **B+ to A-** |
|
||||||
| Qwen3.5-27B | Dense | Q4_K_M | ~17GB | 128K+ | ~25GB | A- |
|
| Qwen3.5-27B | Dense | Q4_K_M | ~17GB | 128K+ | ~25GB | A- |
|
||||||
| Qwen3-Coder-Next (80B/3B active) | MoE | Q4 | ~20GB | 128K | ~28GB | A |
|
| Qwen3-Coder-Next (80B/3B active) | MoE | Q4 | ~20GB | 128K | ~28GB | A |
|
||||||
| Qwen2.5-Coder-14B | Dense | Q4_K_M | ~10GB | 128K | ~22GB | B+ |
|
| Qwen2.5-Coder-14B | Dense | Q4_K_M | ~10GB | 128K | ~22GB | B+ |
|
||||||
|
|||||||
Reference in New Issue
Block a user