Fix: 262K context requires both cards unified, not simultaneous

Both GPUs must be dedicated to the LLM for 262K context window.
Image gen and LLM run one at a time (swap takes seconds).
Simultaneous only possible with smaller context (~64K on one card).

https://claude.ai/code/session_01PtYTPherSJaxDEVPgF6Nxu
This commit is contained in:
Claude
2026-03-22 17:09:20 +00:00
parent 5cfd8b2696
commit 209173e988
+15 -7
View File
@@ -264,16 +264,24 @@ The RTX 5000 has **384 Tensor Cores with native FP16** — full SDXL/Flux suppor
| Flux Dev (high-res / batched) | ~18-24GB | Both cards via NVLink (32GB) |
| ComfyUI / InvokeAI | Works natively | No `--force-fp32` needed |
**Run both workloads simultaneously:**
```bash
# Option A: Dedicated cards (no model swapping)
CUDA_VISIBLE_DEVICES=0 # Ollama — coding LLM on GPU 0
CUDA_VISIBLE_DEVICES=1 # ComfyUI/InvokeAI — image gen on GPU 1
**Important: 262K context requires both cards unified.** You can't split one off for image
gen and keep 262K. It's one task at a time:
# Option B: Both cards unified for whichever task you're doing
# Switch between LLM (32GB, 262K context) and image gen (32GB, high-res batches)
```bash
# CODING SESSION: Both cards unified → 32GB 262K context
ollama run qwen3.5:35b-a3b-q4_K_M # Uses both GPUs via NVLink
# IMAGE GEN SESSION: Stop LLM, run image gen on one card (16GB is plenty)
ollama stop # Frees VRAM
comfyui --listen 0.0.0.0 # SDXL/Flux fits easily in 16GB
# Swap takes a few seconds, not simultaneous but not painful
```
If you want simultaneous coding + image gen, you'd run a smaller model at shorter context
on one card (e.g., Qwen3.5-35B-A3B at ~64K on 16GB) and image gen on the other. But for
full 262K context, both cards must be dedicated to the LLM.
### Hardware Longevity: 3-5 Years Realistic
- **2026-2027**: Sweet spot. MoE + linear attention models are getting smaller active params.