Fix: 262K context requires both cards unified, not simultaneous
Both GPUs must be dedicated to the LLM for 262K context window. Image gen and LLM run one at a time (swap takes seconds). Simultaneous only possible with smaller context (~64K on one card). https://claude.ai/code/session_01PtYTPherSJaxDEVPgF6Nxu
This commit is contained in:
@@ -264,16 +264,24 @@ The RTX 5000 has **384 Tensor Cores with native FP16** — full SDXL/Flux suppor
|
||||
| Flux Dev (high-res / batched) | ~18-24GB | Both cards via NVLink (32GB) |
|
||||
| ComfyUI / InvokeAI | Works natively | No `--force-fp32` needed |
|
||||
|
||||
**Run both workloads simultaneously:**
|
||||
```bash
|
||||
# Option A: Dedicated cards (no model swapping)
|
||||
CUDA_VISIBLE_DEVICES=0 # Ollama — coding LLM on GPU 0
|
||||
CUDA_VISIBLE_DEVICES=1 # ComfyUI/InvokeAI — image gen on GPU 1
|
||||
**Important: 262K context requires both cards unified.** You can't split one off for image
|
||||
gen and keep 262K. It's one task at a time:
|
||||
|
||||
# Option B: Both cards unified for whichever task you're doing
|
||||
# Switch between LLM (32GB, 262K context) and image gen (32GB, high-res batches)
|
||||
```bash
|
||||
# CODING SESSION: Both cards unified → 32GB → 262K context
|
||||
ollama run qwen3.5:35b-a3b-q4_K_M # Uses both GPUs via NVLink
|
||||
|
||||
# IMAGE GEN SESSION: Stop LLM, run image gen on one card (16GB is plenty)
|
||||
ollama stop # Frees VRAM
|
||||
comfyui --listen 0.0.0.0 # SDXL/Flux fits easily in 16GB
|
||||
|
||||
# Swap takes a few seconds, not simultaneous but not painful
|
||||
```
|
||||
|
||||
If you want simultaneous coding + image gen, you'd run a smaller model at shorter context
|
||||
on one card (e.g., Qwen3.5-35B-A3B at ~64K on 16GB) and image gen on the other. But for
|
||||
full 262K context, both cards must be dedicated to the LLM.
|
||||
|
||||
### Hardware Longevity: 3-5 Years Realistic
|
||||
|
||||
- **2026-2027**: Sweet spot. MoE + linear attention models are getting smaller active params.
|
||||
|
||||
Reference in New Issue
Block a user