Fix: 262K context requires both cards unified, not simultaneous
Both GPUs must be dedicated to the LLM for 262K context window. Image gen and LLM run one at a time (swap takes seconds). Simultaneous only possible with smaller context (~64K on one card). https://claude.ai/code/session_01PtYTPherSJaxDEVPgF6Nxu
This commit is contained in:
@@ -264,16 +264,24 @@ The RTX 5000 has **384 Tensor Cores with native FP16** — full SDXL/Flux suppor
|
|||||||
| Flux Dev (high-res / batched) | ~18-24GB | Both cards via NVLink (32GB) |
|
| Flux Dev (high-res / batched) | ~18-24GB | Both cards via NVLink (32GB) |
|
||||||
| ComfyUI / InvokeAI | Works natively | No `--force-fp32` needed |
|
| ComfyUI / InvokeAI | Works natively | No `--force-fp32` needed |
|
||||||
|
|
||||||
**Run both workloads simultaneously:**
|
**Important: 262K context requires both cards unified.** You can't split one off for image
|
||||||
```bash
|
gen and keep 262K. It's one task at a time:
|
||||||
# Option A: Dedicated cards (no model swapping)
|
|
||||||
CUDA_VISIBLE_DEVICES=0 # Ollama — coding LLM on GPU 0
|
|
||||||
CUDA_VISIBLE_DEVICES=1 # ComfyUI/InvokeAI — image gen on GPU 1
|
|
||||||
|
|
||||||
# Option B: Both cards unified for whichever task you're doing
|
```bash
|
||||||
# Switch between LLM (32GB, 262K context) and image gen (32GB, high-res batches)
|
# CODING SESSION: Both cards unified → 32GB → 262K context
|
||||||
|
ollama run qwen3.5:35b-a3b-q4_K_M # Uses both GPUs via NVLink
|
||||||
|
|
||||||
|
# IMAGE GEN SESSION: Stop LLM, run image gen on one card (16GB is plenty)
|
||||||
|
ollama stop # Frees VRAM
|
||||||
|
comfyui --listen 0.0.0.0 # SDXL/Flux fits easily in 16GB
|
||||||
|
|
||||||
|
# Swap takes a few seconds, not simultaneous but not painful
|
||||||
```
|
```
|
||||||
|
|
||||||
|
If you want simultaneous coding + image gen, you'd run a smaller model at shorter context
|
||||||
|
on one card (e.g., Qwen3.5-35B-A3B at ~64K on 16GB) and image gen on the other. But for
|
||||||
|
full 262K context, both cards must be dedicated to the LLM.
|
||||||
|
|
||||||
### Hardware Longevity: 3-5 Years Realistic
|
### Hardware Longevity: 3-5 Years Realistic
|
||||||
|
|
||||||
- **2026-2027**: Sweet spot. MoE + linear attention models are getting smaller active params.
|
- **2026-2027**: Sweet spot. MoE + linear attention models are getting smaller active params.
|
||||||
|
|||||||
Reference in New Issue
Block a user