diff --git a/docs/gpu-setup-research.md b/docs/gpu-setup-research.md index de120f5..dd9e7c3 100644 --- a/docs/gpu-setup-research.md +++ b/docs/gpu-setup-research.md @@ -264,16 +264,24 @@ The RTX 5000 has **384 Tensor Cores with native FP16** — full SDXL/Flux suppor | Flux Dev (high-res / batched) | ~18-24GB | Both cards via NVLink (32GB) | | ComfyUI / InvokeAI | Works natively | No `--force-fp32` needed | -**Run both workloads simultaneously:** -```bash -# Option A: Dedicated cards (no model swapping) -CUDA_VISIBLE_DEVICES=0 # Ollama — coding LLM on GPU 0 -CUDA_VISIBLE_DEVICES=1 # ComfyUI/InvokeAI — image gen on GPU 1 +**Important: 262K context requires both cards unified.** You can't split one off for image +gen and keep 262K. It's one task at a time: -# Option B: Both cards unified for whichever task you're doing -# Switch between LLM (32GB, 262K context) and image gen (32GB, high-res batches) +```bash +# CODING SESSION: Both cards unified → 32GB → 262K context +ollama run qwen3.5:35b-a3b-q4_K_M # Uses both GPUs via NVLink + +# IMAGE GEN SESSION: Stop LLM, run image gen on one card (16GB is plenty) +ollama stop # Frees VRAM +comfyui --listen 0.0.0.0 # SDXL/Flux fits easily in 16GB + +# Swap takes a few seconds, not simultaneous but not painful ``` +If you want simultaneous coding + image gen, you'd run a smaller model at shorter context +on one card (e.g., Qwen3.5-35B-A3B at ~64K on 16GB) and image gen on the other. But for +full 262K context, both cards must be dedicated to the LLM. + ### Hardware Longevity: 3-5 Years Realistic - **2026-2027**: Sweet spot. MoE + linear attention models are getting smaller active params.