diff --git a/docs/gpu-setup-research.md b/docs/gpu-setup-research.md index acafb52..0be7d0d 100644 --- a/docs/gpu-setup-research.md +++ b/docs/gpu-setup-research.md @@ -320,12 +320,110 @@ watch -n 0.5 nvidia-smi **Phase 2 — Add second card + NVLink ($450)** 1. Buy matching Quadro RTX 5000 — ~$400 -2. Buy NVLink HB Bridge 3-slot (part: P4934 / 1JF3K / 6FY12AA) — ~$50 +2. Buy NVLink HB Bridge 2-slot (part: P4934 / 1JF3K / 6FY12AA) — ~$50 3. Install second card in adjacent/nearby x16 slot 4. Connect NVLink bridge 5. Verify with `nvidia-smi nvlink --status` 6. Now running 32GB unified — Qwen3.5-35B-A3B at Q4_K_M with full 262K context +### Dell R720/R730 Installation Guide + +#### Prerequisites (MUST HAVE before buying GPUs) + +| Requirement | R720 | R730 | Why | +|-------------|-------|-------|-----| +| **Dual CPUs** | Required | Required | GPU riser slots are wired to CPU2 — dead without it | +| **2x 1100W PSUs** | Required | Required | 2x 230W GPUs + system = ~600W+ under load | +| **GPU Riser 3** | Required for 2nd GPU | Required for GPUs | Provides the PCIe x16 slot + 8-pin power | +| **GPU Power Cable** | Required | Required | Riser-to-GPU power, not included by default | +| **Low-Profile Heatsinks** | Must swap (part of enablement kit) | Usually pre-installed | Standard heatsinks block GPU riser clearance | +| **Max ambient temp** | 30°C (not the usual 35°C) | 30°C | High GPU TDP restricts cooling headroom | + +#### Shopping List: Dell-Specific Parts + +| Part | Dell P/N | What It Is | Price | Where | +|------|----------|------------|-------|-------| +| **GPU Power Cable** | **9H6FV** (09H6FV) | 8-pin EPS (riser) → 6-pin + 6+2-pin PCIe. One cable powers one GPU | ~$10-15 | Amazon, eBay | +| **GPU Power Cable (alt)** | **N08NH** (0N08NH) | Same function, alternate Dell part number | ~$10-15 | Amazon, eBay | +| **GPU Riser 3** (R720) | Check eBay for "R720 riser 3" or "R720 GPU riser" | Second riser card that provides GPU-capable x16 slot | ~$15-30 | eBay | +| **GPU Riser 3** (R730) | Check eBay for "R730 riser 3" or "R730 GPU riser" | R730 version — NOT interchangeable with R720 | ~$15-30 | eBay | +| **Low-Profile Heatsinks** (R720 only) | Part of original GPU enablement kit | Shorter heatsinks that clear the GPU riser. Search "R720 low profile heatsink" | ~$10-20/pair | eBay | + +**You need 2x power cables** (one per GPU). Search Amazon for "Dell R720 R730 GPU power cable 9H6FV" — multiple sellers (COMeap, ZAHARA, BestParts) stock them for ~$10-15 each. + +#### How It Fits + +``` +Dell R720/R730 Riser Layout (rear view): +┌─────────────────────────────────────┐ +│ Riser 1 Riser 2 Riser 3│ +│ (network/ (GPU 1) (GPU 2)│ +│ storage) PCIe x16 PCIe x16│ +│ Gen2(720) Gen2(720)│ +│ Gen3(730) Gen3(730)│ +└─────────────────────────────────────┘ + ↑ RTX 5000 ↑ ↑ RTX 5000 ↑ + └── NVLink Bridge ──┘ +``` + +- Both GPUs sit on **adjacent risers** (Riser 2 + Riser 3) — this is 2-slot spacing +- The **2-slot NVLink bridge** (P4934) is the correct size for R720/R730 +- The cards mount **vertically** via risers, parallel to each other +- NVLink bridge connects across the top of both cards + +#### R720 vs R730 + +| Feature | R720 | R730 | +|---------|------|------| +| **PCIe** | Gen2 x16 | **Gen3 x16** | +| **Impact on LLM** | None — VRAM-bound | None — VRAM-bound | +| **Impact on NVLink** | None — NVLink bypasses PCIe | None — NVLink bypasses PCIe | +| **GPU power delivery** | Same 8-pin from riser | Same 8-pin from riser | +| **Heatsink swap** | Usually required | Usually already low-profile | +| **Used price** | ~$100-150 cheaper | Preferred if budget allows | +| **Recommendation** | Fine if you already have one | **Buy this one** if shopping new | + +#### Potential Issues + +1. **NVLink bridge clearance in 2U** — The bridge sits on top of both GPUs. In a 2U chassis + this is tight. The R720/R730 riser design mounts cards vertically which actually helps — + the bridge faces the chassis side panel, not the lid. Should fit, but measure before buying. + +2. **Blower fan noise** — The RTX 5000 has an active blower (unlike passive Tesla cards). + The server's own fans may spin higher to compensate. The blower exhausts out the bracket + which is correct for rack airflow. + +3. **"Unsupported" GPU warning** — Dell officially supports Tesla/Quadro cards from their era. + The Quadro RTX 5000 is a later generation than R720/R730 was designed for, but community + reports confirm Quadro RTX and even consumer RTX cards work fine. You won't get Dell support + if something goes wrong, but electrically it's standard PCIe. + +4. **CPU TDP limit** — Dell requires CPUs of 115W or less when GPUs are installed (R720). + Check your CPU model. Most common Xeon E5-2600 v1/v2 (R720) and E5-2600 v3/v4 (R730) + processors are within this range, but some high-core-count variants exceed it. + +5. **PSU mode** — With dual 300W GPUs, set PSU configuration to **non-redundant mode** + to use combined wattage from both PSUs. In redundant mode, you're limited to one PSU's + capacity (1100W) which may not be enough under full GPU + CPU load. + +#### Complete R720/R730 Shopping List + +``` +GPUS + NVLINK + 2x Quadro RTX 5000 ~$800 + 1x NVLink Bridge 2-slot (P4934 / L55997-001) ~$50 + +DELL-SPECIFIC PARTS + 2x GPU Power Cable (9H6FV or N08NH) ~$25 + 1x GPU Riser 3 (match your server model!) ~$20 + 2x Low-Profile Heatsinks (R720 only) ~$15 + +POWER (if not already installed) + 2x Dell 1100W PSU ~$30-50 ea + +TOTAL (assuming you have the server + dual CPUs) ~$940-960 +``` + ## 24GB GPU Options (Previous Research — Still Valid for Tighter Budgets) | GPU | VRAM | Price Range | Best Deals | Notes |