diff --git a/README.md b/README.md index 1e03f4e..f069ec7 100644 --- a/README.md +++ b/README.md @@ -1,50 +1,2992 @@ -# local_proxmox +# Dell R730xd — Complete Setup Walkthrough +### Proxmox VE 9.1 · GPU Passthrough · Frigate NVR -Proxmox VE 9.1 configuration, scripts, and VM layouts for a Dell R730xd homelab. +Full button-by-button guide from bare metal to running VMs. All 12 phases in +one document. -## Hardware +--- -- **Host:** Dell PowerEdge R730xd -- **RAID controller:** PERC H730 (configured in Non-RAID/per-disk passthrough mode) -- **GPUs:** 2x Quadro P3300 4GB (VFIO passthrough, one per VM) -- **TPU:** Google Coral USB (Frigate object detection) -- **Bays:** 12x (not all populated) +## Table of Contents -## VM Layout +- [Before You Start](#before-you-start) +- [Phase 1 — Firmware Updates](#phase-1--firmware-updates) +- [Phase 2 — iDRAC Setup](#phase-2--idrac-setup) +- [Phase 3 — BIOS Settings](#phase-3--bios-settings) +- [Phase 4 — H730 RAID Configuration](#phase-4--h730-raid-configuration) +- [Phase 5 — Install Proxmox VE 9.1](#phase-5--install-proxmox-ve-91) +- [Phase 6 — Proxmox Post-Install](#phase-6--proxmox-post-install) +- [Phase 7 — Fan Control + Staggered Spin-Up](#phase-7--fan-control--staggered-spin-up) +- [Phase 8 — Convert 3.5" Drives to Non-RAID Mode](#phase-8--convert-35-drives-to-non-raid-mode) +- [Phase 9 — Map Physical Bays to Drives](#phase-9--map-physical-bays-to-drives) +- [Phase 10 — GPU Passthrough Setup](#phase-10--gpu-passthrough-setup) +- [Phase 11 — Create the VMs](#phase-11--create-the-vms) +- [Phase 12 — Deploy Frigate in VM 100](#phase-12--deploy-frigate-in-vm-100) -| VM ID | Role | Drives | GPU | Coral | -|-------|------|--------|-----|-------| -| 100 | Frigate NVR | Bays 1–8 | Quadro #1 (VFIO) | Yes | -| 101 | TBD | Bays 9–10 | Quadro #2 (VFIO) | No | -| 102 | TBD | Bays 11–12 | None | No | +--- -## Setup Order +## Before You Start -1. [`scripts/perc-nonraid.sh`](scripts/perc-nonraid.sh) — convert H730 to per-disk non-RAID mode -2. [`scripts/build-bay-map.sh`](scripts/build-bay-map.sh) — map physical bays to stable `/dev/disk/by-id` paths -3. [`scripts/gpu-passthrough-setup.sh`](scripts/gpu-passthrough-setup.sh) — configure IOMMU + VFIO for both Quadros -4. Apply VM configs from [`vm-configs/`](vm-configs/) -5. Deploy Frigate from [`frigate/`](frigate/) +### What is iDRAC and do I need to pay for it? -## Docs +iDRAC stands for **Integrated Dell Remote Access Controller**. Ignore the name. +Here is what it actually is: -- [`docs/hardware-layout.md`](docs/hardware-layout.md) — bay/device/VM mapping worksheet +There is a **second tiny computer built into your server's motherboard**. It has +its own processor, its own RAM, its own network port, and it runs 24 hours a day +as long as the server has power — even when the server is "off". Dell calls this +second computer iDRAC. -Committed and pushed. The guide is at `docs/setup-guide.md` and covers everything in order: +This server has **iDRAC8 Express**, which is the version built into the board +at no cost. There is a paid upgrade called iDRAC8 Enterprise but you do not +need it. Here is exactly what Express gives you and what it does not: -| Phase | What happens | +**Express includes (everything this guide uses):** +- iDRAC web UI — hardware health dashboard, temperatures, fan speeds, event logs +- Power the server on and off remotely from the web UI +- IPMI over LAN — lets `ipmitool` send fan control commands from Proxmox +- racadm — command-line control used by the stagger spin-up script +- SSH directly into iDRAC for management + +**Express does NOT include:** +- Virtual Console — you cannot see the server's screen in a browser window. + A **physical monitor and keyboard are required** for all setup steps until + Proxmox is installed and SSH is working. After that you will never need the + monitor again. +- Virtual Media — you cannot mount an ISO file over the network. A **physical + USB drive** is required for the Proxmox installer. + +Nothing in this guide requires Enterprise. The fan control script and stagger +spin-up both use IPMI over LAN and racadm, which are both available on Express. + +**Do you need to pay for anything?** No. iDRAC Express is already there and +covers everything we need. The only other "subscription" people sometimes ask +about is the **Proxmox subscription** (about €100/year for enterprise update +servers and commercial support). You do not need that either — we configure the +free community repositories in Phase 6. The two are completely unrelated. + +**Summary: pay nothing, skip nothing. Keep a monitor plugged in through Phase 6.** + +--- + +### What you need on the table before starting + +- The server plugged into power and a network switch/router +- A **separate network cable** for the iDRAC port (the small RJ45 on the back + labeled "iDRAC" — it is separate from the four main NIC ports) +- A **monitor and USB keyboard** plugged into the server — required through + Phase 5 (the Proxmox installer). iDRAC Express does not include a remote + KVM, so you cannot see the server's screen from another computer. After + Proxmox is installed and SSH is working (end of Phase 6) the monitor + can be unplugged permanently. +- A USB drive (8 GB or larger) — needed later for the Proxmox installer +- Another computer or phone to read this guide + +--- + +## Phase 1 — Firmware Updates + +**Why first?** Old firmware has bugs. BIOS from 2016 may not correctly expose +the VT-d settings that GPU passthrough requires. The H730 firmware has had fixes +for JBOD/non-RAID mode. Updating now prevents chasing ghosts later. + +**Time required:** 30–90 minutes. The server will reboot several times on its own. + +--- + +### Step 1.1 — Power on and watch the screen + +Press the power button on the front of the server. + +The screen will show a **Dell splash screen** with the PowerEdge logo. At the +bottom of the screen you will see a line of options, something like: + +``` +F2 = System Setup F10 = Lifecycle Controller F11 = Boot Manager F12 = PXE Boot +``` + +These options are only available for about **5–8 seconds** before the server +continues booting. If you miss the window the server will try to boot an OS +(and fail if nothing is installed yet). Just power it off and back on and try again. + +--- + +### Step 1.2 — Enter Lifecycle Controller + +Press **F10** when you see the splash screen. + +The screen will go blank for 10–30 seconds, then show the **Lifecycle Controller** +loading screen. This is a mini operating system built into the server's firmware. +It has nothing to do with your server's main OS. Wait for it to finish loading — +it can take up to 90 seconds the first time. + +When it finishes you will see the **Lifecycle Controller Home** screen with a menu +on the left side: + +``` +Home +Firmware Update +OS Deployment +RAID Configuration +Hardware Configuration +Settings +``` + +If it asks you to complete initial setup (language, network), do that first — +use Tab and arrow keys to navigate, Enter to confirm. + +--- + +### Step 1.3 — Go to Firmware Update + +Click or arrow-key to **Firmware Update** on the left menu, then press Enter. + +You will see: + +``` +Launch Firmware Update +``` + +Press Enter on that. + +--- + +### Step 1.4 — Choose your update method + +You will be asked how you want to get the firmware. You have two options: + +#### Option A — Server has internet access (easiest) + +If your network cable is plugged in and the server can reach the internet: + +1. Select **HTTPS** (it will say something like "downloads.dell.com") +2. Press **Next** +3. Lifecycle Controller will connect to Dell's servers and **automatically scan + your server** to find what firmware versions are installed vs. what is available +4. This takes 2–5 minutes — the screen will show a progress bar +5. Skip to Step 1.5 + +#### Option B — No internet on the server + +You need to download the firmware files on another computer first. + +1. On your other computer, open a browser and go to: + **https://www.dell.com/support/home** + +2. You need your server's **Service Tag** — it is a 7-character code. Find it: + - On the **front of the server**: there is a small pull-out plastic tab (like a + credit card slot) on the left side of the front panel. Pull it out — the + Service Tag is printed on it. + - Or: on a sticker on the **top** of the server chassis. + - Or: it was displayed briefly during POST on the splash screen. + +3. On the Dell support page, click **"View products"** or type the Service Tag + into the search bar at the top. Select your server from the results. + +4. Click **"Drivers & Downloads"** + +5. You will see a filter panel. Under **"Category"**, select each of these one + at a time and download the latest version: + - **iDRAC** — look for "iDRAC8 firmware" — file will end in `.exe` or `.d9` + - **BIOS** — look for "BIOS" — file ends in `.exe` + - **RAID** — look for "PERC H730" — file ends in `.exe` + +6. Copy all downloaded files onto a **FAT32-formatted USB drive** + +7. Plug the USB drive into the server + +8. In Lifecycle Controller: select **Local Drive (USB)** and press Next + +9. Navigate to the USB drive and select the firmware files + +--- + +### Step 1.5 — Review and apply updates + +Lifecycle Controller shows a table of all components with columns like: + +``` +Component Current Version Available Version Select +--------- --------------- ----------------- ------ +iDRAC8 2.40.40.40 2.85.85.85 [ ] +BIOS 1.2.10 2.14.0 [ ] +PERC H730 Mini 25.4.0.0018 25.5.9.0001 [ ] +``` + +1. Check the box next to **every component** that shows a newer Available Version + (you can usually click "Select All") +2. Click **Install and Reboot** (or **Apply**) +3. A warning will say the server will reboot — click **Yes** + +The server will now: +- Apply each firmware update +- Reboot automatically between some updates +- Return to the Lifecycle Controller when finished + +**Do not power off the server during this process.** Just wait. It can take +30–60 minutes if there are many updates. The screen will show progress. + +--- + +### Step 1.6 — Confirm completion + +When all updates are done, Lifecycle Controller returns to its home screen. +You may see a success summary screen first — click **OK** or **Finish**. + +To verify everything worked: +1. In Lifecycle Controller, go to **Firmware Update** → **Launch Firmware Update** + again +2. Run the check one more time +3. The Available Version column should now match Current Version for everything + you updated (or show no newer version) + +Press **Exit** (or **Finish**) to leave Lifecycle Controller. The server will +reboot normally. + +--- + +## Phase 2 — iDRAC Setup + +You need to do two things with iDRAC: +1. Give it a **static IP address** so it is always reachable at the same address +2. Enable **IPMI over LAN** so the fan control script can send commands to it + +--- + +### Step 2.1 — Enter System Setup (F2) + +Power on (or reboot) the server. + +When the Dell splash screen appears, press **F2**. + +The screen will go blank briefly, then show the **System Setup Main Menu**: + +``` +System Setup Main Menu + + System BIOS + iDRAC Settings + Device Settings + Service Tag Settings +``` + +Use the **arrow keys** to highlight **iDRAC Settings** and press **Enter**. + +--- + +### Step 2.2 — Set a static IP for iDRAC + +Inside iDRAC Settings you will see another menu: + +``` +iDRAC Settings + + Network + User Configuration + Smart Card + Update and Rollback + ... +``` + +Arrow down to **Network** and press **Enter**. + +The Network screen has several sections. You are looking for the **IPv4 Settings** +section. Use Tab or arrow keys to navigate to these fields: + +| Field | What to set | |-------|-------------| -| 1 | Firmware updates (do this first — saves pain later) | -| 2 | iDRAC: static IP + enable IPMI over LAN | -| 3 | BIOS: VT-x, VT-d, UEFI mode, HDD stagger — **all required** | -| 4 | H730: RAID1 the 2x 2.5" OS drives, leave 3.5" alone | -| 5 | Install Proxmox onto the RAID1 OS mirror | -| 6 | Post-install: fix repos, install tools, clone this repo | -| 7 | `perc-nonraid.sh` — 3.5" drives to non-RAID mode | -| 8 | `build-bay-map.sh` — walk bays with LED blink, fill in the map | -| 9 | `gpu-passthrough-setup.sh` — IOMMU + vfio-pci, verify with `lspci` | -| 10 | Fan control + stagger spin-up services installed | -| 11 | Create VMs, fill in the conf placeholders | -| 12 | NVIDIA driver + Docker + Frigate inside VM 100 | +| Enable NIC | **Enabled** | +| NIC Selection | **Dedicated** (this uses the dedicated iDRAC port on the back) | +| Enable IPv4 | **Enabled** | +| Enable DHCP | **Disabled** ← change this | +| Static IP Address | e.g. `192.168.1.5` (pick an IP outside your router's DHCP range) | +| Static Gateway | your router's IP, e.g. `192.168.1.1` | +| Static Subnet Mask | `255.255.255.0` | +| DNS Server 1 | your router's IP, e.g. `192.168.1.1` | -The key thing to do **before** touching the installer: Phase 3 BIOS settings, especially VT-d. It's the most common skip that causes passthrough to silently fail. +> **What IP to use?** Log into your router and find its DHCP range — for example +> if DHCP hands out `192.168.1.100` to `192.168.1.200`, pick something outside +> that range like `192.168.1.5`. Write this IP down — you will use it often. + +When done, press **Back** or navigate to the bottom and click **Apply**, then **OK**. + +--- + +### Step 2.3 — Enable IPMI over LAN + +Still inside iDRAC Settings → Network. + +Scroll down past the IPv4 section until you see **IPMI Settings**. + +Find: + +``` +Enable IPMI over LAN: [ Disabled ] +``` + +Change this to **Enabled**. + +This is the setting that allows `ipmitool` commands (used by the fan control +script) to reach iDRAC over the network. + +Click **Apply** at the bottom, then **OK** when asked to confirm. + +--- + +### Step 2.4 — Change the default password + +Go back to the iDRAC Settings main menu (press **Back**). + +Arrow down to **User Configuration** and press **Enter**. + +You will see a list of user slots. **User 1** is the built-in `root` account. +Select it (press Enter). + +> **Why change it?** The factory default password is `calvin`. This is printed +> in Dell's public documentation and is widely known. Any device on your network +> could log into your iDRAC if you leave it as-is. + +Change: +- **User Name:** leave as `root` +- **Change Password:** set to **Enabled** +- **Password:** enter a strong password +- **Confirm Password:** enter it again + +Click **Apply**, then **OK**. + +Press **Finish** to exit iDRAC Settings and return to the System Setup Main Menu. + +**Do not reboot yet** — you still have BIOS settings to configure in Phase 3. + +--- + +### Step 2.5 — Verify iDRAC from your browser (do after Phase 3) + +After you finish Phase 3 and let the server reboot, test iDRAC from your other +computer: + +1. Make sure the **dedicated iDRAC network port** (small RJ45 labeled "iDRAC" on + the server's back panel) is plugged into your switch or router +2. Open a browser on your other computer +3. Go to: `https://192.168.1.5` (whatever static IP you set in Step 2.2) +4. You will see a **certificate warning** — this is normal, iDRAC uses a + self-signed certificate. Click **Advanced** → **Proceed anyway** (wording + varies by browser) +5. The iDRAC8 login page appears — a dark Dell-branded page +6. Log in: Username `root`, Password (the one you set in Step 2.4) + +You should see the iDRAC dashboard showing system health, temperatures, and fans. +This is your remote window into the server — you can now manage it from your desk. + +--- + +## Phase 3 — BIOS Settings + +These settings enable the CPU features that allow devices (GPUs, USB controllers) +to be handed directly to virtual machines. **Every setting below is required.** +Skipping VT-d is the single most common reason GPU passthrough silently fails. + +You are still in System Setup from Phase 2 (or press F2 again on reboot). + +--- + +### Step 3.1 — System BIOS → Processor Settings + +From the System Setup Main Menu, arrow to **System BIOS** and press Enter. + +You will see the System BIOS Settings menu: + +``` +System BIOS Settings + + System Information + Memory Settings + Processor Settings ← go here + SATA Settings + Boot Settings + Integrated Devices + Serial Communication + System Profile Settings + Power Management + Security + Miscellaneous Settings +``` + +Arrow to **Processor Settings** and press **Enter**. + +Find and set these two options: + +--- + +**Virtualization Technology** + +This enables the CPU to run virtual machines efficiently. Without it Proxmox +still works but performance is worse. + +``` +Virtualization Technology: [ Disabled ] + ↓ change to +Virtualization Technology: [ Enabled ] +``` + +Use the arrow keys or spacebar to toggle the value. + +--- + +**VT for Direct I/O** (also shown as "Virtualization Technology for Directed I/O") + +This is the critical one. It enables **IOMMU** — the hardware feature that lets +the CPU safely hand a real PCIe device (like your GPU) directly to a virtual +machine. Without this, passthrough is impossible. + +``` +VT for Direct I/O: [ Disabled ] + ↓ change to +VT for Direct I/O: [ Enabled ] +``` + +> **R730xd BIOS 2.19+ note:** The "VT for Direct I/O" toggle was removed in +> BIOS version 2.19 and later — it does not appear in Processor Settings at all. +> VT-d (IOMMU) is enabled by default on these BIOS versions. If you do not see +> the toggle, that is expected; skip this sub-step and continue. + +Click **Back** to return to System BIOS Settings. + +--- + +### Step 3.2 — System BIOS → Integrated Devices + +From the System BIOS Settings menu, arrow to **Integrated Devices** and press Enter. + +Find: + +**SR-IOV Global Enable** + +SR-IOV is a PCIe feature that allows a single physical device to appear as +multiple devices. Even though the Quadro P2200 does not use SR-IOV, enabling +this globally avoids a class of IOMMU grouping problems. + +``` +SR-IOV Global Enable: [ Disabled ] + ↓ change to +SR-IOV Global Enable: [ Enabled ] +``` + +Click **Back**. + +--- + +### Step 3.3 — System BIOS → Boot Settings + +From System BIOS Settings, arrow to **Boot Settings** and press Enter. + +You will see: + +``` +Boot Settings + + Boot Mode: [ BIOS ] ← must change this + Boot Sequence + ... +``` + +**Boot Mode** — change from BIOS to UEFI: + +``` +Boot Mode: [ BIOS ] + ↓ change to +Boot Mode: [ UEFI ] +``` + +> **Why UEFI?** Proxmox uses a tool called `proxmox-boot-tool` to manage the +> kernel boot parameters (including the IOMMU settings the GPU passthrough script +> writes). This tool only works with UEFI boot. If the server is in Legacy/BIOS +> mode the passthrough script will fail and you will have to reinstall. + +A warning may appear saying the boot sequence will be cleared — click **Yes** or +**OK**. That is fine since we haven't installed anything yet. + +Click **Back**. + +--- + +### Step 3.4 — System BIOS → System Profile Settings (Power Management) + +From System BIOS Settings, arrow to **System Profile Settings** and press Enter. + +> **R730xd BIOS 2.19+ note:** "Power Management" was renamed to **"System +> Profile Settings"** in BIOS 2.19 and later. If you see "System Profile +> Settings" in the menu, that is the same section — go there. + +Find: + +**Hard Disk Drive Sequencing** — this is the built-in stagger that fires at +power-on before the OS loads. It staggers when each drive spins up so they do +not all surge at the same time and overload the PSU. + +``` +Hard Disk Drive Sequencing: [ Disabled ] + ↓ change to +Hard Disk Drive Sequencing: [ Enabled ] +``` + +> **R730xd BIOS 2.19+ note:** The "Hard Disk Drive Sequencing" attribute +> (`BIOS.StorageSettings.HddSeq`) was also removed in BIOS 2.19+. If you do +> not see it here, skip it — the Linux stagger-spinup service (Phase 7, Step +> 7.5) handles spin-up sequencing entirely from the OS side. + +Also check: + +**C States** — these are CPU power-saving sleep states. They can add latency +to VM workloads. Disable them for a server running VMs: + +``` +C States: [ Enabled ] + ↓ change to +C States: [ Disabled ] +``` + +Click **Back**. + +--- + +### Step 3.5 — Apply and exit + +From System BIOS Settings, click **Finish** (at the bottom of the menu, you may +need to scroll). + +A dialog will appear: + +``` +Confirm changes and exit? +[ Yes ] [ No ] +``` + +Click **Yes**. + +The server will reboot. + +--- + +### Step 3.6 — Confirm the settings took + +After the reboot, press **F2** again to re-enter System Setup and spot-check: + +- **Processor Settings:** VT and VT for Direct I/O should both show **Enabled** +- **Boot Settings:** Boot Mode should show **UEFI** + +If either shows Disabled, set it again and apply. Some BIOS versions have a bug +where the first Apply does not persist — applying a second time fixes it. + +--- + +### Phase 3 complete — what you have now + +| Setting | Why it matters | +|---------|---------------| +| VT-x (Virtualization Technology) | Lets the CPU run VMs at near-native speed | +| VT-d (Direct I/O) | Enables IOMMU — the foundation of all device passthrough | +| SR-IOV | Prevents IOMMU group problems with PCIe devices | +| UEFI boot mode | Required for Proxmox's boot management and EFI GPU passthrough | +| HDD sequencing | Staggers drive spin-up at power-on to protect the PSU | + +--- + +## Phase 4 — H730 RAID Configuration + +**Goal:** Mirror the two 2.5" rear drives together so a single drive failure +cannot take down Proxmox. Leave every 3.5" drive untouched — the `perc-nonraid.sh` +script handles those later from inside Proxmox. + +**Why mirror the OS drives here instead of using ZFS?** The H730 presents the +RAID1 mirror as a single virtual disk to the OS. Proxmox installs onto it and +never needs to think about the fact that two physical drives are underneath. +This is the simplest and most reliable approach for an OS volume. + +> **If the 2.5" rear drives appear under a different controller (e.g., PERC H330 +> Mini):** The R730xd sometimes puts the rear bays on a separate mini controller +> rather than the main H730. That is fine — just create the RAID1 on whichever +> controller owns those drives. The rest of this phase is identical regardless +> of which controller is used. + +--- + +### Step 4.1 — Enter Lifecycle Controller + +Power on (or reboot) the server and press **F10** at the Dell splash screen. + +Wait for Lifecycle Controller to load. You will see the main menu on the left: + +``` +Home +Firmware Update +OS Deployment +RAID Configuration ← go here +Hardware Configuration +Settings +``` + +Arrow to **RAID Configuration** and press **Enter**. + +--- + +### Step 4.2 — Open the RAID configuration wizard + +You will see: + +``` +RAID Configuration + + View Current Configuration + Create New VD + ... +``` + +First, click **View Current Configuration** to see what is already there. + +You are looking at the physical drives the H730 can see. Drives are listed under +their controller — it will say something like: + +``` +PERC H730 Mini (Slot 0) + + Physical Disks: + Port 0: SEAGATE ST4000NM0023 4.0 TB State: Unconfigured Good + Port 1: SEAGATE ST4000NM0023 4.0 TB State: Unconfigured Good + ... + Port 8: TOSHIBA MK1401GRRB 146 GB State: Unconfigured Good ← 2.5" + Port 9: TOSHIBA MK1401GRRB 146 GB State: Unconfigured Good ← 2.5" +``` + +> **Identifying the 2.5" OS drives:** They will be noticeably smaller than the +> 3.5" drives — likely 146 GB, 300 GB, 600 GB (SAS) or 120–480 GB (SSD). +> The 3.5" drives will be 1 TB, 2 TB, 4 TB, or larger. Size is the giveaway. + +> **If drives show "Foreign Configuration":** They were previously part of a +> RAID array. See Step 4.3a before continuing. + +Press **Back** to return to the RAID Configuration menu. + +--- + +### Step 4.3a — Clear foreign configurations (only if needed) + +If any drives showed "Foreign Configuration" in Step 4.2, you need to clear +them before you can create a new virtual disk. + +In the RAID Configuration menu, look for: + +``` +Clear Foreign Configuration +``` + +Select it, choose **All Foreign Configurations**, click **Apply**. + +A warning will appear saying all data on those drives will be lost — click **Yes**. +(We are building a new system so there is nothing to keep.) + +After clearing, go back to **View Current Configuration** and confirm the drives +now show **Unconfigured Good**. + +--- + +### Step 4.4 — Create the RAID 1 virtual disk + +In the RAID Configuration menu, select **Create New VD** and press **Enter**. + +The wizard walks you through four screens: + +--- + +**Screen 1 — Select RAID Level** + +``` +Select RAID Level: + + RAID 0 (no redundancy, faster) + RAID 1 (mirror, one drive can fail) ← select this + RAID 5 (requires 3+ drives) + RAID 6 (requires 4+ drives) + RAID 10 (requires 4+ drives) +``` + +Select **RAID 1** and click **Next**. + +--- + +**Screen 2 — Select Physical Disks** + +A list of all available physical disks appears. You need to select **only the +two 2.5" drives**. Do not select any 3.5" drives. + +Click the checkbox next to each of the two 2.5" drives (identified by their +smaller size). Leave all 3.5" drives unchecked. + +Click **Next**. + +--- + +**Screen 3 — Virtual Disk Attributes** + +``` +Virtual Disk Name: OS-Mirror +Virtual Disk Size: (auto-filled — leave it) +Strip Element Size: 64KB (leave default) +Read Policy: Adaptive Read Ahead (leave default) +Write Policy: Write Back (leave default) +Disk Cache Policy: Enabled (leave default) +``` + +The only thing you need to change is the name — type `OS-Mirror` so it is easy +to identify. Everything else can stay at defaults. + +Click **Next**. + +--- + +**Screen 4 — Confirm** + +Review the summary. It should show: +- RAID Level: 1 +- Physical Disks: 2 (your 2.5" drives) +- Virtual Disk Size: whatever the smaller of the two drives is + +Click **Finish**. + +--- + +### Step 4.5 — Initialize the virtual disk + +After creating the VD, Lifecycle Controller will ask about initialization: + +``` +Initialize virtual disk? + Fast Initialize ← select this + Full Initialize (writes zeros to every sector — takes hours, not needed) + Skip +``` + +Select **Fast Initialize** and click **OK**. + +Fast initialization takes 30–60 seconds. A progress bar will appear. Wait for it. + +--- + +### Step 4.6 — Verify and exit + +When initialization finishes, go back to **View Current Configuration**. + +You should now see: + +``` +PERC H730 Mini + + Virtual Disks: + VD 0: OS-Mirror RAID 1 ~146 GB State: Optimal + + Physical Disks: + Port 0–7: 3.5" drives State: Unconfigured Good + Port 8–9: 2.5" drives State: Online (member of VD 0) +``` + +The 3.5" drives should all show **Unconfigured Good**. That is exactly what you +want. Leave them that way. + +Click **Back** repeatedly until you reach the Lifecycle Controller home screen, +then click **Exit**. The server will reboot. + +--- + +## Phase 5 — Install Proxmox VE 9.1 + +### Step 5.1 — Download the Proxmox ISO + +On your other computer, open a browser and go to: + +``` +https://www.proxmox.com/en/downloads +``` + +Click **Proxmox Virtual Environment** → find the latest **Proxmox VE 9.x ISO +Installer** and click **Download**. + +The file will be named something like `proxmox-ve_9.1-1.iso` and is about 1 GB. + +--- + +### Step 5.2 — Write the ISO to a USB drive + +You need to write the ISO as a disk image (not copy the file). Use one of these +tools depending on your computer: + +#### Windows — Rufus (free, no install needed) + +1. Download Rufus from: **https://rufus.ie** (click the first .exe link) +2. Plug in your USB drive (8 GB or larger — all data on it will be erased) +3. Open Rufus +4. **Device:** select your USB drive from the dropdown +5. **Boot selection:** click **SELECT** and choose the Proxmox ISO file +6. **Partition scheme:** select **GPT** +7. **Target system:** select **UEFI (non CSM)** +8. Click **START** +9. A dialog appears asking about ISO mode vs DD mode — select **Write in DD Image + mode** and click **OK** +10. Click **OK** again when warned that the USB will be wiped +11. Wait for it to finish (1–3 minutes), then click **CLOSE** + +#### Mac — Balena Etcher (free) + +1. Download from: **https://etcher.balena.io** — click **Download for macOS** +2. Open Etcher +3. Click **Flash from file** → select the Proxmox ISO +4. Click **Select target** → select your USB drive +5. Click **Flash** — enter your Mac password if prompted +6. Wait for it to finish + +#### Linux — terminal + +```bash +# Find your USB drive device name (look for your drive size) +lsblk + +# Write the ISO (replace sdX with your actual USB device, e.g. sdb — NOT sdb1) +dd if=proxmox-ve_9.1-1.iso of=/dev/sdX bs=1M status=progress conv=fsync +sync +``` + +--- + +### Step 5.3 — Boot the server from USB + +1. Plug the USB drive into one of the USB ports on the front or back of the server +2. Power on (or reboot) the server +3. At the Dell splash screen, press **F11** + +The **Boot Manager** screen appears: + +``` +Boot Manager + + BIOS Boot Menu + UEFI Boot Menu ← go here + One-shot BIOS Boot Menu + ... +``` + +Arrow to **UEFI Boot Menu** and press **Enter**. + +You will see a list of bootable devices. Look for your USB drive — it will be +listed as something like: + +``` +UEFI: SanDisk Ultra USB 3.0, Partition 1 +``` + +Arrow to it and press **Enter**. + +> **If the USB does not appear in the UEFI Boot Menu:** Make sure Rufus wrote in +> DD mode and GPT was selected (Step 5.2). Legacy/MBR USB drives will not appear +> in the UEFI menu. + +--- + +### Step 5.4 — Proxmox installer boot menu + +The server boots from the USB and shows the Proxmox boot menu — white text on +a blue/dark background: + +``` +Proxmox VE Installer + + Install Proxmox VE (Graphical) ← select this + Install Proxmox VE (Terminal UI) + Advanced Options + ... +``` + +Arrow to **Install Proxmox VE (Graphical)** and press **Enter**. + +The graphical installer loads. This takes about 30–60 seconds. + +--- + +### Step 5.5 — End User License Agreement + +The EULA screen appears. Read it or don't — click **I agree** at the bottom right. + +--- + +### Step 5.6 — Target disk selection + +This is the most important screen. You are choosing where Proxmox installs. + +You will see a dropdown labeled **Target Harddisk**. Click it. + +The list shows all visible storage. You are looking for the RAID1 virtual disk +you created in Phase 4. It will appear as a single disk — something like: + +``` +/dev/sda (146.00 GB) ← this is the OS-Mirror RAID1 VD +/dev/sdb (4.00 TB) +/dev/sdc (4.00 TB) +... +``` + +Select the small one — your OS-Mirror virtual disk (146 GB or whatever size your +2.5" drives are). + +> **Do not select a 3.5" drive.** Those are your VM data drives. + +**Filesystem:** Click the **Options** button next to the disk selector. + +``` +Filesystem: ext4 ← leave this as ext4 +``` + +Leave it as `ext4`. The H730 RAID1 already gives you drive redundancy. Adding +ZFS here would be redundant overhead with no benefit. + +Click **Next**. + +--- + +### Step 5.7 — Location and timezone + +``` +Country: [type your country, e.g. United States] +Time Zone: [auto-filled based on country, e.g. America/New_York] +Keyboard: [your keyboard layout, e.g. U.S. English] +``` + +Adjust if needed. Click **Next**. + +--- + +### Step 5.8 — Password and email + +``` +Password: [choose a strong root password — write it down] +Confirm: [same password again] +Email: [any email address — used for system alerts] +``` + +> The root password is how you log into Proxmox. If you forget it, recovery is +> painful. Write it on a piece of paper and put it somewhere safe. + +Click **Next**. + +--- + +### Step 5.9 — Network configuration + +``` +Management Interface: [auto-selected — usually em1 or eno1, the first NIC] +Hostname (FQDN): pve.local +IP Address: 192.168.1.10 ← change to an IP on your network +Netmask: 255.255.255.0 +Gateway: 192.168.1.1 ← your router's IP +DNS Server: 192.168.1.1 ← your router's IP (or 8.8.8.8) +``` + +**Choosing a static IP for Proxmox:** +- Log into your router and find its DHCP range +- Pick an IP outside that range (same subnet) +- Example: if DHCP is 192.168.1.100–200, use 192.168.1.10 +- This IP is how you will reach the Proxmox web UI and SSH from now on + +Click **Next**. + +--- + +### Step 5.10 — Summary and install + +The installer shows a summary of everything you chose. Confirm: +- Target disk: your small RAID1 virtual disk +- Filesystem: ext4 +- Hostname and IP look correct + +Click **Install**. + +The installation takes 5–10 minutes. A progress bar shows the steps: +- Formatting disk +- Copying files +- Setting up bootloader +- Configuring system + +When it finishes you will see: + +``` +Installation successful! +Remove the installation medium and press Enter to reboot. +``` + +**Pull the USB drive out**, then press **Enter**. + +--- + +### Step 5.11 — First login to the Proxmox web UI + +The server reboots. After about 60 seconds it shows a text console with +Proxmox's login prompt and this message: + +``` +Welcome to the Proxmox Virtual Environment. + +Please use your web browser to configure this server - +connect to: https://192.168.1.10:8006/ +``` + +On your other computer, open a browser and go to: + +``` +https://192.168.1.10:8006 +``` + +(Use the IP you set in Step 5.9.) + +**Certificate warning:** Your browser will show a security warning because +Proxmox uses a self-signed SSL certificate. This is expected and safe on +your local network. + +- Chrome/Edge: click **Advanced** → **Proceed to 192.168.1.10 (unsafe)** +- Firefox: click **Advanced** → **Accept the Risk and Continue** + +The Proxmox login page appears — a dark interface with two fields: + +``` +User name: root +Password: [the password you set in Step 5.8] +Realm: Linux PAM standard authentication ← leave this as-is +``` + +Click **Login**. + +**Subscription nag:** A popup immediately appears saying "No valid subscription". +Click **OK** to dismiss it. This appears every login until you fix the repos in +Phase 6. It does not affect anything. + +You are now looking at the Proxmox web interface — the main dashboard. + +--- + +## Phase 6 — Proxmox Post-Install + +From this point you will use **SSH** instead of the web console for most tasks. +SSH is faster and lets you paste commands directly. + +--- + +### Step 6.1 — Connect via SSH + +On your other computer, open a terminal (Mac/Linux) or PuTTY/Windows Terminal +(Windows) and run: + +```bash +ssh root@192.168.1.10 +``` + +Type `yes` when asked to confirm the host fingerprint. Enter your root password. + +You will see the Proxmox shell prompt: + +``` +root@pve:~# +``` + +All the commands in the rest of this guide are typed here. + +--- + +### Step 6.2 — Fix the package repositories + +Proxmox installs with "enterprise" apt repositories configured. These require +a paid subscription and will fail with an authentication error when you try to +update. We switch them to the free community repositories. + +Run these commands one at a time: + +```bash +# Disable the enterprise repo (comment it out) +echo "# disabled - no subscription" > /etc/apt/sources.list.d/pve-enterprise.list + +# Disable the enterprise Ceph repo +echo "# disabled - no subscription" > /etc/apt/sources.list.d/ceph.list + +# Add the no-subscription community repo +echo "deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription" \ + > /etc/apt/sources.list.d/pve-no-subscription.list +``` + +Now update and upgrade all packages: + +```bash +apt update && apt dist-upgrade -y +``` + +This will download and install updates. It may take 5–15 minutes depending on +your internet speed. Answer **Y** if prompted about any config file changes +(just press Enter to accept the default). + +When it finishes, reboot: + +```bash +reboot +``` + +Wait about 60 seconds, then SSH back in: + +```bash +ssh root@192.168.1.10 +``` + +--- + +### Step 6.3 — Install the tools the scripts need + +```bash +apt install -y ipmitool lsscsi ledmon hdparm git curl +``` + +What each tool does: + +| Tool | Used for | +|------|---------| +| ipmitool | Fan control — sends IPMI commands to iDRAC | +| lsscsi | build-bay-map.sh — lists SCSI drives with details | +| ledmon | build-bay-map.sh — blinks drive bay LEDs for identification | +| hdparm | stagger-spinup.sh — checks drive power state, wakes drives | +| git | Cloning this repo onto the host | +| curl | Downloading packages and health checks | + +--- + +### Step 6.4 — Clone this repo onto the Proxmox host + +```bash +git clone https://github.com/outis1one/local_proxmox.git /opt/local_proxmox +chmod +x /opt/local_proxmox/scripts/*.sh +cd /opt/local_proxmox +``` + +All scripts and configs are now at `/opt/local_proxmox/`. + +--- + +### Step 6.5 — Dismiss the subscription warning permanently (optional) + +If the "No valid subscription" popup annoys you every time you log into the +web UI, you can remove it with one command. This does not affect functionality: + +```bash +sed -i.bak "s/data.status !== 'Active'/false/g" \ + /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js +systemctl restart pveproxy +``` + +After running this, refresh the Proxmox web UI — the popup will be gone. + +--- + +### Phase 6 complete — where you are now + +| What is done | Status | +|---|---| +| Proxmox VE 9.1 installed on RAID1 OS mirror | ✓ | +| Free community repos configured | ✓ | +| All packages up to date | ✓ | +| Tools installed | ✓ | +| Repo cloned to /opt/local_proxmox | ✓ | + +--- + +## Phase 7 — Fan Control + Staggered Spin-Up + +**What this phase does:** Fixes two hardware quirks that bite R730xd owners +running non-Dell gear: + +1. **Fan control.** iDRAC detects the two Quadro P2200s as "non-Dell PCIe + cards" and immediately pins every fan at 100% indefinitely to be safe. + The server sounds like a jet engine and never stops. We disable iDRAC's + automatic control and replace it with a daemon that sets fan speed based + on the inlet temperature sensor. +2. **Staggered drive spin-up.** Twelve 3.5" HDDs spinning up simultaneously + at boot spike about 240 W of inrush current for 2–3 seconds — enough to + trip a 750 W PSU under load. We configure two layers of protection: the + iDRAC BIOS setting that fires at POST (before the OS loads), and a Linux + systemd service that handles any drives the BIOS setting missed. + +**Prerequisites from earlier phases:** +- IPMI over LAN enabled in iDRAC (Phase 2) — required for `ipmitool` +- `ipmitool` and `hdparm` installed on the host (Phase 6.3) +- Repo cloned to `/opt/local_proxmox` (Phase 6.4) + +--- + +### Step 7.1 — Test fan control manually before enabling the service + +Before installing anything as a systemd service, verify that the script can +actually talk to iDRAC over IPMI. This catches misconfigured IPMI-over-LAN +credentials before you wonder why the service silently failed. + +SSH into Proxmox: + +```bash +ssh root@192.168.1.10 +``` + +Check the current inlet temperature: + +```bash +bash /opt/local_proxmox/scripts/fan-control.sh --temp +``` + +Expected output: + +``` +Inlet temp: 24°C +``` + +The number will vary with your room temperature — anywhere between 18–35°C +is typical for an idle server. + +> **If you see `Inlet temp: 35°C` exactly every time** — that is the script's +> hard-coded fallback for when the sensor read fails. IPMI is not responding. +> Check Phase 2: iDRAC **Network → IPMI Settings → Enable IPMI Over LAN** +> must be checked, and the channel privilege level must be **Administrator**. + +Now test a manual fan speed override. This is safe — the script has a 15% +minimum floor hard-coded, so you cannot accidentally stop the fans: + +```bash +bash /opt/local_proxmox/scripts/fan-control.sh --set 20 +``` + +Expected output: + +``` +: iDRAC automatic fan control DISABLED +: Fans set to 20% +``` + +**Within 10–30 seconds you should hear the fans audibly slow down.** The +R730xd is loud at 100% — the difference is immediate and obvious. + +Confirm the fans actually dropped: + +```bash +ipmitool sdr type Fan +``` + +Expected output: + +``` +Fan1A | 30h | ok | 7.1 | 2400 RPM +Fan1B | 31h | ok | 7.1 | 2160 RPM +Fan2A | 32h | ok | 7.1 | 2400 RPM +Fan2B | 33h | ok | 7.1 | 2160 RPM +Fan3A | 34h | ok | 7.1 | 2400 RPM +Fan3B | 35h | ok | 7.1 | 2160 RPM +... +``` + +At 20% you should see roughly 2000–3000 RPM. At iDRAC's panic 100% you +would see 12000–15000 RPM. If the numbers look like the former you are in +good shape. + +Now hand control back to iDRAC before proceeding — we want the real service +to be what re-disables automatic control, not this test: + +```bash +bash /opt/local_proxmox/scripts/fan-control.sh --auto +``` + +Expected output: + +``` +: iDRAC automatic fan control RE-ENABLED +``` + +**The fans will immediately ramp back to 100% within a few seconds** — that +is iDRAC reacting to the Quadro again. This confirms the script was in fact +holding them down. Get the service installed quickly so the noise stops. + +--- + +### Step 7.2 — Install the fan control script and service + +Copy the script to the system directory and make it executable: + +```bash +cp /opt/local_proxmox/scripts/fan-control.sh /usr/local/sbin/fan-control.sh +chmod +x /usr/local/sbin/fan-control.sh +``` + +Copy the systemd unit into place: + +```bash +cp /opt/local_proxmox/scripts/fan-control.service /etc/systemd/system/ +``` + +Tell systemd about the new unit, then enable and start it in one command: + +```bash +systemctl daemon-reload +systemctl enable --now fan-control.service +``` + +`enable --now` both enables the unit for future boots and starts it right +now, so the fans should begin quieting within 30 seconds. + +Verify the service is running: + +```bash +systemctl status fan-control.service +``` + +Expected output (abbreviated): + +``` +● fan-control.service - Dell R730xd fan speed control (third-party GPU) + Loaded: loaded (/etc/systemd/system/fan-control.service; enabled; preset: enabled) + Active: active (running) since