# Dell R730xd — Complete Setup Walkthrough ### Proxmox VE 9.1 · GPU Passthrough · Frigate NVR Full button-by-button guide from bare metal to running VMs. All 12 phases in one document. --- ## Table of Contents - [Before You Start](#before-you-start) - [Phase 1 — Firmware Updates](#phase-1--firmware-updates) - [Phase 2 — iDRAC Setup](#phase-2--idrac-setup) - [Phase 3 — BIOS Settings](#phase-3--bios-settings) - [Phase 4 — H730 RAID Configuration](#phase-4--h730-raid-configuration) - [Phase 5 — Install Proxmox VE 9.1](#phase-5--install-proxmox-ve-91) - [Phase 6 — Proxmox Post-Install](#phase-6--proxmox-post-install) - [Phase 7 — Convert 3.5" Drives to Non-RAID Mode](#phase-7--convert-35-drives-to-non-raid-mode) - [Phase 8 — Map Physical Bays to Drives](#phase-8--map-physical-bays-to-drives) - [Phase 9 — GPU Passthrough Setup](#phase-9--gpu-passthrough-setup) - [Phase 10 — Fan Control + Staggered Spin-Up](#phase-10--fan-control--staggered-spin-up) - [Phase 11 — Create the VMs](#phase-11--create-the-vms) - [Phase 12 — Deploy Frigate in VM 100](#phase-12--deploy-frigate-in-vm-100) --- ## Before You Start ### What is iDRAC and do I need to pay for it? iDRAC stands for **Integrated Dell Remote Access Controller**. Ignore the name. Here is what it actually is: There is a **second tiny computer built into your server's motherboard**. It has its own processor, its own RAM, its own network port, and it runs 24 hours a day as long as the server has power — even when the server is "off". Dell calls this second computer iDRAC. This server has **iDRAC8 Express**, which is the version built into the board at no cost. There is a paid upgrade called iDRAC8 Enterprise but you do not need it. Here is exactly what Express gives you and what it does not: **Express includes (everything this guide uses):** - iDRAC web UI — hardware health dashboard, temperatures, fan speeds, event logs - Power the server on and off remotely from the web UI - IPMI over LAN — lets `ipmitool` send fan control commands from Proxmox - racadm — command-line control used by the stagger spin-up script - SSH directly into iDRAC for management **Express does NOT include:** - Virtual Console — you cannot see the server's screen in a browser window. A **physical monitor and keyboard are required** for all setup steps until Proxmox is installed and SSH is working. After that you will never need the monitor again. - Virtual Media — you cannot mount an ISO file over the network. A **physical USB drive** is required for the Proxmox installer. Nothing in this guide requires Enterprise. The fan control script and stagger spin-up both use IPMI over LAN and racadm, which are both available on Express. **Do you need to pay for anything?** No. iDRAC Express is already there and covers everything we need. The only other "subscription" people sometimes ask about is the **Proxmox subscription** (about €100/year for enterprise update servers and commercial support). You do not need that either — we configure the free community repositories in Phase 6. The two are completely unrelated. **Summary: pay nothing, skip nothing. Keep a monitor plugged in through Phase 6.** --- ### What you need on the table before starting - The server plugged into power and a network switch/router - A **separate network cable** for the iDRAC port (the small RJ45 on the back labeled "iDRAC" — it is separate from the four main NIC ports) - A **monitor and USB keyboard** plugged into the server — required through Phase 5 (the Proxmox installer). iDRAC Express does not include a remote KVM, so you cannot see the server's screen from another computer. After Proxmox is installed and SSH is working (end of Phase 6) the monitor can be unplugged permanently. - A USB drive (8 GB or larger) — needed later for the Proxmox installer - Another computer or phone to read this guide --- ## Phase 1 — Firmware Updates **Why first?** Old firmware has bugs. BIOS from 2016 may not correctly expose the VT-d settings that GPU passthrough requires. The H730 firmware has had fixes for JBOD/non-RAID mode. Updating now prevents chasing ghosts later. **Time required:** 30–90 minutes. The server will reboot several times on its own. --- ### Step 1.1 — Power on and watch the screen Press the power button on the front of the server. The screen will show a **Dell splash screen** with the PowerEdge logo. At the bottom of the screen you will see a line of options, something like: ``` F2 = System Setup F10 = Lifecycle Controller F11 = Boot Manager F12 = PXE Boot ``` These options are only available for about **5–8 seconds** before the server continues booting. If you miss the window the server will try to boot an OS (and fail if nothing is installed yet). Just power it off and back on and try again. --- ### Step 1.2 — Enter Lifecycle Controller Press **F10** when you see the splash screen. The screen will go blank for 10–30 seconds, then show the **Lifecycle Controller** loading screen. This is a mini operating system built into the server's firmware. It has nothing to do with your server's main OS. Wait for it to finish loading — it can take up to 90 seconds the first time. When it finishes you will see the **Lifecycle Controller Home** screen with a menu on the left side: ``` Home Firmware Update OS Deployment RAID Configuration Hardware Configuration Settings ``` If it asks you to complete initial setup (language, network), do that first — use Tab and arrow keys to navigate, Enter to confirm. --- ### Step 1.3 — Go to Firmware Update Click or arrow-key to **Firmware Update** on the left menu, then press Enter. You will see: ``` Launch Firmware Update ``` Press Enter on that. --- ### Step 1.4 — Choose your update method You will be asked how you want to get the firmware. You have two options: #### Option A — Server has internet access (easiest) If your network cable is plugged in and the server can reach the internet: 1. Select **HTTPS** (it will say something like "downloads.dell.com") 2. Press **Next** 3. Lifecycle Controller will connect to Dell's servers and **automatically scan your server** to find what firmware versions are installed vs. what is available 4. This takes 2–5 minutes — the screen will show a progress bar 5. Skip to Step 1.5 #### Option B — No internet on the server You need to download the firmware files on another computer first. 1. On your other computer, open a browser and go to: **https://www.dell.com/support/home** 2. You need your server's **Service Tag** — it is a 7-character code. Find it: - On the **front of the server**: there is a small pull-out plastic tab (like a credit card slot) on the left side of the front panel. Pull it out — the Service Tag is printed on it. - Or: on a sticker on the **top** of the server chassis. - Or: it was displayed briefly during POST on the splash screen. 3. On the Dell support page, click **"View products"** or type the Service Tag into the search bar at the top. Select your server from the results. 4. Click **"Drivers & Downloads"** 5. You will see a filter panel. Under **"Category"**, select each of these one at a time and download the latest version: - **iDRAC** — look for "iDRAC8 firmware" — file will end in `.exe` or `.d9` - **BIOS** — look for "BIOS" — file ends in `.exe` - **RAID** — look for "PERC H730" — file ends in `.exe` 6. Copy all downloaded files onto a **FAT32-formatted USB drive** 7. Plug the USB drive into the server 8. In Lifecycle Controller: select **Local Drive (USB)** and press Next 9. Navigate to the USB drive and select the firmware files --- ### Step 1.5 — Review and apply updates Lifecycle Controller shows a table of all components with columns like: ``` Component Current Version Available Version Select --------- --------------- ----------------- ------ iDRAC8 2.40.40.40 2.85.85.85 [ ] BIOS 1.2.10 2.14.0 [ ] PERC H730 Mini 25.4.0.0018 25.5.9.0001 [ ] ``` 1. Check the box next to **every component** that shows a newer Available Version (you can usually click "Select All") 2. Click **Install and Reboot** (or **Apply**) 3. A warning will say the server will reboot — click **Yes** The server will now: - Apply each firmware update - Reboot automatically between some updates - Return to the Lifecycle Controller when finished **Do not power off the server during this process.** Just wait. It can take 30–60 minutes if there are many updates. The screen will show progress. --- ### Step 1.6 — Confirm completion When all updates are done, Lifecycle Controller returns to its home screen. You may see a success summary screen first — click **OK** or **Finish**. To verify everything worked: 1. In Lifecycle Controller, go to **Firmware Update** → **Launch Firmware Update** again 2. Run the check one more time 3. The Available Version column should now match Current Version for everything you updated (or show no newer version) Press **Exit** (or **Finish**) to leave Lifecycle Controller. The server will reboot normally. --- ## Phase 2 — iDRAC Setup You need to do two things with iDRAC: 1. Give it a **static IP address** so it is always reachable at the same address 2. Enable **IPMI over LAN** so the fan control script can send commands to it --- ### Step 2.1 — Enter System Setup (F2) Power on (or reboot) the server. When the Dell splash screen appears, press **F2**. The screen will go blank briefly, then show the **System Setup Main Menu**: ``` System Setup Main Menu System BIOS iDRAC Settings Device Settings Service Tag Settings ``` Use the **arrow keys** to highlight **iDRAC Settings** and press **Enter**. --- ### Step 2.2 — Set a static IP for iDRAC Inside iDRAC Settings you will see another menu: ``` iDRAC Settings Network User Configuration Smart Card Update and Rollback ... ``` Arrow down to **Network** and press **Enter**. The Network screen has several sections. You are looking for the **IPv4 Settings** section. Use Tab or arrow keys to navigate to these fields: | Field | What to set | |-------|-------------| | Enable NIC | **Enabled** | | NIC Selection | **Dedicated** (this uses the dedicated iDRAC port on the back) | | Enable IPv4 | **Enabled** | | Enable DHCP | **Disabled** ← change this | | Static IP Address | e.g. `192.168.1.5` (pick an IP outside your router's DHCP range) | | Static Gateway | your router's IP, e.g. `192.168.1.1` | | Static Subnet Mask | `255.255.255.0` | | DNS Server 1 | your router's IP, e.g. `192.168.1.1` | > **What IP to use?** Log into your router and find its DHCP range — for example > if DHCP hands out `192.168.1.100` to `192.168.1.200`, pick something outside > that range like `192.168.1.5`. Write this IP down — you will use it often. When done, press **Back** or navigate to the bottom and click **Apply**, then **OK**. --- ### Step 2.3 — Enable IPMI over LAN Still inside iDRAC Settings → Network. Scroll down past the IPv4 section until you see **IPMI Settings**. Find: ``` Enable IPMI over LAN: [ Disabled ] ``` Change this to **Enabled**. This is the setting that allows `ipmitool` commands (used by the fan control script) to reach iDRAC over the network. Click **Apply** at the bottom, then **OK** when asked to confirm. --- ### Step 2.4 — Change the default password Go back to the iDRAC Settings main menu (press **Back**). Arrow down to **User Configuration** and press **Enter**. You will see a list of user slots. **User 1** is the built-in `root` account. Select it (press Enter). > **Why change it?** The factory default password is `calvin`. This is printed > in Dell's public documentation and is widely known. Any device on your network > could log into your iDRAC if you leave it as-is. Change: - **User Name:** leave as `root` - **Change Password:** set to **Enabled** - **Password:** enter a strong password - **Confirm Password:** enter it again Click **Apply**, then **OK**. Press **Finish** to exit iDRAC Settings and return to the System Setup Main Menu. **Do not reboot yet** — you still have BIOS settings to configure in Phase 3. --- ### Step 2.5 — Verify iDRAC from your browser (do after Phase 3) After you finish Phase 3 and let the server reboot, test iDRAC from your other computer: 1. Make sure the **dedicated iDRAC network port** (small RJ45 labeled "iDRAC" on the server's back panel) is plugged into your switch or router 2. Open a browser on your other computer 3. Go to: `https://192.168.1.5` (whatever static IP you set in Step 2.2) 4. You will see a **certificate warning** — this is normal, iDRAC uses a self-signed certificate. Click **Advanced** → **Proceed anyway** (wording varies by browser) 5. The iDRAC8 login page appears — a dark Dell-branded page 6. Log in: Username `root`, Password (the one you set in Step 2.4) You should see the iDRAC dashboard showing system health, temperatures, and fans. This is your remote window into the server — you can now manage it from your desk. --- ## Phase 3 — BIOS Settings These settings enable the CPU features that allow devices (GPUs, USB controllers) to be handed directly to virtual machines. **Every setting below is required.** Skipping VT-d is the single most common reason GPU passthrough silently fails. You are still in System Setup from Phase 2 (or press F2 again on reboot). --- ### Step 3.1 — System BIOS → Processor Settings From the System Setup Main Menu, arrow to **System BIOS** and press Enter. You will see the System BIOS Settings menu: ``` System BIOS Settings System Information Memory Settings Processor Settings ← go here SATA Settings Boot Settings Integrated Devices Serial Communication System Profile Settings Power Management Security Miscellaneous Settings ``` Arrow to **Processor Settings** and press **Enter**. Find and set these two options: --- **Virtualization Technology** This enables the CPU to run virtual machines efficiently. Without it Proxmox still works but performance is worse. ``` Virtualization Technology: [ Disabled ] ↓ change to Virtualization Technology: [ Enabled ] ``` Use the arrow keys or spacebar to toggle the value. --- **VT for Direct I/O** (also shown as "Virtualization Technology for Directed I/O") This is the critical one. It enables **IOMMU** — the hardware feature that lets the CPU safely hand a real PCIe device (like your GPU) directly to a virtual machine. Without this, passthrough is impossible. ``` VT for Direct I/O: [ Disabled ] ↓ change to VT for Direct I/O: [ Enabled ] ``` Click **Back** to return to System BIOS Settings. --- ### Step 3.2 — System BIOS → Integrated Devices From the System BIOS Settings menu, arrow to **Integrated Devices** and press Enter. Find: **SR-IOV Global Enable** SR-IOV is a PCIe feature that allows a single physical device to appear as multiple devices. Even though the Quadro P2200 does not use SR-IOV, enabling this globally avoids a class of IOMMU grouping problems. ``` SR-IOV Global Enable: [ Disabled ] ↓ change to SR-IOV Global Enable: [ Enabled ] ``` Click **Back**. --- ### Step 3.3 — System BIOS → Boot Settings From System BIOS Settings, arrow to **Boot Settings** and press Enter. You will see: ``` Boot Settings Boot Mode: [ BIOS ] ← must change this Boot Sequence ... ``` **Boot Mode** — change from BIOS to UEFI: ``` Boot Mode: [ BIOS ] ↓ change to Boot Mode: [ UEFI ] ``` > **Why UEFI?** Proxmox uses a tool called `proxmox-boot-tool` to manage the > kernel boot parameters (including the IOMMU settings the GPU passthrough script > writes). This tool only works with UEFI boot. If the server is in Legacy/BIOS > mode the passthrough script will fail and you will have to reinstall. A warning may appear saying the boot sequence will be cleared — click **Yes** or **OK**. That is fine since we haven't installed anything yet. Click **Back**. --- ### Step 3.4 — System BIOS → Power Management From System BIOS Settings, arrow to **Power Management** and press Enter. Find: **Hard Disk Drive Sequencing** — this is the built-in stagger that fires at power-on before the OS loads. It staggers when each drive spins up so they do not all surge at the same time and overload the PSU. ``` Hard Disk Drive Sequencing: [ Disabled ] ↓ change to Hard Disk Drive Sequencing: [ Enabled ] ``` Also check: **C States** — these are CPU power-saving sleep states. They can add latency to VM workloads. Optional but worth setting for a server: ``` C States: [ Enabled ] ↓ change to C States: [ Disabled ] ``` Click **Back**. --- ### Step 3.5 — Apply and exit From System BIOS Settings, click **Finish** (at the bottom of the menu, you may need to scroll). A dialog will appear: ``` Confirm changes and exit? [ Yes ] [ No ] ``` Click **Yes**. The server will reboot. --- ### Step 3.6 — Confirm the settings took After the reboot, press **F2** again to re-enter System Setup and spot-check: - **Processor Settings:** VT and VT for Direct I/O should both show **Enabled** - **Boot Settings:** Boot Mode should show **UEFI** If either shows Disabled, set it again and apply. Some BIOS versions have a bug where the first Apply does not persist — applying a second time fixes it. --- ### Phase 3 complete — what you have now | Setting | Why it matters | |---------|---------------| | VT-x (Virtualization Technology) | Lets the CPU run VMs at near-native speed | | VT-d (Direct I/O) | Enables IOMMU — the foundation of all device passthrough | | SR-IOV | Prevents IOMMU group problems with PCIe devices | | UEFI boot mode | Required for Proxmox's boot management and EFI GPU passthrough | | HDD sequencing | Staggers drive spin-up at power-on to protect the PSU | --- ## Phase 4 — H730 RAID Configuration **Goal:** Mirror the two 2.5" rear drives together so a single drive failure cannot take down Proxmox. Leave every 3.5" drive untouched — the `perc-nonraid.sh` script handles those later from inside Proxmox. **Why mirror the OS drives here instead of using ZFS?** The H730 presents the RAID1 mirror as a single virtual disk to the OS. Proxmox installs onto it and never needs to think about the fact that two physical drives are underneath. This is the simplest and most reliable approach for an OS volume. --- ### Step 4.1 — Enter Lifecycle Controller Power on (or reboot) the server and press **F10** at the Dell splash screen. Wait for Lifecycle Controller to load. You will see the main menu on the left: ``` Home Firmware Update OS Deployment RAID Configuration ← go here Hardware Configuration Settings ``` Arrow to **RAID Configuration** and press **Enter**. --- ### Step 4.2 — Open the RAID configuration wizard You will see: ``` RAID Configuration View Current Configuration Create New VD ... ``` First, click **View Current Configuration** to see what is already there. You are looking at the physical drives the H730 can see. Drives are listed under their controller — it will say something like: ``` PERC H730 Mini (Slot 0) Physical Disks: Port 0: SEAGATE ST4000NM0023 4.0 TB State: Unconfigured Good Port 1: SEAGATE ST4000NM0023 4.0 TB State: Unconfigured Good ... Port 8: TOSHIBA MK1401GRRB 146 GB State: Unconfigured Good ← 2.5" Port 9: TOSHIBA MK1401GRRB 146 GB State: Unconfigured Good ← 2.5" ``` > **Identifying the 2.5" OS drives:** They will be noticeably smaller than the > 3.5" drives — likely 146 GB, 300 GB, 600 GB (SAS) or 120–480 GB (SSD). > The 3.5" drives will be 1 TB, 2 TB, 4 TB, or larger. Size is the giveaway. > **If drives show "Foreign Configuration":** They were previously part of a > RAID array. See Step 4.3a before continuing. Press **Back** to return to the RAID Configuration menu. --- ### Step 4.3a — Clear foreign configurations (only if needed) If any drives showed "Foreign Configuration" in Step 4.2, you need to clear them before you can create a new virtual disk. In the RAID Configuration menu, look for: ``` Clear Foreign Configuration ``` Select it, choose **All Foreign Configurations**, click **Apply**. A warning will appear saying all data on those drives will be lost — click **Yes**. (We are building a new system so there is nothing to keep.) After clearing, go back to **View Current Configuration** and confirm the drives now show **Unconfigured Good**. --- ### Step 4.4 — Create the RAID 1 virtual disk In the RAID Configuration menu, select **Create New VD** and press **Enter**. The wizard walks you through four screens: --- **Screen 1 — Select RAID Level** ``` Select RAID Level: RAID 0 (no redundancy, faster) RAID 1 (mirror, one drive can fail) ← select this RAID 5 (requires 3+ drives) RAID 6 (requires 4+ drives) RAID 10 (requires 4+ drives) ``` Select **RAID 1** and click **Next**. --- **Screen 2 — Select Physical Disks** A list of all available physical disks appears. You need to select **only the two 2.5" drives**. Do not select any 3.5" drives. Click the checkbox next to each of the two 2.5" drives (identified by their smaller size). Leave all 3.5" drives unchecked. Click **Next**. --- **Screen 3 — Virtual Disk Attributes** ``` Virtual Disk Name: OS-Mirror Virtual Disk Size: (auto-filled — leave it) Strip Element Size: 64KB (leave default) Read Policy: Adaptive Read Ahead (leave default) Write Policy: Write Back (leave default) Disk Cache Policy: Enabled (leave default) ``` The only thing you need to change is the name — type `OS-Mirror` so it is easy to identify. Everything else can stay at defaults. Click **Next**. --- **Screen 4 — Confirm** Review the summary. It should show: - RAID Level: 1 - Physical Disks: 2 (your 2.5" drives) - Virtual Disk Size: whatever the smaller of the two drives is Click **Finish**. --- ### Step 4.5 — Initialize the virtual disk After creating the VD, Lifecycle Controller will ask about initialization: ``` Initialize virtual disk? Fast Initialize ← select this Full Initialize (writes zeros to every sector — takes hours, not needed) Skip ``` Select **Fast Initialize** and click **OK**. Fast initialization takes 30–60 seconds. A progress bar will appear. Wait for it. --- ### Step 4.6 — Verify and exit When initialization finishes, go back to **View Current Configuration**. You should now see: ``` PERC H730 Mini Virtual Disks: VD 0: OS-Mirror RAID 1 ~146 GB State: Optimal Physical Disks: Port 0–7: 3.5" drives State: Unconfigured Good Port 8–9: 2.5" drives State: Online (member of VD 0) ``` The 3.5" drives should all show **Unconfigured Good**. That is exactly what you want. Leave them that way. Click **Back** repeatedly until you reach the Lifecycle Controller home screen, then click **Exit**. The server will reboot. --- ## Phase 5 — Install Proxmox VE 9.1 ### Step 5.1 — Download the Proxmox ISO On your other computer, open a browser and go to: ``` https://www.proxmox.com/en/downloads ``` Click **Proxmox Virtual Environment** → find the latest **Proxmox VE 9.x ISO Installer** and click **Download**. The file will be named something like `proxmox-ve_9.1-1.iso` and is about 1 GB. --- ### Step 5.2 — Write the ISO to a USB drive You need to write the ISO as a disk image (not copy the file). Use one of these tools depending on your computer: #### Windows — Rufus (free, no install needed) 1. Download Rufus from: **https://rufus.ie** (click the first .exe link) 2. Plug in your USB drive (8 GB or larger — all data on it will be erased) 3. Open Rufus 4. **Device:** select your USB drive from the dropdown 5. **Boot selection:** click **SELECT** and choose the Proxmox ISO file 6. **Partition scheme:** select **GPT** 7. **Target system:** select **UEFI (non CSM)** 8. Click **START** 9. A dialog appears asking about ISO mode vs DD mode — select **Write in DD Image mode** and click **OK** 10. Click **OK** again when warned that the USB will be wiped 11. Wait for it to finish (1–3 minutes), then click **CLOSE** #### Mac — Balena Etcher (free) 1. Download from: **https://etcher.balena.io** — click **Download for macOS** 2. Open Etcher 3. Click **Flash from file** → select the Proxmox ISO 4. Click **Select target** → select your USB drive 5. Click **Flash** — enter your Mac password if prompted 6. Wait for it to finish #### Linux — terminal ```bash # Find your USB drive device name (look for your drive size) lsblk # Write the ISO (replace sdX with your actual USB device, e.g. sdb — NOT sdb1) dd if=proxmox-ve_9.1-1.iso of=/dev/sdX bs=1M status=progress conv=fsync sync ``` --- ### Step 5.3 — Boot the server from USB 1. Plug the USB drive into one of the USB ports on the front or back of the server 2. Power on (or reboot) the server 3. At the Dell splash screen, press **F11** The **Boot Manager** screen appears: ``` Boot Manager BIOS Boot Menu UEFI Boot Menu ← go here One-shot BIOS Boot Menu ... ``` Arrow to **UEFI Boot Menu** and press **Enter**. You will see a list of bootable devices. Look for your USB drive — it will be listed as something like: ``` UEFI: SanDisk Ultra USB 3.0, Partition 1 ``` Arrow to it and press **Enter**. > **If the USB does not appear in the UEFI Boot Menu:** Make sure Rufus wrote in > DD mode and GPT was selected (Step 5.2). Legacy/MBR USB drives will not appear > in the UEFI menu. --- ### Step 5.4 — Proxmox installer boot menu The server boots from the USB and shows the Proxmox boot menu — white text on a blue/dark background: ``` Proxmox VE Installer Install Proxmox VE (Graphical) ← select this Install Proxmox VE (Terminal UI) Advanced Options ... ``` Arrow to **Install Proxmox VE (Graphical)** and press **Enter**. The graphical installer loads. This takes about 30–60 seconds. --- ### Step 5.5 — End User License Agreement The EULA screen appears. Read it or don't — click **I agree** at the bottom right. --- ### Step 5.6 — Target disk selection This is the most important screen. You are choosing where Proxmox installs. You will see a dropdown labeled **Target Harddisk**. Click it. The list shows all visible storage. You are looking for the RAID1 virtual disk you created in Phase 4. It will appear as a single disk — something like: ``` /dev/sda (146.00 GB) ← this is the OS-Mirror RAID1 VD /dev/sdb (4.00 TB) /dev/sdc (4.00 TB) ... ``` Select the small one — your OS-Mirror virtual disk (146 GB or whatever size your 2.5" drives are). > **Do not select a 3.5" drive.** Those are your VM data drives. **Filesystem:** Click the **Options** button next to the disk selector. ``` Filesystem: ext4 ← leave this as ext4 ``` Leave it as `ext4`. The H730 RAID1 already gives you drive redundancy. Adding ZFS here would be redundant overhead with no benefit. Click **Next**. --- ### Step 5.7 — Location and timezone ``` Country: [type your country, e.g. United States] Time Zone: [auto-filled based on country, e.g. America/New_York] Keyboard: [your keyboard layout, e.g. U.S. English] ``` Adjust if needed. Click **Next**. --- ### Step 5.8 — Password and email ``` Password: [choose a strong root password — write it down] Confirm: [same password again] Email: [any email address — used for system alerts] ``` > The root password is how you log into Proxmox. If you forget it, recovery is > painful. Write it on a piece of paper and put it somewhere safe. Click **Next**. --- ### Step 5.9 — Network configuration ``` Management Interface: [auto-selected — usually em1 or eno1, the first NIC] Hostname (FQDN): pve.local IP Address: 192.168.1.10 ← change to an IP on your network Netmask: 255.255.255.0 Gateway: 192.168.1.1 ← your router's IP DNS Server: 192.168.1.1 ← your router's IP (or 8.8.8.8) ``` **Choosing a static IP for Proxmox:** - Log into your router and find its DHCP range - Pick an IP outside that range (same subnet) - Example: if DHCP is 192.168.1.100–200, use 192.168.1.10 - This IP is how you will reach the Proxmox web UI and SSH from now on Click **Next**. --- ### Step 5.10 — Summary and install The installer shows a summary of everything you chose. Confirm: - Target disk: your small RAID1 virtual disk - Filesystem: ext4 - Hostname and IP look correct Click **Install**. The installation takes 5–10 minutes. A progress bar shows the steps: - Formatting disk - Copying files - Setting up bootloader - Configuring system When it finishes you will see: ``` Installation successful! Remove the installation medium and press Enter to reboot. ``` **Pull the USB drive out**, then press **Enter**. --- ### Step 5.11 — First login to the Proxmox web UI The server reboots. After about 60 seconds it shows a text console with Proxmox's login prompt and this message: ``` Welcome to the Proxmox Virtual Environment. Please use your web browser to configure this server - connect to: https://192.168.1.10:8006/ ``` On your other computer, open a browser and go to: ``` https://192.168.1.10:8006 ``` (Use the IP you set in Step 5.9.) **Certificate warning:** Your browser will show a security warning because Proxmox uses a self-signed SSL certificate. This is expected and safe on your local network. - Chrome/Edge: click **Advanced** → **Proceed to 192.168.1.10 (unsafe)** - Firefox: click **Advanced** → **Accept the Risk and Continue** The Proxmox login page appears — a dark interface with two fields: ``` User name: root Password: [the password you set in Step 5.8] Realm: Linux PAM standard authentication ← leave this as-is ``` Click **Login**. **Subscription nag:** A popup immediately appears saying "No valid subscription". Click **OK** to dismiss it. This appears every login until you fix the repos in Phase 6. It does not affect anything. You are now looking at the Proxmox web interface — the main dashboard. --- ## Phase 6 — Proxmox Post-Install From this point you will use **SSH** instead of the web console for most tasks. SSH is faster and lets you paste commands directly. --- ### Step 6.1 — Connect via SSH On your other computer, open a terminal (Mac/Linux) or PuTTY/Windows Terminal (Windows) and run: ```bash ssh root@192.168.1.10 ``` Type `yes` when asked to confirm the host fingerprint. Enter your root password. You will see the Proxmox shell prompt: ``` root@pve:~# ``` All the commands in the rest of this guide are typed here. --- ### Step 6.2 — Fix the package repositories Proxmox installs with "enterprise" apt repositories configured. These require a paid subscription and will fail with an authentication error when you try to update. We switch them to the free community repositories. Run these commands one at a time: ```bash # Disable the enterprise repo (comment it out) echo "# disabled - no subscription" > /etc/apt/sources.list.d/pve-enterprise.list # Disable the enterprise Ceph repo echo "# disabled - no subscription" > /etc/apt/sources.list.d/ceph.list # Add the no-subscription community repo echo "deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription" \ > /etc/apt/sources.list.d/pve-no-subscription.list ``` Now update and upgrade all packages: ```bash apt update && apt dist-upgrade -y ``` This will download and install updates. It may take 5–15 minutes depending on your internet speed. Answer **Y** if prompted about any config file changes (just press Enter to accept the default). When it finishes, reboot: ```bash reboot ``` Wait about 60 seconds, then SSH back in: ```bash ssh root@192.168.1.10 ``` --- ### Step 6.3 — Install the tools the scripts need ```bash apt install -y ipmitool lsscsi ledmon hdparm git curl ``` What each tool does: | Tool | Used for | |------|---------| | ipmitool | Fan control — sends IPMI commands to iDRAC | | lsscsi | build-bay-map.sh — lists SCSI drives with details | | ledmon | build-bay-map.sh — blinks drive bay LEDs for identification | | hdparm | stagger-spinup.sh — checks drive power state, wakes drives | | git | Cloning this repo onto the host | | curl | Downloading packages and health checks | --- ### Step 6.4 — Clone this repo onto the Proxmox host ```bash git clone https://github.com/outis1one/local_proxmox.git /opt/local_proxmox chmod +x /opt/local_proxmox/scripts/*.sh cd /opt/local_proxmox ``` All scripts and configs are now at `/opt/local_proxmox/`. --- ### Step 6.5 — Dismiss the subscription warning permanently (optional) If the "No valid subscription" popup annoys you every time you log into the web UI, you can remove it with one command. This does not affect functionality: ```bash sed -i.bak "s/data.status !== 'Active'/false/g" \ /usr/share/javascript/proxmox-widget-toolkit/proxmoxlib.js systemctl restart pveproxy ``` After running this, refresh the Proxmox web UI — the popup will be gone. --- ### Phase 6 complete — where you are now | What is done | Status | |---|---| | Proxmox VE 9.1 installed on RAID1 OS mirror | ✓ | | Free community repos configured | ✓ | | All packages up to date | ✓ | | Tools installed | ✓ | | Repo cloned to /opt/local_proxmox | ✓ | --- ## Phase 7 — Convert 3.5" Drives to Non-RAID Mode **What this does:** Right now the H730 sees your 3.5" drives as "Unconfigured Good" — they exist but the controller is not doing anything with them. We need to put them into **Non-RAID mode**, which tells the H730 to present each drive directly to Linux as an individual block device, with SMART health data intact. Without this step, Linux cannot see the drives at all — the H730 hides them. **The 2.5" OS drives are not affected** — they are a RAID1 virtual disk and the script only targets unconfigured physical drives. --- ### Step 7.1 — Download and install perccli `perccli` is Dell's command-line tool for managing the H730. The `perc-nonraid.sh` script uses it to talk to the controller. SSH into the Proxmox host if you are not already connected: ```bash ssh root@192.168.1.10 ``` Download and install perccli on the Proxmox host. Dell only distributes an RPM, so we convert it with `alien`: ```bash # Download (referer header required — Dell blocks plain curl) curl -L \ --referer "https://www.dell.com/" \ --user-agent "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36" \ "https://dl.dell.com/FOLDER03559396M/1/perccli-1.17.10-1.noarch.rpm" \ -o /tmp/perccli.rpm # Convert RPM → deb and install apt install -y alien alien --to-deb /tmp/perccli.rpm dpkg -i /tmp/perccli_*.deb # Binary lands in /opt/MegaRAID/perccli/ — symlink it into PATH ln -s /opt/MegaRAID/perccli/perccli64 /usr/local/bin/perccli ``` Verify it installed: ```bash perccli show ``` You should see output starting with something like: ``` CLI Version = 007.1907.0000.0000 Operating system = Linux5.x Controller = 0 Status = Success ... ``` If you see `command not found`, try `perccli64` instead — some versions install under that name. --- ### Step 7.2 — Run the non-RAID conversion script ```bash bash /opt/local_proxmox/scripts/perc-nonraid.sh ``` The script first prints a summary of your controller and all drives: ``` === Controller overview === ... === Drives on controller 0 === ------------------------------------------------------------- EID:Slt DID State DG Size Intf Med SED PI SeSz Model Sp ------------------------------------------------------------- 8:0 7 UGood - 3.637 TB SATA HDD N N 512B WD... U 8:1 8 UGood - 3.637 TB SATA HDD N N 512B WD... U ... 8:9 16 Onln 0 136.73 GB SATA SSD N N 512B TOSHIBA U ← OS drive (in VD) 8:10 17 Onln 0 136.73 GB SATA SSD N N 512B TOSHIBA U ← OS drive (in VD) ``` The columns that matter: - **EID:Slt** — Enclosure ID and slot number (physical location in the backplane) - **State** — `UGood` means unconfigured and ready. `Onln` means it is part of a virtual disk (your OS mirror). The script only touches `UGood` drives. - **Size** — confirms which are 3.5" data drives vs the small OS drives The script then asks: ``` Proceed with converting all non-OS drives to Non-RAID mode? [y/N] ``` Type `y` and press Enter. The script works through each drive. For each one you will see: ``` Slot 0: setting to Good... Status = Success Slot 0: setting to Non-RAID... Status = Success Slot 1: setting to Good... Status = Success Slot 1: setting to Non-RAID... Status = Success ... ``` When all drives are done the script prints the final state — drives that were `UGood` should now show `JBOD` (which is Dell's term for non-RAID/per-disk mode): ``` === Final drive state === EID:Slt State 8:0 JBOD ← 8:1 JBOD ← 8:2 JBOD ← ... 8:9 Onln ← OS drive, untouched 8:10 Onln ← OS drive, untouched ``` --- ### Step 7.3 — Reboot ```bash reboot ``` Wait about 60 seconds, then SSH back in: ```bash ssh root@192.168.1.10 ``` ### Step 7.4 — Verify drives are visible ```bash lsblk -d -o NAME,SIZE,TYPE,ROTA ``` You should see your drives listed. Something like: ``` NAME SIZE TYPE ROTA sda 136.7G disk 0 ← OS-Mirror RAID1 virtual disk (the 2.5" drives) sdb 3.6T disk 1 ← 3.5" data drive sdc 3.6T disk 1 sdd 3.6T disk 1 ... ``` If you see only `sda` and nothing else, the non-RAID conversion did not take effect — reboot again and re-run `lsblk`. If still missing, re-run `perc-nonraid.sh` and check the output for any lines that say `Failed`. --- ## Phase 8 — Map Physical Bays to Drives **The problem:** Linux names drives `sdb`, `sdc`, `sdd` etc. based on the order it finds them at boot — that order can change. What you need is the **stable hardware ID** (`/dev/disk/by-id/...`) for each drive, tied to its physical bay number, so you can reliably assign "bay 3" to a specific VM forever. This phase generates that map. --- ### Step 8.1 — Run the bay mapping script ```bash bash /opt/local_proxmox/scripts/build-bay-map.sh ``` Output looks like this (truncated example): ``` === Drive inventory === DEVICE SIZE MODEL SERIAL BY-ID PATH ------ ---- ----- ------ --------- /dev/sdb 3.6T WDC_WD4000FYYZ WD-XXXXXXXXXXXX /dev/disk/by-id/scsi-35000cca23b7d4eb8 /dev/sdc 3.6T WDC_WD4000FYYZ WD-XXXXXXXXXXXX /dev/disk/by-id/scsi-35000cca23b5e1234 /dev/sdd 3.6T ST4000NM0023 Z1Z2XXXXXX /dev/disk/by-id/scsi-35000c500a0000001 ... === Bay identification via LED blink === To confirm which physical bay a device is in, blink its LED: ledctl locate=/dev/sdX # LED on ledctl locate_off=/dev/sdX # LED off ``` The script gives you the by-id path for each device. Now you need to figure out **which physical bay each device is in**. --- ### Step 8.2 — Walk the bays with LED blink This is the physical part. You need to be at the server (or have someone there). For each drive, run the blink command, walk to the server, see which bay's amber LED is lit, note the bay number, then turn it off: ```bash # Blink sdb ledctl locate=/dev/sdb # Walk to the server, find the lit bay — write down: sdb = bay X ledctl locate_off=/dev/sdb # Blink sdc ledctl locate=/dev/sdc # Walk to server, find the lit bay — write down: sdc = bay X ledctl locate_off=/dev/sdc # Repeat for each drive ``` > **Bay numbering on the R730xd:** Bays are numbered left to right, top to > bottom when facing the front of the server. Bay 1 is top-left. The exact > labeling depends on your bezel — some models label them 0–11, others 1–12. > Use whatever number is printed or silk-screened on the chassis next to the bay. > **If the LED does not blink:** The `ledmon` daemon must be running. Start it: > ```bash > systemctl start ledmon > ledctl locate=/dev/sdb > ``` --- ### Step 8.3 — Fill in the hardware layout document Open the layout document: ```bash nano /opt/local_proxmox/docs/hardware-layout.md ``` Fill in the table using what you noted in Step 8.2. It looks like this — fill in the `by-id` column from the script output and the bay number from the LED walk: ``` | Bay | Assigned VM | Size | /dev/disk/by-id (fill in) | |-----|-------------|------|---------------------------------------------| | 1 | 100 | 4TB | scsi-35000cca23b7d4eb8 | | 2 | 100 | 4TB | scsi-35000cca23b5e1234 | ... ``` Save with **Ctrl+O**, Enter, then **Ctrl+X** to exit nano. --- ### Step 8.4 — Fill in the VM config placeholders Now open each VM config example and replace the `PLACEHOLDER_BAYx` entries with the real by-id paths: ```bash nano /opt/local_proxmox/vm-configs/100-frigate.conf.example ``` Change lines like: ``` scsi1: /dev/disk/by-id/PLACEHOLDER_BAY1,size=0 ``` To the real path: ``` scsi1: /dev/disk/by-id/scsi-35000cca23b7d4eb8,size=0 ``` Do this for all 8 drives in VM 100, drives 9–10 in VM 101, and drives 11–12 in VM 102. Save each file. > **The full path starts with `/dev/disk/by-id/`** — but in the Proxmox VM > config you write the full path. Double-check by running: > ```bash > ls -la /dev/disk/by-id/ | grep -v part > ``` > You will see the symlinks and the drives they point to. --- ## Phase 9 — GPU Passthrough Setup **What this does:** Tells the Linux kernel to stop trying to use the two Quadro P2200s itself and instead hand them over to the VFIO driver, which holds them ready to be claimed by a virtual machine. Three things happen: 1. IOMMU is turned on in the kernel (the hardware feature VT-d enables) 2. The host's NVIDIA/nouveau GPU drivers are blocked from loading 3. The VFIO driver claims both GPUs at boot, before any other driver can --- ### Step 9.1 — Check your IOMMU groups first Before running the script, verify that VT-d is actually active: ```bash dmesg | grep -i iommu | head -10 ``` You should see lines like: ``` DMAR: IOMMU enabled Intel-IOMMU: enabled ``` If you see nothing or see "disabled", IOMMU is not active. Go back to Phase 3 and re-check that VT-d is set to **Enabled** in BIOS, then reboot and try again. Also check that each GPU is in its own IOMMU group: ```bash for d in /sys/kernel/iommu_groups/*/devices/*; do n=${d#*/iommu_groups/*}; n=${n%%/*} printf 'IOMMU Group %s ' "$n" lspci -nns "${d##*/}" done | sort -V | grep -i nvidia ``` Example good output — each GPU is in a different group: ``` IOMMU Group 24 03:00.0 VGA compatible controller [0300]: NVIDIA GP106GL [Quadro P2200] [10de:1c35] IOMMU Group 24 03:00.1 Audio device [0403]: NVIDIA GP106 High Definition Audio [10de:10f1] IOMMU Group 31 04:00.0 VGA compatible controller [0300]: NVIDIA GP106GL [Quadro P2200] [10de:1c35] IOMMU Group 31 04:00.1 Audio device [0403]: NVIDIA GP106 High Definition Audio [10de:10f1] ``` Each GPU (`03:00.0` and `04:00.0`) and its audio sibling (`03:00.1` and `04:00.1`) are together in their own group — that is exactly what you want. > **If both GPUs are in the same IOMMU group as other devices** (chipset, NICs, > etc.), you may need to enable **ACS** (Access Control Services). This is rare > on server hardware like the R730xd which has good IOMMU separation. If you > hit this, ask before proceeding. --- ### Step 9.2 — Run the GPU passthrough script ```bash bash /opt/local_proxmox/scripts/gpu-passthrough-setup.sh ``` The script first prints your NVIDIA devices: ``` === Detected NVIDIA devices === 03:00.0 VGA compatible controller [10de:1c35]: NVIDIA GP106GL [Quadro P2200] 03:00.1 Audio device [10de:10f1]: NVIDIA GP106 High Definition Audio 04:00.0 VGA compatible controller [10de:1c35]: NVIDIA GP106GL [Quadro P2200] 04:00.1 Audio device [10de:10f1]: NVIDIA GP106 High Definition Audio GPU PCI IDs to bind to vfio-pci: 10de:1c35,10de:10f1 ``` Then asks: ``` Proceed with configuring VFIO passthrough? [y/N] ``` Type `y` and press Enter. The script runs through these steps — you will see each one printed: ``` --- Configuring kernel cmdline for IOMMU --- Written: /etc/kernel/cmdline Boot tool refreshed. --- Blacklisting nouveau and nvidia on host --- Written: /etc/modprobe.d/blacklist-gpu.conf --- Binding GPU IDs to vfio-pci --- Written: /etc/modprobe.d/vfio.conf IDs: 10de:1c35,10de:10f1 --- Adding vfio modules to initramfs --- Updated: /etc/initramfs-tools/modules Initramfs updated. ``` At the end it prints the PCI addresses you need: ``` === GPU PCI addresses for VM assignment === hostpci0: 0000:03:00,pcie=1 # VGA: NVIDIA GP106GL [Quadro P2200] hostpci0: 0000:04:00,pcie=1 # VGA: NVIDIA GP106GL [Quadro P2200] IMPORTANT: Pass each GPU + its HDMI audio sibling to the same VM. ``` **Write down or copy these addresses** — you need them in Phase 11 when creating the VMs. In this example: - GPU 1 is at `03:00` → goes in VM 100 (Frigate) - GPU 2 is at `04:00` → goes in VM 101 --- ### Step 9.3 — Update the VM configs with the GPU addresses While you have the addresses, add them to the VM configs now: ```bash nano /opt/local_proxmox/vm-configs/100-frigate.conf.example ``` Find the line: ``` hostpci0: 0000:XX:00,pcie=1,x-vga=1 ``` Replace `XX:00` with your GPU 1 address, e.g.: ``` hostpci0: 0000:03:00,pcie=1,x-vga=1 ``` Save, then do the same for VM 101: ```bash nano /opt/local_proxmox/vm-configs/101.conf.example # Change XX:00 to 04:00 (GPU 2) ``` --- ### Step 9.4 — Reboot ```bash reboot ``` Wait about 60 seconds, then SSH back in: ```bash ssh root@192.168.1.10 ``` --- ### Step 9.5 — Verify passthrough is working ```bash lspci -nnk | grep -A3 -i nvidia ``` For each GPU, look for the driver line. It **must** say `vfio-pci`: ``` 03:00.0 VGA compatible controller [10de:1c35]: NVIDIA GP106GL [Quadro P2200] Subsystem: ... Kernel driver in use: vfio-pci ← correct Kernel modules: nouveau 04:00.0 VGA compatible controller [10de:1c35]: NVIDIA GP106GL [Quadro P2200] Subsystem: ... Kernel driver in use: vfio-pci ← correct Kernel modules: nouveau ``` **If it shows `nouveau` or `nvidia` instead of `vfio-pci`:** The blacklist did not take effect. Run: ```bash update-initramfs -u -k all reboot ``` Then check again. If still wrong, verify the blacklist file exists: ```bash cat /etc/modprobe.d/blacklist-gpu.conf # Should show: blacklist nouveau, blacklist nvidia, etc. cat /etc/modprobe.d/vfio.conf # Should show: options vfio-pci ids=10de:1c35,10de:10f1 ``` Also confirm IOMMU is in the kernel command line: ```bash cat /etc/kernel/cmdline # Should contain: intel_iommu=on iommu=pt ``` --- ### Phase 9 complete — where you are now | What is done | Status | |---|---| | 3.5" drives in non-RAID mode, visible to Linux | ✓ | | Physical bay → by-id map documented | ✓ | | VM configs updated with real drive paths and GPU addresses | ✓ | | IOMMU active, both GPUs claimed by vfio-pci | ✓ | --- ## Phase 10 — Fan Control + Staggered Spin-Up **What this phase does:** Fixes two hardware quirks that bite R730xd owners running non-Dell gear: 1. **Fan control.** iDRAC detects the two Quadro P2200s as "non-Dell PCIe cards" and immediately pins every fan at 100% indefinitely to be safe. The server sounds like a jet engine and never stops. We disable iDRAC's automatic control and replace it with a daemon that sets fan speed based on the inlet temperature sensor. 2. **Staggered drive spin-up.** Twelve 3.5" HDDs spinning up simultaneously at boot spike about 240 W of inrush current for 2–3 seconds — enough to trip a 750 W PSU under load. We configure two layers of protection: the iDRAC BIOS setting that fires at POST (before the OS loads), and a Linux systemd service that handles any drives the BIOS setting missed. **Prerequisites from earlier phases:** - IPMI over LAN enabled in iDRAC (Phase 2) — required for `ipmitool` - `ipmitool` and `hdparm` installed on the host (Phase 6.3) - Repo cloned to `/opt/local_proxmox` (Phase 6.4) --- ### Step 10.1 — Test fan control manually before enabling the service Before installing anything as a systemd service, verify that the script can actually talk to iDRAC over IPMI. This catches misconfigured IPMI-over-LAN credentials before you wonder why the service silently failed. SSH into Proxmox: ```bash ssh root@192.168.1.10 ``` Check the current inlet temperature: ```bash bash /opt/local_proxmox/scripts/fan-control.sh --temp ``` Expected output: ``` Inlet temp: 24°C ``` The number will vary with your room temperature — anywhere between 18–35°C is typical for an idle server. > **If you see `Inlet temp: 35°C` exactly every time** — that is the script's > hard-coded fallback for when the sensor read fails. IPMI is not responding. > Check Phase 2: iDRAC **Network → IPMI Settings → Enable IPMI Over LAN** > must be checked, and the channel privilege level must be **Administrator**. Now test a manual fan speed override. This is safe — the script has a 15% minimum floor hard-coded, so you cannot accidentally stop the fans: ```bash bash /opt/local_proxmox/scripts/fan-control.sh --set 20 ``` Expected output: ``` : iDRAC automatic fan control DISABLED : Fans set to 20% ``` **Within 10–30 seconds you should hear the fans audibly slow down.** The R730xd is loud at 100% — the difference is immediate and obvious. Confirm the fans actually dropped: ```bash ipmitool sdr type Fan ``` Expected output: ``` Fan1A | 30h | ok | 7.1 | 2400 RPM Fan1B | 31h | ok | 7.1 | 2160 RPM Fan2A | 32h | ok | 7.1 | 2400 RPM Fan2B | 33h | ok | 7.1 | 2160 RPM Fan3A | 34h | ok | 7.1 | 2400 RPM Fan3B | 35h | ok | 7.1 | 2160 RPM ... ``` At 20% you should see roughly 2000–3000 RPM. At iDRAC's panic 100% you would see 12000–15000 RPM. If the numbers look like the former you are in good shape. Now hand control back to iDRAC before proceeding — we want the real service to be what re-disables automatic control, not this test: ```bash bash /opt/local_proxmox/scripts/fan-control.sh --auto ``` Expected output: ``` : iDRAC automatic fan control RE-ENABLED ``` **The fans will immediately ramp back to 100% within a few seconds** — that is iDRAC reacting to the Quadro again. This confirms the script was in fact holding them down. Get the service installed quickly so the noise stops. --- ### Step 10.2 — Install the fan control script and service Copy the script to the system directory and make it executable: ```bash cp /opt/local_proxmox/scripts/fan-control.sh /usr/local/sbin/fan-control.sh chmod +x /usr/local/sbin/fan-control.sh ``` Copy the systemd unit into place: ```bash cp /opt/local_proxmox/scripts/fan-control.service /etc/systemd/system/ ``` Tell systemd about the new unit, then enable and start it in one command: ```bash systemctl daemon-reload systemctl enable --now fan-control.service ``` `enable --now` both enables the unit for future boots and starts it right now, so the fans should begin quieting within 30 seconds. Verify the service is running: ```bash systemctl status fan-control.service ``` Expected output (abbreviated): ``` ● fan-control.service - Dell R730xd fan speed control (third-party GPU) Loaded: loaded (/etc/systemd/system/fan-control.service; enabled; preset: enabled) Active: active (running) since