local_proxmox/docs/walkthrough-phases-10-12.md

Phases 10–12: Fan Control, VM Creation, Frigate Deploy

Dell R730xd — Button-by-button walkthrough

---

Phase 10 — Fan Control + Stagger Spin-Up Services

What this phase does: Fixes two hardware quirks that bite R730xd owners running non-Dell gear:

1. Fan control. iDRAC detects the two Quadro P2200s as "non-Dell PCIe cards" and immediately pins every fan at 100% indefinitely to be safe. The server sounds like a jet engine and never stops. We disable iDRAC's automatic control and replace it with a daemon that sets fan speed based on the inlet temperature sensor.
2. Staggered drive spin-up. Twelve 3.5" HDDs spinning up simultaneously at boot spike about 240 W of inrush current for 2–3 seconds — enough to trip a 750 W PSU under load. We configure two layers of protection: the iDRAC BIOS setting that fires at POST (before the OS loads), and a Linux systemd service that handles any drives the BIOS setting missed.

Prerequisites from earlier phases:

• IPMI over LAN enabled in iDRAC (Phase 2) — required for ipmitool
• ipmitool and hdparm installed on the host (Phase 6.3)
• Repo cloned to /opt/local_proxmox (Phase 6.4)

---

Step 10.1 — Test fan control manually before enabling the service

Before installing anything as a systemd service, verify that the script can actually talk to iDRAC over IPMI. This catches misconfigured IPMI-over-LAN credentials before you wonder why the service silently failed.

SSH into Proxmox:

ssh root@192.168.1.10

Check the current inlet temperature:

bash /opt/local_proxmox/scripts/fan-control.sh --temp

Expected output:

Inlet temp: 24°C

The number will vary with your room temperature — anywhere between 18–35°C is typical for an idle server.

If you see Inlet temp: 35°C exactly every time — that is the script's hard-coded fallback for when the sensor read fails. IPMI is not responding. Check Phase 2: iDRAC Network → IPMI Settings → Enable IPMI Over LAN must be checked, and the channel privilege level must be Administrator.

Now test a manual fan speed override. This is safe — the script has a 15% minimum floor hard-coded, so you cannot accidentally stop the fans:

bash /opt/local_proxmox/scripts/fan-control.sh --set 20

Expected output:

<date>: iDRAC automatic fan control DISABLED
<date>: Fans set to 20%

Within 10–30 seconds you should hear the fans audibly slow down. The R730xd is loud at 100% — the difference is immediate and obvious.

Confirm the fans actually dropped:

ipmitool sdr type Fan

Expected output:

Fan1A            | 30h | ok  |  7.1 | 2400 RPM
Fan1B            | 31h | ok  |  7.1 | 2160 RPM
Fan2A            | 32h | ok  |  7.1 | 2400 RPM
Fan2B            | 33h | ok  |  7.1 | 2160 RPM
Fan3A            | 34h | ok  |  7.1 | 2400 RPM
Fan3B            | 35h | ok  |  7.1 | 2160 RPM
...

At 20% you should see roughly 2000–3000 RPM. At iDRAC's panic 100% you would see 12000–15000 RPM. If the numbers look like the former you are in good shape.

Now hand control back to iDRAC before proceeding — we want the real service to be what re-disables automatic control, not this test:

bash /opt/local_proxmox/scripts/fan-control.sh --auto

Expected output:

<date>: iDRAC automatic fan control RE-ENABLED

The fans will immediately ramp back to 100% within a few seconds — that is iDRAC reacting to the Quadro again. This confirms the script was in fact holding them down. Get the service installed quickly so the noise stops.

---

Step 10.2 — Install the fan control script and service

Copy the script to the system directory and make it executable:

cp /opt/local_proxmox/scripts/fan-control.sh /usr/local/sbin/fan-control.sh
chmod +x /usr/local/sbin/fan-control.sh

Copy the systemd unit into place:

cp /opt/local_proxmox/scripts/fan-control.service /etc/systemd/system/

Tell systemd about the new unit, then enable and start it in one command:

systemctl daemon-reload
systemctl enable --now fan-control.service

enable --now both enables the unit for future boots and starts it right now, so the fans should begin quieting within 30 seconds.

Verify the service is running:

systemctl status fan-control.service

Expected output (abbreviated):

● fan-control.service - Dell R730xd fan speed control (third-party GPU)
     Loaded: loaded (/etc/systemd/system/fan-control.service; enabled; preset: enabled)
     Active: active (running) since <time>
   Main PID: 12345 (fan-control.sh)
      Tasks: 2 (limit: 154321)
     Memory: 1.5M
     CGroup: /system.slice/fan-control.service
             ├─12345 /bin/bash /usr/local/sbin/fan-control.sh
             └─12350 sleep 30

<date>: Fan control daemon starting
<date>: iDRAC automatic fan control DISABLED
<date>: Inlet 24°C → fans 15%

The key lines to verify:

• Active: active (running) — the daemon is up
• iDRAC automatic fan control DISABLED — iDRAC is no longer in charge
• Inlet NN°C → fans NN% — it read the sensor and set a speed from the map

Press q to exit the status view.

---

Step 10.3 — Verify fans actually slow down over time

Wait 60 seconds, then check fan RPMs again:

ipmitool sdr type Fan

At an idle inlet of 22–30°C, the speed map in fan-control.sh sets fans to 15–20%. You should see roughly 2000–3000 RPM across all fans. The server should be noticeably quiet — closer to a desktop PC than a server.

If the server is still loud (fans 10000+ RPM):

journalctl -u fan-control.service -n 50

Look for errors from ipmitool. The most common failure is Unable to establish IPMI v2 / RMCP+ session — that means IPMI over LAN is not enabled in iDRAC. Revisit Phase 2's iDRAC IPMI settings.

Tuning the temperature curve: The defaults in fan-control.sh are conservative — 15% below 30°C, scaling up to 100% at 60°C inlet. If the server runs hot (40°C+ inlet is common in a closed rack) and the fans feel too aggressive, edit the SPEED_MAP array in /usr/local/sbin/fan-control.sh, then systemctl restart fan-control.service. Never lower MIN_SPEED=15 — below that the drives and CPUs have no airflow safety margin.

---

Step 10.4 — Configure iDRAC staggered spin-up (BIOS layer)

The BIOS layer fires at POST, before the OS even starts. It tells the H730 to spin up drives one at a time during the power-on self-test. This is the strongest protection because it runs when PSU inrush risk is highest.

Run the script's iDRAC helper:

bash /opt/local_proxmox/scripts/stagger-spinup.sh --idrac

If racadm is installed on the Proxmox host (it sometimes ships in the Dell OMSA bundle), the script will run it directly:

Enabling iDRAC hard disk drive sequencing (staggered spin-up)...
[Key=BIOS.Setup.1-1#StorageSettings]
Object value modified successfully
RAC973: Successfully scheduled a job.
Job queued. Reboot for the BIOS setting to take effect.

If racadm is not installed — which is the normal case on a plain Proxmox host — the script prints two manual options:

racadm not found.
Option A: Run from iDRAC SSH:
  ssh root@<idrac-ip>
  racadm set BIOS.StorageSettings.HddSeq Enabled
  racadm jobqueue create BIOS.Setup.1-1
  # Then reboot to apply.

Option B: iDRAC web UI:
  System BIOS → Power Management → Hard Disk Drive Sequencing → Enabled
  Apply and reboot.

Use Option A (iDRAC SSH) — it is the reliable one. The web UI path is buried deep and the wording varies between firmware versions.

SSH directly to iDRAC using the static IP you set in Phase 2:

ssh root@192.168.1.11

(Enter the iDRAC root password you set in Phase 2.)

You will land in iDRAC's own shell — not the Proxmox shell. The prompt looks like:

/admin1-> 

Run the two racadm commands:

racadm set BIOS.StorageSettings.HddSeq Enabled
racadm jobqueue create BIOS.Setup.1-1

Expected output for each:

/admin1-> racadm set BIOS.StorageSettings.HddSeq Enabled
[Key=BIOS.Setup.1-1#StorageSettings]
RAC1017: Successfully modified the object value and the change is in
         pending state.

/admin1-> racadm jobqueue create BIOS.Setup.1-1
RAC1024: Successfully scheduled a job.
Verify the job status using "racadm jobqueue view -i JID_xxxxxxxxxxxx"
Commit JID = JID_123456789012
Reboot Required = Yes

Type exit to leave the iDRAC shell. You are now back on the Proxmox host.

The BIOS setting is queued but not yet applied — the server must POST once for it to take effect. The next reboot (Step 10.5 will trigger one) will apply it automatically.

Why not reboot right now? Because we are about to install the Linux stagger service too, and one reboot covers both changes. Keep going.

---

Step 10.5 — Install the Linux stagger spin-up service

The Linux layer is a safety net: if a drive did not spin up during BIOS POST (for example, it was in standby from a previous run), this service wakes it up during early boot, one at a time, before the storage stack comes online.

Copy the script into place:

cp /opt/local_proxmox/scripts/stagger-spinup.sh /usr/local/sbin/stagger-spinup.sh
chmod +x /usr/local/sbin/stagger-spinup.sh

Copy the systemd unit:

cp /opt/local_proxmox/scripts/stagger-spinup.service /etc/systemd/system/

Enable the service — do not start it manually. The unit is wired into sysinit.target and is meant to run only during the early boot window. Running it ad-hoc after boot is harmless (drives are already spun up, it will log "already active — skipping" for each) but pointless:

systemctl daemon-reload
systemctl enable stagger-spinup.service

Expected output:

Created symlink /etc/systemd/system/sysinit.target.wants/stagger-spinup.service
  → /etc/systemd/system/stagger-spinup.service.

Verify the unit is enabled (but inactive, as expected):

systemctl status stagger-spinup.service

Expected output:

● stagger-spinup.service - Stagger hard drive spin-up to limit PSU current surge
     Loaded: loaded (/etc/systemd/system/stagger-spinup.service; enabled; preset: enabled)
     Active: inactive (dead)

inactive (dead) is correct — it is a Type=oneshot unit that ran once at the last boot (when it did not yet exist) and will run on the next boot.

---

Step 10.6 — Reboot to apply both the BIOS setting and the Linux service

reboot

Pay attention during this boot. You are testing two things at once:

1. Listen to the drives. Instead of the normal "all 12 drives click awake at once" chorus, you should hear them spin up in sequence — a quick series of individual clicks spread across 30–40 seconds during POST. This is the iDRAC BIOS setting doing its job.
2. Watch the fans. Right after POST they will briefly hit 100% again (iDRAC starts in automatic mode every boot). Within 30–60 seconds of Proxmox being up, the fan control service should take over and quiet them down.

Wait about 90 seconds after the reboot, then SSH back in:

ssh root@192.168.1.10

---

Step 10.7 — Verify everything came back correctly

Fan service running and fans quiet:

systemctl is-active fan-control.service
# Expected: active

ipmitool sdr type Fan | head -3
# Expected: fans at ~2000–3000 RPM, not 12000+

Stagger service ran successfully at boot:

systemctl status stagger-spinup.service

Expected output:

● stagger-spinup.service - Stagger hard drive spin-up to limit PSU current surge
     Loaded: loaded (...; enabled; preset: enabled)
     Active: inactive (dead) since <boot time> — oneshot finished
    Process: 456 ExecStart=/usr/local/sbin/stagger-spinup.sh --linux (code=exited, status=0/SUCCESS)
   Main PID: 456 (code=exited, status=0/SUCCESS)

status=0/SUCCESS is the line that matters — the script ran and exited cleanly.

Look at what it actually did:

journalctl -u stagger-spinup.service -b

Expected output (one line per drive):

Staggering spin-up for 12 drives (3s apart)...
 /dev/sda already active (state: active/idle) — skipping
 Waking /dev/sdb (was: standby)
 Waking /dev/sdc (was: standby)
 ...
 Waking /dev/sdm (was: standby)
Stagger complete.

Typical: the OS RAID1 mirror (/dev/sda) is already active from boot, and each of the 12 data drives gets woken in sequence.

Confirm the iDRAC BIOS setting stuck (only works if you installed racadm on the host, otherwise skip — the boot behavior above is the real test):

# Only if racadm is installed on Proxmox:
racadm get BIOS.StorageSettings.HddSeq
# Expected: HddSeq=Enabled

Otherwise SSH into iDRAC and run the same command there.

---

Phase 10 complete — where you are now

What is done	Status
Fan control daemon active, fans quiet	✓
iDRAC BIOS hard disk drive sequencing enabled	✓
Linux stagger spin-up service enabled for future boots	✓
Verified both survive a reboot cleanly	✓

Next: Phase 11 — Create the three VMs in the Proxmox web UI, attach GPUs / Coral USB / raw data drives, and install Ubuntu inside each.

---

Phase 11 — Create the VMs

What this phase does: Creates three VMs in the Proxmox web interface, attaches their hardware (GPUs, Coral USB, raw data drives), installs Ubuntu Server inside each one, and verifies SSH access.

Prerequisites from earlier phases:

• GPU passthrough working: both GPUs show vfio-pci in lspci (Phase 9)
• Bay map filled in: /dev/disk/by-id/... paths known for all 12 drives (Phase 8)
• VM config examples updated with real paths and PCI addresses (Phases 8–9)
• Fan control and stagger services installed and verified (Phase 10)

---

Step 11.1 — Download the Ubuntu installer ISO

The Frigate VM needs Ubuntu Server 24.04 LTS. On your workstation, go to ubuntu.com/download/server and download ubuntu-24.04.1-live-server-amd64.iso (or the latest 24.04 point release).

---

Step 11.2 — Upload the ISO to Proxmox

In the Proxmox web UI (https://192.168.1.10:8006):

1. In the left panel, click your node name (e.g. pve) → local storage
2. Click the ISO Images tab
3. Click Upload
4. Click Select File → choose the ubuntu-24.04.*-live-server-amd64.iso
5. Click Upload

The upload progress bar shows while transferring. When it completes the ISO appears in the list.

Faster alternative — download directly on the Proxmox host:

cd /var/lib/vz/template/iso/
curl -LO "https://releases.ubuntu.com/24.04/ubuntu-24.04.1-live-server-amd64.iso"

The ISO appears in the web UI list immediately after the download finishes.

---

Step 11.3 — Create VM 100 (Frigate) via the wizard

In the Proxmox web UI, click Create VM (top-right button).

Tab: General

Field	Value
Node	pve
VM ID	100
Name	frigate

Click Next.

Tab: OS

Field	Value
ISO image	ubuntu-24.04.1-live-server-amd64.iso
OS Type	Linux
Version	6.x - 2.6 Kernel

Click Next.

Tab: System

Field	Value
Machine	q35
BIOS	OVMF (UEFI)
Add EFI Disk	checked
EFI Storage	local-lvm
Pre-Enrolled Keys	unchecked
SCSI Controller	VirtIO SCSI Single

Why q35 + OVMF? GPU passthrough requires PCIe bus emulation, which only q35 provides. i440fx (the older default) does not support PCIe correctly for VFIO passthrough.

Pre-Enrolled Keys must be unchecked. Enabling it turns on Secure Boot, which blocks NVIDIA kernel modules from loading in the guest.

Click Next.

Tab: Disks

Field	Value
Bus/Device	SCSI / 0
Storage	local-lvm
Disk size (GiB)	64
Cache	Write back
Discard	checked (if local-lvm is on SSD)

Click Next.

Tab: CPU

Field	Value
Sockets	1
Cores	8
Type	host

Why host CPU type? It exposes the real CPU feature flags, which NVIDIA drivers expect. Without it, nvidia-smi may work but performance-sensitive GPU features fail silently.

Click Next.

Tab: Memory

Field	Value
Memory (MiB)	16384

Click Next.

Tab: Network

Field	Value
Bridge	vmbr0
Model	VirtIO (paravirt)
Firewall	checked

Click Next, then Finish.

VM 100 now appears in the left panel. Do not start it yet — the GPU, Coral USB, and data drives must be added first.

---

Step 11.4 — Add GPU passthrough to VM 100

In the left panel, click 100 (frigate) → Hardware tab.

Click Add → PCI Device.

Field	Value
Raw Device	select GPU #1: e.g. 0000:03:00.0 NVIDIA GP106GL [Quadro P2200]
All Functions	checked
Primary GPU	checked
PCI-Express	checked

All Functions passes both 03:00.0 (VGA) and 03:00.1 (HDMI audio) as a unit. Without it the audio sibling stays on the host.

Primary GPU (x-vga=1 in the config) tells the VM this is its display adapter. Required for console output in the guest before GPU drivers load.

Click Add.

---

Step 11.5 — Add Coral USB to VM 100

Still in VM 100's Hardware tab.

Click Add → USB Device.

First entry (pre-init ID):

Field	Value
Use USB Vendor/Device ID	selected
Vendor ID	1a6e
Device ID	089a

Click Add.

Click Add → USB Device again.

Second entry (post-init ID):

Field	Value
Use USB Vendor/Device ID	selected
Vendor ID	18d1
Device ID	9302

Click Add.

Why two entries? The Coral USB presents as 1a6e:089a before Frigate loads firmware onto it, then re-enumerates as 18d1:9302 after firmware loads. Both must be passed through or the second enumeration will not be accessible inside the VM.

The Coral must be physically plugged in to the R730xd before starting the VM. If it is not connected, the VM boots fine but the USB devices will not appear.

---

Step 11.6 — Add raw data drives to VM 100 via CLI

The web UI does not support raw block device passthrough cleanly — use the Proxmox CLI. SSH into the host:

ssh root@192.168.1.10

Run one qm set command per drive, replacing the PLACEHOLDER_BAYx paths with the real by-id paths from your hardware-layout.md:

qm set 100 -scsi1 /dev/disk/by-id/PLACEHOLDER_BAY1
qm set 100 -scsi2 /dev/disk/by-id/PLACEHOLDER_BAY2
qm set 100 -scsi3 /dev/disk/by-id/PLACEHOLDER_BAY3
qm set 100 -scsi4 /dev/disk/by-id/PLACEHOLDER_BAY4
qm set 100 -scsi5 /dev/disk/by-id/PLACEHOLDER_BAY5
qm set 100 -scsi6 /dev/disk/by-id/PLACEHOLDER_BAY6
qm set 100 -scsi7 /dev/disk/by-id/PLACEHOLDER_BAY7
qm set 100 -scsi8 /dev/disk/by-id/PLACEHOLDER_BAY8

Each command is silent on success. Confirm all drives are attached:

grep scsi /etc/pve/qemu-server/100.conf

Expected output:

scsi0: local-lvm:vm-100-disk-0,cache=writeback,size=64G
scsi1: /dev/disk/by-id/scsi-35000cca23b7d4eb8,size=0
scsi2: /dev/disk/by-id/scsi-35000cca23b5e1234,size=0
...
scsi8: /dev/disk/by-id/scsi-35000cca23bab1234,size=0

size=0 is correct for raw disk passthrough — Proxmox defers sizing to the device itself. A non-zero size here indicates the drive was added wrong.

---

Step 11.7 — Create VM 101 (general purpose with GPU)

Click Create VM in the web UI.

Use the same wizard settings as VM 100 except:

Tab	Value
General	VM ID: 101, Name: vm101
CPU	8 cores, host
Memory	16384 MiB
Disk	64 GiB on local-lvm

After the wizard completes, add the second GPU in the Hardware tab:

Click Add → PCI Device → select GPU #2 (e.g. 0000:04:00.0 NVIDIA GP106GL [Quadro P2200]). Check All Functions, Primary GPU, PCI-Express. Click Add.

No Coral USB for VM 101.

Add the two data drives via CLI:

qm set 101 -scsi1 /dev/disk/by-id/PLACEHOLDER_BAY9
qm set 101 -scsi2 /dev/disk/by-id/PLACEHOLDER_BAY10

---

Step 11.8 — Create VM 102 (storage/utility, no GPU)

Click Create VM again.

Tab	Value
General	VM ID: 102, Name: vm102
CPU	4 cores, host
Memory	8192 MiB
Disk	32 GiB on local-lvm

No PCI device, no USB devices.

Add the two data drives:

qm set 102 -scsi1 /dev/disk/by-id/PLACEHOLDER_BAY11
qm set 102 -scsi2 /dev/disk/by-id/PLACEHOLDER_BAY12

---

Step 11.9 — Install Ubuntu in VM 100

In the left panel, click 100 (frigate) → click Start (▶ button).

Open the console: click Console at the top. This opens a noVNC session in your browser.

Wait for the Ubuntu GRUB menu to appear. Select Try or Install Ubuntu Server and press Enter.

Work through the Ubuntu Server installer:

1. Language: English (or your preference)
2. Keyboard: your layout
3. Type of install: Ubuntu Server (not minimised — minimised is missing tools you need)
4. Network connections: the VirtIO NIC auto-detects and gets a DHCP IP. Note the IP address — you will use it for SSH in the next step.
5. Proxy: leave blank
6. Ubuntu archive mirror: leave at default (or a nearby mirror)
7. Storage:
   • Guided storage layout → Use an entire disk
   • Select the 64GB virtual disk (the scsi0 OS disk — shows as ~64.4 GB)
   • Leave LVM enabled
   • Do not select the large 3.6TB data drives — those will become a ZFS pool in Phase 12
8. Profile setup:
   • Server's name: frigate
   • Username: ubuntu (or your preference)
   • Password: something strong
9. SSH: check Install OpenSSH server → Yes
10. Featured server snaps: skip all

Click Done on the summary screen. The install takes 5–10 minutes.

When installation finishes: Reboot Now. Proxmox ejects the ISO automatically. When the login prompt appears in the console, the OS is ready.

---

Step 11.10 — Find VM 100's IP and test SSH

Log in at the console. Check the assigned IP:

ip addr show ens18

Expected output:

2: ens18: <BROADCAST,MULTICAST,UP,LOWER_UP> ...
    inet 192.168.1.XXX/24 brd 192.168.1.255 scope global ens18

From your workstation, verify SSH works:

ssh ubuntu@192.168.1.XXX

If SSH connects, close the noVNC console tab — you will not need it again.

Recommended: Set a DHCP reservation in your router for VM 100's MAC address so its IP stays stable. Frigate's web UI and stream URLs depend on a predictable IP.

---

Step 11.11 — Install Ubuntu in VMs 101 and 102

Repeat steps 11.9–11.10 for VMs 101 and 102. The installer flow is identical. Use vm101 and vm102 as hostnames.

During the storage step in each VM: install Ubuntu only on the small OS disk. Leave the large data drives untouched — they are for you to configure later.

---

Phase 11 complete — where you are now

What is done	Status
Ubuntu Server 24.04 ISO uploaded to Proxmox	✓
VM 100 created: GPU #1, Coral USB (×2), 8 raw data drives	✓
VM 101 created: GPU #2, 2 raw data drives	✓
VM 102 created: 2 raw data drives, no GPU	✓
Ubuntu installed and SSH accessible in all three VMs	✓

Next: Phase 12 — install NVIDIA drivers, Docker, and Frigate inside VM 100.

---

Phase 12 — Deploy Frigate in VM 100

What this phase does: Installs the NVIDIA driver and Docker inside VM 100, creates a ZFS storage pool from the 8 data drives, then deploys Frigate NVR with GPU-accelerated video decode and Coral object detection.

Prerequisites:

• VM 100 running Ubuntu Server 24.04 with SSH access (Phase 11)
• NVIDIA Quadro P2200 passed through (PCIe device visible in guest)
• Google Coral USB passed through (USB device visible in guest)
• 8 raw data drives visible as block devices inside the VM
• Frigate config files from this repo (frigate/docker-compose.yml and frigate/config.yml)

All commands in this phase run inside VM 100 — not on the Proxmox host.

---

Step 12.1 — SSH into VM 100 and switch to root

ssh ubuntu@192.168.1.XXX   # VM 100's IP from Phase 11
sudo -i

---

Step 12.2 — Verify the GPU is visible

lspci | grep -i nvidia

Expected:

06:10.0 VGA compatible controller: NVIDIA Corporation GP106GL [Quadro P2200] (rev a1)
06:10.1 Audio device: NVIDIA Corporation GP106 High Definition Audio Controller (rev a1)

The PCI address inside the VM (e.g. 06:10.0) will differ from the host address (03:00.0). QEMU re-numbers the virtual PCIe bus. This is normal.

If neither device appears, check that hostpci0 is set correctly in /etc/pve/qemu-server/100.conf on the Proxmox host, and that Phase 9 completed successfully (GPU showing vfio-pci on the host).

---

Step 12.3 — Install the NVIDIA driver

apt update
apt install -y nvidia-driver-535

This installs the driver and all required kernel modules. Takes 2–4 minutes.

If the installer asks about Secure Boot MOK (Machine Owner Key), press OK to dismiss — Secure Boot is off in this VM (pre-enrolled-keys was unchecked in Phase 11), so the prompt has no effect.

Reboot the VM to load the driver:

reboot

SSH back in after ~30 seconds:

ssh ubuntu@192.168.1.XXX
sudo -i

---

Step 12.4 — Verify nvidia-smi

nvidia-smi

Expected output:

+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 535.xxx.xx                 Driver Version: 535.xxx.xx    CUDA Version: 12.x |
|-----------------------------------------+------------------------+----------------------|
| GPU  Name                Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
|=========================================+========================+======================|
|   0  Quadro P2200            Off       | 00000000:06:10.0   Off |                  N/A |
| 41%   42C    P8              10W /  75W |      0MiB /  5120MiB  |      0%      Default |
+-----------------------------------------------------------------------------------------+

Key things to verify:

• Driver Version: 535.xxx.xx — driver loaded
• Quadro P2200 — correct GPU model
• 5120 MiB — full 5 GB memory visible (not a partial amount)

If nvidia-smi returns No devices were found:

dmesg | grep -i nvidia | tail -20

Look for Secure Boot signature errors. If present, confirm pre-enrolled-keys=0 in the VM's efidisk0 line in Proxmox.

---

Step 12.5 — Install Docker

curl -fsSL https://get.docker.com | sh

This runs Docker's official install script and installs Docker Engine with the Compose plugin. Takes about 1 minute.

Verify:

docker version

Both Client and Server sections should show a version number.

Enable Docker to start at boot:

systemctl enable docker

---

Step 12.6 — Install NVIDIA Container Toolkit

This allows Docker containers to access the GPU. Without it Frigate cannot use the Quadro for hardware video decode.

curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey \
  | gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg

curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list \
  | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' \
  | tee /etc/apt/sources.list.d/nvidia-container-toolkit.list

apt update
apt install -y nvidia-container-toolkit

nvidia-ctk runtime configure --runtime=docker
systemctl restart docker

Expected output from nvidia-ctk:

INFO    Configured /etc/docker/daemon.json

---

Step 12.7 — Test GPU access in Docker

docker run --rm --runtime=nvidia --gpus all ubuntu nvidia-smi

Expected: the same nvidia-smi table from Step 12.4, now running inside a container. If you see Error: no runtime "nvidia", run systemctl restart docker and try again.

---

Step 12.8 — Verify Coral USB

lsusb

Expected — one of these IDs should be present:

Bus 001 Device 002: ID 1a6e:089a Global Unichip Corp.

or (if previously initialized):

Bus 001 Device 002: ID 18d1:9302 Google Inc.

If neither appears: verify both USB passthrough entries (usb0 and usb1) are in the VM's Hardware tab, and that the Coral is physically plugged in to the R730xd.

---

Step 12.9 — Create the ZFS storage pool

Check which block devices are the data drives (the large ones — not the 64 GB OS disk):

lsblk -d -o NAME,SIZE,TYPE | grep disk

Expected:

NAME   SIZE TYPE
sda     64G disk   ← OS disk — do NOT include in the pool
sdb    3.6T disk   ← data drive
sdc    3.6T disk
sdd    3.6T disk
sde    3.6T disk
sdf    3.6T disk
sdg    3.6T disk
sdh    3.6T disk
sdi    3.6T disk   ← data drive 8

Install ZFS:

apt install -y zfsutils-linux

Create a RAIDZ2 pool from all 8 data drives. RAIDZ2 tolerates up to 2 simultaneous drive failures:

zpool create -f frigate-data raidz2 \
  /dev/sdb /dev/sdc /dev/sdd /dev/sde \
  /dev/sdf /dev/sdg /dev/sdh /dev/sdi

Use /dev/sdX paths here, not by-id paths. The drives have no firmware by-id identity inside the VM — ZFS generates its own stable identifiers from drive serial numbers.

Set the mount point:

zfs set mountpoint=/mnt/frigate frigate-data

Verify:

zpool status frigate-data

Expected:

  pool: frigate-data
 state: ONLINE
config:

    NAME         STATE     READ WRITE CKSUM
    frigate-data ONLINE       0     0     0
      raidz2-0   ONLINE       0     0     0
        sdb      ONLINE       0     0     0
        ...
        sdi      ONLINE       0     0     0

Confirm usable capacity:

df -h /mnt/frigate

With 8 × 4TB drives in RAIDZ2, expect roughly 22–24 TB usable.

---

Step 12.10 — Get the Frigate config files

Clone the repo into the VM:

git clone https://github.com/outis1one/local_proxmox.git /opt/local_proxmox

Copy the Frigate files to the working directory:

mkdir -p /opt/frigate
cp /opt/local_proxmox/frigate/docker-compose.yml /opt/frigate/
cp /opt/local_proxmox/frigate/config.yml /opt/frigate/

---

Step 12.11 — Edit docker-compose.yml

nano /opt/frigate/docker-compose.yml

Change the RTSP password placeholder:

environment:
  FRIGATE_RTSP_PASSWORD: "changeme"   # ← set a real password here

The {FRIGATE_RTSP_PASSWORD} placeholder in config.yml will be substituted with this value at runtime. You will reference it in camera RTSP URLs.

No other changes are needed unless:

• You have 8+ cameras → increase shm_size from 256mb to 512mb
• Your recordings path is different from /mnt/frigate

Save: Ctrl+O, Enter, Ctrl+X.

---

Step 12.12 — Edit config.yml and add your cameras

nano /opt/frigate/config.yml

Find the cameras: section at the bottom and fill in your camera details:

cameras:
  front_door:                          # ← rename to anything (no spaces)
    ffmpeg:
      inputs:
        - path: rtsp://admin:{FRIGATE_RTSP_PASSWORD}@192.168.1.XXX/stream1
          #                                           ^^^^^^^^^^^^^^^^^ your camera IP + RTSP path
          roles:
            - detect
            - record

Common RTSP path formats by brand:

Brand	RTSP URL format
Hikvision	rtsp://user:{password}@IP/Streaming/Channels/101
Dahua	rtsp://user:{password}@IP/cam/realmonitor?channel=1&subtype=0
Reolink	rtsp://user:{password}@IP/h264Preview_01_main
Amcrest	rtsp://user:{password}@IP/cam/realmonitor?channel=1&subtype=0

For H.265 cameras, uncomment the per-camera hwaccel override:

    ffmpeg:
      hwaccel_args: preset-nvidia-h265

To add more cameras, duplicate the entire front_door: block (including indentation) and change the name and IP.

Save: Ctrl+O, Enter, Ctrl+X.

---

Step 12.13 — Start Frigate

cd /opt/frigate
docker compose up -d

On the first run, Docker pulls the Frigate image (~1 GB). Progress bars show the download.

Watch the startup logs:

docker compose logs -f

A successful start looks like:

frigate  | [INFO]    Starting Frigate...
frigate  | [INFO]    Connected to EdgeTPU device: usb
frigate  | [INFO]    Loading NVIDIA GPU: Quadro P2200
frigate  | [INFO]    Starting camera: front_door
frigate  | [INFO]    Frigate is running

Key lines:

• Connected to EdgeTPU device: usb — Coral detected, inference running
• Loading NVIDIA GPU — NVDEC hardware decode active
• Starting camera: [name] — stream connected

Press Ctrl+C to stop following logs. Frigate continues running in the background.

Common first-start errors:

Failed to connect to EdgeTPU — run lsusb in the VM to confirm 1a6e:089a or 18d1:9302 is present. Both USB passthrough entries must be in the VM's Hardware tab.

Failed to connect to [camera name] — RTSP URL is wrong. Check the URL format for your camera model. The Frigate UI shows the stream as Offline.

no runtime "nvidia" — re-run nvidia-ctk runtime configure --runtime=docker && systemctl restart docker.

---

Step 12.14 — Access the Frigate web UI

In a browser on your workstation:

http://192.168.1.XXX:5000

The Frigate dashboard shows live camera feeds, detection events, and system stats. Navigate to System → Stats to confirm:

• GPU utilization is shown (Quadro P2200)
• Coral inference is running
• CPU usage is low (NVDEC handling decode, Coral handling detection)

---

Phase 12 complete — full system summary

What is done	Status
NVIDIA driver 535 installed, verified with nvidia-smi	✓
Docker and NVIDIA Container Toolkit configured	✓
GPU accessible inside Docker containers	✓
ZFS RAIDZ2 pool created from 8 data drives	✓
Frigate running with NVDEC decode and Coral inference	✓
Cameras configured in config.yml	✓
Web UI accessible at http://vm100-ip:5000	✓

---

Full system status:

Layer	Component	Running
Hardware	Dell R730xd, PERC H730, 12 drives, 2× Quadro P2200, Coral USB	✓
Proxmox host	IOMMU active, GPUs held by vfio-pci	✓
Services	fan-control (quiet fans), stagger-spinup (PSU protection)	✓
VM 100	Ubuntu 24.04, GPU passthrough, 8 drives, Frigate NVR	✓
VM 101	Ubuntu 24.04, GPU passthrough, 2 drives	ready
VM 102	Ubuntu 24.04, 2 drives	ready