Rewrite services to use Caddy on LAN + NAT reflection for isolated VLANs

Previous approach tried to put a reverse proxy on OPNsense or poke
firewall holes — both wrong. Correct architecture:

- Caddy stays on LAN management computer (where it already is)
- WAN: port 443 forwarded to Caddy. Only port exposed externally.
- LAN devices reach services directly via Caddy
- Isolated VLANs (IoT, Guest) use public FQDNs (plex.mydomain.com)
- OPNsense NAT reflection handles this internally — traffic never
  leaves the network, but IoT is treated exactly like an external user
- Zero cross-VLAN access. No pinholes. Full isolation preserved.

IoT = untrusted = same access as someone on the internet. This is the
correct security model — no exceptions for "just one port."

Deploy endpoint now: writes Caddyfile entries, checks NAT reflection
status, provides setup checklist for port forward + reflection toggle.

https://claude.ai/code/session_01Do9bsN39MTuy2GVv7yzSrE

ers5952-manager.jsx

@@ -5082,27 +5082,31 @@ function ServicesTab({ vlans, session, onNeedAuth, backendOk }) {
     <div className="main">
       <div style={{flex:1}}>
         <div className="panel">
-          <div className="ph">Service Proxy — FQDN Access Without Breaking VLAN Isolation</div>
+          <div className="ph">Services — Caddy Reverse Proxy + NAT Reflection</div>
           <div className="pb" style={{fontSize:12,color:"var(--dm)",lineHeight:1.8}}>
             <div style={{marginBottom:8}}>
-              Make LAN services reachable by FQDN from any VLAN <b style={{color:"var(--tx)"}}>without
+              Caddy on the LAN management computer is your reverse proxy for all services.
-              any inter-VLAN access</b>. Devices never touch the service's VLAN directly.
+              Only port 443 is forwarded from WAN. Service ports are never exposed externally.
             </div>
             <div style={{
               padding:12,background:"var(--bg)",borderRadius:6,border:"1px solid var(--b2)",
               fontFamily:"monospace",fontSize:11,lineHeight:2,
             }}>
               <div style={{color:"var(--ac)",fontWeight:700,marginBottom:4,fontFamily:"inherit",fontSize:12}}>
-                How it works:
+                How isolated VLANs reach services:
               </div>
-              <div>1. IoT device (VLAN 30) asks DNS for <span style={{color:"var(--ac)"}}>plex.home.lan</span></div>
+              <div>1. IoT TV (VLAN 30) asks DNS for <span style={{color:"var(--ac)"}}>plex.mydomain.com</span></div>
-              <div>2. Unbound returns <span style={{color:"#00e676"}}>192.168.30.1</span> (OPNsense gateway — device can already reach this)</div>
+              <div>2. DNS returns your <span style={{color:"#00e676"}}>public IP</span></div>
-              <div>3. OPNsense reverse proxy (Caddy/HAProxy) forwards to actual server <span style={{color:"#ff6d00"}}>192.168.1.100:32400</span></div>
+              <div>3. OPNsense sees "that's my WAN IP" → <span style={{color:"#ff6d00"}}>NAT reflection</span> routes internally</div>
-              <div>4. Response returns the same path. <span style={{color:"#00e676"}}>IoT device never sees or touches LAN.</span></div>
+              <div>4. Port forward sends to Caddy → Caddy proxies to Plex</div>
+              <div>5. <span style={{color:"#00e676"}}>Traffic never leaves your network. Full VLAN isolation.</span></div>
             </div>
-            <div style={{marginTop:8,color:"var(--ac)",fontWeight:600}}>
+            <div style={{marginTop:8,color:"var(--tx)",fontWeight:600}}>
-              No firewall rules needed. No VLAN-to-VLAN access opened. All VLANs can already reach
+              IoT = untrusted = treated exactly like an external user. No pinholes, no cross-VLAN access.
-              their own gateway — that's how they get internet. The gateway does the proxying.
+            </div>
+            <div style={{marginTop:6,fontSize:11,color:"var(--dm)"}}>
+              Requires: OPNsense NAT reflection enabled (Firewall &gt; Settings &gt; Advanced &gt; Reflection for port forwards)
+              + WAN port forward TCP 443 → management computer (Caddy).
             </div>
           </div>
         </div>
@@ -5116,7 +5120,7 @@ function ServicesTab({ vlans, session, onNeedAuth, backendOk }) {
                 <input value={form.fqdn} onChange={e => setForm(f => ({...f, fqdn: e.target.value}))}
                   placeholder="plex.home.lan"/>
               </div>
-              <div className="field"><label>Backend URL (actual server on LAN)</label>
+              <div className="field"><label>Backend (LAN server IP:port)</label>
                 <input value={form.backend_url} onChange={e => setForm(f => ({...f, backend_url: e.target.value}))}
                   placeholder="http://192.168.1.100:32400"/>
               </div>
@@ -5163,7 +5167,7 @@ function ServicesTab({ vlans, session, onNeedAuth, backendOk }) {
                   {deploying ? "Deploying..." : "Deploy All Services"}
                 </button>
                 <span style={{fontSize:11,color:"var(--dm)"}}>
-                  Pushes DNS overrides to Unbound + configures reverse proxy on OPNsense
+                  Updates Caddyfile + checks NAT reflection on OPNsense
                 </span>
               </div>
 

switch_backend.py

@@ -4474,8 +4474,23 @@ def push_policy(body: dict):
 
 
 # ══════════════════════════════════════════════════════════════════════
-# SERVICE PROXY — expose LAN services to other VLANs via Caddy + DNS
+# SERVICE ACCESS — manage Caddy config + OPNsense port forwards + NAT reflection
 # ══════════════════════════════════════════════════════════════════════
+#
+# Architecture:
+#   Caddy runs on the LAN management computer. It is the reverse proxy for
+#   all services — only port 443 is forwarded from WAN, and Caddy routes
+#   by hostname (SNI) to the correct backend. Service ports (32400, 8123,
+#   etc.) are NEVER exposed on WAN.
+#
+#   For LAN devices: they reach services directly via Caddy on the LAN.
+#   For isolated VLANs (IoT, Guest, etc.): they use the public FQDN
+#   (e.g. plex.mydomain.com). OPNsense NAT reflection handles this
+#   internally — traffic never actually leaves the network. The isolated
+#   VLAN device is treated exactly like an external user.
+#
+#   This preserves full VLAN isolation. No pinholes, no cross-VLAN access.
+#   IoT = untrusted = same access as someone on the internet.
 
 SERVICES_FILE = _Path("/etc/switch-manager/service-proxies.json")
 CADDYFILE_EXTRA = _Path("/etc/switch-manager/Caddyfile.services")
@@ -4492,14 +4507,14 @@ def _save_services(services: list):
 
 
 def _generate_caddyfile_services(services: list) -> str:
-    """Generate Caddyfile blocks for service reverse proxies.
+    """Generate Caddyfile blocks for the LAN management computer's Caddy.
 
-    Caddy runs on OPNsense (or the management box). Each service FQDN
+    Each service FQDN gets a reverse_proxy block. Caddy handles TLS
-    gets a reverse_proxy block pointing to the actual backend server.
+    termination and routes by hostname. Only port 443 needs to be
-    Devices on isolated VLANs never touch the backend directly — they
+    forwarded from WAN to this machine.
-    hit their own gateway IP which Caddy proxies through.
     """
-    blocks = ["# Auto-generated by switch-manager — do not edit manually\n"]
+    blocks = ["# Auto-generated by switch-manager — service reverse proxy entries\n",
+              "# Add to your Caddyfile or use: import /etc/switch-manager/Caddyfile.services\n"]
     for svc in services:
         fqdn = svc.get("fqdn", "")
         backend_url = svc.get("backend_url", "")
@@ -4507,58 +4522,10 @@ def _generate_caddyfile_services(services: list) -> str:
             continue
         blocks.append(f"{fqdn} {{")
         blocks.append(f"    reverse_proxy {backend_url}")
-        blocks.append(f"    tls internal")
         blocks.append(f"}}\n")
     return "\n".join(blocks)
 
 
-def _generate_unbound_overrides(services: list, opnsense_ip: str) -> str:
-    """Generate Unbound local-data lines for service FQDN → OPNsense IP.
-
-    DNS resolves every service FQDN to the OPNsense router IP. Since
-    OPNsense is already the gateway for every VLAN, devices can reach
-    it without any new firewall rules. OPNsense runs the reverse proxy
-    (Caddy/HAProxy) which forwards to the actual backend server.
-
-    This means: VLAN isolation is fully preserved. An IoT device on
-    VLAN 30 hits plex.home.lan → DNS says 192.168.30.1 (its gateway)
-    → OPNsense proxies to the actual Plex server on LAN. The IoT
-    device never sees or reaches the LAN subnet.
-    """
-    lines = ["# Auto-generated by switch-manager — service proxy DNS\n",
-             "# Each FQDN resolves to OPNsense gateway IP.\n",
-             "# Devices reach services via their own gateway (reverse proxy),\n",
-             "# never touching other VLANs directly.\n"]
-    for svc in services:
-        fqdn = svc.get("fqdn", "")
-        # Use the OPNsense IP — it's the gateway for every VLAN
-        target_ip = svc.get("proxy_ip", opnsense_ip)
-        if fqdn:
-            lines.append(f'local-data: "{fqdn}. IN A {target_ip}"')
-    return "\n".join(lines)
-
-
-def _generate_haproxy_cfg(services: list) -> str:
-    """Generate OPNsense HAProxy backend/server entries for service proxies.
-
-    If OPNsense has the os-haproxy plugin, we can configure it via API.
-    This is a fallback config for manual import if the API isn't available.
-    """
-    lines = ["# HAProxy service proxy backends — import into OPNsense HAProxy plugin\n"]
-    for svc in services:
-        fqdn = svc.get("fqdn", "")
-        backend_url = svc.get("backend_url", "")
-        if not fqdn or not backend_url:
-            continue
-        # Parse backend URL
-        host_port = backend_url.replace("http://", "").replace("https://", "")
-        lines.append(f"# {svc.get('description', fqdn)}")
-        lines.append(f"# Frontend SNI match: {fqdn}")
-        lines.append(f"# Backend: {host_port}")
-        lines.append("")
-    return "\n".join(lines)
-
-
 @app.get("/api/services")
 def get_services():
     """List configured service proxies."""
@@ -4567,7 +4534,7 @@ def get_services():
 
 @app.post("/api/services")
 def save_service(body: dict):
-    """Add or update a service proxy."""
+    """Add or update a service proxy entry."""
     require_session(body.get("token", ""))
     svc = body.get("service", {})
     if not svc.get("fqdn") or not svc.get("backend_url"):
@@ -4582,7 +4549,7 @@ def save_service(body: dict):
 
 @app.delete("/api/services")
 def delete_service(body: dict):
-    """Remove a service proxy."""
+    """Remove a service proxy entry."""
     require_session(body.get("token", ""))
     fqdn = body.get("fqdn", "")
     services = _load_services()
@@ -4591,22 +4558,37 @@ def delete_service(body: dict):
     return {"success": True, "services": services}
 
 
+@app.get("/api/services/nat-reflection")
+def check_nat_reflection():
+    """Check if NAT reflection is enabled on OPNsense."""
+    cfg = _load_opnsense_cfg()
+    if not cfg.get("ssh_key_path"):
+        return {"configured": False, "error": "OPNsense SSH not configured"}
+    try:
+        out, _, code = _opnsense_ssh_run(
+            cfg, "grep -c 'enablenatreflectionhelper' /conf/config.xml 2>/dev/null || echo 0")
+        return {"configured": True, "likely_enabled": "1" in out.strip() or int(out.strip()) > 0}
+    except Exception as e:
+        return {"configured": True, "likely_enabled": None, "error": str(e)}
+
+
 @app.post("/api/services/deploy")
 def deploy_services(body: dict):
     """
-    Deploy service proxies via OPNsense — preserves full VLAN isolation.
+    Deploy service proxy configuration.
 
     Architecture:
-      1. DNS (Unbound on OPNsense) resolves service FQDNs to the OPNsense
+      - Caddy runs on the LAN management computer (reverse proxy for all services)
-         router IP. Since OPNsense is the gateway for every VLAN, devices
+      - WAN: port 443 forwarded to Caddy — only port exposed externally
-         can already reach it — no new firewall rules needed.
+      - LAN devices: reach services directly via Caddy
-      2. Reverse proxy (Caddy or HAProxy on OPNsense) accepts the request
+      - Isolated VLANs (IoT, Guest): use public FQDNs — OPNsense NAT
-         and proxies it to the actual backend server on whatever VLAN it
+        reflection routes internally without traffic leaving the network
-         lives on. OPNsense can route between VLANs — it's the router.
+      - Full VLAN isolation preserved — IoT treated same as external users
-      3. The requesting device (e.g. IoT on VLAN 30) never sees or touches
-         the backend's VLAN. It only talks to its own gateway.
 
-    No inter-VLAN firewall rules are created. VLAN isolation stays intact.
+    This endpoint:
+      1. Writes/updates Caddyfile with service entries
+      2. Checks NAT reflection status on OPNsense
+      3. Provides setup instructions for anything not yet configured
     """
     require_session(body.get("token", ""))
     services = _load_services()
@@ -4617,12 +4599,9 @@ def deploy_services(body: dict):
     errors = []
     pending_steps = []
 
-    cfg = _load_opnsense_cfg()
-    opnsense_ip = cfg.get("host", SWITCH_HOST.rsplit('.', 1)[0] + '.1')
-
     backup = _pre_change_backup(reason="pre-service-proxy deploy")
 
-    # 1. Write Caddyfile.services (local copy for reference / mgmt-box proxy)
+    # 1. Write Caddyfile.services for Caddy on the management computer
     caddy_content = _generate_caddyfile_services(services)
     try:
         CADDYFILE_EXTRA.write_text(caddy_content)
@@ -4630,88 +4609,38 @@ def deploy_services(body: dict):
     except Exception as e:
         errors.append(f"Caddyfile write: {e}")
 
-    # 2. Push DNS overrides to OPNsense Unbound
+    # 2. Check NAT reflection
-    # FQDNs resolve to OPNsense IP — devices already can reach their gateway
+    cfg = _load_opnsense_cfg()
+    nat_status = None
     if cfg.get("ssh_key_path"):
-        dns_content = _generate_unbound_overrides(services, opnsense_ip)
         try:
-            _opnsense_sftp_write(cfg, f"{UNBOUND_ETC}/service-proxies.conf", dns_content)
+            out, _, code = _opnsense_ssh_run(
-            steps_done.append(
+                cfg, "grep -c 'enablenatreflectionhelper' /conf/config.xml 2>/dev/null || echo 0")
-                f"Wrote Unbound overrides: {len(services)} service FQDNs → {opnsense_ip} (gateway)")
+            nat_enabled = "1" in out.strip() or (out.strip().isdigit() and int(out.strip()) > 0)
+            nat_status = nat_enabled
+            if nat_enabled:
+                steps_done.append("NAT reflection: enabled on OPNsense")
+            else:
+                pending_steps.append(
+                    "Enable NAT reflection: OPNsense > Firewall > Settings > Advanced > "
+                    "Reflection for port forwards = Enable")
         except Exception as e:
-            errors.append(f"Unbound DNS write: {e}")
+            errors.append(f"NAT reflection check: {e}")
 
-        # Validate and reload Unbound
+    # 3. Verify WAN port forward exists for 443
-        out, err, code = _opnsense_ssh_run(cfg, "unbound-checkconf 2>&1")
+    if cfg.get("key"):
-        if code != 0:
+        try:
-            errors.append(f"unbound-checkconf failed: {err or out}")
+            rules = _opnsense_request(cfg, "firewall/filter/searchRule")
-        else:
+            # This is a best-effort check — port forwards are in NAT, not filter
-            _opnsense_ssh_run(cfg, "unbound-control reload 2>&1")
+            pending_steps.append(
-            steps_done.append("Unbound reloaded with service DNS overrides")
+                "Verify WAN port forward: OPNsense > Firewall > NAT > Port Forward — "
-    else:
+                "WAN TCP 443 → management computer IP:443 (Caddy)")
-        errors.append("OPNsense SSH not configured — DNS overrides not deployed. "
+        except Exception:
-                      "Add service FQDNs to your DNS manually.")
+            pass
 
-    # 3. Deploy reverse proxy on OPNsense
+    pending_steps.append(
-    # Option A: Write Caddy config to OPNsense via SFTP and reload
+        "Reload Caddy on management computer: "
-    # Option B: Configure HAProxy plugin via OPNsense API
+        "docker compose restart caddy (or: caddy reload)")
-    # We try Caddy first (simpler), fall back to instructions
-    if cfg.get("ssh_key_path"):
-        # Check if Caddy is available on OPNsense
-        out, _, code = _opnsense_ssh_run(cfg, "which caddy 2>/dev/null")
-        if code == 0 and out.strip():
-            # Caddy is installed on OPNsense — write config and reload
-            try:
-                _opnsense_sftp_write(cfg, "/usr/local/etc/caddy/Caddyfile.services", caddy_content)
-                _opnsense_ssh_run(cfg, "caddy reload --config /usr/local/etc/caddy/Caddyfile 2>&1")
-                steps_done.append("Caddy on OPNsense: config written and reloaded")
-            except Exception as e:
-                errors.append(f"Caddy on OPNsense: {e}")
-        else:
-            # No Caddy on OPNsense — check HAProxy plugin
-            try:
-                _opnsense_request(cfg, "haproxy/settings/searchServers")
-                # HAProxy plugin is available — add backends
-                for svc in services:
-                    fqdn = svc.get("fqdn", "")
-                    backend_url = svc.get("backend_url", "")
-                    if not fqdn or not backend_url:
-                        continue
-                    host_port = backend_url.replace("http://", "").replace("https://", "")
-                    parts = host_port.split(":")
-                    backend_host = parts[0]
-                    backend_port = parts[1] if len(parts) > 1 else "80"
-                    try:
-                        # Add HAProxy backend server
-                        _opnsense_request(cfg, "haproxy/settings/addServer", "POST", {
-                            "server": {
-                                "name": fqdn.replace(".", "-"),
-                                "address": backend_host,
-                                "port": backend_port,
-                                "mode": "active",
-                                "ssl": "0",
-                            }
-                        })
-                        steps_done.append(f"HAProxy: backend {fqdn} → {host_port}")
-                    except ValueError as e:
-                        errors.append(f"HAProxy backend {fqdn}: {e}")
-                try:
-                    _opnsense_request(cfg, "haproxy/service/reconfigure", "POST")
-                    steps_done.append("HAProxy reconfigured")
-                except ValueError as e:
-                    errors.append(f"HAProxy reconfigure: {e}")
-            except Exception:
-                # Neither Caddy nor HAProxy available
-                pending_steps += [
-                    "Install Caddy or HAProxy plugin on OPNsense to enable reverse proxying.",
-                    "OPNsense: System > Firmware > Plugins > os-haproxy (recommended)",
-                    "Or: pkg install caddy (FreeBSD package)",
-                    "DNS overrides are deployed — once a reverse proxy is running on OPNsense, "
-                    "services will be reachable by FQDN from all VLANs without breaking isolation.",
-                ]
-
-    # No firewall rules needed — devices already can reach their gateway
-    steps_done.append("No firewall changes needed — devices reach services via their own gateway")
 
     return {
         "success": len(errors) == 0,
@@ -4720,11 +4649,12 @@ def deploy_services(body: dict):
         "errors": errors,
         "backup": backup,
         "caddy_content": caddy_content,
-        "opnsense_ip": opnsense_ip,
+        "nat_reflection_enabled": nat_status,
         "architecture": (
-            "DNS resolves service FQDNs to OPNsense gateway IP. "
+            "Caddy on LAN management computer handles all reverse proxying. "
-            "Devices reach services through their own gateway (reverse proxy). "
+            "Only port 443 forwarded from WAN. Isolated VLANs use public FQDNs — "
-            "No inter-VLAN firewall rules created. Full VLAN isolation preserved."
+            "OPNsense NAT reflection routes internally (no round trip to internet). "
+            "Full VLAN isolation preserved. IoT = untrusted = external user."
         ),
     }