From c8d9c2921ddf4eac4608ce0fe6127321877beff1 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 28 Apr 2026 02:48:37 +0000 Subject: [PATCH] Add handoff doc for "add-site" automation script Scoped specifically to a fresh session writing a script that automates the manual two-edit + two-reload workflow for adding a new site behind Authelia. Covers stack context, file targets, edit recipes per case (1/2a/2b/3/4), validation hooks, reload order, idempotency requirements, failure modes, and out-of-scope items. Includes concrete test fixtures the script should pass. https://claude.ai/code/session_013XZ1vmgk78k2PEQ5DmJhF3 --- HANDOFF-add-site-script.md | 456 +++++++++++++++++++++++++++++++++++++ 1 file changed, 456 insertions(+) create mode 100644 HANDOFF-add-site-script.md diff --git a/HANDOFF-add-site-script.md b/HANDOFF-add-site-script.md new file mode 100644 index 0000000..311ef90 --- /dev/null +++ b/HANDOFF-add-site-script.md @@ -0,0 +1,456 @@ +# HANDOFF -- script to add a site behind Authelia + +This doc is for a fresh session where you'll write a script that +automates the "add a new site behind Authelia" workflow currently done +by hand. Everything below is the context you need to write it +correctly. + +## Goal + +One command, e.g. + +``` +./add-site --subdomain doorbell --upstream 192.168.1.60:5555 --case 1 --policy two_factor +``` + +does all of: + +1. Adds the matching rule to `authelia/configuration.yml` under + `access_control.rules:`. +2. Adds (or modifies) the matching site block in the user's Caddyfile. + If a site block already exists with `basic_auth { ... }`, removes + that and inserts `import authelia` instead. +3. Validates Authelia config (`authelia validate-config`). +4. Validates Caddy config (`caddy validate`). +5. Restarts Authelia, then reloads Caddy. Order matters. +6. Optionally `curl`-tests the new URL and reports. + +Idempotent: re-running with the same args is a no-op. + +## Stack context (what's already running) + +- **Authelia 4.39.19**, file backend, SQLite local storage, in-memory + sessions, filesystem notifier. Compose project lives at + `~/docker/authelia/`. Container name `authelia`. +- **fail2ban** as a sidecar in the same compose project. Watches + `./authelia/authelia.log` and `/var/log/caddy/access.log`. +- **Caddy** is in its own compose project at `~/docker/caddy/` + (assumption -- script should accept the path as a parameter). + Container name `caddy`. On the external docker network `caddy_net`. +- Both Caddy and Authelia are on `caddy_net`. Caddy reaches Authelia + as `authelia:9091`. +- The portal is `auth.{DOMAIN}` with `policy: bypass`. +- `default_policy: deny` -- every gated domain MUST have a rule. +- DOMAIN substitution: + - Authelia uses Go templates: `'{{ env "DOMAIN" }}'`. Requires + `X_AUTHELIA_CONFIG_FILTERS=template` env var (already set in the + docker-compose.yml). + - Caddy uses `{env.DOMAIN}`. The Caddy compose passes `DOMAIN` + through to the Caddy container. + +## Files the script touches + +| File | What lives there | Who edits | +|------|------------------|-----------| +| `~/docker/authelia/authelia/configuration.yml` | `access_control.rules:` list | the script | +| `` (path: parameter) | site blocks | the script | +| `~/docker/authelia/.env` | `DOMAIN=...`, `TZ=...`, version pins | read-only (script reads DOMAIN from here OR the environment) | + +The script does NOT touch: +- `authelia/users_database.yml` (user management is separate) +- `authelia/secrets/*` (manual one-time bootstrap) +- `frigate_config/config.yml` or any other app's own config + (case 2a/2b require app-side edits the script can't safely automate + -- it should print instructions instead) +- DNS, TLS, anything outside the local Caddy + Authelia configs + +## Manual workflow the script automates + +For reference, here's what a human does today to add `foo.example.com` +as a case-1 site: + +```bash +# 1. Edit authelia/configuration.yml -- add under access_control.rules: +# - domain: 'foo.{{ env "DOMAIN" }}' +# policy: 'two_factor' + +# 2. Edit your real Caddyfile -- add a new block (or modify existing): +# foo.{env.DOMAIN} { +# import accesslog +# import authelia +# reverse_proxy 192.168.1.60:5555 +# } + +# 3. Validate before reloading +docker compose -f ~/docker/authelia/docker-compose.yml run --rm authelia \ + authelia validate-config --config /config/configuration.yml + +docker compose -f ~/docker/caddy/docker-compose.yml exec caddy \ + caddy validate --config /etc/caddy/Caddyfile + +# 4. Restart Authelia FIRST (so the rule is live before Caddy starts +# forwarding to it -- otherwise default_policy: deny returns 403) +docker compose -f ~/docker/authelia/docker-compose.yml restart authelia + +# 5. Reload Caddy (zero-downtime) +docker compose -f ~/docker/caddy/docker-compose.yml exec caddy \ + caddy reload --config /etc/caddy/Caddyfile + +# 6. Test +curl -sI -o /dev/null -w "%{http_code}\n" https://foo.example.com +# expect 302 redirect to auth.example.com +``` + +## Cases the script must handle + +The full case taxonomy is in `authelia/configuration.yml` and +`caddy/snippets.caddyfile`. Summary: + +### Case 1 -- no app auth, Authelia is the only gate + +**Authelia rule:** +```yaml +- domain: 'SUBDOMAIN.{{ env "DOMAIN" }}' + policy: 'two_factor' # or one_factor +``` + +**Caddy block:** +```caddyfile +SUBDOMAIN.{env.DOMAIN} { + import accesslog + import authelia + reverse_proxy UPSTREAM_IP:UPSTREAM_PORT +} +``` + +Most common case. The migration path from `basic_auth` -- the script +should handle "block exists with basic_auth, swap it for import authelia". + +### Case 2a -- app supports trusted-header proxy auth + +Same Caddy block as case 1. Same Authelia rule. **Plus** an app-side +config change the script CANNOT safely automate (each app is different: +Frigate's `auth.enabled: False`, Grafana's `[auth.proxy]` section, +Gitea's `ENABLE_REVERSE_PROXY_AUTHENTICATION`, etc.). Script should +print app-specific instructions from a lookup table and require +`--ack-app-config-done` to proceed. + +### Case 2b -- app supports OIDC + +Same Caddy block as case 1. Same Authelia rule. **Plus** an +`identity_providers.oidc.clients[]` entry to add to +`configuration.yml`. Each app needs its own `client_id`, +`client_secret`, `redirect_uris`, etc. + +This is more involved. **Suggested**: out of scope for v1 of the script; +print a pointer to Authelia's OIDC docs and skip. + +### Case 3 -- app keeps its own auth, Authelia adds 2FA in front + +Caddy block and Authelia rule are identical to case 1. The user just +keeps logging into the app after Authelia. The script doesn't need to +distinguish case 1 from case 3 mechanically -- the only difference is +the user's mental model. + +### Case 4 -- no Authelia involvement + +```caddyfile +SUBDOMAIN.{env.DOMAIN} { + import accesslog # NO import authelia + reverse_proxy UPSTREAM_IP:UPSTREAM_PORT +} +``` + +**No Authelia rule.** The script's job is just the Caddy edit + reload. +Useful for things like Plex/Emby where Authelia's redirect breaks +native clients. + +## Validation hooks the script must run + +In order, before any reload: + +1. **Authelia config:** + ```bash + docker compose -f ~/docker/authelia/docker-compose.yml run --rm authelia \ + authelia validate-config --config /config/configuration.yml + ``` + Exit 0 = good. Anything else = abort, restore the file from backup. + +2. **Caddy config:** + ```bash + docker compose -f ~/docker/caddy/docker-compose.yml exec caddy \ + caddy validate --config /etc/caddy/Caddyfile + ``` + Exit 0 = good. Anything else = abort, restore Caddyfile from backup. + +After reload: + +3. **HTTP probe:** + ```bash + curl -sI -o /dev/null -w "%{http_code}\n" https://SUBDOMAIN.DOMAIN + ``` + - Case 1 / 2a / 2b / 3: expect `302` (redirect to Authelia). + - Case 4: expect `200` or whatever the upstream returns. + - `403` means the Caddy block has `import authelia` but the Authelia + rule isn't in place (or wasn't picked up). Most common script bug. + +## Reload semantics + +**Order**: Authelia restart, THEN Caddy reload. Reverse order is briefly +broken: Caddy starts forwarding to Authelia for a domain Authelia +doesn't yet have a rule for, and `default_policy: deny` returns 403 to +the user. + +**Authelia restart**: full container restart. ~3-5 second outage on +auth.{DOMAIN}. Acceptable for household use; if you want zero-downtime +later, Authelia supports config reload via SIGHUP -- not used here. + +**Caddy reload**: `caddy reload` is genuinely zero-downtime. It loads +the new config, validates, and atomically swaps. If validation fails +the old config keeps running. + +**Rollback**: if anything fails, the script should revert the file +edits from a `.bak` it took before mutating. Easiest pattern: + +```bash +cp authelia/configuration.yml authelia/configuration.yml.bak +cp $CADDYFILE_PATH $CADDYFILE_PATH.bak +# ... edits ... +# if any validate fails: +mv authelia/configuration.yml.bak authelia/configuration.yml +mv $CADDYFILE_PATH.bak $CADDYFILE_PATH +``` + +## Idempotency + +The script must detect and short-circuit when the desired end state +already exists: + +- **Authelia rule already present**: parse `access_control.rules:`, look + for an entry with matching `domain:`. If found and policy matches, skip + the YAML edit. If found and policy differs, prompt or fail (don't + silently overwrite). +- **Caddy block already present with `import authelia`**: parse the + Caddyfile, look for `SUBDOMAIN.{env.DOMAIN} {` block. If found and + it has `import authelia`, skip the Caddy edit. +- **Caddy block exists with `basic_auth`**: this is the migration case. + Remove the `basic_auth { ... }` lines, add `import authelia` if + missing. Preserve everything else (transport, header_up, etc). +- **Caddy block exists without auth at all**: just add `import authelia`. + +After all checks: if neither file actually changed, skip both reloads +(important -- restarting Authelia for a no-op kicks every active +session). + +## Failure modes the script must handle + +- `DOMAIN` env var not set / `.env` not present -> abort early with + clear error. +- `caddy_net` docker network doesn't exist -> abort. +- Authelia container not running -> can still edit config and validate; + reload step needs to be skipped or attempted with helpful error. +- Caddy container not running -> same. +- Caddyfile path doesn't exist or isn't writable -> abort. +- The Caddyfile doesn't have `(authelia)` and `(accesslog)` snippets + defined -> the script could either inject them at the top of the file + (fragile) or refuse and tell the user to run a one-time setup step + first. **Recommended**: refuse, with a clear "run `./bootstrap-caddy` + first" message. +- Subdomain conflicts with an existing block that's NOT just a + `basic_auth` migration target (e.g. an entirely different upstream) + -> prompt, don't auto-overwrite. +- A site block exists with `basic_auth` AND something else complicated + (custom matchers, multiple `handle` blocks) -> migration is hard. + Recommended: detect the simple case (single `basic_auth { ... }` + inside the block) and refuse the complex case. +- `validate-config` or `caddy validate` fails -> rollback both files, + report the validator's stderr, exit non-zero. +- HTTP probe fails post-reload -> log the symptom but don't auto-revert; + user may have DNS not pointing yet, etc. + +## YAML editing -- preserve comments + +`authelia/configuration.yml` has substantial comments (the +"ALSO PASTE INTO CADDYFILE" blocks, case explanations). A naive YAML +round-trip will eat them. Use a comment-preserving library: + +- **Python**: `ruamel.yaml` with `YAML(typ='rt')` (round-trip mode). +- **Go**: `gopkg.in/yaml.v3` is comment-aware. +- **`yq`** (the Go-based one from mikefarah): preserves comments + reasonably well for simple ops. Adding a list item: + ```bash + yq -i '.access_control.rules += [{"domain": "foo.{{ env \"DOMAIN\" }}", "policy": "two_factor"}]' \ + authelia/configuration.yml + ``` + Note the escaping pain with `{{ env "DOMAIN" }}`. Test before + committing. + +The script should anchor inserts at a stable location. The least-bad +anchor is the END of `access_control.rules:` -- always append, never +splice in the middle. + +## Caddyfile editing -- there is no good parser + +Caddyfile has its own grammar; standard YAML/JSON tools won't touch it. +Options, in order of pragmatism: + +1. **Text-based pattern matching** (recommended for v1). Anchors: + - Find `^SUBDOMAIN\.\{env\.DOMAIN\} \{$` to detect existing block. + - For the basic_auth migration: use a small state machine to find + `basic_auth {` ... `}` inside the matched block and delete those + lines, then ensure `import authelia` and `import accesslog` lines + exist. + - For new-block insertion: append at end of file, separated by a + blank line. + +2. **`caddy adapt`**: converts Caddyfile to JSON. You could edit the + JSON, then... there's no Caddyfile emitter. Adapt is one-way. Skip. + +3. **`caddy fmt`**: normalizes whitespace in a Caddyfile, doesn't + semantically edit. Useful AFTER your edits to clean up. + +The text-based approach is fragile for arbitrary Caddyfiles but +predictable for the conventions this stack uses (one site block per +subdomain, snippets imported at top, no exotic matchers in gated +sites). + +## Suggested architecture + +``` +add-site +├── lib/ +│ ├── env.sh # find DOMAIN, container names, Caddyfile path +│ ├── yaml_edit.sh # ruamel.yaml or yq wrapper for access_control +│ ├── caddyfile_edit.sh # awk/sed-based site-block patcher +│ ├── validate.sh # authelia + caddy validators +│ └── reload.sh # restart authelia + reload caddy in order +├── cases/ +│ ├── case-1.sh # no-app-auth recipe +│ ├── case-2a.md # printable app-side instructions table +│ ├── case-2b.md # OIDC out-of-scope notice + pointer +│ └── case-4.sh # no-Authelia recipe +├── add-site # main entrypoint +└── README.md +``` + +Or in Python with `ruamel.yaml` and a small Caddyfile patcher class. +Either is fine; the bash version has fewer install steps for an +end-user. + +## Concrete test cases the script must pass + +Use these as fixtures. + +### Test 1: fresh case-1 add + +Pre-state: +- `authelia/configuration.yml` has only the `auth.{DOMAIN}` bypass rule. +- Caddyfile has `(authelia)` and `(accesslog)` snippets defined, no + `foo.{env.DOMAIN}` block. + +Invocation: +``` +./add-site --subdomain foo --upstream 192.168.1.60:5555 --case 1 +``` + +Post-state: +- `access_control.rules:` has new entry for `foo.{{ env "DOMAIN" }}`. +- Caddyfile has new `foo.{env.DOMAIN} { ... }` block with `import + authelia` and `import accesslog`. +- `validate-config` and `caddy validate` both pass. +- `curl -sI https://foo.example.com` returns 302. + +### Test 2: idempotent re-run + +Run Test 1's invocation twice. Second run: no file edits, no reloads, +exit 0 with "already configured" message. + +### Test 3: basic_auth migration + +Pre-state: +- Caddyfile has `foo.{env.DOMAIN} { basic_auth { user $2a$... } + reverse_proxy ... }`. + +Invocation: +``` +./add-site --subdomain foo --case 1 --migrate-basic-auth +``` + +Post-state: +- The `basic_auth { ... }` lines are gone. +- `import authelia` and `import accesslog` are present. +- `reverse_proxy` line is unchanged. +- Authelia rule added. + +### Test 4: validation failure rollback + +Pre-state: introduce a typo by hand into the YAML insert template +(e.g. `polciy:` instead of `policy:`). Simulate by mocking the +template. + +Expected: `validate-config` fails, both files restored from `.bak`, +non-zero exit, no reload attempted. + +### Test 5: case-4 (no Authelia) + +Invocation: +``` +./add-site --subdomain plex --upstream 192.168.1.5:32400 --case 4 +``` + +Post-state: +- Caddyfile has new `plex.{env.DOMAIN}` block with `import accesslog`, + NO `import authelia`. +- `access_control.rules:` is UNCHANGED. +- Caddy reloads, Authelia is NOT restarted. + +## Out of scope (tell the user, don't try to automate) + +- DNS A-record creation. Caddy will fail to issue a cert for a domain + that doesn't resolve. Print a "make sure DNS is pointing at this + host" reminder when the script starts. +- TLS / Let's Encrypt failures. Caddy auto-provisions; if it fails, + it's usually DNS or rate-limit. The script should not try to debug. +- App-side proxy-auth config (case 2a). Each app is different. Print + the lookup-table snippet from `caddy/snippets.caddyfile` for that + app and require `--ack-app-config-done` before running. +- OIDC client setup (case 2b). Big enough that it deserves its own + tool. Out of scope. +- User management (`users_database.yml`). +- Secret rotation (`authelia/secrets/*`). +- TOTP enrollment. + +## Quick reference: existing files + +If you want to read what's there to understand the conventions: + +- `docker-compose.yml` -- the auth stack compose, including the + `X_AUTHELIA_CONFIG_FILTERS=template` and `DOMAIN=${DOMAIN}` env vars. +- `authelia/configuration.yml` -- reference for the rule format, + comment style, and the "ALSO PASTE INTO CADDYFILE" blocks under + each case in `access_control.rules:`. +- `caddy/snippets.caddyfile` -- canonical examples of Caddy site + blocks for every case, including app-specific notes the case-2a + table can be extracted from. +- `.env.example` -- the shape of `.env` (DOMAIN, TZ, version pins). +- `README.md` -- the comprehensive bootstrap walkthrough; the + "Adding a new protected site" section is what the script + automates. + +## When in doubt + +- For YAML edits, dry-run with + `docker compose run --rm authelia authelia config template --config + /config/configuration.yml` -- prints the rendered config so you can + see exactly what Authelia will see. +- For Caddyfile edits, `caddy fmt --overwrite Caddyfile` normalizes + whitespace and `caddy validate` catches syntax errors. Run BOTH + before any reload. +- The single most useful debug command for "why is my site getting + 403": + ```bash + docker compose -f ~/docker/authelia/docker-compose.yml exec authelia \ + tail -f /config/authelia.log + ``` + Then hit the URL. The log line tells you exactly which rule (or + default_policy) made the call.