daf0ed11e42ab90591c3ccd8b3a44d119a01e87f
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Claude
|
bec9228c55 |
Back up existing files before every service overwrites them
Confirmed live: install_frigate()'s fresh-install path overwrote a working, hand-crafted docker-compose.yml (Frigate + mosquitto + frigate-notify) with zero backup, because that file's shape didn't match what frigate.sh's own "existing install" detection knew how to recognize. Every service's own detection is a judgment call about what counts as "already installed" and can miss a real setup built outside this repo's conventions. lib/common.sh gains backup_if_exists(FILE) — copies FILE to FILE.bak.<timestamp> if it exists, no-ops otherwise (including DRY_RUN). Applied before every service's own `cat > docker-compose.yml`/`cat > .env` write across all 60 services that do one (115 call sites), plus a matching standalone-mode stub added to every service's own bootstrap block, same convention already used for port_in_use/find_free_port. This doesn't replace a service's own update/fresh-reinstall detection — it's the safety net underneath it, so a wrong detection costs a .bak file to restore from instead of the original silently disappearing. Also fixes the actual gap that surfaced this: services/frigate.sh's Authelia offer only checked for Authelia installed locally on Frigate's own box, which is never true for a dedicated NVR box with no local Caddy either (the common shape — Caddy lives elsewhere, snippet-generation mode already handles that). Now offers Authelia protection unconditionally and, when Authelia isn't local, asks whether it lives on the same machine as Caddy (still "import authelia", since that's local to wherever Caddy ends up) or on a genuinely separate third machine (the explicit header-pinned forward_auth form, per CLAUDE.md's "forward_auth to a remote Authelia" note, needed because a bare authelia:9091 shortcut only works one hop). |
||
|
Claude
|
4717c3a080 |
Add garage-webui: browse Garage's buckets/objects like Backblaze's console
User's actual question: Backblaze B2's web console lets them browse a bucket as folders/files; Garage has no equivalent by default, so after switching an additional backup mirror from Backblaze-only to also target local Garage, they had no way to visually confirm data landed there the way they could on Backblaze. "S3 storage is opaque, you can't browse it" was true of Garage's *own* CLI, but wrong as a blanket statement — Backblaze's browsability comes from a client (its web console) layered on top of the same kind of object storage, and Garage has an actively-maintained equivalent (khairul169/garage-webui, 1.1k stars, "integrated objects/bucket browser") that gives the same experience against Garage's S3 API. services/garage-webui.sh (new): standard service-template Docker service. Requires an existing services/garage.sh install (checks for $DOCKER_DIR/garage/.env, errors with instructions if missing — this is a browser for an existing instance, not a replacement). Reaches Garage over host.docker.internal (both containers' ports are already published to the host — simpler and more robust than trying to join garage's own Compose-project-scoped default network by name). Has its own login (AUTH_USER_PASS, bcrypt via a throwaway `docker run --rm httpd:alpine htpasswd` — same $ -> $$ escaping services/wg-easy.sh already uses for its own bcrypt PASSWORD_HASH, verified here against a real docker compose config run: unescaped, Compose tries to interpolate $2y$05... as variable references and silently corrupts the value with a "not set" warning; escaped, it passes through intact with no warning), so it doesn't need Authelia gating by default. Prerequisite fix in services/garage.sh: its admin API (bucket/key management, object listing — the thing garage-webui talks to) has been running with zero authentication since this service was first built, because admin_token was never set in garage.toml. Nothing in this repo called that API before now, so it went unnoticed; adding a real consumer is what surfaced it. Fixed: generate admin_token (openssl rand -base64 32) alongside the existing rpc_secret, persist GARAGE_ADMIN_TOKEN/GARAGE_ADMIN_PORT to .env for garage-webui to read locally (never sent over SSH, unlike the S3 credentials backup.sh reads remotely). Update mode backfills admin_token into an existing garage.toml (+ restarts just the garage container to apply it) for anyone who installed before this change, same backfill-not-break approach as the GARAGE_S3_API_PORT fix from the previous commit. Verified: bash -n on both files; docker compose config against real Docker Compose for both the primary garage.toml/.env generation (with the new admin_token/GARAGE_ADMIN_PORT fields) and the new garage-webui docker-compose.yml; the bcrypt-escaping behavior specifically (proved via a minimal repro that unescaped $ corrupts the value with a warning, escaped does not); the admin_token/ GARAGE_ADMIN_PORT Update-mode backfill logic against old- and new-style .env/garage.toml fixtures, including idempotency (running it twice adds nothing a second time); and the credential-parsing regexes in garage-webui.sh against both a complete .env fixture and an old one missing the new fields (confirms the "run garage's Update first" error path actually triggers rather than proceeding with blanks). Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H4k6J1qXXyYxhGEgnJaMvn |
||
|
Claude
|
558fc3e75b |
Fix layout-apply crash on Garage reinstall, backfill missing .env fields
Live failure: `sudo ./setup.sh garage` → Full reinstall on a box that
already had a working Garage install crashed with:
Error: ApplyClusterLayout returned InternalError (500): Internal
error: Invalid new layout version
Root cause: a "Full reinstall" deliberately never wipes ./data or
./meta (that's real backup-mirror data — Kopia's sync-to s3 target —
and losing it silently on reinstall would be far worse than the
alternative), but the cluster-init step unconditionally re-ran `garage
layout assign` + `layout apply --version 1` every time it was reached.
Garage requires each apply to be exactly previous_version + 1; a node
that already has a committed layout (from the earlier install, still
sitting in the preserved ./meta) rejects a second "1". Fix: check
`garage status` for "NO ROLE ASSIGNED" first and only run the
assign/apply once, matching what the surrounding comment already
claimed happened ("Only ever run once") but the code didn't enforce.
Second, related issue this would have hit immediately after: the same
reused-./meta state almost always means an existing bucket + key from
the earlier install are still sitting in Garage's storage. The fresh
flow was about to silently create a brand-new bucket/key and overwrite
.env to point at those instead — orphaning any real data already in
the old bucket (nothing left on disk pointing at it, even though it's
still physically stored). Now: when the layout is already applied,
list existing buckets and require an explicit y/n (default n) before
creating new ones, with recovery instructions for reconnecting to an
existing bucket by hand instead.
Third, the actual reason a full reinstall was reached at all: Update
mode never backfills .env fields added to this script after someone's
initial install (GARAGE_S3_API_PORT, needed by services/backup.sh to
read an instance remotely) since Update deliberately never touches
.env otherwise — the only other path was the now-unsafe fresh
reinstall. Update now backfills just that missing key by reading the
real port back out of the already-written docker-compose.yml, so a
future .env schema addition doesn't force this tradeoff again.
Verified with standalone harnesses (not the live install, mocked
`garage status`/bucket-list output and .env/docker-compose.yml
fixtures): all four layout-state branches (fresh node, existing
buckets + decline, existing buckets + confirm, existing role but no
buckets), and both backfill cases (missing key added, existing key
left alone). Caught and fixed a real bug in the first draft of the
port-extraction regex during this testing — grep -oE '^[0-9]+' never
matched because the captured group still had its surrounding quotes.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H4k6J1qXXyYxhGEgnJaMvn
|
||
|
Claude
|
74cab14f86 |
Let the additional-mirror setup read Garage credentials over SSH
Extends the "ADDITIONAL MIRROR" section (previously SFTP-only) with a
type choice: SFTP, or S3 against a Garage instance already running on
that box. For the S3 path, this script never asks the operator to retype
a bucket name or key — it SSHes to the destination, reads
~/docker/garage/.env directly (the real, currently-configured values,
generated once by services/garage.sh and never touched again on its own
Update runs), and uses those for the dry-run verification and the
persisted mirror args. If Garage isn't installed there yet, it says so
plainly with the exact install command instead of failing cryptically or
silently skipping.
Also removed the last hardcoded suggestions from services/garage.sh
itself ("kopia-backup" / "kopia" as fixed prompt defaults) — replaced
with a freshly-generated suggestion each run (timestamp-suffixed), so
nothing about the bucket/key name is a fixed string baked into this repo
at any point in the chain; it's always the operator's actual choice, read
back live wherever it's needed.
Verified end-to-end against a mocked ssh (returning realistic
~/docker/garage/.env content) covering both outcomes: Garage installed
with a real bucket/key correctly parsed, dry-run run, and persisted; and
Garage missing, correctly warning with the install command and leaving
backup.conf untouched either way.
|
||
|
Claude
|
01ca0a7b09 |
Fix leading-whitespace bug in garage key create output parsing
Garage's real CLI output pads labels with extra spaces for column
alignment ("Key ID: GKxxxx"), not a single space like the
mocked test used ("Key ID: GKxxxx") — the fixed ": " field separator left
that padding stuck to the parsed value, so .env ended up with access
key/secret strings carrying leading whitespace inside the quotes.
Confirmed live by the user right after install. This would have broken S3
auth outright once actually used, since access keys have to match exactly.
Switched to ':[[:space:]]+' as a regex field separator, which consumes
however many spaces are actually there instead of assuming exactly one.
Verified against both the single-space and padded/aligned formats — both
now produce the identical clean value with no leading whitespace.
|
||
|
Claude
|
129a97f34b |
Add Garage — MinIO CE's replacement — as a self-hosted S3 object store
MinIO's open-source community edition is dead: console GUI stripped May 2025, Docker images stopped publishing October 2025, repo formally archived April 2026, with MinIO redirecting everyone to their paid AIStor product. Verified this directly before building anything, since recommending a since-abandoned image would have been worse than the SFTP problem this was meant to solve. Garage (Deuxfleurs) is the actively-maintained small-scale self-hosted replacement — single Rust binary, purpose-built for exactly this "one lightweight node" use case (as opposed to SeaweedFS, which targets large object counts / large-scale deployments, more machinery than a single backup-mirror target needs). services/garage.sh follows this repo's standard service template: port scanning for the S3 API/RPC/admin ports, an RPC secret generated once and never touched again on Update, and a one-time cluster init sequence (layout assign/apply, bucket create, key create, bucket allow) gated on whether .env already has a saved access key — Update reruns skip all of it and just refresh the image. Primary intended use: a local S3-compatible target for services/backup.sh's additional-mirror Kopia sync, so a local mirror can reuse the exact same sync-to s3 code path already proven reliable for the Backblaze B2 mirror, instead of Kopia's separate, less-exercised SFTP backend that's been the source of today's connection troubleshooting. Verified end-to-end against a mocked environment (fake docker exec returning realistic `garage status`/`garage key create` output) — caught and fixed a real off-by-one in the status-output parsing this way (grabbed the column-header row's literal "ID" instead of the actual node ID; output has a title line, then a header line, then the data row). Also validated the generated docker-compose.yml with real `docker compose config` in both the no-network and network-created cases. |