Confirmed live and cross-checked against a real, documented Kopia issue
(kopia/kopia#5329): the walkthrough previously told the operator to scope
the Application Key to just the bucket they created — the more
security-conservative default, and correct for B2's own S3-compatible
API in general. But Kopia specifically needs the listBuckets capability
even though it only ever touches the one configured bucket, and B2's
basic "Add a New Application Key" web form doesn't expose a way to grant
listBuckets on a bucket-restricted key — only an account-wide ("All")
key gets it through that form. Without it, the connection fails with
B2's unhelpful "Cannot access bucket" error, which doesn't point at the
actual missing capability at all.
Updated the guidance to "All" with the reasoning inline, and a note that
single-bucket scoping is still possible for anyone willing to create the
key via B2's CLI/API directly (b2_create_key with an explicit
capabilities list including listBuckets) rather than the basic web form
this walkthrough is written for.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H4k6J1qXXyYxhGEgnJaMvn
Two separate fixes from a live report.
1. The DR-spare and SFTP-mirror sections both checked ONLY /root/.ssh
for a key, missing the common case: the person running `sudo
./setup.sh backup` already has a key under their own home directory
(used interactively, quite possibly already authorized on the target
box), while root — who actually runs the scheduled systemd service —
has none. Confirmed live: "the computer has the ssh key for the sudo
user on the box" produced "No SSH key found for root" with no
inline way to do anything about it beyond a pointer to go set one up
elsewhere and re-run.
Factored both call sites into one shared _backup_ensure_root_ssh_key()
that checks root first, then offers to reuse the sudo user's existing
keypair (copied into /root/.ssh with correct ownership/permissions,
root:root 600) before falling back to generating a brand new one —
reusing an existing key can work immediately if it's already
authorized on the target, where a fresh key needs a new ssh-copy-id
round-trip regardless. Verified all three branches (root already has
a key, root has none but the user does and accepts reuse, neither
exists and one gets generated) against a mocked filesystem.
2. The B2 dry-run failure message read like it could be about missing
input even when every field was non-empty — confirmed there's no
code path where non-blank-but-wrong values actually trigger the
separate "Left blank" message (the two are on disjoint branches), but
the dry-run failure text itself didn't rule that out or point at the
actual likely cause. Now echoes back what was entered (bucket,
endpoint, Key ID — never the secret) so it's easy to eyeball against
B2's own confirmation screen, states plainly that this is a rejection
of non-blank input, and names the most likely cause directly: pairing
the Key ID from one Application Key with the Secret from a different
one, which is easy to do after creating more than one while
troubleshooting.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H4k6J1qXXyYxhGEgnJaMvn
Confirmed: Kopia's sync-to sftp has its own SFTP client and doesn't read
~/.ssh/config the way the system ssh/scp binaries do — so an alias set
up via wg-easy's sync-ssh-aliases.sh (or any ~/.ssh/config Host entry)
worked fine for the DR-spare connectivity check (which shells out to
real ssh) but silently failed for this mirror: a plain @-split on an
alias like "main" (no @ present) produced --host=main, a name that only
resolves inside ~/.ssh/config, not real DNS. The dry-run check correctly
rejected it and the mirror was never saved — no error surfaced beyond
that, so it looked like nothing happened.
Now resolves the destination through `ssh -G` before building the Kopia
flags — the same mechanism ssh itself uses to expand config aliases —
and falls back to the previous plain @-split only if that comes back
empty. Verified against three cases: a bare alias (resolves via a mock
~/.ssh/config Host block), an explicit user@ip (passes through
unchanged), and an unrecognized name (falls back to a sane literal
hostname rather than erroring).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H4k6J1qXXyYxhGEgnJaMvn
Requested: mirror to Backblaze B2 AND directly to the IONOS spare box
over Tailscale, at the same time, not one or the other. REMOTE_TYPE/
REMOTE_ARGS was hardcoded to a single mirror target — extending it to a
list would have meant redesigning the one thing that already works and
was already verified against real B2 credentials, so this adds a
separate, additive mechanism instead: EXTRA_MIRROR_NAMES, a space-
separated list, with per-entry MIRROR_<name>_TYPE/_ARGS (same argument
shape as REMOTE_ARGS). An existing B2-only backup.conf keeps working
completely unchanged if this new section is skipped.
install_backup() gets a new "ADDITIONAL MIRROR" prompt after the
existing B2 section: offers a direct SFTP mirror (Kopia's sync-to sftp,
not the deprecated b2 provider — same reasoning as the S3/B2 choice
already made), defaults the destination to whatever was typed at the
DR-spare prompt above (same box, same purpose, no reason to ask twice),
checks passwordless SSH and an SSH key exist first, then verifies with a
--dry-run against the just-created 'default' repo before saving it —
same "don't save something broken" discipline as the B2 flow. Verified
against a mock backup.conf that install-side writes and worker-side
reads agree on the exact format, and that reusing an existing mirror
name reconfigures it instead of duplicating it in the name list.
One correction while researching sync-to sftp's flags: unlike plain ssh,
Kopia doesn't shell out to the system SSH client, so it needs an
explicit --keyfile and --known-hosts path rather than picking up
whatever `ssh` already trusts automatically — checked Kopia's own docs
for the exact flags before writing this, same as the earlier S3 case.
extras/backup_kopia.sh's worker loops through EXTRA_MIRROR_NAMES after
the existing REMOTE_TYPE mirror step, running sync-to for each
destination against each additional mirror and folding failures into
the same FAILED_SVCS/notification reporting the primary mirror already
uses. Verified end-to-end against a mock backup.conf and a stubbed
kp_for: both the B2 and the new SFTP mirror get called in sequence with
the correct arguments.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H4k6J1qXXyYxhGEgnJaMvn
Requested after a live failure: pasting into the hidden Application Key
field silently captured nothing (terminal/SSH-client dependent), and the
only symptom was a generic "one or more fields left blank" warning after
all four prompts had already gone by — no way to tell which field, or
even that the paste itself was the problem rather than something else.
Each of the four fields now echoes its character count right after entry
(never the value for the hidden Application Key field, just its length),
so a failed paste is visible immediately instead of discovered several
prompts later. The blank-field warning now also names exactly which
field(s) were empty instead of a generic message.
Verified against the user's actual reported case: bucket/endpoint/key-ID
entered normally, Application Key came back empty — reproduces as
"(0 characters entered)" on that line and "Left blank: Application Key"
in the warning, both confirmed against a second case where all four
fields are present and it passes through cleanly.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H4k6J1qXXyYxhGEgnJaMvn
Requested: don't push the operator toward installing wg-easy if they
already have a different mesh VPN (Netbird or Tailscale) running —
detect any of the three first, and only offer a choice when none are
present.
Detection checks wg-easy's own directory (this repo's install marker),
then falls back to checking whether the netbird/tailscale binaries exist
AND their systemd services are actually active — not just installed,
since an installed-but-never-connected client isn't a usable path to the
spare box either. wg-easy takes priority if somehow more than one is
present, since it's this repo's own chain-installable option.
When none are detected, offers a numbered choice: wg-easy (chain-installs
via the existing declare -F guard), Netbird, or Tailscale (both via their
official curl-pipe-sh installers — verified the current URLs against
each vendor's own docs rather than guessing, since a wrong URL here would
be a bad thing to ship). Both third-party options still need a manual
follow-up step this script can't complete unattended (Netbird needs a
setup key from the operator's account, Tailscale needs an interactive
auth link) — the success message says so rather than implying the
install alone finishes the job.
Verified the detection branching against all the cases that matter:
nothing present, only wg-easy's directory, only Netbird active, only
Tailscale active, and multiple present at once (wg-easy correctly wins).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H4k6J1qXXyYxhGEgnJaMvn
Requested improvement: the disaster-recovery spare prompt in backup.sh
already ran a live connectivity check and, on failure, printed manual
instructions (set up wg-easy separately if the spare isn't reachable,
run ssh-keygen/ssh-copy-id yourself) — but never offered to do any of it
right there, even though every piece is safe to automate inline.
Now, when the passwordless SSH check fails:
- If wg-easy isn't installed yet, offers to chain-install it (guarded
with declare -F install_wg-easy, same pattern asterisk.sh already uses
for security-dashboard/pstn-trunk) — covers the common case where the
spare is a home box with no port-forward and no path there at all yet,
not just a missing key.
- If root has no SSH key, offers to generate one (ssh-keygen -t ed25519).
- Offers to run ssh-copy-id against the spare interactively right there
— it prompts for the spare's login password itself, so this script
never touches or sees that password, just invokes the real command
inline instead of telling the operator to go run it themselves after.
- Re-runs the connectivity check after ssh-copy-id succeeds, so the
install flow reports the actual current state instead of the
pre-fix failure message.
Verified the has-a-key detection (the part most likely to have a subtle
&&/|| precedence bug) against all four cases — no key, only id_ed25519,
only id_rsa, both — behaves correctly in each.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H4k6J1qXXyYxhGEgnJaMvn
Confirmed live: backup.sh has no update/fresh distinction and re-runs
every prompt on every invocation, including the repository password
prompt — which always minted a fresh (typed or auto-generated) password
regardless of whether a repo already existed at that destination's path.
Re-running the installer (to add a destination, configure the new B2
offsite mirror, or just by habit) then fails to connect to the real,
already-populated repo with "invalid repository password", because the
repo's actual password is permanently whatever was set the first time
and nothing read that back.
Each destination's password is now read back from the existing
backup.conf (if that destination name was already configured there)
before falling through to prompt/auto-generate — same pattern already
applied to REMOTE_TYPE/REMOTE_ARGS, EMBEDDED_COTURN_SLOT, and everywhere
else in this session that re-running a script with no update/fresh gate
turned out to silently regenerate something it shouldn't have. Verified
against a mock backup.conf: an existing destination's password is reused
verbatim, and a genuinely new destination name still falls through to
fresh generation correctly.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H4k6J1qXXyYxhGEgnJaMvn
Answers a direct ask: offsite mirroring existed only as a REMOTE_TYPE/
REMOTE_ARGS placeholder in backup.conf with a comment pointing at
`kopia repository sync-to --help` — no interactive setup at all, B2 or
otherwise.
Checked before building anything: Kopia's dedicated `sync-to b2`
provider is marked [DEPRECATED] on kopia.io's own command reference.
B2 also offers an S3-compatible endpoint (s3.<region>.backblazeb2.com,
same application key works as the access/secret key pair), and Kopia's
`sync-to s3` provider isn't deprecated — so this targets that path
instead of building on a command on its way out.
What's now automated vs. guided, deliberately split:
- Bucket creation and the application key are walked through as console
steps, not automated. Object Lock specifically is a one-time,
bucket-creation-only decision with a real tradeoff (undeletable-by-
design vs. genuinely can't delete early) that shouldn't be silently
flipped either way by a script on someone's behalf.
- Once the operator has a bucket + endpoint + scoped application key
(B2 requires a key scoped to one bucket, not the account master key —
noted in the walkthrough), this becomes mechanical: run a
`sync-to s3 --dry-run` against the just-created 'default' repo to
verify the credentials actually work, and only then write
REMOTE_TYPE=s3 / REMOTE_ARGS into backup.conf. A bad bucket name or
key leaves REMOTE_TYPE at "none" with a clear error instead of saving
a broken config that fails silently at 2am.
- Encryption isn't a separate step — Kopia already encrypts client-side
with the repository password set earlier in this same flow; called
that out explicitly since it was asked about as if it needed its own
setup step.
Also fixed a regression the new prompt would otherwise have caused:
backup.sh has no update/fresh distinction and re-asks everything on
every run, so an already-configured offsite mirror is now read back
from the existing backup.conf and preserved by default — answering "no"
on a re-run no longer silently resets REMOTE_TYPE to "none".
Verified the control flow (not just bash -n) against a mock kopia
binary and stubbed prompts: good credentials wire up REMOTE_TYPE/
REMOTE_ARGS correctly, a rejected credential leaves REMOTE_TYPE at
"none" rather than saving something broken, an existing configured
value survives a "no" answer on re-run, and blank fields skip cleanly
without attempting a dry-run at all.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H4k6J1qXXyYxhGEgnJaMvn
Answers a direct ask: the automated restore-verify test
(extras/test_backup_kopia.sh — verifies the latest snapshot, restores it
over a moved-aside copy, compares, rolls back, reports PASS/FAIL, sends
an ntfy notification) was already fully non-interactive and already
wired to a systemd timer/cron fallback by install_backup() — it just
had no schedule choice at all, hardcoded to weekly (Saturday 03:00).
Every service in this test stops briefly while its data gets moved
aside and restored back, same interruption profile as the main backup
job — so the schedule is a real tradeoff (more frequent verification vs.
more frequent blips), not a free "always pick the most frequent" choice.
Gave it the same Weekly/Monthly/Custom shape the main backup schedule
prompt above it already offers, instead of a single hardcoded option.
Also added an explicit "run the first test now?" prompt right after
scheduling it — otherwise choosing Monthly means waiting up to a month
before finding out whether the test even works, rather than getting
that initial confirmation immediately and then settling into the
chosen cadence.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H4k6J1qXXyYxhGEgnJaMvn
- dr_bringup.sh: bound every kopia call and docker compose up with a
timeout so one stuck service can't stall the rest of the batch, and
only exit non-zero if literally nothing came up — a partial recovery
is a partial success, not a failed run.
- backup_kopia.sh: optional DR_SYNC_HOST/DR_SYNC_PATH in backup.conf
scp's backup.conf + README.md to a spare box over SSH after every
successful backup, so dr_bringup.sh is ready there with no manual
copy step.
- backup.sh: prompts for the spare's SSH destination, verifies
connectivity at install time instead of failing silently at 2am, and
writes ~/docker/backup/README.md (this service never had one) so the
synced copy documents every command listed above.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NQkdAn3iG5A4WoqU9FHMaN
restore_kopia.sh is interactive and one-service-at-a-time, which doesn't
scale to standing up a cold spare box quickly during a real outage.
dr_bringup.sh restores every service's latest snapshot (or one named
service) and runs docker compose up -d with no prompts, so a full-stack
recovery is one command instead of N interactive restores.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NQkdAn3iG5A4WoqU9FHMaN
New site config key: CADDY_REMOTE_HOST (set via 'sudo ./setup.sh configure').
When set, configure_caddy_for_service operates in "remote" mode instead of
writing to a local Caddyfile:
- Upstream uses CADDY_REMOTE_HOST:PORT (host IP, not container name)
- Snippet saved to ~/docker/caddy-snippets/<subdomain>.caddy
- User is shown scp/rsync commands to copy it to the Caddy machine
Three modes in configure_caddy_for_service (lib/common.sh and inline stubs):
local: ~/docker/caddy/ exists → write Caddyfile + reload (existing behavior)
remote: CADDY_REMOTE_HOST set → save snippet, print copy instructions
none: neither configured → silent return (unchanged)
All 31 service standalone bootstrap stubs updated with the new logic.
CADDY_REMOTE_HOST global added to all 42 standalone bootstrap sections.
setup.sh configure now prompts for CADDY_REMOTE_HOST with a clear explanation.
wolf.sh: add missing stubs (configure_caddy_for_service, write_readme,
prompt_yn, ensure_docker_dir_ownership) and the Authelia/Caddy/start calls
that were missing from the install function.
https://claude.ai/code/session_014CCYqVwW6d6f5dw1qRokYt
Each service can now be run directly with sudo bash <service>.sh on any
machine with Docker installed, without needing the full post-install repo.
Uses the shared bootstrap pattern from docs/standalone-template.sh.
https://claude.ai/code/session_014CCYqVwW6d6f5dw1qRokYt
- extras/test_backup_kopia.sh: stop → restore → compare → roll back test for Kopia
- extras/test_backup_borg.sh: stop → extract → compare → roll back test for Borg
- backup workers: timing (duration), service count, and pre-flight disk check (< 512 MB warns)
- backup workers: ntfy notifications include count, duration, and per-failure detail
- services/backup.sh: install test_backup_kopia.sh + optional weekly test timer
- services/borg-backup.sh: install test_backup_borg.sh + optional weekly test timer
extras/test_backup.sh — new unified test script (Kopia + Borg):
• Stops container, moves live data aside, restores latest backup,
compares restored vs live with diff -rq (content, not timestamps),
moves live data back and restarts container
• PASS = restore succeeded; diff output is informational (files changed
since last backup are normal)
• FAIL = restore command failed or target empty after restore
• --list flag, CLI service arg, interactive picker
• Handles both full-service dirs and sub-path sources (gaming-backup)
• Cleanup trap always restores live data even on error
• Sends ntfy notification on pass and fail
extras/backup_kopia.sh, backup_borg.sh, backup_gaming.sh:
• ntfy_send() + categorize_error() helpers added
• Each snapshot/archive failure captures stderr and categorizes:
disk full, remote unreachable, repository not found, wrong passphrase,
permission denied, unknown error
• Single ntfy notification at end: success (low priority) or failure
(urgent) with per-service failure reasons listed
• backup_borg.sh: changed 2>&1 | pipe to 2>"$_ERR" | so stdout logs
cleanly and stderr is captured for error categorization
services/backup.sh, borg-backup.sh, gaming-backup.sh:
• New ntfy prompt section in installer (URL + optional token)
• NTFY_URL / NTFY_TOKEN written to backup.conf
• test_backup.sh copied from extras/ into service dir
• Summary updated to show test_backup.sh commands and ntfy URL
https://claude.ai/code/session_019XgsQ13XKm4Zj3cNsDNwHj
Replace embedded heredocs in the three backup service installers with
cp from versioned source files in extras/:
extras/backup_kopia.sh — Kopia worker (was inline in services/backup.sh)
extras/backup_borg.sh — Borg worker (was inline in services/borg-backup.sh)
extras/backup_gaming.sh — gaming saves worker (was inline in services/gaming-backup.sh)
extras/restore_kopia.sh — unified Kopia restore (multi-dest + single-dest)
extras/restore_borg.sh — unified Borg restore with destination picker
Each installer now does `cp extras/<script>.sh $DIR/<script>.sh` instead of
writing the script inline. Workers and restore scripts are now readable in the
repo rather than buried in heredocs.
Restore scripts are installed flat into the service directory root:
~/docker/backup/backup_kopia.sh ~/docker/backup/restore_kopia.sh
~/docker/borg-backup/backup_borg.sh ~/docker/borg-backup/restore_borg.sh
~/docker/gaming-backup/backup_gaming.sh ~/docker/gaming-backup/restore_kopia.sh
The new restore scripts handle destination selection internally, so a single
script replaces the old per-destination restore/<dest>/ layout.
Also fixes `local` used outside a function in restore_kopia.sh and
restore_borg.sh (destination picker loop), and removes the now-superseded
extras/restore_kopia_backup.sh and extras/restore_borg_backup.sh.
https://claude.ai/code/session_019XgsQ13XKm4Zj3cNsDNwHj
- backup: renamed from disaster-backup; full ~/docker/<service>/ snapshots,
Minecraft flushed not stopped, everything else stop/snap/start for DB
consistency; backup.conf / restore/ layout; post-install-backup timer
- gaming-backup: renamed from backup; frequent game-save snapshots (Minecraft
world data, emulator saves, Steam, Wolf state) with no service downtime;
defaults to hourly schedule; separate gaming-kopia repo + gaming-backup timer
- Deleted services/disaster-backup.sh
- README: updated backup row to reflect new names
https://claude.ai/code/session_019XgsQ13XKm4Zj3cNsDNwHj
- extras/restore_kopia_backup.sh: interactive restore tool — lists all
snapshot sources, pick source + snapshot, choose inspect (restore to
/tmp) or restore-in-place (moves current data aside as .pre-restore-DATE
for easy rollback, auto-stops/starts the associated Docker service)
- services/backup.sh: copy restore script to ~/docker/backup/ during
install; update summary to show restore command
- CLAUDE.md: add note to update README services table when adding a
service; add Authelia section documenting which services have built-in
auth, the import-authelia Caddy pattern for no-auth services, and
remember_me_duration config for kiosk/persistent sessions
https://claude.ai/code/session_019XgsQ13XKm4Zj3cNsDNwHj
Claude