Commit Graph
13 Commits
Author SHA1 Message Date
Claude c48ed039e0 Guide + automate Backblaze B2 offsite mirror setup in backup.sh
Answers a direct ask: offsite mirroring existed only as a REMOTE_TYPE/
REMOTE_ARGS placeholder in backup.conf with a comment pointing at
`kopia repository sync-to --help` — no interactive setup at all, B2 or
otherwise.

Checked before building anything: Kopia's dedicated `sync-to b2`
provider is marked [DEPRECATED] on kopia.io's own command reference.
B2 also offers an S3-compatible endpoint (s3.<region>.backblazeb2.com,
same application key works as the access/secret key pair), and Kopia's
`sync-to s3` provider isn't deprecated — so this targets that path
instead of building on a command on its way out.

What's now automated vs. guided, deliberately split:
- Bucket creation and the application key are walked through as console
  steps, not automated. Object Lock specifically is a one-time,
  bucket-creation-only decision with a real tradeoff (undeletable-by-
  design vs. genuinely can't delete early) that shouldn't be silently
  flipped either way by a script on someone's behalf.
- Once the operator has a bucket + endpoint + scoped application key
  (B2 requires a key scoped to one bucket, not the account master key —
  noted in the walkthrough), this becomes mechanical: run a
  `sync-to s3 --dry-run` against the just-created 'default' repo to
  verify the credentials actually work, and only then write
  REMOTE_TYPE=s3 / REMOTE_ARGS into backup.conf. A bad bucket name or
  key leaves REMOTE_TYPE at "none" with a clear error instead of saving
  a broken config that fails silently at 2am.
- Encryption isn't a separate step — Kopia already encrypts client-side
  with the repository password set earlier in this same flow; called
  that out explicitly since it was asked about as if it needed its own
  setup step.

Also fixed a regression the new prompt would otherwise have caused:
backup.sh has no update/fresh distinction and re-asks everything on
every run, so an already-configured offsite mirror is now read back
from the existing backup.conf and preserved by default — answering "no"
on a re-run no longer silently resets REMOTE_TYPE to "none".

Verified the control flow (not just bash -n) against a mock kopia
binary and stubbed prompts: good credentials wire up REMOTE_TYPE/
REMOTE_ARGS correctly, a rejected credential leaves REMOTE_TYPE at
"none" rather than saving something broken, an existing configured
value survives a "no" answer on re-run, and blank fields skip cleanly
without attempting a dry-run at all.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H4k6J1qXXyYxhGEgnJaMvn
2026-08-13 19:28:00 +00:00
Claude 93d5459a67 Make the backup restore-test schedule configurable, add a run-now option
Answers a direct ask: the automated restore-verify test
(extras/test_backup_kopia.sh — verifies the latest snapshot, restores it
over a moved-aside copy, compares, rolls back, reports PASS/FAIL, sends
an ntfy notification) was already fully non-interactive and already
wired to a systemd timer/cron fallback by install_backup() — it just
had no schedule choice at all, hardcoded to weekly (Saturday 03:00).

Every service in this test stops briefly while its data gets moved
aside and restored back, same interruption profile as the main backup
job — so the schedule is a real tradeoff (more frequent verification vs.
more frequent blips), not a free "always pick the most frequent" choice.
Gave it the same Weekly/Monthly/Custom shape the main backup schedule
prompt above it already offers, instead of a single hardcoded option.

Also added an explicit "run the first test now?" prompt right after
scheduling it — otherwise choosing Monthly means waiting up to a month
before finding out whether the test even works, rather than getting
that initial confirmation immediately and then settling into the
chosen cadence.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H4k6J1qXXyYxhGEgnJaMvn
2026-08-13 19:22:52 +00:00
Claude b7f69090a6 Sync backup.conf/README to a spare box; make DR bring-up fault-tolerant
- dr_bringup.sh: bound every kopia call and docker compose up with a
  timeout so one stuck service can't stall the rest of the batch, and
  only exit non-zero if literally nothing came up — a partial recovery
  is a partial success, not a failed run.
- backup_kopia.sh: optional DR_SYNC_HOST/DR_SYNC_PATH in backup.conf
  scp's backup.conf + README.md to a spare box over SSH after every
  successful backup, so dr_bringup.sh is ready there with no manual
  copy step.
- backup.sh: prompts for the spare's SSH destination, verifies
  connectivity at install time instead of failing silently at 2am, and
  writes ~/docker/backup/README.md (this service never had one) so the
  synced copy documents every command listed above.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NQkdAn3iG5A4WoqU9FHMaN
2026-08-04 13:40:53 +00:00
Claude ed7270dcf2 Add unattended DR bring-up script for the backup service
restore_kopia.sh is interactive and one-service-at-a-time, which doesn't
scale to standing up a cold spare box quickly during a real outage.
dr_bringup.sh restores every service's latest snapshot (or one named
service) and runs docker compose up -d with no prompts, so a full-stack
recovery is one command instead of N interactive restores.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NQkdAn3iG5A4WoqU9FHMaN
2026-08-04 13:31:42 +00:00
Claude ec3f9bfd3f Add remote Caddy support — generate snippet files when Caddy is on another host
New site config key: CADDY_REMOTE_HOST (set via 'sudo ./setup.sh configure').
When set, configure_caddy_for_service operates in "remote" mode instead of
writing to a local Caddyfile:
- Upstream uses CADDY_REMOTE_HOST:PORT (host IP, not container name)
- Snippet saved to ~/docker/caddy-snippets/<subdomain>.caddy
- User is shown scp/rsync commands to copy it to the Caddy machine

Three modes in configure_caddy_for_service (lib/common.sh and inline stubs):
  local:  ~/docker/caddy/ exists → write Caddyfile + reload (existing behavior)
  remote: CADDY_REMOTE_HOST set → save snippet, print copy instructions
  none:   neither configured → silent return (unchanged)

All 31 service standalone bootstrap stubs updated with the new logic.
CADDY_REMOTE_HOST global added to all 42 standalone bootstrap sections.
setup.sh configure now prompts for CADDY_REMOTE_HOST with a clear explanation.
wolf.sh: add missing stubs (configure_caddy_for_service, write_readme,
  prompt_yn, ensure_docker_dir_ownership) and the Authelia/Caddy/start calls
  that were missing from the install function.

https://claude.ai/code/session_014CCYqVwW6d6f5dw1qRokYt
2026-06-09 00:28:30 +00:00
Claude 5aa4a8c91e services: add standalone bootstrap to arm, authelia, backup, borg-backup, caddy, ntfy
Each service can now be run directly with sudo bash <service>.sh on any
machine with Docker installed, without needing the full post-install repo.
Uses the shared bootstrap pattern from docs/standalone-template.sh.

https://claude.ai/code/session_014CCYqVwW6d6f5dw1qRokYt
2026-06-08 17:01:34 +00:00
Outis a3d6a0df3d Add backup test scripts and enhance workers with timing, counts, and pre-flight checks
- extras/test_backup_kopia.sh: stop → restore → compare → roll back test for Kopia
- extras/test_backup_borg.sh:  stop → extract → compare → roll back test for Borg
- backup workers: timing (duration), service count, and pre-flight disk check (< 512 MB warns)
- backup workers: ntfy notifications include count, duration, and per-failure detail
- services/backup.sh: install test_backup_kopia.sh + optional weekly test timer
- services/borg-backup.sh: install test_backup_borg.sh + optional weekly test timer
2026-06-04 17:24:11 -04:00
Claude 862ecf10e9 feat: backup test script, ntfy notifications, and error categorization
extras/test_backup.sh — new unified test script (Kopia + Borg):
  • Stops container, moves live data aside, restores latest backup,
    compares restored vs live with diff -rq (content, not timestamps),
    moves live data back and restarts container
  • PASS = restore succeeded; diff output is informational (files changed
    since last backup are normal)
  • FAIL = restore command failed or target empty after restore
  • --list flag, CLI service arg, interactive picker
  • Handles both full-service dirs and sub-path sources (gaming-backup)
  • Cleanup trap always restores live data even on error
  • Sends ntfy notification on pass and fail

extras/backup_kopia.sh, backup_borg.sh, backup_gaming.sh:
  • ntfy_send() + categorize_error() helpers added
  • Each snapshot/archive failure captures stderr and categorizes:
    disk full, remote unreachable, repository not found, wrong passphrase,
    permission denied, unknown error
  • Single ntfy notification at end: success (low priority) or failure
    (urgent) with per-service failure reasons listed
  • backup_borg.sh: changed 2>&1 | pipe to 2>"$_ERR" | so stdout logs
    cleanly and stderr is captured for error categorization

services/backup.sh, borg-backup.sh, gaming-backup.sh:
  • New ntfy prompt section in installer (URL + optional token)
  • NTFY_URL / NTFY_TOKEN written to backup.conf
  • test_backup.sh copied from extras/ into service dir
  • Summary updated to show test_backup.sh commands and ntfy URL

https://claude.ai/code/session_019XgsQ13XKm4Zj3cNsDNwHj
2026-06-04 20:03:30 +00:00
Claude e55449442f refactor: move backup/restore worker scripts to extras/ as source files
Replace embedded heredocs in the three backup service installers with
cp from versioned source files in extras/:

  extras/backup_kopia.sh   — Kopia worker (was inline in services/backup.sh)
  extras/backup_borg.sh    — Borg worker  (was inline in services/borg-backup.sh)
  extras/backup_gaming.sh  — gaming saves worker (was inline in services/gaming-backup.sh)
  extras/restore_kopia.sh  — unified Kopia restore (multi-dest + single-dest)
  extras/restore_borg.sh   — unified Borg restore with destination picker

Each installer now does `cp extras/<script>.sh $DIR/<script>.sh` instead of
writing the script inline. Workers and restore scripts are now readable in the
repo rather than buried in heredocs.

Restore scripts are installed flat into the service directory root:
  ~/docker/backup/backup_kopia.sh   ~/docker/backup/restore_kopia.sh
  ~/docker/borg-backup/backup_borg.sh   ~/docker/borg-backup/restore_borg.sh
  ~/docker/gaming-backup/backup_gaming.sh   ~/docker/gaming-backup/restore_kopia.sh

The new restore scripts handle destination selection internally, so a single
script replaces the old per-destination restore/<dest>/ layout.

Also fixes `local` used outside a function in restore_kopia.sh and
restore_borg.sh (destination picker loop), and removes the now-superseded
extras/restore_kopia_backup.sh and extras/restore_borg_backup.sh.

https://claude.ai/code/session_019XgsQ13XKm4Zj3cNsDNwHj
2026-06-04 18:50:09 +00:00
Claude 7ae5a20e0e Rename backup services: backup (comprehensive) + gaming-backup (saves-only)
- backup: renamed from disaster-backup; full ~/docker/<service>/ snapshots,
  Minecraft flushed not stopped, everything else stop/snap/start for DB
  consistency; backup.conf / restore/ layout; post-install-backup timer
- gaming-backup: renamed from backup; frequent game-save snapshots (Minecraft
  world data, emulator saves, Steam, Wolf state) with no service downtime;
  defaults to hourly schedule; separate gaming-kopia repo + gaming-backup timer
- Deleted services/disaster-backup.sh
- README: updated backup row to reflect new names

https://claude.ai/code/session_019XgsQ13XKm4Zj3cNsDNwHj
2026-06-04 15:58:15 +00:00
Claude d21a7ea7fb feat: add interactive Kopia restore script, Authelia guidance in CLAUDE.md
- extras/restore_kopia_backup.sh: interactive restore tool — lists all
  snapshot sources, pick source + snapshot, choose inspect (restore to
  /tmp) or restore-in-place (moves current data aside as .pre-restore-DATE
  for easy rollback, auto-stops/starts the associated Docker service)
- services/backup.sh: copy restore script to ~/docker/backup/ during
  install; update summary to show restore command
- CLAUDE.md: add note to update README services table when adding a
  service; add Authelia section documenting which services have built-in
  auth, the import-authelia Caddy pattern for no-auth services, and
  remember_me_duration config for kiosk/persistent sessions

https://claude.ai/code/session_019XgsQ13XKm4Zj3cNsDNwHj
2026-06-04 15:07:04 +00:00
Claude 177598a79e docs: add CLAUDE.md, move backup guide into installer, drop linux-to-sync
- CLAUDE.md: full contributor guide — service template, all helpers,
  globals, DRY_RUN convention, Caddy wiring, non-Docker patterns
- services/backup.sh: print backup strategy guide (Kopia/Borg/rsync/
  rsnapshot + when to use each) at the start of install_backup()
- README.md: remove standalone backup section, fix broken backup row,
  inline base package list, add CLAUDE.md to layout
- services/linux-to-sync.sh: deleted (never worked)
- setup.sh: remove linux-to-sync from is_installed()

https://claude.ai/code/session_019XgsQ13XKm4Zj3cNsDNwHj
2026-06-04 14:29:02 +00:00
Claude 840566e3f8 v0.9.4: gaming modules (wolf, js99er), backup module, versioning
- services/wolf.sh (gaming): Games-on-Whales Wolf / Moonlight, per-service
  folder ~/docker/wolf, wolf-pair dropped, manage.sh pin workflow kept.
- services/js99er.sh (gaming): TI-99/4A emulator, own folder, port 8099,
  Selkies launcher tie-in removed.
- services/backup.sh: Kopia encrypted backups, paths adapted to ~/docker.
- Start versioning: VERSION (0.9.4), CHANGELOG.md, setup.sh --version flag.

All modules pass bash -n; ./setup.sh --list groups base/homelab/gaming/backup;
dry-run run-one exits 0 for every module with real commands guarded.

Note: minecraft module deferred to 0.9.5 (port hit a session limit).

https://claude.ai/code/session_017eA2qqq9jfF2tNtpUYL8vK
2026-06-03 16:25:25 +00:00