Add automatic tab-completion setup and old config-backup pruning
Two things surfaced from actual use this session: 1. Tab completion (tools/setup-completion.bash, added earlier) required manually editing ~/.bashrc — easy to skip or get wrong (confirmed live: the source line never actually landed the first time). base now wires it in automatically (idempotent, checked by grep first), matching how it already touches ~/.bashrc for SSH Host aliases. 2. No pruning existed anywhere for the *.backup.<timestamp> files ~60 different services create before overwriting a live config (Caddyfile, /etc/fstab, etc) — every one of them backs up, none clean up, so they accumulate forever on a box reconfigured regularly. tools/prune-old-backups.sh prunes by file mtime (not by parsing the timestamp out of the filename — robust to the %Y%m%d-%H%M%S vs %Y%m%d_%H%M%S inconsistency across services), always keeping the single newest backup per distinct file regardless of age. Verified both the normal case (mixed old/new, prunes only the old ones) and the edge case (every backup for a file is old, keeps the newest one anyway) against real fixtures. base offers it as a daily systemd timer (prompted, since it deletes files — unlike the tab-completion wiring, which doesn't). Also added logrotate for Caddy's own access logs (/var/log/caddy/*.log), which had no rotation at all and grow unbounded on an active box. Uses copytruncate specifically: the log directory is bind-mounted into the running Caddy container and read live by CrowdSec, so truncating in place avoids either of them needing to notice or react to a rotation happening. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01H4k6J1qXXyYxhGEgnJaMvn
This commit is contained in:
@@ -149,6 +149,10 @@ sudo ./setup.sh --unattended base # non-interactive, use defaults
|
|||||||
|
|
||||||
### Tab completion
|
### Tab completion
|
||||||
|
|
||||||
|
Set up automatically by `base` (checks `~/.bashrc` first, so a rerun never
|
||||||
|
adds it twice) — open a new shell, or `source ~/.bashrc`, and it's active.
|
||||||
|
To add it manually on a box that installed `base` before this existed:
|
||||||
|
|
||||||
```bash
|
```bash
|
||||||
echo "source $(pwd)/tools/setup-completion.bash" >> ~/.bashrc
|
echo "source $(pwd)/tools/setup-completion.bash" >> ~/.bashrc
|
||||||
source ~/.bashrc
|
source ~/.bashrc
|
||||||
@@ -313,6 +317,27 @@ docker compose pull && docker compose up -d # update
|
|||||||
docker compose down # stop
|
docker compose down # stop
|
||||||
```
|
```
|
||||||
|
|
||||||
|
## Old config backup pruning
|
||||||
|
|
||||||
|
Every service in this repo backs up a live config before overwriting it —
|
||||||
|
`Caddyfile.backup.<timestamp>`, `/etc/fstab.backup.<timestamp>`, and so on —
|
||||||
|
but nothing cleans those up afterward, so they build up on any box
|
||||||
|
reconfigured regularly. `base` offers a daily systemd timer
|
||||||
|
(`prune-old-backups`) that removes anything older than 30 days, always
|
||||||
|
keeping at least the single newest backup per file regardless of age — a
|
||||||
|
box left alone for months never ends up with zero backups for something.
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo bash tools/prune-old-backups.sh [KEEP_DAYS] # run by hand, default 30
|
||||||
|
systemctl status prune-old-backups.timer # check the schedule
|
||||||
|
sudo systemctl disable --now prune-old-backups.timer # turn it off
|
||||||
|
```
|
||||||
|
|
||||||
|
Caddy's own access logs (`/var/log/caddy/*.log`) are handled separately —
|
||||||
|
`caddy` sets up `logrotate` for those directly (14 days, `copytruncate` so
|
||||||
|
neither the running container nor CrowdSec's log tailing has to notice a
|
||||||
|
rotation happened).
|
||||||
|
|
||||||
## SSH key import (GitHub/Launchpad) and disabling password login
|
## SSH key import (GitHub/Launchpad) and disabling password login
|
||||||
|
|
||||||
Imports your public keys from GitHub and/or Launchpad (Canonical/Ubuntu's
|
Imports your public keys from GitHub and/or Launchpad (Canonical/Ubuntu's
|
||||||
|
|||||||
@@ -22,6 +22,8 @@ install_base() {
|
|||||||
echo "[DRY-RUN] Would offer Caddy reverse proxy install (full repo only)"
|
echo "[DRY-RUN] Would offer Caddy reverse proxy install (full repo only)"
|
||||||
echo "[DRY-RUN] Would offer CrowdSec intrusion prevention install (full repo only)"
|
echo "[DRY-RUN] Would offer CrowdSec intrusion prevention install (full repo only)"
|
||||||
echo "[DRY-RUN] Would offer to add SSH Host aliases to ~/.ssh/config"
|
echo "[DRY-RUN] Would offer to add SSH Host aliases to ~/.ssh/config"
|
||||||
|
echo "[DRY-RUN] Would add setup.sh tab completion to ~/.bashrc (if not already there)"
|
||||||
|
echo "[DRY-RUN] Would offer daily pruning of old *.backup.* config files (systemd timer)"
|
||||||
return 0
|
return 0
|
||||||
fi
|
fi
|
||||||
|
|
||||||
@@ -82,6 +84,82 @@ install_base() {
|
|||||||
|
|
||||||
# ── SSH Host aliases ─────────────────────────────────────────────────────
|
# ── SSH Host aliases ─────────────────────────────────────────────────────
|
||||||
_base_setup_ssh_aliases
|
_base_setup_ssh_aliases
|
||||||
|
|
||||||
|
# ── setup.sh tab completion ─────────────────────────────────────────────
|
||||||
|
_base_setup_tab_completion
|
||||||
|
|
||||||
|
# ── Old config-backup pruning ────────────────────────────────────────────
|
||||||
|
_base_setup_backup_pruning
|
||||||
|
}
|
||||||
|
|
||||||
|
# Wires tools/setup-completion.bash into ACTUAL_USER's shell automatically —
|
||||||
|
# no reason to make everyone find and run this by hand when base already
|
||||||
|
# touches ~/.bashrc for other things. Idempotent (checked by grep before
|
||||||
|
# appending), so reruns don't pile up duplicate source lines.
|
||||||
|
_base_setup_tab_completion() {
|
||||||
|
local comp_script="$HERE/tools/setup-completion.bash"
|
||||||
|
[ -f "$comp_script" ] || return 0
|
||||||
|
|
||||||
|
local bashrc="$ACTUAL_HOME/.bashrc"
|
||||||
|
[ -f "$bashrc" ] || return 0
|
||||||
|
grep -qF "$comp_script" "$bashrc" 2>/dev/null && return 0
|
||||||
|
|
||||||
|
# No DRY_RUN check here — install_base()'s own top-level one (above)
|
||||||
|
# already returns before this helper is ever called in that mode,
|
||||||
|
# unlike install_glow()'s check further down, which is independently
|
||||||
|
# invokable (sudo ./setup.sh glow --dry-run) and genuinely reachable.
|
||||||
|
{
|
||||||
|
echo ""
|
||||||
|
echo "# ubuntu-post-install: setup.sh tab completion"
|
||||||
|
echo "source $comp_script"
|
||||||
|
} >> "$bashrc"
|
||||||
|
chown "$ACTUAL_USER:$ACTUAL_USER" "$bashrc" 2>/dev/null || true
|
||||||
|
log_success "setup.sh tab completion added to $bashrc (takes effect in new shells, or: source $bashrc)"
|
||||||
|
}
|
||||||
|
|
||||||
|
# Every service in this repo backs up a live config before overwriting it
|
||||||
|
# (Caddyfile, /etc/fstab, ...) but none of them ever clean those up
|
||||||
|
# afterward — see tools/prune-old-backups.sh's own header for the full
|
||||||
|
# reasoning. Offers a daily systemd timer that prunes anything older than
|
||||||
|
# 30 days, always keeping at least the single newest backup per file
|
||||||
|
# regardless of age.
|
||||||
|
_base_setup_backup_pruning() {
|
||||||
|
command -v systemctl >/dev/null 2>&1 || return 0
|
||||||
|
systemctl list-unit-files prune-old-backups.timer --no-legend 2>/dev/null | grep -q . && return 0
|
||||||
|
|
||||||
|
local prune_script="$HERE/tools/prune-old-backups.sh"
|
||||||
|
[ -f "$prune_script" ] || return 0
|
||||||
|
|
||||||
|
echo ""
|
||||||
|
local ENABLE_PRUNE=""
|
||||||
|
prompt_yn "Automatically prune old config backups (Caddyfile.backup.*, fstab.backup.*, etc — keeps 30 days, always keeps at least the newest one)? (y/n):" "y" ENABLE_PRUNE
|
||||||
|
[[ "$ENABLE_PRUNE" =~ ^[Yy]$ ]] || return 0
|
||||||
|
|
||||||
|
cat > /etc/systemd/system/prune-old-backups.service << UNIT
|
||||||
|
[Unit]
|
||||||
|
Description=Prune old *.backup.* config backups (Caddyfile, fstab, etc)
|
||||||
|
|
||||||
|
[Service]
|
||||||
|
Type=oneshot
|
||||||
|
ExecStart=/bin/bash ${prune_script} 30
|
||||||
|
UNIT
|
||||||
|
cat > /etc/systemd/system/prune-old-backups.timer << 'UNIT'
|
||||||
|
[Unit]
|
||||||
|
Description=Daily backup pruning
|
||||||
|
|
||||||
|
[Timer]
|
||||||
|
OnCalendar=daily
|
||||||
|
Persistent=true
|
||||||
|
|
||||||
|
[Install]
|
||||||
|
WantedBy=timers.target
|
||||||
|
UNIT
|
||||||
|
systemctl daemon-reload
|
||||||
|
if systemctl enable --now prune-old-backups.timer >/dev/null 2>&1; then
|
||||||
|
log_success "Old config backups will be pruned daily, keeping 30 days (systemd timer: prune-old-backups)"
|
||||||
|
else
|
||||||
|
log_warning "Couldn't enable the pruning timer — run $prune_script manually to prune old backups."
|
||||||
|
fi
|
||||||
}
|
}
|
||||||
|
|
||||||
_base_setup_nvidia_gpu() {
|
_base_setup_nvidia_gpu() {
|
||||||
|
|||||||
@@ -233,6 +233,7 @@ install_caddy() {
|
|||||||
echo "[DRY-RUN] Would write a starter $CADDY_DIR/Caddyfile (if none exists)"
|
echo "[DRY-RUN] Would write a starter $CADDY_DIR/Caddyfile (if none exists)"
|
||||||
echo "[DRY-RUN] Would write $CADDY_DIR/README.md"
|
echo "[DRY-RUN] Would write $CADDY_DIR/README.md"
|
||||||
echo "[DRY-RUN] Would open 80/tcp, 443/tcp, 443/udp in UFW (Docker's own iptables rules let this traffic through either way, but ufw status should actually reflect it)"
|
echo "[DRY-RUN] Would open 80/tcp, 443/tcp, 443/udp in UFW (Docker's own iptables rules let this traffic through either way, but ufw status should actually reflect it)"
|
||||||
|
echo "[DRY-RUN] Would configure logrotate for /var/log/caddy/*.log (14 days, copytruncate)"
|
||||||
echo "[DRY-RUN] Would optionally start Caddy (docker compose up -d)"
|
echo "[DRY-RUN] Would optionally start Caddy (docker compose up -d)"
|
||||||
return 0
|
return 0
|
||||||
fi
|
fi
|
||||||
@@ -385,6 +386,29 @@ CADDYFILE
|
|||||||
chown -R "$ACTUAL_USER:$ACTUAL_USER" "$CADDY_DIR"
|
chown -R "$ACTUAL_USER:$ACTUAL_USER" "$CADDY_DIR"
|
||||||
echo " ✓ Caddy configured at $CADDY_DIR"
|
echo " ✓ Caddy configured at $CADDY_DIR"
|
||||||
|
|
||||||
|
# Every site's access log (one JSON file per domain, per the "log {}"
|
||||||
|
# block configure_caddy_for_service writes) grows forever otherwise —
|
||||||
|
# nothing in this repo ever rotated them. copytruncate, not the usual
|
||||||
|
# rename+recreate rotation: the log directory is bind-mounted into the
|
||||||
|
# running Caddy container (still holding the file open) and read live
|
||||||
|
# by CrowdSec, so truncating in place avoids both "Caddy keeps writing
|
||||||
|
# to the old, now-unlinked file" and "CrowdSec's tail loses the file"
|
||||||
|
# — neither has to detect or react to a rotation at all this way.
|
||||||
|
if [ -d /etc/logrotate.d ]; then
|
||||||
|
cat > /etc/logrotate.d/caddy << 'LOGROTATE'
|
||||||
|
/var/log/caddy/*.log {
|
||||||
|
daily
|
||||||
|
rotate 14
|
||||||
|
compress
|
||||||
|
delaycompress
|
||||||
|
missingok
|
||||||
|
notifempty
|
||||||
|
copytruncate
|
||||||
|
}
|
||||||
|
LOGROTATE
|
||||||
|
echo " ✓ Access log rotation configured (/var/log/caddy/*.log, 14 days, copytruncate)"
|
||||||
|
fi
|
||||||
|
|
||||||
# Every other service in this repo opens its own UFW rule; this file
|
# Every other service in this repo opens its own UFW rule; this file
|
||||||
# never did — Docker manipulates iptables directly for published
|
# never did — Docker manipulates iptables directly for published
|
||||||
# container ports (the ports: mapping above), which bypasses UFW's own
|
# container ports (the ports: mapping above), which bypasses UFW's own
|
||||||
|
|||||||
@@ -0,0 +1,69 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
# tools/prune-old-backups.sh — Deletes old *.backup.* files this repo's
|
||||||
|
# services create before overwriting a live config (Caddyfile, /etc/fstab,
|
||||||
|
# etc — the pattern is always <name>.backup.<timestamp>, though the exact
|
||||||
|
# timestamp format varies by service: %Y%m%d-%H%M%S vs %Y%m%d_%H%M%S).
|
||||||
|
# Nothing in this repo ever cleans these up at the point they're created —
|
||||||
|
# every service backs up before a destructive write, none of them prune
|
||||||
|
# afterward, so they accumulate forever on any box reconfigured regularly.
|
||||||
|
#
|
||||||
|
# Prunes by file mtime (not by parsing the timestamp out of the filename —
|
||||||
|
# robust to the format inconsistency above, since it never has to parse it
|
||||||
|
# at all), and always keeps the single newest backup per distinct file
|
||||||
|
# regardless of age, so a box that hasn't been touched in months never
|
||||||
|
# ends up with zero backups for something.
|
||||||
|
#
|
||||||
|
# Usage:
|
||||||
|
# sudo bash prune-old-backups.sh [KEEP_DAYS] (default 30)
|
||||||
|
|
||||||
|
set -uo pipefail
|
||||||
|
|
||||||
|
KEEP_DAYS="${1:-30}"
|
||||||
|
[[ "$KEEP_DAYS" =~ ^[0-9]+$ ]] || { echo "KEEP_DAYS must be a number" >&2; exit 1; }
|
||||||
|
|
||||||
|
ACTUAL_USER="${SUDO_USER:-${USER:-root}}"
|
||||||
|
ACTUAL_HOME="$(getent passwd "$ACTUAL_USER" 2>/dev/null | cut -d: -f6 || echo "/home/$ACTUAL_USER")"
|
||||||
|
DOCKER_DIR="${DOCKER_DIR:-$ACTUAL_HOME/docker}"
|
||||||
|
|
||||||
|
SEARCH_DIRS=()
|
||||||
|
[ -d "$DOCKER_DIR" ] && SEARCH_DIRS+=("$DOCKER_DIR")
|
||||||
|
[ -d /etc ] && SEARCH_DIRS+=(/etc)
|
||||||
|
|
||||||
|
if [ "${#SEARCH_DIRS[@]}" -eq 0 ]; then
|
||||||
|
echo "Nothing to scan (no $DOCKER_DIR, no /etc)."
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
# Pass 1: find every distinct "base" (the path with .backup.TIMESTAMP
|
||||||
|
# stripped) and remember its single newest match — that one is always
|
||||||
|
# kept below, regardless of age.
|
||||||
|
declare -A NEWEST_PER_BASE
|
||||||
|
while IFS= read -r -d '' f; do
|
||||||
|
base="${f%.backup.*}"
|
||||||
|
cur="${NEWEST_PER_BASE[$base]:-}"
|
||||||
|
if [ -z "$cur" ] || [ "$f" -nt "$cur" ]; then
|
||||||
|
NEWEST_PER_BASE["$base"]="$f"
|
||||||
|
fi
|
||||||
|
done < <(find "${SEARCH_DIRS[@]}" -maxdepth 6 -type f -name "*.backup.*" -print0 2>/dev/null)
|
||||||
|
|
||||||
|
DELETED=0
|
||||||
|
KEPT_AS_NEWEST=0
|
||||||
|
|
||||||
|
# Pass 2: delete only files older than KEEP_DAYS, skipping any that are
|
||||||
|
# the sole/newest backup for their base (the safety net above).
|
||||||
|
while IFS= read -r -d '' f; do
|
||||||
|
is_newest=false
|
||||||
|
for keep in "${NEWEST_PER_BASE[@]}"; do
|
||||||
|
if [ "$f" = "$keep" ]; then
|
||||||
|
is_newest=true
|
||||||
|
break
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
if [ "$is_newest" = true ]; then
|
||||||
|
KEPT_AS_NEWEST=$((KEPT_AS_NEWEST + 1))
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
rm -f "$f" && DELETED=$((DELETED + 1))
|
||||||
|
done < <(find "${SEARCH_DIRS[@]}" -maxdepth 6 -type f -name "*.backup.*" -mtime "+${KEEP_DAYS}" -print0 2>/dev/null)
|
||||||
|
|
||||||
|
echo "Backup pruning: removed $DELETED file(s) older than ${KEEP_DAYS} days (kept $KEPT_AS_NEWEST as the newest-per-file safety net)"
|
||||||
Reference in New Issue
Block a user