root/ubuntu-post-install
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Merge pull request #48 from outis1one/claude/peaceful-goodall-nIfh9

Browse Source 
docs: add CLAUDE.md, move backup guide into installer, drop linux-to-…
This commit is contained in:
Outis
2026-06-04 10:44:30 -04:00
committed by GitHub
parent b467672a9e 177598a79e
commit ec6007cb5f
5 changed files with 301 additions and 304 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CLAUDE.md
+236
View File
@@ -0,0 +1,236 @@
# CLAUDE.md — ubuntu-post-install contributor guide
Context for adding or modifying services. Read this before touching any
service file so the result matches what's already here.
## How the system works
`setup.sh` sources `lib/common.sh` then globs every `services/*.sh` file.
Each service file self-registers and defines its install function. Nothing
in `setup.sh` needs to change when you add a service — just add the file.
The wizard groups services by category (from `register_service`), shows a
checklist per group, and calls `install_<name>()` for each selected item.
`--list`, `--dry-run`, and `--unattended` all work automatically.
## Adding a service — the three-step rule
1. Create `services/<name>.sh` (kebab-case filename)
2. Call `register_service` at the top of the file
3. Define `install_<name>()` (hyphens → underscores in function name)
That's it. The menu picks it up on the next run.
## Minimal Docker service template
```bash
#!/bin/bash
# services/my-tool.sh — One-line description.
# Part of the modular post-install system (sourced by setup.sh).
register_service my-tool utilities "What it does (My Tool)" 8080
install_my_tool() {
require_docker || return 1
local DIR="$DOCKER_DIR/my-tool"
if [ "$DRY_RUN" = true ]; then
echo "[DRY-RUN] Would create $DIR with docker-compose.yml"
return 0
fi
mkdir -p "$DIR"
ensure_docker_dir_ownership "$DIR"
cd "$DIR" || return 1
cat > docker-compose.yml << 'EOF'
name: my-tool
services:
my-tool:
image: vendor/my-tool:latest
container_name: my-tool
restart: unless-stopped
ports:
- "8080:8080"
volumes:
- ./data:/data
EOF
configure_caddy_for_service "My Tool" "8080" "my-tool"
write_readme "$DIR" << 'MD'
# My Tool
Brief description.
## Manage
```bash
docker compose up -d
docker compose down
docker compose logs -f
docker compose pull && docker compose up -d
```
MD
local START=""
prompt_yn "Start My Tool now? (y/n):" "y" START
if [ "$START" = "y" ] || [ "$START" = "Y" ]; then
docker compose up -d \
&& log_success "My Tool started" \
|| log_warning "Start failed — check: docker compose logs"
fi
}
```
## register_service signature
```bash
register_service <name> <group> "<description>" [port]
```
- `name` — kebab-case, matches the filename and the `install_` function
- `group` — one of the categories below; determines which menu it appears in
- `description` — shown in `--list` and the menu checklist
- `port` — optional; informational only (not used by the framework)
## Available globals
| Variable | Value |
|----------|-------|
| `DOCKER_DIR` | `~/docker` — parent for all Docker service directories |
| `ACTUAL_USER` | The non-root user that invoked sudo |
| `ACTUAL_HOME` | Home directory of `ACTUAL_USER` |
| `SITE_TZ` | Timezone from site config, e.g. `America/New_York` |
| `SITE_DOMAIN` | Base domain from site config, e.g. `example.com` |
| `SITE_CADDY_NET` | Docker network name for Caddy (default: `caddy_net`) |
| `DRY_RUN` | `true`/`false` — set by `--dry-run` flag |
| `UNATTENDED` | `true`/`false` — set by `--unattended` flag |
## Available helpers (lib/common.sh)
### Logging
```bash
log_info "message" # blue [INFO]
log_success "message" # green [OK]
log_warning "message" # yellow [WARN]
log_error "message" # red [ERROR]
```
### Prompts — honor `UNATTENDED` automatically
```bash
prompt_yn "Question? (y/n):" "default_y_or_n" VARNAME
prompt_text "Question? [default]:" "default" VARNAME
```
When `UNATTENDED=true` both functions skip the prompt and use the default.
### Pre-flight
```bash
require_root # exits with an error if not running as root
require_docker # installs Docker CE + Compose plugin if missing, then returns
```
### Execution and ownership
```bash
run_cmd COMMAND [args...] # no-ops in DRY_RUN, executes otherwise
ensure_docker_dir_ownership DIR... # chown -R ACTUAL_USER:ACTUAL_USER (skips in DRY_RUN)
generate_password [length] # alphanumeric random string, default 32 chars
pip_user_install PACKAGE... # pip3 --user with --break-system-packages on 24.04+
```
### Caddy reverse proxy
```bash
configure_caddy_for_service "Display Name" "PORT" "default-subdomain" ["extra-block"]
```
Prompts the user for a domain, appends a site block to the Caddyfile, and
reloads Caddy. No-ops silently if Caddy isn't installed. The fourth argument
is an optional string inserted verbatim inside the Caddy site block (use it
for `import authelia` or custom matchers).
### README generation
```bash
write_readme "$DIR" << 'MD'
# Title
Content
MD
```
Writes `$DIR/README.md` (creates the directory if needed). No-ops in DRY_RUN.
Every Docker service should call this so `~/docker/<name>/README.md` is
self-documenting on the deployed box.
## Categories
| Group | Purpose |
|-------|---------|
| `base` | CLI packages installed on every box |
| `homelab` | Core infrastructure — reverse proxy, auth, intrusion prevention |
| `utilities` | Self-hosted web apps — budget, DNS, files, monitoring, VPN, etc. |
| `media` | Media servers, photo backup, disc ripping |
| `cameras` | NVR and camera tooling (Frigate) |
| `gaming` | Game servers, cloud gaming (Wolf), emulation |
| `extras` | Non-Docker tools and scripts |
| `backup` | Backup solutions |
## Non-Docker services
Not everything is a container. For apt-based or git-clone–based services,
skip `require_docker` and the Docker helpers. See `services/base.sh` (apt
packages + Charm repo) and `services/crowdsec.sh` (official apt repo) as
reference patterns.
For non-Docker services the default `is_installed` check in `setup.sh`
looks for `$DOCKER_DIR/$name`, which won't exist. Add a case to the
`is_installed()` function in `setup.sh` so the `[installed]` marker appears
correctly in the menu:
```bash
# In setup.sh → is_installed()
my-tool) command -v my-tool >/dev/null 2>&1 ;;
```
Docker services use the default case and don't need an entry.
## DRY_RUN convention
Every `install_*` function must check `$DRY_RUN` before touching the
filesystem, installing packages, or starting containers. The pattern is:
```bash
if [ "$DRY_RUN" = true ]; then
echo "[DRY-RUN] Would do X"
echo "[DRY-RUN] Would do Y"
return 0
fi
```
Put the check early — after any pure-display output (banners, info text)
but before the first write.
## .env files and secrets
Generate passwords with `generate_password` (never hardcode them).
Write secrets to `.env` files in the service directory, owned by
`ACTUAL_USER`, permissions 600. Document every variable with a comment
in the `.env` heredoc so the user knows what to change later.
## Caddy network wiring
Services that need to reach Caddy (or each other) over Docker networking
should join the `$SITE_CADDY_NET` network. Add to `docker-compose.yml`:
```yaml
networks:
caddy_net:
external: true
name: ${CADDY_NET:-caddy_net}
```
And read the network name from `.env` using `CADDY_NET=$SITE_CADDY_NET`.
README.md
+3 -173
View File
@@ -49,7 +49,7 @@ sudo ./setup.sh --unattended base # non-interactive, use defaults
## What the wizard does
**First run:**
1. Installs essential CLI packages (see [Base packages](#base-packages) below)
1. Installs essential CLI packages (`net-tools`, `ncdu`, `git`, `curl`, `wget`, `htop`, `tree`, `zip`/`unzip`, `ca-certificates`, `gnupg`, `jq`, `rsync`, `glow`)
2. Checks for Docker CE + Compose plugin; prompts to install if missing
3. Offers to set **site defaults** — timezone, base domain, Caddy Docker network —
so every service picks them up automatically instead of asking each time
@@ -72,181 +72,10 @@ Update them any time with `sudo ./setup.sh configure`.
| `cameras` | `frigate`, `frigate-audio`, `frigate-notify`, `sky-cam` |
| `gaming` | `js99er`, `minecraft`, `wolf`, `wolf-pair` |
| `extras` | `silent-send`, `sync-cc` |
| `backup` | 'Kopia, Borg (encrypted, dedup, scheduled)', rsync (plain or with --link-dest)' rsnapshot |
| `backup` | `backup` — Kopia (built-in, encrypted, dedup, scheduled); Borg, rsync, rsnapshot also supported (see installer) |
Run `./setup.sh --list` to see descriptions.
## Backup
The `backup` service (under the `backup` group) sets up **Kopia** — but you are
not limited to it. Below is a guide to every backup strategy in this repo,
with advice on when to reach for each one.
---
### Kopia — the built-in `backup` service
**What it does:** block-level deduplication + zstd compression + AES-256
encryption, scheduled automatically via a systemd timer (cron fallback).
Each run snapshots configured paths and retains versions according to a
policy (latest N, daily, weekly, monthly). An optional `sync-to` step
mirrors the whole encrypted repository to another computer or a cloud
bucket (SFTP, Backblaze B2, S3, rclone).
**Install:** `sudo ./setup.sh backup`
**When to use:** files that change *constantly* — Minecraft region files,
game saves, Steam Proton prefixes, Docker config volumes. Every changed
block is stored once; identical blocks across snapshots share storage.
Kopia is the right default for homelab data that changes on every write.
---
### Borg Backup
**What it is:** `borgbackup` — chunk-based deduplication + lz4/zstd/zlib
compression + AES-CTR encryption. Mature, battle-tested, wide ecosystem
(Borgmatic for YAML-driven automation, Vorta for a GUI).
```bash
sudo apt install borgbackup
# initialise a repo
borg init --encryption=repokey /backups/borg-repo
# take a snapshot
borg create --stats --progress /backups/borg-repo::'{hostname}-{now}' ~/docker
# list snapshots
borg list /backups/borg-repo
# restore
borg extract /backups/borg-repo::snapshot-name
```
**When to use:** same class of problem as Kopia — frequently-changing files
where deduplication pays off. Choose Borg over Kopia if you prefer its
mature ecosystem, Borgmatic config files, or need to share a repo across
multiple machines. Both are excellent; pick the one whose tooling you
prefer.
---
### rsync — plain mirror (no versioning)
**What it is:** `rsync -av --delete SOURCE/ DEST/` — fast one-way mirror.
`--delete` removes files in `DEST` that no longer exist in `SOURCE`.
The destination is a plain readable copy of the source; no special tool
needed to browse or restore.
```bash
rsync -av --delete /source/media/ /backups/media/
```
**When to use:** files that rarely or never change — media libraries,
ROM collections, game installs you could re-download but prefer to keep
locally. You just need *a copy*, not versioning. rsync is lightweight,
transparent, and the destination needs no special format.
**Not suitable for:** files that change often, because one bad `--delete`
run (source corruption, accidental deletion) immediately destroys the
only copy in the destination.
---
### rsync `--link-dest` — versioned snapshots with original structure
**What it is:** each backup run creates a new dated directory. Unchanged
files are **hard-linked** from the previous backup rather than copied, so
unchanged files cost no extra disk space. Every dated directory looks like
a complete independent snapshot of the source — original folder structure
preserved, no rsnapshot-style naming convention.
```bash
#!/bin/bash
DEST=/backups/snapshots
PREV="$DEST/$(ls -1 "$DEST" | tail -1)" # most recent snapshot
TODAY="$DEST/$(date +%F)"
rsync -av --delete --link-dest="$PREV" /source/ "$TODAY/"
```
Run this daily (via cron or a systemd timer) and you get:
```
/backups/snapshots/
2024-01-13/ ← full copy (first run)
2024-01-14/ ← only changed files stored; rest are hard links
2024-01-15/ ← same
```
If yesterday's run with `--delete` removed everything from the source
(accidental wipe, filesystem corruption), today's snapshot will also be
empty — but `2024-01-14/` and `2024-01-13/` are untouched and fully
restorable with a plain `cp -al` or `rsync`.
**When to use:** general files where you want versioned point-in-time
backups **and** need the original folder structure preserved. No extra
tool required to restore — every dated folder is browsable with `ls` and
copyable with standard Unix tools. Simpler than Borg/Kopia; no encryption
or deduplication across snapshot boundaries.
**Compared to rsnapshot:** rsync `--link-dest` keeps your own naming and
structure; rsnapshot imposes `daily.0/`, `daily.1/`, etc. and manages
rotation automatically. `--link-dest` gives you more control; rsnapshot
gives you easier automation.
---
### rsnapshot — automated versioned snapshots
**What it is:** a wrapper around rsync that manages hard-link snapshots
automatically using a retention scheme (`hourly.0`, `daily.0`, `weekly.0`,
…). Configure sources and retention in `/etc/rsnapshot.conf`, then run on
a schedule.
```bash
sudo apt install rsnapshot
# edit /etc/rsnapshot.conf — set snapshot_root, backup sources, retain counts
rsnapshot daily # run manually or via cron
rsnapshot -t daily # dry-run / test config
```
The resulting layout looks like:
```
/backups/rsnapshot/
daily.0/ ← most recent
daily.1/
daily.2/
weekly.0/
```
Each interval directory contains a full view of the source, hard-linked
where files are unchanged.
**When to use:** you want automated versioned backups without writing your
own `--link-dest` rotation script, and you don't mind that the destination
uses rsnapshot's own directory naming rather than your original structure.
Good for simple setups (home directories, config files) where the rotation
automation saves time.
**Not ideal for:** frequently-changing large files (game saves, databases)
— use Kopia or Borg instead for efficient deduplication across many
changed blocks.
---
### Choosing the right tool
| Scenario | Recommended tool |
|----------|-----------------|
| Minecraft worlds, game saves, Steam prefixes — change on every write | **Kopia** (built-in) or **Borg** |
| Media library, ROMs — rarely change, just need a copy | **rsync plain** |
| General files — want versioning, want original folder structure | **rsync `--link-dest`** |
| Want automated rotation without scripting `--link-dest` yourself | **rsnapshot** |
| Multi-machine deduplicated repo, YAML-driven config (Borgmatic) | **Borg** |
## Layout
```
@@ -254,6 +83,7 @@ setup.sh dispatcher — wizard, direct install, --list, --dry-run
lib/common.sh shared helpers: logging, prompts, site config, OS detection
services/ one file per service (self-registering)
extras/ non-Docker assets bundled with the repo (e.g. sync_cc.py)
CLAUDE.md contributor guide — how to add services, available helpers
```
## Managing installed services
services/backup.sh
+62
View File
@@ -25,6 +25,68 @@ register_service backup backup "Automatic encrypted backups (Kopia)"
install_backup() {
log_info "Setting up automatic encrypted backups (Kopia)..."
echo ""
echo "╔═══════════════════════════════════════════════════════════════════╗"
echo "║ BACKUP STRATEGIES — choose the right tool for your data ║"
echo "╚═══════════════════════════════════════════════════════════════════╝"
echo ""
echo " This installer sets up Kopia (the recommended default), but here"
echo " is a quick guide to all available options so you can pick the"
echo " right tool for each type of data."
echo ""
echo " ┌─────────────────────────────────────────────────────────────────┐"
echo " │ KOPIA (installed here) — block-level dedup + zstd + encryption │"
echo " │ Use for: files that change constantly — Minecraft worlds, │"
echo " │ game saves, Steam prefixes, Docker config volumes. │"
echo " │ Changed blocks are stored once; old blocks are shared. │"
echo " │ Restores via: kopia snapshot list / kopia restore │"
echo " └─────────────────────────────────────────────────────────────────┘"
echo ""
echo " ┌─────────────────────────────────────────────────────────────────┐"
echo " │ BORG (sudo apt install borgbackup) — chunk dedup + encryption │"
echo " │ Use for: same as Kopia. Choose Borg if you prefer Borgmatic │"
echo " │ (YAML config), Vorta (GUI), or multi-machine repos. │"
echo " │ borg init / borg create / borg list / borg extract │"
echo " └─────────────────────────────────────────────────────────────────┘"
echo ""
echo " ┌─────────────────────────────────────────────────────────────────┐"
echo " │ RSYNC plain rsync -av --delete /src/ /dest/ │"
echo " │ Use for: media, ROMs, files that rarely change and you just │"
echo " │ need a copy. Fast, transparent — no special restore tool. │"
echo " │ Not suitable for files that change often (one bad --delete │"
echo " │ run immediately destroys the only copy in the destination). │"
echo " └─────────────────────────────────────────────────────────────────┘"
echo ""
echo " ┌─────────────────────────────────────────────────────────────────┐"
echo " │ RSYNC --link-dest versioned snapshots, original folder layout │"
echo " │ Creates dated dirs (2024-01-15/, 2024-01-16/, …). │"
echo " │ Unchanged files are hard-linked — cost no extra disk space. │"
echo " │ Each dated dir is a complete, browsable snapshot of the │"
echo " │ source. Original folder structure preserved (unlike │"
echo " │ rsnapshot). If today's --delete wiped something, yesterday's │"
echo " │ dated dir is untouched. │"
echo " │ Use for: general files where you want versioning + readable │"
echo " │ snapshot dirs without a special restore tool. │"
echo " └─────────────────────────────────────────────────────────────────┘"
echo ""
echo " ┌─────────────────────────────────────────────────────────────────┐"
echo " │ RSNAPSHOT (sudo apt install rsnapshot) — automated rotation │"
echo " │ Wraps rsync with a retention scheme (daily.0, weekly.0, …). │"
echo " │ Hard-links unchanged files like --link-dest, but dirs are │"
echo " │ named by rsnapshot (not your original structure). │"
echo " │ Use for: automated versioning without scripting --link-dest, │"
echo " │ when the rsnapshot naming convention doesn't bother you. │"
echo " └─────────────────────────────────────────────────────────────────┘"
echo ""
echo " Quick reference:"
echo " Constantly-changing data (saves, worlds, configs) → Kopia or Borg"
echo " Media / ROMs (rarely changes, just need a copy) → rsync plain"
echo " Versioned snapshots, keep original folder layout → rsync --link-dest"
echo " Versioned snapshots, want auto rotation scripted → rsnapshot"
echo ""
echo " Continuing with Kopia setup..."
echo ""
# ── Repo-conventional paths ──────────────────────────────────────────────
local BACKUP_DIR="$DOCKER_DIR/backup"
local CONF_FILE="$BACKUP_DIR/backup.conf" # editable settings
services/linux-to-sync.sh
-130
View File
@@ -1,130 +0,0 @@
#!/bin/bash
# services/linux-to-sync.sh — Clone the private linux-to-sync repository.
# Part of the modular post-install system (sourced by setup.sh).
#
# Ported from ubuntu-post-install-24.04-crowdsec.sh (# ---- LINUX-TO-SYNC ----).
# Clones outis1one/linux-to-sync to ~/linux-to-sync via SSH or HTTPS+PAT.
# No server/container — this is a personal sync/config repo.
register_service linux-to-sync extras "Personal sync & config scripts (linux-to-sync private repo)"
install_linux-to-sync() {
local SYNC_DIR="$ACTUAL_HOME/linux-to-sync"
if [ "$DRY_RUN" = true ]; then
echo "[DRY-RUN] linux-to-sync would:"
echo " - Clone outis1one/linux-to-sync to $SYNC_DIR"
echo " - Authenticate via SSH key or GitHub Personal Access Token"
return 0
fi
# ── Re-run: already cloned → offer pull ──────────────────────────────────
if [ -d "$SYNC_DIR/.git" ]; then
log_info "linux-to-sync already cloned at $SYNC_DIR"
local DO_PULL=""
prompt_yn "Pull latest changes? (y/n) [y]:" "y" DO_PULL
if [[ ${DO_PULL:-y} =~ ^[Yy]$ ]]; then
if sudo -u "$ACTUAL_USER" git -C "$SYNC_DIR" pull; then
log_success "linux-to-sync updated"
else
log_warning "git pull failed — check connectivity and credentials"
fi
fi
return 0
fi
echo ""
echo " Requires access to github.com/outis1one/linux-to-sync"
echo " Authenticate with ONE of:"
echo " [1] SSH key already added to your GitHub account"
echo " [2] GitHub Personal Access Token (PAT)"
echo ""
local AUTH_METHOD=""
prompt_text "Authentication method [1=SSH, 2=PAT, default: 1]:" "1" AUTH_METHOD
AUTH_METHOD="${AUTH_METHOD:-1}"
if [ "$AUTH_METHOD" = "2" ]; then
echo ""
echo " Create a PAT at: https://github.com/settings/tokens/new"
echo " Select the 'repo' scope for full repository access."
echo ""
local GH_TOKEN=""
prompt_text "GitHub Personal Access Token:" "" GH_TOKEN
if [ -z "$GH_TOKEN" ]; then
log_warning "No token provided — skipping."
return 0
fi
log_info "Cloning via HTTPS + PAT..."
if sudo -u "$ACTUAL_USER" \
git clone "https://$GH_TOKEN@github.com/outis1one/linux-to-sync.git" "$SYNC_DIR"; then
# Remove token from remote URL so it isn't stored in plain text
sudo -u "$ACTUAL_USER" git -C "$SYNC_DIR" remote set-url origin \
"https://github.com/outis1one/linux-to-sync.git"
log_success "linux-to-sync cloned to $SYNC_DIR"
echo " Token stripped from remote URL. For future pulls use:"
echo " git -C $SYNC_DIR pull (will prompt for credentials)"
echo " Or set up a credential helper:"
echo " git config --global credential.helper store"
else
log_error "Clone failed — check your PAT and network, then retry."
return 1
fi
else
# SSH auth — git must run as the actual user to use their SSH keys.
echo ""
echo " Checking for SSH key in $ACTUAL_HOME/.ssh/ ..."
local SSH_KEY_FOUND=false
for _k in id_ed25519 id_rsa id_ecdsa; do
if [ -f "$ACTUAL_HOME/.ssh/$_k" ]; then
log_info " Found: $ACTUAL_HOME/.ssh/$_k"
SSH_KEY_FOUND=true
break
fi
done
if [ "$SSH_KEY_FOUND" = false ]; then
log_warning "No SSH key found in $ACTUAL_HOME/.ssh/"
echo ""
echo " To generate one:"
echo " ssh-keygen -t ed25519 -C 'your@email.com'"
echo " cat $ACTUAL_HOME/.ssh/id_ed25519.pub"
echo " → Add the public key at: github.com/settings/keys"
echo ""
local CONTINUE=""
prompt_yn "Continue anyway (will fail if no key on GitHub)? (y/n) [n]:" "n" CONTINUE
[[ ${CONTINUE:-n} =~ ^[Yy]$ ]] || return 0
fi
log_info "Cloning via SSH (running as $ACTUAL_USER)..."
if sudo -u "$ACTUAL_USER" \
git clone git@github.com:outis1one/linux-to-sync.git "$SYNC_DIR"; then
log_success "linux-to-sync cloned to $SYNC_DIR"
else
log_error "SSH clone failed."
echo ""
echo " Common causes:"
echo " • SSH key not added to GitHub — go to github.com/settings/keys"
echo " • Key not accepted by ssh-agent — try: ssh-add $ACTUAL_HOME/.ssh/id_ed25519"
echo " • Test with: sudo -u $ACTUAL_USER ssh -T git@github.com"
echo " Then retry: sudo ./setup.sh linux-to-sync"
return 1
fi
fi
write_readme "$SYNC_DIR" << MD
# linux-to-sync
Private personal sync and config repository cloned from outis1one/linux-to-sync.
## Update
\`\`\`bash
cd $SYNC_DIR
git pull
\`\`\`
MD
echo ""
echo " Cloned to: $SYNC_DIR"
echo ""
}
setup.sh
-1
View File
@@ -80,7 +80,6 @@ is_installed() {
glow) command -v glow >/dev/null 2>&1 ;;
crowdsec) command -v cscli >/dev/null 2>&1 ;;
silent-send) [ -d "$ACTUAL_HOME/silent-send/.git" ] ;;
linux-to-sync) [ -d "$ACTUAL_HOME/linux-to-sync/.git" ] ;;
sync-cc) [ -f "$ACTUAL_HOME/sync-cc/sync_cc.py" ] ;;
sky-cam) [ -d "$ACTUAL_HOME/sky-cam/.git" ] ;;
*) [ -e "$DOCKER_DIR/$1" ] ;;