Files
ubuntu-post-install/services/gatus.sh
T
Claude cd003dbaf3 Add Gatus auto-sync from Caddyfile, promote ensure_yq to lib/common.sh
Adds one Gatus endpoint per Caddy site block automatically, tagged
group: caddy-sync — the sync only ever adds/removes entries in that
exact group, so anything added by hand (the default external checks,
a custom endpoint) is never touched regardless of what the Caddyfile
looks like. Offered at install time (syncs once immediately) and, if
systemd is available, scheduled via a timer every 15 minutes so a site
added or removed later gets picked up without re-running the installer
— matches the "schedule that checks the Caddyfile" shape asked for.

Domain extraction tracks actual brace depth (reusing the same approach
as remove_service's Caddy block removal) rather than a naive
line-by-line scan, so it correctly skips the global options block and
parenthesized snippet definitions like (authelia) without needing to
special-case them by name.

Verified end-to-end against a real Caddyfile/config.yaml fixture with
the actual mikefarah/yq binary: initial sync adds the right entries
and leaves the default "external" group alone, a second run with no
Caddyfile changes is a true no-op (0 added, 0 removed), and changing
the Caddyfile (removing one site, adding another) correctly adds the
new endpoint and removes only the stale one.

Also fixes a real gap surfaced while building this: ensure_yq (used by
both gatus.sh now and onlyoffice.sh already) checked `command -v yq`
alone, which a box can satisfy with a completely different, incompatible
yq — confirmed live in this environment, Debian/Ubuntu's own `yq`
apt package is kislyuk/yq (a Python jq-wrapper) which silently errors
on mikefarah/yq's `e '.path' file` syntax every caller here depends on.
Now checks the version string actually identifies as mikefarah's
before trusting it, installing to /usr/local/bin (which precedes
/usr/bin on Ubuntu's default PATH) if not. Promoted ensure_yq itself
from onlyoffice.sh (its only previous user) to lib/common.sh now that
gatus.sh needs the same thing, so both share one implementation.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01H4k6J1qXXyYxhGEgnJaMvn
2026-08-11 04:26:08 +00:00

541 lines
20 KiB
Bash

#!/bin/bash
# services/gatus.sh — Gatus status/uptime monitoring page.
# Part of the modular post-install system (sourced by setup.sh).
#
# Can also be run standalone on any machine:
# sudo bash gatus.sh
# (Docker must already be installed when run standalone)
#
# Gatus polls endpoints (HTTP, TCP, DNS, ICMP) on a schedule and shows a
# clean status dashboard. Config is hot-reloaded from gatus_config/config.yaml.
# ── Standalone bootstrap ──────────────────────────────────────────────────────
# Detected when the script is executed directly rather than sourced by setup.sh.
# Sets up helpers and globals, then defers execution until after the function
# definition at the bottom of this file.
if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
[[ "$(id -u)" == "0" ]] || { echo "Run with sudo: sudo bash $0"; exit 1; }
_SELF_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
_COMMON="$_SELF_DIR/../lib/common.sh"
if [[ -f "$_COMMON" ]]; then
# Full repo present — use the real helpers (picks up ~/docker/.config too)
# shellcheck source=../lib/common.sh
source "$_COMMON"
else
# One-off copy — inline minimal stubs so the script works without the repo
log_info() { echo -e "\033[0;34m[INFO]\033[0m $*"; }
log_success() { echo -e "\033[0;32m[OK]\033[0m $*"; }
log_warning() { echo -e "\033[1;33m[WARN]\033[0m $*"; }
log_error() { echo -e "\033[0;31m[ERROR]\033[0m $*" >&2; }
require_docker() {
command -v docker &>/dev/null || {
log_error "Docker not found. Install it first:"
log_error " curl -fsSL https://get.docker.com | sudo sh"
return 1
}
docker compose version &>/dev/null || {
log_error "Docker Compose plugin missing:"
log_error " sudo apt-get install -y docker-compose-plugin"
return 1
}
}
ensure_docker_dir_ownership() {
chown -R "$ACTUAL_USER:$ACTUAL_USER" "$@" 2>/dev/null || true
}
port_in_use() {
local _port="$1" _proto="${2:-tcp}"
local _flag="-tlnH"
[ "$_proto" = "udp" ] && _flag="-ulnH"
ss "$_flag" "sport = :${_port}" 2>/dev/null | grep -q .
}
find_free_port() {
local _varname="$1" _port="$2" _proto="${3:-tcp}"
while port_in_use "$_port" "$_proto"; do
_port=$((_port + 1))
done
eval "$_varname='$_port'"
}
# Match common.sh's eval-based pattern so local vars in install_* are set correctly
prompt_text() {
local _q="$1" _def="$2" _var="$3" _r
[[ "${UNATTENDED:-false}" == "true" ]] && { eval "$_var='$_def'"; return; }
read -r -p " $_q " _r
eval "$_var='${_r:-$_def}'"
}
prompt_yn() {
local _q="$1" _def="$2" _var="$3" _r
[[ "${UNATTENDED:-false}" == "true" ]] && { eval "$_var='$_def'"; return; }
read -r -p " $_q " _r
eval "$_var='${_r:-$_def}'"
}
configure_caddy_for_service() {
local _name="$1" _upstream="$2" _subdomain="$3" _extra="${4:-}"
local _caddy_dir="$DOCKER_DIR/caddy"
local _caddyfile="$_caddy_dir/Caddyfile"
local _display_port="${_upstream##*:}"
# Determine mode: local Caddy, remote Caddy, or none
local _mode="none"
[[ -d "$_caddy_dir" ]] && _mode="local"
[[ -n "${CADDY_REMOTE_HOST:-}" ]] && [[ "$_mode" != "local" ]] && _mode="remote"
[[ "$_mode" == "none" ]] && {
log_info "Access $_name directly on port $_display_port."
return 0
}
echo ""
local _do_caddy=""
if [[ "$_mode" == "remote" ]]; then
log_info "Remote Caddy configured (${CADDY_REMOTE_HOST})."
log_info "A snippet file will be saved to ~/docker/caddy-snippets/."
fi
read -r -p " Configure Caddy reverse proxy for $_name? [y/N]: " _do_caddy
[[ "${_do_caddy,,}" == "y" ]] || {
log_info "Skipping — access at: http://localhost:$_display_port"
return 0
}
# Domain prompt — pre-fill from SITE_DOMAIN when available
local _default_domain=""
if [[ -n "${SITE_DOMAIN:-}" ]] && [[ "$SITE_DOMAIN" != "example.com" ]]; then
_default_domain="${_subdomain}.${SITE_DOMAIN}"
log_info "Default: $_default_domain"
fi
local _domain=""
read -r -p " Domain [${_default_domain:-required}]: " _domain
_domain="${_domain:-$_default_domain}"
[[ -n "$_domain" ]] || { log_warning "No domain entered — skipping Caddy."; return 0; }
# Build upstream — remote Caddy uses host IP:port, not container name
local _block_upstream="$_upstream"
if [[ "$_mode" == "remote" ]]; then
_block_upstream="${CADDY_REMOTE_HOST}:${_display_port}"
fi
local _site_block
_site_block="$(cat << CBLOCK
# $_name
${_domain} {
reverse_proxy ${_block_upstream}
header {
Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
X-Content-Type-Options "nosniff"
X-Frame-Options "SAMEORIGIN"
Referrer-Policy "strict-origin-when-cross-origin"
}
log {
output file /var/log/caddy/${_domain}.log
format json
}
${_extra}
}
CBLOCK
)"
if [[ "$_mode" == "local" ]]; then
if [[ -f "$_caddyfile" ]]; then
local _bk="$_caddy_dir/Caddyfile.backup.$(date +%Y%m%d-%H%M%S)"
cp "$_caddyfile" "$_bk"
log_info "Backed up Caddyfile to $(basename "$_bk")"
else
touch "$_caddyfile"
fi
if grep -q "^${_domain}" "$_caddyfile" 2>/dev/null; then
log_warning "$_domain already in Caddyfile"
local _ow=""
read -r -p " Overwrite? [y/N]: " _ow
[[ "${_ow,,}" == "y" ]] || { log_info "Keeping existing entry."; return 0; }
sed -i "/^${_domain}/,/^}/d" "$_caddyfile"
fi
printf '%s\n' "$_site_block" >> "$_caddyfile"
log_success "Added $_domain to Caddyfile"
docker exec caddy caddy fmt --overwrite /etc/caddy/Caddyfile 2>/dev/null || true
if docker exec caddy caddy reload --config /etc/caddy/Caddyfile 2>/dev/null; then
log_success "$_name accessible at: https://$_domain"
else
log_warning "Reload failed — check: docker logs caddy"
log_info "Manual reload: docker exec caddy caddy reload --config /etc/caddy/Caddyfile"
fi
else
local _snippet_dir="$DOCKER_DIR/caddy-snippets"
local _snippet_file="$_snippet_dir/${_subdomain}.caddy"
mkdir -p "$_snippet_dir"
printf '%s\n' "$_site_block" > "$_snippet_file"
chown "$ACTUAL_USER:$ACTUAL_USER" "$_snippet_file" 2>/dev/null || true
log_success "Snippet saved: $_snippet_file"
log_info "Copy to Caddy machine:"
log_info " scp $_snippet_file caddy-host:~/caddy-snippets/"
log_info " rsync -av $_snippet_dir/ caddy-host:~/caddy-snippets/ (all at once)"
fi
}
write_readme() {
local _dir="$1"; shift
mkdir -p "$_dir"
cat > "$_dir/README.md"
}
fi
# Globals — ACTUAL_USER/ACTUAL_HOME must come before DOCKER_DIR
# ($HOME under sudo is /root, not the real user's home)
ACTUAL_USER="${ACTUAL_USER:-${SUDO_USER:-$USER}}"
ACTUAL_HOME="$(getent passwd "$ACTUAL_USER" 2>/dev/null | cut -d: -f6 || echo "${HOME:-/root}")"
DOCKER_DIR="${DOCKER_DIR:-$ACTUAL_HOME/docker}"
DRY_RUN="${DRY_RUN:-false}"
UNATTENDED="${UNATTENDED:-false}"
SITE_TZ="${SITE_TZ:-$(cat /etc/timezone 2>/dev/null || echo UTC)}"
SITE_DOMAIN="${SITE_DOMAIN:-example.com}"
SITE_CADDY_NET="${SITE_CADDY_NET:-caddy_net}"
register_service() { :; } # no-op — no wizard to register into
_RUN_STANDALONE=1
fi
# ─────────────────────────────────────────────────────────────────────────────
register_service gatus utilities "Status & uptime monitoring page (Gatus)" 8086
# ── Caddyfile -> Gatus endpoint sync ────────────────────────────────────────
# Every auto-added endpoint is tagged group: "caddy-sync" — the sync script
# only ever adds or removes entries in that exact group, so anything you add
# by hand (the default external checks, a custom endpoint, different group
# name) is never touched regardless of what the Caddyfile looks like.
_gatus_write_sync_script() {
local gatus_dir="$1" caddy_file="$2"
cat > "$gatus_dir/sync-from-caddy.sh" << 'SYNC_SCRIPT'
#!/bin/bash
# Auto-generated by services/gatus.sh — do not edit by hand, re-run the
# installer to regenerate. Syncs Gatus endpoints with the current
# Caddyfile: adds a group: caddy-sync endpoint for every site block found,
# removes caddy-sync endpoints for sites no longer there. Never touches
# any other endpoint.
set -uo pipefail
CADDYFILE="$1"
GATUS_CONFIG="$2"
# Not just `command -v yq` — some distros package a completely different,
# incompatible yq (kislyuk's Python jq-wrapper) under the same name; check
# it actually identifies as mikefarah's before trusting its syntax.
yq --version 2>/dev/null | grep -q mikefarah || { echo "yq unavailable or incompatible — skipping sync"; exit 0; }
[ -f "$CADDYFILE" ] || { echo "No Caddyfile at $CADDYFILE — skipping sync"; exit 0; }
[ -f "$GATUS_CONFIG" ] || { echo "No Gatus config at $GATUS_CONFIG — skipping sync"; exit 0; }
# Site block headers at brace-depth 0, skipping the bare global-options
# block (no address before its "{") and parenthesized snippet definitions
# like (authelia) — same brace-tracking approach as the Caddy site-block
# removal in lib/common.sh (remove_service), for the same reason: a naive
# "next blank line" scan is exactly the class of bug that already bit this
# repo once on a live Caddy/Samba config.
CURRENT_DOMAINS="$(awk '
BEGIN { depth = 0 }
{
line = $0
opens = gsub(/\{/, "{", line)
closes = gsub(/\}/, "}", line)
if (depth == 0 && opens > 0) {
header = $0
sub(/[[:space:]]*\{.*$/, "", header)
gsub(/^[[:space:]]+|[[:space:]]+$/, "", header)
if (header != "" && header !~ /^\(/) print header
depth += opens - closes
next
}
if (depth > 0) { depth += opens - closes; next }
}
' "$CADDYFILE")"
EXISTING_SYNCED="$(yq e '.endpoints[] | select(.group == "caddy-sync") | .name' "$GATUS_CONFIG" 2>/dev/null)"
ADDED=0
REMOVED=0
while IFS= read -r domain; do
[ -z "$domain" ] && continue
# Domains only — a defensive filter, not strictly needed since these
# come from our own Caddyfile, but cheap insurance against ever
# building a yq expression out of anything unexpected.
[[ "$domain" =~ ^[a-zA-Z0-9.-]+$ ]] || continue
if ! grep -qxF "$domain" <<< "$EXISTING_SYNCED"; then
yq e -i ".endpoints += [{\"name\": \"${domain}\", \"group\": \"caddy-sync\", \"url\": \"https://${domain}\", \"interval\": \"5m\", \"conditions\": [\"[STATUS] < 400\"]}]" "$GATUS_CONFIG" \
&& ADDED=$((ADDED + 1))
fi
done <<< "$CURRENT_DOMAINS"
while IFS= read -r name; do
[ -z "$name" ] && continue
if ! grep -qxF "$name" <<< "$CURRENT_DOMAINS"; then
yq e -i "del(.endpoints[] | select(.group == \"caddy-sync\" and .name == \"${name}\"))" "$GATUS_CONFIG" \
&& REMOVED=$((REMOVED + 1))
fi
done <<< "$EXISTING_SYNCED"
echo "Gatus sync: added $ADDED, removed $REMOVED endpoint(s) from $(basename "$CADDYFILE")"
SYNC_SCRIPT
chmod 700 "$gatus_dir/sync-from-caddy.sh"
chown "$ACTUAL_USER:$ACTUAL_USER" "$gatus_dir/sync-from-caddy.sh" 2>/dev/null || true
}
_gatus_setup_caddy_sync() {
local gatus_dir="$1"
local caddy_file="$DOCKER_DIR/caddy/Caddyfile"
[ -f "$caddy_file" ] || return 0
echo ""
local ENABLE_SYNC=""
prompt_yn "Automatically keep Gatus endpoints in sync with the Caddyfile (syncs now, then every 15 minutes)? (y/n):" "y" ENABLE_SYNC
[[ "$ENABLE_SYNC" =~ ^[Yy]$ ]] || return 0
declare -F ensure_yq >/dev/null 2>&1 && ensure_yq
command -v yq >/dev/null 2>&1 || { log_warning "yq unavailable — skipping Caddy sync."; return 0; }
_gatus_write_sync_script "$gatus_dir" "$caddy_file"
"$gatus_dir/sync-from-caddy.sh" "$caddy_file" "$gatus_dir/gatus_config/config.yaml"
if command -v systemctl >/dev/null 2>&1; then
cat > /etc/systemd/system/gatus-caddy-sync.service << UNIT
[Unit]
Description=Sync Gatus endpoints from Caddyfile
[Service]
Type=oneshot
ExecStart=${gatus_dir}/sync-from-caddy.sh ${caddy_file} ${gatus_dir}/gatus_config/config.yaml
UNIT
cat > /etc/systemd/system/gatus-caddy-sync.timer << 'UNIT'
[Unit]
Description=Periodic Gatus/Caddyfile sync
[Timer]
OnBootSec=5min
OnUnitActiveSec=15min
[Install]
WantedBy=timers.target
UNIT
systemctl daemon-reload
if systemctl enable --now gatus-caddy-sync.timer >/dev/null 2>&1; then
log_success "Sync scheduled every 15 minutes (systemd timer: gatus-caddy-sync)"
else
log_warning "Couldn't enable the sync timer — re-run $gatus_dir/sync-from-caddy.sh manually after Caddyfile changes."
fi
else
log_warning "No systemd — synced once, but won't auto-repeat. Re-run $gatus_dir/sync-from-caddy.sh manually after Caddyfile changes."
fi
}
install_gatus() {
require_docker || return 1
log_info "Installing Gatus..."
local GATUS_DIR="$DOCKER_DIR/gatus"
local WEB_PORT="8086"
if [ "$DRY_RUN" = true ]; then
echo "[DRY-RUN] Would create $GATUS_DIR (gatus_config/, gatus_data/)"
echo "[DRY-RUN] Would deploy twinproduction/gatus:latest"
echo "[DRY-RUN] Port 8086 published (auto-scanned for a free host port),"
echo "[DRY-RUN] config at gatus_config/config.yaml"
echo "[DRY-RUN] Would offer to auto-sync endpoints from the Caddyfile (adds/removes group: caddy-sync entries, every 15min via a systemd timer)"
return 0
fi
# Scan for a free host port — a plain install shouldn't silently claim a
# port another already-running service holds. See CLAUDE.md's "Port
# collision avoidance" section.
find_free_port WEB_PORT "$WEB_PORT"
mkdir -p "$GATUS_DIR/gatus_config" "$GATUS_DIR/gatus_data"
ensure_docker_dir_ownership "$GATUS_DIR"
cd "$GATUS_DIR" || return 1
local TZ_VAL="${SITE_TZ:-$(cat /etc/timezone 2>/dev/null || echo UTC)}"
# Mirrors configure_caddy_for_service's own mode resolution (lib/common.sh):
# explicit CADDY_MODE from the site config wins, then a local ~/docker/caddy,
# then the legacy CADDY_REMOTE_HOST var. Only "local" joins caddy_net — a
# remote Caddy box can't resolve container names on this host's bridge
# network anyway; it reaches this service via the host's published port.
local _CADDY_MODE="${CADDY_MODE:-none}"
[ "$_CADDY_MODE" = "none" ] && [ -d "$DOCKER_DIR/caddy" ] && _CADDY_MODE="local"
[ "$_CADDY_MODE" = "none" ] && [ -n "${CADDY_REMOTE_HOST:-}" ] && _CADDY_MODE="remote"
local _CADDY_NET_BLOCK=""
local _CADDY_NET_SECTION=""
if [ "$_CADDY_MODE" = "local" ]; then
_CADDY_NET_BLOCK=" networks:
- caddy_net
"
_CADDY_NET_SECTION="
networks:
caddy_net:
external: true
name: ${SITE_CADDY_NET:-caddy_net}
"
fi
cat > docker-compose.yml << GATUS_COMPOSE
name: gatus
services:
gatus:
image: twinproduction/gatus:latest
container_name: gatus
hostname: gatus
restart: unless-stopped
env_file: .env
ports:
- "${WEB_PORT}:8080"
volumes:
- ./gatus_config:/config
- ./gatus_data:/data
${_CADDY_NET_BLOCK}${_CADDY_NET_SECTION}
GATUS_COMPOSE
cat > .env << GATUS_ENV
TZ=$TZ_VAL
CADDY_NET=$SITE_CADDY_NET
GATUS_ENV
# Write a sample config if none exists
if [ ! -f gatus_config/config.yaml ]; then
cat > gatus_config/config.yaml << 'GATUS_CFG'
# Gatus configuration — docs: https://github.com/TwiN/gatus
#
# Add or remove endpoints below. Config is hot-reloaded on changes.
# Alert types: ntfy, slack, discord, email, telegram, and more.
storage:
type: sqlite
path: /data/gatus.db
ui:
title: "Status"
header: "Services"
# ── Endpoints ─────────────────────────────────────────────────────────────────
endpoints:
- name: Google DNS
group: external
url: "8.8.8.8"
dns:
query-name: "google.com"
query-type: "A"
interval: 5m
conditions:
- "[DNS_RCODE] == NOERROR"
- name: Example HTTPS
group: external
url: "https://example.com"
interval: 5m
conditions:
- "[STATUS] == 200"
- "[RESPONSE_TIME] < 3000"
- "[CERTIFICATE_EXPIRATION] > 48h"
# ── Add your services below ────────────────────────────────────────────────
# - name: Mealie
# group: homelab
# url: "http://mealie:9000/api/app/about"
# interval: 1m
# conditions:
# - "[STATUS] == 200"
# - "[RESPONSE_TIME] < 500"
#
# - name: Portainer
# group: homelab
# url: "https://portainer:9443"
# interval: 1m
# conditions:
# - "[STATUS] == 200"
# client:
# insecure: true
GATUS_CFG
fi
chown -R "$ACTUAL_USER:$ACTUAL_USER" "$GATUS_DIR"
log_success "Gatus configured at $GATUS_DIR"
configure_caddy_for_service "Gatus" "gatus:8080" "status"
write_readme "$GATUS_DIR" << MD
# Gatus — status & uptime monitoring
Clean, self-hosted status page. Polls HTTP, TCP, DNS, and ICMP endpoints.
## Access
- URL: http://localhost:${WEB_PORT}
## Configuration
Edit \`gatus_config/config.yaml\` — changes are **hot-reloaded** without restarting.
Key concepts:
- \`endpoints:\` — what to check (HTTP, TCP, DNS, ICMP)
- \`interval:\` — how often (e.g. 1m, 5m)
- \`conditions:\` — pass/fail rules ([STATUS], [RESPONSE_TIME], etc.)
- \`alerts:\` — notify via ntfy, Slack, Discord, email, etc.
Full docs: https://github.com/TwiN/gatus
## Caddyfile sync
If you enabled it during install, \`sync-from-caddy.sh\` runs every 15
minutes (systemd timer: \`gatus-caddy-sync\`) and keeps one endpoint per
Caddy site block in sync automatically — added when a new site appears,
removed when one goes away. Every auto-added endpoint is tagged
\`group: caddy-sync\`; the sync only ever touches entries in that group,
so anything you add or edit by hand (a different group, the default
external checks) is left alone no matter what.
Run it by hand any time instead of waiting for the timer:
\`\`\`bash
$GATUS_DIR/sync-from-caddy.sh $DOCKER_DIR/caddy/Caddyfile $GATUS_DIR/gatus_config/config.yaml
\`\`\`
Check/manage the timer:
\`\`\`bash
systemctl status gatus-caddy-sync.timer
systemctl list-timers gatus-caddy-sync.timer
sudo systemctl disable --now gatus-caddy-sync.timer # turn it off
\`\`\`
## Manage
\`\`\`bash
cd $GATUS_DIR
docker compose up -d # start
docker compose down # stop
docker compose logs -f # logs (check config errors here)
docker compose pull && docker compose up -d # update
\`\`\`
MD
local START_GATUS=""
prompt_yn "Start Gatus now? (y/n):" "y" START_GATUS
if [ "$START_GATUS" = "y" ] || [ "$START_GATUS" = "Y" ]; then
docker compose up -d \
&& log_success "Gatus started" \
|| log_warning "Failed to start — check: docker compose logs"
fi
_gatus_setup_caddy_sync "$GATUS_DIR"
echo " Access at: http://localhost:${WEB_PORT}"
echo " Config: $GATUS_DIR/gatus_config/config.yaml (hot-reloaded)"
echo ""
}
# Run immediately when executed directly (deferred until after function definition)
[[ "${_RUN_STANDALONE:-0}" == 1 ]] && install_gatus