1 Commits
15 changed files with 143 additions and 6660 deletions
+1 -20
View File
@@ -623,25 +623,6 @@ in `services/authelia.sh`) — prompts for a new duration (`12h`, `7d`,
Sessions persist through reboots regardless of duration (Redis stores
session state in a volume).
**`inactivity` must track `remember_me`, or a long remember_me is a lie.**
`inactivity` is a separate session field — how long a session can sit idle
before Authelia ends it — and it is NOT extended or bypassed by the
"Remember me" checkbox; the two are independent. Confirmed live: a user
set `remember_me: 1y` expecting "won't be asked to log in again for a
year," but the install default left `inactivity` at a much shorter value
(2h at the time), so ordinary daily gaps between visits (overnight, a
workday) ended the session on inactivity grounds well before remember_me
ever came into play — the 1y setting was doing nothing. Fixed at both ends
so this can't recur silently: `install_authelia()`'s own template now sets
`inactivity: 7d`, matching its `remember_me: 7d` default instead of a
shorter one, and `_authelia_set_remember_me()` now writes the SAME new
duration into both keys on every change, not just `remember_me` alone. If
you ever hand-edit `session:` instead of using the menu option, keep
`inactivity` and `remember_me` equal — a mismatch here is exactly the bug
above, not a valid intentional configuration. `expiration` (the cap for a
session that never checked "Remember me") is a legitimately different,
shorter-by-design setting and is untouched by any of this.
**The config key is `remember_me`, not `remember_me_duration`.** Authelia
renamed it in 4.38; this repo pins `4.39.20`. A stale `remember_me_duration`
key doesn't error, Authelia just silently ignores it — confirmed against
@@ -654,7 +635,7 @@ touch this by hand instead of the menu option, the current schema is:
session:
secret: 'your-existing-secret'
expiration: 1h
inactivity: 1y
inactivity: 5m
remember_me: 1y
cookies:
- domain: 'example.com'
+1 -23
View File
@@ -186,7 +186,7 @@ a ready-to-copy Caddy config snippet to `~/docker/caddy-snippets/`.
|-------|---------|
| `base` | `net-tools`, `ncdu`, `git`, `curl`, `wget`, `htop`, `tree`, `zip`/`unzip`, `ca-certificates`, `gnupg`, `jq`, `rsync`; `glow` (terminal markdown reader, Charm apt repo); Docker CE + Compose plugin; `openssh-server` with GitHub/Launchpad SSH key import, optional password-auth lockdown, and SSH Host aliases; optional NetBird overlay network |
| `homelab` | `caddy`, `crowdsec`, `authelia`, `homeassistant`, `asterisk` (own dedicated coturn for TURN/STUN — see `mattermost` below for the other coturn-owning service), `pstn-trunk`, `sms-inbound`, `security-dashboard`, `sunshine`, `vpn-data-mount` (mount existing SMB shares from a NetBird-connected home box — SSH trust bootstrap, then read-only discovery of shares already configured there; never writes to the home box's Samba config; repeatable, pick from any number of a home box's shares in one pass; optional per-share [gocryptfs decrypt layer](#client-side-encryption-for-vpn-data-mount) so the VPS only ever handles ciphertext) |
| `utilities` | `actualbudget`, `ai-gpu`, `ai-stack`, `anki-progress` (read-only study-progress dashboard for an `anki-sync-server` instance — reviews/accuracy/streak per account, plus an ntfy notification once a study session has been going for a configurable number of minutes; reads collection files with SQLite's read-only mode so it can't interfere with the live sync server), `anki-sync-server` (self-hosted sync backend for the Anki flashcard app — spaced-repetition scheduling stays in the Anki client, this just syncs collections across devices without AnkiWeb; supports multiple independent accounts per instance), `archivebox`, `beszel` (lightweight server + Docker monitoring — CPU/RAM/disk/network, auto-discovers running containers via the Docker socket; complements Gatus rather than replacing it — Gatus is a black-box HTTP check, Beszel is white-box host/process monitoring), `beszel-agent` (agent-only Beszel install for a remote/homelab box reporting to a hub elsewhere — connects outbound over HTTPS, no VPN/port-forwarding/FQDN needed on that box), `changedetection`, `ddclient`, `filebrowser`, `fmd`, `garage` (self-hosted S3-compatible object storage, single node — MinIO CE's actively-maintained replacement), `garage-webui` (browser-based bucket/object browser for an existing `garage` install — folders/files view, the same kind of thing Backblaze's own web console gives you), `gatus`, `gitea` (self-hosted Git server — raw local clones plus optional two-way GitHub mirror sync, standalone from the `ai-stack` bundle's own Gitea container), `homebox`, `iopaint`, `joplin`, `koha`, `magicmirror`, `mail-archiver`, `mattermost`, `mealie`, `meshcentral`, `n8n`, `nextcloud`, `ntfy`, `onlyoffice`, `paintplus`, `pihole` (standalone DNS ad/tracker blocking — not wired into any VPN's DNS push), `portainer`, `pressbooks` (self-hosted book platform — WordPress Multisite, drag-and-drop chapter editing, PDF export via PrinceXML/DocRaptor, Authelia-gated), `rustdesk`, `samba` (SMB/CIFS file sharing — shares, dedicated Samba users/passwords, LAN-scoped firewall by default; also offered as an optional nudge from `base`), `stirling-pdf`, `syncthing`, `traccar`, `unifi`, `uptimekuma`, `vaultwarden`, `watchyourlan`, `watchtower`, `wg-easy`, `wordpress` (multi-site, dedicated MariaDB per site — blogs, business sites, e-commerce via WooCommerce) |
| `utilities` | `actualbudget`, `ai-gpu`, `ai-stack`, `archivebox`, `beszel` (lightweight server + Docker monitoring — CPU/RAM/disk/network, auto-discovers running containers via the Docker socket; complements Gatus rather than replacing it — Gatus is a black-box HTTP check, Beszel is white-box host/process monitoring), `beszel-agent` (agent-only Beszel install for a remote/homelab box reporting to a hub elsewhere — connects outbound over HTTPS, no VPN/port-forwarding/FQDN needed on that box), `changedetection`, `ddclient`, `filebrowser`, `fmd`, `garage` (self-hosted S3-compatible object storage, single node — MinIO CE's actively-maintained replacement), `garage-webui` (browser-based bucket/object browser for an existing `garage` install — folders/files view, the same kind of thing Backblaze's own web console gives you), `gatus`, `gitea` (self-hosted Git server — raw local clones plus optional two-way GitHub mirror sync, standalone from the `ai-stack` bundle's own Gitea container), `homebox`, `iopaint`, `joplin`, `koha`, `magicmirror`, `mail-archiver`, `mattermost`, `mealie`, `meshcentral`, `n8n`, `nextcloud`, `ntfy`, `onlyoffice`, `paintplus`, `pihole` (standalone DNS ad/tracker blocking — not wired into any VPN's DNS push), `portainer`, `pressbooks` (self-hosted book platform — WordPress Multisite, drag-and-drop chapter editing, PDF export via PrinceXML/DocRaptor, Authelia-gated), `rustdesk`, `stirling-pdf`, `syncthing`, `traccar`, `unifi`, `uptimekuma`, `vaultwarden`, `watchyourlan`, `watchtower`, `wg-easy`, `wordpress` (multi-site, dedicated MariaDB per site — blogs, business sites, e-commerce via WooCommerce) |
| `media` | `arm`, `audiobookshelf`, `calibre-web`, `emby`, `immich`, `jellyfin`, `lyrion` |
| `cameras` | `frigate`, `frigate-audio`, `frigate-notify`, `sky-cam` |
| `gaming` | `drum-rhythm-game`, `js99er`, `kyber-launcher`, `kyber-server`, `minecraft`, `wolf`, `wolf-pair` |
@@ -293,28 +293,6 @@ backup
</details>
## Generating Anki decks (tools/anki-deck-*.py)
`anki-sync-server` gives you a self-hosted sync backend, but a fresh
account has no content — `tools/anki-deck-math.py`,
`tools/anki-deck-periodic.py`, and `tools/anki-deck-visual.py` generate
ready-to-import `.apkg` decks (multiplication/division/addition/
subtraction/fractions/decimals, the periodic table, and shapes/clocks/
coin-counting) with Anki's built-in type-the-answer input and offline
neural TTS audio (Piper) on every card. All three are standalone Python
scripts, unrelated to the `services/*.sh` installer framework — run them
on any machine with Python, not necessarily the server itself. Full setup
(a venv, `genanki` + `piper-tts`, downloading a voice) and every deck's
exact usage is documented in `tools/anki-deck-math.py`'s own header
docstring; the other two scripts point back to it rather than repeating
the same instructions three times.
Shapes, clocks, and coin images are drawn programmatically (SVG) rather
than AI-generated — image generation is a poor fit for content that has
to be exactly correct (an exact clock time, an exact side count), not
just plausible-looking; see `tools/anki-deck-visual.py`'s own docstring
for more on that tradeoff.
## Layout
```
-741
View File
@@ -1,741 +0,0 @@
#!/bin/bash
# services/anki-progress.sh — Anki study-progress dashboard + ntfy
# "started studying" notifications. Reads an anki-sync-server instance's
# data directly (read-only) — see services/anki-sync-server.sh, which this
# service requires.
# Part of the modular post-install system (sourced by setup.sh).
#
# Can also be run standalone on any machine:
# sudo bash anki-progress.sh
# (Docker must already be installed, and an anki-sync-server instance must
# already exist on the same box, when run standalone)
# ── Standalone bootstrap ──────────────────────────────────────────────────────
if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
[[ "$(id -u)" == "0" ]] || { echo "Run with sudo: sudo bash $0"; exit 1; }
_SELF_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
_COMMON="$_SELF_DIR/../lib/common.sh"
if [[ -f "$_COMMON" ]]; then
# shellcheck source=../lib/common.sh
source "$_COMMON"
else
log_info() { echo -e "\033[0;34m[INFO]\033[0m $*"; }
log_success() { echo -e "\033[0;32m[OK]\033[0m $*"; }
log_warning() { echo -e "\033[1;33m[WARN]\033[0m $*"; }
log_error() { echo -e "\033[0;31m[ERROR]\033[0m $*" >&2; }
require_docker() {
command -v docker &>/dev/null || {
log_error "Docker not found. Install it first:"
log_error " curl -fsSL https://get.docker.com | sudo sh"
return 1
}
docker compose version &>/dev/null || {
log_error "Docker Compose plugin missing:"
log_error " sudo apt-get install -y docker-compose-plugin"
return 1
}
}
ensure_docker_dir_ownership() {
chown -R "$ACTUAL_USER:$ACTUAL_USER" "$@" 2>/dev/null || true
}
port_in_use() {
local _port="$1" _proto="${2:-tcp}"
local _flag="-tlnH"
[ "$_proto" = "udp" ] && _flag="-ulnH"
ss "$_flag" "sport = :${_port}" 2>/dev/null | grep -q .
}
find_free_port() {
local _varname="$1" _port="$2" _proto="${3:-tcp}"
while port_in_use "$_port" "$_proto"; do
_port=$((_port + 1))
done
eval "$_varname='$_port'"
}
prompt_text() {
local _q="$1" _def="$2" _var="$3" _r
[[ "${UNATTENDED:-false}" == "true" ]] && { eval "$_var='$_def'"; return; }
read -r -p " $_q " _r
eval "$_var='${_r:-$_def}'"
}
prompt_yn() {
local _q="$1" _def="$2" _var="$3" _r
[[ "${UNATTENDED:-false}" == "true" ]] && { eval "$_var='$_def'"; return; }
read -r -p " $_q " _r
eval "$_var='${_r:-$_def}'"
}
prompt_reinstall_mode() {
local _var="$1" _r
if [[ "${UNATTENDED:-false}" == "true" ]]; then eval "$_var='cancel'"; return; fi
echo " Already installed."
read -r -p " (u)pdate / (f)resh reinstall / (c)ancel [c]: " _r
case "${_r,,}" in
u|update) eval "$_var='update'" ;;
f|fresh) eval "$_var='fresh'" ;;
*) eval "$_var='cancel'" ;;
esac
}
configure_caddy_for_service() {
local _name="$1" _upstream="$2" _subdomain="$3" _extra="${4:-}"
local _caddy_dir="$DOCKER_DIR/caddy"
local _caddyfile="$_caddy_dir/Caddyfile"
local _display_port="${_upstream##*:}"
local _mode="none"
[[ -d "$_caddy_dir" ]] && _mode="local"
[[ -n "${CADDY_REMOTE_HOST:-}" ]] && [[ "$_mode" != "local" ]] && _mode="remote"
[[ "$_mode" == "none" ]] && {
log_info "Access $_name directly on port $_display_port."
return 0
}
echo ""
local _do_caddy=""
if [[ "$_mode" == "remote" ]]; then
log_info "Remote Caddy configured (${CADDY_REMOTE_HOST})."
log_info "A snippet file will be saved to ~/docker/caddy-snippets/."
fi
read -r -p " Configure Caddy reverse proxy for $_name? [y/N]: " _do_caddy
[[ "${_do_caddy,,}" == "y" ]] || {
log_info "Skipping — access at: http://localhost:$_display_port"
return 0
}
local _default_domain=""
if [[ -n "${SITE_DOMAIN:-}" ]] && [[ "$SITE_DOMAIN" != "example.com" ]]; then
_default_domain="${_subdomain}.${SITE_DOMAIN}"
log_info "Default: $_default_domain"
fi
local _domain=""
read -r -p " Domain [${_default_domain:-required}]: " _domain
_domain="${_domain:-$_default_domain}"
[[ -n "$_domain" ]] || { log_warning "No domain entered — skipping Caddy."; return 0; }
local _block_upstream="$_upstream"
if [[ "$_mode" == "remote" ]]; then
_block_upstream="${CADDY_REMOTE_HOST}:${_display_port}"
fi
local _site_block
_site_block="$(cat << CBLOCK
# $_name
${_domain} {
reverse_proxy ${_block_upstream}
header {
Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
X-Content-Type-Options "nosniff"
X-Frame-Options "SAMEORIGIN"
Referrer-Policy "strict-origin-when-cross-origin"
}
log {
output file /var/log/caddy/${_domain}.log
format json
}
${_extra}
}
CBLOCK
)"
if [[ "$_mode" == "local" ]]; then
if [[ -f "$_caddyfile" ]]; then
local _bk="$_caddy_dir/Caddyfile.backup.$(date +%Y%m%d-%H%M%S)"
cp "$_caddyfile" "$_bk"
log_info "Backed up Caddyfile to $(basename "$_bk")"
else
touch "$_caddyfile"
fi
if grep -q "^${_domain}" "$_caddyfile" 2>/dev/null; then
log_warning "$_domain already in Caddyfile"
local _ow=""
read -r -p " Overwrite? [y/N]: " _ow
[[ "${_ow,,}" == "y" ]] || { log_info "Keeping existing entry."; return 0; }
sed -i "/^${_domain}/,/^}/d" "$_caddyfile"
fi
printf '%s\n' "$_site_block" >> "$_caddyfile"
log_success "Added $_domain to Caddyfile"
docker exec caddy caddy fmt --overwrite /etc/caddy/Caddyfile 2>/dev/null || true
if docker exec caddy caddy reload --config /etc/caddy/Caddyfile 2>/dev/null; then
log_success "$_name accessible at: https://$_domain"
else
log_warning "Reload failed — check: docker logs caddy"
fi
else
local _snippet_dir="$DOCKER_DIR/caddy-snippets"
local _snippet_file="$_snippet_dir/${_subdomain}.caddy"
mkdir -p "$_snippet_dir"
printf '%s\n' "$_site_block" > "$_snippet_file"
chown "$ACTUAL_USER:$ACTUAL_USER" "$_snippet_file" 2>/dev/null || true
log_success "Snippet saved: $_snippet_file"
fi
}
write_readme() {
local _dir="$1"; shift
mkdir -p "$_dir"
cat > "$_dir/README.md"
}
backup_if_exists() {
local _file="$1"
[ -f "$_file" ] || return 0
cp -p "$_file" "${_file}.bak.$(date +%Y%m%d-%H%M%S)" 2>/dev/null
}
fi
ACTUAL_USER="${ACTUAL_USER:-${SUDO_USER:-$USER}}"
ACTUAL_HOME="$(getent passwd "$ACTUAL_USER" 2>/dev/null | cut -d: -f6 || echo "${HOME:-/root}")"
DOCKER_DIR="${DOCKER_DIR:-$ACTUAL_HOME/docker}"
DRY_RUN="${DRY_RUN:-false}"
UNATTENDED="${UNATTENDED:-false}"
SITE_TZ="${SITE_TZ:-$(cat /etc/timezone 2>/dev/null || echo UTC)}"
SITE_DOMAIN="${SITE_DOMAIN:-example.com}"
SITE_CADDY_NET="${SITE_CADDY_NET:-caddy_net}"
register_service() { :; }
_RUN_STANDALONE=1
fi
# ─────────────────────────────────────────────────────────────────────────────
register_service anki-progress utilities "Anki study-progress dashboard + ntfy 'started studying' notifications (reads an anki-sync-server instance's data read-only)" 8099
install_anki-progress() {
require_docker || return 1
log_info "Installing Anki Progress Dashboard..."
# ── Dependency: needs an anki-sync-server instance already installed ────
# Meaningless on its own — see CLAUDE.md's "Chaining into another
# service" section. Only chains one direction: anki-progress requires
# anki-sync-server, never the reverse.
local _sync_dirs=()
local _d
for _d in "$DOCKER_DIR"/anki-sync-server*; do
[ -d "$_d" ] && _sync_dirs+=("$(basename "$_d")")
done
if [ "${#_sync_dirs[@]}" -eq 0 ]; then
log_error "No anki-sync-server install found — this dashboard reads its data directly."
log_error "Install it first: sudo ./setup.sh anki-sync-server"
return 1
fi
local SYNC_INSTANCE="${_sync_dirs[0]}"
if [ "${#_sync_dirs[@]}" -gt 1 ] && [ "$UNATTENDED" != true ]; then
echo ""
echo " Multiple anki-sync-server instances found:"
local i
for i in "${!_sync_dirs[@]}"; do
echo " $((i + 1))) ${_sync_dirs[$i]}"
done
local _choice=""
prompt_text " Which one should this dashboard monitor? [1]:" "1" _choice
if [[ "$_choice" =~ ^[0-9]+$ ]] && [ "$_choice" -ge 1 ] && [ "$_choice" -le "${#_sync_dirs[@]}" ]; then
SYNC_INSTANCE="${_sync_dirs[$((_choice - 1))]}"
fi
fi
local SYNC_DATA_DIR="$DOCKER_DIR/$SYNC_INSTANCE/data"
# ── Instance selection (of this dashboard itself) ───────────────────────
# A second instance is a real use case (e.g. a second household with its
# own anki-sync-server and its own dashboard) — same multi-instance
# pattern as every other service here (see CLAUDE.md).
local AP_DIR="$DOCKER_DIR/anki-progress"
local INSTANCE_SUFFIX="" CONTAINER="anki-progress"
local WEB_PORT="8099"
if [ "$DRY_RUN" = true ]; then
echo "[DRY-RUN] Would verify an anki-sync-server instance exists ($SYNC_INSTANCE found)"
echo "[DRY-RUN] Would offer to add a new, separate instance if one already exists"
echo "[DRY-RUN] Would create $AP_DIR(-<name>) with app.py, Dockerfile, docker-compose.yml"
echo "[DRY-RUN] Would prompt for ntfy URL/topic and notification timing"
echo "[DRY-RUN] Would auto-scan for a free host port"
return 0
fi
if [ -d "$AP_DIR" ]; then
echo ""
echo " Anki Progress Dashboard is already installed at $AP_DIR."
echo " 1) Manage that install (update / full reinstall / cancel)"
echo " 2) Add a NEW, separate dashboard instance alongside it"
echo ""
local _TOP_CHOICE=""
prompt_text " Choice [1/2]:" "1" _TOP_CHOICE
if [ "$_TOP_CHOICE" = "2" ]; then
local _suffix=""
while true; do
prompt_text " Short name for the new instance (letters/numbers/hyphens):" "" _suffix
_suffix="$(echo "$_suffix" | tr -cs 'a-zA-Z0-9-' '-' | sed 's/^-*//;s/-*$//')"
if [ -z "$_suffix" ]; then
log_warning "Name can't be empty."; continue
fi
if [ -d "$DOCKER_DIR/anki-progress-$_suffix" ]; then
log_warning "anki-progress-$_suffix already exists — pick another name."; continue
fi
break
done
INSTANCE_SUFFIX="$_suffix"
AP_DIR="$DOCKER_DIR/anki-progress-$_suffix"
CONTAINER="anki-progress-$_suffix"
log_info "New instance: $AP_DIR"
else
if [[ -f "$AP_DIR/docker-compose.yml" ]]; then
local MODE=""
prompt_reinstall_mode MODE
case "$MODE" in
update)
log_info "Refreshing app code + rebuilding the image — ntfy config and Caddy setup are left as-is."
( cd "$AP_DIR" && docker compose up -d --build ) \
&& log_success "Anki Progress Dashboard refreshed" \
|| log_warning "Refresh failed — check: docker compose -f $AP_DIR/docker-compose.yml logs"
return 0
;;
cancel)
log_info "Leaving the existing install as-is."
return 0
;;
fresh) ;;
esac
fi
fi
fi
find_free_port WEB_PORT "$WEB_PORT"
# ── ntfy ──────────────────────────────────────────────────────────────
# If ntfy is installed locally, reach it directly over caddy_net by
# container name — avoids a round trip through the public internet for
# a purely internal notification. Otherwise ask for a full URL (a
# remote/self-hosted instance elsewhere, or public ntfy.sh).
local NTFY_URL="" NTFY_TOPIC=""
if [ -d "$DOCKER_DIR/ntfy" ]; then
log_info "Local ntfy install detected — reaching it directly over caddy_net."
NTFY_URL="http://ntfy:80"
else
prompt_text " ntfy server URL (e.g. https://ntfy.yourdomain.com, or https://ntfy.sh):" "https://ntfy.sh" NTFY_URL
fi
prompt_text " ntfy topic to publish 'started studying' notifications to:" "anki-progress" NTFY_TOPIC
local SESSION_GAP_MINUTES="" NOTIFY_DELAY_MINUTES=""
prompt_text " Minutes of inactivity that counts as a new study session starting:" "30" SESSION_GAP_MINUTES
prompt_text " Minutes after a session starts to send the notification:" "10" NOTIFY_DELAY_MINUTES
mkdir -p "$AP_DIR/state"
ensure_docker_dir_ownership "$AP_DIR"
cd "$AP_DIR" || return 1
# Mirrors configure_caddy_for_service's own mode resolution — only
# "local" joins caddy_net.
local _CADDY_MODE="${CADDY_MODE:-none}"
[ "$_CADDY_MODE" = "none" ] && [ -d "$DOCKER_DIR/caddy" ] && _CADDY_MODE="local"
[ "$_CADDY_MODE" = "none" ] && [ -n "${CADDY_REMOTE_HOST:-}" ] && _CADDY_MODE="remote"
local _CADDY_NET_BLOCK=""
local _CADDY_NET_SECTION=""
if [ "$_CADDY_MODE" = "local" ]; then
_CADDY_NET_BLOCK=" networks:
- caddy_net
"
_CADDY_NET_SECTION="
networks:
caddy_net:
external: true
name: ${SITE_CADDY_NET:-caddy_net}
"
fi
backup_if_exists app.py
cat > app.py << 'PYEOF'
#!/usr/bin/env python3
"""Anki study-progress dashboard + ntfy "started studying" notifications.
Reads every account's collection.anki2 directly (READ-ONLY — never opens for
write, so it can't corrupt live data the sync server or a client is using)
from the anki-sync-server's data directory, and:
1. Serves a small web dashboard (reviews today/week, accuracy, streak,
last active) per account.
2. Runs a background loop that detects when a new study session starts
(first review after a gap of SESSION_GAP_MINUTES with no reviews) and
sends one ntfy notification NOTIFY_DELAY_MINUTES after that session
started, if the session is still going (i.e. more reviews happened
after the initial one) — not on every single review.
All configuration (NTFY_URL, NTFY_TOPIC, SESSION_GAP_MINUTES,
NOTIFY_DELAY_MINUTES, ANKI_DATA_DIR, STATE_FILE) comes from environment
variables, set in docker-compose.yml / .env by the installer — nothing to
hand-edit in this file.
"""
import glob
import json
import os
import sqlite3
import threading
import time
from datetime import datetime, timezone
import requests
from flask import Flask, render_template_string
NTFY_URL = os.environ.get("NTFY_URL", "https://ntfy.example.com")
NTFY_TOPIC = os.environ.get("NTFY_TOPIC", "anki-progress")
SESSION_GAP_MINUTES = int(os.environ.get("SESSION_GAP_MINUTES", 30))
NOTIFY_DELAY_MINUTES = int(os.environ.get("NOTIFY_DELAY_MINUTES", 10))
POLL_INTERVAL_SECONDS = 60
ANKI_DATA_DIR = os.environ.get("ANKI_DATA_DIR", "/anki-data")
STATE_FILE = os.environ.get("STATE_FILE", "/app/state/notify_state.json")
app = Flask(__name__)
def find_collections():
"""{username: path-to-collection-file} for every account directory found.
Globs for *.anki2 rather than assuming the exact filename, since that's
an implementation detail of the sync server we shouldn't hardcode."""
result = {}
if not os.path.isdir(ANKI_DATA_DIR):
return result
for entry in sorted(os.listdir(ANKI_DATA_DIR)):
user_dir = os.path.join(ANKI_DATA_DIR, entry)
if not os.path.isdir(user_dir):
continue
matches = glob.glob(os.path.join(user_dir, "*.anki2"))
if matches:
result[entry] = matches[0]
return result
def read_revlog_ids_eases(path):
"""Returns a list of (epoch_ms, ease) tuples sorted by time, read-only.
Opening with mode=ro is what makes this safe to run alongside a live
sync server — it never takes a write lock, so it can't corrupt or
block the account that's actually in use."""
uri = f"file:{path}?mode=ro"
con = sqlite3.connect(uri, uri=True)
try:
rows = con.execute("SELECT id, ease FROM revlog ORDER BY id ASC").fetchall()
except sqlite3.OperationalError:
rows = []
finally:
con.close()
return rows
def compute_stats(revlog_rows, now_ms):
"""Pure function over a list of (epoch_ms, ease) — kept separate from
any file/DB access so it can be unit-tested with synthetic data."""
if not revlog_rows:
return {
"total_reviews": 0, "reviews_today": 0, "reviews_week": 0,
"accuracy_pct": None, "streak_days": 0, "last_active": None,
}
day_ms = 24 * 60 * 60 * 1000
today_day = now_ms // day_ms
today_start = today_day * day_ms
week_start = today_start - 6 * day_ms
reviews_today = sum(1 for ts, _ in revlog_rows if ts >= today_start)
reviews_week = sum(1 for ts, _ in revlog_rows if ts >= week_start)
total = len(revlog_rows)
correct = sum(1 for _, ease in revlog_rows if ease != 1) # ease 1 = "Again" = a miss
accuracy_pct = round(100 * correct / total, 1) if total else None
# Streak: consecutive calendar days with >=1 review, walking backward
# from today. Still "alive" through yesterday if today has no reviews
# yet (so it doesn't reset to 0 first thing each morning) — but not if
# the most recent review is 2+ days old. review_days is unique/sorted
# descending, so any day that isn't exactly "expected" means a gap.
review_days = sorted({ts // day_ms for ts, _ in revlog_rows}, reverse=True)
streak = 0
if review_days and review_days[0] in (today_day, today_day - 1):
expected = review_days[0]
for d in review_days:
if d == expected:
streak += 1
expected -= 1
else:
break
last_active = max(ts for ts, _ in revlog_rows)
return {
"total_reviews": total,
"reviews_today": reviews_today,
"reviews_week": reviews_week,
"accuracy_pct": accuracy_pct,
"streak_days": streak,
"last_active": last_active,
}
def detect_current_session_start(revlog_rows, now_ms):
"""Walk backwards from the most recent review; the session start is the
earliest review such that every gap between consecutive reviews from
there to now is < SESSION_GAP_MINUTES. Returns None if the most recent
review itself is older than the gap threshold (no session "in progress")."""
if not revlog_rows:
return None
gap_ms = SESSION_GAP_MINUTES * 60 * 1000
last_ts = revlog_rows[-1][0]
if now_ms - last_ts > gap_ms:
return None # most recent review is old news, not an active session
session_start = last_ts
for ts, _ in reversed(revlog_rows[:-1]):
if session_start - ts > gap_ms:
break
session_start = ts
return session_start
DASHBOARD_TEMPLATE = """
<!doctype html>
<title>Anki Progress</title>
<meta http-equiv="refresh" content="60">
<style>
body { font-family: Arial, sans-serif; background: #f4f6f8; margin: 0; padding: 24px; }
h1 { color: #333; }
.grid { display: grid; grid-template-columns: repeat(auto-fill, minmax(260px, 1fr)); gap: 16px; }
.card { background: white; border-radius: 10px; padding: 18px 20px; box-shadow: 0 1px 4px rgba(0,0,0,0.1); }
.card h2 { margin: 0 0 10px 0; font-size: 20px; }
.stat { display: flex; justify-content: space-between; margin: 4px 0; font-size: 15px; }
.stat b { color: #1c4587; }
.empty { color: #888; font-style: italic; }
</style>
<h1>Anki Progress</h1>
<div class="grid">
{% for user, s in stats.items() %}
<div class="card">
<h2>{{ user }}</h2>
{% if s.total_reviews == 0 %}
<div class="empty">No reviews yet</div>
{% else %}
<div class="stat"><span>Reviews today</span><b>{{ s.reviews_today }}</b></div>
<div class="stat"><span>Reviews this week</span><b>{{ s.reviews_week }}</b></div>
<div class="stat"><span>Accuracy</span><b>{{ s.accuracy_pct }}%</b></div>
<div class="stat"><span>Streak</span><b>{{ s.streak_days }} day{{ 's' if s.streak_days != 1 else '' }}</b></div>
<div class="stat"><span>Last active</span><b>{{ s.last_active_str }}</b></div>
{% endif %}
</div>
{% endfor %}
</div>
"""
@app.route("/")
def dashboard():
now_ms = int(time.time() * 1000)
stats = {}
for user, path in find_collections().items():
rows = read_revlog_ids_eases(path)
s = compute_stats(rows, now_ms)
if s["last_active"]:
s["last_active_str"] = datetime.fromtimestamp(
s["last_active"] / 1000, tz=timezone.utc
).astimezone().strftime("%b %-d, %-I:%M %p")
else:
s["last_active_str"] = "—"
stats[user] = s
return render_template_string(DASHBOARD_TEMPLATE, stats=stats)
def load_notify_state():
if os.path.isfile(STATE_FILE):
with open(STATE_FILE) as f:
return json.load(f)
return {}
def save_notify_state(state):
os.makedirs(os.path.dirname(STATE_FILE), exist_ok=True)
with open(STATE_FILE, "w") as f:
json.dump(state, f)
def send_ntfy(message):
try:
requests.post(f"{NTFY_URL.rstrip('/')}/{NTFY_TOPIC}",
data=message.encode("utf-8"), timeout=10)
except requests.RequestException as e:
print(f"[ntfy] failed to send: {e}")
def notifier_loop():
state = load_notify_state()
while True:
now_ms = int(time.time() * 1000)
for user, path in find_collections().items():
rows = read_revlog_ids_eases(path)
session_start = detect_current_session_start(rows, now_ms)
entry = state.get(user, {})
if session_start is None:
# No active session right now — clear tracking so the next
# real session starts fresh.
if entry:
state[user] = {}
continue
if entry.get("session_start") != session_start:
# A new session started (different from whatever we were
# tracking) — start the countdown over.
state[user] = {"session_start": session_start, "notified": False}
entry = state[user]
elapsed_minutes = (now_ms - session_start) / 60000
if not entry.get("notified") and elapsed_minutes >= NOTIFY_DELAY_MINUTES:
send_ntfy(f"{user} started studying {NOTIFY_DELAY_MINUTES} minutes ago and is still going.")
entry["notified"] = True
save_notify_state(state)
time.sleep(POLL_INTERVAL_SECONDS)
if __name__ == "__main__":
threading.Thread(target=notifier_loop, daemon=True).start()
app.run(host="0.0.0.0", port=5000)
PYEOF
backup_if_exists Dockerfile
cat > Dockerfile << 'DOCKEREOF'
FROM python:3.12-slim
WORKDIR /app
RUN pip install --no-cache-dir flask requests
COPY app.py .
CMD ["python3", "app.py"]
DOCKEREOF
backup_if_exists docker-compose.yml
cat > docker-compose.yml << COMPOSEEOF
name: $CONTAINER
services:
$CONTAINER:
build: .
container_name: $CONTAINER
hostname: $CONTAINER
restart: unless-stopped
env_file: .env
environment:
- ANKI_DATA_DIR=/anki-data
- STATE_FILE=/app/state/notify_state.json
volumes:
# Read-only — this container only ever reads collection files (see
# app.py's read_revlog_ids_eases, which opens SQLite in mode=ro),
# never writes, so it can't corrupt live data the sync server or a
# client is using.
- $SYNC_DATA_DIR:/anki-data:ro
- ./state:/app/state
ports:
- "${WEB_PORT}:5000"
${_CADDY_NET_BLOCK}${_CADDY_NET_SECTION}
COMPOSEEOF
backup_if_exists .env
cat > .env << ENVEOF
NTFY_URL=$NTFY_URL
NTFY_TOPIC=$NTFY_TOPIC
SESSION_GAP_MINUTES=$SESSION_GAP_MINUTES
NOTIFY_DELAY_MINUTES=$NOTIFY_DELAY_MINUTES
ENVEOF
chmod 600 .env
chown -R "$ACTUAL_USER:$ACTUAL_USER" "$AP_DIR"
echo ""
log_success "Anki Progress Dashboard${INSTANCE_SUFFIX:+ ($INSTANCE_SUFFIX)} configured at $AP_DIR (port $WEB_PORT)"
log_info "Monitoring: $SYNC_INSTANCE"
local START=""
prompt_yn "Start Anki Progress Dashboard${INSTANCE_SUFFIX:+ ($INSTANCE_SUFFIX)} now? (y/n):" "y" START
if [ "$START" = "y" ] || [ "$START" = "Y" ]; then
docker compose up -d --build \
&& log_success "Anki Progress Dashboard started" \
|| log_warning "Start failed — check: docker compose logs"
fi
# ── Caddy + Authelia-aware protection ────────────────────────────────────
# This dashboard shows every account's personal study activity — same
# "sensitive, protect by default" reasoning as
# services/security-dashboard.sh: auto-use local Authelia if present, no
# prompt needed; otherwise warn clearly and offer a remote instance,
# since leaving it open is a real privacy tradeoff, not a neutral default.
local EXTRA_BLOCK=""
if [ -d "$DOCKER_DIR/authelia" ]; then
EXTRA_BLOCK=" import authelia"
log_info "Local Authelia detected — protecting with it."
else
log_warning "No local Authelia found. This dashboard shows every account's"
log_warning "personal study activity — recommend protecting it before"
log_warning "exposing it publicly."
local _use_remote=""
prompt_yn " Protect with a remote Authelia instance (e.g. on a homelab)? (y/n):" "y" _use_remote
if [[ "$_use_remote" =~ ^[Yy]$ ]]; then
local _remote_authelia=""
prompt_text " Remote Authelia address (bare host:port on a private network, or a full https:// URL on its own public domain+TLS):" "" _remote_authelia
if [ -n "$_remote_authelia" ]; then
EXTRA_BLOCK=" forward_auth ${_remote_authelia} {
uri /api/authz/forward-auth
copy_headers Remote-User Remote-Groups Remote-Name Remote-Email
header_up X-Forwarded-Method {method}
header_up X-Forwarded-Proto {scheme}
header_up X-Forwarded-Host {host}
header_up X-Forwarded-Uri {uri}
}"
fi
fi
fi
configure_caddy_for_service "Anki Progress Dashboard${INSTANCE_SUFFIX:+ ($INSTANCE_SUFFIX)}" "${CONTAINER}:5000" "anki-progress${INSTANCE_SUFFIX:+-$INSTANCE_SUFFIX}" "$EXTRA_BLOCK"
declare -F _authelia_scope_access >/dev/null 2>&1 && [ "${CADDY_SERVICE_CONFIGURED:-false}" = true ] \
&& _authelia_scope_access "anki-progress" "$CADDY_SERVICE_DOMAIN"
write_readme "$AP_DIR" << MD
# Anki Progress Dashboard${INSTANCE_SUFFIX:+ — $INSTANCE_SUFFIX}
Read-only study-progress dashboard for the accounts on **$SYNC_INSTANCE**
(reviews today/this week, accuracy, streak, last active), plus an ntfy
notification sent ${NOTIFY_DELAY_MINUTES} minutes after a study session
starts — defined as the first review after ${SESSION_GAP_MINUTES}+ minutes
of inactivity, and only sent if the session is still going at that point
(not on every single review, and not for a session that's already over).
Reads collection files directly with SQLite's read-only mode — never opens
them for write, so it can't corrupt or interfere with the live sync server
or any client actively syncing.
## Access
- URL: $( [ "${CADDY_SERVICE_CONFIGURED:-false}" = true ] && echo "https://${CADDY_SERVICE_DOMAIN}/" || echo "http://localhost:${WEB_PORT}/" )
## Config
- \`$AP_DIR/.env\` — ntfy URL/topic, session-gap and notify-delay minutes
- Edit and \`docker compose up -d\` to apply changes (no rebuild needed —
these are read at container start from environment variables)
## Manage
\`\`\`bash
cd $AP_DIR
docker compose up -d --build
docker compose down
docker compose logs -f
\`\`\`
MD
}
# ── Standalone execution ───────────────────────────────────────────────────
if [[ "${_RUN_STANDALONE:-0}" == "1" ]]; then
install_anki-progress
fi
-82
View File
@@ -1,82 +0,0 @@
## Client setup — pointing Anki at this server instead of AnkiWeb
Every client below needs the **Sync URL** and one of the **accounts** shown
higher up in this README. Do this on every device you want synced — a client
still pointed at AnkiWeb won't see collections synced here, and vice versa.
### Anki Desktop (2.1.66 and newer)
1. **Preferences → Network**
2. Tick **"Self-hosted sync server"**
3. Paste the Sync URL into the field that appears
4. **Sync → log in** with one of the accounts above
### Anki Desktop (older than 2.1.66)
There's no GUI field yet — set an environment variable before launching Anki
instead, then sync normally:
```bash
# Linux/macOS
export SYNC_ENDPOINT="https://your-sync-url/"
anki
# Windows (Command Prompt)
set SYNC_ENDPOINT=https://your-sync-url/
anki.exe
```
Upgrading Anki to 2.1.66+ is the easier long-term fix — do that if you're
setting this up for anyone who isn't comfortable with environment variables.
### AnkiDroid
**Settings → Advanced → Custom sync server**, then enter the Sync URL and
log in with one of the accounts above (AnkiDroid 2.16+; update the app if
this option isn't there).
### AnkiMobile (iOS)
**Settings → Advanced → Custom Sync Server**, same as AnkiDroid — enter the
Sync URL and log in.
### First sync on each device
The very first sync from a device that already has a local collection will
ask whether to upload local data or download from the server — pick upload
from whichever device has your real collection, and download on every other
device, or you'll end up with two different collections that never merge.
## Importing your existing Quizlet sets
This server only handles syncing already-existing Anki collections — it
doesn't import anything itself. Quizlet import happens once, locally, in the
Anki desktop app, before your first sync:
1. **In Quizlet:** open the set → **Export** → choose the plain-text /
tab-separated format (Quizlet's export dialog lets you pick the delimiter
between term and definition, and between rows — tab and newline are the
Anki-friendly defaults) → copy the exported text or download it as a
`.txt`/`.csv` file.
2. **In Anki Desktop:** **File → Import**, pick the file (or paste the text
into a `.txt` file first if you copied it to the clipboard).
3. Map the two columns to **Front** and **Back** in the import dialog, pick
or create the deck and note type, and import.
4. For **math facts or other simple front/back cards**, the Basic note type
is enough. For **more complex cards** (extra example fields, images,
audio, cloze deletions), switch the note type in the import dialog to a
template with more fields, or convert cards afterward — Anki's own
built-in note types (Basic, Basic (and reversed card), Cloze) cover most
of what Quizlet's own card types can do.
5. Sync from this device once the import looks right, so the imported deck
becomes the copy every other device downloads.
### Exporting back out (Anki → Quizlet or anywhere else)
**File → Export**, choose "Notes in Plain Text" and pick the deck — this
produces the same tab-separated format Quizlet's own import expects, so the
round trip works in both directions.
## Why spaced repetition here actually reschedules failed cards
Anki's scheduler (FSRS, the default since recent Anki versions) tracks a
per-card memory-strength estimate and schedules the next review right before
you'd be expected to forget it. Answering "Again" on a card doesn't just
requeue it for later the same session — it lowers that card's estimated
strength, which shortens every subsequent interval for it until you've
proven you know it again, so a card you keep failing gets shown far more
often than one you consistently get right. This is scheduling logic inside
the Anki client itself; this sync server only stores and syncs the resulting
review history, it doesn't change how reviews are scheduled.
-669
View File
@@ -1,669 +0,0 @@
#!/bin/bash
# services/anki-sync-server.sh — Self-hosted Anki flashcard sync server.
# Part of the modular post-install system (sourced by setup.sh).
#
# Can also be run standalone on any machine:
# sudo bash anki-sync-server.sh
# (Docker must already be installed when run standalone)
# ── Standalone bootstrap ──────────────────────────────────────────────────────
# Detected when the script is executed directly rather than sourced by setup.sh.
# Sets up helpers and globals, then defers execution until after the function
# definition at the bottom of this file.
if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
[[ "$(id -u)" == "0" ]] || { echo "Run with sudo: sudo bash $0"; exit 1; }
_SELF_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
_COMMON="$_SELF_DIR/../lib/common.sh"
if [[ -f "$_COMMON" ]]; then
# Full repo present — use the real helpers (picks up ~/docker/.config too)
# shellcheck source=../lib/common.sh
source "$_COMMON"
else
# One-off copy — inline minimal stubs so the script works without the repo
log_info() { echo -e "\033[0;34m[INFO]\033[0m $*"; }
log_success() { echo -e "\033[0;32m[OK]\033[0m $*"; }
log_warning() { echo -e "\033[1;33m[WARN]\033[0m $*"; }
log_error() { echo -e "\033[0;31m[ERROR]\033[0m $*" >&2; }
require_docker() {
command -v docker &>/dev/null || {
log_error "Docker not found. Install it first:"
log_error " curl -fsSL https://get.docker.com | sudo sh"
return 1
}
docker compose version &>/dev/null || {
log_error "Docker Compose plugin missing:"
log_error " sudo apt-get install -y docker-compose-plugin"
return 1
}
}
ensure_docker_dir_ownership() {
chown -R "$ACTUAL_USER:$ACTUAL_USER" "$@" 2>/dev/null || true
}
port_in_use() {
local _port="$1" _proto="${2:-tcp}"
local _flag="-tlnH"
[ "$_proto" = "udp" ] && _flag="-ulnH"
ss "$_flag" "sport = :${_port}" 2>/dev/null | grep -q .
}
find_free_port() {
local _varname="$1" _port="$2" _proto="${3:-tcp}"
while port_in_use "$_port" "$_proto"; do
_port=$((_port + 1))
done
eval "$_varname='$_port'"
}
# Match common.sh's eval-based pattern so local vars in install_* are set correctly
prompt_text() {
local _q="$1" _def="$2" _var="$3" _r
[[ "${UNATTENDED:-false}" == "true" ]] && { eval "$_var='$_def'"; return; }
read -r -p " $_q " _r
eval "$_var='${_r:-$_def}'"
}
prompt_yn() {
local _q="$1" _def="$2" _var="$3" _r
[[ "${UNATTENDED:-false}" == "true" ]] && { eval "$_var='$_def'"; return; }
read -r -p " $_q " _r
eval "$_var='${_r:-$_def}'"
}
prompt_reinstall_mode() {
local _var="$1" _r
if [[ "${UNATTENDED:-false}" == "true" ]]; then eval "$_var='cancel'"; return; fi
echo " Already installed."
read -r -p " (u)pdate / (f)resh reinstall / (c)ancel [c]: " _r
case "${_r,,}" in
u|update) eval "$_var='update'" ;;
f|fresh) eval "$_var='fresh'" ;;
*) eval "$_var='cancel'" ;;
esac
}
generate_password() {
local length="${1:-32}"
openssl rand -base64 48 | tr -dc 'a-zA-Z0-9' | head -c "$length"
}
configure_caddy_for_service() {
local _name="$1" _upstream="$2" _subdomain="$3" _extra="${4:-}"
local _caddy_dir="$DOCKER_DIR/caddy"
local _caddyfile="$_caddy_dir/Caddyfile"
local _display_port="${_upstream##*:}"
# Determine mode: local Caddy, remote Caddy, or none
local _mode="none"
[[ -d "$_caddy_dir" ]] && _mode="local"
[[ -n "${CADDY_REMOTE_HOST:-}" ]] && [[ "$_mode" != "local" ]] && _mode="remote"
[[ "$_mode" == "none" ]] && {
log_info "Access $_name directly on port $_display_port."
return 0
}
echo ""
local _do_caddy=""
if [[ "$_mode" == "remote" ]]; then
log_info "Remote Caddy configured (${CADDY_REMOTE_HOST})."
log_info "A snippet file will be saved to ~/docker/caddy-snippets/."
fi
read -r -p " Configure Caddy reverse proxy for $_name? [y/N]: " _do_caddy
[[ "${_do_caddy,,}" == "y" ]] || {
log_info "Skipping — access at: http://localhost:$_display_port"
return 0
}
# Domain prompt — pre-fill from SITE_DOMAIN when available
local _default_domain=""
if [[ -n "${SITE_DOMAIN:-}" ]] && [[ "$SITE_DOMAIN" != "example.com" ]]; then
_default_domain="${_subdomain}.${SITE_DOMAIN}"
log_info "Default: $_default_domain"
fi
local _domain=""
read -r -p " Domain [${_default_domain:-required}]: " _domain
_domain="${_domain:-$_default_domain}"
[[ -n "$_domain" ]] || { log_warning "No domain entered — skipping Caddy."; return 0; }
# Build upstream — remote Caddy uses host IP:port, not container name
local _block_upstream="$_upstream"
if [[ "$_mode" == "remote" ]]; then
_block_upstream="${CADDY_REMOTE_HOST}:${_display_port}"
fi
local _site_block
_site_block="$(cat << CBLOCK
# $_name
${_domain} {
reverse_proxy ${_block_upstream}
header {
Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
X-Content-Type-Options "nosniff"
X-Frame-Options "SAMEORIGIN"
Referrer-Policy "strict-origin-when-cross-origin"
}
log {
output file /var/log/caddy/${_domain}.log
format json
}
${_extra}
}
CBLOCK
)"
if [[ "$_mode" == "local" ]]; then
if [[ -f "$_caddyfile" ]]; then
local _bk="$_caddy_dir/Caddyfile.backup.$(date +%Y%m%d-%H%M%S)"
cp "$_caddyfile" "$_bk"
log_info "Backed up Caddyfile to $(basename "$_bk")"
else
touch "$_caddyfile"
fi
if grep -q "^${_domain}" "$_caddyfile" 2>/dev/null; then
log_warning "$_domain already in Caddyfile"
local _ow=""
read -r -p " Overwrite? [y/N]: " _ow
[[ "${_ow,,}" == "y" ]] || { log_info "Keeping existing entry."; return 0; }
sed -i "/^${_domain}/,/^}/d" "$_caddyfile"
fi
printf '%s\n' "$_site_block" >> "$_caddyfile"
log_success "Added $_domain to Caddyfile"
docker exec caddy caddy fmt --overwrite /etc/caddy/Caddyfile 2>/dev/null || true
if docker exec caddy caddy reload --config /etc/caddy/Caddyfile 2>/dev/null; then
log_success "$_name accessible at: https://$_domain"
else
log_warning "Reload failed — check: docker logs caddy"
log_info "Manual reload: docker exec caddy caddy reload --config /etc/caddy/Caddyfile"
fi
else
local _snippet_dir="$DOCKER_DIR/caddy-snippets"
local _snippet_file="$_snippet_dir/${_subdomain}.caddy"
mkdir -p "$_snippet_dir"
printf '%s\n' "$_site_block" > "$_snippet_file"
chown "$ACTUAL_USER:$ACTUAL_USER" "$_snippet_file" 2>/dev/null || true
log_success "Snippet saved: $_snippet_file"
log_info "Copy to Caddy machine:"
log_info " scp $_snippet_file caddy-host:~/caddy-snippets/"
log_info " rsync -av $_snippet_dir/ caddy-host:~/caddy-snippets/ (all at once)"
fi
}
write_readme() {
local _dir="$1"; shift
mkdir -p "$_dir"
cat > "$_dir/README.md"
}
backup_if_exists() {
local _file="$1"
[ -f "$_file" ] || return 0
cp -p "$_file" "${_file}.bak.$(date +%Y%m%d-%H%M%S)" 2>/dev/null
}
fi
# Globals — ACTUAL_USER/ACTUAL_HOME must come before DOCKER_DIR
# ($HOME under sudo is /root, not the real user's home)
ACTUAL_USER="${ACTUAL_USER:-${SUDO_USER:-$USER}}"
ACTUAL_HOME="$(getent passwd "$ACTUAL_USER" 2>/dev/null | cut -d: -f6 || echo "${HOME:-/root}")"
DOCKER_DIR="${DOCKER_DIR:-$ACTUAL_HOME/docker}"
DRY_RUN="${DRY_RUN:-false}"
UNATTENDED="${UNATTENDED:-false}"
SITE_TZ="${SITE_TZ:-$(cat /etc/timezone 2>/dev/null || echo UTC)}"
SITE_DOMAIN="${SITE_DOMAIN:-example.com}"
SITE_CADDY_NET="${SITE_CADDY_NET:-caddy_net}"
register_service() { :; } # no-op — no wizard to register into
_RUN_STANDALONE=1
fi
# ─────────────────────────────────────────────────────────────────────────────
register_service anki-sync-server utilities "Self-hosted Anki flashcard sync server (spaced repetition, syncs across devices without AnkiWeb)" 8080
# Reads the current ANKI_SYNC_USERn/ANKI_SYNC_PASSWORDn pairs out of an
# instance's .env into the caller's ANKI_USERS/ANKI_PASSWORDS arrays (bash's
# dynamic scoping means a `local` array declared in the caller is visible
# here without being passed explicitly — same assumption every other helper
# below makes). Numbering is always kept contiguous from 1 by
# _anki_rewrite_account_block, so stopping at the first missing index is
# safe — there's never a gap to skip over.
_anki_load_accounts() {
local _dir="$1" _n=1 _u _p
ANKI_USERS=() ANKI_PASSWORDS=()
while true; do
_u="$(grep "^ANKI_SYNC_USER${_n}=" "$_dir/.env" 2>/dev/null | cut -d= -f2-)"
[ -z "$_u" ] && break
_p="$(grep "^ANKI_SYNC_PASSWORD${_n}=" "$_dir/.env" 2>/dev/null | cut -d= -f2-)"
ANKI_USERS+=("$_u")
ANKI_PASSWORDS+=("$_p")
_n=$((_n + 1))
done
}
# Regenerates the SYNC_USERn=... lines in docker-compose.yml and the
# matching ANKI_SYNC_USERn/ANKI_SYNC_PASSWORDn pairs in .env from the
# caller's current ANKI_USERS/ANKI_PASSWORDS arrays (always renumbered
# contiguously from 1 — see _anki_load_accounts). Used by both the initial
# install and every account-management mutation (add/remove/rotate) so the
# two never drift apart, same reasoning as CLAUDE.md's shared-helper
# guidance for update vs. fresh-install codepaths. Leaves the port, Caddy
# block, and every other line in either file untouched — only lines
# matching the SYNC_USER/ANKI_SYNC_* patterns are touched.
_anki_rewrite_account_block() {
local _dir="$1"
local _compose="$_dir/docker-compose.yml"
local _env="$_dir/.env"
sed -i '/^ - SYNC_USER[0-9]\+=/d' "$_compose"
sed -i '/^ANKI_SYNC_USER[0-9]\+=/d; /^ANKI_SYNC_PASSWORD[0-9]\+=/d' "$_env"
local _compose_lines="" _env_lines="" i idx
for i in "${!ANKI_USERS[@]}"; do
idx=$((i + 1))
_compose_lines+=" - SYNC_USER${idx}=\${ANKI_SYNC_USER${idx}}:\${ANKI_SYNC_PASSWORD${idx}}
"
_env_lines+="ANKI_SYNC_USER${idx}=${ANKI_USERS[$i]}
ANKI_SYNC_PASSWORD${idx}=${ANKI_PASSWORDS[$i]}
"
done
# Insert right after the fixed SYNC_BASE anchor line — always present,
# written by every version of this script's install flow — instead of
# appending at the end, so the block stays grouped with SYNC_HOST/
# SYNC_PORT/SYNC_BASE rather than drifting after `volumes:`.
local _tmp
_tmp="$(mktemp)"
printf '%s' "$_compose_lines" > "$_tmp"
sed -i "\|^ - SYNC_BASE=/data\$|r $_tmp" "$_compose"
rm -f "$_tmp"
printf '%s' "$_env_lines" >> "$_env"
}
# Interactive add/remove/rotate menu for an existing instance's sync
# accounts, offered from install_anki-sync-server's "already installed"
# menu. Every mutation restarts the container (`docker compose up -d`
# re-reads .env for the new/removed/rotated credentials) but never touches
# the port, Caddy config, or the image — the things CLAUDE.md's "update vs.
# fresh reinstall" convention says a non-destructive path must leave alone.
_anki_manage_accounts() {
local _dir="$1"
local ANKI_USERS=() ANKI_PASSWORDS=()
while true; do
_anki_load_accounts "$_dir"
echo ""
echo " Current sync accounts:"
local i
for i in "${!ANKI_USERS[@]}"; do
echo " $((i + 1))) ${ANKI_USERS[$i]}"
done
[ "${#ANKI_USERS[@]}" -eq 0 ] && echo " (none)"
echo ""
echo " a) Add an account"
echo " r) Remove an account"
echo " p) Rotate (reset) an account's password"
echo " 0) Done"
echo ""
local ACTION=""
prompt_text " Choice [a/r/p/0]:" "0" ACTION
case "$ACTION" in
a|A)
if [ "${#ANKI_USERS[@]}" -ge 8 ]; then
log_warning "That's plenty — stopping at 8 accounts."
continue
fi
local _u=""
prompt_text " New username:" "" _u
if [ -z "$_u" ]; then
log_warning "Name can't be empty."; continue
fi
ANKI_USERS+=("$_u")
ANKI_PASSWORDS+=("$(generate_password 24)")
_anki_rewrite_account_block "$_dir"
( cd "$_dir" && docker compose up -d ) \
&& log_success "Account '$_u' added — password: ${ANKI_PASSWORDS[-1]} (also saved in $_dir/.env)" \
|| log_warning "Container restart failed — check: docker compose -f $_dir/docker-compose.yml logs"
;;
r|R)
if [ "${#ANKI_USERS[@]}" -eq 0 ]; then
log_warning "No accounts to remove."; continue
fi
local _n=""
prompt_text " Remove which number?" "" _n
if ! [[ "$_n" =~ ^[0-9]+$ ]] || [ "$_n" -lt 1 ] || [ "$_n" -gt "${#ANKI_USERS[@]}" ]; then
log_warning "Invalid choice."; continue
fi
local _removed="${ANKI_USERS[$((_n - 1))]}"
unset 'ANKI_USERS[_n - 1]' 'ANKI_PASSWORDS[_n - 1]'
ANKI_USERS=("${ANKI_USERS[@]}")
ANKI_PASSWORDS=("${ANKI_PASSWORDS[@]}")
_anki_rewrite_account_block "$_dir"
( cd "$_dir" && docker compose up -d ) \
&& log_success "Account '$_removed' removed" \
|| log_warning "Container restart failed — check: docker compose -f $_dir/docker-compose.yml logs"
;;
p|P)
if [ "${#ANKI_USERS[@]}" -eq 0 ]; then
log_warning "No accounts yet."; continue
fi
local _n=""
prompt_text " Rotate password for which number?" "" _n
if ! [[ "$_n" =~ ^[0-9]+$ ]] || [ "$_n" -lt 1 ] || [ "$_n" -gt "${#ANKI_USERS[@]}" ]; then
log_warning "Invalid choice."; continue
fi
ANKI_PASSWORDS[$((_n - 1))]="$(generate_password 24)"
_anki_rewrite_account_block "$_dir"
( cd "$_dir" && docker compose up -d ) \
&& log_success "New password for '${ANKI_USERS[$((_n - 1))]}': ${ANKI_PASSWORDS[$((_n - 1))]} (also saved in $_dir/.env)" \
|| log_warning "Container restart failed — check: docker compose -f $_dir/docker-compose.yml logs"
;;
0)
break
;;
*)
log_warning "Unrecognized choice."
;;
esac
done
}
install_anki-sync-server() {
require_docker || return 1
log_info "Installing Anki Sync Server..."
# ── Instance selection ───────────────────────────────────────────────────
# First instance keeps the plain "anki-sync-server" name/paths/port exactly
# as before (zero behavior change for anyone with a single instance). Only
# asking to add a second one introduces suffixed naming — same pattern as
# services/ntfy.sh and services/homebox.sh. A second instance is a real
# use case here (e.g. a second household wanting fully separate data on
# the same box) even though one instance already supports multiple
# independent accounts via SYNC_USER1/SYNC_USER2/... — see CLAUDE.md's
# "Multi-instance services" section.
local ANKI_DIR="$DOCKER_DIR/anki-sync-server"
local INSTANCE_SUFFIX="" CONTAINER="anki-sync-server"
local WEB_PORT="8080"
if [ "$DRY_RUN" = true ]; then
echo "[DRY-RUN] Would offer to add a new, separate instance if one already exists"
echo "[DRY-RUN] Would create $ANKI_DIR(-<name>)"
echo "[DRY-RUN] Would prompt for one or more sync accounts and generate passwords"
echo "[DRY-RUN] Would write docker-compose.yml and .env"
echo "[DRY-RUN] Would auto-scan for a free host port"
return 0
fi
if [ -d "$ANKI_DIR" ]; then
echo ""
echo " Anki Sync Server is already installed at $ANKI_DIR."
echo " 1) Manage sync accounts (add / remove / rotate a password — doesn't"
echo " touch the port, Caddy, or the image)"
echo " 2) Manage that install (update image / full reinstall / cancel)"
echo " 3) Add a NEW, separate Anki Sync Server instance alongside it (its"
echo " own data and port — full isolation)"
echo ""
local _TOP_CHOICE=""
prompt_text " Choice [1/2/3]:" "2" _TOP_CHOICE
if [ "$_TOP_CHOICE" = "1" ]; then
_anki_manage_accounts "$ANKI_DIR"
return 0
elif [ "$_TOP_CHOICE" = "3" ]; then
local _suffix=""
while true; do
prompt_text " Short name for the new instance (letters/numbers/hyphens, e.g. 'family'):" "" _suffix
_suffix="$(echo "$_suffix" | tr -cs 'a-zA-Z0-9-' '-' | sed 's/^-*//;s/-*$//')"
if [ -z "$_suffix" ]; then
log_warning "Name can't be empty."; continue
fi
if [ -d "$DOCKER_DIR/anki-sync-server-$_suffix" ]; then
log_warning "anki-sync-server-$_suffix already exists — pick another name."; continue
fi
break
done
INSTANCE_SUFFIX="$_suffix"
ANKI_DIR="$DOCKER_DIR/anki-sync-server-$_suffix"
CONTAINER="anki-sync-server-$_suffix"
log_info "New instance: $ANKI_DIR"
else
# "Manage that install" on THIS instance — the banner above promises
# update/fresh/cancel, so actually offer it instead of falling straight
# through into the same unconditional-overwrite flow as a new install.
if [[ -f "$ANKI_DIR/docker-compose.yml" ]]; then
local MODE=""
prompt_reinstall_mode MODE
case "$MODE" in
update)
log_info "Refreshing the Anki Sync Server image only — existing accounts, port, and Caddy setup are left as-is."
# The image is a Google distroless "nonroot" build (fixed UID/GID
# 65532, no shell — it can't chown anything itself at startup), so
# ./data has to already be writable by that exact UID or the
# container fails to start. Versions of this installer before this
# fix chowned it to ACTUAL_USER instead, which the container can't
# write to — re-asserting the correct ownership here repairs any
# install made under that bug, non-destructively (it's the
# installer's own bug being corrected, not a config choice, so it
# belongs in the non-destructive update path).
chown -R 65532:65532 "$ANKI_DIR/data" 2>/dev/null
( cd "$ANKI_DIR" && docker compose pull && docker compose up -d ) \
&& log_success "Anki Sync Server image refreshed" \
|| log_warning "Refresh failed — check: docker compose -f $ANKI_DIR/docker-compose.yml logs"
return 0
;;
cancel)
log_info "Leaving the existing install as-is."
return 0
;;
fresh) ;; # fall through to the full install flow below
esac
fi
fi
fi
# Scan for a free port unconditionally — not just when adding an explicit
# additional instance. A plain first install can just as easily collide
# with an unrelated service that already claimed this default port — see
# CLAUDE.md's "Port collision avoidance" section.
find_free_port WEB_PORT "$WEB_PORT"
# ── Sync accounts ─────────────────────────────────────────────────────────
# The official sync server has no signup flow of its own — accounts are
# fixed credentials baked in as SYNC_USER1, SYNC_USER2, ... at container
# start, one per line in .env. Ask for at least one now (each Anki client
# — desktop, AnkiDroid, AnkiMobile — logs in with one of these) and offer
# to add more for other people sharing this box, since a single instance
# already keeps each account's collection completely separate.
local ANKI_USERS=() ANKI_PASSWORDS=()
local _u=""
prompt_text " Username for your Anki sync account:" "$ACTUAL_USER" _u
ANKI_USERS+=("$_u")
ANKI_PASSWORDS+=("$(generate_password 24)")
while true; do
local _more=""
prompt_yn " Add another Anki sync account (e.g. for a family member)? (y/n):" "n" _more
[[ "$_more" =~ ^[Yy]$ ]] || break
prompt_text " Username for the additional account:" "" _u
if [ -z "$_u" ]; then
log_warning "Name can't be empty."; continue
fi
ANKI_USERS+=("$_u")
ANKI_PASSWORDS+=("$(generate_password 24)")
if [ "${#ANKI_USERS[@]}" -ge 8 ]; then
log_warning "That's plenty — stopping at 8 accounts."
break
fi
done
mkdir -p "$ANKI_DIR/data"
ensure_docker_dir_ownership "$ANKI_DIR"
cd "$ANKI_DIR" || return 1
# Mirrors configure_caddy_for_service's own mode resolution (lib/common.sh):
# explicit CADDY_MODE from the site config wins, then a local ~/docker/caddy,
# then the legacy CADDY_REMOTE_HOST var. Only "local" joins caddy_net — a
# remote Caddy box can't resolve container names on this host's bridge
# network anyway; it reaches this service via the host's published port.
local _CADDY_MODE="${CADDY_MODE:-none}"
[ "$_CADDY_MODE" = "none" ] && [ -d "$DOCKER_DIR/caddy" ] && _CADDY_MODE="local"
[ "$_CADDY_MODE" = "none" ] && [ -n "${CADDY_REMOTE_HOST:-}" ] && _CADDY_MODE="remote"
local _CADDY_NET_BLOCK=""
local _CADDY_NET_SECTION=""
if [ "$_CADDY_MODE" = "local" ]; then
_CADDY_NET_BLOCK=" networks:
- caddy_net
"
_CADDY_NET_SECTION="
networks:
caddy_net:
external: true
name: ${SITE_CADDY_NET:-caddy_net}
"
fi
# Build the SYNC_USERn=... lines for docker-compose.yml (compose-time
# interpolation of ${ANKI_SYNC_USERn}/${ANKI_SYNC_PASSWORDn} from .env —
# same \${VAR} pattern services/homebox.sh uses for its own .env values)
# and the matching ANKI_SYNC_USERn/ANKI_SYNC_PASSWORDn lines for .env.
local _COMPOSE_USER_LINES="" _ENV_USER_LINES="" i idx
for i in "${!ANKI_USERS[@]}"; do
idx=$((i + 1))
_COMPOSE_USER_LINES+=" - SYNC_USER${idx}=\${ANKI_SYNC_USER${idx}}:\${ANKI_SYNC_PASSWORD${idx}}
"
_ENV_USER_LINES+="ANKI_SYNC_USER${idx}=${ANKI_USERS[$i]}
ANKI_SYNC_PASSWORD${idx}=${ANKI_PASSWORDS[$i]}
"
done
backup_if_exists docker-compose.yml
cat > docker-compose.yml << ANKI_COMPOSE
name: $CONTAINER
services:
anki-sync-server:
image: afrima/anki-sync-server:latest
container_name: $CONTAINER
hostname: $CONTAINER
restart: unless-stopped
environment:
- SYNC_HOST=0.0.0.0
- SYNC_PORT=8080
- SYNC_BASE=/data
${_COMPOSE_USER_LINES} volumes:
- ./data:/data
ports:
- "${WEB_PORT}:8080"
${_CADDY_NET_BLOCK}${_CADDY_NET_SECTION}
ANKI_COMPOSE
backup_if_exists .env
cat > .env << ANKI_ENV
TZ=${SITE_TZ:-$(cat /etc/timezone 2>/dev/null || echo UTC)}
CADDY_NET=$SITE_CADDY_NET
# One username/password pair per Anki sync account (SYNC_USER1, SYNC_USER2,
# ... in docker-compose.yml). Enter these exact values as the account on
# each Anki client (Preferences/Settings → self-hosted sync server). To add,
# remove, or reset one of these later, re-run this installer against the
# existing install and pick "Manage sync accounts" — don't hand-edit these
# lines, the matching docker-compose.yml lines have to change in lockstep.
${_ENV_USER_LINES}
ANKI_ENV
chmod 600 .env
chown -R "$ACTUAL_USER:$ACTUAL_USER" "$ANKI_DIR"
# afrima/anki-sync-server is built on gcr.io/distroless/static-debian12:nonroot
# — the process always runs as that image's fixed "nonroot" UID/GID (65532),
# never as ACTUAL_USER, and distroless has no shell so nothing inside the
# container can chown its own data dir at startup. Applied AFTER the
# ACTUAL_USER chown above (not before — that call would just clobber it,
# since it recurses over the whole $ANKI_DIR including data/) so ./data ends
# up owned by 65532 specifically while docker-compose.yml/.env/README.md
# (which the sysadmin edits, not the container) stay owned by ACTUAL_USER.
# Confirmed live: getting this wrong is exactly what makes the container
# fail to come up with a permissions error the moment it tries to create
# anything under /data (e.g. a new user's collection).
chown -R 65532:65532 "$ANKI_DIR/data"
echo ""
log_success "Anki Sync Server${INSTANCE_SUFFIX:+ ($INSTANCE_SUFFIX)} configured at $ANKI_DIR (port $WEB_PORT)"
echo ""
echo " Sync accounts (also saved in $ANKI_DIR/.env):"
for i in "${!ANKI_USERS[@]}"; do
echo " ${ANKI_USERS[$i]} / ${ANKI_PASSWORDS[$i]}"
done
echo ""
# No Authelia gate here, unlike most other web-facing services in this
# repo: this is a raw HTTP sync API that the Anki client itself talks to
# (not a browser session), so a forward_auth login portal in front of it
# would just break every sync request instead of protecting anything.
# SYNC_USER1/SYNC_USER2/... above is this service's own auth boundary —
# same reasoning as the has-built-in-auth services in CLAUDE.md, just
# with no web UI to additionally gate.
configure_caddy_for_service "Anki Sync Server${INSTANCE_SUFFIX:+ ($INSTANCE_SUFFIX)}" "${CONTAINER}:8080" "anki${INSTANCE_SUFFIX:+-$INSTANCE_SUFFIX}"
local START=""
prompt_yn "Start Anki Sync Server${INSTANCE_SUFFIX:+ ($INSTANCE_SUFFIX)} now? (y/n):" "y" START
if [ "$START" = "y" ] || [ "$START" = "Y" ]; then
docker compose up -d \
&& log_success "Anki Sync Server started" \
|| log_warning "Start failed — check: docker compose logs"
fi
write_readme "$ANKI_DIR" << MD
# Anki Sync Server${INSTANCE_SUFFIX:+ — $INSTANCE_SUFFIX}
Self-hosted sync server for the [Anki](https://apps.ankiweb.net/) flashcard
app — syncs your collection across devices without going through AnkiWeb.
Anki's own spaced-repetition scheduler (FSRS) gives failed cards more
repetition and correctly-recalled cards longer gaps automatically; nothing
here changes that, it's purely the sync backend.
$( [ -n "$INSTANCE_SUFFIX" ] && echo "
This is a separate, fully isolated instance (own data directory, own
accounts, own port) — not shared collections with another Anki Sync Server
instance.")
## Access
- Sync URL: $( [ -n "${CADDY_SERVICE_CONFIGURED:-}" ] && [ "$CADDY_SERVICE_CONFIGURED" = "true" ] && echo "https://${CADDY_SERVICE_DOMAIN}/" || echo "http://localhost:${WEB_PORT}/" )
- Accounts (username / password):
$(for i in "${!ANKI_USERS[@]}"; do echo " - ${ANKI_USERS[$i]} / ${ANKI_PASSWORDS[$i]}"; done)
Enter the Sync URL and one of the above accounts on each Anki client — see
the client setup section below for exactly where.
## Data
- Collections: \`$ANKI_DIR/data\`
- Credentials: \`$ANKI_DIR/.env\` (readable by $ACTUAL_USER only)
## Manage
\`\`\`bash
cd $ANKI_DIR
docker compose up -d
docker compose down
docker compose logs -f
docker compose pull && docker compose up -d
\`\`\`
To add, remove, or reset the password of a sync account later, re-run the
installer against this install and pick **"Manage sync accounts"** —
don't hand-edit \`.env\`, the matching lines in \`docker-compose.yml\` have
to change alongside it:
\`\`\`bash
sudo ./setup.sh anki-sync-server
\`\`\`
MD
log_info "Full client setup + Quizlet import walkthrough written to $ANKI_DIR/README.md"
}
# ── Standalone execution ───────────────────────────────────────────────────
if [[ "${_RUN_STANDALONE:-0}" == "1" ]]; then
install_anki-sync-server
fi
+23 -97
View File
@@ -242,8 +242,7 @@ install_authelia() {
echo " 7) Reconfigure from scratch (regenerates secrets/users — breaks"
echo " existing sessions for every domain already on this instance)"
echo " 8) Show who has universal vs. service-scoped access"
echo " 9) Change \"Remember me\" session duration (stay logged in longer — also"
echo " raises the inactivity timeout to match, so it can't cut it short)"
echo " 9) Change \"Remember me\" session duration (stay logged in longer)"
echo " 10) Protect an existing site with this instance (pick a local Caddy site,"
echo " or type one on a different box — gates it with a login, same as any"
echo " other service already protected this way)"
@@ -502,12 +501,7 @@ access_control:
session:
name: authelia_session
expiration: 12h
# Matches remember_me below, not a shorter default — an idle timeout
# shorter than remember_me silently cuts a "remembered" session short
# regardless of its own duration. See _authelia_set_remember_me()'s
# comment for the live case this caused. Change both together (that
# function does exactly this) rather than one at a time.
inactivity: 7d
inactivity: 2h
remember_me: 7d
cookies:
- domain: ${AUTHELIA_DOMAIN}
@@ -1362,29 +1356,6 @@ _authelia_gen_temp_password() {
| fold -w1 | shuf | tr -d '\n'
}
# Lets the admin type a specific password instead of always getting an
# auto-generated one — same masked-input, "[Enter = auto-generate]"
# convention services/backup.sh/borg-backup.sh/koha.sh already use for their
# own passwords, rather than inventing a separate typed-vs-generated menu
# choice here. Sets two out-params (not `local` — read them after the call
# returns, same convention as OIDC_CLIENT_SECRET_PLAIN elsewhere in this
# file): AUTHELIA_CHOSEN_PASSWORD (the plaintext, never written to disk —
# only its argon2 hash is) and AUTHELIA_PASSWORD_AUTO_GENERATED (so callers
# can word their own "here's the password" message correctly either way).
_authelia_prompt_password() {
AUTHELIA_CHOSEN_PASSWORD=""
AUTHELIA_PASSWORD_AUTO_GENERATED=false
local _pw=""
if [ "$UNATTENDED" != true ]; then
read -rsp " Password [Enter = auto-generate]: " _pw; echo
fi
if [ -z "$_pw" ]; then
_pw="$(_authelia_gen_temp_password)"
AUTHELIA_PASSWORD_AUTO_GENERATED=true
fi
AUTHELIA_CHOSEN_PASSWORD="$_pw"
}
# Adds a new user to an EXISTING Authelia instance's users.yml — the scripted
# version of the manual "generate a hash, paste a users.yml block, restart"
# steps this file's own generated README already documents. Non-destructive:
@@ -1405,9 +1376,8 @@ add_authelia_user() {
echo ""
echo " Add a new user to this Authelia instance."
echo " They log in with their username (not email). You'll set a password"
echo " next — type your own or leave it blank to auto-generate one — shown"
echo " once here either way, never stored in plaintext. \"Forgot Password\""
echo " They log in with their username (not email). A temporary password"
echo " is generated below — hand it to them directly. \"Forgot Password\""
echo " and Authelia's own Settings → Change Password both require working"
echo " SMTP (both email a one-time code), so until that's fixed, use this"
echo " menu's \"Edit an existing user\" → \"Reset password\" for future resets."
@@ -1429,9 +1399,9 @@ add_authelia_user() {
local NEW_ADMIN_YN=""
prompt_yn " Grant admin group membership too? (y/n):" "n" NEW_ADMIN_YN
_authelia_prompt_password
local TEMP_PASS="$AUTHELIA_CHOSEN_PASSWORD" NEW_HASH
log_info "Generating password hash..."
log_info "Generating temporary password + hash..."
local TEMP_PASS NEW_HASH
TEMP_PASS="$(_authelia_gen_temp_password)"
NEW_HASH=$(docker run --rm authelia/authelia:4.39.20 \
authelia crypto hash generate argon2 --password "$TEMP_PASS" 2>/dev/null \
| grep -oP '(?<=Digest: ).*')
@@ -1469,12 +1439,8 @@ ${GROUPS_BLOCK}"
fi
echo ""
echo " New user: ${NEW_USERNAME}"
if [ "$AUTHELIA_PASSWORD_AUTO_GENERATED" = true ]; then
echo " Temp password: ${TEMP_PASS}"
else
echo " Password: ${TEMP_PASS} (the one you just typed)"
fi
echo " New user: ${NEW_USERNAME}"
echo " Temp password: ${TEMP_PASS}"
echo " Give this to them directly (it's shown once, nothing stores it in"
echo " plaintext). They can log in with it as-is and keep using it, or"
echo " change it themselves from Authelia's Settings page — but that page"
@@ -2389,19 +2355,6 @@ _authelia_report_access_scope() {
# earlier version of this very file's own README section) uses the old
# name, which Authelia would just silently ignore rather than error on.
#
# Also writes the SAME value into `inactivity` — a separate session field
# (default 2h, set alongside remember_me in install_authelia()'s own
# template) that ends a session after that much idle time regardless of
# remember_me, since it isn't disabled or extended by the "Remember me"
# checkbox. Confirmed live: a user who'd set remember_me to 1y still got
# logged out after ordinary daily gaps (overnight, a workday) because
# inactivity was still sitting at its 2h default — remember_me alone does
# NOT deliver "won't be asked to log in again for the duration I set"
# without this. Tying the two together is what actually delivers that.
# `expiration` (the session cap when "Remember me" is NOT checked) is left
# alone — a shorter default there for an un-remembered session is correct,
# separate behavior, not the same gap.
#
# This only controls AUTHELIA's own session — it does not touch how long
# a native-OIDC app's (Gitea/Mealie/ActualBudget) own session/token lasts
# after logging in via Authelia. A long remember_me makes re-authenticating
@@ -2412,48 +2365,27 @@ _authelia_set_remember_me() {
local config_file="$DOCKER_DIR/authelia/config/configuration.yml"
[ -f "$config_file" ] || { log_warning "No configuration.yml found — install Authelia first."; return 1; }
local current current_inactivity
local current
current="$(grep -E '^ remember_me:' "$config_file" | awk '{print $2}' | tr -d "'\"")"
current_inactivity="$(grep -E '^ inactivity:' "$config_file" | awk '{print $2}' | tr -d "'\"")"
echo ""
echo " Current \"remember me\" duration: ${current:-not set} (inactivity timeout: ${current_inactivity:-not set})"
echo " Current \"remember me\" duration: ${current:-not set}"
echo " How long a session lasts when someone checks \"Remember me\" at login —"
echo " applies to every domain this Authelia instance protects. Also sets"
echo " \"inactivity\" (idle timeout) to the same value, so a gap between visits"
echo " shorter than this can't log you out early — otherwise inactivity's own"
echo " separate, much shorter default cuts a long remember_me short."
echo " applies to every domain this Authelia instance protects."
echo " Examples: 12h, 7d, 1M (month), 1y. Set to -1 to disable Remember Me entirely."
local new_duration=""
prompt_text " New duration [${current:-7d}]:" "${current:-7d}" new_duration
if [ -z "$new_duration" ]; then
if [ -z "$new_duration" ] || [ "$new_duration" = "$current" ]; then
log_info "No change made."
return 0
fi
# Only truly a no-op if BOTH keys already match — remember_me alone
# matching isn't enough to skip, or an install still carrying the old
# mismatched inactivity default (from before this function synced the
# two) could never actually get inactivity fixed by re-entering the
# same remember_me value. Confirmed live: this is exactly what
# happened on a box that had already set remember_me: 1y before this
# sync existed — re-running with "1y" again hit this early return and
# left inactivity untouched.
if [ "$new_duration" = "$current" ] && [ "$new_duration" = "$current_inactivity" ]; then
log_info "No change made — remember_me and inactivity already both ${new_duration}."
return 0
fi
if grep -qE '^ remember_me:' "$config_file"; then
sed -i "s/^ remember_me:.*/ remember_me: '${new_duration}'/" "$config_file"
else
sed -i "/^session:\$/a\\ remember_me: '${new_duration}'" "$config_file"
fi
if grep -qE '^ inactivity:' "$config_file"; then
sed -i "s/^ inactivity:.*/ inactivity: '${new_duration}'/" "$config_file"
else
sed -i "/^ remember_me:/a\\ inactivity: '${new_duration}'" "$config_file"
fi
chown 1000:1000 "$config_file" 2>/dev/null || true
log_success "\"Remember me\" duration and inactivity timeout both set to ${new_duration}."
log_success "\"Remember me\" duration set to ${new_duration}."
local restart_auth=""
prompt_yn " Restart Authelia to apply? (y/n):" "y" restart_auth
@@ -2465,10 +2397,9 @@ _authelia_set_remember_me() {
echo ""
log_info "Takes effect for NEW logins where \"Remember me\" is checked at Authelia's"
log_info "login page — existing sessions keep whatever expiration/inactivity they"
log_info "already had. The checkbox itself is already on the login form by default;"
log_info "this only changes how long checking it actually keeps you signed in, and"
log_info "stops the separate inactivity timeout from cutting that short."
log_info "login page — existing sessions keep whatever expiration they already had."
log_info "The checkbox itself is already on the login form by default; this only"
log_info "changes how long checking it actually keeps you signed in."
}
# Export/import accounts (+ optionally 2FA/session state) — for migrating to
@@ -2766,7 +2697,7 @@ _authelia_manage_one_user() {
echo ""
echo " Editing user: $TARGET (admin: $IS_ADMIN, 2FA-exempt: $IS_EXEMPT)"
echo " 1) Edit email / display name"
echo " 2) Set/reset password (type your own, or auto-generate)"
echo " 2) Reset password"
echo " 3) Reset 2FA device (they register a new one on next login)"
if [ "$IS_EXEMPT" = "yes" ]; then
echo " 4) Restore the 2FA requirement for this user"
@@ -2798,9 +2729,9 @@ _authelia_manage_one_user() {
log_success "Updated $TARGET's email/display name."
;;
2)
_authelia_prompt_password
local NEW_TEMP_PASS="$AUTHELIA_CHOSEN_PASSWORD" NEW_HASH
log_info "Generating password hash..."
log_info "Generating a new temporary password + hash..."
local NEW_TEMP_PASS NEW_HASH
NEW_TEMP_PASS="$(_authelia_gen_temp_password)"
NEW_HASH=$(docker run --rm authelia/authelia:4.39.20 \
authelia crypto hash generate argon2 --password "$NEW_TEMP_PASS" 2>/dev/null \
| grep -oP '(?<=Digest: ).*')
@@ -2809,13 +2740,8 @@ _authelia_manage_one_user() {
else
_authelia_set_user_field "$USERS_FILE" "$START" "$END" "password" " password: \"${NEW_HASH}\""
chown 1000:1000 "$USERS_FILE" 2>/dev/null || true
if [ "$AUTHELIA_PASSWORD_AUTO_GENERATED" = true ]; then
log_success "Password reset for $TARGET (auto-generated)."
echo " New password: ${NEW_TEMP_PASS}"
else
log_success "Password set for $TARGET."
echo " Password: ${NEW_TEMP_PASS} (the one you just typed)"
fi
log_success "Password reset for $TARGET."
echo " New password: ${NEW_TEMP_PASS}"
echo " Give this to them directly — shown once, not stored in plaintext anywhere."
fi
;;
-25
View File
@@ -21,7 +21,6 @@ install_base() {
echo "[DRY-RUN] Would offer to mount SMB data from a NetBird-connected home box (if NetBird is present)"
echo "[DRY-RUN] Would offer Caddy reverse proxy install (full repo only)"
echo "[DRY-RUN] Would offer CrowdSec intrusion prevention install (full repo only)"
echo "[DRY-RUN] Would offer Samba (SMB/CIFS) file sharing install (full repo only)"
echo "[DRY-RUN] Would offer to add SSH Host aliases to ~/.ssh/config"
return 0
fi
@@ -81,14 +80,6 @@ install_base() {
_base_setup_crowdsec
cd "$_BASE_PWD" 2>/dev/null || true
# ── Samba ────────────────────────────────────────────────────────────────
# Same nudge-not-mandatory shape as Caddy/CrowdSec above: fully optional,
# independently re-runnable later via `sudo ./setup.sh samba`. Defaults to
# n (unlike Caddy/CrowdSec) because it needs real input to be useful — a
# share path and at least one user — not just "yes, with sane defaults".
_base_setup_samba
cd "$_BASE_PWD" 2>/dev/null || true
# ── SSH Host aliases ─────────────────────────────────────────────────────
_base_setup_ssh_aliases
@@ -338,22 +329,6 @@ _base_setup_crowdsec() {
install_crowdsec
}
_base_setup_samba() {
if command -v smbd &>/dev/null; then
log_info "Samba already installed."
return 0
fi
# Only available when the full repo is sourced (setup.sh loads every
# services/*.sh up front) — a standalone copy of base.sh doesn't have
# install_samba, so skip silently rather than error.
declare -F install_samba &>/dev/null || return 0
local INSTALL_SAMBA=""
prompt_yn "Install Samba (SMB/CIFS) file sharing now — shares, users, passwords? (y/n):" "n" INSTALL_SAMBA
[[ "$INSTALL_SAMBA" =~ ^[Yy]$ ]] || return 0
install_samba
}
_base_setup_ssh_aliases() {
local ADD_ALIAS=""
prompt_yn "Add an SSH Host alias now ('ssh myserver' instead of 'ssh user@1.2.3.4')? (y/n):" "n" ADD_ALIAS
+4 -342
View File
@@ -453,11 +453,6 @@ _gitea_remove_sync_timer() {
# reconfigure of an existing one. Always asked (matches pstn-trunk.sh's
# international-calling step reasoning: a live-editable extra, not a
# structural setting tied exclusively to fresh installs).
#
# Sets _GITEA_SYNC_FLAG as an out-param (not `local` — read it after the
# call returns, same convention as CADDY_SERVICE_CONFIGURED) so the caller
# can decide whether the real-time webhook offer even makes sense for the
# direction just chosen.
_gitea_run_sync_direction_step() {
local DIR="$1"
@@ -468,13 +463,12 @@ _gitea_run_sync_direction_step() {
echo " 3) Both directions"
local _DIR_CHOICE=""
prompt_text " Choice [1]:" "1" _DIR_CHOICE
local DIR_DESC=""
local FLAG="" DIR_DESC=""
case "$_DIR_CHOICE" in
2) _GITEA_SYNC_FLAG="--push-only"; DIR_DESC="Gitea -> GitHub only" ;;
3) _GITEA_SYNC_FLAG=""; DIR_DESC="both directions" ;;
*) _GITEA_SYNC_FLAG="--pull-only"; DIR_DESC="GitHub -> Gitea only" ;;
2) FLAG="--push-only"; DIR_DESC="Gitea -> GitHub only" ;;
3) FLAG=""; DIR_DESC="both directions" ;;
*) FLAG="--pull-only"; DIR_DESC="GitHub -> Gitea only" ;;
esac
local FLAG="$_GITEA_SYNC_FLAG"
log_info "Sync direction: $DIR_DESC"
_gitea_remove_sync_timer
@@ -525,250 +519,6 @@ _gitea_run_sync_direction_step() {
esac
}
# ── Real-time sync: a GitHub webhook receiver, not just the timer above ────
# The timer above polls on a fixed schedule (default 6h) — fine for a slow
# backup cadence, but a genuine "GitHub -> Gitea in real time" ask needs
# GitHub to tell Gitea the moment something changes instead of Gitea finding
# out up to one interval late. GitHub's own webhook (repo Settings ->
# Webhooks) is the standard way to do that: it POSTs a JSON payload the
# instant someone pushes. This writes a tiny stdlib-only Python HTTP server
# to receive it — python3 is already a hard dependency of this directory's
# gitea-github-sync.sh itself (used there for JSON parsing), so this adds
# no new dependency — running under its own persistent systemd service,
# and wires it up to Caddy the same way every other web-facing piece of
# this install does.
#
# Deliberately NOT a Docker container: it just shells out to the existing
# gitea-github-sync.sh sitting right next to it in $DIR, the same way the
# timer's own systemd service does — no image to build/pull for what's
# fundamentally a few lines of stdlib HTTP handling.
_gitea_write_webhook_receiver() {
local DIR="$1"
cat > "$DIR/gitea-github-webhook.py" << 'PYEOF'
#!/usr/bin/env python3
"""Gitea <-> GitHub webhook receiver — triggers an immediate, single-repo
mirror sync (gitea-github-sync.sh --repo owner/name --pull-only) the moment
GitHub POSTs a push event, instead of waiting for the scheduled timer.
Written by services/gitea.sh — re-run 'sudo ./setup.sh gitea' (Update mode
is fine) to regenerate this file rather than hand-editing it; a hand edit
survives until the next Update-mode rerun overwrites it again.
WEBHOOK_SECRET is read from .env in this same directory at every request,
never taken from the environment/systemd unit — /etc/systemd/system/*.service
files are world-readable, and .env (chmod 600) is already where every other
token in this directory lives.
"""
import hashlib
import hmac
import http.server
import json
import os
import subprocess
import sys
SYNC_DIR = os.environ.get("GITEA_SYNC_DIR", os.path.dirname(os.path.abspath(__file__)))
ENV_PATH = os.path.join(SYNC_DIR, ".env")
PORT = int(os.environ.get("WEBHOOK_PORT", "3020"))
def _load_env_value(key):
try:
with open(ENV_PATH, "r") as f:
for line in f:
line = line.split("#", 1)[0].strip()
if not line.startswith(key + "="):
continue
return line[len(key) + 1:].strip().strip("'").strip('"')
except OSError:
pass
return ""
class Handler(http.server.BaseHTTPRequestHandler):
def log_message(self, fmt, *args):
sys.stderr.write("%s - %s\n" % (self.address_string(), fmt % args))
def _reply(self, code, body=b""):
self.send_response(code)
self.end_headers()
if body:
self.wfile.write(body)
def do_GET(self):
self._reply(200, b"gitea-github-webhook: listening\n")
def do_POST(self):
secret = _load_env_value("WEBHOOK_SECRET").encode()
if not secret:
self._reply(503, b"WEBHOOK_SECRET not configured")
return
length = int(self.headers.get("Content-Length", 0) or 0)
body = self.rfile.read(length) if length else b""
sig = self.headers.get("X-Hub-Signature-256", "")
expected = "sha256=" + hmac.new(secret, body, hashlib.sha256).hexdigest()
if not sig or not hmac.compare_digest(sig, expected):
self._reply(401, b"bad signature")
return
event = self.headers.get("X-GitHub-Event", "")
if event == "ping":
self._reply(200, b"pong")
return
if event != "push":
self._reply(204)
return
try:
payload = json.loads(body or b"{}")
full_name = payload["repository"]["full_name"]
except (json.JSONDecodeError, KeyError, TypeError):
self._reply(400, b"couldn't find repository.full_name in payload")
return
self._reply(202, b"sync queued\n")
sync_script = os.path.join(SYNC_DIR, "gitea-github-sync.sh")
sync_env = dict(os.environ, SYNC_ENV=ENV_PATH)
subprocess.Popen(
["bash", sync_script, "--repo", full_name, "--pull-only"],
cwd=SYNC_DIR,
env=sync_env,
)
if __name__ == "__main__":
server = http.server.ThreadingHTTPServer(("0.0.0.0", PORT), Handler)
server.serve_forever()
PYEOF
chmod +x "$DIR/gitea-github-webhook.py"
chown "$ACTUAL_USER:$ACTUAL_USER" "$DIR/gitea-github-webhook.py"
}
_gitea_write_webhook_service() {
local DIR="$1" RUN_USER="$2" RUN_HOME="$3" PORT="$4"
local _service="/etc/systemd/system/gitea-github-webhook.service"
cat > "$_service" << UNIT
[Unit]
Description=Gitea-GitHub Webhook Receiver (real-time mirror sync trigger)
After=network-online.target docker.service
Wants=network-online.target
[Service]
Type=simple
User=${RUN_USER}
Environment=HOME=${RUN_HOME}
Environment=GITEA_SYNC_DIR=${DIR}
Environment=WEBHOOK_PORT=${PORT}
ExecStart=/usr/bin/python3 ${DIR}/gitea-github-webhook.py
Restart=on-failure
RestartSec=5
[Install]
WantedBy=multi-user.target
UNIT
systemctl daemon-reload
systemctl enable --now gitea-github-webhook.service
}
_gitea_remove_webhook_service() {
systemctl disable --now gitea-github-webhook.service 2>/dev/null || true
rm -f /etc/systemd/system/gitea-github-webhook.service
systemctl daemon-reload 2>/dev/null || true
}
# Offers the webhook receiver above as an addition to (not a replacement
# for) the timer set up in _gitea_run_sync_direction_step — the timer keeps
# covering the Gitea -> GitHub direction (and acts as a safety net for any
# push GitHub's webhook delivery ever misses), the webhook just gets the
# GitHub -> Gitea direction down from "up to one interval late" to seconds.
# Always asked on every install/reconfigure, same "live-editable extra"
# pattern as the direction+autosync step itself — see that function's own
# comment. Skipped (and any existing webhook torn down) outright when the
# chosen direction is push-only, since GitHub has nothing to notify about
# in that direction.
_gitea_offer_realtime_webhook() {
local DIR="$1" SYNC_FLAG="$2"
if [[ "$SYNC_FLAG" == "--push-only" ]]; then
_gitea_remove_webhook_service
return 0
fi
echo ""
local USE_WEBHOOK=""
prompt_yn " Also add a GitHub webhook for near-instant sync (push on GitHub -> synced here in seconds, instead of waiting for the timer above)? (y/n):" "n" USE_WEBHOOK
if [[ ! "$USE_WEBHOOK" =~ ^[Yy]$ ]]; then
_gitea_remove_webhook_service
return 0
fi
# Reuse an existing secret/port across reruns — rotating either one
# silently breaks a webhook GitHub already has configured against the
# old value, the same reasoning services/asterisk.sh's TURN port-range
# persistence follows for a live coturn install.
local WEBHOOK_SECRET WEBHOOK_PORT
WEBHOOK_SECRET="$(grep '^WEBHOOK_SECRET=' "$DIR/.env" 2>/dev/null | cut -d= -f2- | tr -d "'\"")"
WEBHOOK_PORT="$(grep '^WEBHOOK_PORT=' "$DIR/.env" 2>/dev/null | cut -d= -f2- | tr -d "'\"")"
[[ -z "$WEBHOOK_SECRET" ]] && WEBHOOK_SECRET="$(generate_password 40)"
if [[ -z "$WEBHOOK_PORT" ]]; then
WEBHOOK_PORT=3020
find_free_port WEBHOOK_PORT "$WEBHOOK_PORT"
fi
if grep -q '^WEBHOOK_SECRET=' "$DIR/.env" 2>/dev/null; then
sed -i "s|^WEBHOOK_SECRET=.*|WEBHOOK_SECRET='${WEBHOOK_SECRET}'|" "$DIR/.env"
else
echo "WEBHOOK_SECRET='${WEBHOOK_SECRET}'" >> "$DIR/.env"
fi
if grep -q '^WEBHOOK_PORT=' "$DIR/.env" 2>/dev/null; then
sed -i "s|^WEBHOOK_PORT=.*|WEBHOOK_PORT='${WEBHOOK_PORT}'|" "$DIR/.env"
else
echo "WEBHOOK_PORT='${WEBHOOK_PORT}'" >> "$DIR/.env"
fi
chmod 600 "$DIR/.env"
chown "$ACTUAL_USER:$ACTUAL_USER" "$DIR/.env"
_gitea_write_webhook_receiver "$DIR"
_gitea_write_webhook_service "$DIR" "$ACTUAL_USER" "$ACTUAL_HOME" "$WEBHOOK_PORT"
log_success "Webhook receiver running on port ${WEBHOOK_PORT} (systemctl status gitea-github-webhook)."
# Bare port -> host.docker.internal:PORT, same convention as every other
# host-process (non-container) upstream in this repo — see the
# configure_caddy_for_service usage note in CLAUDE.md.
configure_caddy_for_service "Gitea GitHub Webhook" "$WEBHOOK_PORT" "gitea-webhook"
if [[ "$CADDY_SERVICE_CONFIGURED" == true ]]; then
if command -v ufw &>/dev/null; then
if [[ "$CADDY_SERVICE_MODE" == "local" ]]; then
ufw delete allow "${WEBHOOK_PORT}/tcp" 2>/dev/null || true
ufw_allow_from_caddy_net "${WEBHOOK_PORT}"
else
ufw allow "${WEBHOOK_PORT}/tcp" comment "Gitea GitHub webhook" >/dev/null 2>&1 || true
ensure_ufw_enabled
fi
fi
echo ""
log_success "Now add the webhook on GitHub, for every repo you want instant sync from:"
log_info " Repo -> Settings -> Webhooks -> Add webhook"
log_info " Payload URL: https://${CADDY_SERVICE_DOMAIN}/"
log_info " Content type: application/json"
log_info " Secret: ${WEBHOOK_SECRET}"
log_info " Events: Just the push event"
log_info "The timer above still covers every other repo, and this one too, on its"
log_info "own schedule — the webhook is an addition, not a replacement for it."
else
log_warning "Webhook receiver is running (0.0.0.0:${WEBHOOK_PORT}) but nothing is exposing"
log_warning "it to the internet, so GitHub can't reach it yet — re-run this installer and"
log_warning "configure Caddy for it, or point your own reverse proxy at"
log_warning "127.0.0.1:${WEBHOOK_PORT} (or the container-reachable host IP) by hand."
log_info " Secret (for whenever you do expose it): ${WEBHOOK_SECRET}"
fi
}
install_gitea() {
log_info "Setting up self-hosted Gitea..."
@@ -779,16 +529,12 @@ install_gitea() {
if [ "$DRY_RUN" = true ]; then
echo "[DRY-RUN] Would create $DIR with docker-compose.yml (gitea/gitea:latest)"
echo "[DRY-RUN] Would scan for free host ports (web + SSH) to avoid collisions"
echo "[DRY-RUN] Would open the SSH clone port in UFW (web port too, or scoped to caddy_net"
echo "[DRY-RUN] if Caddy ends up fronting it locally)"
echo "[DRY-RUN] Would prompt for a Gitea admin username/password, then create that account"
echo "[DRY-RUN] and an API token once the container is ready (no manual web wizard)"
echo "[DRY-RUN] Would prompt for a GitHub token and copy in gitea-github-sync.sh"
echo "[DRY-RUN] Would ask sync direction (GitHub->Gitea / Gitea->GitHub / both) and whether"
echo "[DRY-RUN] to install a systemd timer for automatic sync, or print manual instructions"
echo "[DRY-RUN] Would offer to run a sync now (dry-run preview or for real), off-schedule"
echo "[DRY-RUN] Would offer a GitHub webhook receiver for near-instant GitHub->Gitea sync"
echo "[DRY-RUN] (systemd service + Caddy front door), unless direction is push-only"
echo "[DRY-RUN] Would offer \"Sign in with Authelia\" (OIDC) if Authelia is installed"
echo "[DRY-RUN] Would offer zero-click Authelia login (reverse-proxy auth) if Authelia"
echo "[DRY-RUN] and local Caddy are both installed — rewires Gitea onto caddy_net"
@@ -819,7 +565,6 @@ install_gitea() {
&& log_success "Gitea refreshed and restarted." \
|| log_warning "Restart failed — check: docker compose -f $DIR/docker-compose.yml logs"
_gitea_run_sync_direction_step "$DIR"
_gitea_offer_realtime_webhook "$DIR" "$_GITEA_SYNC_FLAG"
_gitea_offer_authelia_sso "$DIR"
_gitea_offer_reverse_proxy_auth "$DIR"
_gitea_offer_actions_runner "$DIR"
@@ -985,7 +730,6 @@ ENV
fi
_gitea_run_sync_direction_step "$DIR"
_gitea_offer_realtime_webhook "$DIR" "$_GITEA_SYNC_FLAG"
# ── Caddy — no forward_auth gate here. Gitea has its own built-in login,
# unlike the no-auth-at-all apps elsewhere in this repo that need Caddy
@@ -994,25 +738,6 @@ ENV
# replacement requiring Caddy involvement. ─────────────────────────────
configure_caddy_for_service "Gitea" "host.docker.internal:${WEB_PORT}" "git"
# ── Firewall ─────────────────────────────────────────────────────────────
# SSH clone (SSH_PORT->22) is a different protocol than the web UI — Caddy
# can't front it no matter what CADDY_SERVICE_MODE came back as, so it
# always needs its own direct rule or `git clone ssh://...` hangs forever
# (a dropped SYN with UFW active, not a fast connection-refused).
if command -v ufw &>/dev/null; then
if [[ "$CADDY_SERVICE_CONFIGURED" == true && "$CADDY_SERVICE_MODE" == "local" ]]; then
ufw delete allow "${WEB_PORT}/tcp" 2>/dev/null || true
ufw_allow_from_caddy_net "${WEB_PORT}"
else
ufw allow "${WEB_PORT}/tcp" comment "Gitea web UI" >/dev/null 2>&1 || true
fi
ufw allow "${SSH_PORT}/tcp" comment "Gitea SSH clone" >/dev/null 2>&1 || true
ensure_ufw_enabled
log_success "UFW: opened SSH clone port ${SSH_PORT}/tcp"
else
log_warning "ufw not installed — if you use a firewall, open TCP ${SSH_PORT} for SSH clones."
fi
_gitea_offer_authelia_sso "$DIR"
_gitea_offer_reverse_proxy_auth "$DIR"
_gitea_offer_actions_runner "$DIR"
@@ -1048,69 +773,6 @@ Config (which repos, private/forks handling) lives at
\`~/.config/gitea-github-sync/config\` — edit directly, or re-run
\`bash gitea-github-sync.sh --init\` to redo it interactively.
## Real-time sync via GitHub webhook (optional)
The setup above only covers the GitHub -> Gitea direction; it doesn't apply
if you chose Gitea -> GitHub only (GitHub has nothing to notify about in
that direction). Adds a small Python HTTP server
(\`gitea-github-webhook.py\`, in this directory) run as its own systemd
service (\`gitea-github-webhook.service\`) that GitHub POSTs to the instant
someone pushes — it verifies the request's HMAC signature against
\`WEBHOOK_SECRET\` in \`.env\`, then runs \`gitea-github-sync.sh --repo
owner/name --pull-only\` for just that one repo. The scheduled timer above
still runs on its own interval regardless — the webhook is an addition
that gets the GitHub -> Gitea direction down to seconds, not a replacement
for it (and still catches anything a missed webhook delivery would have
picked up next interval anyway).
Not set up yet, or want to change the port/secret? Re-run
\`sudo ./setup.sh gitea\` (Update mode is fine) and answer yes to "Also add
a GitHub webhook...". That only stands up the *receiver* on this box — you
still add the actual webhook on GitHub's side afterward, using the payload
URL and secret the installer printed (also readable back from \`.env\` as
\`WEBHOOK_PORT\` / \`WEBHOOK_SECRET\` if you need them again).
**Option A — one repo at a time.** Fastest, but only covers repos you do
this for individually:
repo -> Settings -> Webhooks -> Add webhook
- Payload URL: the URL the installer printed
- Content type: \`application/json\`
- Secret: your \`WEBHOOK_SECRET\`
- Events: "Just the push event"
**Option B — every repo on your account, current AND future, from one
setup.** A plain repo webhook (Option A) is always per-repo, no way around
that — but a personal GitHub App installed with "All repositories" access
covers every repo automatically, including ones you create afterward. No
receiver/code change needed for this: an App's webhook uses the exact same
HMAC-secret mechanism as a repo webhook, so the same \`WEBHOOK_SECRET\`
works for both.
1. GitHub -> Settings -> Developer settings -> GitHub Apps -> New GitHub App
2. Webhook URL: same payload URL as Option A. Webhook secret: your
\`WEBHOOK_SECRET\`. (Homepage URL is a separate, purely cosmetic field —
point it at anything, e.g. your GitHub profile; GitHub never sends
anything there, unlike Webhook URL.)
3. Permissions -> Repository permissions -> Contents: Read-only (required
to unlock the Push event checkbox)
4. Subscribe to events: Push only
5. Where can this GitHub App be installed: "Only on this account"
6. Create it, then Install App -> choose "All repositories" -> Install
If you'd already added Option A webhooks on a few repos, they're now
redundant (not harmful, just two triggers per push) — remove them once
the App is confirmed working.
**Verify either option** — push to a repo, then watch it arrive:
\`\`\`bash
systemctl status gitea-github-webhook # is it running?
journalctl -u gitea-github-webhook -f # watch it receive + trigger syncs
\`\`\`
GitHub also shows delivery attempts and response codes: repo (or App) ->
Settings -> Webhooks -> the webhook -> Recent Deliveries.
## Sign in with Authelia (optional)
If Authelia is installed, re-run \`sudo ./setup.sh gitea\` (Update mode is
-365
View File
@@ -1,365 +0,0 @@
#!/bin/bash
# services/samba.sh — Samba (SMB/CIFS) file sharing: shares, users, passwords.
# Part of the modular post-install system (sourced by setup.sh).
#
# Can also be run standalone on any machine:
# sudo bash samba.sh
#
# Samba is a SYSTEM install (apt package + native smbd/nmbd services), NOT a
# docker-compose service — same shape as services/crowdsec.sh. There is no
# ~/docker/samba compose stack; we only create a docs-only folder there with
# a README pointing at the real config under /etc/samba/smb.conf. This is
# the SERVER side — for mounting an existing remote Samba share instead, see
# services/vpn-data-mount.sh (deliberately the opposite: reads an existing
# smb.conf over SSH, never installs Samba, never creates or resets a share
# password).
# ── Standalone bootstrap ──────────────────────────────────────────────────────
# Detected when the script is executed directly rather than sourced by setup.sh.
# Sets up helpers and globals, then defers execution until after the function
# definition at the bottom of this file.
if [[ "${BASH_SOURCE[0]}" == "${0}" ]]; then
[[ "$(id -u)" == "0" ]] || { echo "Run with sudo: sudo bash $0"; exit 1; }
_SELF_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
_COMMON="$_SELF_DIR/../lib/common.sh"
if [[ -f "$_COMMON" ]]; then
# Full repo present — use the real helpers (picks up ~/docker/.config too)
# shellcheck source=../lib/common.sh
source "$_COMMON"
else
# One-off copy — inline minimal stubs so the script works without the repo
log_info() { echo -e "\033[0;34m[INFO]\033[0m $*"; }
log_success() { echo -e "\033[0;32m[OK]\033[0m $*"; }
log_warning() { echo -e "\033[1;33m[WARN]\033[0m $*"; }
log_error() { echo -e "\033[0;31m[ERROR]\033[0m $*" >&2; }
ensure_docker_dir_ownership() {
chown -R "$ACTUAL_USER:$ACTUAL_USER" "$@" 2>/dev/null || true
}
prompt_text() {
local _q="$1" _def="$2" _var="$3" _r
[[ "${UNATTENDED:-false}" == "true" ]] && { eval "$_var='$_def'"; return; }
read -r -p " $_q " _r
eval "$_var='${_r:-$_def}'"
}
prompt_yn() {
local _q="$1" _def="$2" _var="$3" _r
[[ "${UNATTENDED:-false}" == "true" ]] && { eval "$_var='$_def'"; return; }
read -r -p " $_q " _r
eval "$_var='${_r:-$_def}'"
}
generate_password() {
local _len="${1:-32}"
tr -dc 'A-Za-z0-9' < /dev/urandom | head -c "$_len"
}
ensure_ufw_enabled() {
command -v ufw &>/dev/null || return 0
ufw status 2>/dev/null | grep -q "Status: active" && return 0
local _ssh_port
_ssh_port="$(grep -iE '^[[:space:]]*Port[[:space:]]+[0-9]+' /etc/ssh/sshd_config 2>/dev/null \
| tail -1 | awk '{print $2}')"
_ssh_port="${_ssh_port:-22}"
ufw allow "${_ssh_port}/tcp" comment 'SSH' >/dev/null 2>&1
ufw --force enable >/dev/null 2>&1
log_success "UFW enabled (SSH on port ${_ssh_port} allowed first, so this won't lock you out)."
}
write_readme() {
local _dir="$1"; shift
mkdir -p "$_dir"
cat > "$_dir/README.md"
}
backup_if_exists() {
local _file="$1"
[ -f "$_file" ] || return 0
cp -p "$_file" "${_file}.bak.$(date +%Y%m%d-%H%M%S)" 2>/dev/null
}
fi
# Globals — ACTUAL_USER/ACTUAL_HOME must come before DOCKER_DIR
# ($HOME under sudo is /root, not the real user's home)
ACTUAL_USER="${ACTUAL_USER:-${SUDO_USER:-$USER}}"
ACTUAL_HOME="$(getent passwd "$ACTUAL_USER" 2>/dev/null | cut -d: -f6 || echo "${HOME:-/root}")"
DOCKER_DIR="${DOCKER_DIR:-$ACTUAL_HOME/docker}"
DRY_RUN="${DRY_RUN:-false}"
UNATTENDED="${UNATTENDED:-false}"
register_service() { :; } # no-op — no wizard to register into
_RUN_STANDALONE=1
fi
# ─────────────────────────────────────────────────────────────────────────────
register_service samba utilities "Samba file sharing (SMB/CIFS) — shares, users, passwords"
install_samba() {
local SMB_CONF="/etc/samba/smb.conf"
local DOCS_DIR="$DOCKER_DIR/samba"
if [ "$DRY_RUN" = true ]; then
echo "[DRY-RUN] Would install samba (smbd/nmbd) if not already present"
echo "[DRY-RUN] Would show any shares this installer already manages"
echo "[DRY-RUN] Would prompt to add one or more shares (path, guest-or-authenticated, users)"
echo "[DRY-RUN] Would create a system Linux account + Samba password for any new user"
echo "[DRY-RUN] Would append share stanzas to $SMB_CONF, validate with testparm, restart smbd/nmbd"
echo "[DRY-RUN] Would open UFW for SMB (137/138 udp, 139/445 tcp) — scoped to the LAN by default"
echo "[DRY-RUN] Would write $DOCS_DIR/README.md (docs only — Samba itself runs natively, not in Docker)"
return 0
fi
if ! command -v smbd &>/dev/null; then
log_info "Installing Samba..."
apt-get update -y
apt-get install -y samba || { log_error "Samba install failed"; return 1; }
log_success "Samba installed"
else
log_success "Samba already installed"
fi
backup_if_exists "$SMB_CONF"
if grep -q '^# ubuntu-post-install:share:' "$SMB_CONF" 2>/dev/null; then
echo ""
log_info "Shares already managed by this installer:"
grep '^# ubuntu-post-install:share:' "$SMB_CONF" | sed 's/^# ubuntu-post-install:share:/ - /'
fi
echo ""
local _added_any=false
while true; do
local ADD_SHARE=""
prompt_yn "Add a Samba share now? (y/n):" "y" ADD_SHARE
[[ "$ADD_SHARE" =~ ^[Yy]$ ]] || break
_samba_add_share "$SMB_CONF" && _added_any=true
echo ""
done
if [ "$_added_any" = true ]; then
log_info "Validating smb.conf..."
if testparm -s "$SMB_CONF" &>/dev/null; then
systemctl restart smbd 2>/dev/null
systemctl restart nmbd 2>/dev/null # NetBIOS name resolution — some Samba packages split this out
log_success "smbd/nmbd restarted with the new configuration"
else
log_error "testparm reports smb.conf is invalid — NOT restarting smbd/nmbd."
log_error "Check manually: sudo testparm -s $SMB_CONF"
return 1
fi
else
log_info "No shares added this run."
fi
_samba_configure_firewall
mkdir -p "$DOCS_DIR"
ensure_docker_dir_ownership "$DOCS_DIR"
write_readme "$DOCS_DIR" << MD
# Samba
Samba runs natively on this box (not in Docker) — the real config is
\`/etc/samba/smb.conf\`, managed by \`systemctl\`. This folder just holds this
README; there's no compose stack here.
## Manage
\`\`\`bash
sudo testparm -s # validate smb.conf before restarting
sudo systemctl restart smbd nmbd
sudo systemctl status smbd
\`\`\`
## Shares
Re-run \`sudo ./setup.sh samba\` (or \`sudo bash services/samba.sh\` standalone)
to add another share or another user — existing shares/users are left alone.
Each share this installer wrote is marked in smb.conf with a
\`# ubuntu-post-install:share:<name>\` comment right above its \`[<name>]\`
stanza, so you can find (or hand-edit / remove) them later.
## Users
Samba users need BOTH a Linux account and a separate Samba password
(\`smbpasswd\`) — they are not the same credential. This installer creates a
system account (\`useradd --system --no-create-home\`, no shell login) for
any username that doesn't already exist as a Linux user, adds it to the
\`sambashare\` group, and sets its Samba password with \`smbpasswd\`.
\`\`\`bash
sudo smbpasswd <username> # change an existing user's Samba password
sudo pdbedit -L # list all Samba users
sudo smbpasswd -x <username> # remove a user from Samba (leaves the Linux account alone)
\`\`\`
## Connecting
- Windows: \`\\\\<server-ip>\\<share-name>\`
- macOS Finder: Go -> Connect to Server -> \`smb://<server-ip>/<share-name>\`
- Linux: \`smbclient //<server-ip>/<share-name> -U <username>\` or mount with
\`mount.cifs\` / \`cifs-utils\` (already installed by \`services/base.sh\`).
## Firewall
SMB (137/138 UDP, 139/445 TCP) should almost never be exposed to the public
internet — this installer scopes the UFW rule to your LAN subnet by default.
Check what's currently allowed with \`sudo ufw status | grep -E '13[7-9]|445'\`.
MD
log_success "Samba configured. Re-run 'sudo ./setup.sh samba' any time to add another share or user."
}
# Appends one [share] stanza to smb.conf. Returns non-zero (and adds nothing)
# on a blank/duplicate name so the caller's "did we actually add one" tracking
# stays accurate.
_samba_add_share() {
local _conf="$1"
local NAME="" SHARE_PATH="" GUEST=""
prompt_text " Share name (letters/numbers/hyphens/underscores, e.g. media):" "" NAME
NAME="$(echo "$NAME" | tr -cd 'A-Za-z0-9_-')"
if [[ -z "$NAME" ]]; then
log_warning "Share name required — skipping."
return 1
fi
if grep -q "^\[$NAME\]\$" "$_conf" 2>/dev/null; then
log_warning "A share named [$NAME] already exists in smb.conf — skipping."
log_warning "Edit $_conf by hand to change it, or pick a different name."
return 1
fi
local DEFAULT_PATH="/srv/samba/$NAME"
prompt_text " Path to share [$DEFAULT_PATH]:" "$DEFAULT_PATH" SHARE_PATH
SHARE_PATH="${SHARE_PATH:-$DEFAULT_PATH}"
SHARE_PATH="${SHARE_PATH/#\~/$ACTUAL_HOME}"
mkdir -p "$SHARE_PATH"
prompt_yn " Allow guest (no password) access to '$NAME'? (y/n):" "n" GUEST
local VALID_USERS=""
if [[ ! "$GUEST" =~ ^[Yy]$ ]]; then
echo " Enter Samba usernames to grant access to '$NAME' (blank to stop):"
while true; do
local SUSER=""
prompt_text " Username:" "" SUSER
[[ -z "$SUSER" ]] && break
_samba_ensure_user "$SUSER"
VALID_USERS="${VALID_USERS:+$VALID_USERS }$SUSER"
done
if [[ -z "$VALID_USERS" ]]; then
log_warning "No users added and guest access declined — '$NAME' will be inaccessible until you add a user (re-run this installer, or edit smb.conf by hand)."
fi
fi
getent group sambashare >/dev/null 2>&1 || groupadd sambashare
if [[ "$GUEST" =~ ^[Yy]$ ]]; then
chmod 0777 "$SHARE_PATH"
else
chgrp sambashare "$SHARE_PATH" 2>/dev/null || true
chmod 0770 "$SHARE_PATH"
fi
{
echo ""
echo "# ubuntu-post-install:share:$NAME"
echo "[$NAME]"
echo " path = $SHARE_PATH"
echo " browseable = yes"
echo " read only = no"
if [[ "$GUEST" =~ ^[Yy]$ ]]; then
echo " guest ok = yes"
else
echo " guest ok = no"
[[ -n "$VALID_USERS" ]] && echo " valid users = $VALID_USERS"
fi
} >> "$_conf"
log_success "Share '$NAME' -> $SHARE_PATH added to smb.conf"
}
# Creates the Linux system account (if missing) and sets a Samba password for
# it. Samba users need BOTH — a Linux account and a separate smbpasswd entry
# — they are not the same credential, and smbpasswd -a fails outright against
# a username with no matching Linux account at all.
_samba_ensure_user() {
local _user="$1"
if ! id "$_user" &>/dev/null; then
log_info "Linux account '$_user' doesn't exist — creating a system account (no shell login, no home dir)."
useradd --system --no-create-home --shell /usr/sbin/nologin "$_user"
fi
getent group sambashare >/dev/null 2>&1 || groupadd sambashare
usermod -aG sambashare "$_user"
if pdbedit -L 2>/dev/null | cut -d: -f1 | grep -qx "$_user"; then
log_info "Samba password already set for '$_user' — leaving as-is (change it later with: sudo smbpasswd $_user)."
return 0
fi
local _pass _entered=""
_pass="$(generate_password 16)"
prompt_text " Samba password for '$_user' [$_pass]:" "$_pass" _entered
_pass="${_entered:-$_pass}"
if printf '%s\n%s\n' "$_pass" "$_pass" | smbpasswd -s -a "$_user" >/dev/null 2>&1 \
&& smbpasswd -e "$_user" >/dev/null 2>&1; then
log_success "Samba user '$_user' set — password: $_pass (write this down, it isn't stored anywhere else)"
else
log_warning "Failed to set Samba password for '$_user' — set it manually: sudo smbpasswd $_user"
fi
}
# SMB should almost never face the public internet — scope the UFW rule to
# the LAN by default (LAN-subnet detection borrowed from the same pattern
# services/asterisk.sh uses for its VLAN/local-network prompt).
_samba_configure_firewall() {
command -v ufw &>/dev/null || {
log_warning "ufw not installed — if you use a firewall, open TCP 139/445 and UDP 137/138 for SMB (LAN only, never the internet)."
return 0
}
echo ""
local DETECTED_NETS DEFAULT_SUBNET=""
DETECTED_NETS="$(ip -o -f inet addr show scope global 2>/dev/null \
| awk '{print $2, $4}' \
| grep -Ev '^(docker|br-|veth|tun|tap|wg)' \
| awk '{ split($2,a,"/"); split(a[1],o,"."); print o[1]"."o[2]"."o[3]".0/"a[2] }' \
| sort -u)"
DEFAULT_SUBNET="$(echo "$DETECTED_NETS" | head -1)"
local RESTRICT_LAN=""
prompt_yn "Restrict Samba access to your local network only (recommended — SMB should never face the internet)? (y/n):" "y" RESTRICT_LAN
if [[ "$RESTRICT_LAN" =~ ^[Yy]$ ]]; then
local SUBNET=""
prompt_text " LAN subnet to allow (CIDR)${DEFAULT_SUBNET:+ [$DEFAULT_SUBNET]}:" "$DEFAULT_SUBNET" SUBNET
SUBNET="${SUBNET:-$DEFAULT_SUBNET}"
if [[ -z "$SUBNET" ]]; then
log_warning "No subnet given — skipping UFW rules. Open them manually if needed."
return 0
fi
local p
for p in 137 138; do
ufw allow from "$SUBNET" to any port "$p" proto udp comment "Samba (LAN)" >/dev/null 2>&1
done
for p in 139 445; do
ufw allow from "$SUBNET" to any port "$p" proto tcp comment "Samba (LAN)" >/dev/null 2>&1
done
log_success "UFW: Samba opened to $SUBNET only"
else
log_warning "Opening Samba to ALL sources — not recommended, SMB has a long history of remote exploits."
ufw allow 137/udp comment "Samba" >/dev/null 2>&1
ufw allow 138/udp comment "Samba" >/dev/null 2>&1
ufw allow 139/tcp comment "Samba" >/dev/null 2>&1
ufw allow 445/tcp comment "Samba" >/dev/null 2>&1
log_success "UFW: Samba opened (unrestricted)"
fi
ensure_ufw_enabled
}
[[ "${_RUN_STANDALONE:-0}" == 1 ]] && install_samba
+98 -3034
View File
File diff suppressed because it is too large Load Diff
+1 -2
View File
@@ -121,7 +121,6 @@ is_installed() {
base) command -v ncdu >/dev/null 2>&1 ;;
glow) command -v glow >/dev/null 2>&1 ;;
crowdsec) command -v cscli >/dev/null 2>&1 ;;
samba) command -v smbd >/dev/null 2>&1 ;;
security-dashboard) [ -f /opt/security-dashboard/app.py ] ;;
kdeconnect) command -v kdeconnect >/dev/null 2>&1 ;;
silent-send) [ -d "$ACTUAL_HOME/silent-send/.git" ] ;;
@@ -157,7 +156,7 @@ is_installed() {
# is_installed() as 0 or 1.
install_count() {
case "$1" in
base|glow|crowdsec|samba|security-dashboard|kdeconnect|silent-send|sync-cc|claude-cli|sky-cam|sky-cam-frigate|asterisk|pstn-trunk|sms-inbound|ssh-config|ssh-key-import)
base|glow|crowdsec|security-dashboard|kdeconnect|silent-send|sync-cc|claude-cli|sky-cam|sky-cam-frigate|asterisk|pstn-trunk|sms-inbound|ssh-config|ssh-key-import)
is_installed "$1" && echo 1 || echo 0 ;;
wordpress)
find "$DOCKER_DIR" -mindepth 1 -maxdepth 1 -name 'wordpress-*' -type d 2>/dev/null | wc -l ;;
-420
View File
@@ -1,420 +0,0 @@
#!/usr/bin/env python3
"""tools/anki-deck-math.py — Generate math-fact Anki decks (.apkg) with a
vertical/stacked problem layout, Anki's built-in type-the-answer input, and
Piper (offline, local neural TTS) audio on both the question and answer
side of every card.
Standalone content-generation tool, unrelated to this repo's services/*.sh
installers — run it on any machine with Python (your desktop, laptop, or
the same box running services/anki-sync-server.sh), then import the
resulting .apkg into Anki (File -> Import) or push it into a sync-server
account with AnkiConnect's importPackage action. See services/anki-sync-server.sh
and services/anki-progress.sh for the actual self-hosted sync backend and
progress dashboard this content is meant to be studied through.
Decks:
multiplication 1-12, all 144 ordered pairs (a x b), shuffled
(not sequential — see the note near
random.shuffle(pairs) below for why)
division inverse of the multiplication deck (144 facts)
addsub --lo L --hi H addition + subtraction fact family for [L, H]
(subtraction facts derived from the addition
facts, e.g. 7+3=10 also gives 10-7=3 and
10-3=7 — never negative results)
fractions reducing fractions to lowest terms (denominators 2-12)
decimals fraction -> decimal conversion (only denominators
whose decimal expansion terminates: 2,4,5,8,10,20,25)
Setup (one time):
python3 -m venv ~/anki-deck-venv
source ~/anki-deck-venv/bin/activate
pip install genanki piper-tts
# Download at least one voice (one time per voice you want to try —
# download_voices saves into the CURRENT directory by default, so cd
# somewhere sensible first, e.g. your home directory):
python3 -m piper.download_voices en_US-lessac-medium
# Other options: en_US-amy-medium (warm/friendly), en_US-ryan-high
# (best-quality US male), en_US-libritts_r-medium (multi-speaker),
# en_GB-alba-medium / en_GB-cori-high (British accent). Tiers are
# low < medium < high — higher sounds more natural but is bigger/slower.
# Sanity-check the voice before generating a full deck's worth of clips:
echo "three times seven" | python3 -m piper -m en_US-lessac-medium.onnx -f /tmp/test.wav
# play /tmp/test.wav and confirm it sounds right first.
Usage (run with the venv activated):
python3 anki-deck-math.py --deck multiplication
python3 anki-deck-math.py --deck division
python3 anki-deck-math.py --deck addsub --lo 3 --hi 7
python3 anki-deck-math.py --deck addsub --lo 3 --hi 13
python3 anki-deck-math.py --deck addsub --lo 2 --hi 21
python3 anki-deck-math.py --deck fractions
python3 anki-deck-math.py --deck decimals
(add --voice en_US-amy-medium etc. to any of the above to use a voice other
than the default en_US-lessac-medium; --model-path to point at a voice
file directly if it's not found in any of the usual places checked
automatically; --dry-run-tts to test the deck-building logic itself
without Piper or any voice model at all, using silent placeholder audio)
The addsub --hi 21 deck generates ~1600 audio clips and will take noticeably
longer than the others — consider `nohup python3 anki-deck-math.py --deck
addsub --lo 2 --hi 21 > addsub.log 2>&1 &` if you don't want to wait on it.
"""
import argparse
import hashlib
import genanki
import math
import os
import random
import subprocess
parser = argparse.ArgumentParser()
parser.add_argument("--deck", required=True,
choices=["multiplication", "division", "addsub", "fractions", "decimals"])
parser.add_argument("--lo", type=int, default=None, help="addsub only: low end of range")
parser.add_argument("--hi", type=int, default=None, help="addsub only: high end of range")
parser.add_argument("--voice", default="en_US-lessac-medium")
parser.add_argument("--model-path", default=None)
parser.add_argument("--dry-run-tts", action="store_true",
help="Skip Piper entirely and write silent placeholder audio instead"
" (for testing the deck-building logic without a voice model).")
args = parser.parse_args()
if args.deck == "addsub":
if args.lo is None or args.hi is None:
raise SystemExit("--deck addsub requires --lo and --hi, e.g. --lo 3 --hi 7")
if args.lo >= args.hi:
raise SystemExit("--lo must be less than --hi")
SCRATCH = os.path.dirname(os.path.abspath(__file__))
# ─── Voice resolution (same search order as the multiplication script) ──────
_CANDIDATES = [
args.model_path,
f"{args.voice}.onnx",
os.path.join(SCRATCH, f"{args.voice}.onnx"),
os.path.expanduser(f"~/{args.voice}.onnx"),
os.path.expanduser(f"~/.local/share/piper/voices/{args.voice}.onnx"),
]
VOICE_MODEL = next((p for p in _CANDIDATES if p and os.path.isfile(p)), None)
if VOICE_MODEL is None and not args.dry_run_tts:
raise SystemExit(
f"Voice model for '{args.voice}' not found. Checked:\n"
+ "\n".join(f" {p}" for p in _CANDIDATES if p)
+ f"\n\nFind it with: find / -iname '{args.voice}.onnx' 2>/dev/null"
+ "\nThen pass its exact path with --model-path /the/real/path.onnx"
+ "\n(or pass --dry-run-tts to test deck-building without any voice at all)"
)
def piper_tts(text: str, out_path: str) -> None:
if args.dry_run_tts:
# 44-byte minimal valid WAV header, zero samples — enough for genanki
# to accept it as a real media file without needing Piper installed.
with open(out_path, "wb") as f:
f.write(
b"RIFF$\x00\x00\x00WAVEfmt \x10\x00\x00\x00\x01\x00\x01\x00"
b"\x22\x56\x00\x00\x44\xac\x00\x00\x02\x00\x10\x00data\x00\x00\x00\x00"
)
return
subprocess.run(
["python3", "-m", "piper", "-m", VOICE_MODEL, "-f", out_path],
input=text.encode("utf-8"),
check=True,
capture_output=True,
)
# ─── Number -> words ─────────────────────────────────────────────────────────
ONES = ["zero", "one", "two", "three", "four", "five", "six", "seven",
"eight", "nine", "ten", "eleven", "twelve", "thirteen", "fourteen",
"fifteen", "sixteen", "seventeen", "eighteen", "nineteen"]
TENS = ["", "", "twenty", "thirty", "forty", "fifty", "sixty", "seventy",
"eighty", "ninety"]
def num2words(n):
if n < 0:
return "negative " + num2words(-n)
if n < 20:
return ONES[n]
if n < 100:
t, o = divmod(n, 10)
return TENS[t] + ("-" + ONES[o] if o else "")
h, rem = divmod(n, 100)
return ONES[h] + " hundred" + (" " + num2words(rem) if rem else "")
_NUM_CHECKS = {0: "zero", 9: "nine", 10: "ten", 13: "thirteen", 20: "twenty",
21: "twenty-one", 45: "forty-five", 99: "ninety-nine",
100: "one hundred", 110: "one hundred ten",
121: "one hundred twenty-one", 144: "one hundred forty-four",
441: "four hundred forty-one"}
for _n, _w in _NUM_CHECKS.items():
assert num2words(_n) == _w, f"num2words({_n}) = {num2words(_n)!r}, expected {_w!r}"
# Ordinal words, singular form, denominators 2-21 (covers every deck below).
# Irregular forms (half, third, fifth, eighth, ninth, twelfth) are real
# English irregularities, not a suffix rule, so this is a lookup table, not
# a formula — a formula would get exactly these wrong.
ORDINAL_SINGULAR = {
2: "half", 3: "third", 4: "fourth", 5: "fifth", 6: "sixth",
7: "seventh", 8: "eighth", 9: "ninth", 10: "tenth", 11: "eleventh",
12: "twelfth", 13: "thirteenth", 14: "fourteenth", 15: "fifteenth",
16: "sixteenth", 17: "seventeenth", 18: "eighteenth", 19: "nineteenth",
20: "twentieth", 21: "twenty-first", 25: "twenty-fifth", 50: "fiftieth",
100: "hundredth",
}
def ordinal_plural(n):
s = ORDINAL_SINGULAR[n]
return "halves" if s == "half" else s + "s"
def fraction_words(num, den):
"""'three fourths', 'one half', 'seven tenths'."""
ord_word = ORDINAL_SINGULAR[den] if num == 1 else ordinal_plural(den)
return f"{num2words(num)} {ord_word}"
_FRAC_CHECKS = {
(1, 2): "one half", (3, 4): "three fourths", (1, 4): "one fourth",
(7, 10): "seven tenths", (1, 3): "one third", (2, 3): "two thirds",
(5, 8): "five eighths", (1, 8): "one eighth",
}
for (_n, _d), _w in _FRAC_CHECKS.items():
assert fraction_words(_n, _d) == _w, f"fraction_words({_n},{_d}) = {fraction_words(_n, _d)!r}, expected {_w!r}"
def decimal_words(decimal_str):
"""'0.25' -> 'zero point two five' (each digit spoken individually,
avoids any ambiguity between e.g. 'point two five' vs 'twenty-five
hundredths')."""
whole, frac = decimal_str.split(".")
digit_words = " ".join(ONES[int(d)] for d in frac)
return f"{num2words(int(whole))} point {digit_words}"
assert decimal_words("0.25") == "zero point two five"
assert decimal_words("0.5") == "zero point five"
assert decimal_words("0.375") == "zero point three seven five"
# ─── Shared genanki model builder ────────────────────────────────────────────
# Every deck here renders as two stacked lines with a line under them (same
# visual language as the original multiplication deck): TOP over BOTTOM,
# with an optional prefix (operator) on the bottom line. Fractions/decimals
# reuse the exact same layout as numerator-over-denominator.
def build_model(deck_key):
voice_hash = int(hashlib.sha256(f"{deck_key}:{args.voice}".encode()).hexdigest(), 16)
model_id = 1_600_000_000 + (voice_hash % 90_000_000)
return model_id, genanki.Model(
model_id,
f"Math Fact ({deck_key}, {args.voice})",
fields=[{"name": "Top"}, {"name": "Bottom"}, {"name": "Answer"},
{"name": "QSound"}, {"name": "ASound"}],
templates=[{
"name": "Card",
"qfmt": """
<div class="problem">
<div class="line1">{{Top}}</div>
<div class="line2">{{Bottom}}</div>
<div class="rule"></div>
</div>
{{QSound}}
{{type:Answer}}
""",
"afmt": """
<div class="problem">
<div class="line1">{{Top}}</div>
<div class="line2">{{Bottom}}</div>
<div class="rule"></div>
</div>
<hr id="answer">
{{type:Answer}}
{{ASound}}
""",
}],
css="""
.card { font-family: Arial, sans-serif; font-size: 28px; text-align: center; }
.problem { display: inline-block; text-align: right; margin: 20px auto; }
.line1, .line2 { font-size: 48px; padding: 2px 10px; }
.rule { border-top: 3px solid black; margin-top: 4px; width: 100%; }
""",
)
def build_deck(deck_key, deck_title):
voice_hash = int(hashlib.sha256(f"{deck_key}:{args.voice}".encode()).hexdigest(), 16)
deck_id = 2_000_000_000 + (voice_hash % 90_000_000)
return genanki.Deck(deck_id, deck_title)
def add_note(deck, model, top, bottom, answer, qtext, atext, media_files, tag):
qfile = f"q_{tag}.wav"
afile = f"a_{tag}.wav"
qpath = os.path.join(MEDIA_DIR, qfile)
apath = os.path.join(MEDIA_DIR, afile)
piper_tts(qtext, qpath)
piper_tts(atext, apath)
media_files += [qpath, apath]
deck.add_note(genanki.Note(
model=model,
fields=[top, bottom, answer, f"[sound:{qfile}]", f"[sound:{afile}]"],
))
# ─── Per-deck generators ─────────────────────────────────────────────────────
def gen_multiplication():
deck_key = "multiplication"
model_id, model = build_model(deck_key)
deck = build_deck(deck_key, "Multiplication Facts (1-12)")
media_files = []
pairs = [(a, b) for a in range(1, 13) for b in range(1, 13)]
random.seed(42)
random.shuffle(pairs)
for a, b in pairs:
ans = a * b
add_note(deck, model, str(a), f"&times; {b}", str(ans),
f"{num2words(a)} times {num2words(b)}", num2words(ans),
media_files, f"mul_{a}_{b}")
return deck, media_files, len(pairs)
def gen_division():
deck_key = "division"
model_id, model = build_model(deck_key)
deck = build_deck(deck_key, "Division Facts (inverse of 1-12 times tables)")
media_files = []
# Same (a, b) pairs as multiplication: product / a = b. This is the
# direct inverse of every multiplication card in that deck.
pairs = [(a, b) for a in range(1, 13) for b in range(1, 13)]
random.seed(43)
random.shuffle(pairs)
for a, b in pairs:
product = a * b
add_note(deck, model, str(product), f"&divide; {a}", str(b),
f"{num2words(product)} divided by {num2words(a)}", num2words(b),
media_files, f"div_{a}_{b}")
return deck, media_files, len(pairs)
def gen_addsub(lo, hi):
deck_key = f"addsub_{lo}_{hi}"
model_id, model = build_model(deck_key)
deck = build_deck(deck_key, f"Addition & Subtraction Facts ({lo}-{hi})")
media_files = []
add_pairs = [(a, b) for a in range(lo, hi + 1) for b in range(lo, hi + 1)]
random.seed(hash((lo, hi)) & 0xFFFFFFFF)
random.shuffle(add_pairs)
sub_facts = [] # (minuend, subtrahend, answer)
seen = set()
for a, b in add_pairs:
c = a + b
for minuend, subtrahend, answer in ((c, a, b), (c, b, a)):
key = (minuend, subtrahend)
if key not in seen:
seen.add(key)
sub_facts.append((minuend, subtrahend, answer))
random.shuffle(sub_facts)
count = 0
for a, b in add_pairs:
ans = a + b
add_note(deck, model, str(a), f"+ {b}", str(ans),
f"{num2words(a)} plus {num2words(b)}", num2words(ans),
media_files, f"add_{lo}_{hi}_{a}_{b}")
count += 1
for minuend, subtrahend, answer in sub_facts:
add_note(deck, model, str(minuend), f"&minus; {subtrahend}", str(answer),
f"{num2words(minuend)} minus {num2words(subtrahend)}", num2words(answer),
media_files, f"sub_{lo}_{hi}_{minuend}_{subtrahend}")
count += 1
return deck, media_files, count
def gen_fractions():
deck_key = "fractions"
model_id, model = build_model(deck_key)
deck = build_deck(deck_key, "Reducing Fractions to Lowest Terms")
media_files = []
facts = []
for den in range(2, 13):
for num in range(1, den):
g = math.gcd(num, den)
if g > 1:
facts.append((num, den, num // g, den // g))
random.seed(44)
random.shuffle(facts)
for num, den, rnum, rden in facts:
answer = f"{rnum}/{rden}"
add_note(deck, model, str(num), f"&frasl; {den}", answer,
fraction_words(num, den), fraction_words(rnum, rden),
media_files, f"frac_{num}_{den}")
return deck, media_files, len(facts)
def gen_decimals():
deck_key = "decimals"
model_id, model = build_model(deck_key)
deck = build_deck(deck_key, "Fraction to Decimal Conversion")
media_files = []
# Only denominators whose only prime factors are 2 and 5 terminate in a
# finite decimal (1/3 = 0.333... never terminates) — restricting to
# these avoids ever needing to round/repeat.
facts = []
for den in (2, 4, 5, 8, 10, 20, 25):
for num in range(1, den):
if math.gcd(num, den) != 1:
continue # skip non-lowest-terms fractions (already covered by the fractions deck)
value = num / den
decimal_str = f"{value:.10f}".rstrip("0")
if decimal_str.endswith("."):
decimal_str += "0"
facts.append((num, den, decimal_str))
random.seed(45)
random.shuffle(facts)
for num, den, decimal_str in facts:
add_note(deck, model, str(num), f"&frasl; {den}", decimal_str,
fraction_words(num, den), decimal_words(decimal_str),
media_files, f"dec_{num}_{den}")
return deck, media_files, len(facts)
# ─── Dispatch ─────────────────────────────────────────────────────────────────
if args.deck == "addsub":
deck_key = f"addsub_{args.lo}_{args.hi}"
else:
deck_key = args.deck
MEDIA_DIR = os.path.join(SCRATCH, f"media_{deck_key}_{args.voice}")
os.makedirs(MEDIA_DIR, exist_ok=True)
GENERATORS = {
"multiplication": lambda: gen_multiplication(),
"division": lambda: gen_division(),
"addsub": lambda: gen_addsub(args.lo, args.hi),
"fractions": lambda: gen_fractions(),
"decimals": lambda: gen_decimals(),
}
deck, media_files, count = GENERATORS[args.deck]()
if count == 0:
raise SystemExit(f"No cards generated for --deck {args.deck} — check the range/args.")
package = genanki.Package(deck)
package.media_files = media_files
out_path = os.path.join(SCRATCH, f"{deck_key}_{args.voice}.apkg")
package.write_to_file(out_path)
size_mb = os.path.getsize(out_path) / (1024 * 1024)
print(f"\nDone: {out_path} ({size_mb:.1f} MB, {count} cards, {len(media_files)} audio clips)")
-385
View File
@@ -1,385 +0,0 @@
#!/usr/bin/env python3
"""tools/anki-deck-periodic.py — Generate periodic table Anki decks (.apkg)
with Anki's built-in type-the-answer input and Piper (offline, local
neural TTS) audio on both sides. See tools/anki-deck-math.py's docstring
for one-time setup (venv, genanki + piper-tts, downloading a voice) — same
steps apply here, this is a standalone, self-contained script otherwise.
Decks:
prehs symbol<->name, elements 1-36 (H through Kr)
hs symbol<->name plus number->symbol, all 118 elements
category element category as multiple choice (A/B/C/D shown as
plain text options — not a clickable UI, since that needs
a desktop-only Anki add-on and would break on
AnkiDroid/AnkiMobile), type the letter — only elements
with a confirmed category (excludes 8 very recent
superheavy elements whose category is still officially
unconfirmed)
Element data: Bowserinator/Periodic-Table-JSON (a widely used, actively
maintained public dataset), fetched and spot-checked against known facts
before being embedded below — not typed from memory.
Usage (run with the venv from anki-deck-math.py's docstring activated):
python3 anki-deck-periodic.py --deck prehs
python3 anki-deck-periodic.py --deck hs
python3 anki-deck-periodic.py --deck category
(add --voice en_US-amy-medium etc.; --model-path if a voice isn't found
automatically; --dry-run-tts to test the deck-building logic without any
voice model at all, using silent placeholder audio)
"""
import argparse
import hashlib
import genanki
import os
import random
import subprocess
parser = argparse.ArgumentParser()
parser.add_argument("--deck", required=True, choices=["prehs", "hs", "category"])
parser.add_argument("--voice", default="en_US-lessac-medium")
parser.add_argument("--model-path", default=None)
parser.add_argument("--dry-run-tts", action="store_true")
args = parser.parse_args()
SCRATCH = os.path.dirname(os.path.abspath(__file__))
_CANDIDATES = [
args.model_path,
f"{args.voice}.onnx",
os.path.join(SCRATCH, f"{args.voice}.onnx"),
os.path.expanduser(f"~/{args.voice}.onnx"),
os.path.expanduser(f"~/.local/share/piper/voices/{args.voice}.onnx"),
]
VOICE_MODEL = next((p for p in _CANDIDATES if p and os.path.isfile(p)), None)
if VOICE_MODEL is None and not args.dry_run_tts:
raise SystemExit(
f"Voice model for '{args.voice}' not found. Checked:\n"
+ "\n".join(f" {p}" for p in _CANDIDATES if p)
+ f"\n\nFind it with: find / -iname '{args.voice}.onnx' 2>/dev/null"
+ "\nThen pass its exact path with --model-path /the/real/path.onnx"
+ "\n(or pass --dry-run-tts to test deck-building without any voice at all)"
)
def piper_tts(text: str, out_path: str) -> None:
if args.dry_run_tts:
with open(out_path, "wb") as f:
f.write(
b"RIFF$\x00\x00\x00WAVEfmt \x10\x00\x00\x00\x01\x00\x01\x00"
b"\x22\x56\x00\x00\x44\xac\x00\x00\x02\x00\x10\x00data\x00\x00\x00\x00"
)
return
subprocess.run(
["python3", "-m", "piper", "-m", VOICE_MODEL, "-f", out_path],
input=text.encode("utf-8"),
check=True,
capture_output=True,
)
ONES = ["zero", "one", "two", "three", "four", "five", "six", "seven",
"eight", "nine", "ten", "eleven", "twelve", "thirteen", "fourteen",
"fifteen", "sixteen", "seventeen", "eighteen", "nineteen"]
TENS = ["", "", "twenty", "thirty", "forty", "fifty", "sixty", "seventy",
"eighty", "ninety"]
def num2words(n):
if n < 20:
return ONES[n]
if n < 100:
t, o = divmod(n, 10)
return TENS[t] + ("-" + ONES[o] if o else "")
h, rem = divmod(n, 100)
return ONES[h] + " hundred" + (" " + num2words(rem) if rem else "")
assert num2words(1) == "one"
assert num2words(26) == "twenty-six"
assert num2words(118) == "one hundred eighteen"
# ─── Element data: (atomic_number, symbol, name, category-or-None) ─────────
# category is None for the 8 most recently synthesized superheavy elements
# whose chemical category is still officially unconfirmed (excluded from
# the category deck below, still included in prehs/hs symbol/name/number).
ELEMENTS = [
(1, 'H', 'Hydrogen', 'diatomic nonmetal'),
(2, 'He', 'Helium', 'noble gas'),
(3, 'Li', 'Lithium', 'alkali metal'),
(4, 'Be', 'Beryllium', 'alkaline earth metal'),
(5, 'B', 'Boron', 'metalloid'),
(6, 'C', 'Carbon', 'polyatomic nonmetal'),
(7, 'N', 'Nitrogen', 'diatomic nonmetal'),
(8, 'O', 'Oxygen', 'diatomic nonmetal'),
(9, 'F', 'Fluorine', 'diatomic nonmetal'),
(10, 'Ne', 'Neon', 'noble gas'),
(11, 'Na', 'Sodium', 'alkali metal'),
(12, 'Mg', 'Magnesium', 'alkaline earth metal'),
(13, 'Al', 'Aluminium', 'post-transition metal'),
(14, 'Si', 'Silicon', 'metalloid'),
(15, 'P', 'Phosphorus', 'polyatomic nonmetal'),
(16, 'S', 'Sulfur', 'polyatomic nonmetal'),
(17, 'Cl', 'Chlorine', 'diatomic nonmetal'),
(18, 'Ar', 'Argon', 'noble gas'),
(19, 'K', 'Potassium', 'alkali metal'),
(20, 'Ca', 'Calcium', 'alkaline earth metal'),
(21, 'Sc', 'Scandium', 'transition metal'),
(22, 'Ti', 'Titanium', 'transition metal'),
(23, 'V', 'Vanadium', 'transition metal'),
(24, 'Cr', 'Chromium', 'transition metal'),
(25, 'Mn', 'Manganese', 'transition metal'),
(26, 'Fe', 'Iron', 'transition metal'),
(27, 'Co', 'Cobalt', 'transition metal'),
(28, 'Ni', 'Nickel', 'transition metal'),
(29, 'Cu', 'Copper', 'transition metal'),
(30, 'Zn', 'Zinc', 'transition metal'),
(31, 'Ga', 'Gallium', 'post-transition metal'),
(32, 'Ge', 'Germanium', 'metalloid'),
(33, 'As', 'Arsenic', 'metalloid'),
(34, 'Se', 'Selenium', 'polyatomic nonmetal'),
(35, 'Br', 'Bromine', 'diatomic nonmetal'),
(36, 'Kr', 'Krypton', 'noble gas'),
(37, 'Rb', 'Rubidium', 'alkali metal'),
(38, 'Sr', 'Strontium', 'alkaline earth metal'),
(39, 'Y', 'Yttrium', 'transition metal'),
(40, 'Zr', 'Zirconium', 'transition metal'),
(41, 'Nb', 'Niobium', 'transition metal'),
(42, 'Mo', 'Molybdenum', 'transition metal'),
(43, 'Tc', 'Technetium', 'transition metal'),
(44, 'Ru', 'Ruthenium', 'transition metal'),
(45, 'Rh', 'Rhodium', 'transition metal'),
(46, 'Pd', 'Palladium', 'transition metal'),
(47, 'Ag', 'Silver', 'transition metal'),
(48, 'Cd', 'Cadmium', 'transition metal'),
(49, 'In', 'Indium', 'post-transition metal'),
(50, 'Sn', 'Tin', 'post-transition metal'),
(51, 'Sb', 'Antimony', 'metalloid'),
(52, 'Te', 'Tellurium', 'metalloid'),
(53, 'I', 'Iodine', 'diatomic nonmetal'),
(54, 'Xe', 'Xenon', 'noble gas'),
(55, 'Cs', 'Cesium', 'alkali metal'),
(56, 'Ba', 'Barium', 'alkaline earth metal'),
(57, 'La', 'Lanthanum', 'lanthanide'),
(58, 'Ce', 'Cerium', 'lanthanide'),
(59, 'Pr', 'Praseodymium', 'lanthanide'),
(60, 'Nd', 'Neodymium', 'lanthanide'),
(61, 'Pm', 'Promethium', 'lanthanide'),
(62, 'Sm', 'Samarium', 'lanthanide'),
(63, 'Eu', 'Europium', 'lanthanide'),
(64, 'Gd', 'Gadolinium', 'lanthanide'),
(65, 'Tb', 'Terbium', 'lanthanide'),
(66, 'Dy', 'Dysprosium', 'lanthanide'),
(67, 'Ho', 'Holmium', 'lanthanide'),
(68, 'Er', 'Erbium', 'lanthanide'),
(69, 'Tm', 'Thulium', 'lanthanide'),
(70, 'Yb', 'Ytterbium', 'lanthanide'),
(71, 'Lu', 'Lutetium', 'lanthanide'),
(72, 'Hf', 'Hafnium', 'transition metal'),
(73, 'Ta', 'Tantalum', 'transition metal'),
(74, 'W', 'Tungsten', 'transition metal'),
(75, 'Re', 'Rhenium', 'transition metal'),
(76, 'Os', 'Osmium', 'transition metal'),
(77, 'Ir', 'Iridium', 'transition metal'),
(78, 'Pt', 'Platinum', 'transition metal'),
(79, 'Au', 'Gold', 'transition metal'),
(80, 'Hg', 'Mercury', 'transition metal'),
(81, 'Tl', 'Thallium', 'post-transition metal'),
(82, 'Pb', 'Lead', 'post-transition metal'),
(83, 'Bi', 'Bismuth', 'post-transition metal'),
(84, 'Po', 'Polonium', 'post-transition metal'),
(85, 'At', 'Astatine', 'diatomic nonmetal'),
(86, 'Rn', 'Radon', 'noble gas'),
(87, 'Fr', 'Francium', 'alkali metal'),
(88, 'Ra', 'Radium', 'alkaline earth metal'),
(89, 'Ac', 'Actinium', 'actinide'),
(90, 'Th', 'Thorium', 'actinide'),
(91, 'Pa', 'Protactinium', 'actinide'),
(92, 'U', 'Uranium', 'actinide'),
(93, 'Np', 'Neptunium', 'actinide'),
(94, 'Pu', 'Plutonium', 'actinide'),
(95, 'Am', 'Americium', 'actinide'),
(96, 'Cm', 'Curium', 'actinide'),
(97, 'Bk', 'Berkelium', 'actinide'),
(98, 'Cf', 'Californium', 'actinide'),
(99, 'Es', 'Einsteinium', 'actinide'),
(100, 'Fm', 'Fermium', 'actinide'),
(101, 'Md', 'Mendelevium', 'actinide'),
(102, 'No', 'Nobelium', 'actinide'),
(103, 'Lr', 'Lawrencium', 'actinide'),
(104, 'Rf', 'Rutherfordium', 'transition metal'),
(105, 'Db', 'Dubnium', 'transition metal'),
(106, 'Sg', 'Seaborgium', 'transition metal'),
(107, 'Bh', 'Bohrium', 'transition metal'),
(108, 'Hs', 'Hassium', 'transition metal'),
(109, 'Mt', 'Meitnerium', None),
(110, 'Ds', 'Darmstadtium', None),
(111, 'Rg', 'Roentgenium', None),
(112, 'Cn', 'Copernicium', None),
(113, 'Nh', 'Nihonium', 'post-transition metal'),
(114, 'Fl', 'Flerovium', 'post-transition metal'),
(115, 'Mc', 'Moscovium', None),
(116, 'Lv', 'Livermorium', None),
(117, 'Ts', 'Tennessine', None),
(118, 'Og', 'Oganesson', None),
]
assert len(ELEMENTS) == 118
assert [e[0] for e in ELEMENTS] == list(range(1, 119))
assert ELEMENTS[0] == (1, 'H', 'Hydrogen', 'diatomic nonmetal')
assert ELEMENTS[25] == (26, 'Fe', 'Iron', 'transition metal')
assert ELEMENTS[-1] == (118, 'Og', 'Oganesson', None)
ALL_CATEGORIES = sorted({e[3] for e in ELEMENTS if e[3] is not None})
def build_model(deck_key):
voice_hash = int(hashlib.sha256(f"{deck_key}:{args.voice}".encode()).hexdigest(), 16)
model_id = 1_700_000_000 + (voice_hash % 90_000_000)
return model_id, genanki.Model(
model_id,
f"Periodic Table ({deck_key}, {args.voice})",
fields=[{"name": "Prompt"}, {"name": "Answer"}, {"name": "QSound"}, {"name": "ASound"}],
templates=[{
"name": "Card",
"qfmt": """
<div class="prompt">{{Prompt}}</div>
{{QSound}}
{{type:Answer}}
""",
"afmt": """
<div class="prompt">{{Prompt}}</div>
<hr id="answer">
{{type:Answer}}
{{ASound}}
""",
}],
css="""
.card { font-family: Arial, sans-serif; font-size: 26px; text-align: center; }
.prompt { font-size: 40px; margin: 20px auto; white-space: pre-line; }
""",
)
def build_deck(deck_key, deck_title):
voice_hash = int(hashlib.sha256(f"{deck_key}:{args.voice}".encode()).hexdigest(), 16)
deck_id = 2_100_000_000 + (voice_hash % 90_000_000)
return genanki.Deck(deck_id, deck_title)
def add_note(deck, model, prompt, answer, qtext, atext, media_files, tag):
qfile = f"q_{tag}.wav"
afile = f"a_{tag}.wav"
qpath = os.path.join(MEDIA_DIR, qfile)
apath = os.path.join(MEDIA_DIR, afile)
piper_tts(qtext, qpath)
piper_tts(atext, apath)
media_files += [qpath, apath]
deck.add_note(genanki.Note(
model=model,
fields=[prompt, answer, f"[sound:{qfile}]", f"[sound:{afile}]"],
))
def gen_prehs():
deck_key = "periodic_prehs"
model_id, model = build_model(deck_key)
deck = build_deck(deck_key, "Periodic Table: Symbols & Names (1-36)")
media_files = []
subset = [e for e in ELEMENTS if e[0] <= 36]
cards = []
for number, symbol, name, category in subset:
cards.append(("symbol_to_name", number, symbol, name))
cards.append(("name_to_symbol", number, symbol, name))
random.seed(50)
random.shuffle(cards)
for kind, number, symbol, name in cards:
if kind == "symbol_to_name":
add_note(deck, model, symbol, name,
f"What element has the symbol {symbol}?", name,
media_files, f"prehs_s2n_{number}")
else:
add_note(deck, model, name, symbol,
f"What is the symbol for {name}?", symbol,
media_files, f"prehs_n2s_{number}")
return deck, media_files, len(cards)
def gen_hs():
deck_key = "periodic_hs"
model_id, model = build_model(deck_key)
deck = build_deck(deck_key, "Periodic Table: Symbols, Names & Numbers (1-118)")
media_files = []
cards = []
for number, symbol, name, category in ELEMENTS:
cards.append(("symbol_to_name", number, symbol, name))
cards.append(("name_to_symbol", number, symbol, name))
cards.append(("number_to_symbol", number, symbol, name))
random.seed(51)
random.shuffle(cards)
for kind, number, symbol, name in cards:
if kind == "symbol_to_name":
add_note(deck, model, symbol, name,
f"What element has the symbol {symbol}?", name,
media_files, f"hs_s2n_{number}")
elif kind == "name_to_symbol":
add_note(deck, model, name, symbol,
f"What is the symbol for {name}?", symbol,
media_files, f"hs_n2s_{number}")
else:
add_note(deck, model, f"Element #{number}", symbol,
f"What is the symbol for element number {num2words(number)}?", symbol,
media_files, f"hs_num2s_{number}")
return deck, media_files, len(cards)
def gen_category():
deck_key = "periodic_category"
model_id, model = build_model(deck_key)
deck = build_deck(deck_key, "Periodic Table: Element Categories (multiple choice)")
media_files = []
subset = [e for e in ELEMENTS if e[3] is not None]
random.seed(52)
shuffled = subset[:]
random.shuffle(shuffled)
letters = ["A", "B", "C", "D"]
for number, symbol, name, category in shuffled:
distractor_pool = [c for c in ALL_CATEGORIES if c != category]
distractors = random.sample(distractor_pool, 3)
choices = distractors + [category]
random.shuffle(choices)
correct_letter = letters[choices.index(category)]
prompt_lines = [f"{name} ({symbol})", ""]
for letter, choice in zip(letters, choices):
prompt_lines.append(f"{letter}) {choice}")
prompt = "\n".join(prompt_lines)
qtext = f"What category is {name}?"
atext = f"{category}"
add_note(deck, model, prompt, correct_letter, qtext, atext,
media_files, f"cat_{number}")
return deck, media_files, len(subset)
if args.deck == "prehs":
deck_key = "periodic_prehs"
elif args.deck == "hs":
deck_key = "periodic_hs"
else:
deck_key = "periodic_category"
MEDIA_DIR = os.path.join(SCRATCH, f"media_{deck_key}_{args.voice}")
os.makedirs(MEDIA_DIR, exist_ok=True)
GENERATORS = {"prehs": gen_prehs, "hs": gen_hs, "category": gen_category}
deck, media_files, count = GENERATORS[args.deck]()
package = genanki.Package(deck)
package.media_files = media_files
out_path = os.path.join(SCRATCH, f"{deck_key}_{args.voice}.apkg")
package.write_to_file(out_path)
size_mb = os.path.getsize(out_path) / (1024 * 1024)
print(f"\nDone: {out_path} ({size_mb:.1f} MB, {count} cards, {len(media_files)} audio clips)")
-408
View File
@@ -1,408 +0,0 @@
#!/usr/bin/env python3
"""tools/anki-deck-visual.py — Generate image-based Anki decks (.apkg) for
shapes, clocks, and coin-counting, with Anki's built-in type-the-answer
input and Piper (offline, local neural TTS) audio. See
tools/anki-deck-math.py's docstring for one-time setup (venv, genanki +
piper-tts, downloading a voice) — same steps apply here.
All images are drawn programmatically as SVG (regular-polygon geometry,
clock-hand trigonometry, coin layouts) rather than AI-generated — image
generation (local or cloud) is a poor fit for content that has to be
exactly correct (an exact clock time, an exact side count, an exact coin
total), not just plausible-looking. See this script's own point/angle
generation functions for how each shape's geometry is computed directly
rather than approximated.
Decks:
shapes regular polygons (3-10 sides, image->name and name->sides)
plus 5 quadrilateral types (image->name: square, rectangle,
rhombus, trapezoid, parallelogram — each one's geometry is
genuinely distinct, not just differently labeled)
clocks analog clock faces, all 144 hour/5-min combinations,
type the time as H:MM (the hour hand moves fractionally
with the minutes, e.g. 6:30 sits halfway between 6 and 7 —
a static hour hand is the most common "looks right but
teaches wrong" bug in generated clock faces)
currency US coins (nickel/dime/quarter — no pennies, since they're
barely used day to day at this point), 1-4 coins per card,
type the total in cents
Usage (run with the venv from anki-deck-math.py's docstring activated):
python3 anki-deck-visual.py --deck shapes
python3 anki-deck-visual.py --deck clocks
python3 anki-deck-visual.py --deck currency
(add --voice en_US-amy-medium etc.; --model-path if a voice isn't found
automatically; --dry-run-tts to test the deck-building logic without any
voice model at all, using silent placeholder audio)
"""
import argparse
import hashlib
import genanki
import math
import os
import random
import subprocess
parser = argparse.ArgumentParser()
parser.add_argument("--deck", required=True, choices=["shapes", "clocks", "currency"])
parser.add_argument("--voice", default="en_US-lessac-medium")
parser.add_argument("--model-path", default=None)
parser.add_argument("--dry-run-tts", action="store_true")
args = parser.parse_args()
SCRATCH = os.path.dirname(os.path.abspath(__file__))
_CANDIDATES = [
args.model_path,
f"{args.voice}.onnx",
os.path.join(SCRATCH, f"{args.voice}.onnx"),
os.path.expanduser(f"~/{args.voice}.onnx"),
os.path.expanduser(f"~/.local/share/piper/voices/{args.voice}.onnx"),
]
VOICE_MODEL = next((p for p in _CANDIDATES if p and os.path.isfile(p)), None)
if VOICE_MODEL is None and not args.dry_run_tts:
raise SystemExit(
f"Voice model for '{args.voice}' not found. Checked:\n"
+ "\n".join(f" {p}" for p in _CANDIDATES if p)
+ f"\n\nFind it with: find / -iname '{args.voice}.onnx' 2>/dev/null"
+ "\nThen pass its exact path with --model-path /the/real/path.onnx"
+ "\n(or pass --dry-run-tts to test deck-building without any voice at all)"
)
def piper_tts(text: str, out_path: str) -> None:
if args.dry_run_tts:
with open(out_path, "wb") as f:
f.write(
b"RIFF$\x00\x00\x00WAVEfmt \x10\x00\x00\x00\x01\x00\x01\x00"
b"\x22\x56\x00\x00\x44\xac\x00\x00\x02\x00\x10\x00data\x00\x00\x00\x00"
)
return
subprocess.run(
["python3", "-m", "piper", "-m", VOICE_MODEL, "-f", out_path],
input=text.encode("utf-8"),
check=True,
capture_output=True,
)
ONES = ["zero", "one", "two", "three", "four", "five", "six", "seven",
"eight", "nine", "ten", "eleven", "twelve", "thirteen", "fourteen",
"fifteen", "sixteen", "seventeen", "eighteen", "nineteen"]
TENS = ["", "", "twenty", "thirty", "forty", "fifty", "sixty", "seventy",
"eighty", "ninety"]
def num2words(n):
if n < 20:
return ONES[n]
if n < 100:
t, o = divmod(n, 10)
return TENS[t] + ("-" + ONES[o] if o else "")
h, rem = divmod(n, 100)
return ONES[h] + " hundred" + (" " + num2words(rem) if rem else "")
assert num2words(15) == "fifteen"
assert num2words(40) == "forty"
def time_words(hour, minute):
"""3, 5 -> 'three oh five'; 3, 15 -> 'three fifteen'; 3, 0 -> 'three o'clock'."""
if minute == 0:
return f"{num2words(hour)} o'clock"
if minute < 10:
return f"{num2words(hour)} oh {num2words(minute)}"
return f"{num2words(hour)} {num2words(minute)}"
assert time_words(3, 0) == "three o'clock"
assert time_words(3, 5) == "three oh five"
assert time_words(3, 15) == "three fifteen"
assert time_words(12, 45) == "twelve forty-five"
def build_model(deck_key):
voice_hash = int(hashlib.sha256(f"{deck_key}:{args.voice}".encode()).hexdigest(), 16)
model_id = 1_800_000_000 + (voice_hash % 90_000_000)
return model_id, genanki.Model(
model_id,
f"Visual Fact ({deck_key}, {args.voice})",
fields=[{"name": "Image"}, {"name": "Answer"}, {"name": "QSound"}, {"name": "ASound"}],
templates=[{
"name": "Card",
"qfmt": """
<div class="imgwrap">{{Image}}</div>
{{QSound}}
{{type:Answer}}
""",
"afmt": """
<div class="imgwrap">{{Image}}</div>
<hr id="answer">
{{type:Answer}}
{{ASound}}
""",
}],
css="""
.card { font-family: Arial, sans-serif; font-size: 24px; text-align: center; }
.imgwrap { margin: 10px auto; }
.imgwrap img { max-width: 260px; max-height: 260px; }
""",
)
def build_deck(deck_key, deck_title):
voice_hash = int(hashlib.sha256(f"{deck_key}:{args.voice}".encode()).hexdigest(), 16)
deck_id = 2_200_000_000 + (voice_hash % 90_000_000)
return genanki.Deck(deck_id, deck_title)
def add_note(deck, model, image_html, answer, qtext, atext, media_files, tag):
qfile = f"q_{tag}.wav"
afile = f"a_{tag}.wav"
qpath = os.path.join(MEDIA_DIR, qfile)
apath = os.path.join(MEDIA_DIR, afile)
piper_tts(qtext, qpath)
piper_tts(atext, apath)
media_files += [qpath, apath]
deck.add_note(genanki.Note(
model=model,
fields=[image_html, answer, f"[sound:{qfile}]", f"[sound:{afile}]"],
))
def add_text_note(deck, model, text, answer, qtext, atext, media_files, tag):
"""For directions that don't need an image (e.g. name -> number of sides)."""
add_note(deck, model, f'<div style="font-size:36px;">{text}</div>', answer,
qtext, atext, media_files, tag)
# ─── SVG generation ───────────────────────────────────────────────────────────
def save_svg(svg_body, filename, viewbox="0 0 200 200"):
path = os.path.join(MEDIA_DIR, filename)
with open(path, "w") as f:
f.write(
f'<svg xmlns="http://www.w3.org/2000/svg" viewBox="{viewbox}" '
f'width="200" height="200">{svg_body}</svg>'
)
return path
def regular_polygon_points(n_sides, cx=100, cy=100, r=80):
points = []
# Start pointing up (-90deg) so shapes sit "upright" rather than vertex-right.
start_angle = -90
for i in range(n_sides):
angle_deg = start_angle + i * (360 / n_sides)
angle_rad = math.radians(angle_deg)
x = cx + r * math.cos(angle_rad)
y = cy + r * math.sin(angle_rad)
points.append((round(x, 1), round(y, 1)))
return points
def polygon_svg(points):
pts_str = " ".join(f"{x},{y}" for x, y in points)
return f'<polygon points="{pts_str}" fill="#6fa8dc" stroke="#1c4587" stroke-width="4"/>'
POLYGON_NAMES = {
3: "triangle", 4: "square", 5: "pentagon", 6: "hexagon", 7: "heptagon",
8: "octagon", 9: "nonagon", 10: "decagon",
}
QUADRILATERALS = {
"square": [(50, 50), (150, 50), (150, 150), (50, 150)],
"rectangle": [(30, 60), (170, 60), (170, 140), (30, 140)],
"rhombus": [(100, 20), (170, 100), (100, 180), (30, 100)],
"trapezoid": [(60, 60), (140, 60), (170, 140), (30, 140)],
"parallelogram": [(60, 60), (160, 60), (140, 140), (40, 140)],
}
def clock_svg(hour, minute):
cx, cy, r = 100, 100, 90
minute_angle = minute * 6 - 90
hour_angle = (hour % 12) * 30 + minute * 0.5 - 90
def hand(angle_deg, length, width, color):
rad = math.radians(angle_deg)
x2 = cx + length * math.cos(rad)
y2 = cy + length * math.sin(rad)
return f'<line x1="{cx}" y1="{cy}" x2="{x2:.1f}" y2="{y2:.1f}" stroke="{color}" stroke-width="{width}" stroke-linecap="round"/>'
ticks = []
numerals = []
for h in range(1, 13):
angle = math.radians(h * 30 - 90)
tx1, ty1 = cx + (r - 10) * math.cos(angle), cy + (r - 10) * math.sin(angle)
tx2, ty2 = cx + r * math.cos(angle), cy + r * math.sin(angle)
ticks.append(f'<line x1="{tx1:.1f}" y1="{ty1:.1f}" x2="{tx2:.1f}" y2="{ty2:.1f}" stroke="black" stroke-width="2"/>')
nx, ny = cx + (r - 22) * math.cos(angle), cy + (r - 22) * math.sin(angle)
numerals.append(f'<text x="{nx:.1f}" y="{ny:.1f}" font-size="14" text-anchor="middle" dominant-baseline="middle">{h}</text>')
body = (
f'<circle cx="{cx}" cy="{cy}" r="{r}" fill="white" stroke="black" stroke-width="3"/>'
+ "".join(ticks) + "".join(numerals)
+ hand(hour_angle, 45, 6, "black")
+ hand(minute_angle, 70, 4, "black")
+ f'<circle cx="{cx}" cy="{cy}" r="4" fill="black"/>'
)
return body
COIN_INFO = {5: ("#c0c0c0", ""), 10: ("#d9d9d9", "10¢"), 25: ("#b8b8b8", "25¢")}
COIN_NAMES = {5: "nickel", 10: "dime", 25: "quarter"}
def coins_svg(coin_values):
n = len(coin_values)
spacing = 200 // (n + 1)
parts = []
for i, v in enumerate(coin_values):
cx = spacing * (i + 1)
color, label = COIN_INFO[v]
radius = 30 if v == 25 else (26 if v == 10 else 28)
parts.append(
f'<circle cx="{cx}" cy="100" r="{radius}" fill="{color}" stroke="#444" stroke-width="2"/>'
f'<text x="{cx}" y="105" font-size="14" text-anchor="middle">{label}</text>'
)
return "".join(parts)
def coin_list_words(coin_values):
names = [COIN_NAMES[v] for v in coin_values]
if len(names) == 1:
return f"a {names[0]}"
if len(names) == 2:
return f"a {names[0]} and a {names[1]}"
return ", ".join(f"a {n}" for n in names[:-1]) + f", and a {names[-1]}"
# ─── Per-deck generators ─────────────────────────────────────────────────────
def gen_shapes():
deck_key = "shapes"
model_id, model = build_model(deck_key)
deck = build_deck(deck_key, "Shapes: Polygons & Quadrilaterals")
media_files = []
jobs = []
for n in range(3, 11):
jobs.append(("polygon_image", n))
jobs.append(("polygon_sides", n))
for qname in QUADRILATERALS:
jobs.append(("quad_image", qname))
random.seed(60)
random.shuffle(jobs)
for kind, val in jobs:
if kind == "polygon_image":
n = val
name = POLYGON_NAMES[n]
svg_path = save_svg(polygon_svg(regular_polygon_points(n)), f"poly_{n}.svg")
media_files.append(svg_path)
add_note(deck, model, f'<img src="poly_{n}.svg">', name,
"What shape is this?", name, media_files, f"shape_img_{n}")
elif kind == "polygon_sides":
n = val
name = POLYGON_NAMES[n]
add_text_note(deck, model, name.capitalize(), str(n),
f"How many sides does a {name} have?", num2words(n),
media_files, f"shape_sides_{n}")
else:
qname = val
svg_path = save_svg(polygon_svg(QUADRILATERALS[qname]), f"quad_{qname}.svg")
media_files.append(svg_path)
add_note(deck, model, f'<img src="quad_{qname}.svg">', qname,
"What shape is this?", qname, media_files, f"shape_quad_{qname}")
return deck, media_files, len(jobs)
def gen_clocks():
deck_key = "clocks"
model_id, model = build_model(deck_key)
deck = build_deck(deck_key, "Telling Time: Analog Clocks")
media_files = []
times = [(h, m) for h in range(1, 13) for m in range(0, 60, 5)]
random.seed(61)
random.shuffle(times)
# The question prompt ("What time is it?") is identical for every card —
# generate it once instead of 144 times.
shared_qfile = "q_clock_prompt.wav"
piper_tts("What time is it?", os.path.join(MEDIA_DIR, shared_qfile))
media_files.append(os.path.join(MEDIA_DIR, shared_qfile))
for hour, minute in times:
svg_path = save_svg(clock_svg(hour, minute), f"clock_{hour}_{minute:02d}.svg")
media_files.append(svg_path)
answer = f"{hour}:{minute:02d}"
afile = f"a_clock_{hour}_{minute:02d}.wav"
apath = os.path.join(MEDIA_DIR, afile)
piper_tts(time_words(hour, minute), apath)
media_files.append(apath)
deck.add_note(genanki.Note(
model=model,
fields=[f'<img src="clock_{hour}_{minute:02d}.svg">', answer,
f"[sound:{shared_qfile}]", f"[sound:{afile}]"],
))
return deck, media_files, len(times)
def gen_currency():
deck_key = "currency"
model_id, model = build_model(deck_key)
# Deliberately nickel/dime/quarter only, no pennies — pennies are barely
# used day to day at this point, and skipping them keeps every total a
# multiple of 5 cents, which is a cleaner first pass at coin counting.
deck = build_deck(deck_key, "Counting Coins (nickels, dimes, quarters)")
media_files = []
denoms = [5, 10, 25]
combos = set()
for count in range(1, 5):
def rec(remaining, current):
if remaining == 0:
combos.add(tuple(sorted(current)))
return
for d in denoms:
if not current or d >= current[-1]:
rec(remaining - 1, current + [d])
rec(count, [])
combos = sorted(combos)
random.seed(62)
random.shuffle(combos)
for coin_values in combos:
total = sum(coin_values)
svg_path = save_svg(coins_svg(list(coin_values)), f"coins_{'_'.join(map(str, coin_values))}.svg")
media_files.append(svg_path)
qtext = f"How much money is {coin_list_words(list(coin_values))}?"
atext = f"{num2words(total)} cents"
add_note(deck, model, f'<img src="coins_{"_".join(map(str, coin_values))}.svg">',
str(total), qtext, atext, media_files, f"coins_{'_'.join(map(str, coin_values))}")
return deck, media_files, len(combos)
if args.deck == "shapes":
deck_key = "shapes"
elif args.deck == "clocks":
deck_key = "clocks"
else:
deck_key = "currency"
MEDIA_DIR = os.path.join(SCRATCH, f"media_{deck_key}_{args.voice}")
os.makedirs(MEDIA_DIR, exist_ok=True)
GENERATORS = {"shapes": gen_shapes, "clocks": gen_clocks, "currency": gen_currency}
deck, media_files, count = GENERATORS[args.deck]()
package = genanki.Package(deck)
package.media_files = media_files
out_path = os.path.join(SCRATCH, f"{deck_key}_{args.voice}.apkg")
package.write_to_file(out_path)
size_mb = os.path.getsize(out_path) / (1024 * 1024)
print(f"\nDone: {out_path} ({size_mb:.1f} MB, {count} cards, {len(media_files)} media files)")
+15 -47
View File
@@ -277,34 +277,17 @@ sync_github_to_gitea() {
# old commit indefinitely, with no error at any step. Also no longer
# silencing stderr: a real auth/network failure should be visible in the
# log, not just "Failed to fetch" with no reason why.
local auth_url="${clone_url/https:\/\//https:\/\/$GITHUB_TOKEN@}"
if [[ -d "$local_path" ]]; then
info "Fetching $full_name from GitHub..."
git -C "$local_path" fetch origin '+refs/heads/*:refs/heads/*' --prune --quiet || {
err "Failed to fetch $full_name"; return 1; }
else
info "Cloning $full_name from GitHub..."
mkdir -p "$(dirname "$local_path")"
git init --bare --quiet "$local_path" || { err "Failed to init $full_name"; return 1; }
git -C "$local_path" remote add origin "$auth_url"
local auth_url="${clone_url/https:\/\//https:\/\/$GITHUB_TOKEN@}"
git clone --bare --quiet "$auth_url" "$local_path" || {
err "Failed to clone $full_name"; return 1; }
fi
# Explicit heads+tags refspec on BOTH the initial clone and every later
# fetch, not `git clone --bare` (which pulls every ref the remote
# advertises, refs/pull/*/head included) — GitHub exposes PR refs over
# the same smart-HTTP endpoint a plain bare clone reads from, and those
# live in a namespace Gitea's own PR system reserves for itself. A later
# `git push --mirror` (pushes every local ref verbatim) then gets
# rejected by Gitea's server-side hook — confirmed live: "hook declined
# to update refs/pull/1/head". Scoping fetch AND push to heads/tags only
# avoids ever touching that namespace in either direction.
git -C "$local_path" fetch origin \
'+refs/heads/*:refs/heads/*' '+refs/tags/*:refs/tags/*' \
--prune --quiet || { err "Failed to fetch $full_name"; return 1; }
# Self-heals a repo synced before this fix — a stray refs/pull/* (or any
# other non-heads/tags ref) an earlier run's unscoped `clone --bare`
# already pulled in would otherwise keep tripping the same Gitea hook on
# every sync from here on, with no other way to clear it.
git -C "$local_path" for-each-ref --format='%(refname)' \
'refs/pull/*' 'refs/merge-requests/*' 'refs/changes/*' \
| xargs -r -n1 git -C "$local_path" update-ref -d
# Ensure repo exists on Gitea
local gitea_check
@@ -316,17 +299,12 @@ sync_github_to_gitea() {
>/dev/null || { err "Failed to create $repo_name on Gitea"; return 1; }
fi
# Push to Gitea — same explicit heads+tags scoping as the fetch above,
# not --mirror (which would push refs/pull/* etc. verbatim and hit the
# same rejected-hook failure this whole fix is for). --prune still makes
# Gitea's heads/tags a true mirror of GitHub's (deletes ones GitHub no
# longer has), just without ever touching reserved ref namespaces.
# Push to Gitea
local gitea_push_url="${GITEA_URL/https:\/\//https:\/\/$GITEA_USER:$GITEA_TOKEN@}"
gitea_push_url="${gitea_push_url/http:\/\//http:\/\/$GITEA_USER:$GITEA_TOKEN@}"
gitea_push_url="$gitea_push_url/$GITEA_USER/$repo_name.git"
git -C "$local_path" push --prune --quiet "$gitea_push_url" \
'+refs/heads/*:refs/heads/*' '+refs/tags/*:refs/tags/*' || {
git -C "$local_path" push --mirror "$gitea_push_url" --quiet || {
err "Failed to push $full_name to Gitea"; return 1; }
ok "GitHub → Gitea: $full_name"
_log "PULL $full_name OK"
@@ -342,26 +320,18 @@ sync_gitea_to_github() {
local gitea_auth_url="${clone_url/https:\/\//https:\/\/$GITEA_USER:$GITEA_TOKEN@}"
gitea_auth_url="${gitea_auth_url/http:\/\//http:\/\/$GITEA_USER:$GITEA_TOKEN@}"
# See the matching comment in sync_github_to_gitea() above — same reason
# applies in reverse: Gitea also exposes PR refs (refs/pull/*/head) over
# its git smart-HTTP endpoint, and GitHub rejects direct pushes to that
# same reserved namespace just as Gitea's hook does. Explicit heads+tags
# refspec on the initial clone too, not `git clone --bare`.
# See the matching comment in sync_github_to_gitea() above — same
# explicit-refspec, visible-stderr fix, same reason.
if [[ -d "$local_path" ]]; then
info "Fetching $full_name from Gitea..."
git -C "$local_path" fetch origin '+refs/heads/*:refs/heads/*' --prune --quiet || {
err "Failed to fetch $full_name from Gitea"; return 1; }
else
info "Cloning $full_name from Gitea..."
mkdir -p "$(dirname "$local_path")"
git init --bare --quiet "$local_path" || { err "Failed to init $full_name"; return 1; }
git -C "$local_path" remote add origin "$gitea_auth_url"
git clone --bare --quiet "$gitea_auth_url" "$local_path" || {
err "Failed to clone $full_name from Gitea"; return 1; }
fi
git -C "$local_path" fetch origin \
'+refs/heads/*:refs/heads/*' '+refs/tags/*:refs/tags/*' \
--prune --quiet || { err "Failed to fetch $full_name from Gitea"; return 1; }
# Self-heals a repo synced before this fix — see the matching comment above.
git -C "$local_path" for-each-ref --format='%(refname)' \
'refs/pull/*' 'refs/merge-requests/*' 'refs/changes/*' \
| xargs -r -n1 git -C "$local_path" update-ref -d
# Ensure repo exists on GitHub
local gh_check
@@ -373,11 +343,9 @@ sync_gitea_to_github() {
>/dev/null || { err "Failed to create $repo_name on GitHub"; return 1; }
fi
# Push to GitHub — explicit heads+tags scoping, not --mirror. Same
# reasoning as the Gitea push above.
# Push to GitHub
local github_push_url="https://$GITHUB_TOKEN@github.com/$GITHUB_USER/$repo_name.git"
git -C "$local_path" push --prune --quiet "$github_push_url" \
'+refs/heads/*:refs/heads/*' '+refs/tags/*:refs/tags/*' || {
git -C "$local_path" push --mirror "$github_push_url" --quiet || {
err "Failed to push $full_name to GitHub"; return 1; }
ok "Gitea → GitHub: $full_name"
_log "PUSH $full_name OK"