Add real-time GitHub webhook sync to Gitea
Gitea's GitHub mirror sync previously only ran on a fixed-interval timer. Adds an opt-in GitHub webhook receiver (small stdlib-only Python HTTP server under its own systemd service) that verifies GitHub's HMAC-SHA256 signature and triggers an immediate, single-repo --pull-only sync the moment GitHub receives a push, wired through Caddy the same way every other service in this repo is. The scheduled timer stays in place as a safety net and still covers the Gitea -> GitHub direction.
This commit is contained in:
Claude
+286
-4
@@ -453,6 +453,11 @@ _gitea_remove_sync_timer() {
|
||||
# reconfigure of an existing one. Always asked (matches pstn-trunk.sh's
|
||||
# international-calling step reasoning: a live-editable extra, not a
|
||||
# structural setting tied exclusively to fresh installs).
|
||||
#
|
||||
# Sets _GITEA_SYNC_FLAG as an out-param (not `local` — read it after the
|
||||
# call returns, same convention as CADDY_SERVICE_CONFIGURED) so the caller
|
||||
# can decide whether the real-time webhook offer even makes sense for the
|
||||
# direction just chosen.
|
||||
_gitea_run_sync_direction_step() {
|
||||
local DIR="$1"
|
||||
|
||||
@@ -463,12 +468,13 @@ _gitea_run_sync_direction_step() {
|
||||
echo " 3) Both directions"
|
||||
local _DIR_CHOICE=""
|
||||
prompt_text " Choice [1]:" "1" _DIR_CHOICE
|
||||
local FLAG="" DIR_DESC=""
|
||||
local DIR_DESC=""
|
||||
case "$_DIR_CHOICE" in
|
||||
2) FLAG="--push-only"; DIR_DESC="Gitea -> GitHub only" ;;
|
||||
3) FLAG=""; DIR_DESC="both directions" ;;
|
||||
*) FLAG="--pull-only"; DIR_DESC="GitHub -> Gitea only" ;;
|
||||
2) _GITEA_SYNC_FLAG="--push-only"; DIR_DESC="Gitea -> GitHub only" ;;
|
||||
3) _GITEA_SYNC_FLAG=""; DIR_DESC="both directions" ;;
|
||||
*) _GITEA_SYNC_FLAG="--pull-only"; DIR_DESC="GitHub -> Gitea only" ;;
|
||||
esac
|
||||
local FLAG="$_GITEA_SYNC_FLAG"
|
||||
log_info "Sync direction: $DIR_DESC"
|
||||
|
||||
_gitea_remove_sync_timer
|
||||
@@ -519,6 +525,250 @@ _gitea_run_sync_direction_step() {
|
||||
esac
|
||||
}
|
||||
|
||||
|
||||
# ── Real-time sync: a GitHub webhook receiver, not just the timer above ────
|
||||
# The timer above polls on a fixed schedule (default 6h) — fine for a slow
|
||||
# backup cadence, but a genuine "GitHub -> Gitea in real time" ask needs
|
||||
# GitHub to tell Gitea the moment something changes instead of Gitea finding
|
||||
# out up to one interval late. GitHub's own webhook (repo Settings ->
|
||||
# Webhooks) is the standard way to do that: it POSTs a JSON payload the
|
||||
# instant someone pushes. This writes a tiny stdlib-only Python HTTP server
|
||||
# to receive it — python3 is already a hard dependency of this directory's
|
||||
# gitea-github-sync.sh itself (used there for JSON parsing), so this adds
|
||||
# no new dependency — running under its own persistent systemd service,
|
||||
# and wires it up to Caddy the same way every other web-facing piece of
|
||||
# this install does.
|
||||
#
|
||||
# Deliberately NOT a Docker container: it just shells out to the existing
|
||||
# gitea-github-sync.sh sitting right next to it in $DIR, the same way the
|
||||
# timer's own systemd service does — no image to build/pull for what's
|
||||
# fundamentally a few lines of stdlib HTTP handling.
|
||||
_gitea_write_webhook_receiver() {
|
||||
local DIR="$1"
|
||||
cat > "$DIR/gitea-github-webhook.py" << 'PYEOF'
|
||||
#!/usr/bin/env python3
|
||||
"""Gitea <-> GitHub webhook receiver — triggers an immediate, single-repo
|
||||
mirror sync (gitea-github-sync.sh --repo owner/name --pull-only) the moment
|
||||
GitHub POSTs a push event, instead of waiting for the scheduled timer.
|
||||
|
||||
Written by services/gitea.sh — re-run 'sudo ./setup.sh gitea' (Update mode
|
||||
is fine) to regenerate this file rather than hand-editing it; a hand edit
|
||||
survives until the next Update-mode rerun overwrites it again.
|
||||
|
||||
WEBHOOK_SECRET is read from .env in this same directory at every request,
|
||||
never taken from the environment/systemd unit — /etc/systemd/system/*.service
|
||||
files are world-readable, and .env (chmod 600) is already where every other
|
||||
token in this directory lives.
|
||||
"""
|
||||
import hashlib
|
||||
import hmac
|
||||
import http.server
|
||||
import json
|
||||
import os
|
||||
import subprocess
|
||||
import sys
|
||||
|
||||
SYNC_DIR = os.environ.get("GITEA_SYNC_DIR", os.path.dirname(os.path.abspath(__file__)))
|
||||
ENV_PATH = os.path.join(SYNC_DIR, ".env")
|
||||
PORT = int(os.environ.get("WEBHOOK_PORT", "3020"))
|
||||
|
||||
|
||||
def _load_env_value(key):
|
||||
try:
|
||||
with open(ENV_PATH, "r") as f:
|
||||
for line in f:
|
||||
line = line.split("#", 1)[0].strip()
|
||||
if not line.startswith(key + "="):
|
||||
continue
|
||||
return line[len(key) + 1:].strip().strip("'").strip('"')
|
||||
except OSError:
|
||||
pass
|
||||
return ""
|
||||
|
||||
|
||||
class Handler(http.server.BaseHTTPRequestHandler):
|
||||
def log_message(self, fmt, *args):
|
||||
sys.stderr.write("%s - %s\n" % (self.address_string(), fmt % args))
|
||||
|
||||
def _reply(self, code, body=b""):
|
||||
self.send_response(code)
|
||||
self.end_headers()
|
||||
if body:
|
||||
self.wfile.write(body)
|
||||
|
||||
def do_GET(self):
|
||||
self._reply(200, b"gitea-github-webhook: listening\n")
|
||||
|
||||
def do_POST(self):
|
||||
secret = _load_env_value("WEBHOOK_SECRET").encode()
|
||||
if not secret:
|
||||
self._reply(503, b"WEBHOOK_SECRET not configured")
|
||||
return
|
||||
|
||||
length = int(self.headers.get("Content-Length", 0) or 0)
|
||||
body = self.rfile.read(length) if length else b""
|
||||
|
||||
sig = self.headers.get("X-Hub-Signature-256", "")
|
||||
expected = "sha256=" + hmac.new(secret, body, hashlib.sha256).hexdigest()
|
||||
if not sig or not hmac.compare_digest(sig, expected):
|
||||
self._reply(401, b"bad signature")
|
||||
return
|
||||
|
||||
event = self.headers.get("X-GitHub-Event", "")
|
||||
if event == "ping":
|
||||
self._reply(200, b"pong")
|
||||
return
|
||||
if event != "push":
|
||||
self._reply(204)
|
||||
return
|
||||
|
||||
try:
|
||||
payload = json.loads(body or b"{}")
|
||||
full_name = payload["repository"]["full_name"]
|
||||
except (json.JSONDecodeError, KeyError, TypeError):
|
||||
self._reply(400, b"couldn't find repository.full_name in payload")
|
||||
return
|
||||
|
||||
self._reply(202, b"sync queued\n")
|
||||
sync_script = os.path.join(SYNC_DIR, "gitea-github-sync.sh")
|
||||
sync_env = dict(os.environ, SYNC_ENV=ENV_PATH)
|
||||
subprocess.Popen(
|
||||
["bash", sync_script, "--repo", full_name, "--pull-only"],
|
||||
cwd=SYNC_DIR,
|
||||
env=sync_env,
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
server = http.server.ThreadingHTTPServer(("0.0.0.0", PORT), Handler)
|
||||
server.serve_forever()
|
||||
PYEOF
|
||||
chmod +x "$DIR/gitea-github-webhook.py"
|
||||
chown "$ACTUAL_USER:$ACTUAL_USER" "$DIR/gitea-github-webhook.py"
|
||||
}
|
||||
|
||||
_gitea_write_webhook_service() {
|
||||
local DIR="$1" RUN_USER="$2" RUN_HOME="$3" PORT="$4"
|
||||
local _service="/etc/systemd/system/gitea-github-webhook.service"
|
||||
|
||||
cat > "$_service" << UNIT
|
||||
[Unit]
|
||||
Description=Gitea-GitHub Webhook Receiver (real-time mirror sync trigger)
|
||||
After=network-online.target docker.service
|
||||
Wants=network-online.target
|
||||
|
||||
[Service]
|
||||
Type=simple
|
||||
User=${RUN_USER}
|
||||
Environment=HOME=${RUN_HOME}
|
||||
Environment=GITEA_SYNC_DIR=${DIR}
|
||||
Environment=WEBHOOK_PORT=${PORT}
|
||||
ExecStart=/usr/bin/python3 ${DIR}/gitea-github-webhook.py
|
||||
Restart=on-failure
|
||||
RestartSec=5
|
||||
|
||||
[Install]
|
||||
WantedBy=multi-user.target
|
||||
UNIT
|
||||
|
||||
systemctl daemon-reload
|
||||
systemctl enable --now gitea-github-webhook.service
|
||||
}
|
||||
|
||||
_gitea_remove_webhook_service() {
|
||||
systemctl disable --now gitea-github-webhook.service 2>/dev/null || true
|
||||
rm -f /etc/systemd/system/gitea-github-webhook.service
|
||||
systemctl daemon-reload 2>/dev/null || true
|
||||
}
|
||||
|
||||
# Offers the webhook receiver above as an addition to (not a replacement
|
||||
# for) the timer set up in _gitea_run_sync_direction_step — the timer keeps
|
||||
# covering the Gitea -> GitHub direction (and acts as a safety net for any
|
||||
# push GitHub's webhook delivery ever misses), the webhook just gets the
|
||||
# GitHub -> Gitea direction down from "up to one interval late" to seconds.
|
||||
# Always asked on every install/reconfigure, same "live-editable extra"
|
||||
# pattern as the direction+autosync step itself — see that function's own
|
||||
# comment. Skipped (and any existing webhook torn down) outright when the
|
||||
# chosen direction is push-only, since GitHub has nothing to notify about
|
||||
# in that direction.
|
||||
_gitea_offer_realtime_webhook() {
|
||||
local DIR="$1" SYNC_FLAG="$2"
|
||||
|
||||
if [[ "$SYNC_FLAG" == "--push-only" ]]; then
|
||||
_gitea_remove_webhook_service
|
||||
return 0
|
||||
fi
|
||||
|
||||
echo ""
|
||||
local USE_WEBHOOK=""
|
||||
prompt_yn " Also add a GitHub webhook for near-instant sync (push on GitHub -> synced here in seconds, instead of waiting for the timer above)? (y/n):" "n" USE_WEBHOOK
|
||||
if [[ ! "$USE_WEBHOOK" =~ ^[Yy]$ ]]; then
|
||||
_gitea_remove_webhook_service
|
||||
return 0
|
||||
fi
|
||||
|
||||
# Reuse an existing secret/port across reruns — rotating either one
|
||||
# silently breaks a webhook GitHub already has configured against the
|
||||
# old value, the same reasoning services/asterisk.sh's TURN port-range
|
||||
# persistence follows for a live coturn install.
|
||||
local WEBHOOK_SECRET WEBHOOK_PORT
|
||||
WEBHOOK_SECRET="$(grep '^WEBHOOK_SECRET=' "$DIR/.env" 2>/dev/null | cut -d= -f2- | tr -d "'\"")"
|
||||
WEBHOOK_PORT="$(grep '^WEBHOOK_PORT=' "$DIR/.env" 2>/dev/null | cut -d= -f2- | tr -d "'\"")"
|
||||
[[ -z "$WEBHOOK_SECRET" ]] && WEBHOOK_SECRET="$(generate_password 40)"
|
||||
if [[ -z "$WEBHOOK_PORT" ]]; then
|
||||
WEBHOOK_PORT=3020
|
||||
find_free_port WEBHOOK_PORT "$WEBHOOK_PORT"
|
||||
fi
|
||||
|
||||
if grep -q '^WEBHOOK_SECRET=' "$DIR/.env" 2>/dev/null; then
|
||||
sed -i "s|^WEBHOOK_SECRET=.*|WEBHOOK_SECRET='${WEBHOOK_SECRET}'|" "$DIR/.env"
|
||||
else
|
||||
echo "WEBHOOK_SECRET='${WEBHOOK_SECRET}'" >> "$DIR/.env"
|
||||
fi
|
||||
if grep -q '^WEBHOOK_PORT=' "$DIR/.env" 2>/dev/null; then
|
||||
sed -i "s|^WEBHOOK_PORT=.*|WEBHOOK_PORT='${WEBHOOK_PORT}'|" "$DIR/.env"
|
||||
else
|
||||
echo "WEBHOOK_PORT='${WEBHOOK_PORT}'" >> "$DIR/.env"
|
||||
fi
|
||||
chmod 600 "$DIR/.env"
|
||||
chown "$ACTUAL_USER:$ACTUAL_USER" "$DIR/.env"
|
||||
|
||||
_gitea_write_webhook_receiver "$DIR"
|
||||
_gitea_write_webhook_service "$DIR" "$ACTUAL_USER" "$ACTUAL_HOME" "$WEBHOOK_PORT"
|
||||
log_success "Webhook receiver running on port ${WEBHOOK_PORT} (systemctl status gitea-github-webhook)."
|
||||
|
||||
# Bare port -> host.docker.internal:PORT, same convention as every other
|
||||
# host-process (non-container) upstream in this repo — see the
|
||||
# configure_caddy_for_service usage note in CLAUDE.md.
|
||||
configure_caddy_for_service "Gitea GitHub Webhook" "$WEBHOOK_PORT" "gitea-webhook"
|
||||
if [[ "$CADDY_SERVICE_CONFIGURED" == true ]]; then
|
||||
if command -v ufw &>/dev/null; then
|
||||
if [[ "$CADDY_SERVICE_MODE" == "local" ]]; then
|
||||
ufw delete allow "${WEBHOOK_PORT}/tcp" 2>/dev/null || true
|
||||
ufw_allow_from_caddy_net "${WEBHOOK_PORT}"
|
||||
else
|
||||
ufw allow "${WEBHOOK_PORT}/tcp" comment "Gitea GitHub webhook" >/dev/null 2>&1 || true
|
||||
ensure_ufw_enabled
|
||||
fi
|
||||
fi
|
||||
echo ""
|
||||
log_success "Now add the webhook on GitHub, for every repo you want instant sync from:"
|
||||
log_info " Repo -> Settings -> Webhooks -> Add webhook"
|
||||
log_info " Payload URL: https://${CADDY_SERVICE_DOMAIN}/"
|
||||
log_info " Content type: application/json"
|
||||
log_info " Secret: ${WEBHOOK_SECRET}"
|
||||
log_info " Events: Just the push event"
|
||||
log_info "The timer above still covers every other repo, and this one too, on its"
|
||||
log_info "own schedule — the webhook is an addition, not a replacement for it."
|
||||
else
|
||||
log_warning "Webhook receiver is running (0.0.0.0:${WEBHOOK_PORT}) but nothing is exposing"
|
||||
log_warning "it to the internet, so GitHub can't reach it yet — re-run this installer and"
|
||||
log_warning "configure Caddy for it, or point your own reverse proxy at"
|
||||
log_warning "127.0.0.1:${WEBHOOK_PORT} (or the container-reachable host IP) by hand."
|
||||
log_info " Secret (for whenever you do expose it): ${WEBHOOK_SECRET}"
|
||||
fi
|
||||
}
|
||||
|
||||
install_gitea() {
|
||||
log_info "Setting up self-hosted Gitea..."
|
||||
|
||||
@@ -537,6 +787,8 @@ install_gitea() {
|
||||
echo "[DRY-RUN] Would ask sync direction (GitHub->Gitea / Gitea->GitHub / both) and whether"
|
||||
echo "[DRY-RUN] to install a systemd timer for automatic sync, or print manual instructions"
|
||||
echo "[DRY-RUN] Would offer to run a sync now (dry-run preview or for real), off-schedule"
|
||||
echo "[DRY-RUN] Would offer a GitHub webhook receiver for near-instant GitHub->Gitea sync"
|
||||
echo "[DRY-RUN] (systemd service + Caddy front door), unless direction is push-only"
|
||||
echo "[DRY-RUN] Would offer \"Sign in with Authelia\" (OIDC) if Authelia is installed"
|
||||
echo "[DRY-RUN] Would offer zero-click Authelia login (reverse-proxy auth) if Authelia"
|
||||
echo "[DRY-RUN] and local Caddy are both installed — rewires Gitea onto caddy_net"
|
||||
@@ -567,6 +819,7 @@ install_gitea() {
|
||||
&& log_success "Gitea refreshed and restarted." \
|
||||
|| log_warning "Restart failed — check: docker compose -f $DIR/docker-compose.yml logs"
|
||||
_gitea_run_sync_direction_step "$DIR"
|
||||
_gitea_offer_realtime_webhook "$DIR" "$_GITEA_SYNC_FLAG"
|
||||
_gitea_offer_authelia_sso "$DIR"
|
||||
_gitea_offer_reverse_proxy_auth "$DIR"
|
||||
_gitea_offer_actions_runner "$DIR"
|
||||
@@ -732,6 +985,7 @@ ENV
|
||||
fi
|
||||
|
||||
_gitea_run_sync_direction_step "$DIR"
|
||||
_gitea_offer_realtime_webhook "$DIR" "$_GITEA_SYNC_FLAG"
|
||||
|
||||
# ── Caddy — no forward_auth gate here. Gitea has its own built-in login,
|
||||
# unlike the no-auth-at-all apps elsewhere in this repo that need Caddy
|
||||
@@ -794,6 +1048,34 @@ Config (which repos, private/forks handling) lives at
|
||||
\`~/.config/gitea-github-sync/config\` — edit directly, or re-run
|
||||
\`bash gitea-github-sync.sh --init\` to redo it interactively.
|
||||
|
||||
## Real-time sync via GitHub webhook (optional)
|
||||
|
||||
The setup above only covers the GitHub -> Gitea direction; it doesn't apply
|
||||
if you chose Gitea -> GitHub only (GitHub has nothing to notify about in
|
||||
that direction). Adds a small Python HTTP server
|
||||
(\`gitea-github-webhook.py\`, in this directory) run as its own systemd
|
||||
service (\`gitea-github-webhook.service\`) that GitHub POSTs to the instant
|
||||
someone pushes — it verifies the request's HMAC signature against
|
||||
\`WEBHOOK_SECRET\` in \`.env\`, then runs \`gitea-github-sync.sh --repo
|
||||
owner/name --pull-only\` for just that one repo. The scheduled timer above
|
||||
still runs on its own interval regardless — the webhook is an addition
|
||||
that gets the GitHub -> Gitea direction down to seconds, not a replacement
|
||||
for it (and still catches anything a missed webhook delivery would have
|
||||
picked up next interval anyway).
|
||||
|
||||
Not set up yet, or want to change the port/secret? Re-run
|
||||
\`sudo ./setup.sh gitea\` (Update mode is fine) and answer yes to "Also add
|
||||
a GitHub webhook...". Once it's running, add the webhook itself on GitHub:
|
||||
repo -> Settings -> Webhooks -> Add webhook, Content type
|
||||
\`application/json\`, event \`Just the push event\`, using the payload
|
||||
URL/secret the installer printed (also in \`.env\` as \`WEBHOOK_PORT\` /
|
||||
\`WEBHOOK_SECRET\` if you need them again).
|
||||
|
||||
\`\`\`bash
|
||||
systemctl status gitea-github-webhook # is it running?
|
||||
journalctl -u gitea-github-webhook -f # watch it receive + trigger syncs
|
||||
\`\`\`
|
||||
|
||||
## Sign in with Authelia (optional)
|
||||
|
||||
If Authelia is installed, re-run \`sudo ./setup.sh gitea\` (Update mode is
|
||||
|
||||
Reference in New Issue
Block a user