ubuntu-post-install/SECURITY-IMPROVEMENTS.md

Security and Infrastructure Improvements

Summary of Changes

This document describes the comprehensive security and infrastructure improvements made to the Ubuntu post-installation script.

Issues Fixed

1. Docker Directory Ownership

Problem: Docker directories were being created as root, causing permission issues when running Docker without sudo.

Solution:

• Added ensure_docker_dir_ownership() helper function
• Applied to ALL 25+ services (Immich, Keycloak, ActualBudget, Jellyfin, etc.)
• Fixed disaster recovery path (line 309)
• All Docker directories now properly owned by sudo user

Impact: Docker containers can now be managed without requiring root/sudo for every command.

---

2. Keycloak Security Overhaul

Problem: Weak default passwords, special characters causing issues, development mode in production.

Solutions Implemented:

Password Requirements

• Minimum length: 12 characters (16+ recommended)
• Character set: Letters and numbers ONLY (no special characters)
• Auto-generation: Press ENTER to generate secure passwords automatically
• Validation: Real-time password validation with retry loop

Production vs Development Mode

• Production mode: Uses start command, requires hostname configuration
• Development mode: Uses start-dev command, relaxed security for testing
• Hostname support: Proper KC_HOSTNAME configuration for public deployment

Environment Variables

• All credentials moved to .env file
• Admin password and database password securely stored
• No more hardcoded passwords in docker-compose.yml

Example Keycloak .env file structure:

# Keycloak Environment Variables
KEYCLOAK_ADMIN=admin
KEYCLOAK_ADMIN_PASSWORD=<secure-20-char-password>
POSTGRES_DB=keycloak
POSTGRES_USER=keycloak
POSTGRES_PASSWORD=<secure-32-char-password>
KC_PROXY=edge
KC_HTTP_ENABLED=true
KC_HOSTNAME=auth.yourdomain.com  # (if production mode)

---

3. Environment Variable Management (.env Files)

Services Now Using .env Files:

• ✅ Keycloak (admin + database passwords)
• ✅ ActualBudget (timezone and config)
• ✅ Immich (already had .env)
• ✅ FindMyDevice (already had .env)
• ✅ wg-easy (already had .env)
• ✅ Kopia (already had .env)

Benefits:

• Passwords not visible in docker-compose.yml files
• Easy to backup separately from compose files
• Can be excluded from version control
• Easier credential rotation

---

4. Caddy2 Reverse Proxy Integration

Existing Integration: All services already include Caddy2 reverse proxy configuration via the configure_caddy_for_service() function.

Features:

• Automatic HTTPS via Let's Encrypt
• HTTP/2 support
• Security headers (HSTS, X-Frame-Options, etc.)
• JSON logging for fail2ban
• Automatic certificate renewal

Example Caddy Configuration:

photos.yourdomain.com {
    reverse_proxy localhost:2283

    header {
        Strict-Transport-Security "max-age=31536000; includeSubDomains; preload"
        X-Frame-Options "SAMEORIGIN"
        X-Content-Type-Options "nosniff"
        X-XSS-Protection "1; mode=block"
        Referrer-Policy "strict-origin-when-cross-origin"
    }

    log {
        output file /var/log/caddy/photos-access.log
        format json
    }
}

---

5. Fail2ban Integration

Existing fail2ban Labels: All services already include fail2ban monitoring labels:

labels:
  - "io.podman.annotations.label/fail2ban.enable=true"
  - "io.podman.annotations.label/fail2ban.filter=caddy-auth"

Services with fail2ban monitoring:

• ActualBudget
• Keycloak
• All other internet-facing services

fail2ban Configuration:

• Filter: /etc/fail2ban/filter.d/caddy-auth.conf
• Jail: /etc/fail2ban/jail.d/caddy.conf
• Ban after: 5 failed attempts
• Ban duration: 3600 seconds (1 hour)
• Detection window: 600 seconds

Detailed Setup: See CADDY-FAIL2BAN-SETUP.md for complete configuration.

---

Helper Functions Added

ensure_docker_dir_ownership(dir1 [dir2 ...])

Ensures Docker directories are owned by the actual user (not root).

Usage:

mkdir -p "$SERVICE_DIR"
ensure_docker_dir_ownership "$SERVICE_DIR"

generate_password([length])

Generates secure alphanumeric passwords (no special characters).

Usage:

PASSWORD=$(generate_password 20)  # 20-character password

validate_password(password [min_length])

Validates passwords for Keycloak compatibility.

Validation Rules:

• Minimum length (default: 12 characters)
• Alphanumeric only (a-zA-Z0-9)
• Returns 0 if valid, 1 if invalid

Usage:

if validate_password "$USER_PASSWORD" 12; then
    echo "Password accepted"
fi

---

Keycloak Setup Guide

For ActualBudget on Pikapods

1. Install Keycloak with production mode:

   sudo bash ubuntu-post-install.sh
   # Select Keycloak from menu
   # Choose production mode (y)
   # Enter hostname: auth.yourdomain.com
   # Press ENTER to auto-generate secure passwords
   

2. Configure Caddy2:

   • Script automatically prompts for Caddy configuration
   • Enter your domain (e.g., auth.yourdomain.com)
   • Ensure DNS A record points to your server

3. Configure DNS:

   auth.yourdomain.com  →  Your Server IP
   

4. Access Keycloak:

   https://auth.yourdomain.com
   

5. Set up ActualBudget OAuth:

   • The script automatically creates an OAuth client for ActualBudget
   • Client details saved to: ~/docker/keycloak/actualbudget-oauth.txt
   • Use these credentials in your Pikapod ActualBudget instance

6. Configure ActualBudget on Pikapods:

   • Go to your ActualBudget settings
   • Enable OpenID Connect
   • Enter your Keycloak details:
     • Issuer: https://auth.yourdomain.com/realms/homelab
     • Client ID: (from actualbudget-oauth.txt)
     • Client Secret: (from actualbudget-oauth.txt)

For Other Self-Hosted Services

The script can create generic OAuth clients for other services. After Keycloak installation, you can:

1. Access Keycloak admin console
2. Create new OAuth2/OIDC clients
3. Configure redirect URIs for your services
4. Use the client credentials in your service configuration

Generic Client Template:

• Client ID: your-service-name
• Client Type: Confidential
• Standard Flow Enabled: Yes
• Valid Redirect URIs: https://your-service.com/*

---

Password Requirements Reference

Keycloak Passwords

• Minimum: 12 characters
• Recommended: 16+ characters
• Format: Alphanumeric only (a-zA-Z0-9)
• No special characters: !@#$%^&*() etc. are NOT allowed
• Generation: Press ENTER for auto-generated secure passwords

Why No Special Characters?

Keycloak has issues with special characters in certain authentication flows and database connection strings. Restricting to alphanumeric ensures compatibility.

Password Strength with Alphanumeric Only

• 12 characters: ~62^12 = 3.2 × 10^21 combinations
• 16 characters: ~62^16 = 4.7 × 10^28 combinations
• 20 characters: ~62^20 = 7.0 × 10^35 combinations

This is cryptographically secure for all practical purposes.

---

Verification Checklist

After running the updated script:

Docker Ownership

# Check docker directory ownership
ls -la ~/docker/
# All directories should be owned by your user, not root

# Test docker without sudo
docker ps
# Should work without permission errors

Keycloak

# Check .env file exists
cat ~/docker/keycloak/.env
# Should contain KEYCLOAK_ADMIN_PASSWORD and POSTGRES_PASSWORD

# Check production mode
cat ~/docker/keycloak/docker-compose.yml | grep command
# Should show "start" for production or "start-dev" for development

# Test access
curl http://localhost:8180/health
# Should return health status

Caddy2

# Check Caddy is running
docker ps | grep caddy

# Check logs
docker logs caddy

# Test HTTPS redirect
curl -I http://yourdomain.com
# Should redirect to HTTPS

fail2ban

# Check fail2ban status
sudo fail2ban-client status caddy-auth

# Test ban
# (Make 5 failed login attempts)
sudo fail2ban-client status caddy-auth
# Should show banned IP

---

Migration Guide

If you have existing services:

Existing ActualBudget

1. Backup existing data: cp -r ~/docker/actualbudget ~/docker/actualbudget.backup
2. Run updated script and select "Reconfigure" when prompted
3. New .env file will be created
4. Verify ownership: ls -la ~/docker/actualbudget
5. Restart container: cd ~/docker/actualbudget && docker compose restart

Existing Keycloak

1. IMPORTANT: Backup your data first!

   cp -r ~/docker/keycloak ~/docker/keycloak.backup
   

2. Stop existing container:

   cd ~/docker/keycloak && docker compose down
   

3. Run updated script and select Keycloak
4. Choose whether to keep existing data or start fresh
5. If keeping data, manually update .env with your existing passwords
6. Restart: docker compose up -d

---

Troubleshooting

Permission Denied Errors

# Fix ownership of all docker directories
sudo chown -R $USER:$USER ~/docker

Keycloak Won't Start

# Check logs
docker logs keycloak

# Common issues:
# 1. Missing KC_HOSTNAME in production mode
# 2. Database connection failed (check postgres container)
# 3. Port 8180 already in use

# Fix: Edit .env and docker-compose.yml as needed

Caddy Certificate Errors

# Check Caddy logs
docker logs caddy

# Common issues:
# 1. DNS not pointing to server
# 2. Ports 80/443 not open
# 3. Firewall blocking Let's Encrypt

# Test DNS:
dig auth.yourdomain.com

# Test port accessibility:
sudo ufw status

---

Security Best Practices

1. Change default passwords: Even with auto-generation, review and update if needed
2. Use production mode for Keycloak: Never use development mode for internet-facing deployments
3. Enable fail2ban: Monitor and ban malicious IPs
4. Regular updates: Keep containers updated (use Watchtower in notify mode)
5. Backup .env files: Store securely, separate from compose files
6. Use HTTPS everywhere: Configure Caddy2 for all public services
7. Limit exposed ports: Only expose necessary ports to the internet
8. Monitor logs: Regular review of Caddy and fail2ban logs

---

Additional Resources

• Keycloak Setup Guide: KEYCLOAK-SETUP-GUIDE.md
• Caddy + fail2ban Setup: CADDY-FAIL2BAN-SETUP.md
• Main Script: ubuntu-post-install.sh
• Caddy Helper: caddy-setup-helper.sh

---

Support

If you encounter issues:

1. Check logs: docker logs <container-name>
2. Verify ownership: ls -la ~/docker
3. Review this document for troubleshooting steps
4. Check existing documentation in repository

---

Last Updated: 2026-01-13 Script Version: Latest (with security improvements)