Fix Security Log tab loading the entire multi-GB log into memory

parse_security_log() did f.readlines() on ASTERISK_LOG before slicing the
last 5000 lines - that reads the ENTIRE file into memory first. That log
is Asterisk's unrotated console/security output, and this tab polls it
every 30 seconds from the browser. Confirmed live: on a 1GB-RAM droplet
with a 1.4GB log file, this ballooned the dashboard (explicitly meant to
be a lightweight stdlib-only process) to 677MB RSS / 1.8GB peak swap,
which left CrowdSec unable to even start (boot timeout) and directly
contributed to the droplet becoming unresponsive.

Fixed by reading only a bounded ~2MB tail from the end of the file
(seek + fixed-size read) instead of the whole thing - memory use is now
constant regardless of how large the log grows. Tested against a 180MB
synthetic log: memory delta dropped from being proportional to file size
to ~7MB, in 0.05s.

Also added log rotation (services/asterisk-digital-ocean.sh) for that same
file, which had no rotation at all and reached 1.4GB in about 3 days -
copytruncate avoids needing to signal the containerized Asterisk process
to reopen its log handle.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Ho9mZgAkVpdz7S5wJkg8Nf
This commit is contained in:
Claude
2026-07-23 04:44:19 +00:00
parent 78456e6a3a
commit 616dcdcfbb
2 changed files with 53 additions and 3 deletions
+28 -3
View File
@@ -659,18 +659,43 @@ TIER_RE = re.compile(r"^(internal|restricted|full)$")
NUMBER_RE = re.compile(r"^\d{11}$")
SECURITY_LOG_TAIL_BYTES = 2 * 1024 * 1024 # comfortably enough for 5000 lines
def parse_security_log(limit=200):
"""Tail ASTERISK_LOG and return the most recent SecurityEvent lines,
newest first, as dicts. Missing file / no lines -> empty list, never an
error — this is a convenience view, not load-bearing."""
error — this is a convenience view, not load-bearing.
Reads only a bounded byte window from the END of the file, not the whole
thing — this log is Asterisk's unrotated console/security output and can
grow to multiple GB. The previous version did f.readlines() (loads the
ENTIRE file into memory) before slicing the last 5000 lines, and this
tab polls every 30 seconds from the browser. Confirmed live: on a 1GB-RAM
droplet with a 1.4GB log file, that ballooned this "stdlib only,
deliberately lightweight" process to 677MB RSS / 1.8GB peak swap, which
left CrowdSec unable to even start (boot timeout) and contributed
directly to the droplet becoming unresponsive. Bounding this to a fixed
~2MB window keeps memory use constant regardless of how large the log
file grows.
"""
if not ASTERISK_LOG or not os.path.isfile(ASTERISK_LOG):
return []
events = []
try:
with open(ASTERISK_LOG, "r", errors="replace") as f:
lines = f.readlines()[-5000:] # cap how much we ever scan
with open(ASTERISK_LOG, "rb") as f:
f.seek(0, os.SEEK_END)
size = f.tell()
start = max(0, size - SECURITY_LOG_TAIL_BYTES)
f.seek(start)
data = f.read()
except OSError:
return []
text = data.decode("utf-8", errors="replace")
lines = text.splitlines()
if start > 0 and lines:
lines = lines[1:] # first line is likely truncated mid-line
lines = lines[-5000:]
for line in lines:
if "SecurityEvent=" not in line:
continue