Merge pull request #232 from outis1one/claude/group-call-busy-signal-e8usba

Fix inbound PSTN calls: dialplan context never loaded, Caller-ID never real

docs/anveo-direct-setup-guide.md

@@ -2,9 +2,14 @@
 
 This is the exact sequence that got a real Anveo Direct DID working end to
 end (both outbound and inbound) with `asterisk-digital-ocean.sh` +
-`pstn-trunk.sh`, confirmed live on a real droplet. Follow it in order for
+`pstn-trunk.sh`, confirmed live on a real droplet.
-each additional number — steps 1–2 are one-time account setup; steps
+
-3–7 repeat per DID.
+**Steps 1, 3 and 4 are one-time account setup** — the outbound Service
+Trunk (step 3) and the inbound SIP Trunk (step 4) each cover every DID on
+the account, no matter how many you add. **Only steps 2, 4.5, 6 and 7
+repeat per number**: order the DID, point it at the existing inbound trunk
+(skippable entirely if you set the account default in step 4.6), assign it
+in the dashboard, and test.
 
 ## 0. Prerequisites
 
@@ -91,7 +96,7 @@ catch-all entry (prefix `1`); a few specific area codes have their own
 slightly different override rates, and Virgin Islands is billed
 separately/higher (~$0.02/min) despite looking like a normal US number.
 
-## 4. Create the SIP Trunk (inbound forwarding) — once per DID
+## 4. Create the SIP Trunk (inbound forwarding) — ONE covers every DID
 
 Anveo has **two unrelated "trunk" concepts** — don't confuse them:
 - The **Outbound Service Trunk** from step 3 handles calls *out*.
@@ -99,11 +104,20 @@ Anveo has **two unrelated "trunk" concepts** — don't confuse them:
   routing objects live) handles calls *in* — this is what actually
   populates a DID's "Destination SIP Trunk" dropdown.
 
-For **each DID**, create one of these:
+**You only need ONE of these for the whole account**, no matter how many
+DIDs you have. The `$[E164]$` placeholder in step 2 is what makes it
+reusable — Anveo substitutes each DID's own number at call time, so the
+same trunk object forwards `15551111111@IP:5060` for one DID and
+`15552222222@IP:5060` for another. The Asterisk side matches whatever
+arrives (`_X.` in `[from-pstn-trunk]`) and looks it up in
+`pstn-personal-dids.conf` to decide who rings. Creating one per DID is
+possible but pointless unless different DIDs need to forward to genuinely
+different destinations.
 
-1. **Trunk Name**: any label (e.g. `asterisk-do-inbound`) — can reuse the
+Create it once (steps 1–4), then per DID only step 5 applies — or set the
-   same one for multiple DIDs if you want them all forwarding the same
+account default in step 6 and skip even that:
-   way, or make one per DID for clarity.
+
+1. **Trunk Name**: any label (e.g. `asterisk-do-inbound`).
 2. **Primary**: type **SIP URI**, value:
    ```
    $[E164]$@<this box's public IP>:5060
@@ -115,12 +129,13 @@ For **each DID**, create one of these:
    in the actual INVITE Anveo sends.
 3. **Failover**: leave blank.
 4. Save.
-5. Go to the DID's own **Call Options** tab → set **Destination SIP
+5. **Per DID**: go to that DID's **Call Options** tab → set **Destination
-   Trunk** to this new SIP Trunk object (not the Outbound Service Trunk)
+   SIP Trunk** to this SIP Trunk object (not the Outbound Service Trunk)
    → Save.
-6. **Optional, recommended once you're adding more DIDs**: Account
+6. **Recommended if you have more than one or two DIDs**: Account
    Options → Service Defaults → set **Default Destination Trunk** to this
-   SIP Trunk, so future DIDs auto-route here without repeating step 5.
+   SIP Trunk. New DIDs then auto-route here and step 5 becomes unnecessary
+   entirely.
 
 ## 5. Configure the droplet
 
@@ -216,13 +231,56 @@ installer, not configuration mistakes — listed here for context on what
   under `setup.sh`'s `set -u` on `source`. Now guarded and falls back to
   safe defaults, erroring clearly only if genuinely required fields are
   missing.
+- **The inbound `[from-pstn-trunk]` context never loaded at all** — it used
+  to share one `#include`'d file with the outbound dialplan, where the file
+  continued `[intercom]` first and only declared `[from-pstn-trunk]`
+  partway through. Asterisk silently failed to load the *entire* file
+  (outbound patterns included) with no error anywhere, so every inbound
+  call got "extension not found in context 'from-pstn-trunk'" and an
+  immediate SIP rejection. Inbound now lives in its own file that starts
+  with its context header, matching the structure of the messaging
+  dialplan that always loaded correctly.
+- **Every inbound call's Caller-ID arrived as the DID itself**, not the
+  real caller — the trunk endpoint pinned a static `callerid=<DID>` and
+  didn't set `trust_id_inbound`, so Asterisk ignored the identity Anveo
+  actually sends and fell back to that default. Every restricted-tier
+  `allowed_numbers` check was therefore unwinnable regardless of config.
+  Fixed by dropping the static default (redundant — the outbound dialplan
+  sets `CALLERID(num)` on the channel itself) and adding
+  `trust_id_inbound=yes`.
+- **A leading `+` broke whitelist matching** — once the fix above started
+  surfacing real caller identity, Anveo delivers it `+E.164` style
+  (`+15551234567`). The normalizer only added a leading `1` to 10-digit
+  values and never stripped the `+`, so a 12-character value could never
+  match an 11-digit, digits-only `allowed_numbers` entry no matter how
+  correctly the number was whitelisted. The `+` is now stripped in the
+  dialplan, the group-ring script, and the dashboard's admin-input side
+  (so pasting a number straight from a call log works too).
+- **Bare `Hangup()` on inbound denials** instead of `Busy(15)` — three
+  inbound denial paths didn't produce a proper "486 Busy Here", making a
+  correct permission denial sound like a generic call failure and
+  disguising config mismatches as dialplan bugs.
+
+## Dashboard changes may need "Commit Changes"
+
+Confirmed live: edits made in the Security Dashboard's PSTN Trunk tab
+(permission tiers, groups, personal-DID owners, limits) are written to disk
+immediately, but `AST_CONFIG()` in the dialplan has been observed still
+returning the *old* value until the Asterisk container is fully restarted —
+a `dialplan reload` / `module reload` is not enough. This contradicts the
+"reads fresh on every call, no restart needed" design premise these files
+otherwise rely on; root cause not established.
+
+The PSTN Trunk tab has a **Commit Changes (Restart Asterisk)** button for
+this, which appears after any save on that tab and warns if you navigate
+away with changes uncommitted. Existing calls are unaffected by the
+restart. If a change doesn't seem to be taking effect on a test call, press
+it before assuming the config itself is wrong.
 
 ## Still open
 
-- The 2-phone-number account cap (see step 1) blocks getting all 4
+- Nothing blocking. The two items previously listed here are both resolved:
-  planned numbers until either Anveo lifts it or a tax ID is provided —
+  the 2-phone-number account cap no longer applies (additional DIDs have
-  tabled for now.
+  since been obtained), and the interactive CLI walkthrough was built —
-- The interactive CLI walkthrough (`pstn-trunk.sh` actually prompting
+  see `_pstn_anveo_walkthrough` in `services/pstn-trunk.sh`, reachable from
-  through account setup step by step, not just this static doc) hasn't
+  the provider quick-pick when you choose Anveo Direct.
-  been built yet — this guide is the reference for building that once
-  there's appetite for it.

services/pstn-trunk.sh

@@ -583,30 +583,30 @@ __ALERT_KILLED_IN_LINE__
 ; live config files and applies the identical tier/allowed_numbers logic
 ; in a plain shell loop.
 ;
-; Every OTHER denial path in this file (outbound tier/number/intl/killswitch,
+; Every denial path in this file (outbound tier/number/intl/killswitch, the
-; the shared ring-group's pstn_in_denied, this owner's own denial just
+; shared ring-group's pstn_in_denied, the personal-DID owner and group
-; above) plays a proper Busy(15) tone before Hangup() — pstn_in_denied and
+; denials) plays a proper Busy(15) tone before Hangup(). pstn_in_denied and
-; the personal-DID denial paths were missing that Busy() and fell straight
+; the two personal-DID denial paths used to fall straight to a bare
-; to a bare Hangup() on an unanswered channel instead, which chan_pjsip
+; Hangup() on an unanswered channel instead, which chan_pjsip does NOT map
-; does NOT map to "486 Busy Here" the way Busy() does. NOT yet confirmed
+; to "486 Busy Here" the way Busy() does — making a config mismatch sound
-; live against a real call, but this is a real, reproducible inconsistency
+; like a dialplan failure. Confirmed live (2026-07-24): a group-owned
-; (grep this file for "Hangup()" without a preceding Busy() and these three
+; personal DID is now fully working end to end, ringing every authorized
-; exits were the only inbound ones missing it) and is a plausible cause of
+; member simultaneously, after this plus three other fixes found in the
-; a caller hearing one ring and then an ambiguous fast-busy/call-failed
+; same session (see _pstn_write_inbound_dialplan_include's file-split
-; tone on what looks like a correctly-authorized group member: if the
+; comment, and _pstn_write_pjsip_include's trust_id_inbound/callerid
-; group/tier/allowed_numbers config has ANY mismatch against the test
+; comment).
-; caller ID, the call was always going to be denied here regardless of
+;
-; this fix — the missing Busy() just made a config problem sound like a
+; If a group-assigned personal number goes straight to busy, the NoOp()
-; dialplan problem. Fixed below (Busy(15) added to all three inbound
+; logged right after the SHELL() call in pstn_personal_group_ring prints
-; denial exits). If a group-assigned personal number still goes straight
+; the resolved group name, caller ID, and PSTN_RING_LIST — check the
-; to busy after this, the NoOp() logged right after the SHELL() call in
+; Asterisk console/full log for that line first. An empty list there means
-; pstn_personal_group_ring prints the resolved group name, caller ID, and
+; the config, not the dialplan: confirm the calling number is in
-; PSTN_RING_LIST — check `asterisk -rx "core show channels verbose"` / the
+; allowed_numbers for at least one CURRENT group member whose tier is
-; full log for that line first; an empty list there means the config (not
-; the dialplan) is the next thing to check — confirm the calling number is
-; in allowed_numbers for at least one CURRENT group member whose tier is
 ; "restricted" or "full" in pstn-permissions.conf, and that the group's
-; members= line in pstn-groups.conf actually lists that extension.
+; members= line in pstn-groups.conf actually lists that extension. Note
+; that a dashboard edit may need the "Commit Changes" button (a container
+; restart — see restart_asterisk_container() in services/security-dashboard.sh)
+; before AST_CONFIG() actually returns the new value.
 exten => pstn_personal_inbound,1,GotoIf($["${PSTN_PERSONAL_OWNER:0:1}" = "@"]?pstn_personal_group_ring,1)
  same => n,Set(PSTN_OWNER_TIER=${AST_CONFIG(pstn-permissions.conf,${PSTN_PERSONAL_OWNER},tier)})
  same => n,GotoIf($["${PSTN_OWNER_TIER}" = "full"]?pstn_personal_ring,1)
@@ -672,6 +672,10 @@ EOF
 # Safe REGEX direction: $allowed is admin-entered (pstn-permissions.conf),
 # $caller is the incoming Caller-ID — pattern is always the admin data,
 # string is always the caller-controlled data, never the reverse.
+# Confirmed live (2026-07-24): SHELL() invoking this script from the
+# dialplan works end to end against a real inbound call — the returned
+# &-joined list rings every authorized member simultaneously, first to
+# answer wins, and unauthorized members are correctly skipped.
 _pstn_write_personal_group_ring_script() {
     local FILE="$1"
     cat > "$FILE" << 'SCRIPT'

services/security-dashboard.sh

@@ -458,15 +458,20 @@ _secdash_write_sudoers() {
     # `docker exec -i <container> tee <exact path>` instead of a direct
     # host-side file write (see _secdash_grant_asterisk_access's comment on
     # why), plus the two Asterisk CLI calls needed after a change and the
-    # live registration-status check. All six are exact commands, no
+    # live registration-status check. All seven are exact commands, no
     # wildcards, scoped to the one container actually installed on this box.
+    # The last line (docker restart) backs the PSTN Trunk tab's "Commit
+    # Changes" button — see restart_asterisk_container()'s comment for why
+    # that exists (AST_CONFIG() live-reads not always picking up dashboard
+    # edits without a full container restart).
     if [ -n "$_ea_container" ]; then
         _ea_lines="$_svc_user ALL=(root) NOPASSWD: /usr/bin/docker exec -i $_ea_container tee /etc/asterisk/pjsip.conf
 $_svc_user ALL=(root) NOPASSWD: /usr/bin/docker exec -i $_ea_container tee /etc/easy-asterisk/categories.conf
 $_svc_user ALL=(root) NOPASSWD: /usr/bin/docker exec -i $_ea_container tee /etc/easy-asterisk/rooms.conf
 $_svc_user ALL=(root) NOPASSWD: /usr/bin/docker exec $_ea_container asterisk -rx module\ reload\ res_pjsip.so
 $_svc_user ALL=(root) NOPASSWD: /usr/bin/docker exec $_ea_container asterisk -rx pjsip\ show\ endpoints
-$_svc_user ALL=(root) NOPASSWD: /usr/bin/docker exec $_ea_container /usr/local/bin/easy-asterisk --rebuild-dialplan"
+$_svc_user ALL=(root) NOPASSWD: /usr/bin/docker exec $_ea_container /usr/local/bin/easy-asterisk --rebuild-dialplan
+$_svc_user ALL=(root) NOPASSWD: /usr/bin/docker restart $_ea_container"
     fi
     cat > /etc/sudoers.d/security-dashboard << SUDOERS
 $_svc_user ALL=(root) NOPASSWD: /usr/bin/cscli decisions delete --id [0-9]*
@@ -1644,6 +1649,27 @@ def ea_rebuild_dialplan():
     run_sudo(["docker", "exec", ASTERISK_EA_CONTAINER, "/usr/local/bin/easy-asterisk", "--rebuild-dialplan"])
 
 
+def restart_asterisk_container():
+    """Restarts the Easy Asterisk container - the "Commit Changes" button on
+    the PSTN Trunk tab. Confirmed live: dashboard writes to
+    pstn-permissions.conf/pstn-groups.conf/pstn-personal-dids.conf land on
+    disk immediately (readable via a plain `cat` right after saving), but
+    AST_CONFIG() in the dialplan sometimes kept returning a stale value
+    until the container was fully restarted - not just a `dialplan reload`
+    or `module reload`, an actual container restart. Root cause not fully
+    understood (contradicts AST_CONFIG's whole "reads fresh every call, no
+    restart needed" design premise, which this codebase otherwise relies on
+    throughout), but the restart reliably clears it, so this button exists
+    instead of requiring every admin to rediscover "just restart it" the
+    hard way. Uses the same ASTERISK_EA_CONTAINER/run_sudo mechanism as the
+    Easy Asterisk Admin tab's own docker exec calls - no new sudoers scope
+    needed beyond the one line added for this."""
+    if not ASTERISK_EA_CONTAINER:
+        return False, "No Asterisk container detected on this box"
+    ok, _out, err = run_sudo(["docker", "restart", ASTERISK_EA_CONTAINER], timeout=30)
+    return ok, ("" if ok else (err or "Restart failed"))
+
+
 def ea_get_status():
     """Registered/unregistered per extension — same 'pjsip show endpoints'
     parsing as the vendored get_registered_endpoints()."""
@@ -2311,9 +2337,17 @@ INDEX_HTML = """<!doctype html>
     </div>
   </div>
   <div id="tab-pstn" style="display:none">
+    <div class="card" id="pstn-restart-banner" style="display:none; border-left:4px solid #d9822b">
+      <b>Unsaved changes may not be live yet.</b>
+      <p class="muted" style="margin:0.25rem 0 0.5rem">
+        Edits here are written to disk immediately, but Asterisk doesn't always pick them up without a restart — confirmed on personal-DID group reassignment specifically. Existing calls are never affected.
+      </p>
+      <button class="action" id="pstn-restart-btn">Commit Changes (Restart Asterisk)</button>
+      <span id="pstn-restart-msg" class="muted" style="margin-left:0.5rem"></span>
+    </div>
     <div class="card">
       <h3 style="margin-top:0">Concurrent-call caps</h3>
-      <p class="muted">A call over either cap gets a busy signal (and an ntfy alert, if enabled) — existing calls are never affected. Changes apply live, on the next call.</p>
+      <p class="muted">A call over either cap gets a busy signal (and an ntfy alert, if enabled) — existing calls are never affected. Changes are usually live on the next call; if a call doesn't reflect a recent change, use "Commit Changes" below.</p>
       <div class="row">
         <label class="muted" style="white-space:nowrap">Max outbound<br><input type="text" id="limit-out" style="width:5rem"></label>
         <label class="muted" style="white-space:nowrap">Max inbound<br><input type="text" id="limit-in" style="width:5rem"></label>
@@ -2327,7 +2361,7 @@ INDEX_HTML = """<!doctype html>
         <b>internal</b> — no PSTN, can still call/receive other extensions and internal ring groups.
         <b>restricted</b> — internal, plus only pre-approved US numbers.
         <b>full</b> — internal, plus any US number.
-        Changes apply live, on the next call — no Asterisk restart needed.
+        Changes are usually live on the next call; if one doesn't seem to be taking effect, use "Commit Changes" below.
       </p>
       <p class="muted">
         <b>Messaging</b> — the same internal SIP texting flag as the Extensions tab's checkboxes, independent of the calling tier; this column is just a convenience for setting it alongside tier/numbers on one row. Enforced live by a dedicated dialplan context — see services/asterisk-digital-ocean.sh's README for how, and its caveat on the sender-extraction logic still needing real-traffic confirmation.
@@ -2345,7 +2379,7 @@ INDEX_HTML = """<!doctype html>
     <div class="card">
       <h3 style="margin-top:0">Personal numbers</h3>
       <p class="muted">
-        Multiple DIDs can share this one trunk. Assigning a DID to an extension routes inbound calls to that DID straight to its owner (still gated by the owner's own tier/approved-numbers above — no ring-group fallback), and makes that extension's outbound calls show this DID as Caller-ID instead of the shared trunk DID. You can also assign a DID to a <b>group</b> instead of a single extension — every current member whose own tier/approved-numbers authorize the caller rings (checked fresh on every call, so membership changes apply immediately); a group has no single extension to hang the outbound Caller-ID override on, so that part only applies to single-extension assignments. The shared DID/ring-group keeps working regardless.
+        Multiple DIDs can share this one trunk. Assigning a DID to an extension routes inbound calls to that DID straight to its owner (still gated by the owner's own tier/approved-numbers above — no ring-group fallback), and makes that extension's outbound calls show this DID as Caller-ID instead of the shared trunk DID. You can also assign a DID to a <b>group</b> instead of a single extension — every current member whose own tier/approved-numbers authorize the caller rings, checked fresh against the group's current membership on every call; a group has no single extension to hang the outbound Caller-ID override on, so that part only applies to single-extension assignments. The shared DID/ring-group keeps working regardless. Reassigning a DID's owner has been confirmed to sometimes need "Commit Changes" (below) before Asterisk actually uses the new owner.
       </p>
       <div class="row">
         <input type="text" id="pd-did" placeholder="DID, e.g. 5551234567 (10 digits, no leading 1)" style="width:12rem">
@@ -3118,6 +3152,7 @@ document.getElementById("grp-save").addEventListener("click", async () => {
   });
   const data = await res.json();
   document.getElementById("grp-msg").textContent = data.message || (data.ok ? "Saved" : "Failed");
+  if (data.ok) markPstnDirty();
   loadGroups();
 });
 
@@ -3129,6 +3164,7 @@ async function applyGroupMessaging(name, enabled) {
   });
   const data = await res.json();
   document.getElementById("grp-msg").textContent = data.message || (data.ok ? "Applied" : "Failed");
+  if (data.ok) markPstnDirty();
   loadMessaging();
 }
 
@@ -3140,6 +3176,7 @@ async function deleteGroup(name) {
   });
   const data = await res.json();
   document.getElementById("grp-msg").textContent = data.message || (data.ok ? "Deleted" : "Failed");
+  if (data.ok) markPstnDirty();
   loadGroups();
 }
 
@@ -3152,9 +3189,41 @@ document.getElementById("limits-save").addEventListener("click", async () => {
   });
   const data = await res.json();
   document.getElementById("limits-msg").textContent = data.message || (data.ok ? "Saved" : "Failed");
+  if (data.ok) markPstnDirty();
   loadPstnLimits();
 });
 
+// "Commit Changes" — see restart_asterisk_container()'s comment in app.py
+// for why this button exists: AST_CONFIG() live-reads of these PSTN config
+// files have been confirmed to sometimes stay stale (returning what was on
+// disk at last container start, not the freshly-saved value) until a full
+// container restart, contradicting the "no restart needed" premise the
+// rest of this tab's copy otherwise relies on. pstnDirty tracks whether
+// ANY save on this tab succeeded since the last restart (or page load),
+// shows a persistent banner, and warns on tab-close/navigation so a change
+// doesn't silently sit uncommitted.
+let pstnDirty = false;
+function markPstnDirty() {
+  pstnDirty = true;
+  document.getElementById("pstn-restart-banner").style.display = "";
+}
+window.addEventListener("beforeunload", (e) => {
+  if (!pstnDirty) return;
+  e.preventDefault();
+  e.returnValue = "";
+});
+document.getElementById("pstn-restart-btn").addEventListener("click", async () => {
+  const msg = document.getElementById("pstn-restart-msg");
+  msg.textContent = "Restarting Asterisk…";
+  const res = await fetch("/api/asterisk-restart", { method: "POST" });
+  const data = await res.json();
+  msg.textContent = data.message || (data.ok ? "Restarted" : "Failed");
+  if (data.ok) {
+    pstnDirty = false;
+    document.getElementById("pstn-restart-banner").style.display = "none";
+  }
+});
+
 let lastPstnExts = [];
 let pstnSort = { key: null, dir: 1 };
 
@@ -3243,6 +3312,7 @@ async function savePstnPermission(ext) {
   });
   const data = await res.json();
   document.getElementById("pstn-msg").textContent = (data.message || (data.ok ? "Saved" : "Failed")) + " (extension " + ext + ")";
+  if (data.ok) markPstnDirty();
   loadPstnPermissions();
 }
 
@@ -3308,7 +3378,10 @@ document.getElementById("pd-save").addEventListener("click", async () => {
   });
   const data = await res.json();
   document.getElementById("pd-msg").textContent = data.message || (data.ok ? "Saved" : "Failed");
-  if (data.ok) document.getElementById("pd-did").value = "";
+  if (data.ok) {
+    document.getElementById("pd-did").value = "";
+    markPstnDirty();
+  }
   loadPersonalDids();
 });
 
@@ -3320,6 +3393,7 @@ async function removePersonalDid(did) {
   });
   const data = await res.json();
   document.getElementById("pd-msg").textContent = data.message || (data.ok ? "Removed" : "Failed");
+  if (data.ok) markPstnDirty();
   loadPersonalDids();
 }
 
@@ -3441,6 +3515,9 @@ class Handler(BaseHTTPRequestHandler):
         elif self.path == "/api/pstn-groups/apply-messaging":
             ok, message = apply_group_messaging(payload.get("name", ""), bool(payload.get("enabled", False)))
             self._json({"ok": ok, "message": message})
+        elif self.path == "/api/asterisk-restart":
+            ok, message = restart_asterisk_container()
+            self._json({"ok": ok, "message": message})
         elif self.path == "/api/ea-devices":
             ok, result = ea_add_device(
                 payload.get("name", ""), payload.get("category", ""), payload.get("extension", ""),