# FISHFUSION DEPLOYMENT GUIDE ## FusionPBX @ Max Site — Type C Registered Trunk (Telnyx *or* Flowroute) with E911 **Document scope:** Provider account setup (both Telnyx and Flowroute), headless Debian install over serial console, FusionPBX installation, registered-trunk configuration with E911 DIDs, all Magikarp/switch network changes, and a full acceptance-test matrix. --- ## 0. ENVIRONMENT REFERENCE CARD (verified against your configs) | Item | Value | |---|---| | PBX hostname (local) | `fishfusion.fishtank` | | PBX FQDN (public) | `voip.maxdevnet.cc` | | PBX IP | `172.20.8.5/24` — **static, no IPv6** | | PBX gateway/DNS | `172.20.8.1` (Magikarp br_vlan77) | | HPBX network | VLAN **77**, `172.20.8.0/24` | | VLAN 77 DHCP pool | `172.20.8.40–240` (your .5 is safely outside it) | | Magikarp WAN | `enp16s0f0np0` (`$WAN_IF`), **DHCP** | | Magikarp LAN trunk | `enp16s0f1np1` (`$LAN_IF`) | | Magikarp VLAN 77 | `enp16s0f1np1.77` (`$VLAN77_IF`) | | DDNS | ddclient on Magikarp already updates the A record for `voip.maxdevnet.cc` | | Local DNS | dnsmasq already has `address=/fishfusion/pbx/172.20.8.5` ✔ nothing to add | | Adtran TA908e | `172.20.8.15` (`fishtank908.fishtank`) — not required for this build | | Planned PBX switch port | ICX 1/1/5 (see §5.4 for the VLAN question) | | Trunk type | **Type C — credential/registration-based** (correct choice for DHCP WAN) | | SIP profile facing carrier | FreeSWITCH **external** profile, **UDP 5080** | | RTP range | **UDP 16384–32768** (FreeSWITCH default) | **Pre-flight checks (do these before anything else):** 1. On Cloudflare, confirm `voip.maxdevnet.cc` is **DNS-only (grey cloud)**. A proxied record hands the carrier a Cloudflare edge IP and SIP dies silently. Check now: ``` dig +short voip.maxdevnet.cc ``` The answer must be your current Socket WAN IP, not a 104.x/172.67.x Cloudflare address. 2. Confirm ddclient actually fires: `grep voip /var/cache/ddclient/ddclient.cache` (or your distro's cache path) on Magikarp and compare to `curl -4 ifconfig.me`. 3. Set the Cloudflare TTL on that record to **300 or lower** (Auto is fine; Auto=300). --- # PART 1 — PROVIDER ACCOUNT SETUP Do §1A **or** §1B (or both, if you want to A/B them — nothing conflicts). Everything in Parts 2–3 is provider-agnostic; Part 4 has separate gateway configs per provider. ## 1A. TELNYX ### 1A.1 Account creation & verification 1. Sign up at `telnyx.com` with a real name/address — they're a licensed carrier and KYC-check accounts. 2. Immediately complete **Level 2 verification** (Account Settings → Verifications): government ID + address. This gates number purchasing, porting, higher concurrency, and SMS. It can take minutes to a day or two; front-load it so it's never the blocker. 3. Fund the account: $10–20. Optionally enable auto-recharge later. Set a **low balance email alert** while you're in billing settings. ### 1A.2 Create the credential SIP Connection (this is what makes it Type C) Mission Control → **Voice → SIP Trunking → SIP Connections → Create SIP Connection**. (Telnyx renames menus periodically; the object names below are stable.) 1. **Connection name:** `fishfusion-trunk` 2. **Connection type / Authentication:** **Credentials**. Telnyx generates a SIP **username** and lets you set a **password**. Generate a long random password (24+ chars, no shell-hostile characters — avoid `@ : ; " '` to spare yourself XML/URI escaping pain in FreeSWITCH). **Record both.** 3. Open the connection's settings and set, tab by tab: **Inbound:** - **Destination number format:** `+E.164` (you will type the DID as `+1573…` in FusionPBX — write this down, it's the #1 inbound gotcha). - **Transport:** UDP. - **SIP subdomain:** skip/ignore — you're registering, not receiving by subdomain. - **Codecs:** enable `G711U (PCMU)` first, then `G722`, then `G711A`. Disable the exotic ones; fewer codecs = simpler SDP = fewer interop surprises. **Outbound:** - **Outbound Voice Profile:** create one, name it `fishfusion-ovp`: - **Allowed destinations:** United States (+ Canada if you want it). Leave every other country OFF. This is your toll-fraud seatbelt. - **Traffic type:** Conversational. - **Concurrent call / channel limit:** 2–5. A lab does not need more; a compromised lab REALLY does not need more. - **Daily/maximum spend limits:** set them. $5–10/day is generous for a lab. - **Localization / ANI override:** leave default for now; you'll set caller ID per call from FusionPBX. 4. **Registration specifics you should know:** with credential auth, FreeSWITCH registers to `sip.telnyx.com`. Telnyx honors your requested expiry within their min/max; 120 s (set on the FusionPBX gateway in §4) is safely within bounds. ### 1A.3 Buy the DID Mission Control → **Numbers → Search & Buy Numbers**. 1. Search by area code `573` (or rate center "Columbia, MO") → buy a local DID. 2. On the number's settings: **Connection** = `fishfusion-trunk`. This is what routes inbound calls to your registration. 3. Note the number in +E.164 (`+1573XXXXXXX`). ### 1A.4 E911 on Telnyx 1. Numbers → select your DID → **Emergency (E911)** → enable. 2. Enter the **exact physical address of the Max site**, including unit/suite. It's validated against the MSAG/address database; if validation fails, tweak formatting (St vs Street) until it passes. 3. Understand the billing: small monthly fee per E911-enabled DID, and a **per-call fee for actual 911 use** (and a large fee for false/abusive calls — don't test with real 911, use 933 per §6.7). 4. Telnyx supports the **933 test number**: dialing 933 through the trunk reads back the registered address and callback number without dispatching anyone. (Verify it's listed in their current E911 docs; it has been standard.) ### 1A.5 Collect the Telnyx IP ranges for the firewall Telnyx publishes their **SIP signaling** and **RTP media** IP ranges in their docs (portal → Networking/"IP Addresses" page — search "Telnyx IP addresses" in their docs). Copy the current lists; you'll paste them into nftables sets in §5.1. Do **not** trust any list baked into this document or any blog post — pull it live at deploy time. --- ## 1B. FLOWROUTE ### 1B.1 Account creation 1. Sign up at `flowroute.com`. Business-ish details are fine for a homelab; there's no Telnyx-style tiered ID verification, but expect card verification. 2. Prepay/fund the account (they're metered; $10–25 to start). ### 1B.2 Credentials & registration (Type C on Flowroute) Flowroute's native model is IP-auth "interconnections," but they fully support credential registration, which is what your DHCP WAN wants. 1. Portal → **Interconnection** (menu naming drifts; look for SIP settings / Interconnection / Preferences → API & SIP credentials). 2. Note your **Access Key / tech prefix** (a numeric identifier like `12345678`) — this is your SIP **username** — and set/record the **SIP password**. 3. **Registrar/proxy:** `sip.flowroute.com` (resolves via SRV to their POPs; nearest to you is their central/eastern US POP). 4. **Inbound routing for registration:** Flowroute routes each DID according to its configured route. In the DID's settings, set the primary route to **Registration** (deliver to wherever your credentials are currently registered). If the portal only shows host-based routes, look for the route type literally named `Registration`/`REG` — it exists; that's the Type C path. 5. **DID format note:** Flowroute typically presents the called number as **11-digit `1NXXNXXXXXX` without a leading +** (configurable in DID settings on newer portal versions). You will confirm the actual format with sngrep in §6.3 before creating the FusionPBX destination — do not guess. ### 1B.3 Buy the DID 1. Portal → **DIDs → Purchase**: search `573` / Columbia MO, buy. 2. Set the DID's **route** to Registration (above). 3. While in DID settings: enable **CNAM storage** if you want outbound caller-ID name (free per their docs). ### 1B.4 E911 on Flowroute 1. DIDs → select DID → **E911** → enable and enter the validated Max-site address. 2. Same economics as Telnyx: activation and/or small monthly fee per DID, real 911 calls billed, false-call penalties. Use **933** for testing (Flowroute supports the 933 address-readback test — verify in their E911 docs at deploy time). ### 1B.5 Fraud controls (do not skip) Flowroute's differentiators are here — turn them on: - **International dialing:** restrict to US/CA only (portal → preferences/fraud settings → allowed destinations). - **IP-based auth:** you can *additionally* leave it unset/disabled since you're registering; consider disabling any unused auth path. - **Traffic monitoring alerts:** enable email alerts. ### 1B.6 Collect Flowroute IP ranges Flowroute publishes SIP/media IP lists in their support docs ("Flowroute IP addresses"). Pull the current list for the nftables sets in §5.1. --- # PART 2 — DEBIAN INSTALL, HEADLESS, OVER SERIAL **Version decision:** install **Debian 13 (trixie) stable**. FusionPBX's install script and their prebuilt FreeSWITCH packages target Debian 12/13 by codename; Debian *testing* (forky) has no package builds and the script will add dead repos. If you insist on testing anyway: run the pre-install, then edit the codename the script writes into `/etc/apt/sources.list.d/` from `forky` to `trixie` before running `install.sh`, and apt-pin PHP + freeswitch packages afterward. Recommendation stands: **stable for the PBX**; save testing for boxes where an upgrade surprise costs you an evening, not your phone service. ## 2.1 Prepare the installer USB On any Linux box (Magikarp works): ```bash # Grab the netinst ISO (amd64 assumed; adjust if the server is something else) wget https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-13.*-amd64-netinst.iso # Identify the USB stick CAREFULLY (lsblk), then: dd if=debian-13.*-amd64-netinst.iso of=/dev/sdX bs=4M status=progress oflag=sync ``` ## 2.2 Wire the console - USB-serial adapter into your laptop; null-modem/console cable into the server's COM port (DB9 or RJ45 depending on the box). - On the laptop: ```bash # picocom (or minicom / screen — pick your poison) picocom -b 115200 /dev/ttyUSB0 # screen alternative: screen /dev/ttyUSB0 115200 ``` - In the server's BIOS/UEFI: enable **Serial Console Redirection** on COM1, **115200 8N1, no flow control**, and (if offered) "redirect after POST." Also set the boot order to USB-first for this install. If the BIOS offers "Legacy Console Redirection" for the boot menu, enable it so you can see GRUB. ## 2.3 Boot the installer on serial The Debian ISO's boot menu does **not** output to serial by default; you have to tell it. **UEFI boot (GRUB menu):** 1. When the (blind, if redirection isn't showing it) GRUB menu appears, press `e` on the **Install** entry. If you're truly blind here, the keystrokes from cold boot are usually: wait for the menu, press `↓` then `↑` to be sure you're on *Install*, press `e`. 2. Find the line starting `linux /install.amd/vmlinuz ...` and append: ``` console=ttyS0,115200n8 ``` 3. `Ctrl+X` to boot. The text installer now runs on your serial session. **BIOS/legacy boot (ISOLINUX menu):** 1. At the menu, press `Tab` on *Install* and append the same `console=ttyS0,115200n8`. *(If the server's console header maps to COM2, it's `ttyS1` — you'll find out fast because you'll get nothing on ttyS0.)* ## 2.4 Installer walkthrough (the choices that matter) - Language/locale/keymap: whatever; `en_US.UTF-8`. - **Hostname:** `fishfusion` • **Domain:** `fishtank` - **Network:** the installer will DHCP on VLAN-untagged. That's *fine for the install* (see §5.4 — if the switch port is tagged-only for VLAN 77, temporarily plug the box into an untagged VLAN 77 port, e.g. on the EX3300, or set the ICX port untagged-77 first). You'll set the static 172.20.8.5 post-install either way. - **Users:** set a root password; create your `max` user. - **Partitioning:** Guided, whole disk, **all files in one partition**, ext4. No swap drama — 2–4 GB swap is fine. (LVM if you like; nothing here needs it.) - **Software selection:** UNCHECK "Debian desktop environment" and "GNOME". CHECK **SSH server** and **standard system utilities** only. - GRUB: install to the primary disk. ## 2.5 Post-install base configuration Log in on serial (or SSH to whatever DHCP address it grabbed: check Magikarp's `/var/lib/misc/dnsmasq.leases`). **2.5.1 Make the serial console permanent** (so you can always get in when the network is broken — which is when you need it): ```bash # /etc/default/grub — set/replace these lines: GRUB_CMDLINE_LINUX_DEFAULT="" GRUB_CMDLINE_LINUX="console=tty0 console=ttyS0,115200n8 ipv6.disable=1" GRUB_TERMINAL="serial console" GRUB_SERIAL_COMMAND="serial --speed=115200 --unit=0 --word=8 --parity=no --stop=1" update-grub ``` Note `ipv6.disable=1` snuck in there — that is your **NO IPV6** requirement enforced at the kernel level. Nothing on this box will ever open an AAAA socket. (systemd auto-spawns a serial getty from the `console=` parameter; no extra unit needed.) **2.5.2 Static IP.** `/etc/network/interfaces` (Debian netinst default; substitute your NIC name from `ip -br link`, shown here as `eno1`): *Option A — switch port delivers VLAN 77 **untagged** (recommended, simplest):* ``` source /etc/network/interfaces.d/* auto lo iface lo inet loopback auto eno1 iface eno1 inet static address 172.20.8.5/24 gateway 172.20.8.1 dns-nameservers 172.20.8.1 dns-search fishtank ``` *Option B — switch port delivers VLAN 77 **tagged** (matches ICX 1/1/5 as currently configured "dual-mode 1 + tagged 77"):* ``` apt install -y vlan auto eno1 iface eno1 inet manual auto eno1.77 iface eno1.77 inet static address 172.20.8.5/24 gateway 172.20.8.1 dns-nameservers 172.20.8.1 dns-search fishtank vlan-raw-device eno1 ``` See §5.4 for the switch-side decision. Then: ```bash systemctl restart networking ip -br addr # confirm 172.20.8.5 and NO inet6 lines anywhere ping -c3 172.20.8.1 ping -c3 1.1.1.1 getent hosts fishfusion.fishtank # should return 172.20.8.5 via Magikarp dnsmasq ``` **2.5.3 Time sync** (matters for TLS, CDR timestamps, and Fail2ban windows): ```bash apt install -y chrony # /etc/chrony/chrony.conf — add Socket's NTP that your dnsmasq already advertises: # server 216.106.44.17 iburst systemctl restart chrony && chronyc sources ``` **2.5.4 Bring it current and reboot once:** ```bash apt update && apt full-upgrade -y && reboot ``` After reboot, verify serial console works blind-recovery style, `cat /proc/cmdline` shows `ipv6.disable=1`, and `ip a | grep inet6` returns nothing. --- # PART 3 — INSTALL FUSIONPBX ## 3.1 Run the installer As root: ```bash wget -O - https://raw.githubusercontent.com/fusionpbx/fusionpbx-install.sh/master/debian/pre-install.sh | sh # OPTIONAL but recommended: preset the domain before installing nano /usr/src/fusionpbx-install.sh/debian/resources/config.sh # domain_name=voip.maxdevnet.cc (this becomes the FusionPBX tenant/domain) cd /usr/src/fusionpbx-install.sh/debian && ./install.sh ``` Installs: FusionPBX, packaged FreeSWITCH, nginx, PHP-FPM, PostgreSQL, iptables rules, Fail2ban. ~5 minutes. At the end it prints the login URL, `admin` username, and a random password — **save them**. DB credentials live in `/etc/fusionpbx/config.conf` (older builds: `config.php`) if ever needed. *A note on the script's iptables:* the installer lays down its own iptables ruleset on the PBX. That's a **host** firewall layered under Magikarp's **network** firewall — keep it; defense in depth is free here. Just know it exists when debugging (§6). ## 3.2 First-login hardening & housekeeping 1. Browse to `https://fishfusion.fishtank` (accept the self-signed cert for now) → log in → change the admin password. 2. **Advanced → Domains:** confirm the domain is `voip.maxdevnet.cc`. If the installer created it as the IP, rename it. All extensions/users will live under this domain; logins become `user@voip.maxdevnet.cc`. 3. **Fail2ban:** edit `/etc/fail2ban/jail.local`: - Add `ignoreip = 127.0.0.1/8 172.20.8.0/24 192.168.1.0/24 10.10.0.0/24` so you can't ban yourself from LAN/MGMT during testing. - Per FusionPBX docs, once devices register by FQDN (not bare IP), enable the `[freeswitch-ip]` and `[auth-challenge]` jails to punish IP-based scanner traffic. 4. **NO IPv6, part 2 — FreeSWITCH profiles:** Advanced → SIP Profiles. The stock install ships `internal-ipv6` and `external-ipv6` profiles. With `ipv6.disable=1` they can't bind and will just error-spam logs. Set both to **disabled** (or delete them), then `fs_cli -x "sofia profile internal-ipv6 stop"` etc., or just restart FreeSWITCH after disabling. ## 3.3 Tell FreeSWITCH its public identity (THE dynamic-IP step) Advanced → **Variables** → find `external_sip_ip` and `external_rtp_ip` (a.k.a. `ext-sip-ip`/`ext-rtp-ip` in `vars.xml`). Set BOTH to: ``` host:voip.maxdevnet.cc ``` The `host:` prefix makes FreeSWITCH resolve the name (your ddclient-maintained record) instead of baking in an address. This is what makes SIP `Contact:`/SDP survive a Socket lease change. Then Status → SIP Status → **Rescan/Restart** the external profile (or `systemctl restart freeswitch`). **Verify:** `fs_cli -x "sofia status profile external"` → `Ext-SIP-IP` /` Ext-RTP-IP` must show your **current public IP** (the resolved value), not 172.20.8.5. ## 3.4 HTTPS for the web GUI (your 80/443 problem, addressed) Your Magikarp DNATs `tcp 80,443 → Pleco (192.168.1.125)`, so Let's Encrypt **HTTP-01 on the PBX cannot work** — the challenge lands on Pleco. Options, best-first: - **Option A (recommended): DNS-01 via Cloudflare.** You already have a CF API token on the network (ddclient uses one). On the PBX: ```bash apt install -y certbot python3-certbot-dns-cloudflare # /root/.cloudflare.ini (chmod 600): # dns_cloudflare_api_token = certbot certonly --dns-cloudflare --dns-cloudflare-credentials /root/.cloudflare.ini \ -d voip.maxdevnet.cc ``` Then point nginx's `ssl_certificate`/`ssl_certificate_key` in `/etc/nginx/sites-available/fusionpbx` at the live certs and reload nginx. Renewals are automatic and need **zero inbound ports**. - **Option B:** add a proxy host on Pleco's nginx-proxy-manager: `voip.maxdevnet.cc → https://172.20.8.5:443` with its own LE cert. Works, but now your PBX GUI transits Pleco and is exposed publicly — I'd keep the GUI LAN-only. - **Option C:** live with the self-signed cert and only hit it as `https://fishfusion.fishtank`. Zero effort, perfectly sane for a lab. Whatever you choose: there is **no reason to expose 443 of the PBX to the WAN.** Admin from LAN/WireGuard only. --- # PART 4 — TRUNK CONFIGURATION IN FUSIONPBX Create an extension first (you need somewhere to send calls): **Accounts → Extensions → Add**: extension `1001`, effective caller ID name/number set to your DID (fill the *Outbound Caller ID Number* field with `+1573XXXXXXX` for Telnyx format, `1573XXXXXXX` for Flowroute — see each gateway section). Register a softphone (Linphone on a VLAN 77 laptop, or a Poly when the hardware arrives) to `fishfusion.fishtank:5060` with `1001@voip.maxdevnet.cc` + the extension password. ## 4.1 Gateway — TELNYX Accounts → **Gateways → Add**: | Field | Value | Why | |---|---|---| | Gateway | `telnyx` | name only | | Username | *(Telnyx credential username)* | from §1A.2 | | Password | *(Telnyx credential password)* | from §1A.2 | | From User | *(same username)* | Telnyx expects it | | From Domain | `sip.telnyx.com` | keeps From: URI carrier-side | | Proxy | `sip.telnyx.com` | their SRV-balanced entry point | | Realm | *(leave blank — defaults to proxy)* | | | Expire Seconds | `120` | fast NAT/lease self-heal | | Register | **True** | ← this is the "Type C" | | Retry Seconds | `30` | quick re-register after failure | | Register Transport | `udp` | matches connection settings | | Context | `public` | inbound lands in public dialplan | | Profile | `external` | port 5080, carrier-facing | | Enabled | True | | Save → **Start** on the gateway list → state must reach **REGED**. ```bash fs_cli -x "sofia status gateway telnyx" ``` ## 4.2 Gateway — FLOWROUTE Accounts → **Gateways → Add**: | Field | Value | |---|---| | Gateway | `flowroute` | | Username | *(Access Key / tech prefix)* | | Password | *(SIP password)* | | From User | *(same Access Key)* | | From Domain | `sip.flowroute.com` | | Proxy | `sip.flowroute.com` | | Expire Seconds | `120` | | Register | **True** | | Retry Seconds | `30` | | Context | `public` | | Profile | `external` | | Enabled | True | Same REGED check with `sofia status gateway flowroute`. ## 4.3 Inbound routing (Destinations) **FIRST, empirically confirm the DID format** (§6.3 shows the sngrep procedure). Then: Dialplan → **Destinations → Add**: - Type: `Inbound` - Destination: - Telnyx (with +E.164 selected in §1A.2): `+1573XXXXXXX` - Flowroute (typical default): `1573XXXXXXX` — *but trust sngrep, not this document* - Context: `public` - Action: `1001` (or later an IVR/ring group/time condition) - Description: `Telnyx DID -> x1001` Saving auto-generates the inbound route in the public context. If inbound 404s later, it is a format mismatch here 95% of the time. ## 4.4 Outbound routes (normal calls) Dialplan → **Outbound Routes → Add** — create two (or use the "North America" preset and edit): **Route 1 — 11-digit (1+10):** - Gateway: `telnyx` (or `flowroute`) - Dialplan Expression / condition: `^(1\d{10})$` - Telnyx: prefix the dialed number with `+` (send `+1NXXNXXXXXX`). - Flowroute: send as-dialed `1NXXNXXXXXX` (Flowroute accepts 11-digit and E.164; pick one and be consistent). **Route 2 — 10-digit:** - Expression: `^(\d{10})$` - Telnyx: prepend `+1`. Flowroute: prepend `1`. Set each extension's **Outbound Caller ID Number** to your DID in the same format the provider expects (Telnyx: `+E.164`; Flowroute: 11-digit). Sending a CLI you don't own gets rewritten or rejected, and correct owned-number CLI is what earns you clean STIR/SHAKEN attestation. ## 4.5 Emergency outbound route (E911) — build it deliberately Dialplan → **Outbound Routes → Add**: - **Order/priority:** lower number than every other route (evaluate FIRST). - Expression: `^(911|933)$` - Gateway: the provider where E911 is provisioned. **Never** let 911 fail over to a gateway that lacks an E911 record — better to fail loudly than to place an unlocatable emergency call. - **Caller ID Number for this route: pin it to the E911-registered DID**, overriding any user/extension CID. In the generated dialplan, that's an `effective_caller_id_number` set action on this route — the E911 record is keyed to the DID; if a different CLI goes out, dispatch gets no address. - Description: `EMERGENCY - E911 via - DO NOT REORDER`. Multi-site reminder (future you): E911 address is per-DID. When phones exist at Jeff/Linda sites over the mesh, each site needs its own DID+address, and each site's extensions need their emergency CID pinned to *their site's* DID. ## 4.6 Codec sanity Advanced → Variables → `global_codec_prefs` / `outbound_codec_prefs`: `PCMU,PCMA,G722` (in that order, or G722 first if you want HD wherever possible — PCMU-first is maximally boring and interoperable; boring is good on day one). --- # PART 5 — NETWORK CHANGES (MAGIKARP + SWITCHES) ## 5.1 nftables: new sets (mirror your SOCKET_SIP pattern) In `/etc/nftables.conf`, next to `define SOCKET_SIP`: ```nft # SIP signaling ranges -> HPBX (PULL CURRENT LIST FROM PROVIDER DOCS) define TELNYX_SIP = { x.x.x.x/xx, ... } # RTP media ranges define TELNYX_RTP = { x.x.x.x/xx, ... } # (FLOWROUTE_SIP / FLOWROUTE_RTP identically if going that way) define FISHFUSION = 172.20.8.5 ``` ## 5.2 nftables: forward-chain accepts Add beside your existing `# Socket Telecom SIP ranges -> HPBX` rule: ```nft # Carrier SIP/RTP -> Fishfusion (VLAN77) iifname $WAN_IF oifname $VLAN77_IF ip saddr $TELNYX_SIP udp dport 5080 accept iifname $WAN_IF oifname $VLAN77_IF ip saddr $TELNYX_RTP udp dport 16384-32768 accept ``` **And fix the DNAT-accept scope.** Your current rule is: ```nft iifname $WAN_IF oifname $LAN_IF ct status dnat accept ``` It only matches DNAT toward `$LAN_IF`, so DNATed SIP toward VLAN 77 would die in forward. Either broaden it: ```nft iifname $WAN_IF oifname { $LAN_IF, $VLAN77_IF } ct status dnat accept ``` or rely solely on the explicit accepts above (both is fine; explicit + broad is belt and suspenders). ## 5.3 nftables: DNAT (the "port forwards" — this is the §4 answer) Strictly, a registering trunk can ride conntrack alone — but carrier proxy clusters send INVITEs from hosts other than the one you registered to, and early media can arrive before your first outbound RTP packet. Both produce *intermittent* inbound failures, the worst kind. Add static DNAT, source-locked to the provider: ```nft chain prerouting { ...existing rules... # Carrier SIP -> Fishfusion external profile iifname $WAN_IF ip saddr $TELNYX_SIP udp dport 5080 dnat ip to $FISHFUSION # Carrier RTP -> Fishfusion media range iifname $WAN_IF ip saddr $TELNYX_RTP udp dport 16384-32768 dnat ip to $FISHFUSION } ``` Then `nft -f /etc/nftables.conf` (it flushes and reloads atomically per your file's `flush ruleset`) and verify with `nft list chain inet nat prerouting`. **What NOT to forward:** 5060 (internal profile — extensions only, never WAN-facing), 443/80 of the PBX (admin from LAN/WG only), and nothing at all from non-provider sources. Random-scanner SIP traffic to 5080 keeps falling through to your drop rules, exactly like the Socket pattern today. ## 5.4 The flowtable/offload caveat (your own past decision, now relevant) Your flowtable includes `enp16s0f1np1.77`, and the first forward rule offloads all established UDP/TCP — including RTP. You previously noted (Feb chat) SIP/VoIP as the case to exempt from offload. Now that VLAN 77 carries real calls: if you see one-way audio, dead first-seconds, or audio dying mid-call, exclude VLAN 77 (or just the provider RTP ranges) from the offload rule: ```nft ct state established,related ip saddr != $VLAN77_NET ip daddr != $VLAN77_NET \ flow add @ft counter accept ct state established,related accept ``` Don't pre-pessimize — try with offload first; a couple of G.711 streams is nothing. Just know this lever exists before you spend an evening blaming the carrier. ## 5.5 Switch port for the PBX ICX `1/1/5` per your plan. Two clean choices matching §2.5.2: - **Untagged VLAN 77 (Option A, recommended):** make 1/1/5 an access port in VLAN 77 (`vlan 77 / untagged ethernet 1/1/5`, remove dual-mode-1/voice-vlan from this port). The PBX is a single-purpose VLAN 77 host; untagged is simpler and survives OS reinstalls without VLAN packages. - **Tagged (Option B):** leave 1/1/5 as dual-mode-1 + tagged 77 and run `eno1.77` on Debian. Only worth it if you want the PBX reachable on main LAN too (you don't — inter-VLAN routing through Magikarp already covers management access, and your forward chain already allows LAN↔VLAN77). ## 5.6 DNS / DDNS — already done, just verify - dnsmasq already has `address=/fishfusion/pbx/172.20.8.5` → `fishfusion.fishtank` and `pbx.fishtank` both resolve. ✔ Nothing to add. - ddclient already maintains `voip.maxdevnet.cc`. ✔ Verify grey-cloud + TTL per §0. - Optional pre-staging for a future FQDN-auth migration: SRV record `_sip._udp.voip.maxdevnet.cc. 300 IN SRV 10 10 5080 voip.maxdevnet.cc.` (grey-cloud the A it targets). Harmless now, useful later. --- # PART 6 — ACCEPTANCE TESTING (in order; each gates the next) Install the tools first: `apt install -y sngrep tcpdump` on the PBX. ## 6.1 Layer 0 — plumbing - [ ] `ip -br addr` → `172.20.8.5/24`, zero `inet6` anywhere. - [ ] `ping 172.20.8.1`, `ping 1.1.1.1`, `dig +short voip.maxdevnet.cc @172.20.8.1` returns current WAN IP (compare `curl -4 ifconfig.me` from any LAN box). - [ ] From LAN: `ping fishfusion.fishtank`. From PBX: `getent hosts pbx.fishtank`. - [ ] `chronyc tracking` → sane offset. ## 6.2 Registration - [ ] Gateway shows **REGED** in GUI and in `fs_cli -x "sofia status gateway telnyx"` (`State: REGED`, sane `Ping` times). - [ ] On the PBX, `sngrep` → watch a REGISTER cycle: REGISTER → 401 → REGISTER(auth) → **200 OK**. A 403 or a second 401 = credential problem. - [ ] `fs_cli -x "sofia status profile external"` → Ext-SIP-IP = current public IP. ## 6.3 Inbound DID format discovery (before trusting §4.3) - [ ] `sngrep` running; call the DID from a cell. - [ ] Even if it fails to route, the INVITE shows the exact Request-URI user part (`+1573…` vs `1573…` vs `573…`). Make Destinations match **that**, exactly. ## 6.4 Inbound call - [ ] Cell → DID → softphone/extension 1001 rings; answer; **talk both directions ≥60 s** (past the 32 s ACK-timeout cliff that exposes NAT/ext-ip bugs). - [ ] Repeat 5–10 times spaced out. Intermittent failure = INVITEs from a proxy your DNAT set doesn't cover → recheck provider IP list & `nft list ruleset | grep 5080` counters. - [ ] Hang up from each end once; confirm clean teardown (no zombie channels in `fs_cli -x "show channels"`). ## 6.5 Outbound call - [ ] 1001 → your cell, 10-digit and 1+10-digit forms. Two-way audio ≥60 s. - [ ] Caller ID on the cell shows your DID (CNAM may lag days; number must be right). - [ ] Check the provider portal's CDR/log — the call should show your expected CLI and A-level attestation (Telnyx shows attestation in call detail). - [ ] DTMF test: call something menu-driven (your bank's IVR, or `*97`-style voicemail once configured) and confirm digits register (RFC2833). ## 6.6 Audio-path forensics (only if 6.4/6.5 misbehave) - One-way/no audio → `sngrep`, press Enter on the call, check SDP: the `c=` line your PBX sends **must** be the public IP (§3.3), and inbound RTP must arrive (`tcpdump -ni eno1 udp portrange 16384-32768`). - Audio dies ~30 s in → far end never got your ACK → almost always `ext-sip-ip` stale/wrong or 5080 signaling DNAT missing. - First-second clipping inbound → early-media before outbound RTP → confirms the RTP DNAT from §5.3 is doing its job (or isn't). ## 6.7 E911 — test with 933, never 911 - [ ] Dial **933** from 1001. The system reads back the caller's number and the registered street address. Verify the address is the Max site, verbatim. - [ ] Confirm in `sngrep` that the outbound 933 INVITE carried the **E911 DID** as CLI (your §4.5 route pin working). - [ ] Negative test: temporarily set 1001's CID to garbage, dial 933 again — route should still force the E911 DID. Restore afterward. ## 6.8 The DHCP resilience test (your environment's special) - [ ] Simulate a lease change: on Magikarp, bounce the WAN (`dhclient -r enp16s0f0np0 && dhclient enp16s0f0np0` or your netifrc equivalent) during a quiet moment, ideally forcing a new IP if Socket will give one (spoof the WAN MAC if you want to force it — and note that permanently changes your "sticky" IP). - [ ] Watch the chain heal: ddclient updates Cloudflare (check `dig +short` against the new IP within TTL), gateway re-REGisters within ≤120 s + retry window, `sofia status profile external` shows the **new** Ext-SIP-IP after the profile re-resolves (if it doesn't, `fs_cli -x "sofia profile external restart"` and note that a periodic re-resolve or a hook on ddclient is your permanent fix). - [ ] Inbound and outbound test calls pass post-change. ## 6.9 Security regression - [ ] From an outside host (phone hotspot): `nmap -sU -p 5060,5080 ` → filtered/no-response (source-locked DNAT means outsiders see nothing). - [ ] `nmap -p 80,443,5060 ` → 80/443 land on Pleco as before, no PBX. - [ ] `fail2ban-client status freeswitch` on the PBX → jail active. - [ ] Watch `journalctl -u fail2ban -f` for a day; scanners on 5060 internal should never appear (they can't reach it), which is the point. - [ ] Provider portal: spend limit + destination whitelist still US/CA only. ## 6.10 Sign-off criteria Inbound ×10 clean, outbound ×10 clean, ≥60 s two-way audio, DTMF good, 933 reads the right address with the right CLI, trunk survives a WAN bounce unattended, and nothing answers on 5060/5080 from a random outside IP. That's a production-grade Type C trunk. --- # APPENDIX A — QUICK TROUBLESHOOTING TABLE | Symptom | First suspect | Tool/fix | |---|---|---| | Gateway TRYING/FAIL_WAIT | credentials, or 5080 egress blocked | sngrep REGISTER trace | | REGED but inbound dead | DID→connection mapping (portal) or DNAT/forward | provider call logs + `nft ... counter` | | Inbound 404 | Destination format mismatch | sngrep INVITE Request-URI | | Outbound 403/rejected | CLI not an owned DID, or destination not whitelisted | provider portal call detail | | One-way audio | ext_rtp_ip stale / RTP DNAT / offload | §6.6 | | Dies at ~30 s | ext_sip_ip stale (ACK lost) | §3.3, restart external profile | | Broken after IP change | DDNS lag or profile cached old IP | §6.8 | | Choppy audio under load | flowtable offload interaction | §5.4 exemption | | Random 5080 hits in logs | provider IP list changed | refresh nftables sets | # APPENDIX B — DELIBERATELY OUT OF SCOPE (future work) - FQDN-auth migration (converts Type C → non-registering peer; SRV pre-staged §5.6) - TLS/SRTP on the trunk - Second-provider failover gateway + outbound route ordering - TA908e FXS integration at 172.20.8.15 (fishtank908) - Poly phone auto-provisioning on VLAN 77 (separate guide already discussed) - Jeff/Linda site DIDs + per-site E911 + emergency-CID pinning over the mesh