insights
Abdulmuhsen AlayouniInsights
homeget in touch ↗

/ insights

When Latency Isn't Bandwidth: Symmetric NAT, Relay Detours, and the Static Port Fix

Abdulmuhsen Alayouni · 10 min read

A friend asked me to look at a latency problem. A real-time app felt sluggish, calls lagged, and every speed test came back clean. Their connection was, on paper, excellent: wired to the router, about 2 ms to the ISP, and the service they were reaching had a node roughly 20 ms away. Nothing about the link explained the delay they were feeling.

The link was not the problem. The path was. Their traffic was quietly taking the long way around, and the cause was not bandwidth, not the ISP, and not the app. It was one setting on their own firewall. This is the whole story: how I found it, how I proved it, and the single change that fixed it.

The latency a speed test can't see

Speed tests measure throughput to a nearby server. They tell you how fat the pipe is. They tell you almost nothing about the path a specific application actually takes to a specific peer. You can have a gigabit line, 2 ms to your ISP, and still watch a voice call sit at 130 ms of round-trip latency, because the packets are not going where you assume they are going.

When I looked at the actual flows instead of the speed test, the picture changed. The app was not connecting directly to the node 20 ms away. Every packet was being handed to a relay server in another region first, then forwarded on to the real destination. The direct hop should have been around 20 ms. The detour through the relay was adding somewhere in the range of 80 to 150 ms depending on where that relay sat. Same clean line, same fast ISP, and a session that felt slow for a reason no bandwidth graph would ever show.

How real-time apps actually connect

To understand why the traffic detoured, you have to know how modern real-time apps try to connect two machines that both sit behind home routers. Voice, video, screen sharing, game netcode, and a lot of IoT use a framework called ICE (Interactive Connectivity Establishment, RFC 8445). ICE gathers several candidate ways to reach a peer and then tests them to find one that works.

There are three kinds of candidate:

  • Host candidates: the device's own local addresses. Useless across the internet, fine on a LAN.
  • Server-reflexive candidates: the public address and port your router shows to the outside world. The device learns this by asking a STUN server (RFC 8489) a simple question: "from where you sit, which address and port do you see me on?"
  • Relayed candidates: an address on a TURN relay (RFC 8656), a public server that will forward traffic on your behalf.

ICE always prefers a direct path. If both peers can learn their public server-reflexive addresses and reach each other on them, they establish a direct connection through a technique called NAT traversal, or hole punching. The relay is the fallback. TURN exists precisely because sometimes a direct path cannot be formed, and a working slow path beats a fast path that never connects. The catch is that the fallback is silent. The app does not warn you it gave up on the direct route. It just relays, the session works, and you never learn why it feels slow.

The real culprit: NAT mapping behavior

So why could the direct path not form? The answer lives in a corner of networking most people never touch: how a NAT decides which external port to assign when an internal device opens a connection. RFC 4787 is the document that defines this behavior, and it splits it into two separate characteristics that people constantly blur together.

The first is mapping behavior: when your device sends out from one internal address and port, what external address and port does the NAT put on it, and does that external mapping stay the same when the device talks to a different destination?

  • Endpoint-Independent Mapping (EIM): the NAT reuses the same external mapping no matter which remote host you contact. Talk to server A or server B from the same local port, and the outside world sees you on the same public port. This is the behavior NAT traversal needs, because the address your peer learns from STUN is the same address your peer will actually reach you on.
  • Address-Dependent or Address-and-Port-Dependent Mapping: the NAT hands out a different external mapping depending on the destination. Older docs lump this under symmetric NAT, though strictly that term is the address-and-port-dependent case. RFC 4787 retired the word "symmetric" precisely because it blurred mapping and filtering together. The label matters less than the consequence, which is identical for both: the mapping is not endpoint-independent.

Here is the part that trips people up, and it is worth being precise about because it is the most common myth in this whole topic. Symmetric NAT is not defined by handing out random ports. The defining trait is that the external mapping changes based on where you are sending. The ports might be sequential, they might be random, it does not matter. What breaks traversal is that the public address your peer learns for you (by watching what a STUN server saw) is not the address the NAT will use when your peer tries to reach you, because your peer is a different destination and therefore gets a different mapping. The address your peer was handed is not the address that actually reaches you.

Proving it with STUN

This is the kind of claim you should never make without measuring, so I measured. The test is simple and it is the same question ICE asks: I sent STUN binding requests to several different STUN servers and recorded the external port each one reported back.

One detail matters more than any other here, and it is where a sloppy test gives a false result: every request went out from the same local UDP socket, meaning the same internal source port. That is the only way the answer means anything. If a test tool opened a fresh socket per server, each request would leave from a different internal port, and of course you would see different external ports, from the client, not the NAT. Bind one socket, reuse it, ask everyone the same question. Then the only variable is the NAT's mapping behavior.

Five servers, one socket, one question. The answers came back like this:

  • Google STUN: external port 14983
  • Cloudflare STUN: external port 19635
  • Nextcloud STUN: external port 25650
  • Voipbuster STUN: external port 17358
  • Ekiga STUN: external port 50648

Five destinations, five different external ports, from a single unchanging local port. That is the signature of endpoint-dependent mapping. To be exact about what this proves: five different servers show the mapping is not endpoint-independent, and that is the part that breaks traversal. Proving it is specifically the address-and-port-dependent variety, the strict "symmetric" case, would take a second probe to a different port on the same server IP. For this problem the not-independent part is the whole story, and that is nailed down. The firewall was minting a new public identity for every place the device talked to. No peer could be handed a stable address to reach it on, so ICE could never validate a direct path, and it fell back to the TURN relay every time. The relay was never the bug. It was the symptom. The bug was the mapping.

A quick word on filtering

RFC 4787 defines a second characteristic alongside mapping, called filtering behavior, which controls which inbound packets the NAT will let back in to an existing mapping. It runs on the same scale: endpoint-independent, address-dependent, and address-and-port-dependent filtering. Both mapping and filtering influence whether hole punching succeeds, and a truly stubborn case can involve either. In this case the mapping was the dominant problem, and fixing the mapping was enough to restore the direct path, so I am keeping the focus there. Just know that "it's the NAT" can mean two different knobs, and a complete picture checks both.

The fix: Static Port on the outbound NAT

The firewall was doing outbound NAT with source-port rewriting, assigning a fresh external port per destination. On this platform the lever that changes that is a setting called Static Port on the outbound NAT rule. Static Port tells the NAT to stop rewriting the source port and instead preserve the device's original internal port as the external port.

Be careful here, because this is the second place people overstate the story. Static Port is a port preservation setting. It is not, by definition, the same thing as Endpoint-Independent Mapping. What it does is stop the NAT from inventing a new port per flow. When the application then uses a single stable local port for its traffic, preserving that port means the external mapping stays consistent across destinations, which is exactly the endpoint-independent behavior that RFC 4787 describes and that traversal needs. In other words, Static Port did not "equal" EIM. It produced the EIM-like behavior the app required, in this setup, and the only way to be sure of that is to measure the result rather than trust the label. Some newer firewall builds also expose an explicit Endpoint-Independent Mapping option separate from Static Port. If yours does, that is the more direct lever, and the same measurement confirms it either way.

So I flipped Static Port and ran the identical STUN test again, same socket, same five servers.

  • Google STUN: external port 8676
  • Cloudflare STUN: external port 8676
  • Nextcloud STUN: external port 8676
  • Voipbuster STUN: no answer on the retest
  • Ekiga STUN: external port 8676

One external port, everywhere. (Voipbuster's STUN server simply did not answer the second time, which happens with public STUN servers and does not change the result.) The firewall was now presenting a single stable mapping to the whole internet. The address a peer learned was finally the address it could reach. ICE validated the direct candidate, the relay hop dropped out of the path, and the latency the friend had been living with collapsed back to what the physical link could actually do. Same PC, same servers, same test, minutes apart. The only thing that changed between one column of results and the other was one line of firewall configuration.

The honest caveats

A story like this gets shared, and someone who knows this topic will read it, so here is where it does and does not apply. I would rather you trust the parts that are true than oversell the whole thing.

  • Traversal is a two-sided negotiation. Your NAT is only half the equation. The peer's NAT and filtering behavior matter too, and a hostile combination on the other end can still force a relay even after you fix your side. Here, fixing this edge was enough. It is not guaranteed to always be enough.
  • The latency penalty depends on geography. A relay that happens to sit near both peers adds little. A relay in another region adds the 80 to 150 ms I saw. The number is real for this case, not a universal constant.
  • This is about apps that attempt peer-to-peer. WebRTC-based voice and video, some remote-desktop tools, peer game netcode, and certain IoT protocols try to connect directly and benefit from this. Plenty of "real-time" products are deliberately client-server or always-relayed by design, and no amount of NAT tuning changes their path.
  • Carrier-grade NAT can defeat all of it. If your ISP puts you behind CGNAT, the mapping behavior that matters may live on their equipment, not yours, and your firewall setting cannot override it. Worth checking before you chase your own config.
  • There is a small trade. A predictable, preserved external port is exactly what traversal wants, and it very slightly reduces the randomness of your outbound source ports. For a home or small-office edge enabling this for real-time traffic, that trade is almost always worth it. It is still worth knowing you made it.

How to check your own edge

If your real-time apps feel slow and the speed test keeps coming back clean, look at the path instead of the pipe.

  • Run a STUN reflexive-address check against several STUN servers from the same local socket. If you get a different external port from each server, you have endpoint-dependent (symmetric) mapping, and that is your prime suspect.
  • Watch where your session's packets actually go. If a "nearby" service is reached through a relay in another region, you are paying the detour tax.
  • On the firewall, find the outbound NAT and enable Static Port for the traffic in question, or the explicit Endpoint-Independent Mapping option if your platform has one. Then re-run the STUN check and confirm you now see one consistent external port.
  • Rule out CGNAT first if you can. If your WAN address is already a carrier-shared address (RFC 6598's 100.64.0.0/10 range is the CGNAT giveaway) rather than a real public IP, the lever you need may not be on your box at all.

The reason this problem hides so well is that everything works. The call connects, the screen shares, the app runs. Nothing errors out and points at the cause. It is just slower than it should be, quietly, through a middleman you never chose, because of a mapping decision happening at the edge where almost nobody looks. That is the whole lesson, and it is the one I keep coming back to across every system I touch: you can't fix what you can't see. The fix was one setting. Finding it was the entire job.