How a NAT Gateway Fixes Asymmetric Routing for UDP in AWS Architectures

Problem Analysis

When customers build a cross‑region, high‑availability, low‑latency architecture on AWS using Amazon Global Accelerator (AGA), Network Load Balancer (NLB) and Direct Connect (DX), UDP traffic often fails because the request and response follow different network paths.

TCP vs UDP Handling

TCP case: NLB can disable the source‑IP preservation feature, allowing session tracking and correct return routing.

UDP case: NLB cannot disable source‑IP preservation, so packets keep the original source IP (client or AGA public IP). In scenarios that require bidirectional UDP communication, such as a custom QUIC implementation, this leads to routing problems.

Asymmetric Routing Details

The request path is:

User request reaches the AGA endpoint.

AGA forwards the request to the associated NLB.

NLB receives the request and forwards it via DX to the IDC target group.

Because the source‑IP preservation cannot be disabled for UDP, the traffic arriving at IDC retains the original source IP (client or AGA public IP).

The response path is:

IDC server generates a response packet addressed to the original source IP.

The response tries to return directly to the client or AGA public IP, bypassing the original request path.

Due to differing internet routing configurations of DX and IDC, the response may be sent through IDC’s own internet instead of DX, causing firewall blocks, routing mismatches, or session mismatches, and the packet is dropped.

Symptoms of Asymmetric UDP Communication

No response after sending a request.

Network capture shows outbound packets but missing inbound packets.

One‑way connectivity (client can reach server, but server cannot reach client), e.g., in QUIC.

Solution: Introduce a NAT Gateway for Symmetric Routing

The proposed solution adds a NAT gateway in a private subnet to enforce symmetric routing while preserving the original architecture.

Architecture Improvement Details

Retain the original AGA‑NLB‑DX core.

Insert a NAT gateway in the private subnet.

Adjust route tables precisely:

How the NAT Gateway Resolves Asymmetry

Improved request path:

User request → AGA → NLB → Public subnet.

Public subnet route sends traffic destined for IDC IP targets to the NAT gateway.

NAT gateway performs source‑IP translation, using its own IP as the source.

Translated request → DX → IDC.

Improved response path:

IDC server responds to the NAT gateway IP.

Response reaches the NAT gateway via DX.

NAT gateway looks up the translation table and restores the original client/AGA public IP as the destination.

Response is routed back to the NLB, then to AGA, and finally to the user.

用户 → AGA → NLB → 公有子网 → [路由表] → NAT 网关(源IP转换) → DX → 客户IDC

客户IDC → DX → NAT网关(目的IP转换) → [路由表] → NLB的ENI → NLB → AGA → 用户

Advanced NAT Gateway Configuration: Multi‑IP Support

Amazon NAT gateway can attach up to seven Elastic IP addresses, which linearly scales concurrent UDP connections (each IP provides up to 55,000 connections). For example, adding five IPs supports up to 275,000 concurrent connections.

Efficient port allocation algorithm distributes traffic across all available IPs.

Dynamic port allocation ensures smooth connection establishment without service interruption.

Practical Scale Benefits

Game server scenario: Massive simultaneous player connections use UDP; multi‑IP NAT ensures sufficient port resources per session.

IoT device communication: Burst connections from many devices do not exhaust port resources, maintaining reliable communication.

Configuration Steps (Example with Public Subnet B and Private Subnet B)

Deploy NAT gateway B in private subnet B (Private type). Consider attaching extra IPs based on traffic estimates.

Configure route table for public subnet B:

Add a route directing traffic for the IDC NLB IP target range (e.g., 10.x.x.0/24 → nat-gateway-id-B) to the NAT gateway.

Configure route table for private subnet B:

Add a route directing return traffic to the NLB ENI (e.g., 0.0.0.0 → eni-nlb-interface-id).

Validate symmetric routing using traceroute or packet capture to ensure UDP traffic flows bidirectionally.

Cost‑Benefit Analysis

Introducing a NAT gateway incurs additional costs:

Fixed costs: hourly NAT gateway charge and per‑attached IP address fee.

Variable costs: data processing (per GB) and cross‑AZ data transfer fees.

Conclusion

By inserting a NAT gateway into the AGA‑NLB‑DX architecture and fine‑tuning route tables, the asymmetric routing issue that blocks UDP communication is resolved. The multi‑IP capability of the NAT gateway further enhances scalability, supporting extremely high‑concurrency UDP scenarios while preserving the original architecture’s advantages. Although costs increase, the gains in connection stability, scalability, and business continuity make this a high‑value solution.