How VXLAN Transforms Data Center Networking: Principles and Real-World Deployment

1. Background

With the rapid development of data centers, traditional VLAN limits and Layer‑2 isolation cannot meet the needs of multi‑tenant environments and VM migration. Virtualization, scaling, and cloud computing drive the need for a more flexible network, leading to the emergence of VXLAN.

2. VXLAN Technical Principles

2.1 What is VXLAN?

VXLAN (Virtual Extensible LAN) is a virtualization technology that extends a Layer‑2 network over a Layer‑3 IP fabric, enabling VM migration across subnets, data centers, and regions.

2.2 VXLAN Gateway Deployment

VXLAN typically uses a distributed gateway architecture in a spine‑leaf topology. Leaf nodes act as VXLAN tunnel endpoints (VTEPs) and can serve as both Layer‑2 and Layer‑3 gateways, while spine nodes forward traffic without being aware of the VXLAN tunnels.

2.3 VXLAN Communication Process

Servers communicate through a VXLAN tunnel established between TOR switches. The original Ethernet frames are encapsulated in UDP (port 4789) with a VXLAN header, transmitted over the IP network, and decapsulated at the destination TOR.

3. VXLAN Practical Implementation at Ziroom

3.1 Construction Background

Rapid data‑center expansion and increasing cross‑center traffic exposed limitations of traditional Layer‑2 and core‑TOR architectures, prompting the adoption of VXLAN to enhance scalability and VM mobility.

3.2 Architecture Design

The solution combines BGP‑EVPN with a distributed VXLAN gateway and M‑LAG for link aggregation, providing high availability and improved forwarding performance.

3.3 MTU Planning for Cross‑DC VXLAN Interconnection

To avoid fragmentation of VXLAN packets (which add ~50 bytes), the MTU on the underlay network should be increased, typically to 1600 bytes. Tests showed significant performance improvement after MTU adjustment.

4. Practice Summary

Key considerations for integrating M‑LAG with VXLAN include consistent configuration across primary and backup devices, matching VLANIF/VBDIF IP and virtual MAC settings, and ensuring distinct MAC addresses for NVE interfaces in different M‑LAG systems while keeping them consistent within the same system.