Understanding VXLAN: Concepts, Architecture, and Deployment in Data Center Networks

What is VXLAN

VXLAN (Virtual Extensible LAN) is an IETF‑defined NVO3 (Network Virtualization over Layer 3) standard that extends traditional VLAN by encapsulating Layer‑2 Ethernet frames inside UDP packets, creating a virtual tunnel that can be carried over an IP network.

Why VXLAN is Needed

Data‑center servers are increasingly virtualized, leading to two key requirements: seamless VM migration across physical hosts and isolation for a massive number of tenants. VXLAN satisfies both by providing a large address space (24‑bit VNI) and a virtual Layer‑2 domain that spans the entire IP fabric.

Server Virtualization and VM Migration

Traditional physical servers have low utilization (10‑15%). Server virtualization creates multiple logical VMs on a single host, each with its own MAC and IP, improving resource usage and enabling live migration. Live migration requires the VM to keep its IP and MAC unchanged, which is only possible within a single Layer‑2 domain; VXLAN extends this domain across the data‑center.

VXLAN vs. VLAN

VLAN supports only about 4,000 IDs, insufficient for large‑scale data‑centers. VXLAN introduces a 24‑bit VNI, supporting up to 16 million virtual networks, and builds a virtual Layer‑2 “switch” over the IP fabric, enabling VM migration across a much larger domain.

VXLAN Packet Format

VXLAN Header (8 bytes) – contains the 24‑bit VNI, flags, and reserved fields.

UDP Header – destination port fixed at 4789; source port derived from a hash of the inner frame.

Outer IP Header – source IP is the local VTEP, destination IP is the remote VTEP.

Outer MAC Header – source MAC is the local VTEP, destination MAC is the next‑hop MAC on the underlay network.

VXLAN Tunnel Establishment

Two methods are used:

Manual (static) method : the administrator configures each VTEP with the peer’s IP address. Example configuration on a Huawei CloudEngine switch:

vni 5000 head-end peer-list 2.2.2.2

vni 5000 head-end peer-list 2.2.2.3

This creates a “head‑end replication list” that tells a VTEP which remote VTEPs belong to the same VNI.

Automatic method : uses EVPN (Ethernet VPN) to distribute VTEP information dynamically. The detailed steps are documented in Huawei’s VXLAN configuration guide.

VTEP and VNI

A VTEP (VXLAN Tunnel Endpoint) is the edge device that performs encapsulation and decapsulation of VXLAN packets. It can be a physical switch (e.g., Huawei CloudEngine) or a virtual switch inside a server. The VNI (VXLAN Network Identifier) works like a VLAN ID but with a 24‑bit space, uniquely identifying a tenant or broadcast domain.

Bridge Domain (BD) and Layer‑2 Sub‑Interface

In VXLAN, a Bridge Domain (BD) maps 1:1 to a VNI. Traffic is classified by assigning Layer‑2 sub‑interfaces (dot1q, untag, qinq, default) to a BD. Only packets that match the sub‑interface configuration are sent into the VXLAN tunnel.

VXLAN Gateways

Layer‑2 gateway : provides access to the VXLAN network and enables intra‑VXLAN subnet communication.

Layer‑3 gateway : enables inter‑subnet communication within VXLAN and connectivity to external networks.

Gateways can be centralized (a single device handles all routing) or distributed (each leaf switch acts as a gateway in a spine‑leaf architecture).

Distributed gateways avoid the ARP table scalability issue of centralized designs by keeping ARP entries only for locally attached servers.

Packet Forwarding in VXLAN

For same‑subnet traffic, a VM sends an ARP broadcast; the VTEP replicates the ARP to all peers in the VNI’s replication list, learns MAC‑VNI‑port bindings, and forwards the encapsulated packet. Once MAC addresses are learned, subsequent traffic uses unicast.

For cross‑subnet traffic, the VM’s packet is first sent to its VTEP’s BD interface (Layer‑3 gateway). The VTEP routes the packet, encapsulates it with the appropriate VNI, and forwards it through the VXLAN tunnel to the remote VTEP, which then decapsulates and delivers it to the destination VM.

Both centralized and distributed VXLAN deployments follow the same encapsulation/decapsulation steps; the main difference lies in where the routing decision and ARP learning occur.

Conclusion

VXLAN provides a scalable overlay that turns an IP underlay into a massive virtual Layer‑2 fabric, enabling seamless VM migration, massive tenant isolation via VNI, and flexible gateway designs to suit various data‑center architectures.