How VXLAN Extends VLAN for Scalable Cloud Networks – A Hands‑On Linux and OVS Guide

In cloud environments, multi‑tenant isolation requires each tenant to have its own virtual logical network. Traditional VLANs provide only 12‑bit identifiers (4096 networks), which is insufficient for large‑scale clouds and requires manual configuration of physical switches.

VXLAN Fundamentals

VXLAN (Virtual eXtensible LAN) expands the identifier space to 24 bits, offering 16,777,216 possible VNIs (VXLAN Network Identifiers). It encapsulates original Ethernet frames inside UDP packets, allowing an overlay L2 network to run over an IP underlay.

The encapsulation adds a 50‑byte overhead, so VM MTU may need to be reduced or the physical network MTU increased. Devices that perform encapsulation/decapsulation are called VTEPs (VXLAN Tunnel End Points) and can be hardware or software based.

Linux Kernel Implementation

Three hosts (192.168.33.12‑14) each run a Linux bridge (br0) with IPs 10.1.1.2‑4. After creating the bridges, a VXLAN tunnel (vxlan0) is added on each host, using multicast group 239.1.1.1 for BUM (Broadcast, Unknown‑unicast, Multicast) traffic.

brctl addbr br0

ip addr add 10.1.1.2/24 dev br0
ip link set up br0

ip link add vxlan0 type vxlan id 1 group 239.1.1.1 dev eth1 dstport 4789

brctl addif br0 vxlan0

After configuring all three hosts, the bridge’s forwarding database initially contains only the multicast entry. Once traffic flows, the VTEP learns remote MAC‑VNI mappings and updates the FDB, enabling direct unicast forwarding.

# bridge fdb show dev vxlan0
00:00:00:00:00:00 dst 239.1.1.1 via eth1 self permanent
2e:7a:a2:53:7b:31 dst 192.168.33.13 self

Ping tests confirm connectivity once the learning completes.

Open vSwitch (OVS) Implementation

OVS does not support IP multicast, so unicast VXLAN tunnels are created between each pair of hosts. Two OVS bridges are used: br-int for internal VM traffic and br-tun for VXLAN tunnels. A PATCH port connects the two bridges.

ovs-vsctl add-br br-int
ovs-vsctl add-br br-tun

ovs-vsctl add-port br-int patch-int -- set interface patch-int type=patch options:peer=patch-tun
ovs-vsctl add-port br-tun patch-tun -- set interface patch-tun type=patch options:peer=patch-int

ovs-vsctl add-port br-tun vxlan0 -- set interface vxlan0 type=vxlan options:remote_ip=192.168.33.13
ovs-vsctl add-port br-tun vxlan1 -- set interface vxlan1 type=vxlan options:remote_ip=192.168.33.14

Flow tables on br-tun are programmed to separate traffic sources, handle BUM flooding, and implement learning:

# Table 0 – classify incoming ports
ovs-ofctl add-flow br-tun "table=0,priority=1,in_port=1 actions=resubmit(,1)"
ovs-ofctl add-flow br-tun "table=0,priority=1,in_port=5 actions=resubmit(,2)"
ovs-ofctl add-flow br-tun "table=0,priority=1,in_port=6 actions=resubmit(,2)"
ovs-ofctl add-flow br-tun "table=0,priority=0 actions=drop"

# Table 1 – traffic from br-int (unicast vs broadcast)
ovs-ofctl add-flow br-tun "table=1,priority=0,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,20)"
ovs-ofctl add-flow br-tun "table=1,priority=0,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,21)"

# Table 21 – broadcast to all VTEPs
ovs-ofctl add-flow br-tun "table=21,priority=0,actions=output:5,output:6"

# Table 2 – learning from VTEP ingress
ovs-ofctl add-flow br-tun "table=2,priority=0,actions=learn(table=20,hard_timeout=300,priority=1,NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:NXM_OF_IN_PORT[]),output:1"

# Table 20 – unicast forwarding (default to broadcast if unknown)
ovs-ofctl add-flow br-tun "table=20,priority=0,actions=resubmit(,21)"

After traffic exchange, the learning rule populates table 20 with entries mapping destination MACs to the appropriate VTEP port, eliminating unnecessary flooding.

# Example learned flow
cookie=0x0, duration=164.902s, table=20, n_packets=36, n_bytes=3360, hard_timeout=300, priority=1, dl_dst=ee:2c:09:42:0e:46 actions=load:0->NXM_NX_TUN_ID[],output:6

Conclusion

The article demonstrates that VXLAN provides a scalable overlay solution for cloud data centers, extending VLAN’s limited identifier space and enabling automated, software‑defined networking. Both the native Linux kernel implementation and the Open vSwitch implementation achieve the same goal, though OVS requires explicit flow‑table programming to handle BUM traffic without multicast support. For deeper protocol details, refer to RFC 7348.