How VXLAN Extends VLAN for Scalable Cloud Networks – A Hands‑On Guide

VXLAN Overview

VXLAN (Virtual eXtensible LAN) extends the VLAN identifier space from 12 bits (4,096 IDs) to a 24‑bit VXLAN Network Identifier (VNI), allowing up to 16,777,216 logical networks. It encapsulates Ethernet frames inside UDP packets, building an overlay L2 network on top of an IP fabric. The encapsulation adds a 50‑byte overhead, so the MTU of the underlying network (or the VM MTU) must be increased to avoid fragmentation.

Key Components

VXLAN traffic is processed by VTEPs (VXLAN Tunnel End Points), which can be implemented in hardware or software. Broadcast, Unknown‑unicast, and Multicast (BUM) traffic is normally flooded via IP multicast; Open vSwitch (OvS) implements flooding with multiple unicast streams.

Linux Kernel Implementation

Three hosts (192.168.33.12‑14) each run a Linux bridge br0 with IP addresses 10.1.1.2‑4. A VXLAN tunnel interface vxlan0 is created on each host using VNI 1 and multicast group 239.1.1.1. The configuration steps are:

brctl addbr br0

ip addr add 10.1.1.2/24 dev br0

ip link set up br0

ip link add vxlan0 type vxlan id 1 group 239.1.1.1 dev eth1 dstport 4789

brctl addif br0 vxlan0

After configuring all three hosts, the bridge FDB shows a multicast entry. Ping tests confirm that the overlay network becomes reachable once the VTEPs learn each other's MAC‑VTEP mappings. Packet captures show an ARP request flooded to the multicast group, the reply learned by the remote VTEP, and subsequent ICMP traffic using the learned unicast path.

Open vSwitch (OvS) Implementation

OvS does not support IP multicast, so each pair of hosts is connected via a unicast VXLAN tunnel. The topology uses two bridges per host: br-int for logical network ports and br-tun for VXLAN interfaces, linked by a PATCH port.

Configuration steps:

ovs-vsctl add-br br-int
ovs-vsctl add-br br-tun

ovs-vsctl add-port br-int patch-int -- set interface patch-int type=patch options:peer=patch-tun
ovs-vsctl add-port br-tun patch-tun -- set interface patch-tun type=patch options:peer=patch-int

ovs-vsctl add-port br-tun vxlan0 -- set interface vxlan0 type=vxlan options:remote_ip=192.168.33.13
ovs-vsctl add-port br-tun vxlan1 -- set interface vxlan1 type=vxlan options:remote_ip=192.168.33.14

OpenFlow tables are programmed to separate traffic sources, learn MAC addresses from VTEP packets, and prevent loops.

# Clear existing flows
ovs-ofctl del-flows br-tun

# Table 0 – dispatch based on ingress port
ovs-ofctl add-flow br-tun "table=0,priority=1,in_port=1 actions=resubmit(,1)"
ovs-ofctl add-flow br-tun "table=0,priority=1,in_port=5 actions=resubmit(,2)"
ovs-ofctl add-flow br-tun "table=0,priority=1,in_port=6 actions=resubmit(,2)"
ovs-ofctl add-flow br-tun "table=0,priority=0 actions=drop"

# Table 1 – traffic from br-int (logical ports)
ovs-ofctl add-flow br-tun "table=1,priority=0,dl_dst=00:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,20)"
ovs-ofctl add-flow br-tun "table=1,priority=0,dl_dst=01:00:00:00:00:00/01:00:00:00:00:00 actions=resubmit(,21)"

# Table 2 – learning from VTEP packets
ovs-ofctl add-flow br-tun "table=2,priority=0,actions=learn(table=20,hard_timeout=300,priority=1,NXM_OF_ETH_DST[]=NXM_OF_ETH_SRC[],load:NXM_NX_TUN_ID[]->NXM_NX_TUN_ID[],output:NXM_OF_IN_PORT[]),output:1"

# Table 20 – unicast forwarding (default drop)
ovs-ofctl add-flow br-tun "table=20,priority=0 actions=resubmit(,21)"

# Table 21 – broadcast to all VTEPs
ovs-ofctl add-flow br-tun "table=21,priority=0 actions=output:5,output:6"

After installing the flows, ping tests between the three hosts succeed, and the flow table shows learned entries with priority 1, confirming correct operation of the VXLAN overlay.

Reference

VXLAN protocol details are defined in RFC 7348: https://tools.ietf.org/html/rfc7348