Implementing VM Traffic Mirroring for Security Auditing in Cloud VPC

1. Background

Traffic mirroring provides traffic capture service, allowing traffic within a specified range to be filtered by various conditions and duplicated to a VM in a private VPC. It is suitable for security auditing, risk monitoring, fault troubleshooting, and business analysis.

1.1 Workflow

The key components are the capture source and the receiver. The source is an elastic NIC in the VPC, filtered by network, scope, type, and traffic filters. The receiver is the IP where captured traffic is forwarded.

1.2 Use Cases

Security Auditing

Analyzing abnormal network packets to locate fault causes.

Intrusion Detection

Copying traffic to cloud servers for real‑time analysis to protect confidentiality, integrity, and availability.

Business Analysis

Mirroring business traffic to visualize internal traffic models.

2. Solution Design

2.1 Basic Concepts

Traffic Capture

The source is a VM’s network card.

Capture Types

Supports “all traffic”, “outbound”, and “inbound”.

Traffic Filtering

No filter captures all configured traffic. Five‑tuple filter matches protocol, source/destination subnets, and ports.

Traffic Reception

Typically the capture and receiver IP are in the same VPC, but cross‑VPC (same region) is allowed. The receiver can add an elastic NIC to handle mirrored traffic.

2.2 Solution Options

Traffic mirroring passes through three stages: capture, filter, and reception.

Capture: Use OVS’s built‑in mirror function on the capture node, adding mirror configuration to the port.

Reception: Identify mirrored traffic on the receiver node via the source port’s IP or MAC address; using MAC is more stable. The receiver VM must enable promiscuous mode. <code>ifconfig eth0 promisc # enable promiscuous mode ifconfig eth0 -promisc # disable promiscuous mode</code>

Filtering: Initially use no filter, later add five‑tuple filtering. Two approaches: Distributed: Apply flow‑table filtering on the capture node. Centralized: Use a gateway between capture and reception for filtering.

Overall, the centralized filtering approach is chosen.

2.3 Traffic Identification

Tagging: Mark mirrored traffic with a ToS label via flow tables on the capture node.

UDP Port: Use a special UDP port (e.g., 4790) to distinguish mirrored traffic. Tagging UDP 4790 Pros: Small development effort Pros: Gateway can differentiate by port without deep packet inspection Cons: Requires gateway to parse packets Cons: Larger development effort Considering pros and cons, UDP 4800 is selected to identify mirrored traffic.

2.4 Usage Limits

A capture VM port can belong to only one mirroring instance; a receiver VM port can belong to multiple instances.

2.5 Process Design

2.6 Solution Verification

Before gateway functionality is completed, direct forwarding from the compute node can be verified.

Capture Side

Create a VXLAN port with destination port 4790 and OVS mirror rules. Use “add‑bridge br‑int” to add the mirror.

<code>ovs-vsctl add-port br-int vxlan11 -- set interface vxlan11 type=vxlan \
options:local_ip=10.120.64.238 options:remote_ip=10.220.164.238 options:key=100 \
options:dst_port=4790

ovs-vsctl -- --id=@qvo6fc32225-a8 get port qvo6fc32225-a8 \
-- --id=@vxlan11 get port vxlan11 \
-- --id=@m create mirror name=m0 select-src-port=@qvo6fc32225-a8 select-dst-port=@qvo6fc32225-a8 output-port=@vxlan11 \
-- add bridge br-int mirrors=@m</code>

Receiver Side

Create a VXLAN port with destination port 4790, add flow‑table rules to forward traffic to table 63, then match the source VM’s MAC address to forward to the target NIC.

<code>ovs-vsctl add-port br-int vxlan11 -- set interface vxlan11 type=vxlan \
options:local_ip=10.120.164.238 options:remote_ip=10.220.64.238 options:key=100 \
options:dst_port=4790

ovs-ofctl add-flow br-int -Oopenflow13 "table=0,priority=200,in_port=vxlan11 actions=resubmit(,63)"
ovs-ofctl add-flow br-int -Oopenflow13 "table=63,priority=1,dl_src=fa:16:3e:71:ab:62 actions=output:tap4bd35a8c-a5"
ovs-ofctl add-flow br-int -Oopenflow13 "table=63,priority=1,dl_dst=fa:16:3e:71:ab:62 actions=output:tap4bd35a8c-a5"
ovs-ofctl add-flow br-int -Oopenflow13 "table=63,priority=0 actions=drop"</code>

Ping the capture VM from the receiver and capture packets to confirm mirrored traffic is received.

3. Summary

VM traffic mirroring duplicates and forwards traffic, enabling better monitoring and analysis, enhancing network security, and simplifying intrusion risk mitigation. It supports cross‑VPC mirroring within the same region, extending use cases beyond native cloud provider offerings.