Master Linux Traffic Control: Practical TC Commands and QoS Strategies

Traffic Control

Providing QoS for real‑time services over IP networks is challenging because IP is best‑effort and TCP’s retransmission adds latency.

The simplest solution—adding bandwidth—is costly, so Traffic Control (TC) techniques are used to allocate resources fairly.

Basic Implementation Principle

TC works by placing queues (Qdisc) at the Ingress and Egress points of network devices (NICs, switches, routers) to order and rate‑limit packets. Policies can be applied per‑queue.

Two TC scenarios exist:

L2 Frame Traffic Control

L3 Packet Traffic Control (discussed here)

Three Layers of Traffic Processing

Classifier : Identify packets or flows (e.g., by header or 5‑tuple) and enqueue them into appropriate queues.

Marker : Modify packet headers (e.g., ToS) to mark traffic for later handling.

Policier : Apply policies such as rate limits to the selected queues.

Implementation Model

The diagram below shows the typical flow from Ingress to Egress through classification, marking, and policy enforcement.

Ingress Classification & Marking

Ingress/Egress Traffic Regulation : Drop illegal traffic and enforce thresholds.

Egress Congestion Avoidance : Drop packets or buffer when queues exceed limits.

Bandwidth Management : Limit queue rate.

Interface Rate Limiting

Egress Traffic Shaping : Buffer excess traffic and release smoothly.

Common Queue Types

FIFO Queue

Simple first‑in‑first‑out queue without classification.

PFIFO_FAST Queue

FIFO with basic classification support.

SFQ Queue

Stochastic Fair Queue distributes bandwidth fairly among flows.

Token Bucket Queue

Implements the token‑bucket algorithm: a bucket holds tokens generated at a fixed rate; a packet can dequeue only after acquiring a token, allowing burst control.

A fixed‑capacity bucket fills with tokens at a defined rate.

Each packet consumes one token before leaving the queue.

Token generation rate controls the dequeue rate.

Unused tokens accumulate to handle bursts; burst duration = bucket size / (send rate – token rate).

When tokens are exhausted, policing discards packets, while shaping buffers them.

Linux kernel provides two token‑bucket queues:

TBF (Token Bucket Filter)

HTB (Hierarchical Token Bucket)

Kernel Traffic Control

TC was introduced in Linux 2.4 as a kernel‑level traffic controller, using a Qdisc‑Class‑Filter tree to apply hierarchical control.

Later a Classifier‑Action subsystem allowed custom classifiers.

TC sits at the L2 boundary of a network interface, processing packets on both ingress and egress.

Ingress packets pass through TC before entering the L3 stack.

Egress packets are shaped by TC after L3 routing decisions.

Qdisc (Queue Discipline)

Qdisc defines queue behavior and can be classless (e.g., PFIFO_FAST, SFQ, TBF) or classful (e.g., HTB) which requires associated Classes.

Class

A Class groups configuration parameters and can form a tree of root, inner, and leaf classes, enabling unlimited hierarchical extensions.

Filter

Filters perform actual classification, linking to Qdiscs or Classes, and may include policer actions for over‑limit traffic.

tc CLI Examples

General Configuration Flow

1. Create Queue

$ tc qdisc add dev eth0 root handle 1: htb default 11

2. Create Classes

$ tc class add dev eth0 parent 1: classid 1:11 htb rate 40mbit ceil 40mbit
$ tc class add dev eth0 parent 1: classid 1:12 htb rate 10mbit ceil 10mbit

3. Set Filters

$ tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 match ip dport 80 0xffff flowid 1:11
$ tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 match ip dport 25 0xffff flowid 1:12

HTB Example

# add qdisc
$ tc qdisc add dev eth0 root handle 1: htb default 2 r2q 100

# add default class
$ tc class add dev eth0 parent 1:0 classid 1:1 htb rate 1000mbit ceil 1000mbit
$ tc class add dev eth0 parent 1:1 classid 1:2 htb prio 5 rate 1000mbit ceil 1000mbit
$ tc qdisc add dev eth0 parent 1:2 handle 2: pfifo limit 500

# add default filter
$ tc filter add dev eth0 parent 1:0 prio 5 protocol ip u32
$ tc filter add dev eth0 parent 1:0 prio 5 handle 3: protocol ip u32 divisor 256
$ tc filter add dev eth0 parent 1:0 prio 5 protocol ip u32 ht 800:: match ip src 192.168.0.0/16 hashkey mask 0x000000ff at 12 link 3:

# add egress rules for 192.168.0.9
$ tc class add dev eth0 parent 1:1 classid 1:9 htb prio 5 rate 3mbit ceil 3mbit
$ tc qdisc add dev eth0 parent 1:9 handle 9: pfifo limit 500
$ tc filter add dev eth0 parent 1: protocol ip prio 5 u32 ht 3:9: match ip src "192.168.0.9" flowid 1:9

Egress Bandwidth Limiting

$ tc qdisc del dev eth0 root
$ tc qdisc add dev eth0 root handle 1: htb

$ tc class add dev eth0 parent 1: classid 1:1 htb rate 20mbit ceil 20mbit
$ tc class add dev eth0 parent 1:1 classid 1:10 htb rate 10mbit ceil 10mbit
$ tc qdisc add dev eth0 parent 1:10 sfq perturb 10

# let 172.20.6.0/24 use default queue (no limit)
$ tc filter add dev eth0 protocol ip parent 1: prio 2 u32 match ip dst 172.20.6.0/24 flowid 1:1

# default all traffic to limited queue
$ tc filter add dev eth0 protocol ip parent 1: prio 50 u32 match ip dst 0.0.0.0/0 flowid 1:10

Ingress Bandwidth Limiting (using ifb)

# init ifb
$ modprobe ifb numifbs=1
$ ip link set ifb0 up

# redirect ingress to ifb0
$ tc qdisc add dev eth0 ingress handle ffff:
$ tc filter add dev eth0 parent ffff: protocol ip prio 0 u32 match u32 0 0 flowid ffff: action mirred egress redirect dev ifb0

# add qdisc on ifb0
$ tc qdisc add dev ifb0 root handle 1: htb default 2 r2q 100

# add default class
$ tc class add dev ifb0 parent 1:0 classid 1:1 htb rate 1000mbit ceil 1000mbit
$ tc class add dev ifb0 parent 1:1 classid 1:2 htb prio 5 rate 1000mbit ceil 1000mbit
$ tc qdisc add dev ifb0 parent 1:2 handle 2: pfifo limit 500

# add ingress rules for 192.168.0.9
$ tc class add dev ifb0 parent 1:1 classid 1:9 htb prio 5 rate 3mbit ceil 3mbit
$ tc qdisc add dev ifb0 parent 1:9 handle 9: pfifo limit 500
$ tc filter add dev ifb0 parent 1: protocol ip prio 5 u32 ht 4:9: match ip dst "192.168.0.9" flowid 1:9

Rate Limiting by Source IP

$ tc qdisc add dev ifb0 root handle 1: htb default 20

$ tc class add dev ifb0 parent 1: classid 1:1 htb rate 10000mbit
$ tc class add dev ifb0 parent 1:1 classid 1:10 htb rate 2000mbit
$ tc class add dev ifb0 parent 1:1 classid 1:20 htb rate 1000mbit
$ tc class add dev ifb0 parent 1:1 classid 1:30 htb rate 500mbit

$ tc filter add dev ifb0 protocol ip parent 1:0 prio 1 u32 match ip src 129.9.123.85 flowid 1:10
$ tc filter add dev ifb0 protocol ip parent 1:0 prio 1 u32 match ip src 129.9.123.89 flowid 1:20
$ tc filter add dev ifb0 protocol ip parent 1:0 prio 1 u32 match ip src 129.9.123.88 flowid 1:20

IP QoS Service Model Evolution

Best‑Effort

All traffic is sent without guarantees; suitable for non‑real‑time applications.

IntServ

Requires signaling (RSVP) to reserve resources per flow, providing strict end‑to‑end guarantees but with high complexity and low resource utilization.

DiffServ

Classifies traffic at network edges using DSCP values, allowing scalable per‑class treatment without per‑flow state. Core routers are stateless, applying Per‑Hop Behavior based on DSCP.

DSCP Field

DSCP (6 bits) replaces the IPv4 ToS field and IPv6 Traffic Class for QoS marking. Common classes include EF (Expedited Forwarding), AF (Assured Forwarding), and Best‑Effort.

L2 QoS Service Model

L2 uses the PCP field (0‑7) in Ethernet frames to indicate priority; mapping between DSCP and PCP is shown below.

QoS in Enterprise Networks

QoS functions are distributed across the network: access switches perform traffic identification and marking, while aggregation and core devices enforce shaping, policing, and scheduling based on the marked traffic.