Mastering Enterprise Firewalls: iptables vs nftables Rule Management

Overview

The Linux firewall core is the Netfilter framework, which intercepts packets at hook points such as PREROUTING, INPUT, FORWARD, OUTPUT and POSTROUTING. Historically, iptables has been the user‑space tool to manage Netfilter tables, but its linear‑list storage and full‑chain replacement cause performance degradation when rule counts reach thousands. nftables , introduced in kernel 3.13 and mature in 6.x, replaces the four independent tools (iptables, ip6tables, arptables, ebtables) with a unified nf_tables subsystem, offering native sets, maps, atomic updates and dramatically better scalability.

Architecture Comparison

Kernel interface : iptables uses per‑protocol x_tables modules; nftables uses a single nf_tables subsystem.

User‑space tools : iptables/ip6tables/arptables/ebtables vs. unified nft command.

Rule storage : linear chain (iptables) vs. hash‑based sets/maps (nftables).

Rule changes : full‑chain replacement (iptables) vs. atomic, transactional updates (nftables).

Set support : external ipset (iptables) vs. native sets/maps with timeout (nftables).

Syntax : command‑line flags (iptables) vs. declarative BPF‑style syntax (nftables).

Performance : O(N) matching (iptables) vs. O(1) set lookup (nftables) for large rule sets.

Compatibility layer : none for iptables; iptables-nft provides a compatible CLI for nftables.

Technical Features

Five Netfilter hook points (PREROUTING, INPUT, FORWARD, OUTPUT, POSTROUTING) cover the full packet lifecycle.

Atomic transactions prevent windows of inconsistency during rule updates.

Native sets and maps enable O(1) lookups for large black‑/whitelists.

Connection tracking (conntrack) provides stateful inspection of ESTABLISHED, RELATED and INVALID connections.

Docker inserts NAT rules; the DOCKER-USER chain is the recommended place for host‑level container filtering.

The inet family in nftables handles IPv4 and IPv6 simultaneously, simplifying dual‑stack configuration.

Applicable Scenarios

Server inbound protection (Web, database, SSH brute‑force mitigation).

Gateway NAT (SNAT/MASQUERADE, DNAT for public services).

DMZ isolation with three‑zone model (Internet ↔ DMZ ↔ Internal).

Container networking (Docker/Kubernetes) and preventing rule bypass.

DDoS baseline defenses (SYN flood rate‑limiting, port‑scan detection).

Environment Requirements

Linux kernel 6.x (full nftables feature set).

nftables 1.1.x.

iptables‑nft 1.8.9+ (compatibility layer).

Ubuntu 24.04, Debian 12 or RHEL 9 (default nftables on Debian 12 and RHEL 9).

conntrack‑tools ≥1.4.7.

Implementation

iptables baseline rule set

#!/bin/bash
set -euo pipefail
# Flush existing rules
iptables -F
iptables -X
iptables -Z
# Default policies
iptables -P INPUT DROP
iptables -P FORWARD DROP
iptables -P OUTPUT ACCEPT
# Loopback
iptables -A INPUT -i lo -j ACCEPT
# Established/related
iptables -A INPUT -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT
# Drop invalid
iptables -A INPUT -m conntrack --ctstate INVALID -j DROP
# SSH rate‑limit (6 connections per minute per IP)
iptables -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW \
    -m recent --set --name SSH
iptables -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW \
    -m recent --update --seconds 60 --hitcount 6 --name SSH -j DROP
iptables -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW -j ACCEPT
# Web ports
iptables -A INPUT -p tcp -m multiport --dports 80,443 -m conntrack --ctstate NEW -j ACCEPT
# ICMP rate‑limit (5 packets/sec)
iptables -A INPUT -p icmp --icmp-type echo-request -m limit --limit 5/s --limit-burst 10 -j ACCEPT
iptables -A INPUT -p icmp --icmp-type echo-request -j DROP
# Log dropped packets (rate‑limited)
iptables -A INPUT -m limit --limit 10/min -j LOG --log-prefix "[FW-DROP] " --log-level 4

NAT configuration

# SNAT (static public IP)
iptables -t nat -A POSTROUTING -s 10.0.0.0/24 -o eth0 -j SNAT --to-source 203.0.113.10
# MASQUERADE (dynamic public IP)
iptables -t nat -A POSTROUTING -s 10.0.0.0/24 -o eth0 -j MASQUERADE
# DNAT (public 80 → internal web server)
iptables -t nat -A PREROUTING -i eth0 -p tcp --dport 80 -j DNAT --to-destination 10.0.0.100:8080
# Forwarding for DNAT
iptables -A FORWARD -i eth0 -o eth1 -p tcp --dport 8080 -d 10.0.0.100 -m conntrack --ctstate NEW -j ACCEPT
iptables -A FORWARD -m conntrack --ctstate ESTABLISHED,RELATED -j ACCEPT

nftables rule set

#!/usr/sbin/nft -f
# Flush existing rules (atomic operation)
flush ruleset
# Variable definitions
define WAN_IF = eth0
define DMZ_IF = eth1
define LAN_IF = eth2
define DMZ_NET = 172.16.0.0/24
define LAN_NET = 10.0.0.0/24
# Filter table
table inet filter {
    # Sets
    set blacklist { type ipv4_addr; flags timeout; timeout 10m; }
    set whitelist { type ipv4_addr; elements = { 10.0.0.1, 10.0.0.2 }; }
    set dmz_services { type inet_service; elements = { 80, 443 }; }
    set dmz_to_lan_ports { type inet_service; elements = { 3306, 6379, 5432 }; }
    # SSH limit chain
    chain ssh_limit {
        ct state new meter ssh_meter { ip saddr limit rate 6/minute burst 4 packets } accept
        add @blacklist { ip saddr }
        log prefix "[SSH-BRUTE] " level warn
        drop
    }
    # SYN flood protection chain
    chain syn_protect {
        tcp flags syn limit rate 500/second burst 200 packets accept
        log prefix "[SYN-FLOOD] " level warn
        drop
    }
    # Input chain
    chain input {
        type filter hook input priority 0; policy drop;
        iif "lo" accept
        ip saddr @blacklist counter drop
        ip saddr @whitelist accept
        ct state established,related accept
        ct state invalid counter drop
        iifname $LAN_IF tcp dport 22 ct state new jump ssh_limit
        iifname $WAN_IF tcp dport @dmz_services ct state new accept
        ip protocol icmp icmp type echo-request limit rate 5/second accept
        ip6 nexthdr icmpv6 accept
        limit rate 10/minute log prefix "[NFT-INPUT-DROP] " level warn
    }
    # Forward chain (three‑zone traffic control)
    chain forward {
        type filter hook forward priority 0; policy drop;
        ct state established,related accept
        ct state invalid drop
        iifname $WAN_IF oifname $DMZ_IF tcp dport @dmz_services ct state new accept
        iifname $DMZ_IF oifname $LAN_IF ip daddr $LAN_NET tcp dport @dmz_to_lan_ports ct state new accept
        iifname $LAN_IF oifname $WAN_IF accept
        iifname $LAN_IF oifname $DMZ_IF accept
        limit rate 10/minute log prefix "[NFT-FWD-DROP] " level warn
    }
    chain output { type filter hook output priority 0; policy accept; }
}
# NAT table
table inet nat {
    chain prerouting {
        type nat hook prerouting priority -100; policy accept;
        iifname $WAN_IF tcp dport 80 dnat to 172.16.0.10:80
        iifname $WAN_IF tcp dport 443 dnat to 172.16.0.10:443
    }
    chain postrouting {
        type nat hook postrouting priority 100; policy accept;
        oifname $WAN_IF ip saddr $LAN_NET masquerade
        oifname $WAN_IF ip saddr $DMZ_NET masquerade
    }
}

Migration using iptables‑translate

# Translate a single rule
iptables-translate -A INPUT -p tcp --dport 22 -m conntrack --ctstate NEW -j ACCEPT
# Translate an entire rule set
iptables-save | iptables-restore-translate > /etc/nftables-migrated.conf
# Note: modules such as recent have no direct nftables equivalent and must be rewritten with meters or sets.

Enterprise DMZ model

┌─────────────┐
Internet ──────►│ eth0 (WAN)   │
               │ Firewall    │
               └─────────────┘
      ▲          │ eth1 (DMZ) │   (Web, reverse proxy)
      │          ▼
      │      ┌─────────────┐
      └──────►│ eth2 (LAN) │   (DB, app servers)
               └─────────────┘
Traffic rules:
- Internet → DMZ: only ports 80/443.
- DMZ → LAN: only selected backend ports (e.g., 3306, 6379).
- LAN → Internet: allowed via SNAT.
- Internet → LAN: denied.

Best Practices and Pitfalls

Rule ordering : place high‑frequency matches (e.g., ESTABLISHED,RELATED) at the top of a chain, then specific allow rules, and finally the default DROP.

Set/Map optimization : replace dozens of individual IP or port rules with a single set or map to reduce linear matching overhead.

Conntrack tuning : size the conntrack table as concurrent_connections × 1.5, adjust timeouts (e.g., nf_conntrack_tcp_timeout_established=86400) and hash bucket size to avoid table‑full drops.

Container environments : Docker manipulates the nat table directly; use the DOCKER-USER chain for host‑level filtering or disable Docker’s iptables management via /etc/docker/daemon.json when full control is required.

IPv6 dual‑stack : the inet family handles IPv4/IPv6 together, but IPv6 requires explicit allowance for NDP and DHCPv6 traffic.

Safety measures : when editing rules remotely, schedule a fallback at job that restores a known‑good configuration after a few minutes. Test new rules in a staging environment before production rollout.

Monitoring and Observability

Conntrack usage ratio (< 60% normal, > 80% alarm).

Conntrack drop counter (should stay at 0).

Rule match rate (spikes may indicate attacks).

Per‑rule hit percentages (single rule > 90% suggests consolidation).

Network interface rx_dropped (non‑zero indicates kernel‑level packet loss).

#!/bin/bash
set -euo pipefail
TEXTFILE_DIR="/var/lib/node_exporter/textfile_collector"
METRICS_FILE="$TEXTFILE_DIR/firewall.prom"
mkdir -p "$TEXTFILE_DIR"
{
    echo "# HELP fw_conntrack_usage_ratio conntrack table usage ratio"
    echo "# TYPE fw_conntrack_usage_ratio gauge"
    COUNT=$(cat /proc/sys/net/netfilter/nf_conntrack_count)
    MAX=$(cat /proc/sys/net/netfilter/nf_conntrack_max)
    RATIO=$(awk "BEGIN {printf \"%.4f\", $COUNT/$MAX}")
    echo "fw_conntrack_usage_ratio $RATIO"
    echo "# HELP fw_conntrack_drops_total conntrack dropped connections"
    echo "# TYPE fw_conntrack_drops_total counter"
    DROPS=$(conntrack -S 2>/dev/null | awk -F= '{for(i=1;i<=NF;i++) if($i~/drop/) print $(i+1)}' | awk '{sum+=$1} END {print sum+0}')
    echo "fw_conntrack_drops_total $DROPS"
    echo "# HELP fw_nft_rules_total total nftables rules"
    echo "# TYPE fw_nft_rules_total gauge"
    RULES=$(nft list ruleset 2>/dev/null | grep -c "rule" || echo 0)
    echo "fw_nft_rules_total $RULES"
} > "$METRICS_FILE.tmp"
mv "$METRICS_FILE.tmp" "$METRICS_FILE"

groups:
- name: firewall
  rules:
  - alert: ConntrackTableNearlyFull
    expr: fw_conntrack_usage_ratio > 0.8
    for: 5m
    labels:
      severity: warning
    annotations:
      summary: "conntrack table usage exceeds 80%"
      description: "{{ $labels.instance }} conntrack usage is {{ $value | humanizePercentage }}"
  - alert: ConntrackDropsIncreasing
    expr: rate(fw_conntrack_drops_total[5m]) > 0
    for: 2m
    labels:
      severity: critical
    annotations:
      summary: "conntrack drops detected"
      description: "{{ $labels.instance }} is dropping new connections at {{ $value }}/s"

Backup and Restore

#!/bin/bash
set -euo pipefail
BACKUP_DIR="/opt/firewall-backup"
DATE=$(date +%Y%m%d-%H%M%S)
mkdir -p "$BACKUP_DIR"
# nftables backup
nft list ruleset > "$BACKUP_DIR/nftables-$DATE.conf"
# iptables IPv4 backup
iptables-save > "$BACKUP_DIR/iptables-v4-$DATE.rules"
# iptables IPv6 backup
ip6tables-save > "$BACKUP_DIR/iptables-v6-$DATE.rules"
# Cleanup older than 30 days
find "$BACKUP_DIR" -type f -mtime +30 -delete

# Stop firewall services
systemctl stop nftables || true
# Flush current rules
nft flush ruleset
iptables -F && iptables -t nat -F && iptables -t mangle -F
# Load saved configuration
nft -f /opt/firewall-backup/nftables-20260225-020000.conf
iptables-restore < /opt/firewall-backup/iptables-v4-20260225-020000.rules
ip6tables-restore < /opt/firewall-backup/iptables-v6-20260225-020000.rules
# Apply saved sysctl parameters
sysctl -p /opt/firewall-backup/sysctl-conntrack-20260225-020000.conf
# Verify
nft list ruleset | head -20
iptables -L -n --line-numbers | head -20
# Restart services
systemctl start nftables
systemctl status nftables

Conclusion

nftables is the future‑proof choice for enterprise firewalls, offering atomic updates, native sets/maps, and unified IPv4/IPv6 handling. Proper conntrack sizing, rule ordering, and set‑based optimizations are essential for performance and reliability. Integrating monitoring, automated backups, and safe‑change workflows ensures continuous protection without accidental lock‑outs.