Understanding Netfilter and iptables: Architecture, Hook Mechanism, and Configuration in Linux

Imagine the Linux networking world as a bustling city where data packets are vehicles; Netfilter acts as the traffic police and control center, deciding which packets may pass and which must be intercepted.

1. Netfilter Overview

Introduced in Linux 2.4, Netfilter is a generic, abstract framework that provides a complete set of hook functions, enabling packet filtering, NAT, and connection tracking. It offers fine‑grained control over network packets for system administrators.

Netfilter inserts five hook points into the kernel: NF_IP_PRE_ROUTING , NF_IP_LOCAL_IN , NF_IP_FORWARD , NF_IP_LOCAL_OUT , and NF_IP_POST_ROUTING . Callback functions registered at these points can return values such as NF_ACCEPT , NF_DROP , NF_STOLEN , NF_QUEUE , or NF_REPEAT .

Supported protocol stacks include IPv4, IPv6, and DECnet. Rules (matching fields + action) are organized into chains, and chains are grouped into tables.

2. Netfilter Working Principle

2.1 What is Netfilter

Netfilter is a packet‑processing module in the Linux kernel that provides filtering, forwarding, and NAT. iptables is a user‑space utility that manipulates Netfilter rules.

Three main concepts:

Rule – description of how to handle a specific packet.

Chain – a collection of rules.

Table – a collection of chains with the same purpose.

The five hook points are:

PRE_ROUTING – before routing decision.

LOCAL_IN – after routing, before transport layer for packets destined to the local host.

LOCAL_OUT – after transport layer for locally generated packets, before routing.

FORWARD – after routing for packets being forwarded.

POST_ROUTING – after routing for outgoing packets.

2.2 What is iptables

iptables builds on Netfilter, adding rules to the various hook points. It defines four tables:

filter – default table for packet filtering (chains: INPUT, OUTPUT, FORWARD).

nat – for network address translation (chains: PREROUTING, POSTROUTING, OUTPUT).

mangle – for packet header modification (available on all chains).

raw – for bypassing connection‑tracking (chains: PREROUTING, OUTPUT).

When a packet traverses the kernel, iptables rules are evaluated in order; a failure causes the packet to be dropped.

Table Implementation

The core data structure for a table is struct ipt_table :

struct ipt_table {
  struct list_head list;
  char name[IPT_TABLE_MAXNAMELEN];
  struct ipt_replace *table;
  unsigned int valid_hooks;
  rwlock_t lock;
  struct ipt_table_info *private;
  struct module *me;
};

Key fields:

valid_hooks – bitmap of hooks the table listens to.

struct module *me – prevents removal of a module still referenced by a rule.

Rules are stored as struct ipt_entry linked lists. Each entry contains an ipt_ip description, match offset, target offset, counters, and a flexible elems area for matches and targets.

struct ipt_entry {
    struct ipt_ip ip;
    unsigned int nfcache;
    u_int16_t target_offset;
    u_int16_t next_offset;
    unsigned int comefrom;
    struct ipt_counters counters;
    unsigned char elems[0];
};

Matches are described by struct ipt_entry_match and targets by struct ipt_entry_target . Helper macros such as ipt_get_target() , IPT_MATCH_ITERATE() , and IPT_ENTRY_ITERATE() simplify traversal.

3. Netfilter/iptables‑IPv4 Overall Architecture

3.1 Hook Mechanism

Netfilter provides a generic, protocol‑agnostic hook framework. Modules can register callbacks at any hook point, allowing inspection, modification, or dropping of packets.

3.2 iptables Core Modules

Three basic tables (filter, nat, mangle) implement packet filtering, connection tracking, NAT, and packet modification.

3.3 Netfilter Architecture

Netfilter’s four‑table, five‑chain design enables layered processing: raw → mangle → nat → filter → routing. The stateful firewall tracks connections (NEW, ESTABLISHED, RELATED, INVALID) to improve performance.

4. Basic Netfilter Configuration

4.1 Controlling SSH (port 22)

iptables -A INPUT -p tcp --dport 22 -j DROP

To allow a specific IP:

iptables -I INPUT -s 10.0.0.1 -p tcp --dport 22 -j ACCEPT

4.2 Blocking a subnet

iptables -A INPUT -s 172.16.1.0/24 -d 172.16.1.188 -j DROP

Other examples include port range matching, multi‑port matching, and using -m u32 for advanced packet inspection.

5. Enterprise‑Grade Firewall Practices

Backing up firewall configurations, using firewalld for dynamic rule management, and integrating with VPN or load‑balancing solutions are recommended for production environments.

6. Netfilter Application Cases

Netfilter is used for firewalling, load balancing (via DNAT), VPN integration, and custom kernel modules that drop or log specific traffic.

7. Advantages of Netfilter

Strong extensibility – modules like ebtables, arptables, nftables extend functionality.

Clear four‑table, five‑chain structure makes it easy to understand.

Rich configurability – filtering, NAT, mangle, raw.

Efficient hook‑based workflow.

High flexibility and control over packet handling.

8. Limitations of Netfilter

State table uses hash‑linked lists, which can become a performance bottleneck under heavy load.

Some targets (e.g., logging) are less efficient than dedicated options.

Lacks built‑in features such as SYN proxy.

Difficulty implementing fine‑grained rate limiting or per‑IP connection limits.