Understanding iptables and Netfilter: Hook Points, Tables, and Rule Processing

iptables and Netfilter Overview

iptables is the most common Linux firewall tool. It works by registering hook functions at Netfilter hook points inside the kernel protocol stack, allowing packet filtering and manipulation.

Netfilter Hook Points

NF_IP_PRE_ROUTING

: Triggered immediately after a packet is received, before any routing decision. NF_IP_LOCAL_IN: Triggered for packets whose destination is the local host after routing. NF_IP_FORWARD: Triggered for packets destined for another host after routing. NF_IP_LOCAL_OUT: Triggered for locally generated packets before they enter the protocol stack. NF_IP_POST_ROUTING: Triggered for locally generated or forwarded packets after routing.

Each hook point corresponds to a built‑in chain in iptables (e.g., PREROUTING ↔ NF_IP_PRE_ROUTING).

Example: ip_rcv Function

int ip_rcv(struct sk_buff *skb, struct net_device *dev,
            struct packet_type *pt, struct net_device *orig_dev)
{
    struct net *net = dev_net(dev);
    skb = ip_rcv_core(skb, net);
    if (skb == NULL)
        return NET_RX_DROP;
    return NF_HOOK(NFPROTO_IPV4, NF_INET_PRE_ROUTING,
                   net, NULL, skb, dev, NULL,
                   ip_rcv_finish);
}

The function shows how a packet is handed to NF_HOOK, which then walks the registered hook list.

Hook Execution Flow

Lookup Hook List : Netfilter selects the list of hook functions registered for the given protocol and hook number.

Invoke Hook Functions : It iterates over the list, calling each hook with the packet ( skb) and a nf_hook_state structure.

Process Verdict : Each hook returns a verdict ( NF_ACCEPT, NF_DROP, NF_QUEUE, etc.). The loop stops on NF_DROP or an error; otherwise it continues.

iptables Tables and Chains

iptables organizes rules into tables (filter, nat, mangle, raw, security). Within each table, rules are placed in chains . Built‑in chains are bound to Netfilter hooks; user‑defined chains can only be reached by a jump from another chain.

filter : Decides whether a packet is allowed.

nat : Performs source or destination address translation.

mangle : Modifies packet headers (e.g., TTL) and can set internal marks.

raw : Allows packets to bypass connection tracking.

security : Adds SELinux labels that affect later processing.

Chains are processed in a priority order defined by enum nf_ip_hook_priorities, where a lower numeric value means higher priority.

NAT Example

To masquerade outgoing traffic on interface eth1, the following rule is sufficient:

iptables -t nat -A POSTROUTING -o eth1 -j MASQUERADE

Hook Registration Structures

The kernel uses struct nf_hook_ops to describe a hook function:

struct nf_hook_ops {
    nf_hookfn *hook;          // pointer to the hook function
    struct net_device *dev;  // optional device restriction
    void *priv;              // private data for the hook
    u8 pf;                  // protocol family (e.g., NFPROTO_IPV4)
    enum nf_hook_ops_type hook_ops_type:8;
    unsigned int hooknum;    // hook position (e.g., NF_INET_PRE_ROUTING)
    int priority;            // lower value = higher priority
};

Modules register their hooks during initialization via nf_register_net_hooks (or the older nf_register_net_hook), which ultimately calls nf_hook_entries_grow to insert the new hook into the ordered list based on priority.

From User Space to Kernel

When a user runs an iptables command, the following steps occur:

User‑Space Invocation : The command line tool parses the arguments and builds a rule structure.

libiptc Call : Functions such as iptc_append_entry in libiptc are invoked to add the rule to the appropriate chain.

Socket Communication : libiptc uses a raw socket (via setsockopt) to send the rule to the kernel.

Kernel Processing : Netfilter receives the request, updates internal tables (e.g., ip_tables), and registers the rule via xt_replace_table.

Result Propagation : The kernel returns success or an error code, which the user‑space tool reports to the user.

All of the above logic is implemented in the Linux source tree under net/netfilter (kernel version 6.2.0 in the article).

References

DigitalOcean: "A Deep Dive into Iptables and Netfilter Architecture"

arthurchiao.art translation of the same article

Karl Rupp: "NAT with Linux and iptables"

Tencent Cloud Developer Community tutorial on iptables internals