Passive OS Fingerprinting: Detect Humans vs Bots with SYN Header Analysis

Introduction

Human‑machine identification is a long‑standing topic in the seven‑layer security model. Quickly and minimally invasive differentiation between humans and bots gives an advantage in defense.

Traditional methods such as cookies, JavaScript challenges, and CAPTCHAs are increasingly bypassed by advanced crawlers that can execute JavaScript.

Why OS Fingerprinting?

With the rise of mobile apps, distinguishing between mobile devices, PCs, and servers becomes essential. Operating systems provide a concise way to separate Android, iOS, Windows, macOS, Linux, FreeBSD, etc.

Proposed Passive OS Identification

This article presents a passive method based on network packets to identify the operating system and thus aid human‑machine detection.

Implementation Principle

TL;DR

Capture the full SYN packet’s IP header in user space, extract features to build a fingerprint, and identify the OS.

Typical Internet Company Access Architecture

Client (App) → DNS → IDC → ECMP → L4LB → L7LB → Backend services

Obtaining SYN Header in User Space

Linux kernel 4.5 introduced TCP_SAVE_SYN and TCP_SAVED_SYN socket options to record SYN headers. User‑space programs can retrieve these headers via the options.

case TCP_SAVE_SYN:
    val = tp->save_syn;
    break;
case TCP_SAVED_SYN: {
    if (get_user(len, optlen))
        return -EFAULT;
    lock_sock(sk);
    if (tp->saved_syn) {
        if (len < tp->saved_syn[0]) {
            if (put_user(tp->saved_syn[0], optlen)) {
                release_sock(sk);
                return -EFAULT;
            }
            release_sock(sk);
            return -EINVAL;
        }
        len = tp->saved_syn[0];
        if (put_user(len, optlen)) {
            release_sock(sk);
            return -EFAULT;
        }
        if (copy_to_user(optval, tp->saved_syn + 1, len)) {
            release_sock(sk);
            return -EFAULT;
        }
        tcp_saved_syn_free(tp);
        release_sock(sk);
    } else {
        release_sock(sk);
        len = 0;
        if (put_user(len, optlen))
            return -EFAULT;
    }
    return 0;
}

Building Feature Fingerprint Library

Different OS network stacks emit SYN packets with distinct TTL, window size, and other fields. By collecting these values, a fingerprint database for normal clients and crawlers can be constructed.

Conclusion

OS fingerprinting can be combined with other device and request attributes to form multi‑dimensional challenges, but should not be the sole criterion for bot detection.