Building a Go TCP Scanner to Discover Unauthenticated ClickHouse Services

Part 1 – Finding public IPs with port 9000 open

The previous work produced a list of ~50 million live IPv4 addresses using ICMP ping. To restrict the search to ClickHouse’s default port 9000, the scanner was rewritten to perform a TCP SYN scan. The approach re‑uses the ICMP‑derived list, sends a raw TCP SYN packet to each target, waits for a SYN‑ACK, and immediately replies with a RST to avoid keeping the remote server busy.

Key optimisations:

Reuse the existing live‑host list (≈ 50 M entries).

Construct TCP packets manually using a raw socket ( net.ListenPacket("ip4:tcp", srcIP)).

Use the process PID as the TCP sequence number so the response can be matched.

Compute the TCP checksum locally because the test machine lacks checksum offload.

Send a RST after receiving a SYN‑ACK.

A TCPScanner struct encapsulates the logic:

package fishfinding

import (
    "net"
    "os"
    "time"
    "github.com/kataras/golog"
    "golang.org/x/net/bpf"
    "golang.org/x/net/ipv4"
)

type TCPScanner struct {
    src     net.IP
    srcPort int
    dstPort int
    input   chan string
    output  chan string
}

func NewTCPScanner(srcPort, dstPort int, input, output chan string) *TCPScanner {
    localIP := GetLocalIP()
    return &TCPScanner{
        src:     net.ParseIP(localIP).To4(),
        srcPort: srcPort,
        dstPort: dstPort,
        input:   input,
        output:  output,
    }
}

func (s *TCPScanner) Scan() {
    go s.recv()
    go s.send(s.input)
}

The send goroutine creates a raw socket, builds a TCP SYN header, calculates the checksum with tcpChecksum, and writes the packet to each destination IP:

func (s *TCPScanner) send(input chan string) error {
    defer func() {
        time.Sleep(5 * time.Second)
        close(s.output)
        golog.Infof("send goroutine exit")
    }()
    conn, err := net.ListenPacket("ip4:tcp", s.src.To4().String())
    if err != nil { golog.Fatal(err) }
    defer conn.Close()
    pconn := ipv4.NewPacketConn(conn)
    filter := createEmptyFilter()
    if assembled, err := bpf.Assemble(filter); err == nil {
        pconn.SetBPF(assembled)
    }
    seq := uint32(os.Getpid())
    for ip := range input {
        dstIP := net.ParseIP(ip)
        if dstIP == nil { golog.Errorf("failed to resolve IP address %s", ip); continue }
        tcpHeader := &TCPHeader{Source: uint16(s.srcPort), Destination: uint16(s.dstPort), SeqNum: seq, Flags: 0x002, Window: 65535}
        tcpHeader.Checksum = tcpChecksum(tcpHeader, s.src, dstIP)
        packet := tcpHeader.Marshal()
        if _, err = conn.WriteTo(packet, &net.IPAddr{IP: dstIP}); err != nil {
            golog.Errorf("failed to send TCP packet: %v", err)
        }
    }
    return nil
}

The recv goroutine also uses a raw socket ( net.ListenIP("ip4:tcp", &net.IPAddr{IP: s.src})), enables control messages to obtain source/destination IPs, parses incoming packets, and forwards any SYN‑ACK whose ACK number matches the sent sequence to the output channel:

func (s *TCPScanner) recv() error {
    conn, err := net.ListenIP("ip4:tcp", &net.IPAddr{IP: s.src})
    if err != nil { golog.Fatal(err) }
    defer conn.Close()
    pconn := ipv4.NewPacketConn(conn)
    pconn.SetControlMessage(ipv4.FlagSrc|ipv4.FlagDst|ipv4.FlagInterface, true)
    seq := uint32(os.Getpid()) + 1
    buf := make([]byte, 1024)
    for {
        n, peer, err := conn.ReadFrom(buf)
        if err != nil { golog.Errorf("failed to read: %v", err); continue }
        if n < tcpHeaderLength { continue }
        tcpHeader := ParseTCPHeader(buf[:n])
        if tcpHeader.Destination != uint16(s.srcPort) || tcpHeader.Source != uint16(s.dstPort) { continue }
        if tcpHeader.Flags == 0x012 && tcpHeader.AckNum == seq { // SYN+ACK
            s.output <- peer.String()
        }
    }
}

Running the scanner on a home 100 Mbit network with a decade‑old 4‑core Linux box produced 970 + IPs that responded to the TCP handshake on port 9000.

Part 2 – Checking for unauthenticated ClickHouse instances

A second component, ClickHouseChecker, consumes the candidate IP list, opens a ClickHouse connection on port 9000 using the official Go driver, sets a one‑second dial timeout, and calls Ping. No credentials are supplied; a successful ping indicates the service allows password‑less access.

type ClickHouseChecker struct {
    wg     *sync.WaitGroup
    port   int
    input  chan string
    output chan string
}

func NewClickHouseChecker(port int, input, output chan string, wg *sync.WaitGroup) *ClickHouseChecker {
    return &ClickHouseChecker{port: port, input: input, output: output, wg: wg}
}

func (c *ClickHouseChecker) Check() {
    parallel := runtime.NumCPU()
    for i := 0; i < parallel; i++ {
        c.wg.Add(1)
        go c.check()
    }
}

func (c *ClickHouseChecker) check() {
    defer c.wg.Done()
    for ip := range c.input {
        if isClickHouse(ip, c.port) {
            c.output <- ip
        }
    }
}

func isClickHouse(ip string, port int) bool {
    conn, err := clickhouse.Open(&clickhouse.Options{
        Addr: []string{fmt.Sprintf("%s:%d", ip, port)},
        Settings: clickhouse.Settings{"max_execution_time": 1},
        DialTimeout: time.Second,
        MaxOpenConns: 1,
        MaxIdleConns: 1,
        ConnMaxLifetime: time.Minute,
        ConnOpenStrategy: clickhouse.ConnOpenInOrder,
        BlockBufferSize: 10,
        MaxCompressionBuffer: 1024,
    })
    if err != nil { golog.Errorf("open %s:%d failed: %v", ip, port, err); return false }
    ctx, cancel := context.WithTimeout(context.Background(), time.Second)
    defer cancel()
    if err = conn.Ping(ctx); err != nil { golog.Warnf("ping %s:%d failed: %v", ip, port, err); return false }
    return true
}

Scanning the 3 953 IPs that responded on port 9000 yielded the following outcomes:

~3 950 hosts timed out or were not ClickHouse services.

Four hosts accepted a ClickHouse connection; three required authentication (error:

default: Authentication failed: password is incorrect, or there is no user with such name.

).

Two hosts allowed unauthenticated access.

The low hit rate is explained by common deployment practices such as IP‑based whitelisting or non‑standard ports.

Full‑network scan workflow

1. Download the APNIC delegation file and convert it to CIDR blocks:

wget -c -O- http://ftp.apnic.net/stats/apnic/delegated-apnic-latest | awk -F '|' '/ipv4/ {print $4 "/" 32-log($5)/log(2)}' > ipv4.txt

2. Run icmp_scan against the generated list. The tool reported 15 500 888 live hosts out of ~8 × 10⁸ candidates in 2 h 35 m.

3. Run tcp_scan on the alive hosts to probe port 9000. It found 3 953 reachable ports in 2 m 41 s.

4. Run clickhouse_check to validate each reachable host. The entire pipeline completed in roughly four minutes, confirming only two unauthenticated ClickHouse instances.

The same methodology can be adapted to test other services (e.g., Redis, MySQL) by swapping the service‑specific checker.

Reference implementation: https://github.com/smallnest/fishfinder