Why LoongCollector Beats iLogtail and Open‑Source Log Agents by Up to 10×

Background

Log data is a critical asset for modern cloud‑native environments, but traditional log‑collection agents cannot keep up with the massive, real‑time ingestion demands of containerized and micro‑service workloads.

LoongCollector Overview

LoongCollector is the next‑generation evolution of Alibaba’s iLogtail agent. It retains iLogtail’s stability while redesigning the full stack to reduce lock contention, reuse object pools, and optimize serialization (Protobuf). This yields a substantial throughput boost and lower resource consumption.

Repository: https://github.com/alibaba/loongcollector

Test Methodology

Environment

Alibaba Cloud ECS instance: 32 CPU / 64 GB RAM

OS: Ubuntu 22.04

Network: VPC internal (no external latency)

Log Scenarios

Simple single‑line logs (128 B)

Multi‑line logs (512 B, 23 lines per group)

Complex regex logs (512 B)

Complex JSON logs (512 B)

Delimiter‑separated logs (512 B, ‘|’ delimiter)

Test Steps

Add a collection configuration that sends logs to an SLS Logstore in the same region via internal VPC.

Run a log‑generation process that writes up to 1100 MB/s.

Query the SLS Logstore to obtain the actual traffic.

After a stable one‑minute collection period, compute average traffic, CPU usage, and memory usage.

Vertical Comparison (LoongCollector vs. iLogtail)

Across all five scenarios LoongCollector achieved 78 %–84 % higher single‑thread throughput than iLogtail. Example:

Simple‑line test: LoongCollector 338 MB/s vs. iLogtail 190 MB/s.

CPU usage grew roughly linearly with traffic for LoongCollector, while iLogtail’s CPU consumption inflated disproportionately, indicating less efficient processing. Memory growth for LoongCollector remained modest even at high traffic levels.

Horizontal Comparison (LoongCollector vs. Open‑Source Agents)

Three widely used open‑source agents were benchmarked under identical conditions, sending data to Kafka to ensure a fair comparison.

FluentBit v4.0.0

Vector v0.45.0

Filebeat v8.17.4

Single‑thread, simple‑line (512 B) throughput

LoongCollector: 338 MB/s

FluentBit: 27 MB/s

Vector: 16 MB/s

Filebeat: 7 MB/s

LoongCollector consistently exceeded the best open‑source agent by more than threefold while using less CPU and memory.

Resource‑Usage Findings

At 1 MB/s traffic, LoongCollector’s CPU increase was negligible (~0.7 %).

At 400 MB/s traffic, CPU usage rose to only ~35 % of a single core and memory grew by ~12 %.

Open‑source agents showed steep CPU and memory growth under the same loads.

Sample Log (Complex Regex)

[2025-04-03 10:02:06.561793]    [info]  [131172] /root/stdout-test/server_stdout.go:165 log:ZQsC8JI2l9M6Zum1b09V8QZy7MBk8fI01kmg12XqfHXWxdD4SBYUdGKRH4iRCcjIVIOAXmv8I0TgQlJKtwYxAhJR9O2N1BEirA1v01IqWyGaVsxCxRjCpvhkWQ03wW3CnKUCLCndugLfWYsxiYMJs7YiqYhOlCglTj4XdUQqlOfZTrYdyFNX3fVQk9jwBAO5NEBUAo0VgL7rt86lENPr5wA1UQWNdj2id00ByhsBakCjyRP9tvxDVrTSEq5oEVowKYBYzcjJCK1q56MVDm1BhSfNGrLQifr3nYv5Z8yu1d8EAjK9iQGjVqLxz65IozXKyf40R2TZYR9TS2GAMxMyC7uIvZiMh0TRB4rQL4K4uFeOVkFMGrHG3a3GanWJyrr4wBvFh2ehKh98EGOxo3x7ZVAm3Hz

Conclusions

LoongCollector sets a new performance baseline for log collection in cloud‑native environments:

Up to 10× higher throughput than leading open‑source agents.

Linear CPU growth and modest memory increase even at several hundred MB/s.

Retains iLogtail’s reliability while delivering far better scalability.

Beyond logs, LoongCollector also supports metrics and traces (e.g., Prometheus, eBPF), positioning it as a comprehensive observability collector for modern enterprises.

Future work will focus on further performance optimizations and expanding ecosystem integrations.