How LoongCollector Transforms Log Collection with High‑Performance Pipelines

Overview

LoongCollector is the next‑generation log‑collection agent that evolved from iLogtail. In 2025 it was renamed and rebuilt, delivering deep optimizations in functionality, performance, and stability for cloud‑native environments.

Historical Milestones

2013: iLogtail first released with basic file collection via inotify.

2015: Expanded to Alibaba Cloud, added high‑water‑mark queues, checkpoint, multi‑tenant support.

2017: With SLS commercialisation and ACK, added Go plugin system, container log collection, and support for metrics and traces.

2022: Open‑sourced as 1.0.0, supporting all major container runtimes and profiling data.

2024: 2.0.0 released with further improvements in usability, performance and reliability.

Architecture and Pipeline

Each collection task is represented by a pipeline consisting of an input plugin, a processor plugin and a flusher plugin. LoongCollector supports multiple pipeline combinations, e.g.

C++ Input + C++ native processor

C++ Input + SPL processor

C++ Input + Golang processor

Golang Input + Golang processor

The agent runs three dedicated runner threads (Input, Processor, Flusher) that communicate via buffered queues, providing fair scheduling and isolation.

Hot‑Reload and Isolation

Unlike the previous “Stop‑the‑World” reload, LoongCollector replaces only the pipelines that changed, keeping the rest running and minimizing impact on other teams sharing the same instance.

Performance Gains

Benchmarks show average CPU reduction of 35 % and memory reduction of 10 % compared with iLogtail. In file‑collection scenarios LoongCollector achieves up to 80 % higher throughput, and in container‑standard‑output cases performance improves 2× (containerd) to 1× (docker).

Monitoring and Fault Isolation

LoongCollector provides a built‑in dashboard for CPU, memory, network and pipeline latency. It isolates network‑level failures by applying adaptive AIMD throttling per AZ, project and logstore, ensuring a single region outage does not affect other destinations.

Tag Processing

Tag handling has been unified across C++ and Golang pipelines. Users can add, delete or rename tags via processor plugins, and input‑level tags are managed by each input plugin.

Seamless Upgrade Path

Existing iLogtail configurations, checkpoints and offsets are fully compatible; upgrading to LoongCollector requires no data loss and behaves like a simple restart. In Kubernetes, DaemonSet replacement is performed with affinity control to avoid downtime.

Future Directions

LoongCollector will integrate Prometheus metrics, eBPF collection and other observability sources to become a one‑agent solution for the full observability stack.