Why Alibaba Cloud SLS Beats Open‑Source ELK for Log Management

Background

ELK (Elasticsearch, Logstash, Kibana) is a mainstream open‑source log solution widely used in observability. With accelerating digitalization, machine‑generated logs increase, and self‑built ELK faces challenges in large‑scale data and query performance. Low‑cost, high‑availability observability is a new topic.

SLS vs Elasticsearch History

Elasticsearch started in 2010, built in Java on Lucene, initially for enterprise search and later entered observability. Alibaba Cloud SLS, launched in 2012, is a serverless service built in C++ on the Flying‑Pangu storage, offering high performance and reliability, and has been publicly available since 2017.

SLS Core Architecture

SLS uses Alibaba Cloud Flying‑Pangu distributed file system for storage, supporting Log/Metric/Trace data with multi‑replica backup and various storage tiers (hot, cold, archive). On top of storage it provides SQL (standard SQL‑92), index query (similar to Lucene), data processing, and data pipelines with Kafka‑like consumption.

Feature Comparison

Collection: SLS uses iLogtail (C++ high‑performance, open‑source) vs Beats/Logstash for ELK.

Storage: SLS Logstore supports PB‑scale; ELK index limited to hundreds of GB.

Query: Both support queries; SLS adds SPL for index‑less queries.

SQL: SLS supports full SQL‑92; ELK has incomplete SQL support.

Streaming: SLS supports Flink/Spark consumption via Kafka or native protocol; ELK does not.

Alerting: Native in SLS; ELK requires X‑Pack or third‑party tools.

Visualization: SLS console, Grafana, Kibana; ELK uses Kibana or Grafana.

DevOps integration: SLS console can be embedded; ELK needs SDK/API development.

AIOps: Native in SLS; requires X‑Pack in ELK.

Operational Comparison

Capacity planning: SLS is serverless, no capacity concerns; ELK requires manual capacity planning and can suffer from disk‑full issues.

Machine operation: SLS abstracts machines; ELK requires monitoring of node availability.

Performance tuning: SLS scales by adding Logstore shards; ELK needs expert support.

Version upgrades: SLS upgrades transparently; ELK upgrades may break compatibility.

Data reliability: SLS stores three replicas by default; ELK replica count is configurable and less reliable with single replica.

SLA: SLS guarantees service level; ELK depends on dedicated teams.

Performance Test

In a lab test with 1 billion records, SLS returned query results within seconds, while Elasticsearch latency grew with concurrency and was overall slower. Write throughput of Elasticsearch was about 2 MB/s per core, whereas a single SLS shard handled 10 MB/s, and adding shards increased write speed.

Cost Comparison

SLS charges 0.4 CNY/GB of write volume with 30 days of free storage. For 10 TB daily ingestion, 30‑day, 90‑day, and 180‑day costs are significantly lower than self‑built ELK, which requires expensive ECS instances and storage. Even at 10 GB/day for 30 days, SLS costs ~4 CNY/month versus >200 CNY for comparable ECS resources.

Open‑Source Compatibility

SLS provides Elasticsearch‑compatible and Kafka‑compatible interfaces, translating requests to its own protocol. This enables a single dataset to be accessed via both ES queries and Kafka consumption without additional data pipelines.

Migration Solutions

Dual‑collector migration: Deploy iLogtail on existing machines, collect logs to SLS while keeping existing Beats agents, then switch off Beats after validation.

Direct agent write: Use existing agents to write logs to SLS via Kafka protocol.

Kafka import: Configure SLS Kafka import to ingest existing Kafka topics without deploying instances.

Elasticsearch import: Use SLS Elasticsearch import to migrate historical ES data.

Summary

SLS offers a serverless, high‑availability, low‑cost log service with rich features and open‑source compatibility, making migration from self‑built ELK straightforward and reducing operational overhead.