Real‑Time Data Analysis at AutoHome: Evaluation and Adoption of StarRocks

AutoHome's real‑time data originates from three sources: mobile client logs, server‑side application logs, and relational databases such as MySQL and SQLServer.

The current real‑time analytics stack faces several pain points: Flink‑based metric aggregation is inflexible and costly to develop for changing requirements; Apache Kylin pre‑computes metrics efficiently but cannot query detailed data quickly; TiDB lacks pre‑aggregation support, causing high online computation load and unstable query performance.

StarRocks was selected because it supports flexible query scenarios (both detail‑level aggregation and pre‑aggregated models), is MySQL‑compatible for low‑cost operations, offers a fully vectorized engine with high concurrency and throughput, and has a simplified architecture that eases maintenance.

Performance benchmarks show that StarRocks matches or exceeds Apache Kylin on fixed‑report queries, outperforms ClickHouse on non‑exact distinct (HLL) queries with a 3‑4× speedup, and delivers 2‑7× better query latency than Apache Doris on 600 million‑row workloads. Compared with Presto and Spark, StarRocks consistently achieves the best performance, especially when using the ndv function for approximate distinct counting.

Additional tests reveal that SSD storage can improve query speed by 3‑8× on cold‑cache workloads, while the benefit disappears when the page cache is warm.

In practice, StarRocks has been integrated with both real‑time and offline platforms. Real‑time ingestion uses the Flink‑connector‑StarRocks to map existing StarRocks tables to Flink tables. Offline ingestion employs broker‑load scripts that pull Hive data into StarRocks, with automatic progress monitoring and retry mechanisms.

Monitoring is built on Prometheus and Grafana, collecting StarRocks audit logs, parsing each SQL execution, and visualizing query counts, latency, and errors via AutoBI dashboards.

Two production use cases are highlighted: (1) Real‑time monitoring of recommendation services, ingesting ~200 million rows daily and achieving a TP95 response time of about 1 second; (2) Real‑time search effectiveness analysis, processing tens of billions of rows per day using grouping sets and materialized views, with query latency reduced from over ten seconds to under four seconds after upgrading from StarRocks 1.17 to 1.18.

The overall conclusion emphasizes that StarRocks unifies detail and pre‑aggregation queries, bridges streaming and batch workloads, offers superior query performance, and simplifies operations through MySQL compatibility. Future plans include extending the toolchain for seamless real‑time data distribution, expanding StarRocks use cases, and improving cluster stability.