How 360 Built a Lightning‑Fast Unified Analytics Platform with StarRocks

Business Background

360, a leading internet security service provider, generates huge volumes of data from its diverse business lines. The existing analytics stack—MySQL, Druid, Hive, Flink, and Redis—struggled with billions of rows, high maintenance overhead, low compression, and inadequate query performance, especially for ad‑hoc and retention analyses.

Limitations of the Legacy Architecture

MySQL could not handle the scale without extensive sharding, leading to complex routing logic and high operational cost.

Druid lacked support for array‑type tags, requiring a parallel Hive‑on‑Spark pipeline for user‑portrait data.

Multiple components (Hive, Spark, Flink, Redis) created a tangled data pipeline, increasing development and O&M effort.

Why StarRocks?

StarRocks offered four key capabilities that addressed these pain points:

Ultra‑fast multi‑dimensional queries —benchmark tests (single‑table, two‑table shuffle join) showed that StarRocks could meet the platform’s latency requirements, with bitmap‑based materialized views further accelerating distinct counts.

Support for complex data structures —array functions (e.g., array_contains, array_sum) enable straightforward analysis of tag arrays used in user‑portrait scenarios.

Flexible data ingestion —Broker Load, Routine Load, and Stream Load accommodate HDFS, Kafka, and local files, simplifying integration.

Simplified data pipeline —a single OLAP engine replaces the previous heterogeneous stack, reducing development, maintenance, and storage costs.

Performance Evaluation

Testing environment details (illustrated in the accompanying diagram) compared single‑table scans and two‑table shuffle joins. Results demonstrated that StarRocks consistently delivered sub‑second response times, comfortably satisfying business SLAs.

Implementation in 360’s Data Platform

Both the Radar Analysis Platform and the User‑Portrait Platform now share a unified StarRocks cluster. The radar platform uses StarRocks as a summary layer, handling up to 100 billion rows per second per node, eliminating MySQL sharding and keeping query latency under 2 seconds. The user‑portrait platform stores tag data in a detail model, converting user_id to bitmap format for efficient set operations.

select tag, bitmap_intersect(user_id)
from (
  select tag, date, bitmap_union(user_id) user_id
  from table
  where date in ('2020-05-18', '2020-05-19')
  group by tag, date
) a
group by tag;

This query illustrates how bitmap aggregation simplifies retention calculations across different dates.

Benefits Realized

Data pipeline complexity and maintenance costs were dramatically reduced.

Query performance supports ad‑hoc analysis on billion‑row tables with latencies between 200 ms and 4 s.

Array‑type tag data can be analyzed directly using built‑in functions.

Bitmap operations enable precise distinct counting and fast cohort analysis.

Future Plans

360 intends to migrate additional portrait scenarios to StarRocks, ingest remaining historical Druid data, integrate StarRocks into the data‑governance platform, and explore its data‑lake capabilities, which have already shown several‑fold performance gains over Trino + Iceberg.