StarRocks is an open‑source project under the Linux Foundation, focused on building a high‑performance, scalable analytical database that enables enterprises to create an efficient, unified lake‑house paradigm. It is widely used across many industries worldwide, helping numerous companies enhance their data analytics capabilities.
This guide walks you through using StarRocks—covering effortless expression‑based partitioning, streamlined data loading with INSERT FROM FILES and PIPE, powerful in‑flight data transformation using SELECT/JOIN/UNNEST, and flexible table structure tweaks via ALTER TABLE to boost query performance.
In this interview, graduate student Sun Yinzhen shares how he selected, designed, and implemented an ORC Chunk Writer for the StarRocks database during the Open Source Summer program, detailing the technical challenges, learning outcomes, and his perspective on open‑source contributions for computer science students.
StarRocks’ materialized view feature lets Didi replace costly fuzzy deduplication with precise, high‑concurrency deduplication for real‑time dashboards, using global dictionary mapping, layered ODS/DWD/ADS views, synchronous and asynchronous refreshes, and transparent query rewrite to cut query latency by 80% and boost QPS dramatically.
This article explains how Apache InLong provides automatic, secure, high‑performance real‑time data transfer to StarRocks, detailing the transactional Stream Load API, the two‑phase commit process, Flink‑based ingestion architecture, exactly‑once guarantees, and performance test results across different parallelism levels.
In 2022 Tencent's A/B Test team created the overseas SaaS product ABetterChoice, abstracting internal experiment capabilities, adapting to multi‑cloud compliance, and unifying computation with StarRocks, enabling game titles like Honor of Kings, PUBG Mobile, and Ubisoft to run scalable, compliant A/B experiments worldwide.
Xiaohongshu’s OLAP team migrated from Presto to StarRocks, doubling cluster count to 30, boosting query speed by 6‑7 times, cutting latency to 200 ms, and achieving up to 35% cost reduction through gray‑scale migration and AWS Spot‑based elastic scaling.
StarRocks 3.2, released on December 21, 2023, introduces major usability upgrades—including optimized random bucketing, fast schema evolution, PIPE import, HTTP SQL API, runtime profiling, enhanced storage‑compute separation, data lake analysis, and advanced materialized view capabilities—while refining existing features such as indexing, catalog support, and export syntax.
WeChat transformed its data platform from Hadoop and ClickHouse to a StarRocks‑based lakehouse, tackling massive data volume, ultra‑low latency, and storage fragmentation by deploying lake‑on‑warehouse and warehouse‑lake fusion architectures, real‑time incremental materialized views, and unified SQL access, resulting in dramatic cost cuts and performance gains.
The article explains why analytical databases struggle with real‑time data changes due to columnar storage, complex indexes and distributed processing, and then details StarRocks' primary‑key model, adaptive update mode, bitmap indexes, row/column partial updates, and practical SQL upsert techniques to achieve low‑latency updates without sacrificing query performance.
StarRocks, an open‑source MPP analytical database, consolidates BI, interactive, and real‑time analytics into a single engine by evolving from version 1.0 to 3.x, introducing compute‑storage separation, unified catalog, generated columns, operator spill, and advanced materialized views, while outlining its cloud‑native lakehouse roadmap.
