How Zepto Scaled Real‑Time Brand Analytics with StarRocks: From Postgres MVP to Sub‑Second Queries

Zepto, an Indian instant‑delivery retailer known for its 10‑minute delivery promise, grew to serve 50+ cities and 45,000+ SKUs, generating millions of daily events across brands, cities, and products. To turn raw data into actionable, real‑time insights for partner brands, Zepto built a Brand Analytics dashboard.

Early MVP on PostgreSQL

During the initial phase the team used a lightweight PostgreSQL MVP to quickly launch the product and collect feedback. The core sales table held only a few million rows, which PostgreSQL handled comfortably while tracking GMV, sales, and other key metrics across brand, city, and product dimensions.

Scaling Challenges

Rapid business expansion pushed daily imports to over 30 million rows per table, eventually exceeding 200 million transaction rows. PostgreSQL, optimized for OLTP workloads, began to struggle with the complex, large‑scale analytical queries required by brand partners, prompting the need for a dedicated OLAP system.

Evaluating OLAP Candidates

The team benchmarked ClickHouse, Apache Pinot, and StarRocks. ClickHouse offered fast queries but faltered on complex multi‑table joins; Pinot excelled at low‑latency writes but lacked needed query flexibility. StarRocks satisfied all core requirements and delivered additional benefits.

Why StarRocks?

Lightning‑fast joins: StarRocks’ join optimizer handles complex multi‑table joins efficiently, crucial for Zepto’s analytics.

Sub‑second latency at >300 M rows: Benchmarks showed P99 query latency under 500 ms on 300 M‑row datasets.

Native integration: Built‑in support for Kafka routine load and S3/Parquet enables seamless pipelines with existing Databricks and Flink infrastructure.

Architecture Decisions

Two storage models were considered: integrated storage‑compute (data stored locally in StarRocks) and decoupled storage‑compute (querying data directly from object storage). After evaluation, Zepto chose the integrated model to maximize query performance for brand‑facing dashboards.

Data Ingestion Pipelines

Zepto now imports data via two primary pipelines:

Pipe Load from Databricks (S3/Parquet): A scheduled job syncs Databricks tables to S3 as Parquet files. StarRocks continuously scans the S3 path and loads new files automatically.

Kafka Routine Load: Real‑time event streams (≈60 k events per second) are processed by Apache Flink, aggregated in 5‑minute windows, and written to Kafka topics consumed by StarRocks.

Typical Pipe Load command:

CREATE PIPE <pipe_name>
PROPERTIES (
  "AUTO_INGEST" = "TRUE"
)
AS
INSERT INTO <tablename>
SELECT * FROM FILES (
  "path" = "s3://<bucket>/<folder>/*.parquet",
  "format" = "parquet",
  "aws.s3.region" = "<region>",
  "aws.s3.access_key" = "<access_key>",
  "aws.s3.secret_key" = "<secret_key>"
);

Key tips: enable AUTO_INGEST for automatic loading of new Parquet files, use soft‑delete flags for deletions, and choose appropriate primary keys to support upserts.

Kafka routine load provides exactly‑once semantics, simple configuration, and native support without extra connectors, allowing Zepto to ingest >30 million rows per day into a single core table.

Real‑Time Analytics Flow

Event collection: Over 60 k events per second flow from Kafka, covering exposures, orders, and deliveries.

Flink processing: Events are filtered, window‑aggregated (5‑minute windows), and written to downstream Kafka topics.

StarRocks ingestion: Routine load consumes the processed streams, making data instantly queryable.

Brand dashboards: Queries across SKU, brand, and region dimensions return results in sub‑second latency, enabling brands to make rapid, data‑driven decisions.

Results and Impact

Seamless migration from Postgres MVP to a production‑grade StarRocks analytics architecture.

Achieved sub‑second query performance on dashboards serving brand partners.

Integrated S3 Pipe Load and Kafka Routine Load for real‑time data ingestion.

Ingested >30 million rows per day per core table, with most queries involving 2–3 table joins.

Adopted an integrated storage‑compute model to balance low latency and high performance.

Future work includes deeper performance benchmarking against other databases, addressing scaling challenges, and sharing additional technical learnings.

Thanks to Syed Shah, Rajendra Bera, Ashutosh Gupta, Harshit Gupta, and Deepak Jain for their contributions.