How HuoLala Accelerated User Profiling 30× Faster with Apache Doris

HuoLala, a leading on‑demand freight platform, serves millions of users across dozens of markets and needed a robust user‑profile system to support fine‑grained operations.

Profile Service Background and Architecture

The platform processes over 300 business scenarios, 3,000+ tags, and 50,000+ user groups. Apache Doris serves as a unified query engine, providing efficient analysis for batch profiling, tag aggregation, and audience selection.

Application Architecture

The system consists of two core modules:

API Layer : Provides persona‑api for high‑concurrency queries, persona‑analysis‑api for analysis jobs, and persona‑web‑api for the management UI.

BE Layer : Handles distributed computation (persona‑task) and scheduling (persona‑scheduler) for user‑tag pipelines.

Profile Engine Evolution

Three stages were explored:

Initial stage used Impala+Kudu, suffering >10‑minute query latency and poor scalability.

Second stage introduced Elasticsearch, which reduced some latency but added high development cost and limited multi‑dimensional analysis.

Final stage adopted Apache Doris for its MPP architecture, vectorized engine, rich optimizer, bitmap support, and strong community.

After switching to Doris, single‑group computation became sub‑minute (often seconds), a 30× speedup over Impala, and data import of 400 M rows × 200 columns dropped from >90 minutes to ~30 minutes, a three‑fold improvement.

Data Model Design and Heterogeneous Query Implementation

Core Challenges

Over 3,000 tags span three dimensions: business attribute, aggregation granularity, and update timeliness, requiring different storage and query patterns.

Storage Model Design

Three Doris‑based models were created:

Wide Table : Stores low‑frequency, dense tags using columnar storage, indexes, and materialized views.

High Table : Stores high‑frequency sparse tags with bitmap columns for millisecond‑level set operations.

Population Bitmap Table : Stores pre‑computed audience results using RoaringBitmap to compress user ID sets.

Example DDL for the wide table:

CREATE TABLE wide_table ( user_id varchar(1000) NULL COMMENT "", age bigint(20) REPLACE_IF_NOT_NULL NULL COMMENT "", height bigint(20) REPLACE_IF_NOT_NULL NULL COMMENT "", ... ) ENGINE=OLAP AGGREGATE KEY(user_id) COMMENT "OLAP" DISTRIBUTED BY HASH(user_id) BUCKETS 40 PROPERTIES ( ... );

Example DDL for the high table:

CREATE TABLE high_table ( tag varchar(45) NULL COMMENT "标签名", tag_value varchar(45) NULL COMMENT "标签值", time datetime NOT NULL COMMENT "数据灌入时间", user_ids bitmap BITMAP_UNION NULL COMMENT "用户集" ) ENGINE=OLAP AGGREGATE KEY(tag, tag_value, time) COMMENT "OLAP" DISTRIBUTED BY HASH(tag) BUCKETS 128 PROPERTIES ( ... );

Example DDL for the bitmap table:

CREATE TABLE routine_segmentation_bitmap ( time datetime NOT NULL COMMENT "数据灌入时间", seg_name varchar(45) NULL COMMENT "标签值", user_ids bitmap BITMAP_UNION NULL COMMENT "人群ID集合" ) ENGINE=OLAP AGGREGATE KEY(time, seg_name) COMMENT "OLAP" PARTITION BY RANGE(`time`) (...) DISTRIBUTED BY HASH(seg_name) BUCKETS 128 PROPERTIES (..., "dynamic_partition.enable" = "true", ...);

Population Selection and Heterogeneous Combination Query

All tag results are converted to bitmaps (TO_BITMAP) and combined using UNION ALL, then intersected or united with BITMAP_INTERSECT / BITMAP_UNION, allowing unlimited nested logical rules.

SELECT BITMAP_INTERSECT(b) FROM ( -- Layer1: wide table condition A SELECT TO_BITMAP(...) AS b FROM wide_table WHERE conditionA UNION ALL -- Layer1: high table condition B SELECT user_ids AS b FROM high_table WHERE conditionB UNION ALL -- Layer1: population condition C SELECT user_ids AS b FROM population_table WHERE conditionC UNION ALL -- Layer2: nested conditions ... ) t;

Data Ingestion

Real‑time tags (seconds‑to‑hours) are streamed via Flink into Doris/HBase, while hourly tags use BrokerLoad. Offline tags (daily) are loaded from Hive using BrokerLoad, with parallel tasks to achieve 30‑minute imports for 200‑plus wide‑table tags and 5‑minute imports for high‑frequency tags.

Query Optimization Practices

DSL and SQL Optimization

The DSL built on the front‑end is optimized to merge redundant tag conditions and flatten nested AND/OR structures before being translated into efficient SQL, reducing bitmap reads by 60% and memory usage by up to 50% during peak loads.

-- Before optimization
SELECT BITMAP_INTERSECT(b) AS result FROM ( SELECT BITMAP_INTERSECT(b) AS b FROM ( SELECT user_bitmap as b FROM user_bitmap WHERE "group"='A' UNION ALL SELECT TO_BITMAP(id) AS b FROM wide_table WHERE city='Dongguan' UNION ALL SELECT TO_BITMAP(id) AS b FROM wide_table WHERE sex='Male' ) t1 ) t2;

-- After optimization
SELECT BITMAP_INTERSECT(b) AS result FROM ( SELECT user_bitmap as b FROM user_bitmap WHERE "group"='A' UNION ALL SELECT TO_BITMAP(id) AS b FROM wide_table WHERE city='Dongguan' AND sex='Male' ) t1 ) t2;

Population Bitmap Table Read Optimization

Three approaches are compared: using Doris's bitmap_to_string (simple but heavy), using explore + lateral view (flexible but adds server pressure), and reading raw binary data with return_object_data_as_binary=true (recommended for low‑memory, high‑throughput scenarios).

Summary and Future Plans

Since adopting Apache Doris, HuoLala has achieved nearly 30× query efficiency, 3× faster data import, and 30‑50% lower memory consumption, enabling fine‑grained operations over thousands of tags. Future work focuses on expanding real‑time profiling workloads to Doris and integrating a lake‑house architecture for ultra‑large‑scale analytics.