How Alibaba Cloud Combines RDS PostgreSQL & HybridDB for Real‑Time HTAP Analytics

In 2017 Double‑11, transaction creation peaked at 325,000 per second and payment peaked at 256,000 per second, generating massive real‑time order feed streams that feed into a data‑operation platform for proactive services.

Business Background

An e‑commerce ecosystem involves merchants, stores, logistics, users, payment channels, loans, products, platforms, advertisers, manufacturers, distributors, staff, regulators, tax, quality inspection, etc., producing huge volumes of browsing, order, complaint, refund, and dispute data. FEED logs are recorded across systems to capture user behavior. To simplify analysis, these scattered records are merged into a wide order table, enabling real‑time sentiment display and proactive service identification.

Challenges

Beyond real‑time requirements, the write path must handle data switching, merging, and cleaning. Achieving tens of thousands of writes per second while also providing minute‑level latency for millisecond‑level analysis and proper scheduling is non‑trivial.

Solution Overview

The architecture combines RDS PostgreSQL, HybridDB for PostgreSQL, and OSS for transparent data access.

RDS PostgreSQL – Alibaba Cloud’s managed PostgreSQL service.

HybridDB for PostgreSQL – An MPP‑based distributed analytical database with superior performance for multi‑table joins, complex queries, real‑time statistics, and support for JSON, GIS, HLL, etc.

OSS – Alibaba Cloud’s low‑cost, highly reliable object storage, used here for offline data storage.

The key is transparent access between RDS PostgreSQL, HybridDB, and OSS.

Multiple PostgreSQL instances receive business writes, perform data cleaning, and export cleaned data to OSS via external tables (similar to heap tables). After the write completes, HybridDB reads the same OSS objects through external tables, loading data in parallel for millisecond‑level queries on tens to hundreds of terabytes.

Creating an external table in PostgreSQL maps to an OSS object (image):

Inserting data that meets filter conditions writes to the OSS object "osstest/example.csv".

HybridDB reads the same OSS data via SQL INSERT statements (image):

Advantages

Performance – Merges PostgreSQL’s high‑concurrency write capability with HybridDB’s MPP analytical speed, supporting million‑level writes and real‑time millisecond queries.

Data movement & cleaning – All data transport and cleaning are performed via SQL, eliminating heavy ETL pipelines.

Unified hot‑cold data – Hot data resides in PostgreSQL/HybridDB, cold data in OSS; both are accessed uniformly through SQL.

Dynamic resource scaling – RDS resources and HybridDB segments can be elastically scaled without affecting availability.

Resulting benefits include reduced cost, seamless hot‑cold data access, integrated OLTP/OLAP, and unlimited scaling of RDS instances and HybridDB clusters.

Alibaba Cloud RDS PostgreSQL

Based on the most advanced open‑source PostgreSQL, it offers stability, performance comparable to commercial databases, a rich ecosystem of extensions, plug‑in architecture, FDW for external data sources, and Alibaba‑specific features such as HA, seamless scaling, automatic backup, and transparent OSS access.

Alibaba Cloud HybridDB for PostgreSQL

Derived from Greenplum, this MPP‑based distributed analytical engine supports real‑time GIS, JSON, XML, fuzzy string analysis, ACID consistency, column‑row hybrid storage, linear scalability, and security features like IP whitelisting and DDoS protection.

Conclusion

By leveraging Alibaba Cloud’s ecosystem—RDS PostgreSQL, HybridDB for PostgreSQL, and OSS—enterprises can build an intelligent BI platform that closes the loop from online business to real‑time data analysis.