How GaussDB(DWS) HStore Tables Enable Real‑Time Ingestion and Lightning‑Fast Queries

In the live session “Real‑time Ingestion Tool—HStore Table Principles and Application Practice”, Huawei Cloud invited technical evangelist Ma Jun‑song to discuss GaussDB(DWS) HStore tables.

GaussDB(DWS) evolves from traditional row and column storage to a new HStore table that retains columnar format while supporting high‑concurrency insert, upsert, and update operations with low latency.

Row storage: Stores data by rows, suitable for OLTP, offers fast point queries and updates but limited analytical performance and compression.

Column storage: Designed for OLAP, provides strong compression and batch query performance, but cannot handle small‑batch ingestion efficiently.

Column storage with Delta: Adds a Delta layer to enable small‑batch ingestion for OLAP workloads, though it still cannot support concurrent updates.

HStore table: Combines columnar storage with a Delta‑like mechanism to eliminate lock conflicts, delivering ACID‑consistent, high‑concurrency upsert/insertion and high‑performance queries while keeping disk usage low.

The following diagram compares several GaussDB(DWS) table types:

In real‑time ingestion scenarios, data is streamed via Kafka or Flink into DWS, and downstream applications can query the data instantly. HStore tables support hot‑to‑cold data migration to OBS and external table imports in ORC/Parquet/text formats for AI mining.

Technical characteristics of HStore tables

Full transaction consistency: Immediate visibility of inserts/updates with ACID guarantees.

Comprehensive feature support: Same syntax and capabilities as columnar tables.

Excellent query performance: Optimized distributed query plans, support for complex joins, subqueries, and stored procedures, plus indexing and partitioning.

Fast ingestion: Resolves columnar CU lock conflicts, achieving up to 100× higher concurrent update throughput.

High compression: Columnar MERGE naturally compresses data, saving disk space and I/O.

Delta table vs. traditional Delta table

Practical usage

HStore provides views to monitor Delta tuple counts and inflation, and functions for lightweight and full cleanup of Delta tables.

Developers can validate compression efficiency, concurrent update ingestion, and query performance by watching the live video.

Usage considerations

Parameter settings: HStore relies on a background thread for MERGE cleanup; appropriate GUC parameters must be configured (see recommended settings).

Concurrent updates on the same row: Not supported; concurrent upserts wait for locks then proceed, while concurrent updates/delete wait then error, potentially causing deadlocks.

Indexing: Indexes consume extra space and offer limited point‑query gains; create primary key or btree indexes only when upsert or unique point queries are required.

MERGE handling: Ingestion speed must not exceed MERGE speed to avoid Delta table bloat; control concurrency accordingly.

UPSERT performance: HStore upsert throughput is about one‑third of row storage; for workloads prioritizing single‑point query speed over compression, row storage may be preferable.