Understanding SAP HANA’s Combined Technologies: Memory, Columnar Storage, Compression, and Insert‑Only

How to handle “depends on the situation”

A simple approach is to implement both options, such as storing data in both columnar and row formats, allowing users to choose the most suitable view while accepting the cost of duplicate storage.

A better approach is to combine the strengths of multiple techniques so that the weaknesses of one are mitigated by another, which is the secret behind SAP HANA’s performance.

By integrating in‑memory processing, columnar storage, compression, and an insert‑only model, SAP HANA achieves a level of efficiency that other vendors cannot match due to historical design choices.

Pros and Cons of Each Technology

Memory

Memory is orders of magnitude faster than disk; modern CPUs can deliver >20 GB/s memory bandwidth versus ~550 MB/s for SSDs and ~180 MB/s for HDDs. However, RAM is far more expensive (≈$7/GB) than disk (≈$0.05/GB), making large‑scale memory deployment costly.

Pros: Fast data updates Quick inserts Rapid reads

Cons: Data is lost on power failure High cost for large capacities (e.g., 1 TB RAM ≈ $7,000)

Compression

Compressing data reduces storage size and I/O, but incurs CPU overhead during compression and decompression, especially when updating compressed blocks.

Pros: Reduces required storage size

Cons: Higher CPU usage for inserts, reads, and updates

Columnar Storage

Columnar storage excels for analytical queries (e.g., SUM) because data for a single column is stored contiguously, enabling fast scans. However, retrieving an entire row requires accessing multiple columns stored in different locations.

Pros: All values of a column are tightly packed

Cons: Row reconstruction can be slower

Insert‑Only

Insert‑only guarantees read consistency by never overwriting existing rows; updates are appended as new versions with timestamps. This avoids the need to decompress and recompress data on each update.

Pros: Fast writes

Cons: Potentially slower reads Increased storage growth

Combining the Technologies

When the four techniques are combined, most disadvantages are mitigated: compression becomes an efficient way to reduce I/O, columnar storage optimizes query performance, insert‑only provides consistent reads without locks, and memory enables the entire database to reside in RAM.

Compression: turns storage overhead into an effective read/write strategy.

Columnar storage: optimal for both wide‑row and narrow‑column queries.

Insert‑only: offers strict read consistency without complex locking.

Memory: allows even large databases to fit in RAM.

Remaining challenges include persisting data to disk to avoid loss, merging compressed and uncompressed regions to limit table growth, and hardware costs for very large databases.

Typical solutions involve transaction logs on disk for durability, periodic batch merges of compressed regions, and scaling strategies such as horizontal expansion (multiple smaller servers) or dynamic tiering (hot data in memory, warm data on disk).

Overall, SAP HANA’s uniqueness lies not in inventing each individual technology but in tightly integrating them to form a high‑performance, in‑memory RDBMS that supports both analytical and transactional workloads.

Original source: SAP HANA Blog