Why Aerospike’s Hybrid SSD Architecture Beats Traditional NoSQL for Real‑Time, PB‑Scale Workloads

Many people ask about cache or persistence NoSQL database selection; here is Aerospike, a hybrid‑SSD architecture that delivers micro‑second latency, petabyte‑scale capacity, and distributed real‑time performance.

About Aerospike

Company

Founded: 2009 in Silicon Valley, USA

Customers: over 200 enterprise users

Advantages

Hybrid storage architecture: keys indexed in memory, values stored on SSDs

Scalable and stable as a distributed database for massive data

High performance with strong consistency

Significantly lower cost compared with pure‑memory Redis solutions

Reduced operational cost: easy maintenance and automatic data rebalancing

Aerospike Architecture

1) Shared‑Nothing Architecture

No master/slave distinction among cluster nodes

Each node operates independently, handling its own data

Heartbeat and new‑node detection via multicast or TCP/IP

2) Hybrid Storage Mode

Indexes reside in memory while actual data is on SSD

SSD access bypasses the OS file system, using block‑device reads/writes to avoid extra I/O overhead

Distributed hash partitioning uses RIPEMD‑160 to balance data across nodes and flash devices

Rich data types: Integer, Double, String, Boolean, Blob, List, Map, etc.

3) Smart Client

Supports multiple client languages (Java, C, Go, C#, Python, Rust, Ruby, PHP…)

Data access requires only a single hop

Built‑in TCP/IP connection pool transparently redirects requests when a node fails

4) Multi‑Data‑Center Deployment

Deploy on bare metal or public/private clouds

Synchronous clusters: Aerospike 5 enables multi‑site clusters for 2‑site‑3‑center deployments

Asynchronous clusters: XDR configuration provides active‑passive deployment across city‑level data centers, e.g., one primary and one standby in Beijing

Aerospike Design Optimizations

Why can Aerospike match the performance of an in‑memory database like Redis?

High concurrency with NUMA support

CPU execution and storage engine are highly concurrent; threads are grouped by CPU socket rather than core, aligning with NUMA nodes

Transaction threads are bound to specific I/O devices; network and disk I/O handling are also bound to the same cores, reducing cross‑NUMA data sharing and latency

Reduced thread‑context switching

Some operations run directly in the network listener thread; the system creates as many listener threads as CPU cores, enabling non‑blocking, short, predictable real‑time responses without yielding the CPU

Memory allocation optimizations

Custom allocators independent of language runtimes allocate index memory efficiently

Extended jemalloc from the standard C library allocates memory based on object categories

Objects are grouped by namespace to optimize creation, access, modification, and deletion, minimizing fragmentation

Carefully designed data structures

Partitioned single‑threaded structures use fine‑grained locks per partition, reducing cross‑partition contention and boosting atomic operation performance

Nested structures like index trees have per‑element reference counts and locks, allowing safe concurrent reads/writes without multiple lock acquisitions

Hot data is kept within a single cache line (e.g., Aerospike index entries are 64 bytes, matching X86‑64 cache line size) to improve locality and reduce cache misses

Scheduling and priority optimizations

Thread pools and priorities are assigned based on job type; jobs are broken into work units whose generation rate is controlled by load generators

Cooperative scheduling releases CPU resources after a set of work units, preserving core‑partition affinity and avoiding data contention

Concurrent workloads run first‑come‑first‑served; long‑running tasks like scans are dynamically shifted to round‑robin scheduling for fair resource distribution

Optimized lock‑free cluster management

Network event detection merges multiple changes into a single event, handling rapid network changes with fewer state transitions

Only data related to partitions residing on a specific node is transferred

Client optimizations

Server and client share partition maps, enabling single‑hop data access

Clients dynamically discover current partition state, routing transactions to the correct node even after node additions

High‑concurrency threads match the execution engine’s concurrency level

Supports both synchronous and asynchronous execution

Application Scenarios

Advertising use case: real‑time bidding and personalized ad recommendation. User behavior data (tags, clicks, searches) is stored in Aerospike, enabling sub‑millisecond query responses.

Current key‑value volume exceeds 5 billion records, average record size ~1 KB, with transaction latency required under 1 ms and end‑to‑end processing under 50 ms, handling tens to hundreds of terabytes while meeting low‑latency, high‑throughput demands.

Conclusion

Overall, when the cost of a hundred‑terabyte in‑memory database becomes prohibitive, Aerospike offers a viable NoSQL distributed solution. Interested readers can follow the public account for trial information.