Redis vs Dragonfly: Who Wins the High‑Performance Memory Cache Battle?

In mid‑2023 a former Google and Amazon engineer released the open‑source memory data cache Dragonfly, written in C/C++ and distributed under the Business Source License (BSL). It claims to be the world’s fastest memory storage system, supporting both Memcached and Redis protocols while offering higher query performance and lower runtime memory consumption.

Benchmark data suggest that Dragonfly can achieve up to 25× performance gains over Redis under typical workloads; a single Dragonfly server can process millions of requests per second, and in a 5 GB storage test it uses 30 % less memory than Redis. Within two months the project attracted 9.2 K GitHub stars and 177 forks.

In response, Redis co‑founder and CTO Yiftach Shoolman, together with Redis Labs architects, published an article titled “After 13 years, does Redis need a new architecture?” presenting Redis 7.0 versus Dragonfly benchmark results. Their tests show Redis throughput 18 %–40 % higher than Dragonfly when comparing a 40‑shard Redis 7.0 cluster (using 40 of 64 vCores) against the largest Dragonfly instance used in the original benchmark.

“It absolutely represents how ordinary users run Redis in the real world. Running a cluster on a single machine just to use more than one core adds unnecessary complexity; if a competitor works out‑of‑the‑box on any number of cores, it’s a win.”

Redis acknowledges that while its architecture remains the best for real‑time in‑memory data storage, it also appreciates the fresh ideas introduced by Dragonfly, such as exploring io_uring, modern dictionaries, and more strategic thread usage, some of which may be incorporated in future releases.

Architectural Differences

Background

Multi‑threaded projects typically base architectural decisions on past pain points. Redis was originally designed as a single‑process, single‑core system, but horizontal scaling is achieved by running multiple processes (Redis Cluster) or using Redis Enterprise for large‑scale deployments.

Design Principles

Run multiple Redis instances on each virtual machine

Provides a completely shared‑nothing architecture for linear vertical and horizontal scaling, offering greater flexibility than pure vertical multi‑threaded designs.

Improves replication speed by allowing concurrent replication across processes.

Enables rapid recovery from VM failures because new instances can populate data from multiple external Redis instances.

Limit each Redis process to a reasonable size

Keeping processes under 25 GB (50 GB for Redis on Flash) reduces memory overhead during fork‑based operations such as snapshots and AOF rewrites.

Smaller instances speed up shard migration, rebalancing, and scaling operations.

Horizontal scaling is paramount

More nodes increase cluster resilience; a failure in a three‑node cluster disables one‑third of the service, whereas a nine‑node cluster loses only one‑ninth.

Adding a node and migrating data is far simpler than scaling vertically by replacing a single large node.

Horizontal scaling offers better cost efficiency, especially in cloud environments where vertical scaling can be expensive.

It aligns well with NUMA architectures, avoiding the performance penalties that multi‑threaded designs can incur.

It circumvents storage throughput limits of external disks, ensuring the memory‑centric workload remains unimpeded.

Conclusion

Redis continues to value community‑driven innovations while maintaining its core principles of a shared‑nothing, multi‑process architecture that delivers optimal performance, scalability, and resilience for real‑time in‑memory data platforms.

Appendix: Redis 7.0 vs Dragonfly benchmark details

Result Overview

Version

Redis 7.0.0 built from source.

Dragonfly built from the June 3 source snapshot (hash e806e6c).

Goal

Validate Dragonfly’s published results and reproduce them.

Determine the best achievable OSS Redis 7.0 performance on AWS c6gn.16xlarge and compare it with Dragonfly.

Client Configuration

OSS Redis requires many open connections to the cluster because each memtier_benchmark thread connects to all shards.

Best results were obtained with two memtier_benchmark processes running on the same client VM as the Dragonfly benchmark.

Resource Utilization

Redis performed best with 40 primary shards (24 spare vCPUs) but did not benefit from adding more shards.

Dragonfly fully saturated all 64 vCPUs on the instance.

Benchmark Commands

Redis GET:

memtier_benchmark –ratio 0:1 -t 24 -c 1 –test-time 180 –distinct-client-seed -d 256 –cluster-mode -s 10.3.1.88 –port 30001 –key-maximum 1000000 –hide-histogram

Dragonfly GET:

memtier_benchmark –ratio 0:1 -t 55 -c 30 -n 200000 –distinct-client-seed -d 256 -s 10.3.1.6 –key-maximum 1000000 –hide-histogram

Test Facility Details

VM type: AWS c6gn.16xlarge (aarch64, Arm Neoverse‑N1, 64 cores, 126 GB RAM, 1 NUMA node, Kernel 5.10).