Dragonfly vs Redis: Does the New Cache System Really Outperform?

Dragonfly: A New Open‑Source In‑Memory Cache

Mid‑year, a former Google and Amazon engineer released Dragonfly, an open‑source memory data cache written in C/C++ and distributed under the Business Source License (BSL). Benchmarks suggest Dragonfly may be the fastest memory storage system, supporting Memcached and Redis protocols with higher query performance and lower runtime memory usage.

Compared to Redis, Dragonfly claims a 25× performance boost under typical workloads, handling millions of requests per second on a single server and using 30% less memory in a 5 GB storage test. In just two months, Dragonfly earned 9.2 K GitHub stars and 177 forks.

Other Redis‑compatible in‑memory stores like KeyDB and Skytable have emerged, but none have generated as much buzz because Dragonfly was designed from scratch, shedding legacy constraints.

Redis Responds

To counter Dragonfly, Redis co‑founder and CTO Yiftach Shoolman, along with Redis Labs architects, published an article titled “After 13 Years, Does Redis Need a New Architecture?” presenting Redis 7.0 vs Dragonfly benchmark results, showing Redis throughput 18%–40% higher.

Redis acknowledges Dragonfly’s innovative ideas (e.g., io_uring, modern dictionaries, thread strategies) and notes some may be incorporated in future releases. However, Redis argues the benchmark methodology does not reflect real‑world usage.

It absolutely represents how ordinary users run Redis in the real world. Running a cluster on a single machine just to use more than one core adds unnecessary complexity; if a competitor works “just works” on any core count, it’s preferable.

Redis invested heavily in this article and benchmark, which is a huge compliment to Dragonfly. I’m glad Redis wrote this article, so I had to check out Dragonfly—it looks great.

Speed Considerations

Dragonfly’s benchmarks compare a single‑process Redis instance (single core) with a multithreaded Dragonfly instance (all available cores). This crude comparison doesn’t reflect Redis’s real‑world behavior. A fairer test compares a 40‑shard Redis 7.0 cluster with Dragonfly’s largest benchmark instance on an AWS c4gn.16xlarge server.

In this adjusted test, Redis throughput exceeds Dragonfly by 18%–40% while using only 40 of the 64 vCores.

Architectural Differences

Background

Multithreaded projects often base architecture decisions on past pain points. While a single Redis process on many cores can’t fully utilize resources, Redis was not originally designed for that, and many providers have added solutions.

Redis achieves horizontal scaling by running multiple processes (Redis Cluster), even within a single cloud instance.

Design Principles

Run multiple Redis instances on each VM

This approach provides:

No‑shared architecture for linear vertical and horizontal scaling.

Faster replication through concurrent processes.

Quick recovery from VM failures.

Limit each Redis process to a reasonable size

Processes are capped at 25 GB (50 GB for Redis on Flash) to reduce memory overhead during replication, snapshots, and AOF rewrites, and to keep instances small for faster rebalancing.

Horizontal scaling is paramount

Benefits include better resilience, easier expansion, cost‑effectiveness, higher throughput, alignment with NUMA architectures, and avoidance of storage‑throughput limits.

Conclusion

We appreciate community‑driven ideas like Dragonfly. Some innovations may be adopted by Redis (e.g., io_uring, modern dictionaries, advanced threading). However, Redis will continue to uphold its core principles of no‑shared, multi‑process architecture, which deliver optimal performance, scalability, and flexibility for real‑time in‑memory data platforms.

Appendix: Redis 7.0 vs Dragonfly Benchmark Details

Result Overview

Versions:

Redis 7.0.0 built from source.

Dragonfly built from the June 3 source (hash e806e6c…).

Goals:

Validate Dragonfly’s published results and reproduce them.

Determine the best OSS Redis 7.0.0 cluster performance on AWS c6gn.16xlarge and compare with Dragonfly.

Client Configuration:

OSS Redis 7.0 requires many open connections for cluster access.

Best results achieved with two memtier_benchmark processes on the same client VM.

Resource Utilization:

Redis cluster performs best with 40 primary shards (24 vCPU available).

Dragonfly fully saturates all 64 vCPU.

Our tests reproduced most of Dragonfly’s numbers and even exceeded them under 30‑channel conditions.

Both Redis and Dragonfly are not limited by network packets per second or bandwidth.

Benchmark Commands

Single GET channel (latency < 1 ms):

Redis: memtier_benchmark –ratio 0:1 -t 24 -c 1 –test-time 180 –distinct-client-seed -d 256 –cluster-mode -s 10.3.1.88 –port 30001 –key-maximum 1000000 –hide-histogram

Dragonfly: memtier_benchmark –ratio 0:1 -t 55 -c 30 -n 200000 –distinct-client-seed -d 256 -s 10.3.1.6 –key-maximum 1000000 –hide-histogram

30 GET channels:

Redis: memtier_benchmark –ratio 0:1 -t 24 -c 1 –test-time 180 –distinct-client-seed -d 256 –cluster-mode -s 10.3.1.88 –port 30001 –key-maximum 1000000 –hide-histogram –pipeline 30

Dragonfly: memtier_benchmark –ratio 0:1 -t 55 -c 30 -n 200000 –distinct-client-seed -d 256 -s 10.3.1.6 –key-maximum 1000000 –hide-histogram –pipeline 30

Single SET channel (latency < 1 ms):

Redis: memtier_benchmark –ratio 1:0 -t 24 -c 1 –test-time 180 –distinct-client-seed -d 256 –cluster-mode -s 10.3.1.88 –port 30001 –key-maximum 1000000 –hide-histogram

Dragonfly: memtier_benchmark –ratio 1:0 -t 55 -c 30 -n 200000 –distinct-client-seed -d 256 -s 10.3.1.6 –key-maximum 1000000 –hide-histogram

30 SET channels:

Redis: memtier_benchmark –ratio 1:0 -t 24 -c 1 –test-time 180 –distinct-client-seed -d 256 –cluster-mode -s 10.3.1.88 –port 30001 –key-maximum 1000000 –hide-histogram –pipeline 30

Dragonfly: memtier_benchmark –ratio 1:0 -t 55 -c 30 -n 200000 –distinct-client-seed -d 256 -s 10.3.1.6 –key-maximum 1000000 –hide-histogram –pipeline 30

Test Infrastructure Details

Both client and server used identical VMs:

VM: AWS c6gn.16xlarge

Architecture: aarch64, ARM Neoverse‑N1

Sockets: 1, Cores per socket: 64, Threads per core: 1

NUMA nodes: 1

Kernel: Arm64 5.10

Memory: 126 GB

Reference Links:

https://redis.com/blog/redis-architecture-13-years-later/

https://www.reddit.com/r/programming/comments/wiztpx/redis_hits_back_at_dragonfly/