How Instagram Optimized Redis Memory with Hash Ziplist Settings
Redis’s hash‑max‑ziplist‑entries and hash‑max‑ziplist‑value settings control when hash tables switch to compressed ziplist storage, and by segmenting data into 1,000‑item hash tables, Instagram reduced memory usage from ~20 GB to about 5 GB, dramatically improving efficiency.
Redis configuration includes two parameters: hash-max-ziplist-entries and hash-max-ziplist-value . The term ziplist refers to a compressed data structure that reduces memory usage.
When the number of entries in a hash table is below the hash-max-ziplist-entries threshold (default 512) and each key/value size is under hash-max-ziplist-value bytes (default 64), Redis stores the hash as a ziplist; exceeding either limit switches to a regular hashmap.
Instagram, a large‑scale image service, needed to map image IDs to author IDs. Storing 300 million mappings as simple string key/value pairs would require roughly 20 GB of memory. After consulting a Redis developer, they adopted a segmented hash approach.
They divided data into segments of 1,000 entries per hash table. For an image ID like 1234888, the hash key becomes 1234 and the field 888 stores the author ID. Testing showed that with the entries threshold set to 1,000, memory consumption dropped to about 5 GB with acceptable CPU load.
Similar configuration options exist for other data types, such as list-max-ziplist and zset-max-ziplist .
Signed-in readers can open the original source through BestHub's protected redirect.
This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactand we will review it promptly.
Java High-Performance Architecture
Sharing Java development articles and resources, including SSM architecture and the Spring ecosystem (Spring Boot, Spring Cloud, MyBatis, Dubbo, Docker), Zookeeper, Redis, architecture design, microservices, message queues, Git, etc.
How this landed with the community
Was this worth your time?
0 Comments
Thoughtful readers leave field notes, pushback, and hard-won operational detail here.