Redis Performance Secrets: How Data Types and Internal Encodings Boost Speed

Data Types vs. Encodings

Redis exposes five public data types (String, List, Hash, Set, ZSet). Each type can be backed by one or more internal encodings. The type is the interface you use; the encoding is the hidden implementation that Redis switches automatically based on size thresholds.

String encoding

Strings are stored as SDS (Simple Dynamic String). Three possible encodings are: int – pure integer values, minimal memory, fast increment. embstr – short strings (≈44 bytes) stored together with the object header. raw – long or modified strings, separate allocation.

Example:

127.0.0.1:6379> SET n 12345
OK
127.0.0.1:6379> OBJECT ENCODING n
"int"
127.0.0.1:6379> SET s "hello"
OK
127.0.0.1:6379> OBJECT ENCODING s
"embstr"
127.0.0.1:6379> SET s2 "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"
OK
127.0.0.1:6379> OBJECT ENCODING s2
"raw"

List encoding (quicklist)

Since Redis 3.2, List is implemented as a quicklist: a doubly‑linked list whose nodes each contain a listpack (compact contiguous memory). This gives O(1) head/tail operations and low memory overhead.

Example:

127.0.0.1:6379> RPUSH queue a b c
(integer) 3
127.0.0.1:6379> OBJECT ENCODING queue
"quicklist"

Hash encoding

Small hashes use listpack. When the number of fields exceeds hash-max-listpack-entries (default 512) or any field/value exceeds hash-max-listpack-value (default 64 bytes), Redis upgrades the hash to a hashtable.

Example that triggers an upgrade:

127.0.0.1:6379> HSET big field1 "short"
(integer) 1
# add many fields …
127.0.0.1:6379> OBJECT ENCODING big
"hashtable"

Set encoding

intset

– all members are integers and count ≤ set-max-intset-entries (default 512). listpack – small non‑integer sets (Redis 7.2+). hashtable – large or mixed sets.

Example:

127.0.0.1:6379> SADD ids 1 2 3
(integer) 3
127.0.0.1:6379> OBJECT ENCODING ids
"intset"
127.0.0.1:6379> SADD tags redis mysql
(integer) 2
127.0.0.1:6379> OBJECT ENCODING tags
"listpack"

ZSet encoding

Small sorted sets use listpack. When the number of members exceeds zset-max-listpack-entries (default 128) or a member exceeds zset-max-listpack-value (64 bytes), Redis switches to a skiplist plus a hashtable for O(log N) range queries and O(1) member look‑ups.

Example:

127.0.0.1:6379> ZADD rank 100 Tom 95 Jerry 88 Spike
(integer) 3
127.0.0.1:6379> OBJECT ENCODING rank
"listpack"

Automatic encoding switches

Redis decides to upgrade based on two criteria: element count and the size of a single element. The switch is one‑way; once upgraded, the structure never reverts even if elements are later removed, because the cost of switching back outweighs the benefit.

Observing the internal representation

TYPE key

– shows the public data type. OBJECT ENCODING key – shows the current internal encoding. MEMORY USAGE key – reports the approximate bytes used.

Typical usage:

127.0.0.1:6379> TYPE user:1
hash
127.0.0.1:6379> OBJECT ENCODING user:1
"listpack"
127.0.0.1:6379> MEMORY USAGE user:1
(integer) 120

Practical e‑commerce scenario recommendations

Product details (field‑wise updates) – use Hash (e.g., product:{skuId}) to store title, price, stock, status.

Whole product JSON – use String (e.g., product:json:{skuId}) when the object is read as a whole and rarely modified.

Page view counters – use String with INCR (e.g., product:pv:{skuId}).

Shopping cart per user – use Hash where field=skuId and value=quantity (e.g., cart:{userId}).

Recent‑view history – use List with LPUSH + LTRIM to keep a bounded list (e.g., history:{userId}).

User favorites – use Set for de‑duplication (e.g., fav:{userId}).

Leaderboards / sales ranking – use ZSet where score=metric (e.g., rank:sales:{categoryId}).

Common pitfalls and mitigation

Hash field explosion : when hash-max-listpack-entries or hash-max-listpack-value is exceeded, memory jumps because the hash upgrades to a hashtable. Split large hashes into multiple keys.

Big keys cause blocking : commands like HGETALL, LRANGE 0 -1, SMEMBERS become O(N) and can stall the single Redis thread. Use HSCAN, SSCAN, ZSCAN or the --bigkeys tool to locate and refactor them.

ZSet score precision : scores are stored as double; using raw timestamps larger than 2^53 loses precision. Keep scores within safe ranges or encode high‑precision data into the member string.

Conclusion

The key takeaway is that Redis data types are merely interfaces; the real performance and memory characteristics come from the underlying encodings. By understanding SDS, listpack, quicklist, hashtable, intset, and skiplist, and by monitoring OBJECT ENCODING and MEMORY USAGE, developers can make informed choices, avoid hidden memory bloat, and keep Redis fast at scale.