Mastering Redis: Architecture, Use Cases, Testing Pitfalls & Solutions

1. Quick Introduction to Redis

Redis (Remote Dictionary Server) is a cross‑platform, in‑memory key‑value store that functions as a NoSQL database. It is widely adopted for its high performance, reliability, and scalability.

2. Redis Architecture Design

The architecture is built to satisfy high performance, high reliability, and easy scalability, making it suitable for large‑scale internet services.

3. Common Redis Data Structures

Redis provides five primary data structures that are frequently used:

String

List

Set

Sorted Set (ZSet)

Hash

4. Redis Usage Scenarios

4.1 Caching

Example: Snowball app stores SMS verification codes in Redis with a 5‑minute TTL.

public void setAndExpireSmsVerificationCache(UcTelephone ucTelephone, SmsVerification smsVerification, int expireAt) {
    String cacheKey = SmsCacheKeyEmum.SMS_VERIFICATION.key(ucTelephone.areaCode(), ucTelephone.number());
    usercenterV4.setAndExpire(cacheKey, JsonUtils.write(smsVerification), expireAt);
}

Redis CLI commands to check TTL and retrieve the value:

redis-cli -h ${host} -p ${port} ttl sms:verification:86:${tel}
300
redis-cli -h ${host} -p ${port} get sms:verification:86:${tel}
{"code":"2294","createdAt":"Jun 14, 2022 10:45:28 AM","times":1,"verifyTimes":0,"status":false}

4.2 Counters

Scenario: Limiting a user to follow at most 100 other users per day.

redis-cli -h ${host} -p ${port} get DAILYFUSER_2023-02-07_uid1
100

Increment operation using the incr() interface:

Long followerNum = JedisCluster.master().incr(RelationKeys.createrFollowDailyKey(follower.getId()));

4.3 Distributed Locks

Problem: Concurrent creation of live‑stream rooms could cause duplicate rooms.

Solution: Use a Redis distributed lock implemented with the SET key value EX seconds NX command (wrapped in a setexnx() method). The lock is released in a finally block, and an expiration (e.g., 60 s) prevents deadlocks if the process crashes.

5. Redis Testing Cases

5.1 User Group Query Interface Upgrade

Background: Migration from an RPC interface to a gRPC interface required identical business logic.

Issue: After the upgrade, cached group data became invalid because the score stored in a sorted set changed from a negative timestamp to a positive timestamp.

# Before upgrade (score = -timestamp)
redis-cli -h ${host} -p ${port} zrange MEMBER_MEMBERLIST_GROUPID_3615231762_14257441 0 -1 withscores
9485866208 -1669558282413
7371699344 -1663674830000
# After upgrade (score = timestamp)
redis-cli -h ${host} -p ${port} zrange MEMBER_MEMBERLIST_GROUPID_3615231762_14257441 0 -1 withscores
7371699344 1663674830000
9485866208 1669558282413

Resolution: Restore the negative‑timestamp logic to keep the score consistent with the original implementation.

5.2 Stock‑Page New‑Post Stream Not Updating

Problem: A network glitch prevented the TTL from being set, causing the cache to never expire and the post stream to stay stale.

Solutions:

During code review, ensure atomic operations for setting a value and its expiration (e.g., use SETEX for strings).

Implement fallback mechanisms that detect missing TTLs and reset them, and add monitoring for cache expiration failures.

5.3 Large‑Key Optimization for Tick‑Level Market Data

Test focus:

Transition from single‑level keys to two‑level keys with renamed patterns.

Cover cross‑key read/write scenarios.

Validate data accuracy for high‑volume tick data.

Key coverage includes pre‑market ( trade:ext_pre:{symbol}), intraday ( trade:{symbol}), and post‑market ( trade:ext_after:{symbol}) keys.

#!/bin/bash
key=$1
sum=0
levelkey="shard_${key}"
# Query old key count
old=$(sh connect_redis.sh zcard ${key} | awk '$1>0{print $1}')
# Query new key counts and sum them
list=$(sh connect_redis.sh zrange ${levelkey} 0 -1 | grep trade | grep -v zrange)
for var in ${list}; do
  i=$(sh connect_redis.sh zcard ${var} | awk '$1>0{print $1}')
  sum=$((sum + i))
 done
if [ -z "${old}" ]; then old=0; fi
if [ -z "${sum}" ]; then sum=0; fi
echo "Original total" ${old}
echo "New total" ${sum}
if [ ${old} -eq ${sum} ]; then
  echo "Data totals match"
else
  echo "Data totals mismatch"
fi

Risk assessment includes ensuring no other services still use the old keys (code review + business‑level test coverage) and minimizing rollout risk by gradual traffic increase and feature‑flag switches.

6. Common Redis Problems

6.1 Cache Penetration

When requests query keys that do not exist in both cache and database, the database can be overwhelmed.

Mitigations:

Validate request parameters and reject illegal inputs.

Cache empty results with a short TTL.

Use a Bloom filter to pre‑filter nonexistent keys.

6.2 Cache Breakdown (Cache Stampede)

Hot data expiring simultaneously can cause a surge of database traffic.

Solutions:

Apply a mutex lock so only the first request repopulates the cache.

For extremely hot data, either avoid expiration or proactively refresh before expiry.

6.3 Cache Avalanche

If many keys share the same TTL, they may expire together, leading to massive database load.

Countermeasures:

Stagger expiration times.

Use mutex locks.

For popular topics, set no expiration or extend TTL before it expires.

7. Summary

Redis’s high performance, reliability, and scalability make it a popular choice for caching and as a primary data store. However, misuse can cause issues such as cache penetration, breakdown, and avalanche. This article covered Redis’s architecture, data structures, typical use cases, QA‑driven testing experiences, and concrete solutions to common pitfalls, providing practical guidance for engineers.