Master Redis Set Operations for Scalable Statistics and Analytics

Preface

The article discusses using Redis sets for large‑scale statistical scenarios such as daily sign‑in records, product comment lists, and friend relationships.

Aggregate Statistics

Aggregating multiple sets—intersection, union, and difference—helps solve problems like counting common friends, latest comments, or monthly sign‑ins.

Intersection

To count common friends between two users, use SINTERSTORE:

SINTERSTORE userid:new userid:20002 userid:20003

After execution, the key userid:new stores the intersection of the two user sets.

Difference

To find newly added friends on a specific day, use SDIFFSTORE:

SDIFFSTORE  user:new  userid:20201102 userid:20201101

The resulting set contains the friends added on 2020‑11‑02.

Union

To obtain the total new friends over two days, use SUNIONSTORE:

SUNIONSTORE  userid:new userid:20201102 userid:20201101

The key userid:new now holds the union of both days' friend sets.

Sorted Statistics

Redis provides ordered collections: List preserves insertion order, while Sorted Set orders elements by a score (e.g., timestamp). Use LRANGE for lists and ZRANGEBYSCORE for sorted sets.

Binary State Statistics

Bitmaps (implemented as strings) efficiently store 0/1 states such as sign‑in flags. Commands GETBIT, SETBIT, and BITCOUNT manipulate bits.

Example of setting a sign‑in flag: SETBIT userid:10001:202011 1 1 Checking the flag and counting total sign‑ins:

GETBIT userid:10001:202011 1

BITCOUNT userid:10001:202011

Bitwise AND across daily bitmaps can identify users who signed in for consecutive days.

Cardinality Statistics

HyperLogLog offers approximate distinct counting with about 0.81 % error while using only ~12 KB of memory, suitable for massive UV counting. Use PFADD to add IDs and PFCOUNT to retrieve the estimate. For exact counts, a Set is required.

PFADD p1:uv 10001 10002 10003 10004

PFCOUNT p1:uv

Conclusion

Sets, Sorted Sets, Lists, Bitmaps, and HyperLogLog each have strengths and limitations for aggregation, ordered queries, binary state tracking, and cardinality estimation. Choose the appropriate Redis data type based on the specific statistical requirement.