0 views collected around this technical thread.
This article explains how to efficiently remove duplicates from 4 billion QQ numbers within a 1 GB memory limit by replacing the naïve HashSet approach with a memory‑saving Bitmap data structure, complete with calculations, algorithm steps, Java code, and a discussion of its pros and cons.
This article explains three Redis-based methods—Hash, Bitmap, and HyperLogLog—for tracking website user visits, detailing how each structure works, their implementation steps, memory and accuracy trade‑offs, and guidance on choosing the best approach for different traffic scenarios.
The article explains why large keys, oversized values, massive key counts, and huge Bitmaps or Bloom filters degrade Redis performance and provides practical splitting techniques—using multiple keys, hashes, bucketed hashing, and bitmap partitioning—to reduce memory pressure and improve response times.
By integrating ClickHouse bitmap structures, a dictionary service for dense ID mapping, and Bit‑Slice Indexes, Bilibili’s Data Management Platform now supports flexible, multi‑dimensional audience selection and profiling over petabyte‑scale data with minute‑level latency, cutting storage by over twenty‑fold and query times from hours to seconds.
This article introduces Redis bitmap operations, explains key commands such as BITSET, BITGET, and BITCOUNT, and demonstrates with PHP code how to implement an efficient user sign‑in tracking and counting system using bitmaps for accurate statistics.
This article explains how ClickHouse’s columnar storage, bitmap functions, and distributed architecture can be leveraged to store billions of tag bitmaps, combine them efficiently, and support fast group calculations for customer data platforms, while addressing data‑warehouse integration, storage format, and performance challenges.
This article explains how a CDP system tackles the storage and set‑operation challenges of billions of user‑ID tags and groups by adopting bitmap techniques, especially RoaringBitmap, to dramatically reduce space usage and enable fast union, intersection, and difference calculations.
This article explains the Linux kernel bitmap data structure, its definition, macro-based declarations, related assembly instructions, core bitmap APIs such as set_bit and test_bit, various bit‑operation functions, and practical applications ranging from PID allocation to device management and file‑system indexing.
This article explains why storing daily sign‑in data in a traditional database can cause performance issues at scale and demonstrates how Redis bitmap, a memory‑efficient bit‑level structure, can record and compute continuous user sign‑ins both by date and by user.
This article explains Redis bitmap data structures, introduces the main bitmap commands (BITSET, BITGET, BITCOUNT), and provides a complete PHP example that demonstrates how to record user sign‑ins and count them efficiently using Redis bit operations.
The article explains how to store and deduplicate billions of identifiers by using a bitmap to represent presence with a single bit per value, calculates memory requirements, shows Redis bitmap commands, and introduces Bloom filters as an extension with multiple hash functions for efficient large‑scale data handling.
This article walks through the step‑by‑step design of a simple file system on a 1 TB disk, introducing logical blocks, block bitmaps, inode structures, directory handling, indirect indexing, superblocks, and hierarchical file organization, while comparing the design to the classic ext2 filesystem.
The article explains how to store and deduplicate billions of identifiers efficiently by using a bitmap backed by Redis and extending it with a Bloom filter implementation in Java, highlighting memory calculations, practical commands, and code examples.
To achieve sub‑second, multi‑dimensional analytics on Bilibili’s hundred‑million‑row datasets, the team built a ClickHouse‑based acceleration layer that combines grouping‑set pre‑aggregation, bitmap (RBM) distinct handling, and a dense dictionary encoding service, dramatically cutting CPU, memory and query latency versus traditional OLAP pipelines.
The article explains BitMap fundamentals and introduces Roaring BitMap’s compressed container architecture—Array, BitMap, and Run containers—detailing their conversion logic, Java implementation snippets, performance advantages over traditional BitSets, and practical API usage for high‑performance, memory‑efficient big‑data applications.
This article details how the WeChat team leverages ClickHouse at massive scale, introduces a suite of performance observation tools, describes lakehouse reading and bitmap optimizations, and explains the integration of AI workloads, demonstrating overall query speedups of up to tenfold across diverse scenarios.
This article explains Redis bitmap data structures, introduces common bitmap commands such as BITSET, BITGET, and BITCOUNT, and provides a complete PHP example that records user sign‑ins and calculates daily sign‑in counts using Redis bit operations.
This article introduces Redis's five extended data types—Stream, Geospatial, Bitmap, Bitfield, and HyperLogLog—explaining their concepts, common commands, and practical code examples, and highlights how they address specific internet‑scale scenarios more efficiently than traditional relational solutions.
This article examines how bitmap data structures and multithreading can dramatically accelerate blacklist filtering in large‑scale marketing systems, reducing processing time from tens of minutes to milliseconds while saving memory and improving overall system performance.
Discover ten practical Redis scenarios—from counting page visits and caching category trees to implementing distributed locks, leaderboards, rate limiting, bitmap analytics, message queues, and global ID generation—each illustrated with concise code snippets and best‑practice tips for robust backend development.