Tagged articles
Data Sharding
17 articles · Page 1 of 1
This article details Spring Boot implementation of unit-based multi-active architecture, covering traffic routing with gateway filters and consistent hashing, data sharding via dynamic data sources and ShardingSphere, cross-unit synchronous/async calls with circuit breaking, data synchronization with conflict resolution, and failover switching with three-layer controls and graceful shutdown.
Circuit BreakerConsistent HashingCross-Unit Calls
0 likes · 43 min read

This article explains essential data sharding techniques for large-scale architectures, emphasizing the priority of vertical splitting by business modules, followed by horizontal partitioning of hot tables, and outlines stable sharding rules, hotspot avoidance, and minimizing cross‑shard transactions to ensure optimal performance.
Data ShardingHorizontal Partitioningdatabase design
0 likes · 4 min read

This article analyzes the essential requirements of massive‑scale game leaderboards, explains how Redis sorted sets and hash tables provide fast ranking and lookup, and presents a multi‑layered architecture—including hot‑key sharding, dynamic partitioning, tiered storage, read/write separation, pipeline batching, and hybrid persistence—to achieve real‑time, billion‑user performance.
Data ShardingLeaderboardRedis
0 likes · 5 min read

This article explains Redis cluster hash slots, how they partition data across 16,384 slots, the CRC16-based slot calculation, slot allocation to nodes, and demonstrates dynamic scaling operations such as adding, reshaping, and removing nodes with practical command‑line examples.
ClusterData ShardingDynamic Scaling
0 likes · 9 min read

Redis Cluster provides a scalable, fault‑tolerant distributed Redis solution, explaining why it’s needed, its architecture, virtual slot partitioning, data distribution methods, limitations, smart client optimization, and automatic failover mechanisms, while highlighting key operational considerations for high‑performance deployments.
ClusterData ShardingDistributed Database
0 likes · 11 min read

In an era of exploding data volumes, distributed architecture offers unparalleled scalability, fault tolerance, and parallel performance through sharding, replication, batch and stream processing, with real‑world examples from e‑commerce and social media giants illustrating its practical impact.
Data ShardingParallel Processingbig data
0 likes · 12 min read

This article walks through building a database storage engine from a simple shell script to a full distributed key‑value system, covering in‑memory indexing, SSTable creation, LSM‑Tree architecture with compaction, replication strategies, and sharding techniques for scaling across multiple machines.
B+TreeData ShardingDistributed Database
0 likes · 38 min read

This article presents a comprehensive engineering guide for importing one billion 1 KB unstructured log records stored in HDFS or S3 into MySQL, covering data sizing, B‑tree limits, batch insertion strategies, storage‑engine choices, sharding, file‑reading techniques, concurrency control, and reliable task coordination using Redis, Redisson, and Zookeeper.
Data ShardingMySQLRedis
0 likes · 17 min read

This article outlines a comprehensive strategy for rapidly loading one billion 1 KB log records stored in HDFS or S3 into MySQL, covering data partitioning, B‑tree limits, batch inserts, storage‑engine selection, sharding, concurrent write control, efficient file reading, task coordination, fault‑tolerant progress tracking with Redis, and distributed locking with Redisson.
Data ShardingDistributed LockFile Reading
0 likes · 20 min read

This article explores the comprehensive design of order business systems, covering background considerations, order lifecycle management, technical solutions such as ID generation, parallel and asynchronous processing, timeout handling, distributed transactions, and data strategies like sharding and synchronization.
Data Shardingbackend architecturedistributed transactions
0 likes · 12 min read

MPP (Massive Parallel Processing) databases, designed for large‑scale analytical workloads, use distributed, shared‑nothing architectures with multiple control and compute nodes, offering high scalability, diverse data‑sharding strategies, and powerful SQL compatibility, as illustrated by vendors like Teradata, Vertica, Greenplum, and emerging open‑source solutions.
Data ShardingGreenplumMPP
0 likes · 15 min read


21CTONov 30, 2022 · Big Data
This article explains core data sharding concepts and models—including hash‑based, range‑based, and consistent hashing—detailing their mappings, routing strategies, scalability considerations, and practical implementation examples for handling massive datasets in distributed systems.
Consistent HashingData Shardingbig data
0 likes · 11 min read

The article explains why large financial institutions need a unitized (Set‑based) architecture to improve resource utilization, achieve city‑level disaster recovery, and support massive user traffic, then defines the concept and outlines a four‑step process for building such a system.
Data Shardingdisaster recoveryfinancial IT
0 likes · 8 min read

To support the rapidly growing DeWu App, the team created Rainbow Bridge—a ShardingSphere‑based middleware that unifies horizontal sharding, read‑write separation, shadow‑DB routing, rate limiting, and active‑active traffic correction, while adding a graphical console, stability enhancements, fine‑grained observability, hybrid JDBC/Proxy switching, and ongoing open‑source contributions.
Data ShardingDatabase MiddlewareShardingSphere
0 likes · 18 min read

This article introduces MyBatis‑Mate, an official MyBatis‑Plus extension that provides enterprise‑grade features such as sharding, data auditing, field encryption, sensitive data masking, dictionary binding, dynamic DDL maintenance, and flexible data‑source switching, complete with usage examples and code snippets.
Data AuditingData ShardingField Encryption
0 likes · 10 min read

This article introduces Apache ShardingSphere’s ecosystem, core sharding and governance capabilities, access endpoints such as Sharding-JDBC, Sharding-Proxy and Sharding-Sidecar, and details JD.com’s real‑world implementation, including data‑sharding strategies, distributed primary keys, hint routing, performance optimizations, and future roadmap.
Apache ShardingSphereData ShardingDatabase Middleware
0 likes · 18 min read

This article provides a comprehensive overview of distributed database systems, covering their evolution from centralized architectures, classifications, key characteristics, advantages and disadvantages, data sharding methods, allocation strategies, architectural models, and management considerations, highlighting the benefits and challenges of distributed data management.
DDBMSData ShardingDatabase Architecture
0 likes · 9 min read
