BestHub
Sign in
Tag

Data Sharding

1 views collected around this technical thread.

Cognitive Technology Team
Cognitive Technology Team
Mar 26, 2025 · Game Development

Designing Scalable Game Leaderboards with Redis: Core Requirements, Data Structures, and Architecture

This article analyzes the essential requirements of massive‑scale game leaderboards, explains how Redis sorted sets and hash tables provide fast ranking and lookup, and presents a multi‑layered architecture—including hot‑key sharding, dynamic partitioning, tiered storage, read/write separation, pipeline batching, and hybrid persistence—to achieve real‑time, billion‑user performance.

Data ShardingGame DevelopmentPersistence
0 likes · 5 min read
Designing Scalable Game Leaderboards with Redis: Core Requirements, Data Structures, and Architecture
Full-Stack Internet Architecture
Full-Stack Internet Architecture
Mar 17, 2025 · Databases

Understanding Redis Cluster Hash Slots: Concepts, Calculation, Allocation, and Dynamic Scaling

This article explains Redis cluster hash slots, how they partition data across 16,384 slots, the CRC16-based slot calculation, slot allocation to nodes, and demonstrates dynamic scaling operations such as adding, reshaping, and removing nodes with practical command‑line examples.

ClusterData ShardingDynamic Scaling
0 likes · 9 min read
Understanding Redis Cluster Hash Slots: Concepts, Calculation, Allocation, and Dynamic Scaling
Tencent Cloud Developer
Tencent Cloud Developer
Dec 4, 2024 · Databases

Building a Distributed Database Storage Engine: From LSM Tree to Data Sharding

This article walks through building a database storage engine from a simple shell script to a full distributed key‑value system, covering in‑memory indexing, SSTable creation, LSM‑Tree architecture with compaction, replication strategies, and sharding techniques for scaling across multiple machines.

B+ TreeData ShardingDistributed Database
0 likes · 38 min read
Building a Distributed Database Storage Engine: From LSM Tree to Data Sharding
Architect's Guide
Architect's Guide
Oct 31, 2024 · Databases

Designing an Efficient Pipeline for Importing One Billion Records into MySQL

This article presents a comprehensive engineering guide for importing one billion 1 KB unstructured log records stored in HDFS or S3 into MySQL, covering data sizing, B‑tree limits, batch insertion strategies, storage‑engine choices, sharding, file‑reading techniques, concurrency control, and reliable task coordination using Redis, Redisson, and Zookeeper.

Data ShardingMySQLPerformance Optimization
0 likes · 17 min read
Designing an Efficient Pipeline for Importing One Billion Records into MySQL
macrozheng
macrozheng
Apr 7, 2023 · Backend Development

Designing Scalable Order Systems: Architecture, ID Strategies, and Distributed Transactions

This article explores the comprehensive design of order business systems, covering background considerations, order lifecycle management, technical solutions such as ID generation, parallel and asynchronous processing, timeout handling, distributed transactions, and data strategies like sharding and synchronization.

Data ShardingID generationbackend architecture
0 likes · 12 min read
Designing Scalable Order Systems: Architecture, ID Strategies, and Distributed Transactions
DeWu Technology
DeWu Technology
Jun 22, 2022 · Databases

Rainbow Bridge: A ShardingSphere‑Based Database Middleware for Scalability and Reliability

To support the rapidly growing DeWu App, the team created Rainbow Bridge—a ShardingSphere‑based middleware that unifies horizontal sharding, read‑write separation, shadow‑DB routing, rate limiting, and active‑active traffic correction, while adding a graphical console, stability enhancements, fine‑grained observability, hybrid JDBC/Proxy switching, and ongoing open‑source contributions.

Data ShardingDatabase MiddlewareObservability
0 likes · 18 min read
Rainbow Bridge: A ShardingSphere‑Based Database Middleware for Scalability and Reliability
macrozheng
macrozheng
Nov 16, 2021 · Backend Development

Master Enterprise Data Management with MyBatis‑Mate: Sharding, Encryption & Auditing

This article introduces MyBatis‑Mate, an official MyBatis‑Plus extension that provides enterprise‑grade features such as sharding, data auditing, field encryption, sensitive data masking, dictionary binding, dynamic DDL maintenance, and flexible data‑source switching, complete with usage examples and code snippets.

Data AuditingData ShardingDatabase
0 likes · 10 min read
Master Enterprise Data Management with MyBatis‑Mate: Sharding, Encryption & Auditing
Architecture Digest
Architecture Digest
Mar 9, 2016 · Databases

Overview of Distributed Database Systems: Concepts, Classification, Features, and Data Sharding

This article provides a comprehensive overview of distributed database systems, covering their evolution from centralized architectures, classifications, key characteristics, advantages and disadvantages, data sharding methods, allocation strategies, architectural models, and management considerations, highlighting the benefits and challenges of distributed data management.

DDBMSData ShardingDatabase Architecture
0 likes · 9 min read
Overview of Distributed Database Systems: Concepts, Classification, Features, and Data Sharding