BestHub
Sign in
Tagged articles
602 articles
Page 3 of 7
dbaplus Community
dbaplus Community
Jul 25, 2023 · Databases

How to Seamlessly Migrate a Live System to Sharding with Dual‑Write and Diff

This article explains how Qunar’s ticket ancillary service upgraded from a single-database architecture to a sharded one without downtime, detailing dual-write, transaction handling, mapping-key routing, diff verification, the challenges encountered, and a component-based solution that makes future migrations reusable.

distributed-transactiondual-writemapping key
0 likes · 35 min read
How to Seamlessly Migrate a Live System to Sharding with Dual‑Write and Diff
Programmer DD
Programmer DD
Jul 15, 2023 · Operations

Why Is My Elasticsearch Cluster Using 99% Memory? Sharding, Translog & JVM Insights

This article analyzes a 7‑node Elasticsearch cluster with 500 million documents, revealing excessive shard count, high heap and OS memory usage, large translog, low query‑cache hit rate, and heavy I/O, and offers concrete recommendations on mapping, sharding, JVM tuning, and resource management to restore performance.

Cluster MonitoringElasticsearchJVM
0 likes · 17 min read
Why Is My Elasticsearch Cluster Using 99% Memory? Sharding, Translog & JVM Insights
Architect
Architect
Jul 14, 2023 · Databases

From Single‑Node to Scalable Redis Cluster: A Step‑by‑Step Architecture Guide

This article walks through Redis's evolution from a simple single‑instance cache to a highly available, high‑performance cluster, explaining persistence mechanisms (RDB, AOF, hybrid), master‑slave replication, Sentinel automatic failover, and sharding strategies with concrete examples and trade‑offs.

Database ArchitecturePersistenceReplication
0 likes · 20 min read
From Single‑Node to Scalable Redis Cluster: A Step‑by‑Step Architecture Guide
Su San Talks Tech
Su San Talks Tech
Jul 6, 2023 · Databases

Master ShardingSphere: Quick Sharding with Spring Boot and YML

This tutorial walks through the fundamentals and architecture of ShardingSphere, compares it with other sharding tools, and demonstrates how to implement database sharding in a Spring Boot project using both YML configuration and pure Java code, including detailed code snippets and deployment tips.

ShardingSphereSpring BootYAML
0 likes · 22 min read
Master ShardingSphere: Quick Sharding with Spring Boot and YML
ITPUB
ITPUB
Jun 23, 2023 · Databases

Seamless Sharding Migration: Dual‑Write, Mapping Keys, and Diff Validation

This article explains how to smoothly migrate a legacy single‑database system to a sharded architecture by using dual‑write, mapping‑key routing, custom transaction handling, and offline/real‑time diff checks, while detailing the underlying MyBatis integration and component‑based design.

MyBatisdatabase partitioningdiff-validation
0 likes · 36 min read
Seamless Sharding Migration: Dual‑Write, Mapping Keys, and Diff Validation
Wukong Talks Architecture
Wukong Talks Architecture
Jun 20, 2023 · Databases

Evolution of JD Baitiao’s Data Architecture: From MySQL to Apache ShardingSphere

This article chronicles JD Baitiao’s journey from early MySQL and NoSQL solutions through DBRep to the adoption of Apache ShardingSphere, highlighting the technical motivations, decoupling strategies, performance comparisons, and the broader Database Plus vision for scalable, stable financial‑grade data architectures.

Big DataJD BaitiaoShardingSphere
0 likes · 14 min read
Evolution of JD Baitiao’s Data Architecture: From MySQL to Apache ShardingSphere
JD Cloud Developers
JD Cloud Developers
Jun 15, 2023 · Databases

Mastering Distributed Join Queries: MySQL Sharding-JDBC and Elasticsearch Strategies

This article explores the challenges of distributed join queries, detailing MySQL sharding‑jdbc join implementation, routing strategies, and code examples, then examines Elasticsearch‑SQL join capabilities, various join algorithms, and practical considerations for using nested types, offering insights for optimizing performance in distributed data environments.

DistributedElasticsearchmysql
0 likes · 11 min read
Mastering Distributed Join Queries: MySQL Sharding-JDBC and Elasticsearch Strategies
Qunar Tech Salon
Qunar Tech Salon
May 30, 2023 · Databases

Smooth Migration to Sharding: Design, Implementation, and Componentization of Qunar's Database Sharding Solution

This article describes the challenges of migrating a long‑running monolithic database to a sharded architecture, presents a two‑stage smooth migration strategy using double‑write, diff verification, and mapping‑key techniques, and details the componentized design, core implementation, and configuration of a reusable sharding framework built on MyBatis and Spring.

MyBatisSoftware Architecturedata synchronization
0 likes · 34 min read
Smooth Migration to Sharding: Design, Implementation, and Componentization of Qunar's Database Sharding Solution
Top Architect
Top Architect
May 29, 2023 · Databases

Large Table Splitting in MySQL: Challenges, Solutions, and Lessons Learned

Facing a 50‑million‑row financial transaction table, the team implemented horizontal sharding with sharding‑jdbc, designed migration and pagination strategies, tackled multi‑datasource transaction issues, and outlined a phased rollout, providing practical insights for large‑scale MySQL table splitting and backend system stability.

BackendData Migrationmysql
0 likes · 11 min read
Large Table Splitting in MySQL: Challenges, Solutions, and Lessons Learned
MaGe Linux Operations
MaGe Linux Operations
May 16, 2023 · Databases

How We Split a 50‑Million‑Row MySQL Table: Strategies and Lessons Learned

This article recounts how a finance team tackled a 50‑million‑row MySQL table by analyzing system bottlenecks, defining splitting goals, selecting sharding‑jdbc, addressing multi‑datasource transactions and pagination, designing data migration strategies, and executing a staged rollout to ensure stability and performance.

BackendLarge Tablesdatabase migration
0 likes · 12 min read
How We Split a 50‑Million‑Row MySQL Table: Strategies and Lessons Learned
Wukong Talks Architecture
Wukong Talks Architecture
May 16, 2023 · Databases

Understanding Sharding: 21 Common Concepts of Database Partitioning

This article introduces the fundamental concepts of database sharding, including data fragmentation, nodes, logical and physical tables, broadcast and single tables, sharding keys, strategies, algorithms, binding tables, SQL parsing, routing, rewriting, execution, result merging, distributed primary keys, data masking, distributed transactions, migration, and shadow databases, providing a comprehensive guide for designing and operating a sharded architecture.

Broadcast TableSQL Routingbinding tables
0 likes · 19 min read
Understanding Sharding: 21 Common Concepts of Database Partitioning
Architecture Digest
Architecture Digest
May 15, 2023 · Databases

Case Study: Splitting a Massive MySQL Financial Transaction Table Using Sharding-JDBC

This article details how a finance team tackled a 50‑million‑row MySQL transaction table by designing a horizontal sharding strategy with Sharding‑JDBC, addressing performance bottlenecks, multi‑source transaction handling, pagination across shards, and a phased data migration and rollout plan.

Data Migrationbackend-developmentdatabase scaling
0 likes · 11 min read
Case Study: Splitting a Massive MySQL Financial Transaction Table Using Sharding-JDBC
Architect
Architect
May 13, 2023 · Databases

Splitting a Massive MySQL Financial Transaction Table: Challenges, Strategies, and Implementation

The article details how a finance team tackled a 50‑million‑row MySQL transaction table by analyzing the pre‑split problems, defining split goals, choosing sharding‑jdbc, addressing multi‑datasource transaction and pagination issues, designing a phased migration and rollout plan, and summarizing lessons learned.

BackendData Migrationmysql
0 likes · 13 min read
Splitting a Massive MySQL Financial Transaction Table: Challenges, Strategies, and Implementation
ITPUB
ITPUB
Apr 22, 2023 · Databases

Choosing the Right Sharding Algorithm: From Range to Consistent Hash

This article compares common database sharding strategies—range sharding, mapping tables, ID modulo, hash‑based sharding, and consistent hashing—explaining their implementation complexity, data distribution characteristics, scalability concerns, and practical trade‑offs with code examples.

Consistent HashHashingsharding
0 likes · 11 min read
Choosing the Right Sharding Algorithm: From Range to Consistent Hash
dbaplus Community
dbaplus Community
Apr 19, 2023 · Databases

How to Migrate Large Tables and Sync Data Seamlessly in Production

This article explains common business scenarios such as large‑table splitting, cross‑database migration, and data synchronization, then compares loss‑bearing and loss‑less migration strategies, and provides detailed step‑by‑step implementation guidance for smooth dual‑write migration and real‑world tooling.

Data MigrationDual Writedatabases
0 likes · 17 min read
How to Migrate Large Tables and Sync Data Seamlessly in Production
Efficient Ops
Efficient Ops
Apr 18, 2023 · Databases

Mastering MongoDB Clusters: Setup, Monitoring, Migration, and Optimization

This comprehensive guide explains MongoDB cluster architecture, component roles, common use cases, monitoring commands, essential maintenance operations, data migration steps, troubleshooting of typical production issues, and practical optimization recommendations for high‑performance deployments.

BackupClusterMongoDB
0 likes · 20 min read
Mastering MongoDB Clusters: Setup, Monitoring, Migration, and Optimization
IT Architects Alliance
IT Architects Alliance
Apr 14, 2023 · Databases

Comprehensive Guide to Database Horizontal Scaling, Smooth 2N Expansion, and Keepalived High‑Availability Configuration

This technical guide explains how to scale a sharded database horizontally by introducing five expansion schemes—including shutdown, write‑stop, log‑based, dual‑write, and smooth 2N approaches—covers MariaDB installation, master‑master replication setup, dynamic data‑source configuration with ShardingJDBC, and detailed Keepalived high‑availability configuration for seamless service continuity.

Dual WriteDynamic Data SourceMariaDB
0 likes · 31 min read
Comprehensive Guide to Database Horizontal Scaling, Smooth 2N Expansion, and Keepalived High‑Availability Configuration
Top Architect
Top Architect
Mar 14, 2023 · Databases

Handling Large MySQL Tables: Partitioning, Sharding, and Archiving Strategies

This article explains how to assess MySQL table size, identify performance issues caused by massive data volumes, and presents three practical solutions—table partitioning, database sharding, and hot‑cold data archiving—along with detailed SQL examples and considerations for implementation.

data archivingdatabase partitioningmysql
0 likes · 16 min read
Handling Large MySQL Tables: Partitioning, Sharding, and Archiving Strategies
Su San Talks Tech
Su San Talks Tech
Mar 14, 2023 · Databases

How to Solve Read‑Amplification in MySQL Sharding: Strategies & Tips

This article explains the concepts of database sharding, compares vertical and horizontal partitioning, discusses ID‑range and modulo sharding methods, analyzes the read‑amplification problem caused by non‑sharding keys, and presents practical solutions such as auxiliary index tables, Elasticsearch integration, and TiDB adoption.

ElasticsearchTiDBdatabase scaling
0 likes · 14 min read
How to Solve Read‑Amplification in MySQL Sharding: Strategies & Tips
Top Architect
Top Architect
Mar 4, 2023 · Databases

How to Handle Large MySQL Tables: Partitioning, Sharding, and Archiving Strategies

This article explains why large MySQL tables cause slow queries, analyzes table size, disk usage, and instance capacity, and presents three practical solutions—table partitioning, database sharding (horizontal/vertical), and hot‑cold data archiving—helping developers choose the appropriate strategy for their workloads.

data archivingdatabase partitioningmysql
0 likes · 18 min read
How to Handle Large MySQL Tables: Partitioning, Sharding, and Archiving Strategies
MaGe Linux Operations
MaGe Linux Operations
Feb 22, 2023 · Databases

Mastering MySQL Sharding: When and How to Use Database Partitioning

This article explains the concepts, strategies, and practical scenarios for MySQL database partitioning, covering both table splitting (vertical and horizontal) and database splitting (vertical and horizontal), with examples, advantages, and when to apply each technique to handle large-scale data workloads.

ScalabilityVertical Partitioningdatabase partitioning
0 likes · 9 min read
Mastering MySQL Sharding: When and How to Use Database Partitioning
ITPUB
ITPUB
Feb 18, 2023 · Backend Development

Mastering High‑Performance Backend: Lock‑Free, Zero‑Copy, Serialization, and More

This comprehensive guide explores essential backend techniques—including lock‑free programming, zero‑copy I/O, efficient serialization, pooling, concurrency, async processing, caching strategies, sharding, storage optimizations, and queue mechanisms—to build high‑performance, scalable services while highlighting practical code examples and real‑world trade‑offs.

BackendQueueZero Copy
0 likes · 51 min read
Mastering High‑Performance Backend: Lock‑Free, Zero‑Copy, Serialization, and More
Tencent Cloud Developer
Tencent Cloud Developer
Feb 15, 2023 · Databases

How to Build a Stable, High‑Performance Redis Cluster: Persistence, Replication, Sentinel and Sharding

This guide walks developers through transforming a single‑node Redis cache into a stable, high‑performance cluster by explaining data persistence options (RDB, AOF, hybrid), master‑slave replication, Sentinel‑driven automatic failover, and both client‑side and proxy‑based sharding for horizontal scalability.

ClusterPersistenceReplication
0 likes · 17 min read
How to Build a Stable, High‑Performance Redis Cluster: Persistence, Replication, Sentinel and Sharding
Efficient Ops
Efficient Ops
Feb 12, 2023 · Databases

Why Single‑Node MySQL Is No Longer Enough: Embrace Distributed Databases

The article explains the limitations of standalone MySQL for large‑scale workloads, explores sharding, read/write separation, and middleware approaches, and argues that modern distributed databases and cloud‑native solutions are the sustainable path forward for data‑intensive applications.

Cloud Databasesdatabase scalingdistributed databases
0 likes · 6 min read
Why Single‑Node MySQL Is No Longer Enough: Embrace Distributed Databases
21CTO
21CTO
Feb 11, 2023 · Big Data

Mastering Elasticsearch: Core Concepts, Architecture, and Performance Tips

This comprehensive guide explains what Elasticsearch does, its underlying Lucene technology, core concepts such as clusters, shards, replicas, mapping, indexing and storage mechanisms, and provides practical performance‑tuning advice for building and operating a robust distributed search engine.

ClusterElasticsearchMapping
0 likes · 35 min read
Mastering Elasticsearch: Core Concepts, Architecture, and Performance Tips
dbaplus Community
dbaplus Community
Jan 29, 2023 · Databases

Why Single‑Node MySQL Is Dying and Distributed Databases Are the Future

The article explains how single‑instance MySQL struggles with huge tables, resource‑starved databases, and high read traffic, and argues that sharding middleware is being replaced by modern distributed databases such as TiDB, OceanBase, Aurora, and PolarDB as the sustainable scaling solution.

Database Architecturedistributed databasemysql
0 likes · 9 min read
Why Single‑Node MySQL Is Dying and Distributed Databases Are the Future
ITPUB
ITPUB
Jan 29, 2023 · Databases

How to Query Sharded Orders Efficiently: Redundant, Index, and Gene Strategies

When order_id is not the sharding key, querying a sharded order table can require scanning every shard, but three techniques—data redundancy, an index table, and embedding shard information into the order_id (gene method)—enable fast, single‑shard lookups.

gene methodindex tableorder query
0 likes · 10 min read
How to Query Sharded Orders Efficiently: Redundant, Index, and Gene Strategies
Architect's Guide
Architect's Guide
Jan 26, 2023 · Databases

Primary Key and Index Design for Distributed Databases

The article explains why auto‑increment primary keys are unsuitable for sharded databases, recommends globally unique ordered identifiers, and details index and global‑table strategies—including redundant tables, index tables, and embedding shard information in keys—to achieve efficient single‑shard queries.

distributed databasesglobal unique identifierindex design
0 likes · 10 min read
Primary Key and Index Design for Distributed Databases
Top Architect
Top Architect
Jan 12, 2023 · Databases

MySQL Master‑Slave Replication, Binlog/Redo Log Basics, and Sharding‑JDBC Read‑Write Splitting & Sharding Implementation

This article provides a comprehensive tutorial on configuring MySQL master‑slave replication with Docker, explains the roles of binlog and redo log, describes two‑phase commit, and demonstrates how to use Sharding‑JDBC (ShardingSphere) for read‑write splitting and horizontal sharding in a Spring Boot application, complete with configuration files and code examples.

ReplicationShardingSphereSpringBoot
0 likes · 24 min read
MySQL Master‑Slave Replication, Binlog/Redo Log Basics, and Sharding‑JDBC Read‑Write Splitting & Sharding Implementation
ITPUB
ITPUB
Dec 31, 2022 · Databases

15 Must‑Know Interview Questions on Database Sharding and Partitioning

This article explains why and when to split databases and tables, how to choose sharding keys, various sharding strategies such as range, hash and consistent hash, handling cross‑node joins, pagination, distributed IDs, middleware choices, and step‑by‑step zero‑downtime migration techniques.

Scalabilitymysqlsharding
0 likes · 17 min read
15 Must‑Know Interview Questions on Database Sharding and Partitioning
Java Backend Technology
Java Backend Technology
Dec 29, 2022 · Databases

Why MySQL Must Evolve: From Single‑Node Limits to Distributed Databases

The article explains how traditional single‑node MySQL faces scalability challenges such as massive tables, resource exhaustion, and read‑heavy workloads, and argues that adopting sharding, middleware, or modern distributed databases like TiDB, OceanBase, Aurora, and PolarDB is the inevitable path forward for robust data management.

cloud databasedatabase scalingdistributed database
0 likes · 6 min read
Why MySQL Must Evolve: From Single‑Node Limits to Distributed Databases
Architect
Architect
Dec 28, 2022 · Backend Development

Design and Implementation of Pickup Code Generation and Write‑off Logic for SaaS E‑commerce Platforms

This article explains the end‑to‑end design of a pickup code system—including simple single‑table implementation, sharding‑aware multi‑database strategies, code generation algorithms, concurrency handling with distributed locks, and practical solutions to common pitfalls—targeted at SaaS e‑commerce back‑ends.

Order ManagementSaaSpickup code
0 likes · 12 min read
Design and Implementation of Pickup Code Generation and Write‑off Logic for SaaS E‑commerce Platforms
Top Architect
Top Architect
Dec 24, 2022 · Databases

Elasticsearch Architecture: Inverted Index, Sharding, and Data Operations

This article explains the core concepts of Elasticsearch, including how its inverted index works, the structure of term dictionaries and posting lists, shard and replica configuration, cluster node roles, the detailed write, refresh, flush, and merge processes, as well as how search queries are executed across distributed shards.

distributed architectureinverted indexsearch engine
0 likes · 9 min read
Elasticsearch Architecture: Inverted Index, Sharding, and Data Operations
Architect
Architect
Dec 23, 2022 · Databases

Understanding Elasticsearch Inverted Index, Sharding, and Data Operations

This article explains the core concepts of Elasticsearch, including the structure and purpose of inverted indexes, how shards and replicas are organized in a cluster, and the detailed workflows for writing, reading, searching, and deleting documents within a distributed environment.

Data Writedistributed-architectureinverted-index
0 likes · 8 min read
Understanding Elasticsearch Inverted Index, Sharding, and Data Operations
ITPUB
ITPUB
Dec 8, 2022 · Databases

How to Efficiently Query Sharded Order Tables: Redundant, Index, and Gene Methods

When order_id is not the sharding key, querying a sharded order table can require scanning every shard, but three design patterns—data redundancy, an index table, and gene‑based sharding—enable fast single‑shard lookups while balancing storage and complexity.

Database designgene shardingindex table
0 likes · 9 min read
How to Efficiently Query Sharded Order Tables: Redundant, Index, and Gene Methods
JD Tech
JD Tech
Dec 7, 2022 · Databases

Practical Guide to Large‑Scale Data Migration Using Sharding‑Proxy

This article presents a step‑by‑step practice of migrating massive billing data to 32 sharded databases with Sharding‑Proxy, covering background, objectives, four migration solutions, detailed proxy installation and configuration, debugging, migration workflow, data validation, common issues and their resolutions.

ConfigurationData MigrationSharding-Proxy
0 likes · 14 min read
Practical Guide to Large‑Scale Data Migration Using Sharding‑Proxy
ITPUB
ITPUB
Nov 29, 2022 · Databases

When MySQL Hits Its Limits: Scaling with Horizontal and Vertical Sharding

This article explains MySQL's theoretical row limits, why performance degrades long before reaching them, and how horizontal and vertical database sharding can distribute massive datasets across multiple instances to improve load handling and enable seamless scaling.

Vertical Partitioninghorizontal partitioningmysql
0 likes · 7 min read
When MySQL Hits Its Limits: Scaling with Horizontal and Vertical Sharding
Programmer DD
Programmer DD
Nov 21, 2022 · Databases

How to Achieve Seamless Horizontal Scaling with 2N Expansion and Keepalived

This guide explains how to scale a sharded database horizontally by adding new nodes, handling data migration with stop‑service, stop‑write, log‑based, dual‑write, and smooth 2N strategies, and implementing high‑availability using MariaDB double‑master replication and Keepalived, complete with configuration examples and code snippets.

Data MigrationMariaDBhigh availability
0 likes · 35 min read
How to Achieve Seamless Horizontal Scaling with 2N Expansion and Keepalived
ITPUB
ITPUB
Nov 19, 2022 · Databases

Choosing Between OceanBase and TiDB: Architecture, Performance, and Trade‑offs

This article compares the architectures of OceanBase and TiDB, explaining how their sharding and compute‑storage separation designs affect high availability, latency, and workload suitability, and offers guidance on selecting the right distributed database for complex enterprise applications.

Compute-Storage SeparationOceanBaseTiDB
0 likes · 12 min read
Choosing Between OceanBase and TiDB: Architecture, Performance, and Trade‑offs
Architect's Guide
Architect's Guide
Nov 11, 2022 · Databases

Database Sharding: Table Partitioning, Horizontal and Vertical Splitting, and Implementation with Sharding-JDBC and Mycat

This article explains why high‑concurrency big‑data systems need database sharding, describes table‑level partitioning methods (modulus and range), shows how to implement them with MySQL and Java code, compares vertical and horizontal database splitting, and evaluates middleware solutions such as Sharding‑JDBC and Mycat.

database partitioninghorizontal scalingjava
0 likes · 11 min read
Database Sharding: Table Partitioning, Horizontal and Vertical Splitting, and Implementation with Sharding-JDBC and Mycat
Top Architect
Top Architect
Nov 10, 2022 · Databases

Database Sharding and Partitioning Strategy for High‑Volume Order Systems

This article explains how to handle billions of order records by classifying data into hot and cold tiers, using MySQL sharding with database‑and‑table partitioning, storing cold data in Elasticsearch and Hive, and outlines combined routing formulas for scalable backend architecture.

database partitioningmysqlsharding
0 likes · 8 min read
Database Sharding and Partitioning Strategy for High‑Volume Order Systems
Architects Research Society
Architects Research Society
Nov 4, 2022 · Fundamentals

eBay Scalability Best Practices: Functional Partitioning, Horizontal Sharding, Async Decoupling, and More

This article outlines eBay's key scalability best practices—including functional decomposition, horizontal sharding, avoiding distributed transactions, asynchronous decoupling, virtualization, and intelligent caching—to demonstrate how large‑scale web systems can achieve linear resource growth and high availability.

AsynchronousOperationsScalability
0 likes · 14 min read
eBay Scalability Best Practices: Functional Partitioning, Horizontal Sharding, Async Decoupling, and More
HomeTech
HomeTech
Nov 2, 2022 · Backend Development

Technical Architecture and Performance Optimization of the Intelligent Focus Auto‑Placement Platform

This article presents the design, implementation challenges, and optimization strategies of an intelligent auto‑placement platform, covering background analysis, distributed task scheduling with XXL‑Job, sharding with ShardingSphere, caching via Caffeine, message‑queue integration using Kafka, and the resulting performance gains.

Backend ArchitectureDistributed Schedulingsharding
0 likes · 16 min read
Technical Architecture and Performance Optimization of the Intelligent Focus Auto‑Placement Platform
Java Architect Essentials
Java Architect Essentials
Nov 1, 2022 · Databases

Designing Routing Keys for Sharding in an Order Platform

The article explains how to select and implement routing keys for database sharding in an order platform, covering time‑based and business‑related keys, user and merchant scenarios, hash‑modulo strategies for single and multi‑database environments, and the overall data flow.

BackendHashingRouting Key
0 likes · 5 min read
Designing Routing Keys for Sharding in an Order Platform
Top Architect
Top Architect
Oct 19, 2022 · Databases

MySQL Replication, High Availability, and Sharding: Concepts and Solutions

This article explains the evolution from single‑node MySQL databases to master‑slave replication, various replication modes, high‑availability strategies, and both vertical and horizontal sharding techniques, while discussing the associated challenges such as distributed transactions, routing, and operational complexity.

Distributed TransactionsReplicationdatabase scaling
0 likes · 11 min read
MySQL Replication, High Availability, and Sharding: Concepts and Solutions
Architecture Digest
Architecture Digest
Oct 17, 2022 · Databases

Designing Routing Keys for Sharding in an Order Platform

The article explains how to choose and implement routing keys for database sharding in an order platform, covering time‑based and business‑related keys, user and merchant scenarios, hash‑mod routing strategies, and the complete data flow with diagrams.

DesignRouting Keyorder platform
0 likes · 5 min read
Designing Routing Keys for Sharding in an Order Platform
ITPUB
ITPUB
Oct 12, 2022 · Databases

Optimizing Distributed Database Sorting: In-Memory Merge to Disk Buffering

This article examines the challenges of sorting queries in distributed databases, outlines the limitations of in‑memory proxy sorting, and presents a step‑by‑step optimized approach that leverages per‑shard sorting, disk‑based buffering, and priority‑queue merging to reduce memory pressure and I/O overhead.

disk bufferingpriority-queueproxy merge
0 likes · 12 min read
Optimizing Distributed Database Sorting: In-Memory Merge to Disk Buffering
IT Architects Alliance
IT Architects Alliance
Oct 10, 2022 · Databases

Designing Routing Keys for Database Sharding in an Order Platform

The article explains how to choose and implement routing keys for database sharding in an online food ordering system, covering supported scenarios, user‑side and merchant‑side keys, single‑ and multi‑database strategies, and the impact on data distribution and query efficiency.

Routing Keydistributed databasesorder platform
0 likes · 6 min read
Designing Routing Keys for Database Sharding in an Order Platform
ITPUB
ITPUB
Oct 7, 2022 · Databases

How to Efficiently Paginate Across Sharded Tables: Three Proven Methods

This article explains why sharding tables improves read/write speed, then compares three pagination strategies—global query, no‑skip pagination, and two‑phase query—detailing their SQL rewrites, execution steps, performance trade‑offs, and practical examples.

Distributedpaginationsharding
0 likes · 10 min read
How to Efficiently Paginate Across Sharded Tables: Three Proven Methods
IT Architects Alliance
IT Architects Alliance
Sep 15, 2022 · Databases

Designing Indexes and Primary Keys for Distributed Database Architectures

Effective index and primary key design in distributed database architectures requires using globally unique, ordered identifiers instead of auto‑increment keys, embedding shard information in keys, employing auxiliary index tables, and handling global tables to ensure queries target a single shard, thereby preserving scalability and performance.

distributed databasesglobal tablesindex design
0 likes · 11 min read
Designing Indexes and Primary Keys for Distributed Database Architectures
ITPUB
ITPUB
Sep 6, 2022 · Databases

From Monolith to Sharded MySQL: A Complete End‑to‑End Sharding Case Study

This article walks through a real‑world large‑scale MySQL sharding project, covering business refactoring, storage architecture design, data migration, incremental upgrades, best‑practice tips, and stability safeguards, while sharing concrete steps, pitfalls, and lessons learned from start to production rollout.

mysqlshardingstability
0 likes · 27 min read
From Monolith to Sharded MySQL: A Complete End‑to‑End Sharding Case Study
IT Architects Alliance
IT Architects Alliance
Sep 4, 2022 · Databases

Mastering MySQL: From Replication to High Availability and Sharding Strategies

This article examines why single-node databases no longer meet modern internet workloads, explores MySQL replication models (master‑slave, asynchronous, semi‑synchronous, group replication), discusses high‑availability solutions such as MHA, MGR and Orchestrator, and outlines vertical and horizontal sharding techniques along with their trade‑offs.

Database ArchitectureMGRMHA
0 likes · 13 min read
Mastering MySQL: From Replication to High Availability and Sharding Strategies
Java High-Performance Architecture
Java High-Performance Architecture
Sep 2, 2022 · Backend Development

Eight Essential Software Architecture Patterns Every Backend Engineer Should Master

This article explains eight core software architecture patterns—from single‑database single‑application to microservices, multi‑level caching, and multi‑datacenter deployments—detailing their designs, typical use cases, advantages, and drawbacks to help engineers choose the right approach for scalable, high‑performance systems.

ScalabilitySoftware Architecturecaching
0 likes · 23 min read
Eight Essential Software Architecture Patterns Every Backend Engineer Should Master
WeChat Client Technology Team
WeChat Client Technology Team
Aug 30, 2022 · Databases

How WeChat Optimized Its Desktop Database for Speed, Size, and Reliability

This article analyzes the performance and storage problems of the Windows version of WeChat, explains why message growth causes slowness, large file size, and corruption risk, and presents a multi‑pronged solution involving sharding, indexing, payload compression, and robustness improvements that halve database size and boost I/O performance by about ten percent.

Database OptimizationSQLitecompression
0 likes · 11 min read
How WeChat Optimized Its Desktop Database for Speed, Size, and Reliability
Su San Talks Tech
Su San Talks Tech
Aug 18, 2022 · Fundamentals

What Hidden Pitfalls Await in Distributed Systems? A Deep Dive into Messaging, Caching, and Sharding

This article explores the fundamental concepts and common pitfalls of distributed systems—including CAP and BASE theories, message‑queue reliability issues, Redis sentinel challenges, sharding strategies, and distributed transaction models—offering practical guidance to avoid costly failures in real‑world deployments.

CAP theoremsharding
0 likes · 24 min read
What Hidden Pitfalls Await in Distributed Systems? A Deep Dive into Messaging, Caching, and Sharding
Wukong Talks Architecture
Wukong Talks Architecture
Aug 11, 2022 · Fundamentals

Common Pitfalls and Core Concepts of Distributed Systems, Message Queues, Caching, Sharding, and Transactions

This article provides a comprehensive overview of distributed system fundamentals—including CAP and BASE theories—while detailing typical pitfalls and mitigation strategies for message queues, distributed caches, database sharding, and distributed transactions, helping engineers decide when and how to adopt distributed architectures.

BASE modelCAP theoremcaching
0 likes · 23 min read
Common Pitfalls and Core Concepts of Distributed Systems, Message Queues, Caching, Sharding, and Transactions
Sohu Tech Products
Sohu Tech Products
Aug 3, 2022 · Backend Development

Design and Evolution of the Price‑Increase Coupon Service for a C2B Recycling Platform

This article details the architectural evolution of a price‑increase coupon system used in a C2B digital‑product recycling business, covering its initial experimental phase, platformization, sharding, intelligent recommendation, Elasticsearch integration, and stability enhancements through NoSQL and read‑replica strategies.

CouponElasticsearchSystem Design
0 likes · 10 min read
Design and Evolution of the Price‑Increase Coupon Service for a C2B Recycling Platform
Senior Brother's Insights
Senior Brother's Insights
Jul 24, 2022 · Databases

Understanding Redis High Availability: Master‑Slave, Sentinel, and Cluster Explained

This article explains why single‑node Redis suffers from single‑point failures, describes the master‑slave replication model, details the Sentinel automatic failover mechanism, compares various sharding solutions like client‑side sharding, Twemproxy, Codis, and outlines the features and deployment considerations of Redis Cluster.

Clusterdatabasehigh availability
0 likes · 19 min read
Understanding Redis High Availability: Master‑Slave, Sentinel, and Cluster Explained
FunTester
FunTester
Jul 21, 2022 · Databases

Scaling Databases with Distributed Sharding and Peak‑Shaving Strategies

The article explains why simply adding hardware cannot keep up with growing database workloads and presents a three‑step approach—business isolation, horizontal sharding, and advanced reporting using PolarDB‑X—to achieve high concurrency, elastic capacity, and efficient caching for modern high‑traffic applications.

Distributed SystemsPeak ShavingPolarDB-X
0 likes · 7 min read
Scaling Databases with Distributed Sharding and Peak‑Shaving Strategies
Su San Talks Tech
Su San Talks Tech
Jul 21, 2022 · Databases

Mastering Database Sharding: When and How to Split Databases and Tables

This article explains the three aspects of database sharding—database‑only, table‑only, and both—detailing when to apply each, how to choose sharding keys, common algorithms, ID generation strategies, available open‑source tools, and the new challenges introduced by sharding.

ID generationVertical Scalingdatabase partitioning
0 likes · 16 min read
Mastering Database Sharding: When and How to Split Databases and Tables
Programmer DD
Programmer DD
Jul 20, 2022 · Databases

Mastering MySQL Sharding: Strategies, ID Generation, and Seamless Scaling

This article explains why and how to apply database sharding, introduces key terminology, compares global ID generation methods such as auto‑increment, UUID, COMB and Snowflake, outlines sharding algorithms, discusses challenges like distributed transactions, and presents practical expansion and implementation solutions.

ID generationdatabase scalingmysql
0 likes · 12 min read
Mastering MySQL Sharding: Strategies, ID Generation, and Seamless Scaling
Zhuanzhuan Tech
Zhuanzhuan Tech
Jul 20, 2022 · Backend Development

Design and Evolution of the Price‑Increase Coupon Service for a C2B Recycling Platform

This article details the design, evolution, and scaling strategies of a price‑increase coupon system for a C2B digital product recycling platform, covering its initial experimental phase, platformization, sharding‑JDBC implementation, intelligent coupon recommendation, Elasticsearch integration, and operational optimizations for high‑throughput stability.

BackendCouponMicroservices
0 likes · 11 min read
Design and Evolution of the Price‑Increase Coupon Service for a C2B Recycling Platform
MaGe Linux Operations
MaGe Linux Operations
Jul 17, 2022 · Databases

Understanding Redis High Availability: Master‑Slave, Sentinel, and Cluster Explained

This article explains how Redis tackles single‑point failures with master‑slave replication, introduces Sentinel for automatic failover, compares client‑side and proxy sharding solutions like Twemproxy and Codis, and details the native Redis Cluster architecture for true distributed storage and high availability.

Clusterhigh availabilityredis
0 likes · 18 min read
Understanding Redis High Availability: Master‑Slave, Sentinel, and Cluster Explained
Aikesheng Open Source Community
Aikesheng Open Source Community
Jul 14, 2022 · Backend Development

Introducing and Evaluating the DBLE Split Command for Accelerated Data Import

This article explains the DBLE split feature that partitions large mysqldump files according to sharding configuration, provides command syntax and usage examples, and presents a performance test showing that split‑based import can be up to 18 times faster than direct DBLE import while preserving data correctness.

DBLEPerformance Testingdata import
0 likes · 12 min read
Introducing and Evaluating the DBLE Split Command for Accelerated Data Import
JavaEdge
JavaEdge
Jul 10, 2022 · Databases

How to Partition Massive Datasets: Strategies, Trade‑offs, and Hotspot Prevention

This article explains why large‑scale systems need data partitioning beyond replication, defines partitions and sharding, compares key‑range and hash‑based partitioning methods, discusses load‑balancing and hotspot avoidance, and shows how replication works together with partitioning to improve scalability.

HashingPartitioningScalability
0 likes · 12 min read
How to Partition Massive Datasets: Strategies, Trade‑offs, and Hotspot Prevention
ITPUB
ITPUB
Jul 8, 2022 · Databases

Uncovering the Flaws: Why No Distributed Database Architecture Is Perfect

This article examines the most common Chinese distributed database architectures, comparing integrated SHARDING designs like OceanBase with separated compute‑storage models such as TiDB and SQL‑proxy based systems, highlighting their inherent trade‑offs, performance challenges, and practical considerations for choosing the right solution.

OceanBaseSQL proxyTiDB
0 likes · 11 min read
Uncovering the Flaws: Why No Distributed Database Architecture Is Perfect
Selected Java Interview Questions
Selected Java Interview Questions
Jul 5, 2022 · Big Data

Understanding Elasticsearch: Core Concepts, Architecture, Indexing Mechanics and Performance Optimization

This article explains the fundamentals of structured and unstructured data, introduces Lucene's inverted index, describes Elasticsearch's distributed cluster architecture, node roles, sharding and replication mechanisms, indexing workflow with refresh and translog, storage segment model, and provides practical performance‑tuning recommendations.

ClusterElasticsearchReplication
0 likes · 36 min read
Understanding Elasticsearch: Core Concepts, Architecture, Indexing Mechanics and Performance Optimization
Selected Java Interview Questions
Selected Java Interview Questions
Jul 4, 2022 · Backend Development

Introducing MyBatis-Mate: Enterprise‑Level Features for MyBatis‑Plus

This article introduces MyBatis‑Mate, an official MyBatis‑Plus extension that provides enterprise‑grade capabilities such as dictionary binding, field encryption, data masking, dynamic table maintenance, audit logging, sharding, multi‑datasource switching, and customizable data‑scope handling, all illustrated with code examples.

MyBatisdata encryptionsensitive data
0 likes · 13 min read
Introducing MyBatis-Mate: Enterprise‑Level Features for MyBatis‑Plus
Sanyou's Java Diary
Sanyou's Java Diary
Jun 30, 2022 · Databases

Mastering Redis High Availability: Sharding, Consistent Hashing, and Sentinel Explained

This article explains Redis high‑availability strategies, covering basic hash sharding, the advantages of consistent hashing, client‑side versus proxy‑based partitioning, master‑slave replication, and the Sentinel failover mechanism, with diagrams illustrating node addition, removal, and failover decision processes.

consistent hashingdatabasehigh availability
0 likes · 10 min read
Mastering Redis High Availability: Sharding, Consistent Hashing, and Sentinel Explained
政采云技术
政采云技术
Jun 30, 2022 · Backend Development

ElasticJob‑Lite Overview and Implementation Details

This article provides a comprehensive overview of ElasticJob‑Lite, a distributed scheduling solution for high‑concurrency environments, covering its architecture, ZooKeeper‑based registration and sharding mechanisms, core listener components, key code examples, and the UI console deployment.

Distributed SchedulingJob ManagementZooKeeper
0 likes · 14 min read
ElasticJob‑Lite Overview and Implementation Details
IT Architects Alliance
IT Architects Alliance
Jun 27, 2022 · Databases

Mastering Cluster Terminology and Database Cluster Architectures

This article explains core cluster concepts, the benefits of building database clusters, classifies common cluster types, and compares scalable architectures such as Oracle RAC, MySQL Cluster, sharding, CAP/BAS​E theory, and cross‑database transaction strategies for high availability and performance.

CAP theoremDatabase ClusterMySQL Cluster
0 likes · 22 min read
Mastering Cluster Terminology and Database Cluster Architectures
IT Architects Alliance
IT Architects Alliance
Jun 14, 2022 · Backend Development

8 Essential Backend Architecture Patterns Every Engineer Should Master

This article explores eight common backend architecture patterns—from simple single‑database setups to microservices, caching, sharding, elastic scaling, and multi‑region deployments—detailing their design principles, typical use cases, advantages, drawbacks, and practical implementation steps.

Backend ArchitectureDesign PatternsMicroservices
0 likes · 23 min read
8 Essential Backend Architecture Patterns Every Engineer Should Master
Tencent Cloud Developer
Tencent Cloud Developer
Jun 7, 2022 · Databases

Mastering MongoDB: From Basics to Advanced Performance Tuning

This comprehensive guide explores MongoDB’s core features—including schema flexibility, high availability, sharding, storage engine internals, indexing, and performance tuning—while providing practical examples, configuration tips, and best‑practice recommendations for developers and architects seeking to efficiently deploy and operate MongoDB in production environments.

MongoDBReplicationWiredTiger
0 likes · 40 min read
Mastering MongoDB: From Basics to Advanced Performance Tuning
Top Architect
Top Architect
May 26, 2022 · Backend Development

Eight Common Software Architecture Design Patterns and Their Advantages & Disadvantages

The article introduces eight widely used software architecture design patterns—including single‑database, content distribution, query separation, microservices, multi‑level caching, sharding, elastic scaling, and multi‑datacenter deployment—explaining their typical structures, suitable scenarios, and the key pros and cons of each.

BackendDesign PatternsMicroservices
0 likes · 21 min read
Eight Common Software Architecture Design Patterns and Their Advantages & Disadvantages
Bilibili Tech
Bilibili Tech
May 17, 2022 · Cloud Computing

Bilibili Object Storage Service (BOSS) Design: Building a Large-Scale Distributed Storage System in 13 Days

In just 13 days Bilibili transformed a simple MySQL‑based S3 prototype into the BOSS distributed object storage system by separating metadata and data, adding an RPC abstraction layer, implementing two‑level sharding, switching to RocksDB, and deploying a three‑replica, multi‑zone high‑availability architecture.

ReplicationRocksDBS3 protocol
0 likes · 15 min read
Bilibili Object Storage Service (BOSS) Design: Building a Large-Scale Distributed Storage System in 13 Days
Xiao Lou's Tech Notes
Xiao Lou's Tech Notes
May 5, 2022 · Databases

How We Scaled a Ride‑Hailing Order System: From Single DB to Sharded MySQL

This article chronicles the step‑by‑step evolution of a ride‑hailing order service, detailing how a single SQLServer database was optimized, split by business domain, migrated to MySQL, and finally sharded with a custom client‑side component, while leveraging read/write separation, caching, MQ, and data‑exchange platforms to sustain massive traffic.

cachingdatabasesmigration
0 likes · 15 min read
How We Scaled a Ride‑Hailing Order System: From Single DB to Sharded MySQL
Java Backend Technology
Java Backend Technology
May 1, 2022 · Databases

Mastering Database Performance: 8 Proven Optimization Strategies

This article explains why databases become slow, introduces a four‑layer thinking model, and presents eight practical solutions—including reducing data volume, leveraging space for performance, and selecting the right storage system—to help backend engineers resolve 80‑90% of common performance problems.

CQRSDatabase OptimizationNoSQL
0 likes · 19 min read
Mastering Database Performance: 8 Proven Optimization Strategies
dbaplus Community
dbaplus Community
Apr 24, 2022 · Databases

Eight Practical Strategies to Tackle Database Performance Bottlenecks

This article outlines a systematic approach for backend engineers to identify root causes of database slowness and presents eight concrete optimization techniques—ranging from data reduction and space‑for‑performance trade‑offs to appropriate storage system selection—while explaining when each method is most effective and what trade‑offs to consider.

CQRSDatabase PerformanceNoSQL
0 likes · 18 min read
Eight Practical Strategies to Tackle Database Performance Bottlenecks
dbaplus Community
dbaplus Community
Apr 20, 2022 · Backend Development

How to Build a Robust E‑Commerce Backend: Idempotency, Snapshots, and Sharding

This article explores common e‑commerce backend challenges such as duplicate orders, order snapshots, cart storage, inventory oversell, logistics updates, account balance consistency, read‑write splitting, hot‑cold data separation, and sharding, and provides concrete design patterns and SQL examples to solve them.

BackendIdempotencye‑commerce
0 likes · 17 min read
How to Build a Robust E‑Commerce Backend: Idempotency, Snapshots, and Sharding