This article explains how to assess MySQL table size using table capacity, disk usage, and instance limits, reveals why large tables slow down due to B+‑tree overhead, and presents three practical solutions—partitioning, sharding, and hot‑cold archiving—to restore query performance.
This article walks through a real e‑commerce MySQL performance crisis, showing how to pinpoint bottlenecks, analyze slow‑query logs, use EXPLAIN, add composite indexes, rewrite SQL, apply partitioning, read/write splitting and caching, and achieve sub‑second response times with 99% CPU reduction.
This article explains why interviewers probe sharding, walks through the typical chain of questions, compares partition tables and read/write splitting, shows how to decide between sharding strategies, estimate capacity, plan expansion, and safely migrate data with traffic replication.
This article walks through the end‑to‑end design, configuration, and implementation of splitting massive loan and repayment tables into 50 sharded MySQL tables using ShardingSphere, Spring Boot, and a custom suffix algorithm, while covering data migration, DBA coordination, dynamic switches, and scheduled consistency checks.
The article walks through why sharding is needed, outlines IO and CPU bottlenecks, presents two design principles, shows how to estimate capacity from existing data and growth trends, compares range, modulo, consistent‑hash and Snowflake sharding schemes, and details migration strategies for expanding nodes without downtime.
This article explains how to handle MySQL order tables that have grown to billions of rows by applying vertical and horizontal sharding, gene‑based partitioning, cross‑shard query strategies, data migration techniques, and practical pitfalls to achieve massive performance improvements.
This article explains how to assess MySQL table size from table capacity, disk usage, and instance limits, describes why large tables degrade performance due to B+‑tree depth, and presents three practical solutions—partitioning, sharding, and hot‑cold archiving—to improve query speed and manage growth.
This article explains the fundamental concepts of database sharding—including data nodes, logical and broadcast tables, sharding keys, strategies, algorithms, SQL parsing, routing, rewriting, execution, result merging, distributed primary keys, data masking, distributed transactions, data migration, and shadow databases—providing clear examples and code snippets for real‑world implementation.
This article details the background, challenges, and technical solutions behind the partitioning redesign of the KPOS reporting system on OceanBase, covering deployment choices, parameter tuning, data migration, phased rollout, performance gains, encountered issues, and future scaling plans.
When a MySQL table grows to tens of millions of rows, insert and query latency spikes, schema changes become painful, and only a subset of data is needed; this guide explains how to assess table size, why large tables slow down, and three practical solutions—partitioning, sharding, and hot/cold archiving—along with their trade‑offs and implementation steps.
This article describes how a senior architect designed, configured, and implemented a 50‑table sharding solution for loan and repayment request data using ShardingSphere, Spring Boot 3, MySQL, and custom synchronization scripts, while also addressing historical data migration, backend query changes, and operational safeguards.
This article examines the fundamental drawbacks of MySQL sharding in large‑scale e‑commerce, highlighting distributed transaction challenges, limited cross‑shard queries, global key constraints, scaling difficulties, operational overhead, and why distributed databases may be a better long‑term solution.
This article walks through evaluating massive MySQL tables, explains why large row counts slow queries, and compares three practical solutions—table partitioning, database sharding, and hot/cold archiving—while highlighting their trade‑offs and offering guidance on selecting the right approach for a given workload.
This article details the design, planning, and step‑by‑step implementation of a database table‑partitioning solution for loan and repayment request tables, covering schema decisions, sharding strategy, historical data migration, dynamic synchronization, code examples with ShardingSphere, SpringBoot, and operational considerations for seamless migration.
After a year of business coding, the author details the design, implementation, and migration strategy for splitting loan and repayment tables into 50 sharded tables using ShardingSphere with Spring Boot, covering schema design, data synchronization, dynamic switches, and practical pitfalls.
The article describes how a one‑year‑old loan/repayment service was refactored by introducing a 50‑table sharding scheme with ShardingSphere, detailing the design decisions, historical data migration, code changes, configuration files, pitfalls, and runtime synchronization strategies to achieve efficient query performance on billions of rows.
This article explains the motivation, architecture, resource estimation, code generation algorithms, data modeling, database partitioning, caching strategies, and implementation details for building a scalable short URL system that handles billions of links and high QPS workloads.
This article explains how to design a SaaS system that isolates each subsidiary's data by adding a tenant_id column to a shared MySQL table, converting the table to LIST‑partitioned storage, and enforcing tenant‑level access through stored procedures and a MyBatis interceptor, complete with code examples and testing steps.
This article demonstrates how to convert a single‑tenant MySQL application into a multi‑tenant SaaS solution by adding a tenant_id column, using shared‑database partitioning, creating tenant tables, writing stored procedures for dynamic partition management, and implementing a MyBatis interceptor for hierarchical data permissions.
This article walks through converting a single‑tenant MySQL application into a SaaS solution by adding a tenant_id column, using shared‑database table partitioning, creating tenant metadata tables, writing stored procedures to add columns and partitions, and implementing a MyBatis interceptor for automatic data‑permission enforcement.
This article introduces the essential concepts of sharding architecture—including data partitioning, logical and physical tables, sharding keys, routing strategies, SQL rewriting, distributed primary keys, data masking, transactions, migration, and shadow databases—providing a comprehensive guide for building high‑performance, horizontally scaled database systems.
This article explains why database sharding is needed, the problems caused by oversized databases and tables, and how to apply vertical and horizontal splitting using MyCat, including configuration examples, splitting strategies, and testing procedures.
This article explains how to evaluate MySQL table size, understand B+‑tree limitations, and improve query performance by using table partitioning, horizontal/vertical sharding, and hot‑cold data archiving, while also comparing the trade‑offs of each approach for practical selection.
This article explains the technical reasons why a MySQL single table should not exceed about 20 million rows, covering limits from auto‑increment primary keys, data‑page structures, B+‑tree storage calculations, and practical partitioning recommendations for large‑scale applications.
This article details a practical approach to handling massive billing data by dynamically creating monthly tables in MySQL via stored procedures, integrating them with a SpringBoot‑MyBatis backend, and providing APIs for insertion, serial‑number lookup, and time‑range queries, complete with code examples and operational tips.
This article explains why relational databases hit performance bottlenecks at large scale, introduces vertical and horizontal sharding techniques, compares their pros and cons, discusses common challenges such as distributed transactions, joins, pagination and global key generation, and offers practical solutions and middleware options.
The article explains the principle of hot‑cold data separation, when it should be applied, how to distinguish hot versus cold data, and three practical implementation approaches—code modification, binlog listening, and scheduled scanning—to improve database performance and maintain consistency.
This article explains how to smoothly migrate a legacy single‑database system to a sharded architecture by using dual‑write, mapping‑key routing, custom transaction handling, and offline/real‑time diff checks, while detailing the underlying MyBatis integration and component‑based design.
This article introduces the fundamental concepts of database sharding, including data fragmentation, nodes, logical and physical tables, broadcast and single tables, sharding keys, strategies, algorithms, binding tables, SQL parsing, routing, rewriting, execution, result merging, distributed primary keys, data masking, distributed transactions, migration, and shadow databases, providing a comprehensive guide for designing and operating a sharded architecture.
This article explains how to assess MySQL table size, identify performance issues caused by massive data volumes, and presents three practical solutions—table partitioning, database sharding, and hot‑cold data archiving—along with detailed SQL examples and considerations for implementation.
This article explains why large MySQL tables cause slow queries, analyzes table size, disk usage, and instance capacity, and presents three practical solutions—table partitioning, database sharding (horizontal/vertical), and hot‑cold data archiving—helping developers choose the appropriate strategy for their workloads.
This article explains the concepts, strategies, and practical scenarios for MySQL database partitioning, covering both table splitting (vertical and horizontal) and database splitting (vertical and horizontal), with examples, advantages, and when to apply each technique to handle large-scale data workloads.
This article explains how to assess MySQL table size, why massive tables degrade performance due to B+‑tree depth, and presents three practical solutions—table partitioning, database sharding, and hot‑cold data archiving—along with guidance on selecting the appropriate approach.
Facing massive user and order data growth, this article examines storage options, compares relational, NoSQL, and NewSQL databases, and provides a detailed guide to implementing MySQL‑based sharding with proxy and client modes, key selection, partition strategies, migration steps, and future scaling considerations.
This article explains the fundamentals of database sharding, why and when to apply it, various vertical and horizontal splitting strategies, routing algorithms, common pitfalls such as pagination and distributed transactions, and compares client‑side and proxy‑based ShardingSphere deployment patterns.
This article explains why high‑concurrency big‑data systems need database sharding, describes table‑level partitioning methods (modulus and range), shows how to implement them with MySQL and Java code, compares vertical and horizontal database splitting, and evaluates middleware solutions such as Sharding‑JDBC and Mycat.
This article explains how to handle billions of order records by classifying data into hot and cold tiers, using MySQL sharding with database‑and‑table partitioning, storing cold data in Elasticsearch and Hive, and outlines combined routing formulas for scalable backend architecture.
The article explains how to select and implement routing keys for database sharding in an order platform, covering time‑based and business‑related keys, user and merchant scenarios, hash‑modulo strategies for single and multi‑database environments, and the overall data flow.
This article explains the three aspects of database sharding—database‑only, table‑only, and both—detailing when to apply each, how to choose sharding keys, common algorithms, ID generation strategies, available open‑source tools, and the new challenges introduced by sharding.
This article explains the concepts, methods, advantages, and drawbacks of database sharding, covering vertical and horizontal partitioning, sharding rules, global primary key generation, distributed transaction issues, cross‑node queries, pagination, data migration, and practical case studies, with code examples and diagrams.
The article explains the hot‑cold data separation pattern, describing its purpose, when to use it, how to distinguish hot versus cold data, and three practical implementation approaches—code modification, binlog listening, and scheduled scanning—to improve performance and maintain data consistency in large‑scale systems.
This article explains the concepts, types, advantages, and challenges of database sharding—including vertical and horizontal partitioning—covers when to consider splitting, practical strategies for handling transactions, joins, pagination, global IDs, and lists popular middleware solutions for implementing sharding in production.
This article explains the concepts, characteristics, and practical scenarios of database sharding—including vertical and horizontal table partitioning as well as vertical and horizontal database partitioning—helping developers decide when and how to apply each strategy to improve scalability and performance.
This article explains MySQL's native horizontal sharding using MERGE tables, covering the concepts of horizontal versus vertical splitting, table‑level sharding and partitioning, practical examples, advantages, limitations, and suitable use cases for MERGE tables.
This article explains the concepts, characteristics, and practical scenarios of vertical and horizontal table partitioning as well as vertical and horizontal database sharding in MySQL, helping developers decide when and how to apply each strategy for large‑scale applications.
This article explains why monolithic systems need to be split, outlines how to assess business complexity, define service boundaries, and execute database vertical and horizontal partitioning with global ID generators, migration strategies, cross‑database query refactoring, cut‑over plans, and post‑split stability measures.
This article explains the concepts, types, advantages, and drawbacks of vertical and horizontal database sharding, discusses the challenges such as distributed transactions, cross‑node joins, pagination, global primary‑key generation, migration, and provides practical guidelines and case studies for deciding when and how to apply sharding.
This comprehensive guide explains why and when to split relational databases, details vertical and horizontal sharding techniques, discusses associated challenges such as distributed transactions, cross‑node joins, pagination, global primary keys, and offers practical solutions and middleware options.
This article explains the fundamentals of database sharding, covering both vertical and horizontal partitioning concepts, routing algorithms, advantages and drawbacks, practical implementation steps using sharding-jdbc, and discusses related challenges such as distributed transactions, pagination, global IDs, and tool selection.
Sharding splits a single database into multiple servers to improve performance, covering vertical and horizontal partitioning, middleware options, and key challenges such as distributed transactions, cross‑node joins, ID generation, pagination, routing transparency, and choosing between frameworks or custom solutions.
This guide details recommended MySQL versions for Zabbix, proper grant statements for read/write and read‑only users, essential MySQL configuration tweaks, SQL techniques for item and function updates, read‑write splitting, upgrade procedures, backup strategies, and comprehensive partition maintenance scripts to improve performance and manage data growth.
This article explains MySQL performance bottlenecks, compares IO and CPU limits, introduces horizontal and vertical sharding and partitioning concepts, presents practical scenarios, outlines common tools such as Sharding‑Sphere, TDDL and Mycat, and provides step‑by‑step guidance for implementation, troubleshooting and scaling.
This article explains why relational databases become bottlenecks at large scale, introduces vertical and horizontal sharding techniques, discusses the challenges of distributed transactions, cross‑shard queries, pagination, and global primary‑key generation, and provides practical guidelines and middleware options for implementing sharding in production systems.
The article explains why sharding (splitting databases and tables) is the preferred solution for handling massive user, order, and transaction data in high‑traffic internet applications, comparing it with partitioning and NoSQL/NewSQL alternatives, and detailing practical middleware choices, sharding column selection, and integration with Elasticsearch and HBase.
This article explains vertical and horizontal database partitioning, compares hash‑modulo and range sharding schemes, and proposes a hybrid approach that balances load, prevents hotspots, and eliminates the need for costly data migrations during scaling.
This article shares a practical journey of handling MySQL tables that grew beyond a billion rows, describing a temporary backup‑table fix, a custom hash‑based sharding implementation, data migration tactics, validation steps, and key lessons learned for large‑scale database management.
Database bottlenecks often require sharding; this guide explains core concepts of database partitioning, global ID generation methods, shard allocation strategies, smooth scaling techniques, and popular solutions, helping engineers decide when and how to implement effective sharding without premature over‑design.
This article explains why modern applications dealing with massive user, order, and transaction data must move beyond single‑table designs, compares partitioning, sharding (分库分表), and NoSQL/NewSQL solutions, and provides practical guidance on choosing middleware, sharding columns, and hybrid architectures such as es + HBase.
This article describes the background, analysis, solution selection, architecture redesign, execution steps, and results of a large‑scale order table splitting project that separates hot and cold data, applies horizontal sharding, and uses MySQL partitioning to control table size and boost query performance.
This article explores MySQL sharding fundamentals, detailing vertical, horizontal, and combined partitioning methods, their advantages and drawbacks, and surveys various open‑source implementations such as MySQL Proxy, HSCALE, Amoeba, HiveDB, and PL/Proxy, while offering practical guidance on configuration, transaction handling, and common pitfalls.
This article explains the motivations behind database sharding, defines the concept, explores its historical background, discusses storage stretching, indexing and consistency challenges, and presents a practical sharding design example for transaction systems.
This article explains what MariaDB Spider is, its role as a pluggable storage engine for MySQL/MariaDB sharding, demonstrates vertical and horizontal partitioning with real‑world Tencent game case, provides installation steps, configuration commands, performance considerations, tunable parameters, and load‑balancing architecture.
An experienced DBA shares a real-world case of optimizing a nightly Oracle reporting pipeline, detailing how partitioned tables, truncate operations, table compression, disabling logging, and parallel execution reduced a 20‑hour data extraction to under one hour, and compiles community‑sourced tuning tips for large‑scale data warehouses.
This article walks through Oracle 11g's Reference Partition feature, demonstrating table creation, handling ORA‑14652 errors, inserting data, and analyzing partition alignment between master and detail tables to reveal performance and management benefits.
This article explains the core concepts of database sharding, detailing vertical and horizontal partitioning methods, strategies for combining them, handling shared data, and addressing challenges such as distributed transactions, cross‑node joins, and aggregation queries, with practical insights for scalable system design.
