MySQL 8.2 adds native read/write separation using MySQL Router 8.2 with InnoDB ReplicaSet, automatically routing SELECT queries to replicas and writes to the primary without application changes, with transaction-aware routing and configurable access modes.
The article explains why high‑traffic MySQL deployments hit performance limits, introduces the concepts of vertical and horizontal sharding, and provides a step‑by‑step guide—including necessity assessment, shard key selection, schema design, middleware integration, and data migration—using an e‑commerce order system as a concrete example.
The article explains why sharding should be a last‑resort scaling option, describes vertical and horizontal partitioning, compares hash, range and directory sharding schemes, discusses hotspots, routing, data colocation, migration steps and tool choices, and warns about the operational complexity involved.
The article explains why Redis must be highly available and walks through three progressive architectures—master‑slave replication, Sentinel automatic failover, and Redis Cluster—detailing their mechanisms, advantages, drawbacks, and when to choose each for small, medium, or large‑scale production systems.
The article walks through a real‑world order system that spikes to 300,000 QPS, explaining why the original single‑node MySQL design fails, and detailing a step‑by‑step evolution—index tuning, transaction fixes, read‑write splitting, vertical and horizontal sharding, plus data‑pipeline integration—to achieve stable low latency at massive scale.
The article analyzes why traditional offset‑based pagination breaks after data is split across multiple databases, explains the global query method with its pros and cons, and introduces the no‑skip optimization that reduces data transfer and improves deep‑page performance.
The article reviews the rise and decline of traditional database sharding, explains its technical pitfalls such as wrong shard keys, cross‑database joins, and distributed transactions, compares it with emerging NewSQL solutions like TiDB, OceanBase and PolarDB‑X, and offers practical criteria for choosing the right architecture.
The article revisits the rise and fall of database sharding, explains why it became problematic, and evaluates newer cloud‑native, distributed‑SQL, and serverless databases as modern replacements, offering a practical four‑step guide to help engineers choose the right solution for their workload and team.
The article dissects Alibaba’s second‑round interview question on handling 10 million daily order queries, exposing why a single sharding answer fails and presenting a comprehensive architecture‑seven‑suite combined with heterogeneous storage (MySQL, HBase, ClickHouse, ES, Redis, MQ) to achieve high concurrency, low latency, and reliable data consistency.
MySQL Router 8.2 now supports automatic read/write splitting for InnoDB ReplicaSet clusters, allowing reads to be routed to replicas and writes to the primary without any changes to application code, as demonstrated through configuration steps, connection examples, and practical observations.
Redis introduces a multithreaded query engine that keeps average latency under 10 ms while delivering up to 16× higher throughput for vector‑search workloads, enabling faster retrieval‑augmented generation (RAG) applications and outperforming pure vector databases and managed Redis services in benchmark tests.
This article details a comprehensive redesign of a high‑traffic order system, covering the challenges of massive data volume, concurrency pressure, and poor scalability, and presenting a step‑by‑step solution that includes sharding strategy selection, database product comparison, unique ID generation, middleware choice, migration plan, risk mitigation, and FAQ handling.
This article explains how Volcano Engine's veDB leverages compute‑storage separation, Kubernetes operators, and declarative operations to achieve extreme deployment density, seamless scaling, and high‑availability for millions of database instances, while addressing the challenges of traditional VM‑based deployments.
OpenAI runs a single primary PostgreSQL with about 50 replicas to serve roughly 800 million ChatGPT users, employing PgBouncer connection pooling, cache‑lock mechanisms, query and schema optimizations, workload isolation, multi‑layer throttling, and cascade replication, while acknowledging the architecture’s limits and the complexity of full sharding.
Increasing a database connection pool size often worsens performance under massive traffic because the bottleneck lies in hardware limits and management overhead, so developers must adopt async processing, read‑write splitting, caching, sharding, and proper pool tuning to sustain high concurrency.
Asynchronous ProcessingBackend Performanceconnection pool
MySQL 8.2 introduces native read/write splitting via MySQL Router, allowing automatic routing of reads to replicas and writes to the primary in an InnoDB ReplicaSet without any application changes.
When a single‑node MongoDB instance can no longer handle tens of millions of records, this guide walks you through the theory, architecture, deployment steps, shard key strategies, performance tuning, monitoring, backup, and troubleshooting needed to build a robust, production‑grade sharded cluster.
This article walks through the challenges of a growing order system, analyzes current bottlenecks, compares sharding and distributed database products, proposes a unique ID strategy, outlines sharding key selection, presents a migration plan with ShardingSphere‑JDBC, and discusses risks and FAQs for a robust backend architecture.
When a core order table with tens of millions of rows needs a new column, naïve ALTER TABLE can lock the table and disrupt services, so this article explores safe alternatives such as master‑slave switching, online DDL tools, extension tables, JSON‑based schema‑less designs, and clever reuse of existing redundant fields, complete with practical code snippets and lessons learned.
Redis has introduced a multithreaded query engine that dramatically reduces latency and increases throughput—up to 16×—for vector similarity searches, enabling vertical scaling and better support for real‑time RAG applications compared to traditional single‑threaded architectures and competing vector databases.
The article analyzes a JD interview question on scaling a MySQL order table from 100 million to billions of rows, detailing four practical expansion strategies—stop‑service migration, double‑write migration, ShardingSphere middleware, and dual‑VIP routing—each with step‑by‑step procedures, code samples, risk assessments, and validation methods to enable near‑instant capacity growth without service interruption.
This article explains the concepts, differences, advantages, and disadvantages of database sharding and partitioning, provides practical MySQL examples for horizontal sharding by region and range partitioning, and offers guidance on choosing the right approach for scalability and maintainability.
Designing high‑traffic, high‑concurrency systems requires careful planning across architecture, client optimization, CDN usage, clustering, caching, database tuning, and service governance, with principles such as statelessness, modularity, and fault‑tolerance to ensure scalability, reliability, and maintainability.
This comprehensive guide walks you through real-world MySQL performance tuning, covering slow query analysis, index design, query optimization, parameter tuning, hardware considerations, sharding, and fault handling, with practical scripts and case studies to help you transform sluggish databases into high‑throughput, scalable systems.
This article explains how Redis achieves high concurrency through non‑blocking I/O, an event‑driven reactor model, efficient I/O multiplexing, in‑memory data structures, and cluster sharding to support millions of simultaneous connections.
This article examines how auto‑increment primary keys work, the limits of common integer types, the failures that occur when those limits are reached, and practical strategies—including type upgrades, distributed ID generators, sharding, and sequence cycling—to prevent and mitigate ID exhaustion in high‑traffic MySQL and PostgreSQL deployments.
This comprehensive guide explains MongoDB sharding fundamentals, walks through step‑by‑step deployment of config servers, shard replica sets, and mongos routers, compares range, hash, and compound shard keys, and provides performance tuning, security, backup, monitoring, and troubleshooting best practices for production‑grade clusters.
This article explores the performance bottlenecks of a 700‑million‑row MySQL orders table and presents a comprehensive sharding solution—including vertical and horizontal partitioning, gene‑based shard keys, routing logic, global secondary indexes, data migration strategies, and common pitfalls—demonstrating how query latency can drop from seconds to milliseconds while supporting billions of records.
This article explores the pain points of a 700‑million‑row MySQL order table, presents vertical and horizontal sharding strategies, introduces gene‑based Snowflake IDs, details routing logic, migration steps, common pitfalls, and shows performance gains after applying the final architecture.
This article explains how to decompose a monolithic Nginx‑LNMP setup by moving the database, web nodes, and static assets to separate servers, configuring load balancing, shared storage, and session persistence with Redis, and provides complete command‑line examples for each step.
When MySQL auto‑increment IDs approach their limit, this guide explains six practical solutions—including expanding column types, using UUIDs, segmenting ID generation, composite keys, adjusting increment steps, and sharding databases—to ensure scalable, unique identifiers for large‑scale applications.
This article explains why MongoDB sharding is required for scaling storage and performance, describes the shard, config server, and mongos components, outlines range, hash, and compound sharding strategies, and provides practical guidance on shard key selection, balancing, backup, tuning, and security.
This article examines seven typical challenges that arise after implementing database sharding—such as global ID conflicts, cross‑shard queries, distributed transactions, shard‑key design, data migration, pagination, and operational overhead—and provides practical solutions and code examples to address each issue.
This article explains common data‑migration strategies such as hash‑modulo, time‑range and ID‑range sharding, and compares practical migration solutions including downtime migration, dual‑write, and an MQ + Redis approach, with detailed workflow diagrams and implementation tips.
The article examines why MySQL sharding (分库分表) is adopted for high‑traffic e‑commerce, outlines its inherent problems such as distributed transaction handling, index and global key constraints, operational overhead, and argues that distributed databases like OceanBase offer a more robust alternative.
This extensive series of 50 video lessons covers fundamental and advanced topics such as technology selection, capacity planning, micro‑service design, database scaling, caching strategies, load balancing, decoupling, layering, and service mesh, providing engineers with the knowledge needed to become proficient architects.
This article details the end‑to‑end architecture and step‑by‑step performance tuning of a high‑traffic lottery platform, covering server‑level rate limiting, application‑level throttling, semaphore usage, user‑behavior detection, caching strategies, database optimizations, and hardware upgrades that together enabled stable handling of millions of daily requests.
The article walks through how to assess MySQL table size, explains why large tables slow down due to B+‑tree index overhead, and compares three practical solutions—table partitioning, database sharding (horizontal, vertical, hash‑range), and hot‑cold archiving—while outlining their trade‑offs and selection criteria.
This article details the end‑to‑end architecture, rate‑limiting strategies, caching layers, database optimizations, and hardware upgrades that enabled a lottery service to handle daily traffic exceeding one million users during peak promotional events.
The article walks through every stage of a web request—from IPv4 addressing, NAT, and DNS resolution through CDN proximity and TCP stream handling, to HTTP framing, CGI versus FastCGI processing, stateless versus stateful server models, and finally database scaling techniques such as sharding and Redis caching.
This article explains how to build high‑concurrency systems by analyzing hardware and code optimizations on a single machine, exploring multi‑machine scaling, database and business‑application considerations, and presenting a practical inventory‑stock case that evolves from simple locking to partitioned, asynchronous, Redis‑based designs.
This article examines Pinterest's evolution through four development phases, the core technologies it adopted, and how it transitioned from complex clustering to a simpler sharding architecture to achieve scalable, high‑availability data storage.
The tutorial project showcases how to implement database sharding with ShardingSphere-JDBC (versions 4 and 5) in a SpringBoot O2O order service, providing modules for sharding, a gRPC ID generator, detailed configuration of four data sources, sharding algorithms, and Swagger-based testing of order creation and queries.
The article analyzes a recent outage of Ant's internal flash‑sale service built with AI‑generated low‑code, explains why the AI‑written business logic was not the cause, details the database capacity bottleneck that triggered a snowball effect, and discusses future automation and operational strategies to prevent similar failures.
This comprehensive guide explains why MySQL sharding (分库分表) is essential for handling massive data growth, outlines various horizontal and vertical partitioning strategies, discusses common challenges such as hot‑spot, cross‑database queries, pagination, and transaction handling, and compares popular middleware solutions.
This article details how Bilibili’s membership‑purchase platform tackled massive traffic spikes by redesigning the order call chain, introducing concurrent and asynchronous processing, and applying a sharding strategy that split databases and tables, ultimately boosting latency performance and supporting over 4,000 TPS during peak sales.
This article presents five expansion strategies—shutdown, write‑stop, log‑based, dual‑write, and smooth 2N—detailing step‑by‑step procedures for MariaDB installation, master‑master replication, dynamic data‑source configuration, and Keepalived high‑availability setup, enabling seamless horizontal scaling and minimal service disruption for large‑scale databases.
This article details eBay's event monitoring platform architecture, explains the challenges of high‑load OLAP workloads on ClickHouse clusters, describes the design and implementation of read/write separation and multi‑shard Keeper coordination, and shares concrete configuration snippets, performance observations, and production lessons learned.
The article describes how a finance team tackled a 50‑million‑row MySQL transaction table by analyzing the pain points, defining split goals, selecting sharding‑jdbc, designing multi‑source transaction handling, pagination across shards, data migration strategies, and a staged rollout to ensure stability and performance.
This article details how a finance team tackled a 50‑million‑row MySQL transaction table by designing a horizontal sharding strategy with Sharding‑JDBC, addressing performance bottlenecks, multi‑source transaction handling, pagination across shards, and a phased data migration and rollout plan.
This technical guide explains how to scale a sharded database horizontally by introducing five expansion schemes—including shutdown, write‑stop, log‑based, dual‑write, and smooth 2N approaches—covers MariaDB installation, master‑master replication setup, dynamic data‑source configuration with ShardingJDBC, and detailed Keepalived high‑availability configuration for seamless service continuity.
This article explains the concepts of database sharding, compares vertical and horizontal partitioning, discusses ID‑range and modulo sharding methods, analyzes the read‑amplification problem caused by non‑sharding keys, and presents practical solutions such as auxiliary index tables, Elasticsearch integration, and TiDB adoption.
The article explains the limitations of standalone MySQL for large‑scale workloads, explores sharding, read/write separation, and middleware approaches, and argues that modern distributed databases and cloud‑native solutions are the sustainable path forward for data‑intensive applications.
This article explores how YouTube handles massive video uploads and billions of daily views by employing a sophisticated backend stack—including MySQL clusters powered by Vitess, sharding, replication, caching, disaster recovery, and cloud‑native deployment on Kubernetes—ensuring scalability, reliability, and low‑latency delivery.
This article explains how to handle massive order volumes by classifying data into hot and cold tiers, storing them in MySQL, Elasticsearch, and Hive, and implementing sharding and partitioning strategies—including shard keys, modulo routing, and combined database‑table distribution—to achieve high throughput and low cost.
Even non-programmers notice MySQL’s growing pains—large tables, oversized databases, and read-heavy workloads—prompting a shift toward distributed databases, where sharding, read/write separation, and cloud-native solutions like TiDB or Aurora replace traditional middleware, marking the end of single-node MySQL dominance.
The article explains how traditional single‑node MySQL faces scalability challenges such as massive tables, resource exhaustion, and read‑heavy workloads, and argues that adopting sharding, middleware, or modern distributed databases like TiDB, OceanBase, Aurora, and PolarDB is the inevitable path forward for robust data management.
The article explains the limitations of standalone MySQL, outlines common scaling problems such as oversized tables and high read traffic, and compares middleware‑based sharding solutions with modern distributed databases, highlighting why the latter are becoming the preferred choice for growing applications.
The technical‑assurance team secured Bilibili’s high‑write live‑stream gift system by expanding capacity, isolating hot keys, refactoring pipelines, adding asynchronous writes, employing horizontal scaling and full‑link load testing, converting uncertain dependencies into graceful fallbacks, and deploying dual‑active, chaos‑engineered disaster‑resilience architecture aligned with business usage patterns.
Managing 20,000+ concurrent SQL queries requires moving beyond a single‑instance design; this guide explains why simple architectures fail, how to shard databases into multiple instances, route traffic using geographic hashing, and implement read‑write separation to sustain high read loads while keeping latency low.
This article explains the evolution from single‑node MySQL databases to master‑slave replication, various replication modes, high‑availability strategies, and both vertical and horizontal sharding techniques, while discussing the associated challenges such as distributed transactions, routing, and operational complexity.
This article explains why single‑node databases cannot keep up with rapid business growth and describes MySQL replication methods, high‑availability solutions such as MHA and MGR, and the challenges and techniques of vertical and horizontal sharding for large‑scale systems.
This article explains the evolution from single‑node MySQL to master‑slave replication, various replication modes, high‑availability solutions such as MHA and MGR, and the challenges and techniques of vertical and horizontal sharding for scaling large‑scale internet applications.
The article explains why simply adding hardware cannot keep up with growing database workloads and presents a three‑step approach—business isolation, horizontal sharding, and advanced reporting using PolarDB‑X—to achieve high concurrency, elastic capacity, and efficient caching for modern high‑traffic applications.
This article explains why and how to apply database sharding, introduces key terminology, compares global ID generation methods such as auto‑increment, UUID, COMB and Snowflake, outlines sharding algorithms, discusses challenges like distributed transactions, and presents practical expansion and implementation solutions.
In early 2021 Notion transformed its monolithic PostgreSQL into a horizontally partitioned fleet by sharding the block table and related data, using workspace IDs as the partition key, migrating through double‑write, backfill, and verification steps, and learning hard lessons about timing, zero‑downtime, and schema design.
This article explains MySQL's lack of a hard row limit, the industry rule of thumb around 5 million rows, how InnoDB buffer size and hardware affect performance, and demonstrates a simple insertion test that reveals slowdown when a table grows too large.
This article describes how a finance team tackled a 500 million‑row MySQL transaction table by analyzing pre‑split issues, defining sharding goals, selecting sharding‑jdbc, addressing multi‑datasource transaction and pagination challenges, designing a hybrid data‑migration plan, and executing a three‑stage rollout to ensure system stability and performance.
This article shares a practical experience of replacing a massive SQL Server data store with Elasticsearch, covering the motivations, architectural design, performance advantages, indexing mechanisms, data synchronization strategies, implementation details, and operational considerations for handling tens of billions of records.
This comprehensive guide walks through MySQL performance fundamentals, real‑world testing results, slow‑query detection, index impact, pagination pitfalls, count optimizations, sharding techniques, and step‑by‑step migration plans for large tables, equipping engineers with actionable tuning insights.
This article explains six typical system architecture designs, focusing on the first three—single‑database single‑app, content‑delivery, and read‑write separation—detailing their structures, advantages, disadvantages, and practical implementation steps for modern backend development.
Vitess 12, the latest major release of the MySQL clustering solution, introduces the experimental Gen4 query planner, enhanced VTAdmin multi‑cluster management, role‑based access control, updated benchmarking tools, and more inclusive naming conventions to improve scalability and cloud deployment.
This article explains the background, design criteria, and practical implementations of horizontal database sharding—including range, hash, gene, routing‑table, and consistent‑hash methods—while discussing common pitfalls, data‑skew mitigation, and both doubling and consistent‑hash expansion techniques for MySQL clusters.
This article details how a rapidly growing e‑commerce platform migrated from a single MySQL instance to a 16‑shard architecture using Sharding‑Jdbc, introduced consistent‑hashing to mitigate data skew, leveraged ES+HBase for multi‑dimensional queries, and implemented zero‑downtime migration strategies such as dual‑write and Canal replication.
This article explains how to build a highly available, scalable system by decoupling components, adding redundancy, choosing appropriate scaling strategies, employing load balancers, caching, CDN, database replication, sharding, and stateless design to support over 100 million users.
The article explains how Ant Group’s Alipay handles massive double‑11 payment traffic by using logical data centers (LDC), unit‑based architecture with RZone, GZone and CZone, traffic routing, disaster‑recovery strategies, and a CAP‑aware design built on the OceanBase distributed database.
The article explains how to scale MySQL for massive traffic by combining Redis caching, read‑write splitting, data archiving, and sharding techniques, while highlighting their benefits, implementation steps, and potential consistency challenges in modern systems.
The article outlines the essential duties, required competencies, decision‑making principles, and optimization techniques for software architects, covering system decomposition, technology selection, communication, technical depth, performance tuning, database scaling, and the broader mindset needed to lead complex development projects.
This article uses a fictional bank scenario to illustrate how database architectures evolve from a single MySQL instance through caching, master‑slave replication, vertical splitting, horizontal sharding, clustering, and finally the adoption of NoSQL and search engines to handle growing traffic and data complexity.
The article explains how to evolve a simple single‑instance database into a highly available, horizontally sharded system, detailing offline scaling, online dual‑write migration, and a smooth four‑node expansion process with configuration changes, data migration, and cleanup steps.
This article explains how to handle multi-table cross-database transactions by aligning sharding strategies for related tables, using a case study of shop_order_detail and shop_order_stat, and provides detailed formulas and TDDL configuration code to ensure consistent routing within the same database shard.
This case study details how JD’s "Browse" UGC project evolved from rapid agile delivery to a performance bottleneck as data grew, and how introducing Elasticsearch, redesigning the query flow, and refactoring storage components restored fast, flexible searches for both front‑end and operations users.
The 10th PostgreSQL China Technical Conference in January 2021 featured expert talks on scaling Citus clusters with logical replication and on applying PostgreSQL to Suning’s retail digitalization, highlighting architecture, challenges, technology choices, and future directions for database-driven business transformation.
This article details how Vivo transformed its monolithic e‑commerce order service into a scalable, service‑oriented system by applying data archiving, sharding‑JDBC based sharding, MySQL‑to‑Elasticsearch sync, zero‑downtime migration, and distributed‑transaction techniques, while sharing practical pitfalls and performance results.
This article details how Vivo's cloud service tackled explosive data growth by applying horizontal and vertical sharding, routing‑table based dynamic expansion, and MySQL InnoDB compression, providing step‑by‑step guidance, performance results, and practical recommendations for large‑scale database deployments.
During the pandemic, the Yunnan “Anti‑Epidemic” mini‑program faced over 80 W requests per minute, prompting an analysis of how cloud development’s serverless, auto‑scaling architecture—covering data pipelines, cloud functions, and cloud databases—can ensure high performance, availability, and cost‑effective handling of massive concurrent traffic.
This article explains the concept of read‑write splitting for databases, compares client‑side and proxy implementations, discusses data lag and forced routing techniques, and provides code examples and custom load‑balancing algorithms using Sharding‑JDBC and Spring.
This article explores the end‑to‑end design of a flash‑sale (秒杀) system, detailing business flow, technical challenges, architectural principles, front‑end and back‑end design, concurrency handling, anti‑cheating measures, data safety, and practical solutions for high‑traffic e‑commerce scenarios.
This article explains how to extract maximum performance and scalability from PostgreSQL for multi‑terabyte workloads by leveraging specialized indexes, declarative partitioning, tablespaces, parallel query execution, read‑only replica load‑balancing, and foreign‑table sharding techniques.
This article shares a detailed case study of how a startup managed a massive cinema‑ticket dataset with PostgreSQL, evaluated cloud and self‑hosted options, refined materialized‑view strategies, built a reliable job queue, and distilled practical takeaways for large‑scale database operations.
This article details a comprehensive experiment that evaluates MySQL's ability to sustain up to 100,000 simultaneous connections using Percona Server's thread‑pool plugin, sysbench clients, adjusted OS/network limits, and multiple IP addresses, presenting hardware specs, configuration steps, observed errors, performance metrics, and practical recommendations.
This article objectively compares NewSQL databases with middleware‑based sharding approaches, examining their architectural differences, transaction models, CAP implications, high‑availability designs, scaling mechanisms, storage engines, maturity, and provides a decision guide to help engineers choose the right solution for their workloads.
Zhihu’s Moneta service, handling over 1.3 trillion rows and billions of daily writes, migrated from MySQL sharding to TiDB, achieving millisecond query latency, high availability, and horizontal scalability, while sharing architectural choices, performance metrics, migration challenges, and future expectations for TiDB 3.0.
This article explains how Ant Financial’s massive Double‑11 payment traffic is handled through logical data centers (LDC), unit‑based architecture (RZone, GZone, CZone), traffic routing, disaster‑recovery strategies, and a CAP analysis that highlights the role of OceanBase’s Paxos‑based consensus in achieving high availability and eventual consistency.
Zhihu's Moneta service stores over a trillion rows of user‑read posts and, facing massive write throughput and strict sub‑90 ms query latency requirements, migrated to the open‑source HTAP database TiDB, achieving horizontal scalability, high availability, and dramatically improved performance.
The article details how Alipay engineers tackled the massive traffic spikes of Double 11 by addressing external payment bottlenecks, implementing recharge‑based balances, building capacity‑planning platforms, adopting logical data‑center (LDC) and CRG zone architectures, deploying elastic scaling, and evolving their OceanBase database and service‑mesh infrastructure to sustain millions of transactions per second.
This article details how Weibo leverages Redis across billions of requests, describing its massive scale, the challenges of trillion‑level reads/writes, the technical choices and customizations made—including LongSet, HA solutions, multi‑level caching, RocksDB integration—and outlines ongoing capacity and future development strategies.
This article explores practical techniques for handling massive, high‑concurrency data workloads, covering relational database limits, read/write separation, vertical and horizontal sharding, key selection, archival to NoSQL stores, and the use of heterogeneous index tables to maintain performance.
A MySQL‑based receipt‑tracking service suffered a sudden 100‑fold load increase, prompting a step‑by‑step optimization that combined read‑write splitting, middleware‑less data routing, columnar storage with Infobright, and dynamic scheduling to dramatically lower CPU/IO pressure and restore performance.
This article outlines ten crucial design principles for building robust microservice systems, covering API gateways, stateless services, database scaling, caching strategies, service decomposition, orchestration, configuration management, logging, fault tolerance, and comprehensive monitoring, each illustrated with clear diagrams.