This article provides a step‑by‑step tutorial on configuring MySQL master‑slave replication using Docker, explains redo log and binlog mechanisms, and demonstrates how to implement Sharding‑JDBC read/write splitting and database sharding with Spring Boot, including code examples and configuration details.
This article provides a comprehensive overview of the design principles, architectural patterns, performance, availability, scalability, security, and operational considerations for building large distributed web sites, illustrated with a step‑by‑step evolution from a single‑server setup to a multi‑layer, cloud‑native architecture.
The article explains how Ant Financial scales Double‑11 payment traffic to hundreds of thousands of TPS by employing logical data centers (LDC), unit‑based system design (RZone, GZone, CZone), database sharding, CAP theorem analysis, Paxos‑based consensus, and the OceanBase distributed database, while also detailing disaster‑recovery and traffic‑shifting mechanisms.
Effective MySQL sharding requires sustainable, low‑skew designs, favoring hash‑based methods with proper coprime counts, two‑stage partitioning, routing tables, or consistent hashing, while supporting expansion via doubling or flexible consistent‑hash growth to avoid hot spots and uneven data distribution.
This article chronicles JD Baitiao's data‑architecture evolution from Solr + HBase to MongoDB and finally to Apache ShardingSphere, highlighting the challenges of massive data growth, the need for decoupling, and the performance, scalability, and operational benefits achieved by adopting ShardingSphere.
This article provides a comprehensive guide on using ShardingSphere-JDBC for read‑write splitting, configuring MySQL master‑slave replication with Docker, explaining redo log and binlog mechanisms, demonstrating two‑phase commit, and showing how to implement database sharding and distributed primary keys in a Spring Boot application.
This article explains how to decide between database and table sharding, describes vertical, horizontal, and range partitioning techniques, discusses their trade‑offs and implementation details, and offers practical guidance for solving query challenges in high‑traffic, write‑intensive systems.
The article explains MySQL vertical partitioning techniques—including instance‑level database splitting, schema‑level splitting, and column‑level table splitting—detailing their advantages, drawbacks, and step‑by‑step command‑line procedures with code examples for implementing each method in production environments.
This article introduces Sharding-JDBC (now ShardingSphere), explains core sharding concepts such as shards, data nodes, logical and physical tables, sharding keys, algorithms and strategies, shows how it extends JDBC, and provides a step‑by‑step Spring Boot + MyBatis‑Plus example for building a sharded database application.
The article outlines the defining traits of large-scale websites—high concurrency, massive data, 24/7 availability, security, and rapid iteration—and discusses major technical challenges such as scaling, caching, clustering, database read/write separation, CDN acceleration, distributed storage, and service decomposition.
Shopee’s Logistics Channel Service migrated from a shared‑cluster MySQL setup to dedicated clusters, added TiDB for transient tracking data, and ultimately adopted hash‑based sharding with separate order and tracking services plus an asynchronous compensation mechanism, enabling uniform data distribution, cross‑database consistency, six‑month archiving, and scalable growth without rebalancing.
The article explains how to transition from client‑side sharding with Sharding‑JDBC to proxy‑side sharding using Rainbow Bridge DAL by adding new, switchable data sources, employing AbstractRoutingDataSource and configuration flags for gradual migration, while addressing read‑write separation, unsupported SQL functions, and ensuring backward compatibility.
The article explains what high concurrency means, outlines key performance metrics such as response time, throughput, QPS, TPS and concurrent users, and presents vertical and horizontal scaling techniques—including hardware upgrades, caching, load balancing, sharding, micro‑services, multithreading and CDN—to improve a system's ability to handle massive parallel requests.
This article outlines the career path toward software architecture, detailing the market expectations, essential competencies such as project management, technical depth, business insight, communication, decision‑making, and optimization techniques for systems, backend services, and databases, while emphasizing continuous learning and community involvement.
This article introduces ShardingSphere‑Jdbc as a lightweight Java JDBC framework, details step‑by‑step Docker‑based MySQL master‑slave replication configuration, reviews redo log and binlog mechanisms, explains two‑phase commit, and demonstrates read‑write splitting and sharding strategies with Spring Boot examples.
This article explains how to use Spring Boot to quickly integrate ShardingSphere for database sharding, covering the background of massive data storage challenges, the advantages of ShardingSphere, step‑by‑step project setup, configuration of data sources and sharding strategies, and a complete test example.
This guide walks through configuring ShardingSphere with Oracle in a Spring Boot‑MyBatis‑JPA project, shows custom sharding algorithms, entity/DAO/service code, and details the step‑by‑step debugging that uncovered schema handling bugs and version‑specific quirks, ending with a working solution.
This article explains the performance bottlenecks of relational databases, introduces sharding concepts such as horizontal, vertical, and table partitioning, discusses associated challenges like distributed transactions and global primary keys, and provides practical solutions and tool recommendations for implementing effective database partitioning.
The article explains how increasing business complexity and system throughput drive the need for system splitting, decoupling, and architectural evolution—covering horizontal scaling, vertical and business splitting, database sharding, caching, and the transition to micro‑services to improve capacity, stability, and performance.
The article explains how traditional scaling methods like sharding and RPC eventually hit connection limits, and introduces unitization as a strategy to achieve unlimited service expansion by limiting each application to a single database instance.
This article explains how, in a Spring Boot 2.2.13 environment with ShardingSphere 4.0.1, enabling data encryption causes the expected ShardingDataSource bean not to be created, walks through the configuration and source code analysis, and provides a custom bean solution to resolve the issue.
This article provides a comprehensive technical overview of large‑scale distributed website architecture, covering characteristics, goals, common patterns, high‑performance and high‑availability designs, scalability, extensibility, security, agility, a seven‑layer reference model, and the evolutionary steps of modern e‑commerce systems.
This article explains why database performance bottlenecks arise, outlines criteria for deciding when to shard databases and tables, and provides detailed strategies for SQL tuning, table restructuring, horizontal and vertical splitting, single‑ versus multi‑database sharding, as well as handling cross‑database joins, distributed transactions, pagination, and ID generation.
This article examines common distributed ID generation methods—including UUID, MySQL auto‑increment, multi‑instance auto‑increment, Snowflake, and Redis—detailing their advantages, drawbacks, and practical use cases, and presents advanced designs such as database‑driven ID blocks, concurrency handling, and double‑buffer techniques for high‑traffic systems.
This article explains vertical and horizontal database partitioning, compares hash‑modulo and range sharding schemes, and proposes a hybrid group‑based design that balances uniform data distribution with painless scaling, eliminating hotspots while avoiding costly data migrations.
The article explains vertical and horizontal database sharding, compares hash-modulo and range partitioning schemes with their advantages and drawbacks, and proposes a hybrid group‑based design that balances load, avoids hotspots, and simplifies scaling without costly data migrations.
This article explains what database sharding (splitting databases and tables) is, why it improves performance and availability, how to choose between horizontal and vertical partitioning, design ID generation, handle distributed transactions, migrate data smoothly, and configure Alibaba's TDDL middleware with practical code examples.
This article explains vertical and horizontal database sharding, compares hash‑modulo and range‑based partitioning schemes, and discusses their advantages, drawbacks, and the impact on scaling and hotspot problems in large‑scale applications.
This article explains how to split large-scale database tables using vertical and horizontal partitioning, compares hash-modulo and range-based sharding, and proposes a hybrid approach that leverages both methods to balance data distribution, avoid hotspot issues, and eliminate the need for costly data migrations during scaling.
The article outlines a microservice‑based user‑center architecture for handling over one hundred million users, detailing gateway, core, and async services, interface design, vertical and horizontal database sharding, token graceful degradation, multi‑layer password security, event‑driven async processing, and comprehensive monitoring for high availability, performance, and security.
This article explains how Ant Financial’s payment system grew from 20,000 transactions per minute in 2010 to 540,000 TPS in 2019 by adopting logical data centers (LDC), unitized architecture (RZone, GZone, CZone), OceanBase’s Paxos‑based consensus, and sophisticated traffic steering and disaster‑recovery strategies.
This article explains why large tables hurt performance, defines database sharding and its vertical and horizontal strategies, compares partitioning by key, size, or time, and outlines practical implementation options and migration steps for robust, scalable backend systems.
This article outlines the step‑by‑step evolution of a high‑traffic server architecture—from a single‑machine deployment to distributed clusters, caching, load balancing, database sharding, microservices, containerization, and cloud platforms—highlighting the technologies involved at each stage and summarizing key design principles for scalable, highly available systems.
The article explains why growing business complexity and throughput demands force system decomposition and architectural upgrades, detailing horizontal scaling, vertical and business splitting, database sharding, caching strategies, and the evolution from monolithic to micro‑service structures.
The article explains how increasing business complexity and throughput demands drive system splitting, architectural evolution, and the adoption of scaling, sharding, caching, and asynchronous messaging techniques to improve capacity, robustness, and maintainability of large‑scale backend services.
This article explains how Ant Financial’s logical data center (LDC) architecture uses unitization, database sharding, and CAP‑aware design—including RZone, GZone, and CZone—to achieve tens of millions of TPS during Double‑11, while providing multi‑active disaster recovery and high availability.
This article analyzes common database performance bottlenecks, explains when to apply IO‑ or CPU‑focused optimizations, and provides a detailed guide to horizontal and vertical sharding, partitioning strategies, tooling choices, implementation steps, and practical pitfalls for scalable systems.
This article explains why monolithic systems need to be split, how to assess business complexity, define service boundaries, perform vertical and horizontal database sharding, adopt global ID generators, migrate data safely, and ensure consistency and stability during cut‑over using stop‑write or dual‑write strategies.
This article introduces Sharding-JDBC, explaining its basic usage, core concepts, sharding strategies, SQL processing workflow, configuration examples, and code snippets, while emphasizing the importance of understanding its underlying principles for effective database horizontal partitioning in Java applications.
This article analyzes the business flow of flash‑sale (seckill) systems, identifies ten technical challenges such as high concurrency, bandwidth, stock‑deduction and cheating, and presents architecture principles, design patterns, code examples, and safety mechanisms to build a reliable, scalable backend for flash‑sale scenarios.
This article walks through the step‑by‑step evolution of a Java‑based e‑commerce system—from a single‑server monolith to a fully distributed architecture with load balancing, read/write separation, caching, sharding, and service decomposition—highlighting the motivations, challenges, and key design decisions at each stage.
This article compiles the most frequently asked MySQL‑related questions from developers and operations teams in banks, covering why MySQL is chosen over Oracle, compatibility gaps, migration methods, sharding strategies, performance tuning, consistency guarantees, and resource planning.
This article walks through the step‑by‑step evolution of a Java‑Web e‑commerce system—from a single‑server monolith to clustered services, read‑write separation, search integration, caching, sharding, and finally service decomposition—highlighting the architectural decisions and challenges at each stage.
This article walks through eight progressive architecture upgrades—starting with physically separating the web server and database, adding page and fragment caches, scaling web servers, sharding databases, and finally adopting distributed caching—to illustrate how a high‑traffic site can maintain performance and reliability.
This article provides a step‑by‑step tutorial on using Sharding-JDBC with Spring Boot to implement horizontal database and table sharding across multiple MySQL instances, including project setup, Maven dependencies, entity and controller code, sharding configuration, and testing via Postman.
This article walks through the step‑by‑step evolution of a high‑concurrency backend—from a single‑machine setup to a cloud‑native, micro‑service architecture—highlighting the technical challenges, key technologies, and design principles at each stage.
This article recounts the 2007 challenges faced by Taobao's monolithic MySQL setup and explains how the TDDL middleware introduced SQL optimization, sharding, read‑write separation, caching, and dynamic scaling to transform the system into a distributed, high‑performance database solution.
This article explains common I/O and CPU bottlenecks in databases, compares horizontal and vertical sharding techniques, outlines practical partitioning strategies, introduces popular sharding tools, and provides step‑by‑step guidance for implementing and scaling sharded architectures.
This article demonstrates how to set up Sharding-JDBC in a Spring Boot project, covering project structure, Maven configuration, entity, controller, service, mapper code, database DDL for multiple shards, sharding rules in application.properties, and API testing with Postman.
This article examines how Ant Financial redesigned its payment system’s data layer after retiring mainframes, detailing RTO/RPO goals, CAP trade‑offs, vertical and horizontal sharding, blacklist‑based accounting DR, failover mechanisms for transactions, and the role of OceanBase in achieving strong consistency and low‑latency recovery.
The article explains why and when to apply database sharding, describes horizontal and vertical partitioning methods, lists common sharding tools, and discusses challenges such as distributed transactions, join avoidance, pagination, and global primary key generation.
This article explains how to use ShardingSphere‑JDBC to achieve horizontal database sharding and read/write splitting in a Spring Boot application, covering ShardingSphere concepts, configuration, entity and DAO code, custom sharding algorithm, and practical testing with multiple MySQL instances.
This article recounts the four‑year journey of Ele.me's order platform, detailing the transition from a monolithic Zeus system to service‑oriented components, the challenges of sharding, message broadcasting, performance testing, Redis optimization, and the cultural practices that shaped a resilient backend architecture.
This article explains how Ant Financial’s LDC (Logical Data Center) architecture, unitized RZone/GZone/CZone design, OceanBase database, and CAP-aware strategies enable the payment platform to handle double‑11 traffic peaks of over 540,000 transactions per second while ensuring high availability, disaster recovery, and eventual consistency.
This article walks through the step‑by‑step evolution of a Java‑based e‑commerce platform, illustrating how a single‑machine deployment can be transformed into a scalable architecture using techniques such as server‑database separation, clustering, load‑balancing algorithms, read‑write splitting, caching, sharding, micro‑services and message queues.
This article explains common database performance bottlenecks such as IO and CPU limits, introduces horizontal and vertical sharding concepts, compares sharding tools, outlines practical implementation steps, discusses common pitfalls, and provides scaling strategies to keep your data layer responsive under heavy load.
This article examines Taobao’s pivotal architectural transformation from a monolithic codebase to a large‑scale distributed system, detailing the challenges of personnel, business complexity, Oracle bottlenecks, and hardware limits, and describing the comprehensive solutions involving code and interface vertical splitting, custom middleware, database sharding, and new infrastructure.
The article summarizes Xu Wenqi's 2019 Alibaba Cloud Summit talk on Ant Financial's distributed architecture, covering the shift from monolithic to microservices, modular development, load‑balancing, database sharding, the distributed TA system, task scheduling, gray‑release, full‑link stress testing, and OceanBase high‑availability solutions.
This article explains why and when to split databases, compares vertical and horizontal sharding methods, outlines practical steps for determining split mode, fields, rules and quantity, and discusses scaling, read/write separation, and common challenges with solutions.
The article describes how a traditional monolithic ERP system can be transformed into a scalable, high‑performance solution by splitting functional modules into independent micro‑services, using separate databases, adopting distributed transaction patterns, and optionally adding a dedicated reporting subsystem.
This comprehensive guide explains the motivations, methods, and trade‑offs of database sharding—including vertical and horizontal partitioning, sharding rules, handling cross‑node joins, distributed transactions, global primary key generation, and practical migration strategies—so you can scale relational databases effectively.
This article examines Suning's bargain‑group platform transformation, detailing its strategic shift to a platform model, high‑availability architecture, vertical and horizontal decomposition, data sharding, cache design, dual‑data‑center deployment, and link optimizations for handling massive concurrent traffic.
This article walks through the progressive transformation of a simple Java‑Web e‑commerce prototype—from a single‑machine setup to multi‑server clusters, load‑balancing, database read/write splitting, caching, sharding, service‑oriented architecture, and message middleware—illustrating practical solutions to scalability, reliability, and performance challenges.
This article explains the reasons for splitting databases, describes vertical and horizontal sharding techniques, compares their benefits and drawbacks, and advises performing horizontal sharding before vertical partitioning to improve scalability and performance in large‑scale applications.
This article explains the concepts, benefits, and challenges of database sharding—including vertical and horizontal partitioning, common splitting strategies, handling distributed transactions, cross‑node joins, pagination, global primary keys, and migration—while offering practical guidelines on when to adopt sharding and which tools can assist.
This talk walks through Sharding-Proxy’s role within Apache ShardingSphere, detailing its architecture, SQL lifecycle from protocol decoding to routing, rewriting, execution, and result merging, while highlighting core principles, performance bottlenecks, and practical optimization techniques for high‑throughput, low‑latency database proxying.
The article outlines Taobao's backend architecture evolution—from a single Tomcat‑MySQL server through caching, load‑balancing, database sharding, MPP, DNS‑based multi‑datacenter routing, NoSQL/search integration, microservices, ESB, containerization, and finally cloud‑native deployment—while summarizing the associated technologies and design principles.
This article summarizes the key concepts, advantages, and practical implementations of distributed architecture at Ant Financial, covering microservice migration, modular development, database vertical and horizontal sharding, high‑availability mechanisms, gray‑release, and fault‑tolerant settlement workflows.
This article examines common database performance bottlenecks such as I/O and CPU limits, explains when to apply horizontal or vertical sharding, partitioning, and scaling techniques, compares tools like ShardingSphere, TDDL, and Mycat, and provides practical steps for implementation, migration, and troubleshooting.
This article explains Ant Financial's practical implementation of a distributed architecture—including micro‑service migration, modular development, database vertical and horizontal sharding, high‑availability mechanisms, task‑scheduling platforms, gray‑release strategies, and full‑link stress testing—to achieve reliable, scalable payment processing.
This article outlines how large‑scale web applications evolve from simple single‑server deployments through static‑dynamic separation, clustering, database read/write splitting, sharding, service‑oriented architecture, and message queues, emphasizing the trade‑offs and best practices for each architectural stage.
This article uses Taobao as a case study to trace the evolution of its server‑side architecture from a single‑machine setup to a cloud‑native, micro‑service ecosystem capable of handling tens of millions of concurrent users, highlighting key concepts, technologies, and design principles at each stage.
This article reviews the complete process of database and table sharding, covering when to shard, range and hash strategies, combining both, data migration challenges, business compatibility, and long‑term archiving to keep MySQL workloads manageable.
The article presents a detailed interview with Yan Huqing, the lead of the open‑source DBLE project, covering his background, the upcoming Shanghai 3306π meetup agenda, and practical advice on using Dble for sharding, read‑write splitting, and advanced monitoring in distributed database environments.
This article uses Taobao as a case study to trace the step‑by‑step evolution of server‑side architecture from a single‑machine deployment to distributed clusters, caching, load‑balancing, database sharding, micro‑services, containerization and finally cloud platforms, summarizing the key technologies and design principles at each stage.
This article analyzes the business flow of flash‑sale (seckill) activities, identifies the high‑concurrency technical challenges, and presents a comprehensive backend architecture—including frontend static page handling, request throttling, service‑layer queuing, database sharding, caching, optimistic locking, and anti‑cheating measures—to ensure reliability, scalability, and data consistency.
This article explains how to design a database architecture that can handle millions of daily active users and tens of thousands of concurrent requests by using multi‑server sharding, extensive table partitioning, read‑write separation, and distributed unique‑ID generation techniques such as Snowflake.
The article analyzes QQGame’s room‑entry failures caused by massive concurrent users, explores the limits of client‑side data synchronization, and proposes a scalable backend architecture using region‑based partitioning, autonomous server processing, and distributed database sharding to achieve high availability and data consistency.
This article explains the responsibilities, architectural design, critical construction points, functional modules, hotspot account handling, database sharding strategies, and diverse settlement rule implementations of JD Digits' accounting and settlement system, which is the core financial control component of the payment platform.
This article explains why and when to shard databases, compares sharding methods such as key‑based, range‑based, and dictionary approaches, and outlines the performance gains, availability improvements, and new challenges like increased complexity, ID handling, cross‑shard queries, and distributed transactions.
Database sharding, a horizontal partitioning technique that splits a table’s rows across multiple nodes, enables scalable performance and fault isolation for high‑traffic applications, but introduces complexity, potential data imbalance, and recovery challenges, so it should be adopted only after simpler optimizations are exhausted.
This article presents a practical performance test of Sharding‑Sphere using MySQL, showing how cross‑shard queries incur significant latency due to parallel routing and result merging, and discusses the underlying thread‑pool execution model and its impact on OLTP versus OLAP workloads.
This article explains how to move beyond naive copy‑paste development of lottery back‑ends by abstracting business logic into a reusable platform, covering database sharding, micro‑service architecture, common and custom modules, performance, deployment, and security considerations for a robust, SaaS‑like solution.
The article outlines how increasing business complexity and throughput demands drive the need for system splitting, architectural upgrades, and micro‑service adoption, detailing horizontal scaling, vertical and business partitioning, database sharding, and the evolution toward resilient, high‑performance backend infrastructures.
This article outlines the progressive evolution of large‑scale web architectures—from monolithic mainframe‑style deployments to separated application and database servers, clustering, load balancing, read/write splitting, search integration, caching, sharding, and finally micro‑service decomposition—illustrated with a simple Java‑based e‑commerce example.
To address growing business complexity and throughput demands, this article outlines systematic approaches to decompose monolithic systems—covering horizontal and vertical scaling, application and database sharding, service governance, caching, and the evolution toward microservices—highlighting practical techniques and real-world experiences.
This article reviews the background, side effects, sharding strategies, and key considerations of database partitioning, illustrating challenges such as merge‑sort pagination, deep‑paging performance, hash‑based routing, and integration issues with Sharding‑JDBC, while offering practical solutions and best‑practice recommendations.
This article explains why growing business complexity and throughput demand system decoupling, outlines horizontal scaling, vertical and business splitting, database sharding techniques, and shows how architectural evolution leads to micro‑service‑based, resilient infrastructures.
The article explains how increasing business complexity and throughput require system splitting, decoupling, and architectural evolution—covering horizontal and vertical scaling, database sharding, caching, and micro‑service patterns—to improve capacity, robustness, and performance of large‑scale applications.
This article describes how Ctrip's hotel R&D team tackled growing order‑volume challenges by sharding the database, building a real‑time Elasticsearch index, implementing a custom replication pipeline, and applying various write‑ and read‑optimizations to achieve low latency and stable performance.
The article explains how the slowdown of Moore's Law drives the shift from monolithic applications to distributed systems, covering load balancing, session handling strategies, read‑write separation, and database sharding to achieve scalable, high‑availability backend architectures.
This article explains why growing business complexity and throughput demands force a system to be split and refactored, describing horizontal scaling, vertical and business splitting, database sharding, and the resulting architectural evolution toward micro‑services.
The article reviews how increasing business complexity and throughput demands drive the need to split applications and databases, adopt horizontal and vertical scaling, implement business and service-level partitioning, and evolve system architecture toward micro‑services, caching, and asynchronous processing.
This article outlines how a .NET‑based system evolves from a single‑server deployment to a multi‑layered, load‑balanced, clustered architecture with database sharding, distributed file storage, caching, micro‑service communication, and cloud services to handle high traffic and massive data volumes.
This article examines why large‑scale MySQL deployments encounter CPU, I/O and memory bottlenecks, compares vertical and horizontal data‑sharding strategies, evaluates open‑source middleware ShardingJdbc versus MyCat, and provides a detailed Spring Boot integration design with routing rules and practical use‑case examples.
This article explains the essence of software architecture as a process of reducing system entropy through splitting and merging, outlines the capabilities required of architects, and details the step‑by‑step evolution of large‑scale website infrastructures including caching, CDN, database sharding, and messaging systems.
This article explains the multi‑layer data architecture design for microservice systems, covering concepts such as data usability, primary and secondary data decoupling, sharding, multi‑source data adaptation and caching, and introduces data marts to improve scalability and maintainability.
This article walks through the step‑by‑step evolution of a Java website—from a single‑machine setup to a multi‑tier, load‑balanced, sharded, cached, and service‑oriented architecture capable of handling billions of requests—detailing the challenges and solutions at each stage.
The article outlines a step‑by‑step evolution of a web application from a single‑machine deployment to a fully distributed architecture, covering server separation, clustering, load‑balancing algorithms, session handling, read‑write splitting, search engine integration, caching, database sharding, service‑oriented decomposition, and message‑queue middleware.
Facing billions of rows spread across 1024 tables in 64 databases, a Chinese gaming platform replaced its custom sharding framework with TiDB, achieving real‑time OLTP consistency, elastic scaling, robust backup, high availability, and lower migration costs while preserving MySQL compatibility.
