This article explains Redis as an open‑source in‑memory data store, compares its main deployment modes—single instance, Redis Cluster, and Redis Sentinel—detailing their architectures, replication, sharding, failover mechanisms, client options, and backup strategies such as RDB and AOF.
An in‑depth walkthrough of Oracle 12c R2 Sharding, covering its core concepts, architecture, environment setup across three virtual machines, detailed GSM and shard catalog configuration commands, test model, results, limitations, and troubleshooting tips for real‑world OLTP deployments.
The article outlines how a Discuz website’s MySQL infrastructure progresses through five stages—from a single‑machine setup for low traffic to multi‑master, read‑write split, and finally sharding and partitioning—to handle daily page views growing from under ten thousand to ten million.
The article outlines the complex business logic of JD Daojia's transaction system and presents architectural decisions such as service setization, database sharding, frontend‑backend separation, parallel execution, asynchronous processing, dependency governance, and future enhancements to improve scalability, reliability, and performance.
This article outlines practical techniques for optimizing large MySQL tables—including column and index tuning, query improvements, engine selection, system parameters, hardware upgrades, caching, partitioning, vertical and horizontal sharding, and discusses client‑side and proxy‑based sharding solutions and compatible databases.
This article explains how to scale MySQL for large‑scale, high‑concurrency applications by using business splitting, master‑slave replication, database sharding and partitioning, and provides practical implementation strategies, code examples, and references to useful tools and middleware.
MTDDL, Meituan’s in‑house client‑side sharding middleware, was created to address rapid product‑database growth by offering dynamic data‑source routing, read‑write separation, distributed ID generation, configurable database‑and‑table sharding, connection‑pool management, SQL monitoring, and live configuration updates via annotations and extensible strategies.
This guide walks through the reasons for sharding in MongoDB, explains the sharding architecture, and provides detailed commands to configure config servers, shard servers, and mongos, add shards, enable sharding on databases and collections, insert test data, and verify successful data distribution across shards.
Pinterest rebuilt its storage layer using a MySQL‑centric sharding design that meets strict stability, scalability, and performance requirements, detailing business needs, shard key generation, object and mapping tables, expansion strategies, and practical lessons learned over three years of production use.
This comprehensive guide presents the most common MongoDB interview questions, covering NoSQL fundamentals, replication, sharding, transactions, profiling, storage nuances, and operational details, providing concise yet thorough answers to help candidates confidently tackle technical interviews.
This article explains core database concepts such as sharding, replication, routing strategies, and high‑availability designs, detailing how to achieve read/write separation, cache integration, and consistency handling in a master‑slave architecture for large‑scale systems.
This article explains the three scenarios that trigger MongoDB chunk migration, describes the Balancer's lock‑based operation, and walks through the six detailed steps—including tag‑based splits, draining shard handling, and load‑balancing thresholds—that ensure data is evenly distributed across shards.
This article reviews Redis's native limitations, explains three clustering approaches—client‑side sharding, proxy sharding (e.g., Twemproxy), and the official Redis Cluster—then compares Twemproxy's drawbacks and introduces Codis as a modern, open‑source alternative with practical deployment tips.
The article recaps Meituan‑Dianping’s 10th Tech Salon where senior DBA Ping Zhong detailed the Atlas database middleware, explaining its three‑layer architecture, connection‑string simplification, load balancing, read/write separation, automated slave management, monitoring, sharding capabilities, current 87% deployment across services, and upcoming enhancements such as stronger SQL processing and distributed transaction support.
The article explains when MySQL sharding is needed, outlines five practical principles for deciding to split databases or tables, provides real‑world examples, and shares operational tips for implementing horizontal partitioning to improve performance, availability, and manageability.
The article details how Mogujie's e‑commerce transaction system evolved from a monolithic PHP application to a service‑oriented architecture through DB vertical sharding, service decomposition, custom sharding component TSharding, and performance optimizations such as asynchronous processing, caching, and distributed transaction handling to support rapid growth and high‑availability requirements.
This article provides a comprehensive overview of Elasticsearch 2.x, covering its distributed architecture, core concepts such as clusters, nodes, indices, shards and replicas, the ZenDiscovery master‑election process, scaling mechanisms, recovery, query features, and the underlying system components like Guice, Netty, and thread‑pool designs.
The article analyzes write‑mostly, read‑light cache scenarios such as driver location updates and counter increments, discusses lock bottlenecks, proposes horizontal sharding and per‑record locking, explores lock‑free approaches with data‑integrity signatures, and summarizes practical optimization techniques for high‑concurrency back‑end systems.
This guide explains how to build Docker images, configure Redis, and orchestrate containers to create a fully automated, one‑click Redis sharding cluster, covering preparation, scripting, configuration, and verification steps.
The article introduces MyCat, an open‑source distributed database middleware that provides transparent sharding, multi‑backend support, AI‑driven catlet extensions, and advanced read‑write splitting to enable low‑cost migration of single‑node databases to cloud environments while addressing performance and scalability challenges.
This article explains Oracle 12.2 sharding concepts—including table families, sharded versus duplicated tables, chunk definition and movement, shardspace usage, sharding methods, and step‑by‑step deployment commands—providing detailed guidance for DBA practitioners.
This article outlines eBay's seven key scalability best practices—including functional partitioning, horizontal sharding, avoiding distributed transactions, asynchronous decoupling, stream processing, virtualization, and strategic caching—to illustrate how large‑scale web platforms can achieve linear resource growth and high availability.
This article explains how to build a micro‑blogging social system using Redis, covering core data models, read‑through replication, Sentinel failover, and write‑side sharding techniques that enable the platform to handle hundreds of millions of users and massive traffic.
This article explains how LeTV's payment platform handled a hundred‑fold surge in traffic by redesigning its architecture with database sharding, a custom order‑ID scheme, asynchronous consistency, high‑availability clustering, data tiering, and a coarse‑fine request pipeline to sustain up to 100,000 orders per second.
In this technical share, the operations director explains how his team tackled PB‑scale data challenges by scaling Redis with Codis, implementing multi‑dimensional MySQL sharding, using vertical and horizontal partitioning, and optimizing storage with TokuDB, offering practical insights for high‑throughput system design.
This article introduces Mycat, an open‑source distributed database middleware that implements the MySQL protocol, explains its core concepts such as sharding, read/write separation, high availability, deployment architectures, optimization strategies, and provides practical configuration examples for schema, rule, and server files.
The article analyzes the architecture of TDSQL, a MySQL‑based distributed SQL system designed for Tencent’s billing platform, detailing its transition from an in‑memory NoSQL solution to a high‑consistency, auto‑scaling, fault‑tolerant database with sharding, scheduler, agent, and gateway components.
This article details Dazhong Dianping's practical approach to sharding its massive order database, covering vertical and horizontal partitioning strategies, specific sharding algorithms, unique ID design, data migration phases, and lessons learned for long‑term scalability.
This article explains how large‑scale internet applications can overcome database bottlenecks by using load‑balancing clusters, implementing read/write separation, and applying vertical or horizontal sharding, while outlining each method's principles, implementation steps, advantages, and drawbacks.
This article explains the core concepts of database sharding, detailing vertical and horizontal partitioning methods, strategies for combining them, handling shared data, and addressing challenges such as distributed transactions, cross‑node joins, and aggregation queries, with practical insights for scalable system design.
Pinterest grew from zero to over 100 billion monthly page views by rapidly expanding its infrastructure, adopting simple, mature technologies like MySQL, Redis, and Memcache, and transitioning from clustering to sharding, offering practical lessons on scaling, tool selection, ID design, and migration strategies for massive growth.
This article compares MongoDB to traditional relational databases, highlighting advantages such as eventual consistency, document storage, built‑in GridFS and sharding, rich third‑party support, and performance, while also discussing drawbacks like lack of transactions, high disk usage, and limited maintenance tools.
The article outlines the design requirements, sharding strategies, capacity planning, cost trade‑offs, and indexing techniques for building a highly available, scalable distributed storage solution for Weibo, focusing on storage‑layer decisions without involving caching or ID generation.
This article explains how to query sharded tables by routing single‑value and multi‑value lookups using hash functions, handling non‑sharded fields, and performing cross‑database joins, illustrating each case with SQL examples and diagrams of shard selection.
This article explains a practical method for performing pagination on a sharded table by querying each database up to the target offset, merging and sorting the results, and extracting the required page while discussing performance drawbacks and an optimization when data distribution is uniform.
This article explains essential database design principles—including normalization, denormalization, join avoidance, and various sharding techniques—while also covering scaling strategies such as vertical upgrades, horizontal partitioning, and the use of flash storage to boost performance.
This article compiles Russell Smith’s comprehensive MongoDB best‑practice guide, covering architecture choices, file limits, write safety, schema design, replication, sharding, security, and performance tuning to help engineers avoid common pitfalls and optimize production deployments.
This article explains the fundamentals of distributed computing, covering sharding algorithms, message‑queue based task distribution, an overview of Hadoop and its MapReduce model, and the characteristics of offline batch processing for large‑scale data workloads.
This article details a backend redesign of an instant‑messaging platform, covering Java interfaces for unread messages, Redis‑based unread counters, persistent queue processing, MySQL sharding and partitioning, SQL retrieval logic, and performance tuning to achieve high insert throughput.
This article explores the essential design considerations for building a short URL service, covering data structures, encoding algorithms, key length choices, capacity planning, sharding strategies, concurrency handling, network architecture, security measures, and a real‑world example.
Designing a short URL service involves choosing appropriate key‑value storage, simple incremental or base‑36 encoding for keys, estimating data capacity and sharding strategies, handling concurrent reads/writes with thread‑safe structures or Redis, selecting network event loops, and addressing security concerns such as abuse prevention.
The article describes how the limitations of an in‑memory NoSQL HOLD platform led to the creation of TDSQL, a distributed MySQL‑based SQL system featuring ZooKeeper‑coordinated Scheduler, Agent, and Gateway modules, automatic cross‑IDC disaster recovery, transparent horizontal scaling, strong synchronous replication, and future integration with container technologies.
This article explains how TDSQL transforms MySQL into a distributed, high‑availability SQL system by addressing NoSQL limitations, introducing a Scheduler‑Agent‑Gateway architecture, automatic scaling, sharding, robust disaster‑recovery mechanisms, and future integration with container technologies.
This article explains how scalability ties to concurrency and walks through architectural evolution—from a single MySQL instance to vertical partitioning, master‑slave replication, horizontal sharding, and finally cloud‑native SaaS databases—highlighting key challenges and solutions.
This article shares practical Elasticsearch experience covering index and query performance optimization, shard routing strategies, JVM tuning, daily maintenance, and answers to common production questions, providing actionable guidance for building high‑availability search clusters.
MyCat is an open‑source distributed database middleware that transparently shreds tables across multiple backend databases, solves connection overload, provides ER‑based sharding, global partitioning, AI‑driven catlets, and advanced read‑write separation, enabling low‑cost migration of single‑node databases to the cloud.
This article compiles expert insights from a high‑efficiency operations forum, covering MySQL performance optimization techniques such as read‑write separation, sharding, schema and parameter tuning, hardware and SSD enhancements, as well as a detailed FAQ addressing replication lag, Aurora challenges, and backup tools.
The article describes the evolution, architecture, automatic scaling, sharding, disaster‑recovery mechanisms, and strong synchronous replication strategy of TDSQL, a MySQL‑engine based distributed database built to meet the high‑availability and high‑consistency requirements of Tencent's billing platform.
This article presents 58.com’s experience with MySQL under massive data volumes, covering core concepts such as single‑instance, sharding, replication and grouping, common availability and read‑write challenges, detailed sharding implementations for user, post, friend and order tables, and post‑sharding business practices including IN queries, non‑partition key queries, and cross‑database pagination.
MongoDB 3.0 introduces a pluggable storage engine API with WiredTiger support, expands backup set capacity, adds sharding enhancements, improves security with SCRAM‑SHA‑1 authentication, and upgrades tooling and logging, while retaining MMAPv1 as the default engine.
The article explains how to achieve horizontal scalability by making applications stateless, using client‑side cookies for session data, applying various caching layers, splitting services and databases with sharding, adopting asynchronous messaging, storing unstructured data, and integrating monitoring with alerting.
This article outlines eBay's practical scalability best practices—including functional partitioning, horizontal sharding, avoiding distributed transactions, asynchronous design, virtualization, and judicious caching—to help large‑scale web systems achieve reliable, cost‑effective growth.
This article compiles practical discussions on database sharding and partitioning, covering criteria such as user ID, time, and hash, redundancy techniques, caching with Redis, query handling, pagination, consistency, and distributed transaction approaches for scalable backend systems.
This comprehensive tutorial walks you through Redis Cluster fundamentals, including installation, hash slot sharding, node replication, fault tolerance, configuration files, cluster creation commands, and client testing, all without delving into complex distributed theory.
This article traces MySQL’s architectural evolution—from a single‑instance setup to vertical partitioning, master‑slave replication, horizontal sharding, and finally cloud‑native SaaS—highlighting scalability strategies, sharding routing methods, consistency challenges, and automated scaling techniques for high‑concurrency web applications.
The article introduces SessionDB, a Java‑implemented, LSM‑tree‑based key/value storage engine designed to eliminate sticky sessions by providing high‑throughput, durable, and scalable session data handling, and details its architecture, optimizations, sharding strategy, and benchmark comparisons with BerkeleyDB, LevelDB, and RocksDB.
