BestHub
Sign in
Tagged articles
602 articles
Page 5 of 7
Java Architect Essentials
Java Architect Essentials
Dec 31, 2020 · Databases

MySQL Single Table Optimization, Sharding, Partitioning, and Scaling Techniques

When a MySQL table grows large, performance degrades sharply, so this guide explains single‑table tuning, proper field choices, index strategies, query best practices, engine differences, system parameters, hardware upgrades, read‑write splitting, caching layers, table partitioning, vertical and horizontal sharding, and how to choose suitable sharding solutions.

DatabaseScalingPartitioningindexing
0 likes · 25 min read
MySQL Single Table Optimization, Sharding, Partitioning, and Scaling Techniques
Liangxu Linux
Liangxu Linux
Dec 26, 2020 · Databases

Master MySQL Performance: Practical Soft and Hard Optimization Techniques

This guide explains how to boost MySQL performance through soft optimizations like query analysis, index usage, and table restructuring, as well as hard optimizations involving hardware upgrades, configuration tuning, sharding, read‑write splitting, and cache clustering.

CacheDatabase Optimizationindexing
0 likes · 8 min read
Master MySQL Performance: Practical Soft and Hard Optimization Techniques
Tencent Cloud Developer
Tencent Cloud Developer
Dec 24, 2020 · Big Data

Distributed Search Engine Design and Index Management in WeChat Search

The article details WeChat Search’s practical distributed architecture—using a Chubby‑elected leader for shard‑to‑node mapping, hash‑based sharding with dynamic rebalancing, a Lambda‑style batch and near‑real‑time indexing pipeline, relaxed monotonic consistency, and group‑based searcher scaling—to illustrate trade‑offs and lessons for building scalable, reliable search services.

Distributed SystemsIndex ManagementLSM
0 likes · 28 min read
Distributed Search Engine Design and Index Management in WeChat Search
vivo Internet Technology
vivo Internet Technology
Dec 23, 2020 · Backend Development

How Vivo Scaled Its Order System: Sharding, Migration, and Real‑World Lessons

This article details how Vivo transformed its monolithic e‑commerce order service into a scalable, service‑oriented system by applying data archiving, sharding‑JDBC based sharding, MySQL‑to‑Elasticsearch sync, zero‑downtime migration, and distributed‑transaction techniques, while sharing practical pitfalls and performance results.

Distributed SystemsElasticsearchMicroservices
0 likes · 17 min read
How Vivo Scaled Its Order System: Sharding, Migration, and Real‑World Lessons
Java Interview Crash Guide
Java Interview Crash Guide
Dec 17, 2020 · Databases

Mastering Data Sharding: When and How to Split Your Database

This article explains the concepts, types, advantages, and drawbacks of vertical and horizontal database sharding, discusses the challenges such as distributed transactions, cross‑node joins, pagination, global primary‑key generation, migration, and provides practical guidelines and case studies for deciding when and how to apply sharding.

Distributed Transactionsdatabase partitioningglobal IDs
0 likes · 28 min read
Mastering Data Sharding: When and How to Split Your Database
Laravel Tech Community
Laravel Tech Community
Dec 10, 2020 · Databases

MySQL Single‑Table Optimization, Partitioning, Sharding and Scaling Strategies

This article provides a comprehensive guide to improving MySQL performance by optimizing single‑table schemas, indexes, queries, engine settings, system parameters, and then discusses read/write separation, caching layers, table partitioning, vertical and horizontal sharding, compatible scalable databases and when to consider NoSQL alternatives.

Database OptimizationPartitioningScalability
0 likes · 21 min read
MySQL Single‑Table Optimization, Partitioning, Sharding and Scaling Strategies
dbaplus Community
dbaplus Community
Dec 6, 2020 · Databases

Sharding Massive MySQL Tables: End‑to‑End Architecture, Migration & Best Practices

This article presents a comprehensive, step‑by‑step case study of a large‑scale MySQL sharding project, covering business refactoring, storage architecture design, data migration, synchronization strategies, capacity planning, validation, rollout procedures, stability safeguards, and cross‑team collaboration.

Database Architecturemysqlsharding
0 likes · 28 min read
Sharding Massive MySQL Tables: End‑to‑End Architecture, Migration & Best Practices
Senior Brother's Insights
Senior Brother's Insights
Dec 2, 2020 · Databases

Mastering Data Sharding: When, How, and What to Watch Out For

This comprehensive guide explains why and when to split relational databases, details vertical and horizontal sharding techniques, discusses associated challenges such as distributed transactions, cross‑node joins, pagination, global primary keys, and offers practical solutions and middleware options.

Distributed TransactionsGlobal ID Generationdatabase partitioning
0 likes · 24 min read
Mastering Data Sharding: When, How, and What to Watch Out For
ITPUB
ITPUB
Nov 12, 2020 · Databases

How Vivo Scaled to Billions of Records: Sharding and InnoDB Compression Strategies

This article details how Vivo's cloud service tackled explosive data growth by applying horizontal and vertical sharding, routing‑table based dynamic expansion, and MySQL InnoDB compression, providing step‑by‑step guidance, performance results, and practical recommendations for large‑scale database deployments.

InnoDBPartitioningdata compression
0 likes · 16 min read
How Vivo Scaled to Billions of Records: Sharding and InnoDB Compression Strategies
Practical DevOps Architecture
Practical DevOps Architecture
Nov 10, 2020 · Databases

Design of Ant Financial's Logical Data Center (LDC) and Unitization for High‑TPS Payments

The article explains how Ant Financial’s Logical Data Center (LDC) and unit‑based architecture, combined with sharding, CAP analysis, and OceanBase’s Paxos‑based consensus, enable the payment platform to sustain tens of millions of transactions per second during Double‑11 events while ensuring high availability and disaster recovery.

CAP theoremDistributed SystemsHigh TPS
0 likes · 42 min read
Design of Ant Financial's Logical Data Center (LDC) and Unitization for High‑TPS Payments
Architect
Architect
Oct 27, 2020 · Databases

Database Bottlenecks and Sharding: Strategies for Horizontal and Vertical Partitioning

This article examines common database performance bottlenecks such as I/O and CPU limits, explains the concepts, results, and scenarios of horizontal and vertical sharding—including database and table partitioning—and outlines tools, steps, and challenges for implementing effective sharding solutions.

databaseshorizontal-partitioningsharding
0 likes · 11 min read
Database Bottlenecks and Sharding: Strategies for Horizontal and Vertical Partitioning
Top Architect
Top Architect
Oct 20, 2020 · Databases

Database Bottlenecks and Sharding Strategies

This article analyzes common database performance bottlenecks such as I/O and CPU limits, explains how horizontal and vertical sharding (including database and table partitioning) can alleviate these issues, introduces practical tools and step‑by‑step implementation guidance, and discusses related challenges like non‑partition key queries and scaling.

ScalabilityVertical Partitioningdatabases
0 likes · 8 min read
Database Bottlenecks and Sharding Strategies
JD Cloud Developers
JD Cloud Developers
Oct 14, 2020 · Databases

How ClickHouse Distributed Tables Write Data: Sync vs Async Explained

This article dissects ClickHouse's Distributed table write path, detailing how the Distributed engine creates tables, decides between synchronous and asynchronous insertion, manages local and remote shards, handles sharding keys, and ensures atomic data distribution across the cluster.

ClickHouseData InsertionDistributed Table
0 likes · 14 min read
How ClickHouse Distributed Tables Write Data: Sync vs Async Explained
macrozheng
macrozheng
Oct 9, 2020 · Databases

How to Implement Database Sharding: Horizontal & Vertical Partitioning with Sharding-JDBC

This article explains the fundamentals of database sharding, covering both vertical and horizontal partitioning concepts, routing algorithms, advantages and drawbacks, practical implementation steps using sharding-jdbc, and discusses related challenges such as distributed transactions, pagination, global IDs, and tool selection.

Distributed TransactionsJavaVertical Scaling
0 likes · 17 min read
How to Implement Database Sharding: Horizontal & Vertical Partitioning with Sharding-JDBC
Wukong Talks Architecture
Wukong Talks Architecture
Sep 24, 2020 · Fundamentals

Common Pitfalls in Distributed Systems: Message Queues, Caches, Sharding, and Transactions

This article systematically explains the fundamental concepts and typical pitfalls of distributed systems—including CAP and BASE theories, message‑queue reliability issues, distributed cache challenges, sharding strategies, and transaction models—while offering practical mitigation techniques for each problem.

CAP theoremDistributed Systemsdistributed-transaction
0 likes · 24 min read
Common Pitfalls in Distributed Systems: Message Queues, Caches, Sharding, and Transactions
Full-Stack Internet Architecture
Full-Stack Internet Architecture
Sep 20, 2020 · Databases

Database Read/Write Separation: Principles, Advantages, Disadvantages, and Implementation Strategies

Database read/write separation, alongside sharding and master‑slave architectures, mitigates high‑concurrency bottlenecks by distributing reads to replicas and writes to a primary, offering scalability and performance benefits while introducing challenges such as replication lag, routing complexity, and resource utilization trade‑offs.

Master‑SlaveRead-Write Separationhigh concurrency
0 likes · 10 min read
Database Read/Write Separation: Principles, Advantages, Disadvantages, and Implementation Strategies
Huawei Cloud Developer Alliance
Huawei Cloud Developer Alliance
Sep 17, 2020 · Fundamentals

Why Distributed Systems Matter: Core Concepts, Design Trade‑offs & CAP

This article explores the fundamentals of distributed systems, explaining what they are, why they’re used, design considerations such as replication and partitioning, the implications of the CAP theorem, common distribution strategies, typical architectural patterns, and the advantages and challenges of building and operating such systems.

CAP theoremDistributed SystemsScalability
0 likes · 14 min read
Why Distributed Systems Matter: Core Concepts, Design Trade‑offs & CAP
Architecture Digest
Architecture Digest
Sep 14, 2020 · Databases

Understanding the Underlying Mechanics of Elasticsearch and Lucene

This article provides a comprehensive, top‑down and bottom‑up explanation of Elasticsearch’s internal architecture, covering clusters, nodes, shards, Lucene segments, inverted indexes, stored fields, document values, caching, merging, routing, scaling, and query processing, while addressing common performance questions.

Elasticsearchcachinglucene
0 likes · 11 min read
Understanding the Underlying Mechanics of Elasticsearch and Lucene
IT Xianyu
IT Xianyu
Sep 11, 2020 · Backend Development

Beyond Sharding: Unitization as a Solution to Unlimited Service Scaling

The article examines why traditional sharding and database partitioning cannot alone achieve limitless scaling, explains the problem of excessive RPC‑to‑DB connections, and proposes a unitization approach that limits each service to a single database shard to enable true horizontal expansion.

Scalabilitydatabase connectionssharding
0 likes · 8 min read
Beyond Sharding: Unitization as a Solution to Unlimited Service Scaling
JavaEdge
JavaEdge
Sep 8, 2020 · Databases

Understanding Redis Sharding: Benefits, Strategies, and Implementation

This article explains Redis sharding, covering its purpose, various partitioning methods such as range and hash partitioning, implementation approaches like client‑side and proxy‑assisted sharding, their drawbacks, and practical steps for migration and using Redis Cluster or Twemproxy.

PartitioningProxyconsistent hashing
0 likes · 9 min read
Understanding Redis Sharding: Benefits, Strategies, and Implementation
Java Backend Technology
Java Backend Technology
Aug 30, 2020 · Databases

Mastering Database Sharding: Core Concepts, Middleware, and Common Challenges

Sharding splits a single database into multiple servers to improve performance, covering vertical and horizontal partitioning, middleware options, and key challenges such as distributed transactions, cross‑node joins, ID generation, pagination, routing transparency, and choosing between frameworks or custom solutions.

Distributed TransactionsID generationdatabase partitioning
0 likes · 16 min read
Mastering Database Sharding: Core Concepts, Middleware, and Common Challenges
MaGe Linux Operations
MaGe Linux Operations
Aug 23, 2020 · Databases

Choosing the Right Primary Key Strategy: UUID, Snowflake, Redis, and More

This article reviews common primary key generation methods—including auto‑increment drawbacks and five alternatives such as UUID, step, segment, Snowflake, and Redis—explaining their mechanisms, advantages, and trade‑offs to help you select the best fit for high‑concurrency and distributed systems.

databaseprimary keysharding
0 likes · 9 min read
Choosing the Right Primary Key Strategy: UUID, Snowflake, Redis, and More
Programmer DD
Programmer DD
Aug 10, 2020 · Big Data

Master ElasticSearch: How Its Distributed Architecture Powers Scalable Search

ElasticSearch achieves distributed search by organizing data into indices, types, mappings, documents, and fields, splitting indices into primary and replica shards across multiple nodes, with automatic master election and shard allocation, enabling horizontal scaling, high availability, and improved performance for large‑scale data workloads.

distributed architecturesearch enginesharding
0 likes · 7 min read
Master ElasticSearch: How Its Distributed Architecture Powers Scalable Search
Selected Java Interview Questions
Selected Java Interview Questions
Aug 8, 2020 · Databases

MySQL Single‑Table Optimization, Sharding, and Scaling Strategies

This article explains why MySQL tables with massive row counts suffer performance degradation and provides practical guidance on single‑table optimization, field and index design, query tuning, engine selection, system parameters, hardware upgrades, read‑write splitting, caching layers, partitioning, vertical and horizontal sharding, as well as client‑side and proxy‑side sharding solutions.

Database Optimizationindexingmysql
0 likes · 26 min read
MySQL Single‑Table Optimization, Sharding, and Scaling Strategies
Programmer DD
Programmer DD
Aug 7, 2020 · Databases

Database Architecture Guide: Principles, Patterns, and Consistency Solutions

This article outlines core database architecture principles—high availability, performance, scalability, and consistency—examines four common deployment patterns (primary‑standby, dual‑primary, primary‑replica with read/write separation, and hybrid), and presents practical consistency solutions and personal insights for selecting and evolving database designs.

Database ArchitectureRead-Write SeparationScalability
0 likes · 11 min read
Database Architecture Guide: Principles, Patterns, and Consistency Solutions
Programmer DD
Programmer DD
Jul 30, 2020 · Databases

Understanding Redis Cluster: Sharding, High Availability, and Hash Slot Mechanics

This article explains Redis Cluster’s automatic sharding, built‑in high availability, the role of the cluster bus ports, the limitations of traditional hash algorithms, and how consistent hashing with virtual nodes and the 16384‑slot hash slot mechanism enable efficient data distribution and low‑cost node scaling.

ClusterHash Slotconsistent hashing
0 likes · 3 min read
Understanding Redis Cluster: Sharding, High Availability, and Hash Slot Mechanics
dbaplus Community
dbaplus Community
Jul 14, 2020 · Databases

How to Install and Configure MyCat 1.6.5 for a Distributed MySQL Cluster

This guide walks through installing MySQL nodes, Java, and MyCat 1.6.5, configuring schemas, data nodes, sharding rules, adjusting ports, adding users, and verifying the MyCat service, providing complete code snippets and troubleshooting tips for a functional distributed database middleware setup.

ConfigurationInstallationMycat
0 likes · 16 min read
How to Install and Configure MyCat 1.6.5 for a Distributed MySQL Cluster
Big Data Technology & Architecture
Big Data Technology & Architecture
Jun 25, 2020 · Databases

MySQL Replication, Data Sharding, and Distributed Database Deployment Overview

This article explains MySQL replication (master‑slave and master‑master), data sharding principles and implementations, and various distributed database deployment architectures, covering replication mechanisms, advantages, failure recovery, sharding strategies, middleware like Mycat, and scaling solutions for high‑availability systems.

Database ArchitectureReplicationdistributed databases
0 likes · 14 min read
MySQL Replication, Data Sharding, and Distributed Database Deployment Overview
ITPUB
ITPUB
Jun 17, 2020 · Databases

Why Go All‑in on MongoDB? Architecture, HA, Sharding & Schema Design Explained

This article explains why a fast‑growing e‑commerce platform chose MongoDB, covering its high‑availability replica‑set architecture, Raft‑based election algorithm, replica‑set size limits, write‑concern trade‑offs, sharding components and load‑balancing, as well as the flexible document schema with practical code examples.

Database ArchitectureMongoDBhigh availability
0 likes · 13 min read
Why Go All‑in on MongoDB? Architecture, HA, Sharding & Schema Design Explained
Architecture Digest
Architecture Digest
Jun 10, 2020 · Databases

Comprehensive MySQL Single‑Table Optimization and Scaling Strategies

This article presents a thorough guide to improving MySQL performance for large single tables, covering field choices, indexing best practices, query tuning, engine selection, system parameters, hardware upgrades, read/write separation, caching layers, partitioning, vertical and horizontal sharding, and compatible scalable database alternatives.

Database OptimizationPartitioningindexing
0 likes · 21 min read
Comprehensive MySQL Single‑Table Optimization and Scaling Strategies
MaGe Linux Operations
MaGe Linux Operations
May 29, 2020 · Databases

Search Engine Architecture: Indexing, Querying, and Elasticsearch Basics

This article explains what a search engine is, describes its core components—indexing and search modules—detailing the workflow from content acquisition to result rendering, and provides an in‑depth overview of Elasticsearch, including its architecture, clusters, shards, replicas, mappings, and basic configuration.

Elasticsearchindexingsharding
0 likes · 16 min read
Search Engine Architecture: Indexing, Querying, and Elasticsearch Basics
ITPUB
ITPUB
May 26, 2020 · Databases

Master Redis: Core Data Types, Commands, and Advanced Features Explained

Redis is a high‑performance, open‑source key‑value store offering persistence, rich data structures, replication, pub/sub, transactions, and clustering; this guide explains its fundamentals, data types (string, hash, list, set, sorted set), essential commands, and advanced features such as persistence options, replication, Sentinel, and sharding.

Data StructuresPersistenceTransactions
0 likes · 16 min read
Master Redis: Core Data Types, Commands, and Advanced Features Explained
Java Architect Essentials
Java Architect Essentials
May 25, 2020 · Databases

MySQL Sharding and Partitioning: Strategies, Tools, and Best Practices

This article explains MySQL performance bottlenecks, compares IO and CPU limits, introduces horizontal and vertical sharding and partitioning concepts, presents practical scenarios, outlines common tools such as Sharding‑Sphere, TDDL and Mycat, and provides step‑by‑step guidance for implementation, troubleshooting and scaling.

Vertical Partitioningdatabase partitioninghorizontal scaling
0 likes · 8 min read
MySQL Sharding and Partitioning: Strategies, Tools, and Best Practices
dbaplus Community
dbaplus Community
May 24, 2020 · Big Data

Why Cross-Index Queries Matter in Elasticsearch and How to Implement Them

This article explains why Elasticsearch cross-index queries are essential, outlines their technical principles, showcases classic use cases such as business analytics, big‑data pipelines and log management, and provides practical methods, code examples, and performance considerations for effective implementation.

Big DataCross-Index QueryElasticsearch
0 likes · 10 min read
Why Cross-Index Queries Matter in Elasticsearch and How to Implement Them
Aikesheng Open Source Community
Aikesheng Open Source Community
May 15, 2020 · Databases

DBLE 3.0 Planning Overview and Community Q&A Summary

The article recaps the final DBLE open class session on May 12, presenting DBLE 3.0 roadmap, sharing PPT and video resources, and providing detailed Q&A covering features such as connection pooling, read‑write separation, distributed transactions, sharding, high‑availability, and future development plans.

DBLERead-Write Separationdistributed middleware
0 likes · 9 min read
DBLE 3.0 Planning Overview and Community Q&A Summary
Programmer DD
Programmer DD
May 14, 2020 · Databases

17 Essential Database Lessons Every Developer Should Know

Google Cloud engineer Jaana Dogan shares 17 hard‑earned database lessons—from ACID nuances and isolation levels to optimistic locking, sharding strategies, AUTOINCREMENT pitfalls, clock drift, and online migrations—providing a practical guide to help developers avoid common pitfalls and design more reliable data systems.

ACIDoptimistic lockingsharding
0 likes · 27 min read
17 Essential Database Lessons Every Developer Should Know
Tencent Cloud Developer
Tencent Cloud Developer
May 7, 2020 · Databases

TDSQL Horizontal Scaling: Design Principles and Practice for Non‑Disruptive Linear Expansion

TDSQL enables Tencent Meeting’s non‑disruptive, linear horizontal scaling by using a stateless SQL engine, shard‑key selection, two‑phase distributed transactions, and one‑click set expansion, delivering near‑linear performance growth, high availability, and seamless capacity increase for millions of daily users worldwide.

TDSQLcloud video conferencedistributed database
0 likes · 17 min read
TDSQL Horizontal Scaling: Design Principles and Practice for Non‑Disruptive Linear Expansion
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Apr 26, 2020 · Backend Development

How to Build a Million‑Message‑Per‑Second RabbitMQ Cluster: Lessons from Google

This article explains how to design and scale a RabbitMQ cluster capable of handling millions of messages per second, covering core concepts, Google’s large‑scale test setup, sharding and federation plugins, mirror queues, reliability mechanisms, and practical tips for high‑availability and performance optimization.

Message QueueRabbitMQclustering
0 likes · 25 min read
How to Build a Million‑Message‑Per‑Second RabbitMQ Cluster: Lessons from Google
Java Backend Technology
Java Backend Technology
Apr 26, 2020 · Databases

When to Shard Your Database? A Practical Guide to Partitioning Strategies

This article explains database bottlenecks caused by IO and CPU limits, introduces horizontal and vertical sharding for databases and tables, compares popular sharding tools, discusses challenges such as distributed transactions, cross‑node joins, pagination and global ID generation, and offers guidance on when and how to apply sharding in real‑world systems.

Distributed SystemsPartitioningScalability
0 likes · 14 min read
When to Shard Your Database? A Practical Guide to Partitioning Strategies
Architecture Digest
Architecture Digest
Apr 19, 2020 · Databases

Sharding (Partitioning) vs NewSQL Databases: A Comparative Analysis

This article objectively compares traditional sharding‑plus‑middleware architectures with modern NewSQL distributed databases, examining their design principles, distributed transaction handling, high‑availability mechanisms, scaling and sharding strategies, SQL support, storage engines, and overall maturity to help practitioners choose the most suitable solution for their workloads.

Database ArchitectureNewSQLhigh availability
0 likes · 21 min read
Sharding (Partitioning) vs NewSQL Databases: A Comparative Analysis
Java Backend Technology
Java Backend Technology
Apr 6, 2020 · Databases

Why NewSQL Databases Outperform Middleware Sharding? A Deep Comparison

This article objectively compares NewSQL databases with middleware‑based sharding solutions, examining architecture, distributed transactions, CAP constraints, performance, high availability, scaling, SQL support, storage engines, maturity, and provides practical guidance for choosing the right approach.

CAP theoremNewSQLdistributed databases
0 likes · 18 min read
Why NewSQL Databases Outperform Middleware Sharding? A Deep Comparison
Java Captain
Java Captain
Mar 31, 2020 · Databases

Optimizing Large MySQL Tables: Indexes, Partitioning, Sharding, and System Tuning

When a MySQL table grows to millions of rows, query, insert, update and delete performance degrades sharply, but by applying proper field design, index strategies, SQL best practices, engine selection, system parameters, caching, partitioning, vertical and horizontal sharding, and choosing suitable client‑side or proxy‑side architectures, the database can efficiently handle tens of millions of records.

Database TuningPartitioningmysql
0 likes · 24 min read
Optimizing Large MySQL Tables: Indexes, Partitioning, Sharding, and System Tuning
Cloud Native Technology Community
Cloud Native Technology Community
Mar 30, 2020 · Cloud Native

Building a Cloud‑Native Large‑Scale Distributed Monitoring System with Prometheus

This article explains how to design and implement a cloud‑native, large‑scale distributed monitoring system using Prometheus, covering its limitations, service‑level sharding, centralized storage, federation, and high‑availability strategies to overcome scaling challenges in Kubernetes environments.

Cloud NativeFederationPrometheus
0 likes · 12 min read
Building a Cloud‑Native Large‑Scale Distributed Monitoring System with Prometheus
Java Captain
Java Captain
Mar 22, 2020 · Databases

Database Bottlenecks and Sharding Strategies: Horizontal/Vertical Partitioning, Tools, and Best Practices

This article analyzes common database I/O and CPU bottlenecks, explains horizontal and vertical sharding concepts, presents practical sharding rules, discusses tooling such as Sharding‑Sphere, TDDL and Mycat, and outlines step‑by‑step procedures and common pitfalls for scaling relational databases.

Performance OptimizationVertical Partitioningdatabases
0 likes · 11 min read
Database Bottlenecks and Sharding Strategies: Horizontal/Vertical Partitioning, Tools, and Best Practices
Youzan Coder
Youzan Coder
Mar 11, 2020 · Backend Development

Youzan Task Scheduling Platform (TSP): Design, Implementation, and Future Roadmap

Youzan’s Task Scheduling Platform (TSP) unifies its legacy cron and timeout systems into a modular, extensible framework that supports both timed and delayed tasks, offering per‑task rate limiting, sharding, isolation, retry policies, and a rich SDK, while its roadmap adds end‑to‑end monitoring, workflow orchestration, domain‑level clustering, transactional messaging, and dynamic task registration.

BackendTSParchitecture
0 likes · 14 min read
Youzan Task Scheduling Platform (TSP): Design, Implementation, and Future Roadmap
Architects Research Society
Architects Research Society
Feb 3, 2020 · Databases

MapReduce‑Style Parallel Query Processing with Citus

The article explains how Citus enables sharding and MapReduce‑style parallel query execution in PostgreSQL, showing performance gains, example bucket algorithms, and how standard SQL can replace custom MapReduce code for large‑scale data analytics.

CitusDistributed PostgreSQLMapReduce
0 likes · 6 min read
MapReduce‑Style Parallel Query Processing with Citus
dbaplus Community
dbaplus Community
Jan 9, 2020 · Databases

Mastering Vitess: Scaling MySQL with Cloud‑Native Sharding and Resharding

This article introduces Vitess, a cloud‑native MySQL sharding middleware, explains its key features, architecture, core concepts such as cells, keyspaces, shards and vindexes, and provides a step‑by‑step guide to performing a reshard from two to four shards while highlighting operational challenges and practical recommendations.

Cloud NativeDatabase MiddlewareResharding
0 likes · 13 min read
Mastering Vitess: Scaling MySQL with Cloud‑Native Sharding and Resharding
Architecture Digest
Architecture Digest
Dec 17, 2019 · Databases

Scaling Dada's Delivery Platform: From Simple Architecture to Read/Write Separation, Vertical and Horizontal Sharding

This article chronicles Dada's rapid growth from a single‑database backend to a multi‑layer architecture that employs MySQL read/write separation, vertical partitioning, and horizontal sharding with a custom ID generator to sustain millions of daily orders while maintaining performance and stability.

BackendArchitectureDatabaseScalingReadWriteSeparation
0 likes · 14 min read
Scaling Dada's Delivery Platform: From Simple Architecture to Read/Write Separation, Vertical and Horizontal Sharding
Architecture Digest
Architecture Digest
Dec 15, 2019 · Databases

MongoDB High‑Throughput Cluster Optimization: Software, Configuration, and Storage Engine Tuning

This article details how a high‑traffic MongoDB sharded cluster exceeding one million TPS was optimized through software‑level tweaks, configuration changes, WiredTiger storage‑engine tuning, and hardware upgrades, resulting in latency reductions from hundreds of milliseconds to a few milliseconds and significantly improved stability.

Database TuningIO optimizationMongoDB
0 likes · 33 min read
MongoDB High‑Throughput Cluster Optimization: Software, Configuration, and Storage Engine Tuning
ITPUB
ITPUB
Nov 25, 2019 · Databases

Which Database Architecture Best Balances Availability, Performance, and Consistency?

This article examines four common database architecture patterns—primary‑standby, dual‑primary, master‑slave read/write separation, and a hybrid dual‑primary + master‑slave—evaluating each against high availability, performance, consistency, and scalability, and then presents practical consistency‑resolution techniques and cache strategies.

CacheConsistencyDatabase Architecture
0 likes · 10 min read
Which Database Architecture Best Balances Availability, Performance, and Consistency?
Programmer DD
Programmer DD
Oct 30, 2019 · Backend Development

Beyond Sharding: How Service Unitization Solves Unlimited Scaling

The article explains why traditional sharding and database partitioning cannot guarantee limitless scalability, examines the connection‑limit problems of RPC‑based services, and introduces unitization as a practical approach to achieve true unlimited scaling while managing database connections efficiently.

BackendScalabilityService Architecture
0 likes · 7 min read
Beyond Sharding: How Service Unitization Solves Unlimited Scaling
Selected Java Interview Questions
Selected Java Interview Questions
Oct 11, 2019 · Databases

Primary Key Strategies After Database Sharding

After splitting a database into multiple shards, generating globally unique primary keys becomes essential, and this article examines various solutions—including auto‑increment IDs, sequence steps, UUIDs, timestamp concatenation, and the Snowflake algorithm—detailing their advantages, drawbacks, and suitable scenarios.

ID generationprimary keysharding
0 likes · 11 min read
Primary Key Strategies After Database Sharding
Aikesheng Open Source Community
Aikesheng Open Source Community
Aug 12, 2019 · Databases

Choosing Between NewSQL Databases and Middleware‑Based Sharding: A Comparative Analysis

This article objectively compares NewSQL distributed databases with middleware‑based sharding solutions, examining their architectures, distributed transaction handling, scalability, performance, high‑availability, and operational considerations, and provides guidance on selecting the appropriate approach based on workload, consistency, and organizational constraints.

CAP theoremDistributed TransactionsNewSQL
0 likes · 19 min read
Choosing Between NewSQL Databases and Middleware‑Based Sharding: A Comparative Analysis
Big Data Technology Architecture
Big Data Technology Architecture
Aug 9, 2019 · Databases

Understanding Elasticsearch: Architecture, Core Concepts, and Performance Optimization

This article provides a comprehensive overview of Elasticsearch, covering its role in handling structured and unstructured data, core concepts such as Lucene, inverted indexes, clusters, shards, replicas, mapping, indexing processes, storage mechanisms, and practical performance tuning tips for deployment.

ElasticsearchReplicationinverted index
0 likes · 35 min read
Understanding Elasticsearch: Architecture, Core Concepts, and Performance Optimization
Aikesheng Open Source Community
Aikesheng Open Source Community
Jul 29, 2019 · Databases

Analysis and Resolution of MongoDB Sharding Balancer Chunk Migration Failures in Version 3.4.x

A MongoDB client reported severe chunk imbalance and nightly balancer migration failures in a sharded cluster, which were traced to a known bug causing conflicting operations, and the issue was resolved by disabling the balancer for the affected collection and upgrading the cluster to version 3.4.11 or later.

BalancerChunk MigrationDatabase Bug
0 likes · 6 min read
Analysis and Resolution of MongoDB Sharding Balancer Chunk Migration Failures in Version 3.4.x
ITPUB
ITPUB
Jul 12, 2019 · Databases

Choosing the Right Database Architecture: Master‑Slave, Dual‑Master, and Beyond

This article outlines core database architecture principles—high availability, performance, consistency, and scalability—then compares four common setups (primary‑backup, dual‑master, primary‑replica, and hybrid), analyzes their trade‑offs, and presents practical consistency and cache‑sync solutions along with personal insights.

ConsistencyDatabase ArchitectureRead-Write Separation
0 likes · 11 min read
Choosing the Right Database Architecture: Master‑Slave, Dual‑Master, and Beyond
dbaplus Community
dbaplus Community
Jul 9, 2019 · Databases

How Gaea Reinvents MySQL Middleware with Go‑Based Sharding and Hot‑Reload

Gaea is an open‑source MySQL middleware created by Xiaomi that replaces MyCAT, offering read‑write splitting, sharding, namespace‑based multi‑tenant configuration, Go‑driven architecture, hot‑reload via etcd, comprehensive monitoring, and a visual management console, with its source hosted on GitHub.

GoOpenSourcemiddleware
0 likes · 14 min read
How Gaea Reinvents MySQL Middleware with Go‑Based Sharding and Hot‑Reload
Aikesheng Open Source Community
Aikesheng Open Source Community
Jun 4, 2019 · Databases

How DBLE Maps Logical Shards to Physical Partitions Using a Hash Modulo

This article explains DBLE's hash‑based sharding algorithm that first computes a modulo to obtain a logical shard number and then directly maps it to a physical shard via a mapping table derived from partitionLength and partitionCount arrays, including configuration steps, development tips, operational guidance, and a comparison with MyCat.

DBLEDatabase MiddlewareMycat
0 likes · 7 min read
How DBLE Maps Logical Shards to Physical Partitions Using a Hash Modulo
21CTO
21CTO
May 9, 2019 · Databases

How to Identify Database Bottlenecks and Choose the Right Sharding Strategy

This article explains common I/O and CPU bottlenecks in databases, compares horizontal and vertical sharding techniques, introduces practical tools, outlines migration steps, and discusses typical challenges such as non‑partition‑key queries, cross‑shard pagination, and scaling.

Vertical Partitioningbottleneckdatabases
0 likes · 8 min read
How to Identify Database Bottlenecks and Choose the Right Sharding Strategy
Architect's Tech Stack
Architect's Tech Stack
Apr 27, 2019 · Databases

Hybrid Hash‑Range Sharding Strategy with Group‑Based Allocation

This article presents a hybrid sharding approach that combines range partitioning to assign ID ranges to groups and hash modulo on the total number of tables to achieve uniform data distribution while avoiding hotspots and eliminating the need for data migration during scaling.

Distributed SystemsHashScalability
0 likes · 7 min read
Hybrid Hash‑Range Sharding Strategy with Group‑Based Allocation
Architecture Digest
Architecture Digest
Apr 26, 2019 · Databases

Database Bottlenecks and Sharding Strategies (Horizontal & Vertical Partitioning)

The article explains common database performance bottlenecks such as I/O and CPU limits, and details horizontal and vertical sharding techniques—including database and table partitioning—along with tools, implementation steps, common issues, scaling strategies, and practical examples for improving scalability and reliability.

ScalabilityVertical Partitioningdatabase
0 likes · 10 min read
Database Bottlenecks and Sharding Strategies (Horizontal & Vertical Partitioning)
Architecture Digest
Architecture Digest
Apr 23, 2019 · Databases

Database Architecture Principles, Common Schemes, and Consistency Solutions

This article outlines core database architecture principles—high availability, performance, consistency, and scalability—examines four typical deployment schemes with their trade‑offs, and presents multiple consistency‑preserving strategies for both primary/replica and DB‑cache interactions.

Consistencyhigh availabilitysharding
0 likes · 10 min read
Database Architecture Principles, Common Schemes, and Consistency Solutions
Programmer DD
Programmer DD
Apr 18, 2019 · Databases

How We Scaled Billion‑Row MySQL Tables with Simple Sharding Strategies

This article shares a practical journey of handling MySQL tables that grew beyond a billion rows, describing a temporary backup‑table fix, a custom hash‑based sharding implementation, data migration tactics, validation steps, and key lessons learned for large‑scale database management.

database partitioningmysqlsharding
0 likes · 12 min read
How We Scaled Billion‑Row MySQL Tables with Simple Sharding Strategies
21CTO
21CTO
Apr 15, 2019 · Big Data

Mastering High‑Concurrency Big Data: Sharding, Partitioning, and Index Strategies

This article explores practical techniques for handling massive, high‑concurrency data workloads, covering relational database limits, read/write separation, vertical and horizontal sharding, key selection, archival to NoSQL stores, and the use of heterogeneous index tables to maintain performance.

Big DataPartitioningdatabase scaling
0 likes · 6 min read
Mastering High‑Concurrency Big Data: Sharding, Partitioning, and Index Strategies
Xianyu Technology
Xianyu Technology
Mar 6, 2019 · Databases

Seamless Migration of Xianyu Product Database Using TDDL and DTS

Xianyu migrated its tens‑billion‑row product database from a shared MySQL cluster to an isolated instance by refactoring with dual‑write, employing TDDL for sharding and sequence management, and using Alibaba Cloud DTS for full‑load, incremental sync, consistency verification, and a no‑data‑loss rollback, achieving seamless, user‑transparent migration and improved stability.

DTSData ConsistencyOnline Services
0 likes · 9 min read
Seamless Migration of Xianyu Product Database Using TDDL and DTS
Java High-Performance Architecture
Java High-Performance Architecture
Mar 4, 2019 · Databases

Data Heterogeneity Explained: Sharding, Query Dimensions & MySQL Binlog with Canal

This article introduces the concept of data heterogeneity in large-scale systems, explains how sharding creates query challenges, describes query‑dimension and aggregation heterogeneity, and details implementation techniques such as subscribing to MySQL binlog and using Alibaba’s Canal for reliable data synchronization and storage.

BinlogCanaldata heterogeneity
0 likes · 7 min read
Data Heterogeneity Explained: Sharding, Query Dimensions & MySQL Binlog with Canal
Aikesheng Open Source Community
Aikesheng Open Source Community
Feb 26, 2019 · Databases

Community Contributions on DBLE: Case Analysis, Configuration Guides, and Custom Sharding Algorithms

The announcement showcases a 30‑day community sharing event for the open‑source DBLE middleware, highlighting three member submissions that cover a cross‑shard query case study, detailed rule.xml and server.xml configuration guides, and a tutorial on creating custom sharding algorithms, along with reward details.

Case StudyConfigurationDBLE
0 likes · 5 min read
Community Contributions on DBLE: Case Analysis, Configuration Guides, and Custom Sharding Algorithms
21CTO
21CTO
Jan 28, 2019 · Databases

How Alibaba’s TDDL Evolved from Cobar to Power Billions of Daily Queries

This article traces the evolution of Alibaba’s distributed data layer—from the early Cobar system to the modern TDDL framework and DRDS service—explaining their architectures, limitations, sharding principles, transaction‑boundary strategies, heterogeneous index tables, and the Jingwei data‑replication platform that together enable seamless scaling and high‑performance SQL processing across thousands of databases.

AlibabaDRDSJingwei
0 likes · 29 min read
How Alibaba’s TDDL Evolved from Cobar to Power Billions of Daily Queries
Architect's Tech Stack
Architect's Tech Stack
Jan 20, 2019 · Backend Development

From Monolith to Unitized Architecture: Overcoming Unlimited Scaling Limits of Sharding

The article traces the evolution of backend services from monolithic applications through RPC-based scaling and database sharding, explains why sharding alone cannot achieve unlimited scalability due to connection limits, and proposes a unitized architecture that partitions databases per service to overcome these constraints.

Scalabilitydatabase connectionssharding
0 likes · 6 min read
From Monolith to Unitized Architecture: Overcoming Unlimited Scaling Limits of Sharding
dbaplus Community
dbaplus Community
Dec 11, 2018 · Databases

How We Fixed MongoDB Outages and Boosted Performance in Production

This article outlines MongoDB's key features, describes a real‑world outage caused by misconfigured connection limits, details the root‑cause analysis and temporary remediation, and presents a comprehensive set of configuration, sharding, and hardware optimizations that dramatically improved the system's reliability and throughput.

ConfigurationMongoDBOps
0 likes · 14 min read
How We Fixed MongoDB Outages and Boosted Performance in Production
Youzan Coder
Youzan Coder
Dec 7, 2018 · Backend Development

How Youzan Scaled Order Management: Sharding, Elasticsearch, and HBase

The article details Youzan's three‑stage evolution of its order management system—from database sharding to Elasticsearch‑based cross‑table search and finally HBase for fast detail assembly—while addressing data sync, real‑time consistency, and idempotency strategies.

Backend ArchitectureOrder Managementdata synchronization
0 likes · 9 min read
How Youzan Scaled Order Management: Sharding, Elasticsearch, and HBase
Architects Research Society
Architects Research Society
Nov 25, 2018 · Operations

eBay Scalability Best Practices: Functional Partitioning, Horizontal Sharding, Asynchronous Decoupling, and More

The article outlines eBay's key scalability best practices—including functional partitioning, horizontal sharding, avoiding distributed transactions, aggressive asynchronous decoupling, moving work to async pipelines, pervasive virtualization, and intelligent caching—to achieve linear or better resource usage as load grows.

AsynchronousOperationscaching
0 likes · 14 min read
eBay Scalability Best Practices: Functional Partitioning, Horizontal Sharding, Asynchronous Decoupling, and More
Qunar Tech Salon
Qunar Tech Salon
Nov 20, 2018 · Databases

Strategies for Massive Data Storage and Sharding in High‑Scale Systems

This article examines various approaches to storing massive data, including table partitioning, NoSQL/NewSQL, and database sharding, analyzes their advantages and drawbacks, and presents practical sharding designs for user, forum, and order databases, with considerations for caching, redundancy, and query routing.

Partitioningcachingsharding
0 likes · 19 min read
Strategies for Massive Data Storage and Sharding in High‑Scale Systems
Programmer DD
Programmer DD
Nov 8, 2018 · Databases

Mastering Sharding: When and How to Use Partitioning, Sharding, and NoSQL

This article explains why modern applications dealing with massive user, order, and transaction data must move beyond single‑table designs, compares partitioning, sharding (分库分表), and NoSQL/NewSQL solutions, and provides practical guidance on choosing middleware, sharding columns, and hybrid architectures such as es + HBase.

Scalabilitydatabase partitioningmiddleware
0 likes · 15 min read
Mastering Sharding: When and How to Use Partitioning, Sharding, and NoSQL
Tencent Cloud Developer
Tencent Cloud Developer
Nov 1, 2018 · Databases

Experience and Optimization of MongoDB for Mini‑Game Operations and Cloud Integration

Li Xiaohui shares Tencent Cloud MongoDB’s real‑world mini‑game operations, detailing schema‑free design, sharding, thread‑per‑connection tuning, snapshot‑based read fixes, and table‑level rollback, then demonstrates a one‑click cloud stack that provisions MongoDB, serverless functions, storage, monitoring and security for mini‑program developers.

Cloud ServicesGame DevelopmentMongoDB
0 likes · 12 min read
Experience and Optimization of MongoDB for Mini‑Game Operations and Cloud Integration
JD Tech Talk
JD Tech Talk
Nov 1, 2018 · Databases

The Evolution and Release of ShardingSphere 3.0.0: From Sharding‑JDBC to Sharding‑Proxy

This article chronicles the eight‑month development of ShardingSphere 3.0.0, detailing its origins from Sharding‑JDBC, the addition of Sharding‑Proxy and Sidecar, core feature enhancements, distributed transaction redesign, community growth, and future roadmap for the open‑source database middleware ecosystem.

Database MiddlewareDistributed TransactionsJava
0 likes · 12 min read
The Evolution and Release of ShardingSphere 3.0.0: From Sharding‑JDBC to Sharding‑Proxy