BestHub
Sign in
Tagged articles

Sharding

656 articles · Page 6 of 7
Java Backend Technology
Java Backend Technology
Apr 26, 2020 · Databases

When to Shard Your Database? A Practical Guide to Partitioning Strategies

This article explains database bottlenecks caused by IO and CPU limits, introduces horizontal and vertical sharding for databases and tables, compares popular sharding tools, discusses challenges such as distributed transactions, cross‑node joins, pagination and global ID generation, and offers guidance on when and how to apply sharding in real‑world systems.

DatabasePartitioningSharding
0 likes · 14 min read
When to Shard Your Database? A Practical Guide to Partitioning Strategies
Architecture Digest
Architecture Digest
Apr 19, 2020 · Databases

Sharding (Partitioning) vs NewSQL Databases: A Comparative Analysis

This article objectively compares traditional sharding‑plus‑middleware architectures with modern NewSQL distributed databases, examining their design principles, distributed transaction handling, high‑availability mechanisms, scaling and sharding strategies, SQL support, storage engines, and overall maturity to help practitioners choose the most suitable solution for their workloads.

Database ArchitectureHigh AvailabilityNewSQL
0 likes · 21 min read
Sharding (Partitioning) vs NewSQL Databases: A Comparative Analysis
Java Backend Technology
Java Backend Technology
Apr 6, 2020 · Databases

Why NewSQL Databases Outperform Middleware Sharding? A Deep Comparison

This article objectively compares NewSQL databases with middleware‑based sharding solutions, examining architecture, distributed transactions, CAP constraints, performance, high availability, scaling, SQL support, storage engines, maturity, and provides practical guidance for choosing the right approach.

CAP theoremDistributed DatabasesNewSQL
0 likes · 18 min read
Why NewSQL Databases Outperform Middleware Sharding? A Deep Comparison
Java Captain
Java Captain
Mar 31, 2020 · Databases

Optimizing Large MySQL Tables: Indexes, Partitioning, Sharding, and System Tuning

When a MySQL table grows to millions of rows, query, insert, update and delete performance degrades sharply, but by applying proper field design, index strategies, SQL best practices, engine selection, system parameters, caching, partitioning, vertical and horizontal sharding, and choosing suitable client‑side or proxy‑side architectures, the database can efficiently handle tens of millions of records.

MySQLOptimizationPartitioning
0 likes · 24 min read
Optimizing Large MySQL Tables: Indexes, Partitioning, Sharding, and System Tuning
Cloud Native Technology Community
Cloud Native Technology Community
Mar 30, 2020 · Cloud Native

Building a Cloud‑Native Large‑Scale Distributed Monitoring System with Prometheus

This article explains how to design and implement a cloud‑native, large‑scale distributed monitoring system using Prometheus, covering its limitations, service‑level sharding, centralized storage, federation, and high‑availability strategies to overcome scaling challenges in Kubernetes environments.

Cloud NativeFederationHigh Availability
0 likes · 12 min read
Building a Cloud‑Native Large‑Scale Distributed Monitoring System with Prometheus
Java Captain
Java Captain
Mar 22, 2020 · Databases

Database Bottlenecks and Sharding Strategies: Horizontal/Vertical Partitioning, Tools, and Best Practices

This article analyzes common database I/O and CPU bottlenecks, explains horizontal and vertical sharding concepts, presents practical sharding rules, discusses tooling such as Sharding‑Sphere, TDDL and Mycat, and outlines step‑by‑step procedures and common pitfalls for scaling relational databases.

DatabasesHorizontal PartitioningSharding
0 likes · 11 min read
Database Bottlenecks and Sharding Strategies: Horizontal/Vertical Partitioning, Tools, and Best Practices
Youzan Coder
Youzan Coder
Mar 11, 2020 · Backend Development

Youzan Task Scheduling Platform (TSP): Design, Implementation, and Future Roadmap

Youzan’s Task Scheduling Platform (TSP) unifies its legacy cron and timeout systems into a modular, extensible framework that supports both timed and delayed tasks, offering per‑task rate limiting, sharding, isolation, retry policies, and a rich SDK, while its roadmap adds end‑to‑end monitoring, workflow orchestration, domain‑level clustering, transactional messaging, and dynamic task registration.

ShardingTSParchitecture
0 likes · 14 min read
Youzan Task Scheduling Platform (TSP): Design, Implementation, and Future Roadmap
Architects Research Society
Architects Research Society
Feb 3, 2020 · Databases

MapReduce‑Style Parallel Query Processing with Citus

The article explains how Citus enables sharding and MapReduce‑style parallel query execution in PostgreSQL, showing performance gains, example bucket algorithms, and how standard SQL can replace custom MapReduce code for large‑scale data analytics.

CitusDistributed PostgreSQLMapReduce
0 likes · 6 min read
MapReduce‑Style Parallel Query Processing with Citus
dbaplus Community
dbaplus Community
Jan 9, 2020 · Databases

Mastering Vitess: Scaling MySQL with Cloud‑Native Sharding and Resharding

This article introduces Vitess, a cloud‑native MySQL sharding middleware, explains its key features, architecture, core concepts such as cells, keyspaces, shards and vindexes, and provides a step‑by‑step guide to performing a reshard from two to four shards while highlighting operational challenges and practical recommendations.

Cloud NativeDatabase MiddlewareMySQL
0 likes · 13 min read
Mastering Vitess: Scaling MySQL with Cloud‑Native Sharding and Resharding
Architecture Digest
Architecture Digest
Dec 17, 2019 · Databases

Scaling Dada's Delivery Platform: From Simple Architecture to Read/Write Separation, Vertical and Horizontal Sharding

This article chronicles Dada's rapid growth from a single‑database backend to a multi‑layer architecture that employs MySQL read/write separation, vertical partitioning, and horizontal sharding with a custom ID generator to sustain millions of daily orders while maintaining performance and stability.

BackendArchitectureDatabaseScalingMySQL
0 likes · 14 min read
Scaling Dada's Delivery Platform: From Simple Architecture to Read/Write Separation, Vertical and Horizontal Sharding
Architecture Digest
Architecture Digest
Dec 15, 2019 · Databases

MongoDB High‑Throughput Cluster Optimization: Software, Configuration, and Storage Engine Tuning

This article details how a high‑traffic MongoDB sharded cluster exceeding one million TPS was optimized through software‑level tweaks, configuration changes, WiredTiger storage‑engine tuning, and hardware upgrades, resulting in latency reductions from hundreds of milliseconds to a few milliseconds and significantly improved stability.

IO optimizationMongoDBSharding
0 likes · 33 min read
MongoDB High‑Throughput Cluster Optimization: Software, Configuration, and Storage Engine Tuning
ITPUB
ITPUB
Nov 25, 2019 · Databases

Which Database Architecture Best Balances Availability, Performance, and Consistency?

This article examines four common database architecture patterns—primary‑standby, dual‑primary, master‑slave read/write separation, and a hybrid dual‑primary + master‑slave—evaluating each against high availability, performance, consistency, and scalability, and then presents practical consistency‑resolution techniques and cache strategies.

Database ArchitectureShardingcache
0 likes · 10 min read
Which Database Architecture Best Balances Availability, Performance, and Consistency?
Programmer DD
Programmer DD
Oct 30, 2019 · Backend Development

Beyond Sharding: How Service Unitization Solves Unlimited Scaling

The article explains why traditional sharding and database partitioning cannot guarantee limitless scalability, examines the connection‑limit problems of RPC‑based services, and introduces unitization as a practical approach to achieve true unlimited scaling while managing database connections efficiently.

Database ConnectionsShardingbackend
0 likes · 7 min read
Beyond Sharding: How Service Unitization Solves Unlimited Scaling
Selected Java Interview Questions
Selected Java Interview Questions
Oct 11, 2019 · Databases

Primary Key Strategies After Database Sharding

After splitting a database into multiple shards, generating globally unique primary keys becomes essential, and this article examines various solutions—including auto‑increment IDs, sequence steps, UUIDs, timestamp concatenation, and the Snowflake algorithm—detailing their advantages, drawbacks, and suitable scenarios.

Shardingid generationprimary key
0 likes · 11 min read
Primary Key Strategies After Database Sharding
Aikesheng Open Source Community
Aikesheng Open Source Community
Aug 12, 2019 · Databases

Choosing Between NewSQL Databases and Middleware‑Based Sharding: A Comparative Analysis

This article objectively compares NewSQL distributed databases with middleware‑based sharding solutions, examining their architectures, distributed transaction handling, scalability, performance, high‑availability, and operational considerations, and provides guidance on selecting the appropriate approach based on workload, consistency, and organizational constraints.

CAP theoremDistributed DatabasesHigh Availability
0 likes · 19 min read
Choosing Between NewSQL Databases and Middleware‑Based Sharding: A Comparative Analysis
Big Data Technology Architecture
Big Data Technology Architecture
Aug 9, 2019 · Databases

Understanding Elasticsearch: Architecture, Core Concepts, and Performance Optimization

This article provides a comprehensive overview of Elasticsearch, covering its role in handling structured and unstructured data, core concepts such as Lucene, inverted indexes, clusters, shards, replicas, mapping, indexing processes, storage mechanisms, and practical performance tuning tips for deployment.

ElasticsearchShardinginverted index
0 likes · 35 min read
Understanding Elasticsearch: Architecture, Core Concepts, and Performance Optimization
Aikesheng Open Source Community
Aikesheng Open Source Community
Jul 29, 2019 · Databases

Analysis and Resolution of MongoDB Sharding Balancer Chunk Migration Failures in Version 3.4.x

A MongoDB client reported severe chunk imbalance and nightly balancer migration failures in a sharded cluster, which were traced to a known bug causing conflicting operations, and the issue was resolved by disabling the balancer for the affected collection and upgrading the cluster to version 3.4.11 or later.

BalancerChunk MigrationDatabase Bug
0 likes · 6 min read
Analysis and Resolution of MongoDB Sharding Balancer Chunk Migration Failures in Version 3.4.x
ITPUB
ITPUB
Jul 12, 2019 · Databases

Choosing the Right Database Architecture: Master‑Slave, Dual‑Master, and Beyond

This article outlines core database architecture principles—high availability, performance, consistency, and scalability—then compares four common setups (primary‑backup, dual‑master, primary‑replica, and hybrid), analyzes their trade‑offs, and presents practical consistency and cache‑sync solutions along with personal insights.

Database ArchitectureRead-Write SeparationSharding
0 likes · 11 min read
Choosing the Right Database Architecture: Master‑Slave, Dual‑Master, and Beyond
dbaplus Community
dbaplus Community
Jul 9, 2019 · Databases

How Gaea Reinvents MySQL Middleware with Go‑Based Sharding and Hot‑Reload

Gaea is an open‑source MySQL middleware created by Xiaomi that replaces MyCAT, offering read‑write splitting, sharding, namespace‑based multi‑tenant configuration, Go‑driven architecture, hot‑reload via etcd, comprehensive monitoring, and a visual management console, with its source hosted on GitHub.

GoMiddlewareMySQL
0 likes · 14 min read
How Gaea Reinvents MySQL Middleware with Go‑Based Sharding and Hot‑Reload
Aikesheng Open Source Community
Aikesheng Open Source Community
Jun 4, 2019 · Databases

How DBLE Maps Logical Shards to Physical Partitions Using a Hash Modulo

This article explains DBLE's hash‑based sharding algorithm that first computes a modulo to obtain a logical shard number and then directly maps it to a physical shard via a mapping table derived from partitionLength and partitionCount arrays, including configuration steps, development tips, operational guidance, and a comparison with MyCat.

DBLEDatabase MiddlewareMycat
0 likes · 7 min read
How DBLE Maps Logical Shards to Physical Partitions Using a Hash Modulo
21CTO
21CTO
May 9, 2019 · Databases

How to Identify Database Bottlenecks and Choose the Right Sharding Strategy

This article explains common I/O and CPU bottlenecks in databases, compares horizontal and vertical sharding techniques, introduces practical tools, outlines migration steps, and discusses typical challenges such as non‑partition‑key queries, cross‑shard pagination, and scaling.

BottleneckDatabasesHorizontal Partitioning
0 likes · 8 min read
How to Identify Database Bottlenecks and Choose the Right Sharding Strategy
Architecture Digest
Architecture Digest
Apr 26, 2019 · Databases

Database Bottlenecks and Sharding Strategies (Horizontal & Vertical Partitioning)

The article explains common database performance bottlenecks such as I/O and CPU limits, and details horizontal and vertical sharding techniques—including database and table partitioning—along with tools, implementation steps, common issues, scaling strategies, and practical examples for improving scalability and reliability.

DatabaseHorizontal PartitioningSharding
0 likes · 10 min read
Database Bottlenecks and Sharding Strategies (Horizontal & Vertical Partitioning)
Architecture Digest
Architecture Digest
Apr 23, 2019 · Databases

Database Architecture Principles, Common Schemes, and Consistency Solutions

This article outlines core database architecture principles—high availability, performance, consistency, and scalability—examines four typical deployment schemes with their trade‑offs, and presents multiple consistency‑preserving strategies for both primary/replica and DB‑cache interactions.

High AvailabilityShardingconsistency
0 likes · 10 min read
Database Architecture Principles, Common Schemes, and Consistency Solutions
Programmer DD
Programmer DD
Apr 18, 2019 · Databases

How We Scaled Billion‑Row MySQL Tables with Simple Sharding Strategies

This article shares a practical journey of handling MySQL tables that grew beyond a billion rows, describing a temporary backup‑table fix, a custom hash‑based sharding implementation, data migration tactics, validation steps, and key lessons learned for large‑scale database management.

MySQLShardingdatabase partitioning
0 likes · 12 min read
How We Scaled Billion‑Row MySQL Tables with Simple Sharding Strategies
21CTO
21CTO
Apr 15, 2019 · Big Data

Mastering High‑Concurrency Big Data: Sharding, Partitioning, and Index Strategies

This article explores practical techniques for handling massive, high‑concurrency data workloads, covering relational database limits, read/write separation, vertical and horizontal sharding, key selection, archival to NoSQL stores, and the use of heterogeneous index tables to maintain performance.

Big DataDatabase ScalingPartitioning
0 likes · 6 min read
Mastering High‑Concurrency Big Data: Sharding, Partitioning, and Index Strategies
Xianyu Technology
Xianyu Technology
Mar 6, 2019 · Databases

Seamless Migration of Xianyu Product Database Using TDDL and DTS

Xianyu migrated its tens‑billion‑row product database from a shared MySQL cluster to an isolated instance by refactoring with dual‑write, employing TDDL for sharding and sequence management, and using Alibaba Cloud DTS for full‑load, incremental sync, consistency verification, and a no‑data‑loss rollback, achieving seamless, user‑transparent migration and improved stability.

DTSData ConsistencyDatabase Migration
0 likes · 9 min read
Seamless Migration of Xianyu Product Database Using TDDL and DTS
Java High-Performance Architecture
Java High-Performance Architecture
Mar 4, 2019 · Databases

Data Heterogeneity Explained: Sharding, Query Dimensions & MySQL Binlog with Canal

This article introduces the concept of data heterogeneity in large-scale systems, explains how sharding creates query challenges, describes query‑dimension and aggregation heterogeneity, and details implementation techniques such as subscribing to MySQL binlog and using Alibaba’s Canal for reliable data synchronization and storage.

CanalData synchronizationMySQL
0 likes · 7 min read
Data Heterogeneity Explained: Sharding, Query Dimensions & MySQL Binlog with Canal
Aikesheng Open Source Community
Aikesheng Open Source Community
Feb 26, 2019 · Databases

Community Contributions on DBLE: Case Analysis, Configuration Guides, and Custom Sharding Algorithms

The announcement showcases a 30‑day community sharing event for the open‑source DBLE middleware, highlighting three member submissions that cover a cross‑shard query case study, detailed rule.xml and server.xml configuration guides, and a tutorial on creating custom sharding algorithms, along with reward details.

Case studyConfigurationDBLE
0 likes · 5 min read
Community Contributions on DBLE: Case Analysis, Configuration Guides, and Custom Sharding Algorithms
21CTO
21CTO
Jan 28, 2019 · Databases

How Alibaba’s TDDL Evolved from Cobar to Power Billions of Daily Queries

This article traces the evolution of Alibaba’s distributed data layer—from the early Cobar system to the modern TDDL framework and DRDS service—explaining their architectures, limitations, sharding principles, transaction‑boundary strategies, heterogeneous index tables, and the Jingwei data‑replication platform that together enable seamless scaling and high‑performance SQL processing across thousands of databases.

AlibabaDRDSData Replication
0 likes · 29 min read
How Alibaba’s TDDL Evolved from Cobar to Power Billions of Daily Queries
Architect's Tech Stack
Architect's Tech Stack
Jan 20, 2019 · Backend Development

From Monolith to Unitized Architecture: Overcoming Unlimited Scaling Limits of Sharding

The article traces the evolution of backend services from monolithic applications through RPC-based scaling and database sharding, explains why sharding alone cannot achieve unlimited scalability due to connection limits, and proposes a unitized architecture that partitions databases per service to overcome these constraints.

Database ConnectionsShardingscalability
0 likes · 6 min read
From Monolith to Unitized Architecture: Overcoming Unlimited Scaling Limits of Sharding
dbaplus Community
dbaplus Community
Dec 11, 2018 · Databases

How We Fixed MongoDB Outages and Boosted Performance in Production

This article outlines MongoDB's key features, describes a real‑world outage caused by misconfigured connection limits, details the root‑cause analysis and temporary remediation, and presents a comprehensive set of configuration, sharding, and hardware optimizations that dramatically improved the system's reliability and throughput.

ConfigurationDatabaseMongoDB
0 likes · 14 min read
How We Fixed MongoDB Outages and Boosted Performance in Production
Youzan Coder
Youzan Coder
Dec 7, 2018 · Backend Development

How Youzan Scaled Order Management: Sharding, Elasticsearch, and HBase

The article details Youzan's three‑stage evolution of its order management system—from database sharding to Elasticsearch‑based cross‑table search and finally HBase for fast detail assembly—while addressing data sync, real‑time consistency, and idempotency strategies.

Backend ArchitectureData synchronizationSharding
0 likes · 9 min read
How Youzan Scaled Order Management: Sharding, Elasticsearch, and HBase
Architects Research Society
Architects Research Society
Nov 25, 2018 · Operations

eBay Scalability Best Practices: Functional Partitioning, Horizontal Sharding, Asynchronous Decoupling, and More

The article outlines eBay's key scalability best practices—including functional partitioning, horizontal sharding, avoiding distributed transactions, aggressive asynchronous decoupling, moving work to async pipelines, pervasive virtualization, and intelligent caching—to achieve linear or better resource usage as load grows.

AsynchronousOperationsSharding
0 likes · 14 min read
eBay Scalability Best Practices: Functional Partitioning, Horizontal Sharding, Asynchronous Decoupling, and More
Qunar Tech Salon
Qunar Tech Salon
Nov 20, 2018 · Databases

Strategies for Massive Data Storage and Sharding in High‑Scale Systems

This article examines various approaches to storing massive data, including table partitioning, NoSQL/NewSQL, and database sharding, analyzes their advantages and drawbacks, and presents practical sharding designs for user, forum, and order databases, with considerations for caching, redundancy, and query routing.

PartitioningShardingcaching
0 likes · 19 min read
Strategies for Massive Data Storage and Sharding in High‑Scale Systems
Programmer DD
Programmer DD
Nov 8, 2018 · Databases

Mastering Sharding: When and How to Use Partitioning, Sharding, and NoSQL

This article explains why modern applications dealing with massive user, order, and transaction data must move beyond single‑table designs, compares partitioning, sharding (分库分表), and NoSQL/NewSQL solutions, and provides practical guidance on choosing middleware, sharding columns, and hybrid architectures such as es + HBase.

MiddlewareShardingdatabase partitioning
0 likes · 15 min read
Mastering Sharding: When and How to Use Partitioning, Sharding, and NoSQL
Tencent Cloud Developer
Tencent Cloud Developer
Nov 1, 2018 · Databases

Experience and Optimization of MongoDB for Mini‑Game Operations and Cloud Integration

Li Xiaohui shares Tencent Cloud MongoDB’s real‑world mini‑game operations, detailing schema‑free design, sharding, thread‑per‑connection tuning, snapshot‑based read fixes, and table‑level rollback, then demonstrates a one‑click cloud stack that provisions MongoDB, serverless functions, storage, monitoring and security for mini‑program developers.

Game DevelopmentMongoDBOperations
0 likes · 12 min read
Experience and Optimization of MongoDB for Mini‑Game Operations and Cloud Integration
JD Tech Talk
JD Tech Talk
Nov 1, 2018 · Databases

The Evolution and Release of ShardingSphere 3.0.0: From Sharding‑JDBC to Sharding‑Proxy

This article chronicles the eight‑month development of ShardingSphere 3.0.0, detailing its origins from Sharding‑JDBC, the addition of Sharding‑Proxy and Sidecar, core feature enhancements, distributed transaction redesign, community growth, and future roadmap for the open‑source database middleware ecosystem.

Database MiddlewareShardingdistributed transactions
0 likes · 12 min read
The Evolution and Release of ShardingSphere 3.0.0: From Sharding‑JDBC to Sharding‑Proxy
Node Underground
Node Underground
Oct 31, 2018 · Cloud Computing

Unlock 32 Cloud Design Patterns to Solve Distributed System Challenges

This article introduces the free classic book “Cloud Design Patterns,” which catalogs 32 design patterns across eight problem domains—availability, data management, design & implementation, messaging, management & monitoring, performance & scalability, resiliency, and security—offering practical solutions and C# examples for modern cloud and distributed systems.

Shardingdistributed systemsresiliency
0 likes · 5 min read
Unlock 32 Cloud Design Patterns to Solve Distributed System Challenges
21CTO
21CTO
Oct 17, 2018 · Databases

Scaling Payment Systems: Sharding, Snowflake IDs, and High‑Availability

This article explains how to design a high‑throughput payment system using database sharding, Snowflake‑style globally unique order IDs, eventual consistency via message queues, high‑availability architectures, data tiering, and coarse‑fine traffic control to handle massive request spikes.

Data TieringDatabaseSharding
0 likes · 15 min read
Scaling Payment Systems: Sharding, Snowflake IDs, and High‑Availability
Architects' Tech Alliance
Architects' Tech Alliance
Oct 15, 2018 · Databases

Data Sharding in Distributed Systems: Partitioning Strategies, Metadata Management, and Consistency Mechanisms

The article explains how distributed storage systems solve the fundamental problems of data sharding and redundancy by describing three sharding methods (hash, consistent‑hash, and range‑based), the criteria for choosing a shard key, the role of metadata servers, and consistency techniques such as leasing, all illustrated with concrete examples and code snippets.

DatabasesMetadataSharding
0 likes · 25 min read
Data Sharding in Distributed Systems: Partitioning Strategies, Metadata Management, and Consistency Mechanisms
dbaplus Community
dbaplus Community
Oct 10, 2018 · Databases

Why MyCat’s Pseudo‑Distributed MySQL Solution Fails and What to Do Instead

The article examines MyCat’s middleware‑based pseudo‑distributed MySQL architecture, outlines its storage, scalability, and reliability shortcomings, walks through common solutions like disk expansion, compression, and sharding, and finally offers practical steps and alternative technologies for building truly distributed database systems.

Database ArchitectureMiddlewareMySQL
0 likes · 24 min read
Why MyCat’s Pseudo‑Distributed MySQL Solution Fails and What to Do Instead
dbaplus Community
dbaplus Community
Sep 24, 2018 · Databases

Master MySQL Performance: Practical Optimizations for Large Tables

This guide explains why MySQL performance drops when a single table grows huge and provides a comprehensive set of optimization steps—including field type choices, index design, query tuning, engine selection, system parameters, hardware scaling, read/write splitting, caching, partitioning, vertical and horizontal sharding, and architecture recommendations—to keep MySQL fast and reliable at scale.

IndexingMySQLSharding
0 likes · 24 min read
Master MySQL Performance: Practical Optimizations for Large Tables
Full-Stack Internet Architecture
Full-Stack Internet Architecture
Aug 3, 2018 · Databases

Understanding Redis Cluster: Architecture, Slot Sharding, Node Management, and High Availability

Redis Cluster is a distributed system that partitions its keyspace into 16,384 hash slots across up to 16,384 nodes, enabling automatic sharding, slot migration, replication, and high‑availability features such as automatic node discovery, master‑slave election, and online resharding without service interruption.

ClusterDatabaseHigh Availability
0 likes · 5 min read
Understanding Redis Cluster: Architecture, Slot Sharding, Node Management, and High Availability
JD Tech
JD Tech
Aug 3, 2018 · Backend Development

High‑Concurrency Price Protection System: Rate Limiting, Degradation, Caching, Sharding, and Scalable Task Processing

This article describes how JD built a high‑concurrency price‑protection service for the 618 promotion, covering rate‑limiting, degradation, CDN and data caching, database sharding with smooth expansion, and a multi‑stage, fault‑tolerant task‑processing workflow to achieve unlimited scalability.

ShardingTask Processingbackend
0 likes · 13 min read
High‑Concurrency Price Protection System: Rate Limiting, Degradation, Caching, Sharding, and Scalable Task Processing
System Architect Go
System Architect Go
Jul 29, 2018 · Databases

What Is Elasticsearch? Core Concepts and Fundamentals

Elasticsearch is an open‑source, scalable, high‑availability distributed full‑text search engine that operates in near real‑time, using clusters of nodes, indexes, documents, shards and replicas to efficiently store and retrieve large volumes of data.

ClusterElasticsearchFull-text Search
0 likes · 4 min read
What Is Elasticsearch? Core Concepts and Fundamentals
Architects' Tech Alliance
Architects' Tech Alliance
Jul 27, 2018 · Backend Development

Designing Data Architecture for Microservices: Principles, Patterns, and Database Choices

This article explains how to design data architecture for microservice systems, covering microservice fundamentals, advantages, decoupling, lightweight APIs, continuous delivery, database per service versus shared databases, polyglot persistence, scaling dimensions, sharding strategies, and why MongoDB is a suitable choice.

MongoDBPolyglot PersistenceSharding
0 likes · 15 min read
Designing Data Architecture for Microservices: Principles, Patterns, and Database Choices
Efficient Ops
Efficient Ops
Jun 10, 2018 · Databases

How MongoDB Enables Active‑Active Multi‑Data‑Center Architectures

This article examines the requirements of active‑active applications across multiple data centers, compares distributed database architectures—including two‑phase commit, multi‑master, and sharded designs—and explains how MongoDB’s sharding and write/read concerns enable high availability, consistency, and rapid failover in such environments.

Active-ActiveDistributed DatabasesMongoDB
0 likes · 16 min read
How MongoDB Enables Active‑Active Multi‑Data‑Center Architectures
Meitu Technology
Meitu Technology
Jun 8, 2018 · Blockchain

Technical Overview of Telegram Open Network (TON) Blockchain

Telegram’s open‑source TON blockchain combines a master chain with dynamically sharded work chains, uses hyper‑cube routing for fast or reliable inter‑shard messages, employs PoS‑BFT consensus with validators, nominators and fishermen, supports services like DNS, Proxy, Storage and Pay, and manages a 5‑billion Gram token economy to enable scalable decentralized applications.

BlockchainConsensusSharding
0 likes · 14 min read
Technical Overview of Telegram Open Network (TON) Blockchain
ITPUB
ITPUB
May 28, 2018 · Databases

Mastering Database Sharding: When and How to Split Your Data

This article explains why and how relational databases should be sharded, covering vertical and horizontal partitioning techniques, common pitfalls such as distributed transactions and join queries, and practical guidelines for deciding when to split data and which tools to use.

Horizontal PartitioningMiddlewareSharding
0 likes · 22 min read
Mastering Database Sharding: When and How to Split Your Data
Efficient Ops
Efficient Ops
May 22, 2018 · Databases

Scaling Consumer‑Finance Databases: Architecture, DevOps, and Future Trends

This talk walks through the fundamentals of consumer‑finance services, details the evolving database architecture—including sharding, connection‑pool handling, resource isolation, and high‑performance caching—and shares the team's DevOps transformation and outlook toward NewSQL and AI‑driven operations.

DatabasesDevOpsSharding
0 likes · 15 min read
Scaling Consumer‑Finance Databases: Architecture, DevOps, and Future Trends
AntTech
AntTech
May 17, 2018 · Backend Development

Challenges and Solutions for Transforming Traditional Payment Systems to Distributed Architecture

The article examines how the rapid growth of mobile and aggregated payments strains traditional IOE‑based payment systems, and outlines a comprehensive distributed transformation strategy that includes vertical service‑oriented splitting, horizontal sharding by data type, and practical approaches to handling distributed transactions, cross‑shard queries, scaling, disaster recovery, and operational monitoring.

Shardinghorizontal-splittingpayment
0 likes · 15 min read
Challenges and Solutions for Transforming Traditional Payment Systems to Distributed Architecture
dbaplus Community
dbaplus Community
May 13, 2018 · Backend Development

Mastering Distributed Cache: Programming Techniques, Sharding Strategies, and Migration Plans

This article explains why distributed caching is essential for high‑traffic services, presents three programming approaches (direct code, Spring‑Data‑Redis injection, and annotation‑based), compares three cache‑access patterns, details sharding models, outlines smooth and offline migration steps, and addresses cache penetration, concurrency, avalanche, and transaction support with concrete code examples.

MigrationRedisSharding
0 likes · 22 min read
Mastering Distributed Cache: Programming Techniques, Sharding Strategies, and Migration Plans
Java Backend Technology
Java Backend Technology
May 8, 2018 · Databases

How MyCat Simplifies Database Sharding and Read/Write Routing

This article explains MyCat's role as a database middleware, tracing the evolution from single‑database setups through distributed deployment, caching, master‑slave, vertical and horizontal sharding, and shows how MyCat automates data‑source and shard selection to reduce code complexity.

Database MiddlewareHorizontal ShardingMycat
0 likes · 11 min read
How MyCat Simplifies Database Sharding and Read/Write Routing
dbaplus Community
dbaplus Community
May 6, 2018 · Databases

NewSQL Explained: Storage Engines, Sharding, and Distributed Transactions

This article examines the core technologies behind NewSQL databases for OLTP workloads, contrasting them with NoSQL and traditional relational systems by exploring storage engines such as B‑Tree and LSM‑Tree, sharding strategies, replication models, and distributed transaction mechanisms like two‑phase commit.

Distributed DatabasesNewSQLSharding
0 likes · 20 min read
NewSQL Explained: Storage Engines, Sharding, and Distributed Transactions
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Apr 22, 2018 · Databases

Mastering MySQL Sharding: Vertical, Horizontal, and Hybrid Strategies Explained

This article explores MySQL sharding fundamentals, detailing vertical, horizontal, and combined partitioning methods, their advantages and drawbacks, and surveys various open‑source implementations such as MySQL Proxy, HSCALE, Amoeba, HiveDB, and PL/Proxy, while offering practical guidance on configuration, transaction handling, and common pitfalls.

AmoebaHorizontal ShardingMySQL
0 likes · 22 min read
Mastering MySQL Sharding: Vertical, Horizontal, and Hybrid Strategies Explained
Qunar Tech Salon
Qunar Tech Salon
Nov 29, 2017 · Backend Development

Optimizing the Pricing Engine for High‑Volume Flight Search: Hash Sharding, Caching, and Response‑Time Improvements

This article explains how Qunar's international ticket pricing engine was optimized by redesigning hash rules for sharding, enhancing local cache replication, employing multi‑stage result delivery, and applying common distributed‑system patterns to boost computational capacity and reduce response latency for massive flight‑search workloads.

Pricing EngineShardingcaching
0 likes · 8 min read
Optimizing the Pricing Engine for High‑Volume Flight Search: Hash Sharding, Caching, and Response‑Time Improvements
Architecture Digest
Architecture Digest
Nov 3, 2017 · Databases

Understanding Database Sharding: Why It Matters and How to Implement It

Database sharding, or splitting databases into multiple instances and tables, is essential for scaling backend systems; this article explains its importance, historical context, core concepts, storage stretching techniques, index and consistency handling, and provides a practical design example for transaction systems.

Backend ArchitectureDatabase ScalingIndexing
0 likes · 11 min read
Understanding Database Sharding: Why It Matters and How to Implement It
JD Retail Technology
JD Retail Technology
Oct 16, 2017 · Databases

Design and Evolution of JD Elastic Database: Architecture, Sharding, and Automatic Failover

This article details the evolution of JD's Elastic Database, describing the challenges of scaling MySQL, the staged solutions including sharding, JProxy, and the final elastic architecture with services like Topology, JED‑Gate, and JED‑Tablet, and explains its query processing, dynamic resharding, and automatic failover mechanisms.

DatabasesElastic ArchitectureFailover
0 likes · 11 min read
Design and Evolution of JD Elastic Database: Architecture, Sharding, and Automatic Failover
dbaplus Community
dbaplus Community
Oct 3, 2017 · Databases

What Oracle 18c Sharding and MySQL 8.0 Reveal About Modern Database Evolution

The article reviews Oracle's high‑availability and sharding advances in 18c, including RAC Sharding 2.0 and Active Data Guard features, and examines MySQL 8.0's major improvements such as the removal of MyISAM, data‑dictionary optimizations, self‑tuning, enhanced security, and new performance capabilities, while also comparing MySQL with PostgreSQL.

8.0High AvailabilityMySQL
0 likes · 12 min read
What Oracle 18c Sharding and MySQL 8.0 Reveal About Modern Database Evolution
Meituan Technology Team
Meituan Technology Team
Sep 21, 2017 · Databases

Database Q&A: Book Recommendations, Engine Differences, Optimization, Sharding, and Operational Practices

The article compiles Meituan‑Dianping engineers' Q&A covering SQL book suggestions, relational versus NoSQL engine choices, performance tuning techniques, time‑series database uses, sharding versus partitioning strategies, query handling in distributed systems, proxy limitations, and practical advice on replication and pre‑database setups.

MySQLNoSQLOptimization
0 likes · 12 min read
Database Q&A: Book Recommendations, Engine Differences, Optimization, Sharding, and Operational Practices
Qunar Tech Salon
Qunar Tech Salon
Sep 20, 2017 · Backend Development

Design and Implementation of a One‑Stop Fund Processing Platform

This article describes the design and implementation of a five‑layer one‑stop fund processing platform, covering architecture, rapid business support, hotspot data handling, storage optimization, and query improvements, including sharding, asynchronous processing, and dynamic data source strategies to ensure scalability and reliability.

Dynamic Data SourceShardingasynchronous processing
0 likes · 13 min read
Design and Implementation of a One‑Stop Fund Processing Platform
dbaplus Community
dbaplus Community
Sep 6, 2017 · Databases

How to Build a MongoDB Database-as-a-Service Platform: Architecture, Features, and Best Practices

This article explains the concept of Database-as-a-Service (DBaaS), introduces MongoDB’s core capabilities, outlines the essential components such as lifecycle management, disaster recovery, monitoring, and value‑added services, and provides practical guidance on building a production‑grade MongoDB DBaaS platform.

BackupCloud DatabaseDBaaS
0 likes · 19 min read
How to Build a MongoDB Database-as-a-Service Platform: Architecture, Features, and Best Practices
dbaplus Community
dbaplus Community
Aug 7, 2017 · Databases

Mastering MySQL Architecture: Standards, HA, Sharding, and Redis Integration

This article outlines comprehensive MySQL best practices, covering development and operational standards, high‑availability architecture choices such as Keepalived, MHA and Percona XtraDB Cluster, sharding strategies (vertical and horizontal), and how Redis can be leveraged to offload read pressure.

Database DesignHigh AvailabilityMySQL
0 likes · 19 min read
Mastering MySQL Architecture: Standards, HA, Sharding, and Redis Integration
dbaplus Community
dbaplus Community
Jul 27, 2017 · Databases

How Sharding-JDBC Transforms Relational Databases for Scalable Cloud‑Native Applications

This article explains the core functions of relational‑database middleware, dives into Sharding-JDBC’s architecture, performance characteristics, and implementation details such as sharding rules, SQL parsing, routing, rewrite, execution, result merging, and distributed primary‑key generation, and outlines its future roadmap.

Database MiddlewareSQL RoutingSharding
0 likes · 21 min read
How Sharding-JDBC Transforms Relational Databases for Scalable Cloud‑Native Applications
ITPUB
ITPUB
May 27, 2017 · Databases

Building a High‑Performance OAuth Token Service with Tarantool, Raft, and Sharding

This article explains how we designed and implemented a scalable OAuth token storage and refresh system using Tarantool’s in‑memory database, Raft leader election, sharding across multiple data centers, and a custom lightweight queue to handle high‑throughput token updates while maintaining consistency and fault tolerance.

OAuthRaftSharding
0 likes · 20 min read
Building a High‑Performance OAuth Token Service with Tarantool, Raft, and Sharding
dbaplus Community
dbaplus Community
May 21, 2017 · Databases

Unlocking PostgreSQL: From BRIN Indexes to Real Sharding and Machine Learning

The article summarizes Digoal's 2017 Gdevops talk, detailing PostgreSQL's advanced features such as block-level BRIN indexes, high‑efficiency range queries, asynchronous messaging, data pumps, real sharding, MADlib machine‑learning integration, FDW data federation, and large‑scale use cases in IoT, finance, and geospatial analytics.

BRINData PumpFDW
0 likes · 19 min read
Unlocking PostgreSQL: From BRIN Indexes to Real Sharding and Machine Learning
ITPUB
ITPUB
May 9, 2017 · Databases

Master MySQL Single-Table Optimization and Scaling Strategies

Learn how to boost MySQL performance for massive tables by optimizing schema design, indexing, query patterns, engine selection, system parameters, hardware upgrades, read/write splitting, caching layers, partitioning, vertical and horizontal sharding, and choosing appropriate client or proxy sharding solutions.

MySQLOptimizationPartitioning
0 likes · 25 min read
Master MySQL Single-Table Optimization and Scaling Strategies
Architecture Digest
Architecture Digest
May 6, 2017 · Databases

MongoDB Architecture and Practical Usage at the MamaBang Platform

The talk outlines MamaBang's evolution from MySQL to a multi‑zone MongoDB cluster, compares MongoDB with relational databases, discusses data‑model design, sharding, replication, transaction handling, operational challenges, and offers practical advice for teams considering MongoDB adoption.

Data ModelingDatabase ArchitectureMongoDB
0 likes · 19 min read
MongoDB Architecture and Practical Usage at the MamaBang Platform
dbaplus Community
dbaplus Community
Apr 19, 2017 · Databases

How to Use MariaDB Spider for Horizontal & Vertical Sharding

This article explains what MariaDB Spider is, its role as a pluggable storage engine for MySQL/MariaDB sharding, demonstrates vertical and horizontal partitioning with real‑world Tencent game case, provides installation steps, configuration commands, performance considerations, tunable parameters, and load‑balancing architecture.

MariaDBShardingSpider
0 likes · 8 min read
How to Use MariaDB Spider for Horizontal & Vertical Sharding
Alibaba Cloud Developer
Alibaba Cloud Developer
Apr 6, 2017 · Backend Development

How Alibaba Scaled GitLab to Support Millions of Users with Sharding and High‑Availability

This article details Alibaba Group's journey of transforming its GitLab deployment from a single‑node setup to a distributed, sharded architecture that handles tens of millions of daily requests, achieves near‑perfect reliability, and incorporates performance, monitoring, and disaster‑recovery innovations.

GitLabHigh AvailabilityMonitoring
0 likes · 15 min read
How Alibaba Scaled GitLab to Support Millions of Users with Sharding and High‑Availability
Alibaba Cloud Infrastructure
Alibaba Cloud Infrastructure
Apr 6, 2017 · Operations

Scaling Alibaba Group GitLab: Distributed Architecture, Sharding, and Operational Practices

This article describes how Alibaba Group transformed its GitLab platform from a single‑node deployment to a distributed, sharded architecture with proxy services, performance optimizations, multi‑datacenter backup, and automated operations to support millions of users and dramatically increase request throughput.

GitLabScalingSharding
0 likes · 14 min read
Scaling Alibaba Group GitLab: Distributed Architecture, Sharding, and Operational Practices
Baidu Intelligent Testing
Baidu Intelligent Testing
Mar 22, 2017 · Operations

Load Balancing: Concepts, Mechanisms, and Enterprise Practices

This article explains the principles of load balancing, distinguishes stateless service and stateful data balancing, describes DNS, hardware and software solutions such as F5, HAProxy, LVS, and GSLB, and illustrates real‑world implementations at Alibaba and Tencent while offering practical guidance on sharding, caching, and fault tolerance.

DNSGSLBHAProxy
0 likes · 18 min read
Load Balancing: Concepts, Mechanisms, and Enterprise Practices
ITPUB
ITPUB
Mar 15, 2017 · Databases

From Table Design to Scalable MySQL: A Complete Optimization Roadmap

This article outlines the five evolutionary stages of a MySQL deployment—table design, deployment, performance tuning, architecture scaling, and maintenance—detailing practical configuration tweaks, kernel and hardware optimizations, replication strategies, caching, sharding, partitioning, and essential monitoring and backup techniques.

BackupMySQLSharding
0 likes · 18 min read
From Table Design to Scalable MySQL: A Complete Optimization Roadmap