BestHub
Sign in
Tagged articles

sharding

677 articles · Page 1 of 7
Random Bulletin
Random Bulletin
Oct 1, 2026 · Backend Development

Fault Domain Design: Turning Blast Radius from 100% into a Tunable 1/N Parameter

The article presents a layered fault-domain strategy—physical anti-affinity, logical isolation (sharding, cluster groups, swimlanes, bulkheads), cell-based architecture, chaos-engineering validation, and quantitative governance metrics—to shrink the blast radius of a ten-million-QPS system from a fixed 100% to a controllable 1/N design parameter.

KubernetesSLOanti-affinity
0 likes · 25 min read
Fault Domain Design: Turning Blast Radius from 100% into a Tunable 1/N Parameter
Cloud Architecture
Cloud Architecture
Sep 17, 2026 · Databases

Indexes Aren't Free: Production Index Governance for High-Write Order Systems

This article presents a comprehensive index governance methodology for high-write MySQL order systems, demonstrating through a real incident how a read-optimized index caused write latency, replica lag, and timeouts, and detailing a reusable process covering query-driven design, cost measurement, validation, change buffer limits, index convergence, safe deletion, read-write routing, sharding, transactional outbox, idempotent consumers, online DDL safeguards, gating, and long-term ownership.

Capacity PlanningChange BufferDescending Indexes
0 likes · 29 min read
Indexes Aren't Free: Production Index Governance for High-Write Order Systems
Java Tech Enthusiast
Java Tech Enthusiast
Sep 14, 2026 · Backend Development

Easy-Query: A Strongly-Typed Java ORM with LINQ-Style Queries and Zero Dependencies

This article introduces Easy-Query, a Java ORM framework inspired by .NET's LINQ that enables strongly-typed, chainable query expressions for 90% of database scenarios, demonstrating queries, joins, subqueries, pagination, streaming, dynamic conditions, grouping, and encryption with generated SQL examples.

Database EncryptionEasy-QueryJava ORM
0 likes · 13 min read
Easy-Query: A Strongly-Typed Java ORM with LINQ-Style Queries and Zero Dependencies
dbaplus Community
dbaplus Community
Sep 13, 2026 · Backend Development

Solving Sharding Routing Latency by Embedding Route Keys in Order IDs

A large OTA platform eliminated sharding query latency by encoding a 4-bit routeKey into 64-bit Snowflake order IDs, enabling direct shard lookup without index table queries, reducing P99 latency from over 1 second to tens of milliseconds and cutting database load by 80% through a three-layer fallback strategy.

Performance OptimizationSnowflakebit-manipulation
0 likes · 20 min read
Solving Sharding Routing Latency by Embedding Route Keys in Order IDs
ITPUB
ITPUB
Sep 6, 2026 · Backend Development

How WeChat Resets 1 Billion Step Counts at Midnight Without Crashing

WeChat avoids server crashes during midnight step-count resets for 1 billion users by using logical time-based versioning instead of physical updates, a custom PaxosStore for atomic increments, delayed double-write buffers for clock skew, Redis ZSet sharding for rankings, and asynchronous cold-data archival during low-traffic hours.

High ConcurrencyPaxosStoreRedis
0 likes · 18 min read
How WeChat Resets 1 Billion Step Counts at Midnight Without Crashing
Java Architect Handbook
Java Architect Handbook
Sep 3, 2026 · Databases

Why ElasticSearch Is Blazing Fast: Inverted Indexes, FST, and Distributed Architecture Explained

This article breaks down ElasticSearch's performance advantages across three layers—data structures (inverted index, FST, compressed posting lists), storage (immutable segments, Doc Values), and architecture (shard parallelism, near-real-time writes, multi-level caching)—with concrete examples and interview-focused explanations.

ElasticsearchNear Real-Timecaching
0 likes · 16 min read
Why ElasticSearch Is Blazing Fast: Inverted Indexes, FST, and Distributed Architecture Explained
liandk
liandk
Sep 1, 2026 · Databases

Master MySQL Sharding with Sharding-JDBC: Sorting, Distributed IDs, and Scaling

This article explains what MySQL sharding and partitioning are, when they are necessary, outlines four splitting patterns, details four sharding key strategies, shows a production‑ready Sharding-JDBC configuration, and discusses five common distributed challenges with practical solutions.

MySQLSharding-JDBCdistributed-id
0 likes · 10 min read
Master MySQL Sharding with Sharding-JDBC: Sorting, Distributed IDs, and Scaling
Architect's Guide
Architect's Guide
Aug 26, 2026 · Backend Development

10 Powerful Performance‑Optimization Techniques You Should Try

The article surveys ten practical performance‑optimization tactics—from classic indexing, compression, and caching to prefetching, peak‑shaving, batch processing, and advanced methods such as resource squeezing, horizontal scaling, sharding, and lock‑free designs—explaining their trade‑offs, concrete examples, and when to apply each in real‑world systems.

Batch ProcessingCompressionIndexing
0 likes · 36 min read
10 Powerful Performance‑Optimization Techniques You Should Try
samdeepthink
samdeepthink
Aug 23, 2026 · Databases

Is Splitting Data Across Machines Distributed? Understand Sharding vs Distributed Systems

The article explains that merely placing data on multiple machines constitutes sharding—a way to split data for capacity—but true distributed systems require coordinated nodes that communicate, replicate, and handle failures, illustrated with an e‑commerce warehouse analogy and guidance on choosing between sharding and distributed databases.

System Designdatabase architecturedistributed systems
0 likes · 8 min read
Is Splitting Data Across Machines Distributed? Understand Sharding vs Distributed Systems
Ray's Galactic Tech
Ray's Galactic Tech
Aug 22, 2026 · Backend Development

High-Concurrency Wallet Design: Hot Accounts, Idempotent Deductions, and Sharding Practices

The article analyses why wallet balance deduction is more error‑prone than inventory, outlines the root causes of a real‑world 60 k TPS outage, and presents a production‑grade architecture that separates strong‑consistent user deductions from asynchronous hot‑account crediting using idempotent requests, unique DB constraints, sharding, outbox messaging, and comprehensive monitoring to guarantee financial correctness under extreme load.

High ConcurrencyOutboxidempotency
0 likes · 32 min read
High-Concurrency Wallet Design: Hot Accounts, Idempotent Deductions, and Sharding Practices
Mike Chen Rui
Mike Chen Rui
Aug 21, 2026 · Databases

Mastering MySQL Sharding: Principles, Architecture, and Real‑World Implementation

The article explains why high‑traffic MySQL deployments hit performance limits, introduces the concepts of vertical and horizontal sharding, and provides a step‑by‑step guide—including necessity assessment, shard key selection, schema design, middleware integration, and data migration—using an e‑commerce order system as a concrete example.

Database ScalingHorizontal ShardingMySQL
0 likes · 5 min read
Mastering MySQL Sharding: Principles, Architecture, and Real‑World Implementation
Ray's Galactic Tech
Ray's Galactic Tech
Aug 20, 2026 · Backend Development

Payment Reconciliation: Detecting Lost Orders, Over‑payments and Auto‑Repairing Them

The article explains why successful payment does not guarantee correct accounting, defines discrepancy types such as lost orders, over‑payments and shortfalls, and presents a production‑grade reconciliation architecture with T+1 bill ingestion, shard‑based diff detection, state‑machine driven error handling, automatic repair workflows, and comprehensive monitoring.

auto-repairlost ordersmonitoring
0 likes · 45 min read
Payment Reconciliation: Detecting Lost Orders, Over‑payments and Auto‑Repairing Them
Ray's Galactic Tech
Ray's Galactic Tech
Aug 20, 2026 · Backend Development

Why Wallet Balance Shouldn't Be Calculated On‑The‑Fly: Double‑Entry Accounting, Snapshots, and Immutable Ledger Practices

The article explains that a production‑grade wallet must store balance snapshots instead of summing transaction flows, using double‑entry bookkeeping, ACID guarantees, idempotent request handling, outbox messaging, sharding, and comprehensive reconciliation to ensure correctness and high‑throughput under heavy concurrency.

High ConcurrencyOutbox Patternbalance snapshot
0 likes · 43 min read
Why Wallet Balance Shouldn't Be Calculated On‑The‑Fly: Double‑Entry Accounting, Snapshots, and Immutable Ledger Practices
YiSu Grain
YiSu Grain
Aug 19, 2026 · Databases

Optimizing Slow Queries, Sharding and Cache Consistency for Appointment System

This article walks through a comprehensive case study of a provincial medical appointment platform, diagnosing slow‑query bottlenecks, proposing patient‑id + create_time composite indexes, designing read‑write separation with replication, selecting patient_id for horizontal sharding, and implementing cache‑aside strategies to ensure consistency while handling cache avalanche, penetration and thundering‑herd scenarios.

IndexingMySQLRedis
0 likes · 30 min read
Optimizing Slow Queries, Sharding and Cache Consistency for Appointment System
samdeepthink
samdeepthink
Aug 18, 2026 · Databases

When a Single Database Hits Its Limits: Sharding Principles and Migration Path

The article explains why sharding should be a last‑resort scaling option, describes vertical and horizontal partitioning, compares hash, range and directory sharding schemes, discusses hotspots, routing, data colocation, migration steps and tool choices, and warns about the operational complexity involved.

Data ConsistencyDatabase ScalingHorizontal Partitioning
0 likes · 16 min read
When a Single Database Hits Its Limits: Sharding Principles and Migration Path
Code Farming
Code Farming
Aug 14, 2026 · Backend Development

How Weibo Handles 100k QPS During Viral Peaks: Architectural Secrets

The article breaks down how Weibo’s backend was engineered to sustain 100,000 queries per second during traffic surges by first quantifying load, then using a push‑pull hybrid feed, a four‑tier cache funnel, and user‑ID sharding with write throttling.

High Concurrencybackend designcaching
0 likes · 8 min read
How Weibo Handles 100k QPS During Viral Peaks: Architectural Secrets
Cloud Architecture
Cloud Architecture
Aug 12, 2026 · Databases

How Redis Cluster’s Decentralized Design Powers Billion‑Scale Traffic

When a single Redis instance can no longer hold the data volume or write load of e‑commerce workloads, the traditional master‑slave with Sentinel model reaches its limits, and Redis Cluster—by sharding data across 16,384 slots, using gossip‑based topology, and removing a central control plane—delivers horizontal scaling and fault‑tolerance for billions of requests, provided key design, hash tags, hot‑key mitigation, and client routing are applied.

Rediscachecluster
0 likes · 32 min read
How Redis Cluster’s Decentralized Design Powers Billion‑Scale Traffic
YiSu Grain
YiSu Grain
Aug 8, 2026 · Databases

Day 51: Database Architecture – Master‑Slave Replication, Read‑Write Separation, Sharding & Consistency

The article walks through diagnosing database bottlenecks, explains MySQL replication flow, read‑write separation benefits and limits, shows when to apply vertical versus horizontal partitioning, details sharding‑key selection, cross‑shard challenges, high‑availability steps, and presents a complete e‑commerce case study.

Database ReplicationHorizontal PartitioningRead-Write Separation
0 likes · 29 min read
Day 51: Database Architecture – Master‑Slave Replication, Read‑Write Separation, Sharding & Consistency
Xike
Xike
Aug 7, 2026 · Databases

Embedding Sharding Genes in Business IDs for Direct Routing in Sharded Databases

By embedding a shard identifier (“gene”) into the low bits of a business ID generated via Redis INCR or similar sequencers, the article shows how to achieve direct table routing without broadcast queries or extra mapping tables, detailing the algorithm, implementation, integration with Snowflake and Leaf, and common pitfalls.

JavaLeafRedis
0 likes · 11 min read
Embedding Sharding Genes in Business IDs for Direct Routing in Sharded Databases
DeepNoMind
DeepNoMind
Aug 2, 2026 · Databases

Understand Partitioning vs Sharding in 5 Minutes

The article explains how partitioning splits tables within a single database and how sharding distributes data across multiple database instances, comparing their types, advantages, limitations, and trade‑offs, and provides practical examples and a decision framework for choosing the right strategy.

Partitioningdatabasesdistributed systems
0 likes · 7 min read
Understand Partitioning vs Sharding in 5 Minutes
Ray's Galactic Tech
Ray's Galactic Tech
Aug 1, 2026 · Databases

Beyond CRUD: Full‑Scale Production Guide for MySQL 8.4 LTS

This article walks through a complete production‑grade view of MySQL 8.4 LTS, explaining how a chain of traffic spikes, connection‑pool exhaustion, long transactions and replication lag can cause an avalanche, and then detailing the five core modules, seven production mechanisms, architectural evolution steps, incident post‑mortems, and concrete configuration and code examples to build a resilient MySQL service.

InnoDBMySQLObservability
0 likes · 36 min read
Beyond CRUD: Full‑Scale Production Guide for MySQL 8.4 LTS
Code Farming
Code Farming
Jul 29, 2026 · Backend Development

Billions of Logs in Seconds: How Elasticsearch Makes It Possible

The article explains how Elasticsearch achieves sub‑second search over billions of log entries by combining sharding, immutable segment writes, a three‑layer inverted index, and a two‑phase query‑then‑fetch process that distributes work across nodes.

Distributed SearchElasticsearchinverted index
0 likes · 6 min read
Billions of Logs in Seconds: How Elasticsearch Makes It Possible
Java Architect Handbook
Java Architect Handbook
Jul 26, 2026 · Databases

Interview Question: What Problems Arise After Sharding? List the Key Issues

The article outlines six major challenges introduced by database sharding—cross‑database joins, pagination, distributed transactions, global ID generation, data migration, and aggregation—along with practical solutions such as binding tables, cursor pagination, Seata AT mode, Snowflake IDs, dual‑write scaling, and summary tables, providing interview‑ready answers.

SeataSnowflakedatabase partitioning
0 likes · 17 min read
Interview Question: What Problems Arise After Sharding? List the Key Issues
Dabaoshi
Dabaoshi
Jul 26, 2026 · Databases

When to Scale MySQL: From Single Instance to Distributed Architecture

The article explains why a single MySQL server eventually hits read, write, or availability limits, outlines a step‑by‑step evolution—from SQL tuning and caching to read‑write splitting, high‑availability setups, and finally vertical or horizontal sharding—while warning against premature distribution.

MySQLRead-Write Splittingdistributed systems
0 likes · 15 min read
When to Scale MySQL: From Single Instance to Distributed Architecture
Java Architect Handbook
Java Architect Handbook
Jul 24, 2026 · Databases

What Are Database Sharding, Table Sharding, and Combined Sharding? – Xianyu Interview Guide

The article explains the differences between database sharding, table sharding, and combined sharding, outlines interview expectations, details vertical and horizontal splitting strategies, discusses when to apply each technique, and reviews practical tools such as ShardingSphere and MyCat for Java projects.

BackendInterview PreparationMySQL
0 likes · 10 min read
What Are Database Sharding, Table Sharding, and Combined Sharding? – Xianyu Interview Guide
Cloud Architecture
Cloud Architecture
Jul 20, 2026 · Databases

Designing MySQL for Millions of QPS: From Single Server to Distributed Architecture

The article walks through a real‑world order system that spikes to 300,000 QPS, explaining why the original single‑node MySQL design fails, and detailing a step‑by‑step evolution—index tuning, transaction fixes, read‑write splitting, vertical and horizontal sharding, plus data‑pipeline integration—to achieve stable low latency at massive scale.

Database ScalingIndex OptimizationMySQL
0 likes · 20 min read
Designing MySQL for Millions of QPS: From Single Server to Distributed Architecture
Top Architect
Top Architect
Jul 20, 2026 · Databases

Why You Should Stop Using Snowflake for IDs and Try a Shorter MySQL Auto‑Increment Solution

The article examines the drawbacks of using Snowflake for generating short numeric user IDs, details a MySQL auto‑increment based approach, reveals deadlock problems with REPLACE INTO, evaluates alternative schemes, and presents a final sharding‑friendly short‑ID design that meets performance and usability requirements.

MySQLSnowflakeauto-increment
0 likes · 14 min read
Why You Should Stop Using Snowflake for IDs and Try a Shorter MySQL Auto‑Increment Solution
samdeepthink
samdeepthink
Jul 17, 2026 · Databases

Designing Redis Leaderboards: From a Single ZSet to Billion‑Scale Rankings

The article walks through the evolution of Redis leaderboard architectures, covering why ZSets are used, handling same‑score ordering, seasonal resets, access patterns, synchronous vs asynchronous updates, big‑key and hot‑key issues, sharding with buckets, and data durability strategies.

LeaderboardRedisZset
0 likes · 12 min read
Designing Redis Leaderboards: From a Single ZSet to Billion‑Scale Rankings
Code Farming
Code Farming
Jul 13, 2026 · Backend Development

Designing Trillion‑Scale Counters: From MySQL to a Custom Redis Engine

The article dissects a proven trillion‑level counter architecture, tracing its evolution from a simple MySQL table through hash sharding, a full Redis migration, deep Redis memory optimizations, and hot‑cold separation, while detailing the trade‑offs and performance gains at each step.

High ConcurrencyRedis optimizationcounter architecture
0 likes · 6 min read
Designing Trillion‑Scale Counters: From MySQL to a Custom Redis Engine
IoT Full-Stack Technology
IoT Full-Stack Technology
Jul 13, 2026 · Databases

Is Sharding Dead? A Technical Comparison with NewSQL Databases

The article objectively compares middleware‑based sharding with NewSQL distributed databases, examining distributed transactions, CAP constraints, HA, scaling, storage engines, maturity, and ecosystem, and offers a decision framework for choosing the appropriate architecture based on concrete requirements.

CAP theoremNewSQLdatabase architecture
0 likes · 18 min read
Is Sharding Dead? A Technical Comparison with NewSQL Databases
Cloud Architecture
Cloud Architecture
Jul 12, 2026 · Databases

Database Performance Optimization: 100× Speed Gains Without Changing SQL

Even without rewriting any SQL, database performance can improve up to a hundredfold by first diagnosing bottlenecks, reducing unnecessary traffic, layering read paths, optimizing indexes, tuning connection pools, and progressively evolving from a single‑node setup to read‑write separation, sharding, and distributed read models.

MySQLPerformance OptimizationRead-Write Separation
0 likes · 38 min read
Database Performance Optimization: 100× Speed Gains Without Changing SQL
Code Farming
Code Farming
Jul 7, 2026 · Backend Development

Four Ways to Ensure Unique IDs After Database Sharding

When a system is split into multiple databases and tables, the article explains the root cause of ID collisions and compares four mainstream ID generation strategies—UUID, database auto‑increment, Snowflake, and segment allocation—detailing their trade‑offs, pitfalls, and selection guidelines for different concurrency levels.

High ConcurrencySnowflakeUUID
0 likes · 9 min read
Four Ways to Ensure Unique IDs After Database Sharding
Linyb Geek Road
Linyb Geek Road
Jun 30, 2026 · Backend Development

How to Design Pagination for Billion‑Row Sharded Databases in an Interview

The article systematically breaks down pagination challenges in billion‑row sharded databases, compares common sharding strategies and middleware architectures, analyzes the performance drawbacks of a naïve global‑query approach, and presents several practical alternatives—including keyset pagination, two‑stage queries, index‑table tricks, and external search or NewSQL solutions—while highlighting their trade‑offs for interview discussions.

SQLdistributed databasespagination
0 likes · 24 min read
How to Design Pagination for Billion‑Row Sharded Databases in an Interview
Linyb Geek Road
Linyb Geek Road
Jun 30, 2026 · Databases

Implementing Efficient Pagination Across Sharded Databases

The article analyzes why traditional LIMIT/OFFSET pagination fails when data is split across multiple databases, presents a global query approach with its trade‑offs, and proposes an optimized "no‑skip" method plus practical tips using ShardingSphere and Elasticsearch.

ElasticsearchShardingSpheredatabase
0 likes · 7 min read
Implementing Efficient Pagination Across Sharded Databases
Shepherd Advanced Notes
Shepherd Advanced Notes
Jun 24, 2026 · Backend Development

Boosting Throughput 10×: Architecture Evolution and Tuning for High‑Concurrency Batch Processing

The article details how a sluggish batch‑processing system handling millions of records was redesigned with XXL‑JOB sharding, Redis‑based dynamic task distribution, cursor pagination, and selective transaction scopes, achieving nearly ten‑fold throughput improvement while addressing resource contention, load‑balancing, and reliable result reconciliation.

Batch ProcessingJavaMySQL
0 likes · 19 min read
Boosting Throughput 10×: Architecture Evolution and Tuning for High‑Concurrency Batch Processing
samdeepthink
samdeepthink
Jun 22, 2026 · Databases

How to Handle 30,000 Writes per Second with Oracle, Java, and Spring

The article analyzes the write bottleneck of processing 30,000 payment orders per second on an Oracle‑Java‑Spring stack, explains why sharding was used historically, why MQ‑based peak‑shaving is discouraged, and compares self‑built sharding with modern distributed databases such as PolarDB‑X, TiDB and OceanBase, while summarizing public practices from major Chinese tech firms.

MySQLOceanBaseOracle
0 likes · 13 min read
How to Handle 30,000 Writes per Second with Oracle, Java, and Spring
Code Farming
Code Farming
Jun 20, 2026 · Backend Development

How This Architecture Handles Tens‑Fold Traffic Spikes Without Crashing

The article breaks down a complete flash‑sale system into four phases and explains how Redis distributed locks, CDN static pages, Nginx rate limiting, message‑queue peak shaving, and sharding together prevent overselling, crashes, and lost orders even when traffic surges dozens of times.

Flash SaleHigh ConcurrencyNginx
0 likes · 6 min read
How This Architecture Handles Tens‑Fold Traffic Spikes Without Crashing
Architecture Digest
Architecture Digest
May 15, 2026 · Databases

Why Alibaba Bans Joins Over Three Tables – A Must‑Know Rule for SQL Engineers

Alibaba’s Java Development Manual mandates that any SQL involving more than three tables must be avoided, a rule that stems from the exponential cost of multi‑table joins in a single‑instance database, prompting engineers to rethink data modeling, adopt denormalization, wide tables, materialized views, CQRS or application‑level assembly instead of relying on complex joins.

AlibabaCQRSJOIN
0 likes · 12 min read
Why Alibaba Bans Joins Over Three Tables – A Must‑Know Rule for SQL Engineers
Shuge Unlimited
Shuge Unlimited
Apr 30, 2026 · Databases

Milvus VTS Deep Dive: Two Write Modes, Sharding, and Migration Best Practices

The article provides a source‑code level analysis of Milvus Vector Transport Service (VTS), detailing its three‑stage architecture, partition‑aware sharding logic, two distinct sink write mechanisms (BufferBatchWriter and BulkWriter), schema conversion rules, error‑handling strategies, performance tuning parameters, and practical configuration examples for efficient vector data migration across various data sources.

BufferBatchWriterBulkWriterMilvus
0 likes · 20 min read
Milvus VTS Deep Dive: Two Write Modes, Sharding, and Migration Best Practices
ITPUB
ITPUB
Apr 29, 2026 · Backend Development

How to Rescue a System When QPS Jumps 100× in 10 Minutes

When a service experiences a sudden 100‑fold QPS surge, this guide walks through immediate emergency measures such as rate limiting, circuit breaking, and traffic shedding, followed by systematic analysis of traffic origins, robust architectural redesign including horizontal scaling, micro‑service decomposition, sharding, pooling, caching, and asynchronous processing, and finally stress testing to ensure resilience.

Circuit BreakingMicroservicesRate Limiting
0 likes · 13 min read
How to Rescue a System When QPS Jumps 100× in 10 Minutes
Cloud Architecture
Cloud Architecture
Apr 27, 2026 · Backend Development

Building an Enterprise‑Level MyBatis Persistence Layer from Zero to One

The article walks through a real production incident caused by a massive IN‑list query, then presents a complete methodology for designing, implementing, and tuning an enterprise‑grade MyBatis persistence layer—including core execution chain, caching strategies, batch processing, read/write splitting, sharding, observability, and deployment best practices.

Batch ProcessingMicroservicesMyBatis
0 likes · 39 min read
Building an Enterprise‑Level MyBatis Persistence Layer from Zero to One
Java Companion
Java Companion
Apr 26, 2026 · Databases

Is Sharding Ready to Retire? Why the Classic Split‑Database Approach Is Becoming Legacy

The article reviews the rise and decline of traditional database sharding, explains its technical pitfalls such as wrong shard keys, cross‑database joins, and distributed transactions, compares it with emerging NewSQL solutions like TiDB, OceanBase and PolarDB‑X, and offers practical criteria for choosing the right architecture.

Database ScalingNewSQLOceanBase
0 likes · 14 min read
Is Sharding Ready to Retire? Why the Classic Split‑Database Approach Is Becoming Legacy
Cloud Architecture
Cloud Architecture
Apr 25, 2026 · Databases

How a Phone Number Field Can Crash a 2B‑User Database – Architect’s Design Guide

A mis‑designed phone number column can silently degrade index performance, overload CPU, and cause a cascade of timeouts in a 2‑billion‑user system, but by treating the phone as a domain value object, using a normalized VARCHAR, aligning indexing rules, and applying consistent sharding and migration strategies, you can prevent the database from collapsing under high concurrency.

High ConcurrencyMySQLValue Object
0 likes · 31 min read
How a Phone Number Field Can Crash a 2B‑User Database – Architect’s Design Guide
Sohu Tech Products
Sohu Tech Products
Apr 22, 2026 · Databases

How to Resolve ID Conflicts After Sharding? 5 Proven Solutions

When sharding databases, independent auto‑increment IDs can collide, causing data integrity issues; this article explains why the conflict occurs and walks through five practical solutions—including UUID, custom auto‑increment steps, segment mode, Snowflake, and Meituan Leaf—detailing their trade‑offs, performance, and implementation steps.

LeafSnowflakeUUID
0 likes · 15 min read
How to Resolve ID Conflicts After Sharding? 5 Proven Solutions
Architecture & Thinking
Architecture & Thinking
Apr 22, 2026 · Backend Development

Why Traditional Limit‑Offset Fails on Billion‑Row Tables and How to Fix It

This article dissects the performance collapse of classic LIMIT‑OFFSET pagination on tables with hundreds of millions of rows, explains the underlying execution steps, presents benchmark data, and walks through a progressive set of solutions—including index tuning, Seek pagination, Cursor pagination, and sharding—complete with Go code examples and practical trade‑offs.

Cursor PaginationMySQLgolang
0 likes · 30 min read
Why Traditional Limit‑Offset Fails on Billion‑Row Tables and How to Fix It
Architect's Guide
Architect's Guide
Apr 18, 2026 · Databases

How to Import 1 Billion Records into MySQL at Lightning Speed

This article analyzes the constraints of loading a billion 1‑KB log records from HDFS/S3 into MySQL, evaluates single‑table limits, proposes batch inserts, sharding, file‑reading techniques, task coordination with Redis, Redisson, and Zookeeper, and offers practical performance‑tuning recommendations.

Bulk InsertMySQLRedis
0 likes · 21 min read
How to Import 1 Billion Records into MySQL at Lightning Speed
Java Companion
Java Companion
Apr 17, 2026 · Backend Development

Resolving ID Conflicts After Sharding: 5 Practical Solutions

When sharding databases, independent auto‑increment IDs can collide; this article explains why the conflict occurs and evaluates five concrete solutions—UUID, MySQL auto‑increment step, segment allocation, Snowflake algorithm, and Meituan Leaf—detailing their mechanisms, trade‑offs, performance, and suitability.

Meituan LeafMySQLSnowflake
0 likes · 14 min read
Resolving ID Conflicts After Sharding: 5 Practical Solutions
Lobster Programming
Lobster Programming
Apr 15, 2026 · Databases

Choosing the Right Redis Architecture: From Single Node to Cluster

This article reviews the main Redis deployment options—including single‑node, master‑slave with Sentinel, sharding via consistent hashing, and Redis Cluster—explaining their advantages, high‑availability mechanisms, scalability limits, and recommending suitable scenarios for each architecture.

Redisclusterdeployment
0 likes · 7 min read
Choosing the Right Redis Architecture: From Single Node to Cluster
IT Services Circle
IT Services Circle
Apr 11, 2026 · Databases

Why Sharding Isn’t Dead: Modern Alternatives and When to Use Them

The article revisits the rise and fall of database sharding, explains why it became problematic, and evaluates newer cloud‑native, distributed‑SQL, and serverless databases as modern replacements, offering a practical four‑step guide to help engineers choose the right solution for their workload and team.

Database ScalingDistributed SQLServerless
0 likes · 23 min read
Why Sharding Isn’t Dead: Modern Alternatives and When to Use Them
Cloud Architecture
Cloud Architecture
Apr 9, 2026 · Databases

Redis Cluster Deep Dive: Sharding, Replication, Failover & Smart Client

This article provides a comprehensive technical analysis of Redis Cluster, covering its data sharding via 16,384 hash slots, master‑slave replication, automated failover mechanisms, and the essential role of smart clients in routing, along with practical engineering guidelines and production‑grade code examples.

FailoverRedisReplication
0 likes · 41 min read
Redis Cluster Deep Dive: Sharding, Replication, Failover & Smart Client
Architect's Guide
Architect's Guide
Apr 3, 2026 · Databases

How to Diagnose and Solve MySQL Table Size Bottlenecks

This article explains how to assess MySQL table size using table capacity, disk usage, and instance limits, reveals why large tables slow down due to B+‑tree overhead, and presents three practical solutions—partitioning, sharding, and hot‑cold archiving—to restore query performance.

B+ TreeHot Cold ArchivingMySQL
0 likes · 17 min read
How to Diagnose and Solve MySQL Table Size Bottlenecks
Tech Freedom Circle
Tech Freedom Circle
Mar 17, 2026 · Databases

Why HyperLogLog Misses 100M Daily Active Users and How Bitmap Solves It

The article dissects an Alibaba interview question on counting 100 million daily active users, showing why HyperLogLog’s error and lack of per‑user state make it unsuitable, and presents a detailed Bitmap‑based architecture—including sharding, pre‑computation, and ClickHouse integration—to achieve precise, high‑performance analytics.

BitmapClickHouseDailyActiveUsers
0 likes · 16 min read
Why HyperLogLog Misses 100M Daily Active Users and How Bitmap Solves It
Selected Java Interview Questions
Selected Java Interview Questions
Mar 9, 2026 · Databases

How to Paginate Efficiently Across Sharded Databases Without OOM

This article explains why pagination only becomes complex when querying across sharded databases, compares three common solutions—including global search, sequential ID‑based paging, and the discouraged secondary query method—provides practical SQL examples, outlines pros and cons, and offers a STAR‑style interview answer.

SQLpaginationsharding
0 likes · 6 min read
How to Paginate Efficiently Across Sharded Databases Without OOM
Senior Xiao Ying
Senior Xiao Ying
Feb 26, 2026 · Databases

Vertical vs Horizontal Sharding in MySQL: Strategy Selection and Middleware Guide

The article explains vertical and horizontal sharding concepts, compares their advantages and drawbacks, evaluates popular sharding middleware such as ShardingSphere, details distributed ID generation with Snowflake, offers cross‑database query solutions, and provides practical guidance on when and how to apply sharding in MySQL.

Cross-Database QueriesHorizontal ShardingMySQL
0 likes · 13 min read
Vertical vs Horizontal Sharding in MySQL: Strategy Selection and Middleware Guide
Architect's Guide
Architect's Guide
Feb 26, 2026 · Backend Development

8 Essential Software Architecture Patterns and When to Use Them

This article explains eight common software architecture patterns—from single‑database apps to microservices, caching, sharding, elastic scaling and multi‑datacenter deployment—detailing their designs, typical use cases, advantages, drawbacks, and practical implementation steps.

Design PatternsElastic Scalingbackend scaling
0 likes · 23 min read
8 Essential Software Architecture Patterns and When to Use Them
DeepNoMind
DeepNoMind
Feb 19, 2026 · Artificial Intelligence

High‑Reliability AI Agent Design: Sharding and Decentralized Retrieval

This article explains how sharding a knowledge base and using decentralized retrieval can boost the reliability, precision, and latency of AI agents, providing code examples, a LangGraph workflow, and a performance comparison that shows a 28% speed gain over a monolithic setup.

AI agentsLangChainRAG
0 likes · 12 min read
High‑Reliability AI Agent Design: Sharding and Decentralized Retrieval
IT Learning Made Simple
IT Learning Made Simple
Feb 13, 2026 · Databases

Designing Databases as Intelligent Warehouses: A Supermarket Analogy

This article explains core database design concepts—from relational vs. NoSQL choices and normalization rules to indexing strategies, ACID transactions, SQL optimization, and sharding—using supermarket‑style analogies and concrete exam‑style examples to help readers build fast, safe, and scalable data stores.

IndexingSQL optimizationTransactions
0 likes · 38 min read
Designing Databases as Intelligent Warehouses: A Supermarket Analogy
Selected Java Interview Questions
Selected Java Interview Questions
Feb 4, 2026 · Databases

How to Handle Pagination Across Sharded Databases Without Performance Pitfalls

This article explains why pagination only becomes complex when queries span multiple database shards, compares three mainstream solutions—including middleware, open‑source frameworks, and business compromises—highlights the pitfalls of global search for deep pages, and recommends sequential paging based on the previous page’s max ID, with interview‑ready STAR guidance.

SQLinterviewpagination
0 likes · 7 min read
How to Handle Pagination Across Sharded Databases Without Performance Pitfalls
Sohu Tech Products
Sohu Tech Products
Jan 28, 2026 · Backend Development

How We Evolved a News App Comment System: From Threaded Views to AI‑Driven Ranking

This article details the evolution of a news‑app comment backend, covering early thread‑based displays, the transition to sharded databases and mixed adjacency‑path models, current hot‑comment ranking strategies, an in‑house experiment platform, topic aggregation via Kafka, and future AI‑driven architectural enhancements.

AIBackendcomment system
0 likes · 16 min read
How We Evolved a News App Comment System: From Threaded Views to AI‑Driven Ranking
Architect's Guide
Architect's Guide
Jan 27, 2026 · Databases

How to Scale an Order System with Sharding, Distributed IDs, and Seata Transactions

This article details a comprehensive redesign of a high‑traffic order system, covering the challenges of massive data volume, concurrency pressure, and poor scalability, and presenting a step‑by‑step solution that includes sharding strategy selection, database product comparison, unique ID generation, middleware choice, migration plan, risk mitigation, and FAQ handling.

Database ScalingSeatabackend architecture
0 likes · 17 min read
How to Scale an Order System with Sharding, Distributed IDs, and Seata Transactions
Architect's Guide
Architect's Guide
Jan 17, 2026 · Backend Development

How We Split a 500M‑Row MySQL Table: Lessons and Strategies

Facing a 50‑million‑row financial transaction table that grew 600,000 rows each month, the team designed a sharding solution using sharding‑jdbc, tackled multi‑datasource transaction and pagination challenges, and executed a staged migration that kept the system stable while eliminating MySQL performance bottlenecks.

data-migrationpaginationsharding
0 likes · 13 min read
How We Split a 500M‑Row MySQL Table: Lessons and Strategies
Raymond Ops
Raymond Ops
Jan 9, 2026 · Databases

Master MongoDB Sharding: From Single Server to Enterprise-Scale Cluster

When a single‑node MongoDB instance can no longer handle tens of millions of records, this guide walks you through the theory, architecture, deployment steps, shard key strategies, performance tuning, monitoring, backup, and troubleshooting needed to build a robust, production‑grade sharded cluster.

Database ScalingMongoDBbackup
0 likes · 14 min read
Master MongoDB Sharding: From Single Server to Enterprise-Scale Cluster
Ray's Galactic Tech
Ray's Galactic Tech
Jan 8, 2026 · Databases

Boost SQL Performance Without Rewriting Queries: Indexes, Partitioning, Caching

This guide presents a comprehensive, step‑by‑step roadmap for accelerating slow SQL queries without altering the original statements, covering index creation, database parameter tuning, table partitioning, caching layers, read‑write splitting, sharding, statistics updates, hardware choices, and middleware routing.

Performance OptimizationSQLcaching
0 likes · 8 min read
Boost SQL Performance Without Rewriting Queries: Indexes, Partitioning, Caching
Smart Sea Tide
Smart Sea Tide
Jan 7, 2026 · Databases

Designing Distributed System Architecture with Elasticsearch

The article explains Elasticsearch's cluster architecture, covering nodes, shards, replicas, two deployment models, data‑layer design, and the trade‑offs of each approach, while also comparing alternative distributed storage architectures and their impact on reliability and performance.

Data StorageDeployment ModesDistributed Architecture
0 likes · 14 min read
Designing Distributed System Architecture with Elasticsearch
java1234
java1234
Jan 3, 2026 · Backend Development

Ditch the Heavyweight XXL‑Job: An Elegant Nacos‑Based Scheduling Solution

The article analyses the friction between XXL‑Job and a Nacos‑centric stack, proposes the JobFlow design that removes redundant registration, adds full‑link TraceId, strong sharding with distributed locks, intelligent retries and cloud‑native configuration, and demonstrates how these changes simplify operations and improve observability in microservice environments.

JobFlowMicroservicesNacos
0 likes · 19 min read
Ditch the Heavyweight XXL‑Job: An Elegant Nacos‑Based Scheduling Solution
Tech Freedom Circle
Tech Freedom Circle
Jan 3, 2026 · Databases

How Many Values Can MySQL IN Handle? 100K‑10M IDs and the Ultimate Solution

An interview scenario reveals that using a massive IN list (hundreds of thousands of IDs) in MySQL quickly exceeds packet limits, forces full‑table scans, and overloads CPU, while batch IN queries cause network storms; the article dissects these pitfalls and presents three progressive strategies—splitting IN, temporary tables with JOIN, and sharding with stream‑merge—to safely query massive ID sets.

IN ClauseMySQLQuery Optimization
0 likes · 32 min read
How Many Values Can MySQL IN Handle? 100K‑10M IDs and the Ultimate Solution
Java Companion
Java Companion
Jan 3, 2026 · Cloud Native

Ditch the Bulky XXL‑Job? Try This Elegant Nacos‑Based Scheduling Solution

The article analyzes the friction between XXL‑Job and Nacos in cloud‑native environments, proposes the JobFlow design that removes redundant registration and configuration, adds full‑traceability, true sharding with distributed locks, smart retries and cloud‑native configuration, and demonstrates how these changes improve consistency, observability and operational cost.

JobFlowMicroservicesNacos
0 likes · 19 min read
Ditch the Bulky XXL‑Job? Try This Elegant Nacos‑Based Scheduling Solution
Architect
Architect
Dec 28, 2025 · Databases

How to Scale an Order System: Sharding, ID Generation, and Database Choices

This article walks through the challenges of a growing order system, analyzes current bottlenecks, compares sharding and distributed database products, proposes a unique ID strategy, outlines sharding key selection, presents a migration plan with ShardingSphere‑JDBC, and discusses risks and FAQs for a robust backend architecture.

Database ScalingUnique IDdistributed transactions
0 likes · 18 min read
How to Scale an Order System: Sharding, ID Generation, and Database Choices
macrozheng
macrozheng
Dec 26, 2025 · Databases

NewSQL vs Middleware Sharding: Which Architecture Really Wins?

This article objectively compares middleware‑based sharding solutions with native NewSQL distributed databases, examining their architectural differences, transaction handling, high‑availability, scaling, SQL support, storage engines, and maturity to help engineers decide which approach best fits their workload.

CAP theoremNewSQLPaxos
0 likes · 20 min read
NewSQL vs Middleware Sharding: Which Architecture Really Wins?
Architect Chen
Architect Chen
Dec 23, 2025 · Databases

Boost MySQL to 10k QPS: Multi‑Layer Optimization Strategies

When MySQL QPS reaches 10,000, common bottlenecks include CPU saturation, low buffer pool hit rate, connection limits, and lock contention, which can be mitigated through internal tuning, cache interception, read‑write splitting, and horizontal sharding.

MySQLOptimizationQPS
0 likes · 4 min read
Boost MySQL to 10k QPS: Multi‑Layer Optimization Strategies
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Dec 21, 2025 · Backend Development

How Elasticsearch Scales to Billions of Queries: Sharding, Inverted Index, Distributed Execution, and Replication

Elasticsearch achieves billion‑scale search performance by combining horizontal sharding, immutable inverted‑index segments, a two‑stage distributed Query/FETCH model, and multiple replicas with a coordinator node to ensure high concurrency, scalability, and availability.

Distributed QueryElasticsearchReplication
0 likes · 4 min read
How Elasticsearch Scales to Billions of Queries: Sharding, Inverted Index, Distributed Execution, and Replication
Su San Talks Tech
Su San Talks Tech
Dec 11, 2025 · Databases

How to Scale a 900 M‑Row Message Table to 60 B Rows Without Downtime

This article details a comprehensive sharding strategy for a high‑traffic message table, explaining why early partitioning is essential, how to design language‑based vertical shards combined with hash‑based horizontal shards, calculate table counts and virtual nodes, and execute a zero‑downtime migration using dual‑writes and gradual traffic rollout.

Horizontal PartitioningMySQLPerformance Optimization
0 likes · 16 min read
How to Scale a 900 M‑Row Message Table to 60 B Rows Without Downtime
Java Architect Handbook
Java Architect Handbook
Nov 30, 2025 · Databases

When Sharding Fails: Debugging Pagination Bugs and Optimizing MySQL for Massive Tables

The article recounts a real‑world case where a massive device table (over 500 million rows) suffered from sharding and Elasticsearch integration issues, describes a pagination bug caused by Sharding‑JDBC, and presents a comprehensive set of hardware, software, and SQL optimization techniques—including MySQL configuration examples—to improve performance.

Database OptimizationElasticsearchMySQL
0 likes · 13 min read
When Sharding Fails: Debugging Pagination Bugs and Optimizing MySQL for Massive Tables
Code Wrench
Code Wrench
Nov 27, 2025 · Databases

Build a Mini Olric KV Store in Go: 300 Lines of Sharding, TTL, and Performance Tuning

This article walks through implementing a compact, 300‑line Go version of Olric—a distributed key‑value store—covering core data structures, shard routing, simplified RPC, TTL handling, node replication, rebalancing, concurrency safety, and performance experiments with benchmarks, profiling, and memory optimizations.

BenchmarkDistributed KVGo
0 likes · 9 min read
Build a Mini Olric KV Store in Go: 300 Lines of Sharding, TTL, and Performance Tuning
Java Companion
Java Companion
Nov 22, 2025 · Databases

Why ShardingJDBC Can Be a Trap: Rethinking Sharding‑Based Database Optimization

The article recounts a real‑world case where using ShardingJDBC, Elasticsearch and Logstash dramatically increased system complexity and caused a pagination bug, then walks through hardware, software and SQL‑level optimization techniques and suggests more balanced alternatives to blind sharding.

Database OptimizationElasticsearchLogstash
0 likes · 12 min read
Why ShardingJDBC Can Be a Trap: Rethinking Sharding‑Based Database Optimization
Java Companion
Java Companion
Nov 21, 2025 · Databases

How to Query Phone Number Suffix in Milliseconds on 50 Million Records

When faced with 50 million user rows, using LIKE '%1234' triggers a full table scan, but adding a suffix column, reversing the phone number, leveraging function indexes, or integrating Elasticsearch can reduce query time from minutes to a few milliseconds, each with its own trade‑offs.

ElasticsearchIndexingMySQL
0 likes · 12 min read
How to Query Phone Number Suffix in Milliseconds on 50 Million Records
Architect-Kip
Architect-Kip
Nov 21, 2025 · Backend Development

Mastering Pagination Performance: Single‑DB and Cross‑DB Strategies

This guide explores common pagination bottlenecks and presents practical solutions for single‑database and sharded environments, covering keyset pagination, bidirectional paging, join‑based paging, index design, middleware rewriting, approximate paging, and two‑phase query techniques.

SQLkeyset paginationmiddleware
0 likes · 14 min read
Mastering Pagination Performance: Single‑DB and Cross‑DB Strategies
Architect Chen
Architect Chen
Nov 18, 2025 · Backend Development

Designing High‑Concurrency Microservices: Splitting, Sharding, and Resilience

This guide explains how to break a monolithic system into independent microservices, apply vertical and horizontal data sharding, and implement rate limiting, circuit breaking, and degradation to sustain millions of concurrent requests while keeping the backend stable and scalable.

Circuit BreakingMicroservicesservice degradation
0 likes · 5 min read
Designing High‑Concurrency Microservices: Splitting, Sharding, and Resilience
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Nov 13, 2025 · Backend Development

Designing High‑Concurrency Microservice Architectures: Splitting, Sharding, Rate Limiting, and Circuit Breaking

This guide explains how to build a million‑request‑per‑second microservice system by properly splitting business domains, partitioning data with vertical and horizontal sharding, applying robust rate‑limiting techniques, and implementing circuit‑breaking and degradation strategies to maintain stability.

Circuit BreakingHigh ConcurrencyService Splitting
0 likes · 6 min read
Designing High‑Concurrency Microservice Architectures: Splitting, Sharding, Rate Limiting, and Circuit Breaking
Selected Java Interview Questions
Selected Java Interview Questions
Nov 6, 2025 · Databases

Master Spring Boot ShardingSphere: Step-by-Step MySQL Sharding Guide

This article walks through building a Spring Boot project with ShardingSphere‑JDBC, covering Maven dependencies, YAML configuration, database setup, sharding rules, distributed ID generation, binding tables, and testing, providing a complete, hands‑on guide to implementing MySQL horizontal partitioning in a real‑world application.

MySQLShardingSphereSpring Boot
0 likes · 18 min read
Master Spring Boot ShardingSphere: Step-by-Step MySQL Sharding Guide
Senior Brother's Insights
Senior Brother's Insights
Oct 29, 2025 · Databases

Sharding vs Partitioning: When and How to Scale Your Database

This article explains the concepts, differences, advantages, and disadvantages of database sharding and partitioning, provides practical MySQL examples for horizontal sharding by region and range partitioning, and offers guidance on choosing the right approach for scalability and maintainability.

Data DistributionDatabase ScalingMySQL
0 likes · 10 min read
Sharding vs Partitioning: When and How to Scale Your Database
Tech Freedom Circle
Tech Freedom Circle
Oct 25, 2025 · Databases

Designing a 10 WQPS Redis Counter Component: A Systematic Timer Solution

This article presents a complete, step‑by‑step analysis of a high‑concurrency Redis counter component that supports up to 100 000 QPS, covering business pain points, architectural design, two core counting strategies, sharding, local batch optimization, code walkthroughs, and performance benchmark results.

AOPCounterHigh Concurrency
0 likes · 33 min read
Designing a 10 WQPS Redis Counter Component: A Systematic Timer Solution
Tech Freedom Circle
Tech Freedom Circle
Oct 23, 2025 · Databases

Why Consistent Hashing Fails: Why Redis, HBase, TiDB and Ceph Have Dropped It

The article examines the fundamental limitations of consistent hashing—its inability to preserve data locality, support range queries, and handle topology awareness—explaining why major storage systems such as Redis Cluster, TiDB, Ceph, and HBase have adopted alternative sharding strategies like hash slots, range partitioning, and CRUSH.

CRUSHCephHBase
0 likes · 45 min read
Why Consistent Hashing Fails: Why Redis, HBase, TiDB and Ceph Have Dropped It