Skip to content
Tagged articles

Sharding

500 articles · Page 1 of 5
Cilantro Loves Linux
Cilantro Loves Linux
Oct 18, 2026 · Backend Development

Batch Processing Scheduling Architecture: Reliability Beyond Scheduled Triggers

This chapter distinguishes scheduling from batch execution, covering single-machine, cluster, distributed, and pipeline batch patterns, emphasizing reliability through state tracking, idempotency, sharding, checkpoints, and explicit dependencies over fixed timing. It also outlines practical platform capabilities like versioned task definitions, unique run instances, state machines, leases, business idempotency keys, resource quotas, and end-to-end reconciliation.

IdempotencyPipelineScheduling
0 likes · 9 min read
Batch Processing Scheduling Architecture: Reliability Beyond Scheduled Triggers
Cilantro Loves Linux
Cilantro Loves Linux
Oct 18, 2026 · Databases

Why Adding Indexes Fails at Scale: Relational Big Data Architecture Patterns

This chapter from 'Architecture Fundamentals' details relational big data architecture patterns: sharding strategies, query optimization avoiding unbounded scans, async import/export with streaming I/O, idempotency, and a bottleneck-first evolution checklist that prioritizes product-level constraints over premature scaling.

Shardingasync processingbig data architecture
0 likes · 9 min read
Why Adding Indexes Fails at Scale: Relational Big Data Architecture Patterns
dbaplus Community
dbaplus Community
Oct 18, 2026 · Databases

Why MongoDB Struggles vs Redis in China: The Positioning Dilemma

This article analyzes why MongoDB adoption lags behind Redis in China, citing MongoDB's ambiguous positioning between cache and database, operational complexity, weak JOIN support, sharding difficulties, lack of SQL compatibility, and a scarcity of specialized DBAs as key factors.

China Tech MarketDBA SkillsMongoDB
0 likes · 9 min read
Why MongoDB Struggles vs Redis in China: The Positioning Dilemma
Java Architect Handbook
Java Architect Handbook
Oct 9, 2026 · Databases

TiDB: The NewSQL Database That Makes Sharding Obsolete

This article explains how TiDB, a distributed NewSQL database, combines horizontal scalability with ACID transactions and MySQL compatibility, eliminating the need for manual sharding middleware while supporting both OLTP and OLAP workloads through its HTAP architecture.

Database ArchitectureDistributed DatabaseHTAP
0 likes · 27 min read
TiDB: The NewSQL Database That Makes Sharding Obsolete
Random Bulletin
Random Bulletin
Oct 1, 2026 · Backend Development

Fault Domain Design: Turning Blast Radius from 100% into a Tunable 1/N Parameter

The article presents a layered fault-domain strategy—physical anti-affinity, logical isolation (sharding, cluster groups, swimlanes, bulkheads), cell-based architecture, chaos-engineering validation, and quantitative governance metrics—to shrink the blast radius of a ten-million-QPS system from a fixed 100% to a controllable 1/N design parameter.

High AvailabilityKubernetesSLO
0 likes · 25 min read
Fault Domain Design: Turning Blast Radius from 100% into a Tunable 1/N Parameter
Cloud Architecture
Cloud Architecture
Sep 17, 2026 · Databases

Indexes Aren't Free: Production Index Governance for High-Write Order Systems

This article presents a comprehensive index governance methodology for high-write MySQL order systems, demonstrating through a real incident how a read-optimized index caused write latency, replica lag, and timeouts, and detailing a reusable process covering query-driven design, cost measurement, validation, change buffer limits, index convergence, safe deletion, read-write routing, sharding, transactional outbox, idempotent consumers, online DDL safeguards, gating, and long-term ownership.

Change BufferDescending IndexesHigh-Write Systems
0 likes · 29 min read
Indexes Aren't Free: Production Index Governance for High-Write Order Systems
Java Tech Enthusiast
Java Tech Enthusiast
Sep 14, 2026 · Backend Development

Easy-Query: A Strongly-Typed Java ORM with LINQ-Style Queries and Zero Dependencies

This article introduces Easy-Query, a Java ORM framework inspired by .NET's LINQ that enables strongly-typed, chainable query expressions for 90% of database scenarios, demonstrating queries, joins, subqueries, pagination, streaming, dynamic conditions, grouping, and encryption with generated SQL examples.

Database EncryptionEasy-QueryJava ORM
0 likes · 13 min read
Easy-Query: A Strongly-Typed Java ORM with LINQ-Style Queries and Zero Dependencies
dbaplus Community
dbaplus Community
Sep 13, 2026 · Backend Development

Solving Sharding Routing Latency by Embedding Route Keys in Order IDs

A large OTA platform eliminated sharding query latency by encoding a 4-bit routeKey into 64-bit Snowflake order IDs, enabling direct shard lookup without index table queries, reducing P99 latency from over 1 second to tens of milliseconds and cutting database load by 80% through a three-layer fallback strategy.

RoutingShardingSnowflake
0 likes · 20 min read
Solving Sharding Routing Latency by Embedding Route Keys in Order IDs
ITPUB
ITPUB
Sep 6, 2026 · Backend Development

How WeChat Resets 1 Billion Step Counts at Midnight Without Crashing

WeChat avoids server crashes during midnight step-count resets for 1 billion users by using logical time-based versioning instead of physical updates, a custom PaxosStore for atomic increments, delayed double-write buffers for clock skew, Redis ZSet sharding for rankings, and asynchronous cold-data archival during low-traffic hours.

PaxosStoreRedisSharding
0 likes · 18 min read
How WeChat Resets 1 Billion Step Counts at Midnight Without Crashing
Java Architect Handbook
Java Architect Handbook
Sep 3, 2026 · Databases

Why ElasticSearch Is Blazing Fast: Inverted Indexes, FST, and Distributed Architecture Explained

This article breaks down ElasticSearch's performance advantages across three layers—data structures (inverted index, FST, compressed posting lists), storage (immutable segments, Doc Values), and architecture (shard parallelism, near-real-time writes, multi-level caching)—with concrete examples and interview-focused explanations.

CachingElasticsearchInterview questions
0 likes · 16 min read
Why ElasticSearch Is Blazing Fast: Inverted Indexes, FST, and Distributed Architecture Explained
liandk
liandk
Sep 1, 2026 · Databases

Master MySQL Sharding with Sharding-JDBC: Sorting, Distributed IDs, and Scaling

This article explains what MySQL sharding and partitioning are, when they are necessary, outlines four splitting patterns, details four sharding key strategies, shows a production‑ready Sharding-JDBC configuration, and discusses five common distributed challenges with practical solutions.

MySQLShardingSharding-JDBC
0 likes · 10 min read
Master MySQL Sharding with Sharding-JDBC: Sorting, Distributed IDs, and Scaling
Architect's Guide
Architect's Guide
Aug 26, 2026 · Backend Development

10 Powerful Performance‑Optimization Techniques You Should Try

The article surveys ten practical performance‑optimization tactics—from classic indexing, compression, and caching to prefetching, peak‑shaving, batch processing, and advanced methods such as resource squeezing, horizontal scaling, sharding, and lock‑free designs—explaining their trade‑offs, concrete examples, and when to apply each in real‑world systems.

CachingCompressionHorizontal Scaling
0 likes · 36 min read
10 Powerful Performance‑Optimization Techniques You Should Try
samdeepthink
samdeepthink
Aug 23, 2026 · Databases

Is Splitting Data Across Machines Distributed? Understand Sharding vs Distributed Systems

The article explains that merely placing data on multiple machines constitutes sharding—a way to split data for capacity—but true distributed systems require coordinated nodes that communicate, replicate, and handle failures, illustrated with an e‑commerce warehouse analogy and guidance on choosing between sharding and distributed databases.

Database ArchitectureShardingdistributed systems
0 likes · 8 min read
Is Splitting Data Across Machines Distributed? Understand Sharding vs Distributed Systems
Ray's Galactic Tech
Ray's Galactic Tech
Aug 22, 2026 · Backend Development

High-Concurrency Wallet Design: Hot Accounts, Idempotent Deductions, and Sharding Practices

The article analyses why wallet balance deduction is more error‑prone than inventory, outlines the root causes of a real‑world 60 k TPS outage, and presents a production‑grade architecture that separates strong‑consistent user deductions from asynchronous hot‑account crediting using idempotent requests, unique DB constraints, sharding, outbox messaging, and comprehensive monitoring to guarantee financial correctness under extreme load.

IdempotencyOutboxSharding
0 likes · 32 min read
High-Concurrency Wallet Design: Hot Accounts, Idempotent Deductions, and Sharding Practices
Mike Chen Rui
Mike Chen Rui
Aug 21, 2026 · Databases

Mastering MySQL Sharding: Principles, Architecture, and Real‑World Implementation

The article explains why high‑traffic MySQL deployments hit performance limits, introduces the concepts of vertical and horizontal sharding, and provides a step‑by‑step guide—including necessity assessment, shard key selection, schema design, middleware integration, and data migration—using an e‑commerce order system as a concrete example.

Horizontal ShardingMySQLSharding
0 likes · 5 min read
Mastering MySQL Sharding: Principles, Architecture, and Real‑World Implementation
Ray's Galactic Tech
Ray's Galactic Tech
Aug 20, 2026 · Backend Development

Payment Reconciliation: Detecting Lost Orders, Over‑payments and Auto‑Repairing Them

The article explains why successful payment does not guarantee correct accounting, defines discrepancy types such as lost orders, over‑payments and shortfalls, and presents a production‑grade reconciliation architecture with T+1 bill ingestion, shard‑based diff detection, state‑machine driven error handling, automatic repair workflows, and comprehensive monitoring.

ShardingTransactional Outboxauto-repair
0 likes · 45 min read
Payment Reconciliation: Detecting Lost Orders, Over‑payments and Auto‑Repairing Them
Ray's Galactic Tech
Ray's Galactic Tech
Aug 20, 2026 · Backend Development

Why Wallet Balance Shouldn't Be Calculated On‑The‑Fly: Double‑Entry Accounting, Snapshots, and Immutable Ledger Practices

The article explains that a production‑grade wallet must store balance snapshots instead of summing transaction flows, using double‑entry bookkeeping, ACID guarantees, idempotent request handling, outbox messaging, sharding, and comprehensive reconciliation to ensure correctness and high‑throughput under heavy concurrency.

IdempotencyOutbox PatternSharding
0 likes · 43 min read
Why Wallet Balance Shouldn't Be Calculated On‑The‑Fly: Double‑Entry Accounting, Snapshots, and Immutable Ledger Practices
YiSu Grain
YiSu Grain
Aug 19, 2026 · Databases

Optimizing Slow Queries, Sharding and Cache Consistency for Appointment System

This article walks through a comprehensive case study of a provincial medical appointment platform, diagnosing slow‑query bottlenecks, proposing patient‑id + create_time composite indexes, designing read‑write separation with replication, selecting patient_id for horizontal sharding, and implementing cache‑aside strategies to ensure consistency while handling cache avalanche, penetration and thundering‑herd scenarios.

Cache ConsistencyIndexingMySQL
0 likes · 30 min read
Optimizing Slow Queries, Sharding and Cache Consistency for Appointment System
samdeepthink
samdeepthink
Aug 18, 2026 · Databases

When a Single Database Hits Its Limits: Sharding Principles and Migration Path

The article explains why sharding should be a last‑resort scaling option, describes vertical and horizontal partitioning, compares hash, range and directory sharding schemes, discusses hotspots, routing, data colocation, migration steps and tool choices, and warns about the operational complexity involved.

Data ConsistencyHorizontal PartitioningSharding
0 likes · 16 min read
When a Single Database Hits Its Limits: Sharding Principles and Migration Path
Code Farming
Code Farming
Aug 14, 2026 · Backend Development

How Weibo Handles 100k QPS During Viral Peaks: Architectural Secrets

The article breaks down how Weibo’s backend was engineered to sustain 100,000 queries per second during traffic surges by first quantifying load, then using a push‑pull hybrid feed, a four‑tier cache funnel, and user‑ID sharding with write throttling.

CachingShardingbackend design
0 likes · 8 min read
How Weibo Handles 100k QPS During Viral Peaks: Architectural Secrets
Internet Backend Architecture Stack
Internet Backend Architecture Stack
Aug 13, 2026 · Databases

Redis Cluster Production Guide: HA, Sharding, Hot Keys & Pitfall Avoidance

This article provides a comprehensive production guide for Redis Cluster, covering cluster selection, hash slot principles, data skew solutions for big and hot keys, scaling strategies, master-slave replication latency handling, failure scenarios like split-brain, and a 7-point checklist with a real e-commerce case study.

Big KeyCaffeineData Skew
0 likes · 13 min read
Redis Cluster Production Guide: HA, Sharding, Hot Keys & Pitfall Avoidance
Cloud Architecture
Cloud Architecture
Aug 12, 2026 · Databases

How Redis Cluster’s Decentralized Design Powers Billion‑Scale Traffic

When a single Redis instance can no longer hold the data volume or write load of e‑commerce workloads, the traditional master‑slave with Sentinel model reaches its limits, and Redis Cluster—by sharding data across 16,384 slots, using gossip‑based topology, and removing a central control plane—delivers horizontal scaling and fault‑tolerance for billions of requests, provided key design, hash tags, hot‑key mitigation, and client routing are applied.

CacheClusterHigh Availability
0 likes · 32 min read
How Redis Cluster’s Decentralized Design Powers Billion‑Scale Traffic

MySQL Large Table Optimization: 5 Strategies for 100M+ Row Queries

This article presents a five-step practical guide for optimizing MySQL queries on tables exceeding 100 million rows, covering index optimization, SQL rewriting, cursor-based pagination, table structure improvements including partitioning, and architectural upgrades like read-write separation and sharding, with concrete code examples and performance comparisons.

Index OptimizationLarge TablesMySQL
0 likes · 12 min read
MySQL Large Table Optimization: 5 Strategies for 100M+ Row Queries
YiSu Grain
YiSu Grain
Aug 8, 2026 · Databases

Day 51: Database Architecture – Master‑Slave Replication, Read‑Write Separation, Sharding & Consistency

The article walks through diagnosing database bottlenecks, explains MySQL replication flow, read‑write separation benefits and limits, shows when to apply vertical versus horizontal partitioning, details sharding‑key selection, cross‑shard challenges, high‑availability steps, and presents a complete e‑commerce case study.

ConsistencyDatabase ReplicationHigh Availability
0 likes · 29 min read
Day 51: Database Architecture – Master‑Slave Replication, Read‑Write Separation, Sharding & Consistency
Xike
Xike
Aug 7, 2026 · Databases

Embedding Sharding Genes in Business IDs for Direct Routing in Sharded Databases

By embedding a shard identifier (“gene”) into the low bits of a business ID generated via Redis INCR or similar sequencers, the article shows how to achieve direct table routing without broadcast queries or extra mapping tables, detailing the algorithm, implementation, integration with Snowflake and Leaf, and common pitfalls.

JavaLeafRedis
0 likes · 11 min read
Embedding Sharding Genes in Business IDs for Direct Routing in Sharded Databases
DeepNoMind
DeepNoMind
Aug 2, 2026 · Databases

Understand Partitioning vs Sharding in 5 Minutes

The article explains how partitioning splits tables within a single database and how sharding distributes data across multiple database instances, comparing their types, advantages, limitations, and trade‑offs, and provides practical examples and a decision framework for choosing the right strategy.

PartitioningShardingdatabases
0 likes · 7 min read
Understand Partitioning vs Sharding in 5 Minutes
Ray's Galactic Tech
Ray's Galactic Tech
Aug 1, 2026 · Databases

Beyond CRUD: Full‑Scale Production Guide for MySQL 8.4 LTS

This article walks through a complete production‑grade view of MySQL 8.4 LTS, explaining how a chain of traffic spikes, connection‑pool exhaustion, long transactions and replication lag can cause an avalanche, and then detailing the five core modules, seven production mechanisms, architectural evolution steps, incident post‑mortems, and concrete configuration and code examples to build a resilient MySQL service.

High AvailabilityInnoDBMySQL
0 likes · 36 min read
Beyond CRUD: Full‑Scale Production Guide for MySQL 8.4 LTS
Code Farming
Code Farming
Jul 29, 2026 · Backend Development

Billions of Logs in Seconds: How Elasticsearch Makes It Possible

The article explains how Elasticsearch achieves sub‑second search over billions of log entries by combining sharding, immutable segment writes, a three‑layer inverted index, and a two‑phase query‑then‑fetch process that distributes work across nodes.

Distributed SearchElasticsearchSharding
0 likes · 6 min read
Billions of Logs in Seconds: How Elasticsearch Makes It Possible
Java Architect Handbook
Java Architect Handbook
Jul 26, 2026 · Databases

Interview Question: What Problems Arise After Sharding? List the Key Issues

The article outlines six major challenges introduced by database sharding—cross‑database joins, pagination, distributed transactions, global ID generation, data migration, and aggregation—along with practical solutions such as binding tables, cursor pagination, Seata AT mode, Snowflake IDs, dual‑write scaling, and summary tables, providing interview‑ready answers.

PaginationSeataSharding
0 likes · 17 min read
Interview Question: What Problems Arise After Sharding? List the Key Issues
Dabaoshi
Dabaoshi
Jul 26, 2026 · Databases

When to Scale MySQL: From Single Instance to Distributed Architecture

The article explains why a single MySQL server eventually hits read, write, or availability limits, outlines a step‑by‑step evolution—from SQL tuning and caching to read‑write splitting, high‑availability setups, and finally vertical or horizontal sharding—while warning against premature distribution.

High AvailabilityMySQLSharding
0 likes · 15 min read
When to Scale MySQL: From Single Instance to Distributed Architecture
Java Architect Handbook
Java Architect Handbook
Jul 24, 2026 · Databases

What Are Database Sharding, Table Sharding, and Combined Sharding? – Xianyu Interview Guide

The article explains the differences between database sharding, table sharding, and combined sharding, outlines interview expectations, details vertical and horizontal splitting strategies, discusses when to apply each technique, and reviews practical tools such as ShardingSphere and MyCat for Java projects.

BackendInterview PreparationMySQL
0 likes · 10 min read
What Are Database Sharding, Table Sharding, and Combined Sharding? – Xianyu Interview Guide
Cloud Architecture
Cloud Architecture
Jul 20, 2026 · Databases

Designing MySQL for Millions of QPS: From Single Server to Distributed Architecture

The article walks through a real‑world order system that spikes to 300,000 QPS, explaining why the original single‑node MySQL design fails, and detailing a step‑by‑step evolution—index tuning, transaction fixes, read‑write splitting, vertical and horizontal sharding, plus data‑pipeline integration—to achieve stable low latency at massive scale.

Index OptimizationMySQLPerformance Tuning
0 likes · 20 min read
Designing MySQL for Millions of QPS: From Single Server to Distributed Architecture
Top Architect
Top Architect
Jul 20, 2026 · Databases

Why You Should Stop Using Snowflake for IDs and Try a Shorter MySQL Auto‑Increment Solution

The article examines the drawbacks of using Snowflake for generating short numeric user IDs, details a MySQL auto‑increment based approach, reveals deadlock problems with REPLACE INTO, evaluates alternative schemes, and presents a final sharding‑friendly short‑ID design that meets performance and usability requirements.

MySQLShardingSnowflake
0 likes · 14 min read
Why You Should Stop Using Snowflake for IDs and Try a Shorter MySQL Auto‑Increment Solution
samdeepthink
samdeepthink
Jul 17, 2026 · Databases

Designing Redis Leaderboards: From a Single ZSet to Billion‑Scale Rankings

The article walks through the evolution of Redis leaderboard architectures, covering why ZSets are used, handling same‑score ordering, seasonal resets, access patterns, synchronous vs asynchronous updates, big‑key and hot‑key issues, sharding with buckets, and data durability strategies.

CacheLeaderboardRedis
0 likes · 12 min read
Designing Redis Leaderboards: From a Single ZSet to Billion‑Scale Rankings
Code Farming
Code Farming
Jul 13, 2026 · Backend Development

Designing Trillion‑Scale Counters: From MySQL to a Custom Redis Engine

The article dissects a proven trillion‑level counter architecture, tracing its evolution from a simple MySQL table through hash sharding, a full Redis migration, deep Redis memory optimizations, and hot‑cold separation, while detailing the trade‑offs and performance gains at each step.

Message QueueRedis optimizationSharding
0 likes · 6 min read
Designing Trillion‑Scale Counters: From MySQL to a Custom Redis Engine
IoT Full-Stack Technology
IoT Full-Stack Technology
Jul 13, 2026 · Databases

Is Sharding Dead? A Technical Comparison with NewSQL Databases

The article objectively compares middleware‑based sharding with NewSQL distributed databases, examining distributed transactions, CAP constraints, HA, scaling, storage engines, maturity, and ecosystem, and offers a decision framework for choosing the appropriate architecture based on concrete requirements.

CAP theoremDatabase ArchitectureHigh Availability
0 likes · 18 min read
Is Sharding Dead? A Technical Comparison with NewSQL Databases
Cloud Architecture
Cloud Architecture
Jul 12, 2026 · Databases

Database Performance Optimization: 100× Speed Gains Without Changing SQL

Even without rewriting any SQL, database performance can improve up to a hundredfold by first diagnosing bottlenecks, reducing unnecessary traffic, layering read paths, optimizing indexes, tuning connection pools, and progressively evolving from a single‑node setup to read‑write separation, sharding, and distributed read models.

CachingMySQLRead-Write Separation
0 likes · 38 min read
Database Performance Optimization: 100× Speed Gains Without Changing SQL
Code Farming
Code Farming
Jul 7, 2026 · Backend Development

Four Ways to Ensure Unique IDs After Database Sharding

When a system is split into multiple databases and tables, the article explains the root cause of ID collisions and compares four mainstream ID generation strategies—UUID, database auto‑increment, Snowflake, and segment allocation—detailing their trade‑offs, pitfalls, and selection guidelines for different concurrency levels.

ShardingSnowflakeUUID
0 likes · 9 min read
Four Ways to Ensure Unique IDs After Database Sharding
Linyb Geek Road
Linyb Geek Road
Jun 30, 2026 · Backend Development

How to Design Pagination for Billion‑Row Sharded Databases in an Interview

The article systematically breaks down pagination challenges in billion‑row sharded databases, compares common sharding strategies and middleware architectures, analyzes the performance drawbacks of a naïve global‑query approach, and presents several practical alternatives—including keyset pagination, two‑stage queries, index‑table tricks, and external search or NewSQL solutions—while highlighting their trade‑offs for interview discussions.

PaginationSQLSharding
0 likes · 24 min read
How to Design Pagination for Billion‑Row Sharded Databases in an Interview
Linyb Geek Road
Linyb Geek Road
Jun 30, 2026 · Databases

Implementing Efficient Pagination Across Sharded Databases

The article analyzes why traditional LIMIT/OFFSET pagination fails when data is split across multiple databases, presents a global query approach with its trade‑offs, and proposes an optimized "no‑skip" method plus practical tips using ShardingSphere and Elasticsearch.

ElasticsearchPaginationSharding
0 likes · 7 min read
Implementing Efficient Pagination Across Sharded Databases
Shepherd Advanced Notes
Shepherd Advanced Notes
Jun 24, 2026 · Backend Development

Boosting Throughput 10×: Architecture Evolution and Tuning for High‑Concurrency Batch Processing

The article details how a sluggish batch‑processing system handling millions of records was redesigned with XXL‑JOB sharding, Redis‑based dynamic task distribution, cursor pagination, and selective transaction scopes, achieving nearly ten‑fold throughput improvement while addressing resource contention, load‑balancing, and reliable result reconciliation.

JavaMySQLPerformance Tuning
0 likes · 19 min read
Boosting Throughput 10×: Architecture Evolution and Tuning for High‑Concurrency Batch Processing
samdeepthink
samdeepthink
Jun 22, 2026 · Databases

How to Handle 30,000 Writes per Second with Oracle, Java, and Spring

The article analyzes the write bottleneck of processing 30,000 payment orders per second on an Oracle‑Java‑Spring stack, explains why sharding was used historically, why MQ‑based peak‑shaving is discouraged, and compares self‑built sharding with modern distributed databases such as PolarDB‑X, TiDB and OceanBase, while summarizing public practices from major Chinese tech firms.

MySQLOceanBaseOracle
0 likes · 13 min read
How to Handle 30,000 Writes per Second with Oracle, Java, and Spring
Code Farming
Code Farming
Jun 20, 2026 · Backend Development

How This Architecture Handles Tens‑Fold Traffic Spikes Without Crashing

The article breaks down a complete flash‑sale system into four phases and explains how Redis distributed locks, CDN static pages, Nginx rate limiting, message‑queue peak shaving, and sharding together prevent overselling, crashes, and lost orders even when traffic surges dozens of times.

Message QueueRedisSharding
0 likes · 6 min read
How This Architecture Handles Tens‑Fold Traffic Spikes Without Crashing
Architecture Digest
Architecture Digest
May 15, 2026 · Databases

Why Alibaba Bans Joins Over Three Tables – A Must‑Know Rule for SQL Engineers

Alibaba’s Java Development Manual mandates that any SQL involving more than three tables must be avoided, a rule that stems from the exponential cost of multi‑table joins in a single‑instance database, prompting engineers to rethink data modeling, adopt denormalization, wide tables, materialized views, CQRS or application‑level assembly instead of relying on complex joins.

AlibabaDatabase DesignSQL
0 likes · 12 min read
Why Alibaba Bans Joins Over Three Tables – A Must‑Know Rule for SQL Engineers
Shuge Unlimited
Shuge Unlimited
Apr 30, 2026 · Databases

Milvus VTS Deep Dive: Two Write Modes, Sharding, and Migration Best Practices

The article provides a source‑code level analysis of Milvus Vector Transport Service (VTS), detailing its three‑stage architecture, partition‑aware sharding logic, two distinct sink write mechanisms (BufferBatchWriter and BulkWriter), schema conversion rules, error‑handling strategies, performance tuning parameters, and practical configuration examples for efficient vector data migration across various data sources.

BufferBatchWriterBulkWriterMilvus
0 likes · 20 min read
Milvus VTS Deep Dive: Two Write Modes, Sharding, and Migration Best Practices
ITPUB
ITPUB
Apr 29, 2026 · Backend Development

How to Rescue a System When QPS Jumps 100× in 10 Minutes

When a service experiences a sudden 100‑fold QPS surge, this guide walks through immediate emergency measures such as rate limiting, circuit breaking, and traffic shedding, followed by systematic analysis of traffic origins, robust architectural redesign including horizontal scaling, micro‑service decomposition, sharding, pooling, caching, and asynchronous processing, and finally stress testing to ensure resilience.

CachingHorizontal ScalingSharding
0 likes · 13 min read
How to Rescue a System When QPS Jumps 100× in 10 Minutes
Cloud Architecture
Cloud Architecture
Apr 27, 2026 · Backend Development

Building an Enterprise‑Level MyBatis Persistence Layer from Zero to One

The article walks through a real production incident caused by a massive IN‑list query, then presents a complete methodology for designing, implementing, and tuning an enterprise‑grade MyBatis persistence layer—including core execution chain, caching strategies, batch processing, read/write splitting, sharding, observability, and deployment best practices.

CachingMyBatisPerformance Tuning
0 likes · 39 min read
Building an Enterprise‑Level MyBatis Persistence Layer from Zero to One
Java Companion
Java Companion
Apr 26, 2026 · Databases

Is Sharding Ready to Retire? Why the Classic Split‑Database Approach Is Becoming Legacy

The article reviews the rise and decline of traditional database sharding, explains its technical pitfalls such as wrong shard keys, cross‑database joins, and distributed transactions, compares it with emerging NewSQL solutions like TiDB, OceanBase and PolarDB‑X, and offers practical criteria for choosing the right architecture.

NewSQLOceanBasePolarDB-X
0 likes · 14 min read
Is Sharding Ready to Retire? Why the Classic Split‑Database Approach Is Becoming Legacy
Cloud Architecture
Cloud Architecture
Apr 25, 2026 · Databases

How a Phone Number Field Can Crash a 2B‑User Database – Architect’s Design Guide

A mis‑designed phone number column can silently degrade index performance, overload CPU, and cause a cascade of timeouts in a 2‑billion‑user system, but by treating the phone as a domain value object, using a normalized VARCHAR, aligning indexing rules, and applying consistent sharding and migration strategies, you can prevent the database from collapsing under high concurrency.

Database DesignMySQLSharding
0 likes · 31 min read
How a Phone Number Field Can Crash a 2B‑User Database – Architect’s Design Guide
Sohu Tech Products
Sohu Tech Products
Apr 22, 2026 · Databases

How to Resolve ID Conflicts After Sharding? 5 Proven Solutions

When sharding databases, independent auto‑increment IDs can collide, causing data integrity issues; this article explains why the conflict occurs and walks through five practical solutions—including UUID, custom auto‑increment steps, segment mode, Snowflake, and Meituan Leaf—detailing their trade‑offs, performance, and implementation steps.

LeafShardingSnowflake
0 likes · 15 min read
How to Resolve ID Conflicts After Sharding? 5 Proven Solutions
Architecture & Thinking
Architecture & Thinking
Apr 22, 2026 · Backend Development

Why Traditional Limit‑Offset Fails on Billion‑Row Tables and How to Fix It

This article dissects the performance collapse of classic LIMIT‑OFFSET pagination on tables with hundreds of millions of rows, explains the underlying execution steps, presents benchmark data, and walks through a progressive set of solutions—including index tuning, Seek pagination, Cursor pagination, and sharding—complete with Go code examples and practical trade‑offs.

Cursor PaginationMySQLPagination
0 likes · 30 min read
Why Traditional Limit‑Offset Fails on Billion‑Row Tables and How to Fix It
Architect's Guide
Architect's Guide
Apr 18, 2026 · Databases

How to Import 1 Billion Records into MySQL at Lightning Speed

This article analyzes the constraints of loading a billion 1‑KB log records from HDFS/S3 into MySQL, evaluates single‑table limits, proposes batch inserts, sharding, file‑reading techniques, task coordination with Redis, Redisson, and Zookeeper, and offers practical performance‑tuning recommendations.

MySQLPerformance TuningRedis
0 likes · 21 min read
How to Import 1 Billion Records into MySQL at Lightning Speed
Java Companion
Java Companion
Apr 17, 2026 · Backend Development

Resolving ID Conflicts After Sharding: 5 Practical Solutions

When sharding databases, independent auto‑increment IDs can collide; this article explains why the conflict occurs and evaluates five concrete solutions—UUID, MySQL auto‑increment step, segment allocation, Snowflake algorithm, and Meituan Leaf—detailing their mechanisms, trade‑offs, performance, and suitability.

Meituan LeafMySQLSharding
0 likes · 14 min read
Resolving ID Conflicts After Sharding: 5 Practical Solutions
Lobster Programming
Lobster Programming
Apr 15, 2026 · Databases

Choosing the Right Redis Architecture: From Single Node to Cluster

This article reviews the main Redis deployment options—including single‑node, master‑slave with Sentinel, sharding via consistent hashing, and Redis Cluster—explaining their advantages, high‑availability mechanisms, scalability limits, and recommending suitable scenarios for each architecture.

ClusterHigh AvailabilityRedis
0 likes · 7 min read
Choosing the Right Redis Architecture: From Single Node to Cluster
IT Services Circle
IT Services Circle
Apr 11, 2026 · Databases

Why Sharding Isn’t Dead: Modern Alternatives and When to Use Them

The article revisits the rise and fall of database sharding, explains why it became problematic, and evaluates newer cloud‑native, distributed‑SQL, and serverless databases as modern replacements, offering a practical four‑step guide to help engineers choose the right solution for their workload and team.

OperationsShardingcloud-native
0 likes · 23 min read
Why Sharding Isn’t Dead: Modern Alternatives and When to Use Them
Cloud Architecture
Cloud Architecture
Apr 9, 2026 · Databases

Redis Cluster Deep Dive: Sharding, Replication, Failover & Smart Client

This article provides a comprehensive technical analysis of Redis Cluster, covering its data sharding via 16,384 hash slots, master‑slave replication, automated failover mechanisms, and the essential role of smart clients in routing, along with practical engineering guidelines and production‑grade code examples.

ClusterFailoverRedis
0 likes · 41 min read
Redis Cluster Deep Dive: Sharding, Replication, Failover & Smart Client
Architect's Guide
Architect's Guide
Apr 3, 2026 · Databases

How to Diagnose and Solve MySQL Table Size Bottlenecks

This article explains how to assess MySQL table size using table capacity, disk usage, and instance limits, reveals why large tables slow down due to B+‑tree overhead, and presents three practical solutions—partitioning, sharding, and hot‑cold archiving—to restore query performance.

B+TreeHot Cold ArchivingMySQL
0 likes · 17 min read
How to Diagnose and Solve MySQL Table Size Bottlenecks
Tech Freedom Circle
Tech Freedom Circle
Mar 17, 2026 · Databases

Why HyperLogLog Misses 100M Daily Active Users and How Bitmap Solves It

The article dissects an Alibaba interview question on counting 100 million daily active users, showing why HyperLogLog’s error and lack of per‑user state make it unsuitable, and presents a detailed Bitmap‑based architecture—including sharding, pre‑computation, and ClickHouse integration—to achieve precise, high‑performance analytics.

BitmapClickHouseDailyActiveUsers
0 likes · 16 min read
Why HyperLogLog Misses 100M Daily Active Users and How Bitmap Solves It
Selected Java Interview Questions
Selected Java Interview Questions
Mar 9, 2026 · Databases

How to Paginate Efficiently Across Sharded Databases Without OOM

This article explains why pagination only becomes complex when querying across sharded databases, compares three common solutions—including global search, sequential ID‑based paging, and the discouraged secondary query method—provides practical SQL examples, outlines pros and cons, and offers a STAR‑style interview answer.

PaginationSQLSharding
0 likes · 6 min read
How to Paginate Efficiently Across Sharded Databases Without OOM
Senior Xiao Ying
Senior Xiao Ying
Feb 26, 2026 · Databases

Vertical vs Horizontal Sharding in MySQL: Strategy Selection and Middleware Guide

The article explains vertical and horizontal sharding concepts, compares their advantages and drawbacks, evaluates popular sharding middleware such as ShardingSphere, details distributed ID generation with Snowflake, offers cross‑database query solutions, and provides practical guidance on when and how to apply sharding in MySQL.

Cross-Database QueriesHorizontal ShardingMySQL
0 likes · 13 min read
Vertical vs Horizontal Sharding in MySQL: Strategy Selection and Middleware Guide
Architect's Guide
Architect's Guide
Feb 26, 2026 · Backend Development

8 Essential Software Architecture Patterns and When to Use Them

This article explains eight common software architecture patterns—from single‑database apps to microservices, caching, sharding, elastic scaling and multi‑datacenter deployment—detailing their designs, typical use cases, advantages, drawbacks, and practical implementation steps.

CachingElastic ScalingSharding
0 likes · 23 min read
8 Essential Software Architecture Patterns and When to Use Them
DeepNoMind
DeepNoMind
Feb 19, 2026 · Artificial Intelligence

High‑Reliability AI Agent Design: Sharding and Decentralized Retrieval

This article explains how sharding a knowledge base and using decentralized retrieval can boost the reliability, precision, and latency of AI agents, providing code examples, a LangGraph workflow, and a performance comparison that shows a 28% speed gain over a monolithic setup.

AI agentsLangChainRAG
0 likes · 12 min read
High‑Reliability AI Agent Design: Sharding and Decentralized Retrieval
IT Learning Made Simple
IT Learning Made Simple
Feb 13, 2026 · Databases

Designing Databases as Intelligent Warehouses: A Supermarket Analogy

This article explains core database design concepts—from relational vs. NoSQL choices and normalization rules to indexing strategies, ACID transactions, SQL optimization, and sharding—using supermarket‑style analogies and concrete exam‑style examples to help readers build fast, safe, and scalable data stores.

Database DesignIndexingSQL Optimization
0 likes · 38 min read
Designing Databases as Intelligent Warehouses: A Supermarket Analogy
Selected Java Interview Questions
Selected Java Interview Questions
Feb 4, 2026 · Databases

How to Handle Pagination Across Sharded Databases Without Performance Pitfalls

This article explains why pagination only becomes complex when queries span multiple database shards, compares three mainstream solutions—including middleware, open‑source frameworks, and business compromises—highlights the pitfalls of global search for deep pages, and recommends sequential paging based on the previous page’s max ID, with interview‑ready STAR guidance.

InterviewPaginationSQL
0 likes · 7 min read
How to Handle Pagination Across Sharded Databases Without Performance Pitfalls
Sohu Tech Products
Sohu Tech Products
Jan 28, 2026 · Backend Development

How We Evolved a News App Comment System: From Threaded Views to AI‑Driven Ranking

This article details the evolution of a news‑app comment backend, covering early thread‑based displays, the transition to sharded databases and mixed adjacency‑path models, current hot‑comment ranking strategies, an in‑house experiment platform, topic aggregation via Kafka, and future AI‑driven architectural enhancements.

AIBackendRanking
0 likes · 16 min read
How We Evolved a News App Comment System: From Threaded Views to AI‑Driven Ranking
Architect's Guide
Architect's Guide
Jan 27, 2026 · Databases

How to Scale an Order System with Sharding, Distributed IDs, and Seata Transactions

This article details a comprehensive redesign of a high‑traffic order system, covering the challenges of massive data volume, concurrency pressure, and poor scalability, and presenting a step‑by‑step solution that includes sharding strategy selection, database product comparison, unique ID generation, middleware choice, migration plan, risk mitigation, and FAQ handling.

SeataShardingbackend architecture
0 likes · 17 min read
How to Scale an Order System with Sharding, Distributed IDs, and Seata Transactions
Architect's Guide
Architect's Guide
Jan 17, 2026 · Backend Development

How We Split a 500M‑Row MySQL Table: Lessons and Strategies

Facing a 50‑million‑row financial transaction table that grew 600,000 rows each month, the team designed a sharding solution using sharding‑jdbc, tackled multi‑datasource transaction and pagination challenges, and executed a staged migration that kept the system stable while eliminating MySQL performance bottlenecks.

PaginationShardingdata-migration
0 likes · 13 min read
How We Split a 500M‑Row MySQL Table: Lessons and Strategies
Raymond Ops
Raymond Ops
Jan 9, 2026 · Databases

Master MongoDB Sharding: From Single Server to Enterprise-Scale Cluster

When a single‑node MongoDB instance can no longer handle tens of millions of records, this guide walks you through the theory, architecture, deployment steps, shard key strategies, performance tuning, monitoring, backup, and troubleshooting needed to build a robust, production‑grade sharded cluster.

MongoDBPerformance TuningSharding
0 likes · 14 min read
Master MongoDB Sharding: From Single Server to Enterprise-Scale Cluster
Smart Sea Tide
Smart Sea Tide
Jan 7, 2026 · Databases

Designing Distributed System Architecture with Elasticsearch

The article explains Elasticsearch's cluster architecture, covering nodes, shards, replicas, two deployment models, data‑layer design, and the trade‑offs of each approach, while also comparing alternative distributed storage architectures and their impact on reliability and performance.

Data StorageDeployment ModesElasticsearch
0 likes · 14 min read
Designing Distributed System Architecture with Elasticsearch
java1234
java1234
Jan 3, 2026 · Backend Development

Ditch the Heavyweight XXL‑Job: An Elegant Nacos‑Based Scheduling Solution

The article analyses the friction between XXL‑Job and a Nacos‑centric stack, proposes the JobFlow design that removes redundant registration, adds full‑link TraceId, strong sharding with distributed locks, intelligent retries and cloud‑native configuration, and demonstrates how these changes simplify operations and improve observability in microservice environments.

JobFlowNacosSharding
0 likes · 19 min read
Ditch the Heavyweight XXL‑Job: An Elegant Nacos‑Based Scheduling Solution
Tech Freedom Circle
Tech Freedom Circle
Jan 3, 2026 · Databases

How Many Values Can MySQL IN Handle? 100K‑10M IDs and the Ultimate Solution

An interview scenario reveals that using a massive IN list (hundreds of thousands of IDs) in MySQL quickly exceeds packet limits, forces full‑table scans, and overloads CPU, while batch IN queries cause network storms; the article dissects these pitfalls and presents three progressive strategies—splitting IN, temporary tables with JOIN, and sharding with stream‑merge—to safely query massive ID sets.

IN ClauseMySQLSharding
0 likes · 32 min read
How Many Values Can MySQL IN Handle? 100K‑10M IDs and the Ultimate Solution
Java Companion
Java Companion
Jan 3, 2026 · Cloud Native

Ditch the Bulky XXL‑Job? Try This Elegant Nacos‑Based Scheduling Solution

The article analyzes the friction between XXL‑Job and Nacos in cloud‑native environments, proposes the JobFlow design that removes redundant registration and configuration, adds full‑traceability, true sharding with distributed locks, smart retries and cloud‑native configuration, and demonstrates how these changes improve consistency, observability and operational cost.

JobFlowNacosSharding
0 likes · 19 min read
Ditch the Bulky XXL‑Job? Try This Elegant Nacos‑Based Scheduling Solution
Architect
Architect
Dec 28, 2025 · Databases

How to Scale an Order System: Sharding, ID Generation, and Database Choices

This article walks through the challenges of a growing order system, analyzes current bottlenecks, compares sharding and distributed database products, proposes a unique ID strategy, outlines sharding key selection, presents a migration plan with ShardingSphere‑JDBC, and discusses risks and FAQs for a robust backend architecture.

ShardingUnique IDdatabase scaling
0 likes · 18 min read
How to Scale an Order System: Sharding, ID Generation, and Database Choices
macrozheng
macrozheng
Dec 26, 2025 · Databases

NewSQL vs Middleware Sharding: Which Architecture Really Wins?

This article objectively compares middleware‑based sharding solutions with native NewSQL distributed databases, examining their architectural differences, transaction handling, high‑availability, scaling, SQL support, storage engines, and maturity to help engineers decide which approach best fits their workload.

CAP theoremNewSQLPaxos
0 likes · 20 min read
NewSQL vs Middleware Sharding: Which Architecture Really Wins?
Architect Chen
Architect Chen
Dec 23, 2025 · Databases

Boost MySQL to 10k QPS: Multi‑Layer Optimization Strategies

When MySQL QPS reaches 10,000, common bottlenecks include CPU saturation, low buffer pool hit rate, connection limits, and lock contention, which can be mitigated through internal tuning, cache interception, read‑write splitting, and horizontal sharding.

CachingMySQLQPS
0 likes · 4 min read
Boost MySQL to 10k QPS: Multi‑Layer Optimization Strategies
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Dec 21, 2025 · Backend Development

How Elasticsearch Scales to Billions of Queries: Sharding, Inverted Index, Distributed Execution, and Replication

Elasticsearch achieves billion‑scale search performance by combining horizontal sharding, immutable inverted‑index segments, a two‑stage distributed Query/FETCH model, and multiple replicas with a coordinator node to ensure high concurrency, scalability, and availability.

Distributed QueryElasticsearchReplication
0 likes · 4 min read
How Elasticsearch Scales to Billions of Queries: Sharding, Inverted Index, Distributed Execution, and Replication
Su San Talks Tech
Su San Talks Tech
Dec 11, 2025 · Databases

How to Scale a 900 M‑Row Message Table to 60 B Rows Without Downtime

This article details a comprehensive sharding strategy for a high‑traffic message table, explaining why early partitioning is essential, how to design language‑based vertical shards combined with hash‑based horizontal shards, calculate table counts and virtual nodes, and execute a zero‑downtime migration using dual‑writes and gradual traffic rollout.

Horizontal PartitioningMySQLSharding
0 likes · 16 min read
How to Scale a 900 M‑Row Message Table to 60 B Rows Without Downtime
Java Architect Handbook
Java Architect Handbook
Nov 30, 2025 · Databases

When Sharding Fails: Debugging Pagination Bugs and Optimizing MySQL for Massive Tables

The article recounts a real‑world case where a massive device table (over 500 million rows) suffered from sharding and Elasticsearch integration issues, describes a pagination bug caused by Sharding‑JDBC, and presents a comprehensive set of hardware, software, and SQL optimization techniques—including MySQL configuration examples—to improve performance.

ElasticsearchMySQLPagination
0 likes · 13 min read
When Sharding Fails: Debugging Pagination Bugs and Optimizing MySQL for Massive Tables
Code Wrench
Code Wrench
Nov 27, 2025 · Databases

Build a Mini Olric KV Store in Go: 300 Lines of Sharding, TTL, and Performance Tuning

This article walks through implementing a compact, 300‑line Go version of Olric—a distributed key‑value store—covering core data structures, shard routing, simplified RPC, TTL handling, node replication, rebalancing, concurrency safety, and performance experiments with benchmarks, profiling, and memory optimizations.

BenchmarkDistributed KVGo
0 likes · 9 min read
Build a Mini Olric KV Store in Go: 300 Lines of Sharding, TTL, and Performance Tuning
Java Companion
Java Companion
Nov 21, 2025 · Databases

How to Query Phone Number Suffix in Milliseconds on 50 Million Records

When faced with 50 million user rows, using LIKE '%1234' triggers a full table scan, but adding a suffix column, reversing the phone number, leveraging function indexes, or integrating Elasticsearch can reduce query time from minutes to a few milliseconds, each with its own trade‑offs.

Database DesignElasticsearchIndexing
0 likes · 12 min read
How to Query Phone Number Suffix in Milliseconds on 50 Million Records
Architect-Kip
Architect-Kip
Nov 21, 2025 · Backend Development

Mastering Pagination Performance: Single‑DB and Cross‑DB Strategies

This guide explores common pagination bottlenecks and presents practical solutions for single‑database and sharded environments, covering keyset pagination, bidirectional paging, join‑based paging, index design, middleware rewriting, approximate paging, and two‑phase query techniques.

Keyset PaginationSQLSharding
0 likes · 14 min read
Mastering Pagination Performance: Single‑DB and Cross‑DB Strategies
Architect Chen
Architect Chen
Nov 18, 2025 · Backend Development

Designing High‑Concurrency Microservices: Splitting, Sharding, and Resilience

This guide explains how to break a monolithic system into independent microservices, apply vertical and horizontal data sharding, and implement rate limiting, circuit breaking, and degradation to sustain millions of concurrent requests while keeping the backend stable and scalable.

Shardingcircuit breakingmicroservices
0 likes · 5 min read
Designing High‑Concurrency Microservices: Splitting, Sharding, and Resilience