BestHub
Sign in
Tagged articles
602 articles
Page 1 of 7
Shuge Unlimited
Shuge Unlimited
Apr 30, 2026 · Databases

Milvus VTS Deep Dive: Two Write Modes, Sharding, and Migration Best Practices

The article provides a source‑code level analysis of Milvus Vector Transport Service (VTS), detailing its three‑stage architecture, partition‑aware sharding logic, two distinct sink write mechanisms (BufferBatchWriter and BulkWriter), schema conversion rules, error‑handling strategies, performance tuning parameters, and practical configuration examples for efficient vector data migration across various data sources.

BufferBatchWriterBulkWriterMilvus
0 likes · 20 min read
Milvus VTS Deep Dive: Two Write Modes, Sharding, and Migration Best Practices
ITPUB
ITPUB
Apr 29, 2026 · Backend Development

How to Rescue a System When QPS Jumps 100× in 10 Minutes

When a service experiences a sudden 100‑fold QPS surge, this guide walks through immediate emergency measures such as rate limiting, circuit breaking, and traffic shedding, followed by systematic analysis of traffic origins, robust architectural redesign including horizontal scaling, micro‑service decomposition, sharding, pooling, caching, and asynchronous processing, and finally stress testing to ensure resilience.

Circuit BreakingMicroservicescaching
0 likes · 13 min read
How to Rescue a System When QPS Jumps 100× in 10 Minutes
Java Companion
Java Companion
Apr 26, 2026 · Databases

Is Sharding Ready to Retire? Why the Classic Split‑Database Approach Is Becoming Legacy

The article reviews the rise and decline of traditional database sharding, explains its technical pitfalls such as wrong shard keys, cross‑database joins, and distributed transactions, compares it with emerging NewSQL solutions like TiDB, OceanBase and PolarDB‑X, and offers practical criteria for choosing the right architecture.

NewSQLOceanBasePolarDB-X
0 likes · 14 min read
Is Sharding Ready to Retire? Why the Classic Split‑Database Approach Is Becoming Legacy
Sohu Tech Products
Sohu Tech Products
Apr 22, 2026 · Databases

How to Resolve ID Conflicts After Sharding? 5 Proven Solutions

When sharding databases, independent auto‑increment IDs can collide, causing data integrity issues; this article explains why the conflict occurs and walks through five practical solutions—including UUID, custom auto‑increment steps, segment mode, Snowflake, and Meituan Leaf—detailing their trade‑offs, performance, and implementation steps.

ID generationLeafdatabase
0 likes · 15 min read
How to Resolve ID Conflicts After Sharding? 5 Proven Solutions
Architecture & Thinking
Architecture & Thinking
Apr 22, 2026 · Backend Development

Why Traditional Limit‑Offset Fails on Billion‑Row Tables and How to Fix It

This article dissects the performance collapse of classic LIMIT‑OFFSET pagination on tables with hundreds of millions of rows, explains the underlying execution steps, presents benchmark data, and walks through a progressive set of solutions—including index tuning, Seek pagination, Cursor pagination, and sharding—complete with Go code examples and practical trade‑offs.

Golangcursor paginationmysql
0 likes · 30 min read
Why Traditional Limit‑Offset Fails on Billion‑Row Tables and How to Fix It
Architect's Guide
Architect's Guide
Apr 18, 2026 · Databases

How to Import 1 Billion Records into MySQL at Lightning Speed

This article analyzes the constraints of loading a billion 1‑KB log records from HDFS/S3 into MySQL, evaluates single‑table limits, proposes batch inserts, sharding, file‑reading techniques, task coordination with Redis, Redisson, and Zookeeper, and offers practical performance‑tuning recommendations.

Bulk InsertZooKeeperdata import
0 likes · 21 min read
How to Import 1 Billion Records into MySQL at Lightning Speed
Java Companion
Java Companion
Apr 17, 2026 · Backend Development

Resolving ID Conflicts After Sharding: 5 Practical Solutions

When sharding databases, independent auto‑increment IDs can collide; this article explains why the conflict occurs and evaluates five concrete solutions—UUID, MySQL auto‑increment step, segment allocation, Snowflake algorithm, and Meituan Leaf—detailing their mechanisms, trade‑offs, performance, and suitability.

ID generationMeituan LeafSegment
0 likes · 14 min read
Resolving ID Conflicts After Sharding: 5 Practical Solutions
Lobster Programming
Lobster Programming
Apr 15, 2026 · Databases

Choosing the Right Redis Architecture: From Single Node to Cluster

This article reviews the main Redis deployment options—including single‑node, master‑slave with Sentinel, sharding via consistent hashing, and Redis Cluster—explaining their advantages, high‑availability mechanisms, scalability limits, and recommending suitable scenarios for each architecture.

ClusterDeploymenthigh availability
0 likes · 7 min read
Choosing the Right Redis Architecture: From Single Node to Cluster
IT Services Circle
IT Services Circle
Apr 11, 2026 · Databases

Why Sharding Isn’t Dead: Modern Alternatives and When to Use Them

The article revisits the rise and fall of database sharding, explains why it became problematic, and evaluates newer cloud‑native, distributed‑SQL, and serverless databases as modern replacements, offering a practical four‑step guide to help engineers choose the right solution for their workload and team.

Cloud NativeDistributed SQLOperations
0 likes · 23 min read
Why Sharding Isn’t Dead: Modern Alternatives and When to Use Them
Architect's Guide
Architect's Guide
Apr 3, 2026 · Databases

How to Diagnose and Solve MySQL Table Size Bottlenecks

This article explains how to assess MySQL table size using table capacity, disk usage, and instance limits, reveals why large tables slow down due to B+‑tree overhead, and presents three practical solutions—partitioning, sharding, and hot‑cold archiving—to restore query performance.

B+TreeHot/Cold Archivingdatabase partitioning
0 likes · 17 min read
How to Diagnose and Solve MySQL Table Size Bottlenecks
Tech Freedom Circle
Tech Freedom Circle
Mar 17, 2026 · Databases

Why HyperLogLog Misses 100M Daily Active Users and How Bitmap Solves It

The article dissects an Alibaba interview question on counting 100 million daily active users, showing why HyperLogLog’s error and lack of per‑user state make it unsuitable, and presents a detailed Bitmap‑based architecture—including sharding, pre‑computation, and ClickHouse integration—to achieve precise, high‑performance analytics.

BitmapClickHouseDailyActiveUsers
0 likes · 16 min read
Why HyperLogLog Misses 100M Daily Active Users and How Bitmap Solves It
Selected Java Interview Questions
Selected Java Interview Questions
Mar 9, 2026 · Databases

How to Paginate Efficiently Across Sharded Databases Without OOM

This article explains why pagination only becomes complex when querying across sharded databases, compares three common solutions—including global search, sequential ID‑based paging, and the discouraged secondary query method—provides practical SQL examples, outlines pros and cons, and offers a STAR‑style interview answer.

SQLpaginationsharding
0 likes · 6 min read
How to Paginate Efficiently Across Sharded Databases Without OOM
Architect's Guide
Architect's Guide
Feb 26, 2026 · Backend Development

8 Essential Software Architecture Patterns and When to Use Them

This article explains eight common software architecture patterns—from single‑database apps to microservices, caching, sharding, elastic scaling and multi‑datacenter deployment—detailing their designs, typical use cases, advantages, drawbacks, and practical implementation steps.

Design PatternsSoftware Architecturebackend scaling
0 likes · 23 min read
8 Essential Software Architecture Patterns and When to Use Them
Selected Java Interview Questions
Selected Java Interview Questions
Feb 4, 2026 · Databases

How to Handle Pagination Across Sharded Databases Without Performance Pitfalls

This article explains why pagination only becomes complex when queries span multiple database shards, compares three mainstream solutions—including middleware, open‑source frameworks, and business compromises—highlights the pitfalls of global search for deep pages, and recommends sequential paging based on the previous page’s max ID, with interview‑ready STAR guidance.

SQLinterviewpagination
0 likes · 7 min read
How to Handle Pagination Across Sharded Databases Without Performance Pitfalls
Sohu Tech Products
Sohu Tech Products
Jan 28, 2026 · Backend Development

How We Evolved a News App Comment System: From Threaded Views to AI‑Driven Ranking

This article details the evolution of a news‑app comment backend, covering early thread‑based displays, the transition to sharded databases and mixed adjacency‑path models, current hot‑comment ranking strategies, an in‑house experiment platform, topic aggregation via Kafka, and future AI‑driven architectural enhancements.

AIBackendComment System
0 likes · 16 min read
How We Evolved a News App Comment System: From Threaded Views to AI‑Driven Ranking
Architect's Guide
Architect's Guide
Jan 27, 2026 · Databases

How to Scale an Order System with Sharding, Distributed IDs, and Seata Transactions

This article details a comprehensive redesign of a high‑traffic order system, covering the challenges of massive data volume, concurrency pressure, and poor scalability, and presenting a step‑by‑step solution that includes sharding strategy selection, database product comparison, unique ID generation, middleware choice, migration plan, risk mitigation, and FAQ handling.

Backend ArchitectureSeatadatabase scaling
0 likes · 17 min read
How to Scale an Order System with Sharding, Distributed IDs, and Seata Transactions
Architect's Guide
Architect's Guide
Jan 17, 2026 · Backend Development

How We Split a 500M‑Row MySQL Table: Lessons and Strategies

Facing a 50‑million‑row financial transaction table that grew 600,000 rows each month, the team designed a sharding solution using sharding‑jdbc, tackled multi‑datasource transaction and pagination challenges, and executed a staged migration that kept the system stable while eliminating MySQL performance bottlenecks.

data-migrationpaginationsharding
0 likes · 13 min read
How We Split a 500M‑Row MySQL Table: Lessons and Strategies
Raymond Ops
Raymond Ops
Jan 9, 2026 · Databases

Master MongoDB Sharding: From Single Server to Enterprise-Scale Cluster

When a single‑node MongoDB instance can no longer handle tens of millions of records, this guide walks you through the theory, architecture, deployment steps, shard key strategies, performance tuning, monitoring, backup, and troubleshooting needed to build a robust, production‑grade sharded cluster.

BackupMongoDBOps
0 likes · 14 min read
Master MongoDB Sharding: From Single Server to Enterprise-Scale Cluster
Ray's Galactic Tech
Ray's Galactic Tech
Jan 8, 2026 · Databases

Boost SQL Performance Without Rewriting Queries: Indexes, Partitioning, Caching

This guide presents a comprehensive, step‑by‑step roadmap for accelerating slow SQL queries without altering the original statements, covering index creation, database parameter tuning, table partitioning, caching layers, read‑write splitting, sharding, statistics updates, hardware choices, and middleware routing.

Database TuningPerformance OptimizationSQL
0 likes · 8 min read
Boost SQL Performance Without Rewriting Queries: Indexes, Partitioning, Caching
java1234
java1234
Jan 3, 2026 · Backend Development

Ditch the Heavyweight XXL‑Job: An Elegant Nacos‑Based Scheduling Solution

The article analyses the friction between XXL‑Job and a Nacos‑centric stack, proposes the JobFlow design that removes redundant registration, adds full‑link TraceId, strong sharding with distributed locks, intelligent retries and cloud‑native configuration, and demonstrates how these changes simplify operations and improve observability in microservice environments.

Cloud NativeJobFlowMicroservices
0 likes · 19 min read
Ditch the Heavyweight XXL‑Job: An Elegant Nacos‑Based Scheduling Solution
Tech Freedom Circle
Tech Freedom Circle
Jan 3, 2026 · Databases

How Many Values Can MySQL IN Handle? 100K‑10M IDs and the Ultimate Solution

An interview scenario reveals that using a massive IN list (hundreds of thousands of IDs) in MySQL quickly exceeds packet limits, forces full‑table scans, and overloads CPU, while batch IN queries cause network storms; the article dissects these pitfalls and presents three progressive strategies—splitting IN, temporary tables with JOIN, and sharding with stream‑merge—to safely query massive ID sets.

IN clausemysqlquery optimization
0 likes · 32 min read
How Many Values Can MySQL IN Handle? 100K‑10M IDs and the Ultimate Solution
Java Companion
Java Companion
Jan 3, 2026 · Cloud Native

Ditch the Bulky XXL‑Job? Try This Elegant Nacos‑Based Scheduling Solution

The article analyzes the friction between XXL‑Job and Nacos in cloud‑native environments, proposes the JobFlow design that removes redundant registration and configuration, adds full‑traceability, true sharding with distributed locks, smart retries and cloud‑native configuration, and demonstrates how these changes improve consistency, observability and operational cost.

Cloud NativeJobFlowMicroservices
0 likes · 19 min read
Ditch the Bulky XXL‑Job? Try This Elegant Nacos‑Based Scheduling Solution
Architect
Architect
Dec 28, 2025 · Databases

How to Scale an Order System: Sharding, ID Generation, and Database Choices

This article walks through the challenges of a growing order system, analyzes current bottlenecks, compares sharding and distributed database products, proposes a unique ID strategy, outlines sharding key selection, presents a migration plan with ShardingSphere‑JDBC, and discusses risks and FAQs for a robust backend architecture.

Distributed TransactionsUnique IDdatabase scaling
0 likes · 18 min read
How to Scale an Order System: Sharding, ID Generation, and Database Choices
macrozheng
macrozheng
Dec 26, 2025 · Databases

NewSQL vs Middleware Sharding: Which Architecture Really Wins?

This article objectively compares middleware‑based sharding solutions with native NewSQL distributed databases, examining their architectural differences, transaction handling, high‑availability, scaling, SQL support, storage engines, and maturity to help engineers decide which approach best fits their workload.

CAP theoremNewSQLPaxos
0 likes · 20 min read
NewSQL vs Middleware Sharding: Which Architecture Really Wins?
Architect Chen
Architect Chen
Dec 23, 2025 · Databases

Boost MySQL to 10k QPS: Multi‑Layer Optimization Strategies

When MySQL QPS reaches 10,000, common bottlenecks include CPU saturation, low buffer pool hit rate, connection limits, and lock contention, which can be mitigated through internal tuning, cache interception, read‑write splitting, and horizontal sharding.

QPSReadWriteSplittingcaching
0 likes · 4 min read
Boost MySQL to 10k QPS: Multi‑Layer Optimization Strategies
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Dec 21, 2025 · Backend Development

How Elasticsearch Scales to Billions of Queries: Sharding, Inverted Index, Distributed Execution, and Replication

Elasticsearch achieves billion‑scale search performance by combining horizontal sharding, immutable inverted‑index segments, a two‑stage distributed Query/FETCH model, and multiple replicas with a coordinator node to ensure high concurrency, scalability, and availability.

Distributed QueryElasticsearchReplication
0 likes · 4 min read
How Elasticsearch Scales to Billions of Queries: Sharding, Inverted Index, Distributed Execution, and Replication
Su San Talks Tech
Su San Talks Tech
Dec 11, 2025 · Databases

How to Scale a 900 M‑Row Message Table to 60 B Rows Without Downtime

This article details a comprehensive sharding strategy for a high‑traffic message table, explaining why early partitioning is essential, how to design language‑based vertical shards combined with hash‑based horizontal shards, calculate table counts and virtual nodes, and execute a zero‑downtime migration using dual‑writes and gradual traffic rollout.

Performance Optimizationhorizontal partitioningmysql
0 likes · 16 min read
How to Scale a 900 M‑Row Message Table to 60 B Rows Without Downtime
Java Architect Handbook
Java Architect Handbook
Nov 30, 2025 · Databases

When Sharding Fails: Debugging Pagination Bugs and Optimizing MySQL for Massive Tables

The article recounts a real‑world case where a massive device table (over 500 million rows) suffered from sharding and Elasticsearch integration issues, describes a pagination bug caused by Sharding‑JDBC, and presents a comprehensive set of hardware, software, and SQL optimization techniques—including MySQL configuration examples—to improve performance.

Database OptimizationElasticsearchSharding-JDBC
0 likes · 13 min read
When Sharding Fails: Debugging Pagination Bugs and Optimizing MySQL for Massive Tables
Code Wrench
Code Wrench
Nov 27, 2025 · Databases

Build a Mini Olric KV Store in Go: 300 Lines of Sharding, TTL, and Performance Tuning

This article walks through implementing a compact, 300‑line Go version of Olric—a distributed key‑value store—covering core data structures, shard routing, simplified RPC, TTL handling, node replication, rebalancing, concurrency safety, and performance experiments with benchmarks, profiling, and memory optimizations.

BenchmarkDistributed KVGo
0 likes · 9 min read
Build a Mini Olric KV Store in Go: 300 Lines of Sharding, TTL, and Performance Tuning
Java Companion
Java Companion
Nov 22, 2025 · Databases

Why ShardingJDBC Can Be a Trap: Rethinking Sharding‑Based Database Optimization

The article recounts a real‑world case where using ShardingJDBC, Elasticsearch and Logstash dramatically increased system complexity and caused a pagination bug, then walks through hardware, software and SQL‑level optimization techniques and suggests more balanced alternatives to blind sharding.

Database OptimizationElasticsearchLogstash
0 likes · 12 min read
Why ShardingJDBC Can Be a Trap: Rethinking Sharding‑Based Database Optimization
Java Companion
Java Companion
Nov 21, 2025 · Databases

How to Query Phone Number Suffix in Milliseconds on 50 Million Records

When faced with 50 million user rows, using LIKE '%1234' triggers a full table scan, but adding a suffix column, reversing the phone number, leveraging function indexes, or integrating Elasticsearch can reduce query time from minutes to a few milliseconds, each with its own trade‑offs.

Database designElasticsearchindexing
0 likes · 12 min read
How to Query Phone Number Suffix in Milliseconds on 50 Million Records
Architect-Kip
Architect-Kip
Nov 21, 2025 · Backend Development

Mastering Pagination Performance: Single‑DB and Cross‑DB Strategies

This guide explores common pagination bottlenecks and presents practical solutions for single‑database and sharded environments, covering keyset pagination, bidirectional paging, join‑based paging, index design, middleware rewriting, approximate paging, and two‑phase query techniques.

Keyset PaginationSQLmiddleware
0 likes · 14 min read
Mastering Pagination Performance: Single‑DB and Cross‑DB Strategies
Architect Chen
Architect Chen
Nov 18, 2025 · Backend Development

Designing High‑Concurrency Microservices: Splitting, Sharding, and Resilience

This guide explains how to break a monolithic system into independent microservices, apply vertical and horizontal data sharding, and implement rate limiting, circuit breaking, and degradation to sustain millions of concurrent requests while keeping the backend stable and scalable.

Circuit BreakingMicroservicesservice degradation
0 likes · 5 min read
Designing High‑Concurrency Microservices: Splitting, Sharding, and Resilience
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Nov 13, 2025 · Backend Development

Designing High‑Concurrency Microservice Architectures: Splitting, Sharding, Rate Limiting, and Circuit Breaking

This guide explains how to build a million‑request‑per‑second microservice system by properly splitting business domains, partitioning data with vertical and horizontal sharding, applying robust rate‑limiting techniques, and implementing circuit‑breaking and degradation strategies to maintain stability.

Circuit BreakingService Splittinghigh concurrency
0 likes · 6 min read
Designing High‑Concurrency Microservice Architectures: Splitting, Sharding, Rate Limiting, and Circuit Breaking
Selected Java Interview Questions
Selected Java Interview Questions
Nov 6, 2025 · Databases

Master Spring Boot ShardingSphere: Step-by-Step MySQL Sharding Guide

This article walks through building a Spring Boot project with ShardingSphere‑JDBC, covering Maven dependencies, YAML configuration, database setup, sharding rules, distributed ID generation, binding tables, and testing, providing a complete, hands‑on guide to implementing MySQL horizontal partitioning in a real‑world application.

ShardingSphereSpring Bootdistributed-id
0 likes · 18 min read
Master Spring Boot ShardingSphere: Step-by-Step MySQL Sharding Guide
Senior Brother's Insights
Senior Brother's Insights
Oct 29, 2025 · Databases

Sharding vs Partitioning: When and How to Scale Your Database

This article explains the concepts, differences, advantages, and disadvantages of database sharding and partitioning, provides practical MySQL examples for horizontal sharding by region and range partitioning, and offers guidance on choosing the right approach for scalability and maintainability.

Partitioningdata distributiondatabase scaling
0 likes · 10 min read
Sharding vs Partitioning: When and How to Scale Your Database
Tech Freedom Circle
Tech Freedom Circle
Oct 25, 2025 · Databases

Designing a 10 WQPS Redis Counter Component: A Systematic Timer Solution

This article presents a complete, step‑by‑step analysis of a high‑concurrency Redis counter component that supports up to 100 000 QPS, covering business pain points, architectural design, two core counting strategies, sharding, local batch optimization, code walkthroughs, and performance benchmark results.

CounterSliding WindowSpring Boot
0 likes · 33 min read
Designing a 10 WQPS Redis Counter Component: A Systematic Timer Solution
Tech Freedom Circle
Tech Freedom Circle
Oct 23, 2025 · Databases

Why Consistent Hashing Fails: Why Redis, HBase, TiDB and Ceph Have Dropped It

The article examines the fundamental limitations of consistent hashing—its inability to preserve data locality, support range queries, and handle topology awareness—explaining why major storage systems such as Redis Cluster, TiDB, Ceph, and HBase have adopted alternative sharding strategies like hash slots, range partitioning, and CRUSH.

CRUSHCephHBase
0 likes · 45 min read
Why Consistent Hashing Fails: Why Redis, HBase, TiDB and Ceph Have Dropped It
Senior Brother's Insights
Senior Brother's Insights
Oct 15, 2025 · Databases

Master MySQL Partitioning: When, How, and Why It Beats Sharding

This guide explains MySQL partitioning fundamentals, shows how to create RANGE, LIST, and KEY partitions with concrete CREATE TABLE examples, compares the performance benefits and maintenance trade‑offs, and clarifies the differences between partitioning and sharding to help you decide when to use each technique.

Database OptimizationKEYList
0 likes · 7 min read
Master MySQL Partitioning: When, How, and Why It Beats Sharding
Su San Talks Tech
Su San Talks Tech
Oct 9, 2025 · Backend Development

Designing a Scalable Real‑Time Video Ranking System for Billions of Views

This article walks through the end‑to‑end system design of a TikTok‑style video leaderboard, covering functional and non‑functional requirements, traffic estimation, single‑node prototypes, fault‑tolerant replication, sharding, elastic partitioning, sliding‑window counters, and caching strategies to achieve low‑latency, high‑throughput ranking at massive scale.

Sliding WindowSystem Designcaching
0 likes · 21 min read
Designing a Scalable Real‑Time Video Ranking System for Billions of Views
Su San Talks Tech
Su San Talks Tech
Sep 28, 2025 · Databases

5 Redis Deployment Patterns Every Engineer Should Know

The article outlines five Redis deployment patterns—from a single‑instance setup to official Redis Cluster—detailing their architectures, advantages, drawbacks, and suitable scenarios, helping engineers choose the right solution for scalability, high availability, and operational complexity.

Consistent HashDeploymentRedis Cluster
0 likes · 8 min read
5 Redis Deployment Patterns Every Engineer Should Know
Raymond Ops
Raymond Ops
Sep 22, 2025 · Databases

Master‑Slave, Sentinel, and Sharding: Complete Guide to Redis Cluster Architectures

This article explains Redis’s three clustering options—master‑slave replication, Sentinel high‑availability, and sharding—detailing their architectures, setup steps, synchronization mechanisms, advantages, drawbacks, and common interview questions, helping readers choose and implement the right solution for high‑performance, scalable data storage.

ClusterReplicationhigh availability
0 likes · 18 min read
Master‑Slave, Sentinel, and Sharding: Complete Guide to Redis Cluster Architectures
Su San Talks Tech
Su San Talks Tech
Sep 22, 2025 · Backend Development

Mastering Pagination in Billion‑Row Sharded Databases: Interview‑Ready Strategies

This article dissects the challenges of implementing pagination on billion‑row sharded tables, explains common sharding strategies, compares SDK, Proxy, and Sidecar architectures, and presents practical solutions such as global query, infinite scroll, two‑phase queries, index tables, and external storage to help engineers ace interview questions.

Performance Optimizationbackend interviewdistributed databases
0 likes · 24 min read
Mastering Pagination in Billion‑Row Sharded Databases: Interview‑Ready Strategies
Code Wrench
Code Wrench
Sep 17, 2025 · Databases

Mastering GORM Sharding: A Lightweight Go Plugin for Scalable Databases

This article explains why horizontal sharding becomes essential for billion‑row tables, introduces the non‑intrusive GORM Sharding plugin, walks through installation, configuration, and code examples, and evaluates its advantages, limitations, common issues, and suitable use cases for Go back‑end services.

GORMGoperformance
0 likes · 11 min read
Mastering GORM Sharding: A Lightweight Go Plugin for Scalable Databases
Architect's Journey
Architect's Journey
Sep 10, 2025 · Databases

Why Taobao Order IDs End with the Same Six Digits: Inside the Gene Method

The article explains why the last six digits of Taobao order numbers are constant, revealing the “gene” method that embeds a hashed user identifier to bind orders to specific shards, thereby improving query performance, ensuring balanced data distribution, and supporting scalability in large e‑commerce systems.

Database designTaobaodistributed databases
0 likes · 8 min read
Why Taobao Order IDs End with the Same Six Digits: Inside the Gene Method
Java Backend Full-Stack
Java Backend Full-Stack
Sep 9, 2025 · Backend Development

Why Do the Last Six Digits of Taobao Order IDs Remain Constant?

The article explains how Taobao embeds a user‑specific “gene” in the last six digits of its order numbers, enabling efficient sharding routing, uniform data distribution, and idempotent order handling while maintaining global uniqueness in a large e‑commerce system.

Taobaodatabase routinge‑commerce
0 likes · 7 min read
Why Do the Last Six Digits of Taobao Order IDs Remain Constant?
JD Cloud Developers
JD Cloud Developers
Sep 2, 2025 · Databases

Unlocking ClickHouse’s Lightning‑Fast Queries: The ‘Nine Swords’ Architecture Explained

This article explores ClickHouse’s high‑performance OLAP design—including its MPP architecture, columnar storage, vectorized execution, pre‑sorting, sharding, replication, index strategies, and compute engine—showing how each innovation contributes to ultra‑fast, scalable data analysis in the big‑data era.

ClickHouseColumnar StorageOLAP
0 likes · 14 min read
Unlocking ClickHouse’s Lightning‑Fast Queries: The ‘Nine Swords’ Architecture Explained
Raymond Ops
Raymond Ops
Aug 22, 2025 · Operations

Mastering ELK Stack: From Installation to Advanced Sharding Strategies

This guide introduces the ELK stack fundamentals, explains Elasticsearch, Logstash, and Kibana roles, walks through environment preparation, installation, configuration, head plugin setup, shard and replica concepts, scaling recommendations, and provides scripts for monitoring cluster health, offering a comprehensive hands‑on reference for log analytics operations.

ELKElasticsearchKibana
0 likes · 16 min read
Mastering ELK Stack: From Installation to Advanced Sharding Strategies
MaGe Linux Operations
MaGe Linux Operations
Aug 12, 2025 · Databases

Master MongoDB Sharding: From Single Server to Scalable Cluster Deployment

This comprehensive guide explains MongoDB sharding fundamentals, walks through step‑by‑step deployment of config servers, shard replica sets, and mongos routers, compares range, hash, and compound shard keys, and provides performance tuning, security, backup, monitoring, and troubleshooting best practices for production‑grade clusters.

Cluster DeploymentConfig ServersMongoDB
0 likes · 13 min read
Master MongoDB Sharding: From Single Server to Scalable Cluster Deployment
Tech Freedom Circle
Tech Freedom Circle
Aug 4, 2025 · Operations

How Do Projects Achieve High Availability Without Multi‑Site Active‑Active? – A Meituan Interview Question

The article analyzes high‑availability concepts, from single‑machine risks to multi‑site active‑active architectures, compares cold and hot backup strategies, discusses network latency challenges, and presents Ele.me’s cell‑based, sharding‑driven multi‑region solution with concrete examples, tables, and code snippets.

cell-based architecturedata replicationdisaster recovery
0 likes · 28 min read
How Do Projects Achieve High Availability Without Multi‑Site Active‑Active? – A Meituan Interview Question
MaGe Linux Operations
MaGe Linux Operations
Aug 1, 2025 · Databases

Master MySQL Slow Query Fixes: From Index Tuning to Distributed Sharding

This comprehensive guide walks MySQL administrators through diagnosing slow queries, configuring the slow‑query log, leveraging EXPLAIN, optimizing indexes, rewriting joins, implementing pagination tricks, and scaling with read‑write splitting, master‑slave replication, and horizontal sharding to dramatically improve performance in high‑traffic production environments.

Index Optimizationmysqlsharding
0 likes · 21 min read
Master MySQL Slow Query Fixes: From Index Tuning to Distributed Sharding
Top Architect
Top Architect
Jul 24, 2025 · Databases

How to Efficiently Shard Billion‑Row Tables with ShardingSphere and Spring Boot

This article walks through the end‑to‑end design, configuration, and implementation of splitting massive loan and repayment tables into 50 sharded MySQL tables using ShardingSphere, Spring Boot, and a custom suffix algorithm, while covering data migration, DBA coordination, dynamic switches, and scheduled consistency checks.

Data MigrationShardingSphereSpring Boot
0 likes · 19 min read
How to Efficiently Shard Billion‑Row Tables with ShardingSphere and Spring Boot
ITPUB
ITPUB
Jul 14, 2025 · Databases

How to Compute Mutual Friends for 100M Users in Seconds with Redis

This article breaks down the massive‑scale challenge of calculating mutual friends for hundreds of millions of users, analyzes data‑size, latency, and memory constraints, and presents three Redis‑based solutions—basic Set, bitmap, and a sharded architecture—complete with code snippets, performance numbers, and practical trade‑offs.

BitmapMutual FriendsScalable Computing
0 likes · 10 min read
How to Compute Mutual Friends for 100M Users in Seconds with Redis
Tech Freedom Circle
Tech Freedom Circle
Jul 14, 2025 · Databases

How to Estimate Sharding Capacity: Calculating Required Databases and Tables for an Alibaba Interview

The article walks through why sharding is needed, outlines IO and CPU bottlenecks, presents two design principles, shows how to estimate capacity from existing data and growth trends, compares range, modulo, consistent‑hash and Snowflake sharding schemes, and details migration strategies for expanding nodes without downtime.

Data Migrationcapacity planningconsistent hashing
0 likes · 22 min read
How to Estimate Sharding Capacity: Calculating Required Databases and Tables for an Alibaba Interview
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Jul 10, 2025 · Big Data

Master ElasticSearch: Core Concepts, Architecture, and Search Process Explained

This article provides a comprehensive overview of ElasticSearch, covering its role as a distributed full‑text search engine built on Lucene, key concepts such as index, type, document, field, shard and replica, the analysis pipeline, inverted index mechanics, and the two‑phase query‑fetch search workflow.

AnalysisElasticsearchFull‑Text Search
0 likes · 7 min read
Master ElasticSearch: Core Concepts, Architecture, and Search Process Explained
Selected Java Interview Questions
Selected Java Interview Questions
Jul 8, 2025 · Databases

How to Scale 1 Billion Orders with MySQL Sharding: Strategies, Gene IDs, and Migration

This article explores the performance bottlenecks of a 700‑million‑row MySQL orders table and presents a comprehensive sharding solution—including vertical and horizontal partitioning, gene‑based shard keys, routing logic, global secondary indexes, data migration strategies, and common pitfalls—demonstrating how query latency can drop from seconds to milliseconds while supporting billions of records.

Data MigrationPartitioningdatabase scaling
0 likes · 9 min read
How to Scale 1 Billion Orders with MySQL Sharding: Strategies, Gene IDs, and Migration
Su San Talks Tech
Su San Talks Tech
Jul 3, 2025 · Databases

Mastering MySQL Sharding: Strategies for 1 Billion Orders

This article explores the pain points of a 700‑million‑row MySQL order table, presents vertical and horizontal sharding strategies, introduces gene‑based Snowflake IDs, details routing logic, migration steps, common pitfalls, and shows performance gains after applying the final architecture.

Distributed SystemsPerformance Optimizationdatabase scaling
0 likes · 9 min read
Mastering MySQL Sharding: Strategies for 1 Billion Orders
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Jul 1, 2025 · Big Data

Master ElasticSearch: Core Concepts, Architecture, and Search Workflow Explained

This article provides a comprehensive overview of ElasticSearch, covering its definition, core components such as indexes, shards and replicas, the analysis pipeline, inverted index mechanics, and the two‑stage search process that enables scalable, fault‑tolerant full‑text search in big‑data environments.

AnalyzersBig DataDistributed Search
0 likes · 7 min read
Master ElasticSearch: Core Concepts, Architecture, and Search Workflow Explained
Top Architect
Top Architect
Jun 27, 2025 · Databases

How to Implement High‑Performance Database Sharding with ShardingSphere and SpringBoot

This article walks through the complete process of designing, configuring, and coding a sharding solution for loan and repayment tables—including database basics, historical data migration, backend query refactoring, a three‑write strategy, scheduled consistency checks, and full SpringBoot/ShardingSphere implementation details.

Data MigrationMyBatisShardingSphere
0 likes · 20 min read
How to Implement High‑Performance Database Sharding with ShardingSphere and SpringBoot
vivo Internet Technology
vivo Internet Technology
Jun 18, 2025 · Backend Development

Cutting Spring Boot Startup Time by 50%: From 280 s to 159 s

This article systematically analyzes a Spring Boot project's 280‑second startup, identifies bottlenecks such as bean initialization and sharding data source loading, and applies listener‑based timing, bean‑post‑processor profiling, and asynchronous initialization to reduce launch time to 159 seconds, improving developer efficiency.

BackendJavaSpring Boot
0 likes · 13 min read
Cutting Spring Boot Startup Time by 50%: From 280 s to 159 s
Senior Tony
Senior Tony
Jun 18, 2025 · Backend Development

Mastering High‑Concurrency Account Balance Deductions with Kafka, Sharding, and Parallel Processing

This article examines the challenges of high‑concurrency write scenarios in ad‑billing systems, explains why row‑level locking becomes a bottleneck, and presents a step‑by‑step solution using Kafka for asynchronous peak shaving, parallel consumer processing, database sharding, hotspot dispersion, and batch deduction to achieve reliable real‑time balance updates.

Kafkaaccount balance deductionhigh concurrency
0 likes · 8 min read
Mastering High‑Concurrency Account Balance Deductions with Kafka, Sharding, and Parallel Processing
Baidu Tech Salon
Baidu Tech Salon
Jun 17, 2025 · Operations

How Baidu Scaled Its Vertical Search: Elastic Scheduling and Data Management Secrets

This article explains how Baidu's vertical search platform tackled massive data growth and scaling challenges by redesigning its data management system, introducing elastic scheduling, decoupling ETCD access, implementing auto‑scaling, and advancing shard expansion to improve performance, stability, and cost efficiency.

Auto ScalingData ManagementSearch Architecture
0 likes · 18 min read
How Baidu Scaled Its Vertical Search: Elastic Scheduling and Data Management Secrets
Java Captain
Java Captain
Jun 10, 2025 · Databases

Master MySQL Performance: Slow Queries, Indexes, Transactions & Replication

This article explains how to locate and analyze MySQL slow queries, understand index structures and types, apply best practices for index creation, handle large pagination, manage transaction isolation and MVCC, and implement master‑slave replication and sharding strategies for high‑performance databases.

Index Optimizationmysqlsharding
0 likes · 13 min read
Master MySQL Performance: Slow Queries, Indexes, Transactions & Replication
Lobster Programming
Lobster Programming
Jun 9, 2025 · Databases

How to Add a Column to Billion‑Row Tables Without Downtime

This article explains a metadata‑driven approach for extending massive tables—using a separate extension table, sharding, and Elasticsearch sync—to add new fields to billion‑row databases without locking the primary table or disrupting online services.

Big DataElasticsearchdatabase schema
0 likes · 6 min read
How to Add a Column to Billion‑Row Tables Without Downtime
Code Ape Tech Column
Code Ape Tech Column
May 28, 2025 · Databases

Evaluating MySQL Table Size and Optimizing Large Tables with Partitioning, Sharding, and Hot‑Cold Archiving

This article explains how to assess MySQL table size from table capacity, disk usage, and instance limits, describes why large tables degrade performance due to B+‑tree depth, and presents three practical solutions—partitioning, sharding, and hot‑cold archiving—to improve query speed and manage growth.

B+TreeHot/Cold ArchivingPerformance Optimization
0 likes · 16 min read
Evaluating MySQL Table Size and Optimizing Large Tables with Partitioning, Sharding, and Hot‑Cold Archiving
Top Architect
Top Architect
May 26, 2025 · Databases

Design and Implementation of MySQL Table Sharding with ShardingSphere and Spring Boot

This article walks through the full process of designing, configuring, and implementing a sharding solution for large loan and repayment tables using ShardingSphere, Spring Boot 3, MySQL, and MyBatis‑Plus, covering schema planning, data migration, DBA coordination, code examples, common pitfalls, and operational considerations.

BackendSpringBootcode
0 likes · 19 min read
Design and Implementation of MySQL Table Sharding with ShardingSphere and Spring Boot
Aikesheng Open Source Community
Aikesheng Open Source Community
May 22, 2025 · Databases

Root Cause Analysis of MySQL Replication Lag Caused by Large Transactions in a Sharding Scenario

This article examines a real‑world MySQL master‑slave replication lag issue caused by massive multi‑table transactions during a sharding operation, details the investigative steps using system tables, binlog parsing, and my2sql analysis, and proposes three practical mitigation strategies.

Large TransactionsPerformance OptimizationReplication Lag
0 likes · 8 min read
Root Cause Analysis of MySQL Replication Lag Caused by Large Transactions in a Sharding Scenario
Cognitive Technology Team
Cognitive Technology Team
May 15, 2025 · Databases

How to Solve MySQL Deep Pagination Performance Issues

This article analyzes why large OFFSET values cause severe MySQL performance degradation and presents multiple optimization techniques—including range queries, subqueries, delayed joins, covering indexes, sharding, caching, and search engine integration—along with their advantages, limitations, and practical recommendations.

Database Optimizationcachingindexing
0 likes · 11 min read
How to Solve MySQL Deep Pagination Performance Issues
dbaplus Community
dbaplus Community
May 13, 2025 · Databases

Master Elasticsearch Index Design: From Mapping to Sharding Best Practices

This article provides a comprehensive guide to Elasticsearch index architecture, covering fundamental concepts, index mapping, field types, alias usage, shard and replica strategies, shard planning, resource impact, and practical recommendations for optimizing performance and stability in production environments.

BackendElasticsearchMapping
0 likes · 21 min read
Master Elasticsearch Index Design: From Mapping to Sharding Best Practices
Architect
Architect
May 4, 2025 · Databases

NewSQL vs Middleware Sharding: Which Architecture Truly Wins?

This article objectively compares middleware‑based sharding with NewSQL distributed databases, examining architecture, distributed transactions, performance, high availability, scaling, SQL support, storage engines, and ecosystem maturity to help architects decide which solution fits their specific workload and operational constraints.

CAP theoremDatabase ArchitectureNewSQL
0 likes · 20 min read
NewSQL vs Middleware Sharding: Which Architecture Truly Wins?
Cognitive Technology Team
Cognitive Technology Team
May 1, 2025 · Databases

MySQL Optimization Strategies for Read‑Heavy and Write‑Heavy Scenarios

This article systematically examines MySQL optimization techniques for both read‑many/write‑few and write‑many/read‑few business scenarios, covering architecture design, caching, sharding, asynchronous processing, monitoring, and emerging NewSQL/HTAP solutions to achieve high performance, consistency, and scalability.

Database Optimizationcachingmysql
0 likes · 9 min read
MySQL Optimization Strategies for Read‑Heavy and Write‑Heavy Scenarios
Su San Talks Tech
Su San Talks Tech
Apr 29, 2025 · Databases

Why Split Databases? Master Sharding Concepts, Strategies, and Practical SQL Routing

This article explains the fundamental concepts of database sharding—including data nodes, logical and broadcast tables, sharding keys, strategies, algorithms, SQL parsing, routing, rewriting, execution, result merging, distributed primary keys, data masking, distributed transactions, data migration, and shadow databases—providing clear examples and code snippets for real‑world implementation.

Data MigrationDistributed TransactionsSQL Routing
0 likes · 21 min read
Why Split Databases? Master Sharding Concepts, Strategies, and Practical SQL Routing
Full-Stack Internet Architecture
Full-Stack Internet Architecture
Apr 25, 2025 · Databases

Essential Knowledge Points of Redis Cluster

Redis cluster relies on two ports per node, a hash‑slot based sharding scheme, a master‑slave replication model, and provides eventual consistency, making these four fundamentals essential knowledge for developers and operators to.

ClusterConsistencyMaster‑Slave
0 likes · 6 min read
Essential Knowledge Points of Redis Cluster
Architect's Guide
Architect's Guide
Apr 25, 2025 · Databases

Solutions for MySQL Auto Increment ID Exhaustion

This article discusses the problem of MySQL auto‑increment ID exhaustion and presents six practical solutions—including changing column type to BIGINT, using UUIDs, segmenting ID generation, composite keys, adjusting auto‑increment steps, and database sharding—to ensure scalability and uniqueness.

Database designID Exhaustionauto_increment
0 likes · 6 min read
Solutions for MySQL Auto Increment ID Exhaustion
Architect
Architect
Apr 23, 2025 · Databases

Mastering Elasticsearch Index Design: From Basics to Shard Planning

This article provides a comprehensive guide to Elasticsearch index concepts, covering index definitions, alias usage, mapping and field types, shard architecture, and practical recommendations for planning shards and optimizing resource consumption to ensure stable and efficient ES clusters.

ElasticsearchMappingPerformance Optimization
0 likes · 23 min read
Mastering Elasticsearch Index Design: From Basics to Shard Planning
Java Captain
Java Captain
Apr 17, 2025 · Databases

Choosing Between Sharding Middleware and NewSQL Distributed Databases: An Objective Comparison

This article objectively compares middleware‑based sharding with NewSQL distributed databases, examining their architectural differences, transaction models, high‑availability mechanisms, scaling, SQL support, storage engines, and maturity to help practitioners decide which approach best fits their workload and operational constraints.

Database ArchitectureDistributed TransactionsNewSQL
0 likes · 17 min read
Choosing Between Sharding Middleware and NewSQL Distributed Databases: An Objective Comparison
macrozheng
macrozheng
Apr 16, 2025 · Databases

Why Enterprises Shun Foreign Keys: Performance Risks and Alternatives

Although foreign keys enforce referential integrity, many enterprises avoid them because they can slow inserts, cause blocking cascade updates, trigger database update storms in high‑concurrency environments, and complicate sharding, leading developers to prefer redundant fields and handle relationships at the application layer.

Database PerformanceForeign Keymysql
0 likes · 6 min read
Why Enterprises Shun Foreign Keys: Performance Risks and Alternatives
DeWu Technology
DeWu Technology
Apr 9, 2025 · Databases

Understanding Elasticsearch Indexes, Mappings, and Shard Architecture

The article explains Elasticsearch indexes as logical namespaces for JSON documents, details how mappings define immutable field types such as text, keyword, and numeric, describes primary and replica shard roles, and offers practical guidance on alias usage, shard sizing, replica settings, and performance optimizations to ensure cluster stability.

ElasticsearchMappingindex design
0 likes · 20 min read
Understanding Elasticsearch Indexes, Mappings, and Shard Architecture
vivo Internet Technology
vivo Internet Technology
Apr 2, 2025 · Databases

Case Study of Slow SQL Governance in an Activity Middleware System

The case study details how an activity‑middleware platform tackled slow SQL problems by cleaning historic data, introducing sharding and Elasticsearch off‑loading, optimizing queries with proper indexes and simplified joins, and automating nightly deletions, ultimately cutting daily slow queries from thousands to double‑digit levels and boosting system stability.

Database PerformanceSQL Optimizationindexing
0 likes · 17 min read
Case Study of Slow SQL Governance in an Activity Middleware System
Su San Talks Tech
Su San Talks Tech
Mar 30, 2025 · Databases

How to Supercharge Large Database Tables: Proven Optimization Techniques

This article explains why massive tables become slow, identifies common bottlenecks such as disk I/O, missing indexes, deep pagination and lock contention, and provides a step‑by‑step guide covering table design, indexing, SQL tuning, sharding, caching and a real‑world case study to dramatically improve performance.

Index TuningLarge TablesSQL Performance
0 likes · 13 min read
How to Supercharge Large Database Tables: Proven Optimization Techniques
Java Architect Essentials
Java Architect Essentials
Mar 14, 2025 · Databases

Comparing NewSQL Databases with Middleware‑Based Sharding: Advantages, Trade‑offs, and Selection Guidance

This article objectively compares NewSQL distributed databases with traditional middleware‑based sharding solutions, examining their architectures, distributed transaction handling, high‑availability, scaling, storage engines, and ecosystem maturity, and provides guidance on selecting the appropriate approach based on consistency, growth, operational capacity, and performance requirements.

Database ArchitectureDistributed TransactionsNewSQL
0 likes · 19 min read
Comparing NewSQL Databases with Middleware‑Based Sharding: Advantages, Trade‑offs, and Selection Guidance
Architecture Digest
Architecture Digest
Mar 12, 2025 · Backend Development

Designing and Optimizing a High‑Concurrency Counting System for Large‑Scale Social Media

This article explains the challenges of storing and serving trillion‑level count data for a massive micro‑blogging platform and presents a step‑by‑step backend design that evolves from simple MySQL tables to sharded databases, Redis caching, message‑queue write‑sharding, and memory‑efficient custom data structures to achieve high performance, low cost, and strong consistency.

Performance Optimizationcounting systemhigh concurrency
0 likes · 12 min read
Designing and Optimizing a High‑Concurrency Counting System for Large‑Scale Social Media
Java Web Project
Java Web Project
Mar 6, 2025 · Databases

NewSQL vs Middleware Sharding: Which Architecture Truly Wins?

This article objectively compares NewSQL databases with middleware‑based sharding, dissecting their core architectures, distributed transaction handling, high‑availability designs, scaling mechanisms, SQL support, storage engines, and maturity to help engineers decide the most suitable solution for their workloads.

CAP theoremDatabase ArchitectureDistributed Transactions
0 likes · 20 min read
NewSQL vs Middleware Sharding: Which Architecture Truly Wins?