Skip to content
Tagged articles

Sharding

500 articles · Page 2 of 5
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Nov 13, 2025 · Backend Development

Designing High‑Concurrency Microservice Architectures: Splitting, Sharding, Rate Limiting, and Circuit Breaking

This guide explains how to build a million‑request‑per‑second microservice system by properly splitting business domains, partitioning data with vertical and horizontal sharding, applying robust rate‑limiting techniques, and implementing circuit‑breaking and degradation strategies to maintain stability.

Shardingcircuit breakinghigh concurrency
0 likes · 6 min read
Designing High‑Concurrency Microservice Architectures: Splitting, Sharding, Rate Limiting, and Circuit Breaking
Selected Java Interview Questions
Selected Java Interview Questions
Nov 6, 2025 · Databases

Master Spring Boot ShardingSphere: Step-by-Step MySQL Sharding Guide

This article walks through building a Spring Boot project with ShardingSphere‑JDBC, covering Maven dependencies, YAML configuration, database setup, sharding rules, distributed ID generation, binding tables, and testing, providing a complete, hands‑on guide to implementing MySQL horizontal partitioning in a real‑world application.

MySQLShardingShardingSphere
0 likes · 18 min read
Master Spring Boot ShardingSphere: Step-by-Step MySQL Sharding Guide
Senior Brother's Insights
Senior Brother's Insights
Oct 29, 2025 · Databases

Sharding vs Partitioning: When and How to Scale Your Database

This article explains the concepts, differences, advantages, and disadvantages of database sharding and partitioning, provides practical MySQL examples for horizontal sharding by region and range partitioning, and offers guidance on choosing the right approach for scalability and maintainability.

Data DistributionHorizontal ScalingMySQL
0 likes · 10 min read
Sharding vs Partitioning: When and How to Scale Your Database
Tech Freedom Circle
Tech Freedom Circle
Oct 25, 2025 · Databases

Designing a 10 WQPS Redis Counter Component: A Systematic Timer Solution

This article presents a complete, step‑by‑step analysis of a high‑concurrency Redis counter component that supports up to 100 000 QPS, covering business pain points, architectural design, two core counting strategies, sharding, local batch optimization, code walkthroughs, and performance benchmark results.

AOPCounterRedis
0 likes · 33 min read
Designing a 10 WQPS Redis Counter Component: A Systematic Timer Solution
Tech Freedom Circle
Tech Freedom Circle
Oct 23, 2025 · Databases

Why Consistent Hashing Fails: Why Redis, HBase, TiDB and Ceph Have Dropped It

The article examines the fundamental limitations of consistent hashing—its inability to preserve data locality, support range queries, and handle topology awareness—explaining why major storage systems such as Redis Cluster, TiDB, Ceph, and HBase have adopted alternative sharding strategies like hash slots, range partitioning, and CRUSH.

CRUSHCephConsistent Hashing
0 likes · 45 min read
Why Consistent Hashing Fails: Why Redis, HBase, TiDB and Ceph Have Dropped It
Senior Brother's Insights
Senior Brother's Insights
Oct 15, 2025 · Databases

Master MySQL Partitioning: When, How, and Why It Beats Sharding

This guide explains MySQL partitioning fundamentals, shows how to create RANGE, LIST, and KEY partitions with concrete CREATE TABLE examples, compares the performance benefits and maintenance trade‑offs, and clarifies the differences between partitioning and sharding to help you decide when to use each technique.

ListMySQLPartitioning
0 likes · 7 min read
Master MySQL Partitioning: When, How, and Why It Beats Sharding
Su San Talks Tech
Su San Talks Tech
Oct 9, 2025 · Backend Development

Designing a Scalable Real‑Time Video Ranking System for Billions of Views

This article walks through the end‑to‑end system design of a TikTok‑style video leaderboard, covering functional and non‑functional requirements, traffic estimation, single‑node prototypes, fault‑tolerant replication, sharding, elastic partitioning, sliding‑window counters, and caching strategies to achieve low‑latency, high‑throughput ranking at massive scale.

CachingShardingSliding Window
0 likes · 21 min read
Designing a Scalable Real‑Time Video Ranking System for Billions of Views
Su San Talks Tech
Su San Talks Tech
Sep 28, 2025 · Databases

5 Redis Deployment Patterns Every Engineer Should Know

The article outlines five Redis deployment patterns—from a single‑instance setup to official Redis Cluster—detailing their architectures, advantages, drawbacks, and suitable scenarios, helping engineers choose the right solution for scalability, high availability, and operational complexity.

Consistent HashRedis ClusterSharding
0 likes · 8 min read
5 Redis Deployment Patterns Every Engineer Should Know
Raymond Ops
Raymond Ops
Sep 22, 2025 · Databases

Master‑Slave, Sentinel, and Sharding: Complete Guide to Redis Cluster Architectures

This article explains Redis’s three clustering options—master‑slave replication, Sentinel high‑availability, and sharding—detailing their architectures, setup steps, synchronization mechanisms, advantages, drawbacks, and common interview questions, helping readers choose and implement the right solution for high‑performance, scalable data storage.

ClusterHigh AvailabilityRedis
0 likes · 18 min read
Master‑Slave, Sentinel, and Sharding: Complete Guide to Redis Cluster Architectures
Su San Talks Tech
Su San Talks Tech
Sep 22, 2025 · Backend Development

Mastering Pagination in Billion‑Row Sharded Databases: Interview‑Ready Strategies

This article dissects the challenges of implementing pagination on billion‑row sharded tables, explains common sharding strategies, compares SDK, Proxy, and Sidecar architectures, and presents practical solutions such as global query, infinite scroll, two‑phase queries, index tables, and external storage to help engineers ace interview questions.

Shardingbackend interviewdistributed databases
0 likes · 24 min read
Mastering Pagination in Billion‑Row Sharded Databases: Interview‑Ready Strategies
Code Wrench
Code Wrench
Sep 17, 2025 · Databases

Mastering GORM Sharding: A Lightweight Go Plugin for Scalable Databases

This article explains why horizontal sharding becomes essential for billion‑row tables, introduces the non‑intrusive GORM Sharding plugin, walks through installation, configuration, and code examples, and evaluates its advantages, limitations, common issues, and suitable use cases for Go back‑end services.

GORMGoSharding
0 likes · 11 min read
Mastering GORM Sharding: A Lightweight Go Plugin for Scalable Databases
Architect's Journey
Architect's Journey
Sep 10, 2025 · Databases

Why Taobao Order IDs End with the Same Six Digits: Inside the Gene Method

The article explains why the last six digits of Taobao order numbers are constant, revealing the “gene” method that embeds a hashed user identifier to bind orders to specific shards, thereby improving query performance, ensuring balanced data distribution, and supporting scalability in large e‑commerce systems.

Database DesignRoutingSharding
0 likes · 8 min read
Why Taobao Order IDs End with the Same Six Digits: Inside the Gene Method
Java Backend Full-Stack
Java Backend Full-Stack
Sep 9, 2025 · Backend Development

Why Do the Last Six Digits of Taobao Order IDs Remain Constant?

The article explains how Taobao embeds a user‑specific “gene” in the last six digits of its order numbers, enabling efficient sharding routing, uniform data distribution, and idempotent order handling while maintaining global uniqueness in a large e‑commerce system.

ShardingTaobaodatabase routing
0 likes · 7 min read
Why Do the Last Six Digits of Taobao Order IDs Remain Constant?
JD Cloud Developers
JD Cloud Developers
Sep 2, 2025 · Databases

Unlocking ClickHouse’s Lightning‑Fast Queries: The ‘Nine Swords’ Architecture Explained

This article explores ClickHouse’s high‑performance OLAP design—including its MPP architecture, columnar storage, vectorized execution, pre‑sorting, sharding, replication, index strategies, and compute engine—showing how each innovation contributes to ultra‑fast, scalable data analysis in the big‑data era.

ClickHouseOLAPSharding
0 likes · 14 min read
Unlocking ClickHouse’s Lightning‑Fast Queries: The ‘Nine Swords’ Architecture Explained
Raymond Ops
Raymond Ops
Aug 22, 2025 · Operations

Mastering ELK Stack: From Installation to Advanced Sharding Strategies

This guide introduces the ELK stack fundamentals, explains Elasticsearch, Logstash, and Kibana roles, walks through environment preparation, installation, configuration, head plugin setup, shard and replica concepts, scaling recommendations, and provides scripts for monitoring cluster health, offering a comprehensive hands‑on reference for log analytics operations.

ElasticsearchKibanaOperations
0 likes · 16 min read
Mastering ELK Stack: From Installation to Advanced Sharding Strategies
MaGe Linux Operations
MaGe Linux Operations
Aug 12, 2025 · Databases

Master MongoDB Sharding: From Single Server to Scalable Cluster Deployment

This comprehensive guide explains MongoDB sharding fundamentals, walks through step‑by‑step deployment of config servers, shard replica sets, and mongos routers, compares range, hash, and compound shard keys, and provides performance tuning, security, backup, monitoring, and troubleshooting best practices for production‑grade clusters.

Cluster DeploymentConfig ServersMongoDB
0 likes · 13 min read
Master MongoDB Sharding: From Single Server to Scalable Cluster Deployment
Tech Freedom Circle
Tech Freedom Circle
Aug 4, 2025 · Operations

How Do Projects Achieve High Availability Without Multi‑Site Active‑Active? – A Meituan Interview Question

The article analyzes high‑availability concepts, from single‑machine risks to multi‑site active‑active architectures, compares cold and hot backup strategies, discusses network latency challenges, and presents Ele.me’s cell‑based, sharding‑driven multi‑region solution with concrete examples, tables, and code snippets.

High AvailabilityShardingcell-based architecture
0 likes · 28 min read
How Do Projects Achieve High Availability Without Multi‑Site Active‑Active? – A Meituan Interview Question
MaGe Linux Operations
MaGe Linux Operations
Aug 1, 2025 · Databases

Master MySQL Slow Query Fixes: From Index Tuning to Distributed Sharding

This comprehensive guide walks MySQL administrators through diagnosing slow queries, configuring the slow‑query log, leveraging EXPLAIN, optimizing indexes, rewriting joins, implementing pagination tricks, and scaling with read‑write splitting, master‑slave replication, and horizontal sharding to dramatically improve performance in high‑traffic production environments.

Index OptimizationMySQLSharding
0 likes · 21 min read
Master MySQL Slow Query Fixes: From Index Tuning to Distributed Sharding
Top Architect
Top Architect
Jul 24, 2025 · Databases

How to Efficiently Shard Billion‑Row Tables with ShardingSphere and Spring Boot

This article walks through the end‑to‑end design, configuration, and implementation of splitting massive loan and repayment tables into 50 sharded MySQL tables using ShardingSphere, Spring Boot, and a custom suffix algorithm, while covering data migration, DBA coordination, dynamic switches, and scheduled consistency checks.

MySQLShardingShardingSphere
0 likes · 19 min read
How to Efficiently Shard Billion‑Row Tables with ShardingSphere and Spring Boot
ITPUB
ITPUB
Jul 14, 2025 · Databases

How to Compute Mutual Friends for 100M Users in Seconds with Redis

This article breaks down the massive‑scale challenge of calculating mutual friends for hundreds of millions of users, analyzes data‑size, latency, and memory constraints, and presents three Redis‑based solutions—basic Set, bitmap, and a sharded architecture—complete with code snippets, performance numbers, and practical trade‑offs.

BitmapMutual FriendsRedis
0 likes · 10 min read
How to Compute Mutual Friends for 100M Users in Seconds with Redis
Tech Freedom Circle
Tech Freedom Circle
Jul 14, 2025 · Databases

How to Estimate Sharding Capacity: Calculating Required Databases and Tables for an Alibaba Interview

The article walks through why sharding is needed, outlines IO and CPU bottlenecks, presents two design principles, shows how to estimate capacity from existing data and growth trends, compares range, modulo, consistent‑hash and Snowflake sharding schemes, and details migration strategies for expanding nodes without downtime.

Consistent HashingShardingcapacity planning
0 likes · 22 min read
How to Estimate Sharding Capacity: Calculating Required Databases and Tables for an Alibaba Interview
Selected Java Interview Questions
Selected Java Interview Questions
Jul 8, 2025 · Databases

How to Scale 1 Billion Orders with MySQL Sharding: Strategies, Gene IDs, and Migration

This article explores the performance bottlenecks of a 700‑million‑row MySQL orders table and presents a comprehensive sharding solution—including vertical and horizontal partitioning, gene‑based shard keys, routing logic, global secondary indexes, data migration strategies, and common pitfalls—demonstrating how query latency can drop from seconds to milliseconds while supporting billions of records.

MySQLPartitioningSharding
0 likes · 9 min read
How to Scale 1 Billion Orders with MySQL Sharding: Strategies, Gene IDs, and Migration
Su San Talks Tech
Su San Talks Tech
Jul 3, 2025 · Databases

Mastering MySQL Sharding: Strategies for 1 Billion Orders

This article explores the pain points of a 700‑million‑row MySQL order table, presents vertical and horizontal sharding strategies, introduces gene‑based Snowflake IDs, details routing logic, migration steps, common pitfalls, and shows performance gains after applying the final architecture.

MySQLShardingdatabase scaling
0 likes · 9 min read
Mastering MySQL Sharding: Strategies for 1 Billion Orders
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Jul 1, 2025 · Big Data

Master ElasticSearch: Core Concepts, Architecture, and Search Workflow Explained

This article provides a comprehensive overview of ElasticSearch, covering its definition, core components such as indexes, shards and replicas, the analysis pipeline, inverted index mechanics, and the two‑stage search process that enables scalable, fault‑tolerant full‑text search in big‑data environments.

AnalyzersDistributed SearchSharding
0 likes · 7 min read
Master ElasticSearch: Core Concepts, Architecture, and Search Workflow Explained
Top Architect
Top Architect
Jun 27, 2025 · Databases

How to Implement High‑Performance Database Sharding with ShardingSphere and SpringBoot

This article walks through the complete process of designing, configuring, and coding a sharding solution for loan and repayment tables—including database basics, historical data migration, backend query refactoring, a three‑write strategy, scheduled consistency checks, and full SpringBoot/ShardingSphere implementation details.

MyBatisShardingShardingSphere
0 likes · 20 min read
How to Implement High‑Performance Database Sharding with ShardingSphere and SpringBoot
vivo Internet Technology
vivo Internet Technology
Jun 18, 2025 · Backend Development

Cutting Spring Boot Startup Time by 50%: From 280 s to 159 s

This article systematically analyzes a Spring Boot project's 280‑second startup, identifies bottlenecks such as bean initialization and sharding data source loading, and applies listener‑based timing, bean‑post‑processor profiling, and asynchronous initialization to reduce launch time to 159 seconds, improving developer efficiency.

BackendJavaSharding
0 likes · 13 min read
Cutting Spring Boot Startup Time by 50%: From 280 s to 159 s
Senior Tony
Senior Tony
Jun 18, 2025 · Backend Development

Mastering High‑Concurrency Account Balance Deductions with Kafka, Sharding, and Parallel Processing

This article examines the challenges of high‑concurrency write scenarios in ad‑billing systems, explains why row‑level locking becomes a bottleneck, and presents a step‑by‑step solution using Kafka for asynchronous peak shaving, parallel consumer processing, database sharding, hotspot dispersion, and batch deduction to achieve reliable real‑time balance updates.

KafkaParallel ProcessingSharding
0 likes · 8 min read
Mastering High‑Concurrency Account Balance Deductions with Kafka, Sharding, and Parallel Processing
Baidu Tech Salon
Baidu Tech Salon
Jun 17, 2025 · Operations

How Baidu Scaled Its Vertical Search: Elastic Scheduling and Data Management Secrets

This article explains how Baidu's vertical search platform tackled massive data growth and scaling challenges by redesigning its data management system, introducing elastic scheduling, decoupling ETCD access, implementing auto‑scaling, and advancing shard expansion to improve performance, stability, and cost efficiency.

Auto ScalingData ManagementSearch Architecture
0 likes · 18 min read
How Baidu Scaled Its Vertical Search: Elastic Scheduling and Data Management Secrets
Java Captain
Java Captain
Jun 10, 2025 · Databases

Master MySQL Performance: Slow Queries, Indexes, Transactions & Replication

This article explains how to locate and analyze MySQL slow queries, understand index structures and types, apply best practices for index creation, handle large pagination, manage transaction isolation and MVCC, and implement master‑slave replication and sharding strategies for high‑performance databases.

Index OptimizationMySQLSharding
0 likes · 13 min read
Master MySQL Performance: Slow Queries, Indexes, Transactions & Replication
Lobster Programming
Lobster Programming
Jun 9, 2025 · Databases

How to Add a Column to Billion‑Row Tables Without Downtime

This article explains a metadata‑driven approach for extending massive tables—using a separate extension table, sharding, and Elasticsearch sync—to add new fields to billion‑row databases without locking the primary table or disrupting online services.

ElasticsearchShardingbig data
0 likes · 6 min read
How to Add a Column to Billion‑Row Tables Without Downtime
Code Ape Tech Column
Code Ape Tech Column
May 28, 2025 · Databases

Evaluating MySQL Table Size and Optimizing Large Tables with Partitioning, Sharding, and Hot‑Cold Archiving

This article explains how to assess MySQL table size from table capacity, disk usage, and instance limits, describes why large tables degrade performance due to B+‑tree depth, and presents three practical solutions—partitioning, sharding, and hot‑cold archiving—to improve query speed and manage growth.

B+TreeHot Cold ArchivingMySQL
0 likes · 16 min read
Evaluating MySQL Table Size and Optimizing Large Tables with Partitioning, Sharding, and Hot‑Cold Archiving
Top Architect
Top Architect
May 26, 2025 · Databases

Design and Implementation of MySQL Table Sharding with ShardingSphere and Spring Boot

This article walks through the full process of designing, configuring, and implementing a sharding solution for large loan and repayment tables using ShardingSphere, Spring Boot 3, MySQL, and MyBatis‑Plus, covering schema planning, data migration, DBA coordination, code examples, common pitfalls, and operational considerations.

BackendShardingSpringBoot
0 likes · 19 min read
Design and Implementation of MySQL Table Sharding with ShardingSphere and Spring Boot
Aikesheng Open Source Community
Aikesheng Open Source Community
May 22, 2025 · Databases

Root Cause Analysis of MySQL Replication Lag Caused by Large Transactions in a Sharding Scenario

This article examines a real‑world MySQL master‑slave replication lag issue caused by massive multi‑table transactions during a sharding operation, details the investigative steps using system tables, binlog parsing, and my2sql analysis, and proposes three practical mitigation strategies.

Large TransactionsMySQLReplication Lag
0 likes · 8 min read
Root Cause Analysis of MySQL Replication Lag Caused by Large Transactions in a Sharding Scenario
Cognitive Technology Team
Cognitive Technology Team
May 15, 2025 · Databases

How to Solve MySQL Deep Pagination Performance Issues

This article analyzes why large OFFSET values cause severe MySQL performance degradation and presents multiple optimization techniques—including range queries, subqueries, delayed joins, covering indexes, sharding, caching, and search engine integration—along with their advantages, limitations, and practical recommendations.

CachingIndexingMySQL
0 likes · 11 min read
How to Solve MySQL Deep Pagination Performance Issues
dbaplus Community
dbaplus Community
May 13, 2025 · Databases

Master Elasticsearch Index Design: From Mapping to Sharding Best Practices

This article provides a comprehensive guide to Elasticsearch index architecture, covering fundamental concepts, index mapping, field types, alias usage, shard and replica strategies, shard planning, resource impact, and practical recommendations for optimizing performance and stability in production environments.

BackendElasticsearchMapping
0 likes · 21 min read
Master Elasticsearch Index Design: From Mapping to Sharding Best Practices
Architect
Architect
May 4, 2025 · Databases

NewSQL vs Middleware Sharding: Which Architecture Truly Wins?

This article objectively compares middleware‑based sharding with NewSQL distributed databases, examining architecture, distributed transactions, performance, high availability, scaling, SQL support, storage engines, and ecosystem maturity to help architects decide which solution fits their specific workload and operational constraints.

CAP theoremDatabase ArchitectureNewSQL
0 likes · 20 min read
NewSQL vs Middleware Sharding: Which Architecture Truly Wins?
Cognitive Technology Team
Cognitive Technology Team
May 1, 2025 · Databases

MySQL Optimization Strategies for Read‑Heavy and Write‑Heavy Scenarios

This article systematically examines MySQL optimization techniques for both read‑many/write‑few and write‑many/read‑few business scenarios, covering architecture design, caching, sharding, asynchronous processing, monitoring, and emerging NewSQL/HTAP solutions to achieve high performance, consistency, and scalability.

CachingMySQLPerformance Tuning
0 likes · 9 min read
MySQL Optimization Strategies for Read‑Heavy and Write‑Heavy Scenarios
Su San Talks Tech
Su San Talks Tech
Apr 29, 2025 · Databases

Why Split Databases? Master Sharding Concepts, Strategies, and Practical SQL Routing

This article explains the fundamental concepts of database sharding—including data nodes, logical and broadcast tables, sharding keys, strategies, algorithms, SQL parsing, routing, rewriting, execution, result merging, distributed primary keys, data masking, distributed transactions, data migration, and shadow databases—providing clear examples and code snippets for real‑world implementation.

SQL routingShardingbroadcast tables
0 likes · 21 min read
Why Split Databases? Master Sharding Concepts, Strategies, and Practical SQL Routing
Full-Stack Internet Architecture
Full-Stack Internet Architecture
Apr 25, 2025 · Databases

Essential Knowledge Points of Redis Cluster

Redis cluster relies on two ports per node, a hash‑slot based sharding scheme, a master‑slave replication model, and provides eventual consistency, making these four fundamentals essential knowledge for developers and operators to.

ClusterConsistencyMaster‑Slave
0 likes · 6 min read
Essential Knowledge Points of Redis Cluster
Architect's Guide
Architect's Guide
Apr 25, 2025 · Databases

Solutions for MySQL Auto Increment ID Exhaustion

This article discusses the problem of MySQL auto‑increment ID exhaustion and presents six practical solutions—including changing column type to BIGINT, using UUIDs, segmenting ID generation, composite keys, adjusting auto‑increment steps, and database sharding—to ensure scalability and uniqueness.

Database DesignID ExhaustionMySQL
0 likes · 6 min read
Solutions for MySQL Auto Increment ID Exhaustion
Architect
Architect
Apr 23, 2025 · Databases

Mastering Elasticsearch Index Design: From Basics to Shard Planning

This article provides a comprehensive guide to Elasticsearch index concepts, covering index definitions, alias usage, mapping and field types, shard architecture, and practical recommendations for planning shards and optimizing resource consumption to ensure stable and efficient ES clusters.

ElasticsearchMappingSharding
0 likes · 23 min read
Mastering Elasticsearch Index Design: From Basics to Shard Planning
Java Captain
Java Captain
Apr 17, 2025 · Databases

Choosing Between Sharding Middleware and NewSQL Distributed Databases: An Objective Comparison

This article objectively compares middleware‑based sharding with NewSQL distributed databases, examining their architectural differences, transaction models, high‑availability mechanisms, scaling, SQL support, storage engines, and maturity to help practitioners decide which approach best fits their workload and operational constraints.

Database ArchitectureHigh AvailabilityNewSQL
0 likes · 17 min read
Choosing Between Sharding Middleware and NewSQL Distributed Databases: An Objective Comparison
macrozheng
macrozheng
Apr 16, 2025 · Databases

Why Enterprises Shun Foreign Keys: Performance Risks and Alternatives

Although foreign keys enforce referential integrity, many enterprises avoid them because they can slow inserts, cause blocking cascade updates, trigger database update storms in high‑concurrency environments, and complicate sharding, leading developers to prefer redundant fields and handle relationships at the application layer.

Database PerformanceForeign KeyMySQL
0 likes · 6 min read
Why Enterprises Shun Foreign Keys: Performance Risks and Alternatives
DeWu Technology
DeWu Technology
Apr 9, 2025 · Databases

Understanding Elasticsearch Indexes, Mappings, and Shard Architecture

The article explains Elasticsearch indexes as logical namespaces for JSON documents, details how mappings define immutable field types such as text, keyword, and numeric, describes primary and replica shard roles, and offers practical guidance on alias usage, shard sizing, replica settings, and performance optimizations to ensure cluster stability.

ElasticsearchMappingSharding
0 likes · 20 min read
Understanding Elasticsearch Indexes, Mappings, and Shard Architecture
vivo Internet Technology
vivo Internet Technology
Apr 2, 2025 · Databases

Case Study of Slow SQL Governance in an Activity Middleware System

The case study details how an activity‑middleware platform tackled slow SQL problems by cleaning historic data, introducing sharding and Elasticsearch off‑loading, optimizing queries with proper indexes and simplified joins, and automating nightly deletions, ultimately cutting daily slow queries from thousands to double‑digit levels and boosting system stability.

Database PerformanceIndexingMySQL
0 likes · 17 min read
Case Study of Slow SQL Governance in an Activity Middleware System
Su San Talks Tech
Su San Talks Tech
Mar 30, 2025 · Databases

How to Supercharge Large Database Tables: Proven Optimization Techniques

This article explains why massive tables become slow, identifies common bottlenecks such as disk I/O, missing indexes, deep pagination and lock contention, and provides a step‑by‑step guide covering table design, indexing, SQL tuning, sharding, caching and a real‑world case study to dramatically improve performance.

CachingLarge TablesSQL Performance
0 likes · 13 min read
How to Supercharge Large Database Tables: Proven Optimization Techniques
Java Architect Essentials
Java Architect Essentials
Mar 14, 2025 · Databases

Comparing NewSQL Databases with Middleware‑Based Sharding: Advantages, Trade‑offs, and Selection Guidance

This article objectively compares NewSQL distributed databases with traditional middleware‑based sharding solutions, examining their architectures, distributed transaction handling, high‑availability, scaling, storage engines, and ecosystem maturity, and provides guidance on selecting the appropriate approach based on consistency, growth, operational capacity, and performance requirements.

Database ArchitectureHigh AvailabilityNewSQL
0 likes · 19 min read
Comparing NewSQL Databases with Middleware‑Based Sharding: Advantages, Trade‑offs, and Selection Guidance
Architecture Digest
Architecture Digest
Mar 12, 2025 · Backend Development

Designing and Optimizing a High‑Concurrency Counting System for Large‑Scale Social Media

This article explains the challenges of storing and serving trillion‑level count data for a massive micro‑blogging platform and presents a step‑by‑step backend design that evolves from simple MySQL tables to sharded databases, Redis caching, message‑queue write‑sharding, and memory‑efficient custom data structures to achieve high performance, low cost, and strong consistency.

MySQLShardingcounting system
0 likes · 12 min read
Designing and Optimizing a High‑Concurrency Counting System for Large‑Scale Social Media
Java Web Project
Java Web Project
Mar 6, 2025 · Databases

NewSQL vs Middleware Sharding: Which Architecture Truly Wins?

This article objectively compares NewSQL databases with middleware‑based sharding, dissecting their core architectures, distributed transaction handling, high‑availability designs, scaling mechanisms, SQL support, storage engines, and maturity to help engineers decide the most suitable solution for their workloads.

CAP theoremDatabase ArchitectureHigh Availability
0 likes · 20 min read
NewSQL vs Middleware Sharding: Which Architecture Truly Wins?
macrozheng
macrozheng
Mar 1, 2025 · Databases

NewSQL vs Middleware Sharding: Which Database Architecture Truly Wins?

An in‑depth comparison of NewSQL databases and middleware‑based sharding reveals each approach’s architectural strengths, distributed transaction handling, scalability, HA mechanisms, storage engine design, and ecosystem maturity, guiding readers on when to adopt NewSQL versus traditional sharding solutions.

NewSQLShardingdistributed databases
0 likes · 18 min read
NewSQL vs Middleware Sharding: Which Database Architecture Truly Wins?
JD Tech
JD Tech
Jan 9, 2025 · Databases

Challenges and Practices of Distributed Data Systems: Master‑Slave Replication, Partitioning, and High‑Availability Strategies

This article examines the core challenges of distributed data systems—including consistency, availability, and partition tolerance—then details master‑slave replication mechanisms for MySQL and Redis, various replication modes and binlog formats, and explores data partitioning, sharding, and hot‑key mitigation techniques for scalable, high‑availability deployments.

High AvailabilityMySQLReplication
0 likes · 23 min read
Challenges and Practices of Distributed Data Systems: Master‑Slave Replication, Partitioning, and High‑Availability Strategies
Su San Talks Tech
Su San Talks Tech
Jan 7, 2025 · Databases

7 Common Pitfalls of Database Sharding and How to Solve Them

This article examines seven typical challenges that arise after implementing database sharding—such as global ID conflicts, cross‑shard queries, distributed transactions, shard‑key design, data migration, pagination, and operational overhead—and provides practical solutions and code examples to address each issue.

PaginationShardingdatabase scaling
0 likes · 12 min read
7 Common Pitfalls of Database Sharding and How to Solve Them
Architect
Architect
Jan 3, 2025 · Operations

Designing Multi‑Active Distributed Systems: Overcoming Write Latency and Data Replication Challenges

This article analyzes the architectural impact of cross‑city multi‑active deployments, focusing on data‑layer design, write latency, sharding strategies, replication topologies, and routing considerations to achieve high availability, performance, and scalability in large‑scale distributed systems.

Multi-Active ArchitectureShardingdata replication
0 likes · 22 min read
Designing Multi‑Active Distributed Systems: Overcoming Write Latency and Data Replication Challenges
ITPUB
ITPUB
Jan 3, 2025 · Databases

Why ClickHouse Sharded Table Queries Return Inconsistent Row Counts—and How to Fix It

A ClickHouse cluster showed wildly varying row counts when querying sharded tables, while local tables behaved correctly; the article analyses the root cause in the cluster and table configuration, explains why the inconsistency occurs, and provides a step‑by‑step fix by switching to replicated tables.

ClickHouseQuery InconsistencyReplication
0 likes · 7 min read
Why ClickHouse Sharded Table Queries Return Inconsistent Row Counts—and How to Fix It
dbaplus Community
dbaplus Community
Dec 29, 2024 · Databases

How to Speed Up Massive MySQL Tables: Partitioning, Sharding, and Hot/Cold Archiving

When a MySQL table grows to tens of millions of rows, insert and query latency spikes, schema changes become painful, and only a subset of data is needed; this guide explains how to assess table size, why large tables slow down, and three practical solutions—partitioning, sharding, and hot/cold archiving—along with their trade‑offs and implementation steps.

Hot Cold ArchivingMySQLSharding
0 likes · 17 min read
How to Speed Up Massive MySQL Tables: Partitioning, Sharding, and Hot/Cold Archiving
Lobster Programming
Lobster Programming
Dec 26, 2024 · Databases

How to Migrate Tens of Millions of Rows: Strategies and Practical Steps

This article explains common data‑migration strategies such as hash‑modulo, time‑range and ID‑range sharding, and compares practical migration solutions including downtime migration, dual‑write, and an MQ + Redis approach, with detailed workflow diagrams and implementation tips.

Message QueueShardingdata migration
0 likes · 8 min read
How to Migrate Tens of Millions of Rows: Strategies and Practical Steps
Top Architect
Top Architect
Dec 20, 2024 · Databases

Database Optimization, Sharding, and Pagination Issues: Lessons from a Large‑Scale MySQL Deployment

The article shares a senior architect’s experience with large‑scale MySQL tables, discussing why sharding and Elasticsearch were introduced, the pagination bug caused by Sharding‑JDBC, and a comprehensive set of hardware, software, and SQL tuning recommendations, including a full MySQL configuration example.

ElasticsearchMySQLPagination
0 likes · 14 min read
Database Optimization, Sharding, and Pagination Issues: Lessons from a Large‑Scale MySQL Deployment
Radish, Keep Going!
Radish, Keep Going!
Dec 6, 2024 · Backend Development

How BigCache Achieves Ultra‑Fast In‑Memory Caching in Go

This article explains how the Go library BigCache uses sharding, efficient hash functions, and memory‑optimised data structures to deliver high‑concurrency, low‑latency caching, while avoiding GC bottlenecks and providing practical code examples and performance benchmarks.

CachingGoMemory Optimization
0 likes · 12 min read
How BigCache Achieves Ultra‑Fast In‑Memory Caching in Go
Top Architect
Top Architect
Nov 12, 2024 · Databases

Design and Implementation of Table Sharding for Loan Repayment Applications Using ShardingSphere and Spring Boot

This article describes how a senior architect designed, configured, and implemented a 50‑table sharding solution for loan and repayment request data using ShardingSphere, Spring Boot 3, MySQL, and custom synchronization scripts, while also addressing historical data migration, backend query changes, and operational safeguards.

MySQLShardingShardingSphere
0 likes · 23 min read
Design and Implementation of Table Sharding for Loan Repayment Applications Using ShardingSphere and Spring Boot
dbaplus Community
dbaplus Community
Nov 6, 2024 · Databases

Why Sharding Fails: Hidden Pitfalls of Database Partitioning in E‑commerce

This article examines the fundamental drawbacks of MySQL sharding in large‑scale e‑commerce, highlighting distributed transaction challenges, limited cross‑shard queries, global key constraints, scaling difficulties, operational overhead, and why distributed databases may be a better long‑term solution.

Shardingdatabase partitioningdistributed transactions
0 likes · 11 min read
Why Sharding Fails: Hidden Pitfalls of Database Partitioning in E‑commerce
Selected Java Interview Questions
Selected Java Interview Questions
Oct 26, 2024 · Databases

Handling MySQL Auto Increment ID Exhaustion: Strategies and Solutions

This article explains why MySQL auto‑increment IDs can run out in large or special‑case tables and presents six practical solutions—including changing the column type to BIGINT, using UUIDs, segmenting IDs, composite keys, adjusting auto‑increment steps, and sharding databases—to ensure scalable and unique primary keys.

ID ExhaustionMySQLSharding
0 likes · 6 min read
Handling MySQL Auto Increment ID Exhaustion: Strategies and Solutions
Tencent Cloud Developer
Tencent Cloud Developer
Oct 15, 2024 · Industry Insights

Why Write Latency Drives Multi‑Active Distributed Architecture Design

This article analyzes how write latency, write volume, isolation, and data replication strategies influence the design of multi‑active distributed systems, offering practical guidance on sharding, synchronous and asynchronous replication, routing, and architecture selection for high availability and performance across regions.

High AvailabilityMulti-ActiveSharding
0 likes · 23 min read
Why Write Latency Drives Multi‑Active Distributed Architecture Design
Programmer XiaoFu
Programmer XiaoFu
Oct 10, 2024 · Databases

Three Strategies Big Companies Use to Manage Massive MySQL Tables

The article walks through how to assess MySQL table size, explains why large tables slow down due to B+‑tree index overhead, and compares three practical solutions—table partitioning, database sharding (horizontal, vertical, hash‑range), and hot‑cold archiving—while outlining their trade‑offs and selection criteria.

Hot Cold ArchivingMySQLSharding
0 likes · 16 min read
Three Strategies Big Companies Use to Manage Massive MySQL Tables
21CTO
21CTO
Oct 3, 2024 · Databases

MongoDB 8.0 Unveiled: Massive Performance Gains and New Vector Support

MongoDB 8.0 launches with up to 36% higher read throughput, 56% faster batch writes, 200% faster time‑series processing, 50‑fold faster sharding distribution, quantized vector search, and queryable encryption, delivering significant performance, cost, and security improvements for modern workloads.

MongoDBShardingVector Search
0 likes · 4 min read
MongoDB 8.0 Unveiled: Massive Performance Gains and New Vector Support
Architect
Architect
Oct 3, 2024 · Databases

How to Tame Massive MySQL Tables: Partitioning, Sharding, and Archiving Strategies

This article walks through evaluating massive MySQL tables, explains why large row counts slow queries, and compares three practical solutions—table partitioning, database sharding, and hot/cold archiving—while highlighting their trade‑offs and offering guidance on selecting the right approach for a given workload.

Hot Cold ArchivingLarge TablesMySQL
0 likes · 18 min read
How to Tame Massive MySQL Tables: Partitioning, Sharding, and Archiving Strategies
Programmer1970
Programmer1970
Oct 2, 2024 · Databases

A Deep Dive into MongoDB’s Technical Architecture

This article outlines MongoDB’s core features, its layered technical architecture—including storage, data model, query language, indexing, replication, sharding, and transactions—and explains the three cluster deployment modes with detailed roles and request flow for sharded clusters.

Database ArchitectureIndexingMongoDB
0 likes · 10 min read
A Deep Dive into MongoDB’s Technical Architecture
Architect's Guide
Architect's Guide
Sep 29, 2024 · Databases

Advantages of NewSQL Databases Over Middleware‑Based Sharding: Architecture, Transactions, HA, and Scaling

This article objectively compares NewSQL distributed databases with traditional middleware‑based sharding, examining their architectural advantages, distributed transaction handling, high‑availability mechanisms, scaling and sharding strategies, storage engine differences, ecosystem maturity, and provides guidance on selecting the appropriate solution for various workloads.

Database ArchitectureNewSQLPaxos
0 likes · 21 min read
Advantages of NewSQL Databases Over Middleware‑Based Sharding: Architecture, Transactions, HA, and Scaling
Programmer1970
Programmer1970
Sep 25, 2024 · Backend Development

How to Use XXL‑JOB in SpringBoot for Flexible Sharding of Massive Data

This article explains how to configure XXL‑JOB with a sharding‑broadcast strategy in a SpringBoot application, use ID modulo hashing to partition large tables, implement the task with XxlJobHelper, write MyBatis sharding SQL, and verify correct parallel processing through detailed logs.

MyBatisShardingSpringBoot
0 likes · 7 min read
How to Use XXL‑JOB in SpringBoot for Flexible Sharding of Massive Data
Architecture Digest
Architecture Digest
Sep 23, 2024 · Databases

Evaluating and Optimizing Large MySQL Tables: Partitioning, Sharding, and Hot‑Cold Archiving Strategies

This article explains how to assess MySQL table size using table capacity, disk usage, and instance resources, describes why large tables degrade performance, and presents three practical solutions—partitioning, sharding (horizontal/vertical), and hot‑cold data archiving—along with their trade‑offs and implementation tips.

B+TreeHotColdArchivingMySQL
0 likes · 15 min read
Evaluating and Optimizing Large MySQL Tables: Partitioning, Sharding, and Hot‑Cold Archiving Strategies
Programmer1970
Programmer1970
Sep 22, 2024 · Backend Development

15 Proven Techniques to Optimize API Performance

This article enumerates fifteen practical techniques—including local and distributed caching, parallelism, asynchrony, pooling, sharding, SQL tuning, pre‑computation, transaction trimming, bulk I/O, lock granularity, context propagation, collection sizing, and query optimization—to systematically improve the response time and throughput of service interfaces.

APICachingConcurrency
0 likes · 14 min read
15 Proven Techniques to Optimize API Performance
Java High-Performance Architecture
Java High-Performance Architecture
Sep 21, 2024 · Backend Development

Master MyBatis-Mate: Advanced Features for Secure, Sharded Java Backends

This article introduces MyBatis‑Mate, an enterprise‑grade MyBatis extension that provides sharding, data auditing, sensitive‑word filtering, field encryption, dictionary binding, dynamic DDL maintenance, data‑scope permissions and performance monitoring, complete with practical code examples and configuration guidance.

JavaMyBatisSharding
0 likes · 15 min read
Master MyBatis-Mate: Advanced Features for Secure, Sharded Java Backends
Top Architect
Top Architect
Sep 17, 2024 · Databases

Design and Implementation of Database Table Sharding for Loan and Repayment Applications Using ShardingSphere and Spring Boot

This technical article describes how a senior architect designed, configured, and migrated loan and repayment application tables to a sharded MySQL setup using ShardingSphere, Spring Boot, and MyBatis‑Plus, covering schema design, historical data sync, dual‑write switches, code integration, and common pitfalls.

BackendShardingdata migration
0 likes · 19 min read
Design and Implementation of Database Table Sharding for Loan and Repayment Applications Using ShardingSphere and Spring Boot
21CTO
21CTO
Aug 31, 2024 · Databases

MongoDB Claims Edge Over PostgreSQL with Atlas Wins and Strong Q2 Growth

MongoDB reported a 13% revenue rise to $478.1 million in Q2 2025, highlighted a major win over PostgreSQL at Fanatics Betting & Gaming, and emphasized its flexible JSON‑like architecture, horizontal scaling, and sharding advantages while noting ongoing competition and a revised growth outlook.

AtlasMongoDBNoSQL
0 likes · 5 min read
MongoDB Claims Edge Over PostgreSQL with Atlas Wins and Strong Q2 Growth
Architecture Digest
Architecture Digest
Aug 29, 2024 · Backend Development

mybatis-mate – Enterprise‑level MyBatis Extension for Sharding, Auditing, Encryption, and Data Permissions

The article introduces mybatis-mate, an enterprise‑grade MyBatis module that supports sharding, multi‑datasource, data auditing, field encryption, sensitive‑data masking, automatic DDL maintenance, performance logging, and fine‑grained data‑scope control, providing code examples and configuration guidelines for Spring Boot projects.

Data AuditingJavaMyBatis
0 likes · 15 min read
mybatis-mate – Enterprise‑level MyBatis Extension for Sharding, Auditing, Encryption, and Data Permissions
Java High-Performance Architecture
Java High-Performance Architecture
Aug 28, 2024 · Databases

NewSQL vs Sharding: Which Database Architecture Truly Wins?

This article objectively compares NewSQL distributed databases with traditional middleware‑based sharding solutions, examining their architectural differences, transaction support, scalability, HA, storage engines, and ecosystem maturity, to help readers decide which approach best fits their performance, consistency, and operational needs.

Distributed DatabaseHANewSQL
0 likes · 18 min read
NewSQL vs Sharding: Which Database Architecture Truly Wins?
macrozheng
macrozheng
Aug 23, 2024 · Databases

NewSQL vs Middleware Sharding: Which Architecture Truly Wins?

This article objectively compares NewSQL databases with middleware‑based sharding solutions, examining architecture, distributed transactions, CAP constraints, high availability, scaling, SQL support, storage engines, and maturity to help readers choose the right approach for their workloads.

CAP theoremHigh AvailabilityNewSQL
0 likes · 19 min read
NewSQL vs Middleware Sharding: Which Architecture Truly Wins?
Top Architect
Top Architect
Aug 18, 2024 · Databases

Design and Implementation of Database Table Partitioning for Loan and Repayment Systems

This article details the design, planning, and step‑by‑step implementation of a database table‑partitioning solution for loan and repayment request tables, covering schema decisions, sharding strategy, historical data migration, dynamic synchronization, code examples with ShardingSphere, SpringBoot, and operational considerations for seamless migration.

ShardingShardingSphereSpringBoot
0 likes · 18 min read
Design and Implementation of Database Table Partitioning for Loan and Repayment Systems
Architecture & Thinking
Architecture & Thinking
Aug 15, 2024 · Databases

How to Evolve Your Database Architecture for Massive Scale: From Monolith to Sharding

This article outlines the progressive evolution of database architectures—from simple monolithic designs to cold‑hot separation, read‑write splitting, and horizontal sharding—explaining each pattern, its components, and the performance and scalability benefits they bring to large‑scale systems.

Cold Hot SeparationDatabase ArchitectureMySQL
0 likes · 8 min read
How to Evolve Your Database Architecture for Massive Scale: From Monolith to Sharding
FunTester
FunTester
Aug 15, 2024 · Backend Development

9 Proven Techniques to Supercharge Service Performance

This article outlines nine practical methods—caching, parallelization, batch processing, data compression, lock‑free design, sharding, request avoidance, pooling, and asynchronous handling—demonstrating how each can be applied to backend services to dramatically reduce latency and improve throughput.

CachingCompressionParallelism
0 likes · 25 min read
9 Proven Techniques to Supercharge Service Performance
Top Architect
Top Architect
Aug 9, 2024 · Databases

Design and Implementation of Database Sharding for Loan and Repayment Tables Using ShardingSphere and Spring Boot

After a year of business coding, the author details the design, implementation, and migration strategy for splitting loan and repayment tables into 50 sharded tables using ShardingSphere with Spring Boot, covering schema design, data synchronization, dynamic switches, and practical pitfalls.

ShardingSpring Bootbackend development
0 likes · 20 min read
Design and Implementation of Database Sharding for Loan and Repayment Tables Using ShardingSphere and Spring Boot
Su San Talks Tech
Su San Talks Tech
Aug 5, 2024 · Fundamentals

Mastering Distributed Systems: Common Pitfalls and How to Avoid Them

This article explains the core concepts of distributed systems—including the CAP theorem, BASE theory, message‑queue challenges, Redis sentinel issues, sharding strategies, unique ID generation, and distributed transaction patterns—while offering practical guidance to prevent common pitfalls and improve reliability.

CAP theoremMessage QueueSharding
0 likes · 27 min read
Mastering Distributed Systems: Common Pitfalls and How to Avoid Them