BestHub
Sign in
Tagged articles

Sharding

656 articles · Page 2 of 7
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Jul 10, 2025 · Big Data

Master ElasticSearch: Core Concepts, Architecture, and Search Process Explained

This article provides a comprehensive overview of ElasticSearch, covering its role as a distributed full‑text search engine built on Lucene, key concepts such as index, type, document, field, shard and replica, the analysis pipeline, inverted index mechanics, and the two‑phase query‑fetch search workflow.

AnalysisElasticsearchFull-text Search
0 likes · 7 min read
Master ElasticSearch: Core Concepts, Architecture, and Search Process Explained
Selected Java Interview Questions
Selected Java Interview Questions
Jul 8, 2025 · Databases

How to Scale 1 Billion Orders with MySQL Sharding: Strategies, Gene IDs, and Migration

This article explores the performance bottlenecks of a 700‑million‑row MySQL orders table and presents a comprehensive sharding solution—including vertical and horizontal partitioning, gene‑based shard keys, routing logic, global secondary indexes, data migration strategies, and common pitfalls—demonstrating how query latency can drop from seconds to milliseconds while supporting billions of records.

Data MigrationDatabase ScalingMySQL
0 likes · 9 min read
How to Scale 1 Billion Orders with MySQL Sharding: Strategies, Gene IDs, and Migration
Su San Talks Tech
Su San Talks Tech
Jul 3, 2025 · Databases

Mastering MySQL Sharding: Strategies for 1 Billion Orders

This article explores the pain points of a 700‑million‑row MySQL order table, presents vertical and horizontal sharding strategies, introduces gene‑based Snowflake IDs, details routing logic, migration steps, common pitfalls, and shows performance gains after applying the final architecture.

Database ScalingMySQLSharding
0 likes · 9 min read
Mastering MySQL Sharding: Strategies for 1 Billion Orders
Top Architect
Top Architect
Jun 27, 2025 · Databases

How to Implement High‑Performance Database Sharding with ShardingSphere and SpringBoot

This article walks through the complete process of designing, configuring, and coding a sharding solution for loan and repayment tables—including database basics, historical data migration, backend query refactoring, a three‑write strategy, scheduled consistency checks, and full SpringBoot/ShardingSphere implementation details.

Data MigrationDatabaseMyBatis
0 likes · 20 min read
How to Implement High‑Performance Database Sharding with ShardingSphere and SpringBoot
vivo Internet Technology
vivo Internet Technology
Jun 18, 2025 · Backend Development

Cutting Spring Boot Startup Time by 50%: From 280 s to 159 s

This article systematically analyzes a Spring Boot project's 280‑second startup, identifies bottlenecks such as bean initialization and sharding data source loading, and applies listener‑based timing, bean‑post‑processor profiling, and asynchronous initialization to reduce launch time to 159 seconds, improving developer efficiency.

ShardingSpring Bootasynchronous-initialization
0 likes · 13 min read
Cutting Spring Boot Startup Time by 50%: From 280 s to 159 s
Senior Tony
Senior Tony
Jun 18, 2025 · Backend Development

Mastering High‑Concurrency Account Balance Deductions with Kafka, Sharding, and Parallel Processing

This article examines the challenges of high‑concurrency write scenarios in ad‑billing systems, explains why row‑level locking becomes a bottleneck, and presents a step‑by‑step solution using Kafka for asynchronous peak shaving, parallel consumer processing, database sharding, hotspot dispersion, and batch deduction to achieve reliable real‑time balance updates.

KafkaShardingaccount balance deduction
0 likes · 8 min read
Mastering High‑Concurrency Account Balance Deductions with Kafka, Sharding, and Parallel Processing
Baidu Tech Salon
Baidu Tech Salon
Jun 17, 2025 · Operations

How Baidu Scaled Its Vertical Search: Elastic Scheduling and Data Management Secrets

This article explains how Baidu's vertical search platform tackled massive data growth and scaling challenges by redesigning its data management system, introducing elastic scheduling, decoupling ETCD access, implementing auto‑scaling, and advancing shard expansion to improve performance, stability, and cost efficiency.

Auto ScalingData ManagementSearch Architecture
0 likes · 18 min read
How Baidu Scaled Its Vertical Search: Elastic Scheduling and Data Management Secrets
Java Captain
Java Captain
Jun 10, 2025 · Databases

Master MySQL Performance: Slow Queries, Indexes, Transactions & Replication

This article explains how to locate and analyze MySQL slow queries, understand index structures and types, apply best practices for index creation, handle large pagination, manage transaction isolation and MVCC, and implement master‑slave replication and sharding strategies for high‑performance databases.

MySQLShardingSlow Query
0 likes · 13 min read
Master MySQL Performance: Slow Queries, Indexes, Transactions & Replication
Lobster Programming
Lobster Programming
Jun 9, 2025 · Databases

How to Add a Column to Billion‑Row Tables Without Downtime

This article explains a metadata‑driven approach for extending massive tables—using a separate extension table, sharding, and Elasticsearch sync—to add new fields to billion‑row databases without locking the primary table or disrupting online services.

Big DataElasticsearchSharding
0 likes · 6 min read
How to Add a Column to Billion‑Row Tables Without Downtime
Code Ape Tech Column
Code Ape Tech Column
May 28, 2025 · Databases

Evaluating MySQL Table Size and Optimizing Large Tables with Partitioning, Sharding, and Hot‑Cold Archiving

This article explains how to assess MySQL table size from table capacity, disk usage, and instance limits, describes why large tables degrade performance due to B+‑tree depth, and presents three practical solutions—partitioning, sharding, and hot‑cold archiving—to improve query speed and manage growth.

B+TreeHot Cold ArchivingMySQL
0 likes · 16 min read
Evaluating MySQL Table Size and Optimizing Large Tables with Partitioning, Sharding, and Hot‑Cold Archiving
Top Architect
Top Architect
May 26, 2025 · Databases

Design and Implementation of MySQL Table Sharding with ShardingSphere and Spring Boot

This article walks through the full process of designing, configuring, and implementing a sharding solution for large loan and repayment tables using ShardingSphere, Spring Boot 3, MySQL, and MyBatis‑Plus, covering schema planning, data migration, DBA coordination, code examples, common pitfalls, and operational considerations.

MigrationShardingSpringBoot
0 likes · 19 min read
Design and Implementation of MySQL Table Sharding with ShardingSphere and Spring Boot
Aikesheng Open Source Community
Aikesheng Open Source Community
May 22, 2025 · Databases

Root Cause Analysis of MySQL Replication Lag Caused by Large Transactions in a Sharding Scenario

This article examines a real‑world MySQL master‑slave replication lag issue caused by massive multi‑table transactions during a sharding operation, details the investigative steps using system tables, binlog parsing, and my2sql analysis, and proposes three practical mitigation strategies.

Large TransactionsMySQLReplication Lag
0 likes · 8 min read
Root Cause Analysis of MySQL Replication Lag Caused by Large Transactions in a Sharding Scenario
Cognitive Technology Team
Cognitive Technology Team
May 15, 2025 · Databases

How to Solve MySQL Deep Pagination Performance Issues

This article analyzes why large OFFSET values cause severe MySQL performance degradation and presents multiple optimization techniques—including range queries, subqueries, delayed joins, covering indexes, sharding, caching, and search engine integration—along with their advantages, limitations, and practical recommendations.

IndexingMySQLSharding
0 likes · 11 min read
How to Solve MySQL Deep Pagination Performance Issues
dbaplus Community
dbaplus Community
May 13, 2025 · Databases

Master Elasticsearch Index Design: From Mapping to Sharding Best Practices

This article provides a comprehensive guide to Elasticsearch index architecture, covering fundamental concepts, index mapping, field types, alias usage, shard and replica strategies, shard planning, resource impact, and practical recommendations for optimizing performance and stability in production environments.

ElasticsearchShardingbackend
0 likes · 21 min read
Master Elasticsearch Index Design: From Mapping to Sharding Best Practices
Architect
Architect
May 4, 2025 · Databases

NewSQL vs Middleware Sharding: Which Architecture Truly Wins?

This article objectively compares middleware‑based sharding with NewSQL distributed databases, examining architecture, distributed transactions, performance, high availability, scaling, SQL support, storage engines, and ecosystem maturity to help architects decide which solution fits their specific workload and operational constraints.

CAP theoremDatabase ArchitectureDistributed Databases
0 likes · 20 min read
NewSQL vs Middleware Sharding: Which Architecture Truly Wins?
Su San Talks Tech
Su San Talks Tech
Apr 29, 2025 · Databases

Why Split Databases? Master Sharding Concepts, Strategies, and Practical SQL Routing

This article explains the fundamental concepts of database sharding—including data nodes, logical and broadcast tables, sharding keys, strategies, algorithms, SQL parsing, routing, rewriting, execution, result merging, distributed primary keys, data masking, distributed transactions, data migration, and shadow databases—providing clear examples and code snippets for real‑world implementation.

Data MigrationSQL RoutingSharding
0 likes · 21 min read
Why Split Databases? Master Sharding Concepts, Strategies, and Practical SQL Routing
Full-Stack Internet Architecture
Full-Stack Internet Architecture
Apr 25, 2025 · Databases

Essential Knowledge Points of Redis Cluster

Redis cluster relies on two ports per node, a hash‑slot based sharding scheme, a master‑slave replication model, and provides eventual consistency, making these four fundamentals essential knowledge for developers and operators to.

ClusterRedisSharding
0 likes · 6 min read
Essential Knowledge Points of Redis Cluster
Architect's Guide
Architect's Guide
Apr 25, 2025 · Databases

Solutions for MySQL Auto Increment ID Exhaustion

This article discusses the problem of MySQL auto‑increment ID exhaustion and presents six practical solutions—including changing column type to BIGINT, using UUIDs, segmenting ID generation, composite keys, adjusting auto‑increment steps, and database sharding—to ensure scalability and uniqueness.

Database DesignID ExhaustionMySQL
0 likes · 6 min read
Solutions for MySQL Auto Increment ID Exhaustion
Architect
Architect
Apr 23, 2025 · Databases

Mastering Elasticsearch Index Design: From Basics to Shard Planning

This article provides a comprehensive guide to Elasticsearch index concepts, covering index definitions, alias usage, mapping and field types, shard architecture, and practical recommendations for planning shards and optimizing resource consumption to ensure stable and efficient ES clusters.

ElasticsearchShardingindex design
0 likes · 23 min read
Mastering Elasticsearch Index Design: From Basics to Shard Planning
Java Captain
Java Captain
Apr 17, 2025 · Databases

Choosing Between Sharding Middleware and NewSQL Distributed Databases: An Objective Comparison

This article objectively compares middleware‑based sharding with NewSQL distributed databases, examining their architectural differences, transaction models, high‑availability mechanisms, scaling, SQL support, storage engines, and maturity to help practitioners decide which approach best fits their workload and operational constraints.

Database ArchitectureHigh AvailabilityNewSQL
0 likes · 17 min read
Choosing Between Sharding Middleware and NewSQL Distributed Databases: An Objective Comparison
macrozheng
macrozheng
Apr 16, 2025 · Databases

Why Enterprises Shun Foreign Keys: Performance Risks and Alternatives

Although foreign keys enforce referential integrity, many enterprises avoid them because they can slow inserts, cause blocking cascade updates, trigger database update storms in high‑concurrency environments, and complicate sharding, leading developers to prefer redundant fields and handle relationships at the application layer.

Database PerformanceForeign KeyMySQL
0 likes · 6 min read
Why Enterprises Shun Foreign Keys: Performance Risks and Alternatives
DeWu Technology
DeWu Technology
Apr 9, 2025 · Databases

Understanding Elasticsearch Indexes, Mappings, and Shard Architecture

The article explains Elasticsearch indexes as logical namespaces for JSON documents, details how mappings define immutable field types such as text, keyword, and numeric, describes primary and replica shard roles, and offers practical guidance on alias usage, shard sizing, replica settings, and performance optimizations to ensure cluster stability.

ElasticsearchShardingindex design
0 likes · 20 min read
Understanding Elasticsearch Indexes, Mappings, and Shard Architecture
vivo Internet Technology
vivo Internet Technology
Apr 2, 2025 · Databases

Case Study of Slow SQL Governance in an Activity Middleware System

The case study details how an activity‑middleware platform tackled slow SQL problems by cleaning historic data, introducing sharding and Elasticsearch off‑loading, optimizing queries with proper indexes and simplified joins, and automating nightly deletions, ultimately cutting daily slow queries from thousands to double‑digit levels and boosting system stability.

Database PerformanceIndexingMySQL
0 likes · 17 min read
Case Study of Slow SQL Governance in an Activity Middleware System
Su San Talks Tech
Su San Talks Tech
Mar 30, 2025 · Databases

How to Supercharge Large Database Tables: Proven Optimization Techniques

This article explains why massive tables become slow, identifies common bottlenecks such as disk I/O, missing indexes, deep pagination and lock contention, and provides a step‑by‑step guide covering table design, indexing, SQL tuning, sharding, caching and a real‑world case study to dramatically improve performance.

Index TuningLarge TablesSQL Performance
0 likes · 13 min read
How to Supercharge Large Database Tables: Proven Optimization Techniques
Java Architect Essentials
Java Architect Essentials
Mar 14, 2025 · Databases

Comparing NewSQL Databases with Middleware‑Based Sharding: Advantages, Trade‑offs, and Selection Guidance

This article objectively compares NewSQL distributed databases with traditional middleware‑based sharding solutions, examining their architectures, distributed transaction handling, high‑availability, scaling, storage engines, and ecosystem maturity, and provides guidance on selecting the appropriate approach based on consistency, growth, operational capacity, and performance requirements.

Database ArchitectureHigh AvailabilityNewSQL
0 likes · 19 min read
Comparing NewSQL Databases with Middleware‑Based Sharding: Advantages, Trade‑offs, and Selection Guidance
Architecture Digest
Architecture Digest
Mar 12, 2025 · Backend Development

Designing and Optimizing a High‑Concurrency Counting System for Large‑Scale Social Media

This article explains the challenges of storing and serving trillion‑level count data for a massive micro‑blogging platform and presents a step‑by‑step backend design that evolves from simple MySQL tables to sharded databases, Redis caching, message‑queue write‑sharding, and memory‑efficient custom data structures to achieve high performance, low cost, and strong consistency.

MySQLShardingcounting system
0 likes · 12 min read
Designing and Optimizing a High‑Concurrency Counting System for Large‑Scale Social Media
Java Web Project
Java Web Project
Mar 6, 2025 · Databases

NewSQL vs Middleware Sharding: Which Architecture Truly Wins?

This article objectively compares NewSQL databases with middleware‑based sharding, dissecting their core architectures, distributed transaction handling, high‑availability designs, scaling mechanisms, SQL support, storage engines, and maturity to help engineers decide the most suitable solution for their workloads.

CAP theoremDatabase ArchitectureHigh Availability
0 likes · 20 min read
NewSQL vs Middleware Sharding: Which Architecture Truly Wins?
macrozheng
macrozheng
Mar 1, 2025 · Databases

NewSQL vs Middleware Sharding: Which Database Architecture Truly Wins?

An in‑depth comparison of NewSQL databases and middleware‑based sharding reveals each approach’s architectural strengths, distributed transaction handling, scalability, HA mechanisms, storage engine design, and ecosystem maturity, guiding readers on when to adopt NewSQL versus traditional sharding solutions.

Distributed DatabasesNewSQLSharding
0 likes · 18 min read
NewSQL vs Middleware Sharding: Which Database Architecture Truly Wins?
JD Tech
JD Tech
Jan 9, 2025 · Databases

Challenges and Practices of Distributed Data Systems: Master‑Slave Replication, Partitioning, and High‑Availability Strategies

This article examines the core challenges of distributed data systems—including consistency, availability, and partition tolerance—then details master‑slave replication mechanisms for MySQL and Redis, various replication modes and binlog formats, and explores data partitioning, sharding, and hot‑key mitigation techniques for scalable, high‑availability deployments.

DatabasesHigh AvailabilityMySQL
0 likes · 23 min read
Challenges and Practices of Distributed Data Systems: Master‑Slave Replication, Partitioning, and High‑Availability Strategies
Su San Talks Tech
Su San Talks Tech
Jan 7, 2025 · Databases

7 Common Pitfalls of Database Sharding and How to Solve Them

This article examines seven typical challenges that arise after implementing database sharding—such as global ID conflicts, cross‑shard queries, distributed transactions, shard‑key design, data migration, pagination, and operational overhead—and provides practical solutions and code examples to address each issue.

Database ScalingShardingTransaction Management
0 likes · 12 min read
7 Common Pitfalls of Database Sharding and How to Solve Them
Architect
Architect
Jan 3, 2025 · Operations

Designing Multi‑Active Distributed Systems: Overcoming Write Latency and Data Replication Challenges

This article analyzes the architectural impact of cross‑city multi‑active deployments, focusing on data‑layer design, write latency, sharding strategies, replication topologies, and routing considerations to achieve high availability, performance, and scalability in large‑scale distributed systems.

Data ReplicationShardingdistributed systems
0 likes · 22 min read
Designing Multi‑Active Distributed Systems: Overcoming Write Latency and Data Replication Challenges
ITPUB
ITPUB
Jan 3, 2025 · Databases

Why ClickHouse Sharded Table Queries Return Inconsistent Row Counts—and How to Fix It

A ClickHouse cluster showed wildly varying row counts when querying sharded tables, while local tables behaved correctly; the article analyses the root cause in the cluster and table configuration, explains why the inconsistency occurs, and provides a step‑by‑step fix by switching to replicated tables.

ClickHouseDatabaseQuery Inconsistency
0 likes · 7 min read
Why ClickHouse Sharded Table Queries Return Inconsistent Row Counts—and How to Fix It
dbaplus Community
dbaplus Community
Dec 29, 2024 · Databases

How to Speed Up Massive MySQL Tables: Partitioning, Sharding, and Hot/Cold Archiving

When a MySQL table grows to tens of millions of rows, insert and query latency spikes, schema changes become painful, and only a subset of data is needed; this guide explains how to assess table size, why large tables slow down, and three practical solutions—partitioning, sharding, and hot/cold archiving—along with their trade‑offs and implementation steps.

Hot Cold ArchivingMySQLSharding
0 likes · 17 min read
How to Speed Up Massive MySQL Tables: Partitioning, Sharding, and Hot/Cold Archiving
Lobster Programming
Lobster Programming
Dec 26, 2024 · Databases

How to Migrate Tens of Millions of Rows: Strategies and Practical Steps

This article explains common data‑migration strategies such as hash‑modulo, time‑range and ID‑range sharding, and compares practical migration solutions including downtime migration, dual‑write, and an MQ + Redis approach, with detailed workflow diagrams and implementation tips.

Data MigrationDatabase ScalingMessage Queue
0 likes · 8 min read
How to Migrate Tens of Millions of Rows: Strategies and Practical Steps
Top Architect
Top Architect
Dec 20, 2024 · Databases

Database Optimization, Sharding, and Pagination Issues: Lessons from a Large‑Scale MySQL Deployment

The article shares a senior architect’s experience with large‑scale MySQL tables, discussing why sharding and Elasticsearch were introduced, the pagination bug caused by Sharding‑JDBC, and a comprehensive set of hardware, software, and SQL tuning recommendations, including a full MySQL configuration example.

ElasticsearchMySQLOptimization
0 likes · 14 min read
Database Optimization, Sharding, and Pagination Issues: Lessons from a Large‑Scale MySQL Deployment
Radish, Keep Going!
Radish, Keep Going!
Dec 6, 2024 · Backend Development

How BigCache Achieves Ultra‑Fast In‑Memory Caching in Go

This article explains how the Go library BigCache uses sharding, efficient hash functions, and memory‑optimised data structures to deliver high‑concurrency, low‑latency caching, while avoiding GC bottlenecks and providing practical code examples and performance benchmarks.

GoMemory OptimizationSharding
0 likes · 12 min read
How BigCache Achieves Ultra‑Fast In‑Memory Caching in Go
Top Architect
Top Architect
Nov 12, 2024 · Databases

Design and Implementation of Table Sharding for Loan Repayment Applications Using ShardingSphere and Spring Boot

This article describes how a senior architect designed, configured, and implemented a 50‑table sharding solution for loan and repayment request data using ShardingSphere, Spring Boot 3, MySQL, and custom synchronization scripts, while also addressing historical data migration, backend query changes, and operational safeguards.

MySQLShardingShardingSphere
0 likes · 23 min read
Design and Implementation of Table Sharding for Loan Repayment Applications Using ShardingSphere and Spring Boot
dbaplus Community
dbaplus Community
Nov 6, 2024 · Databases

Why Sharding Fails: Hidden Pitfalls of Database Partitioning in E‑commerce

This article examines the fundamental drawbacks of MySQL sharding in large‑scale e‑commerce, highlighting distributed transaction challenges, limited cross‑shard queries, global key constraints, scaling difficulties, operational overhead, and why distributed databases may be a better long‑term solution.

Shardingdatabase partitioningdistributed transactions
0 likes · 11 min read
Why Sharding Fails: Hidden Pitfalls of Database Partitioning in E‑commerce
Selected Java Interview Questions
Selected Java Interview Questions
Oct 26, 2024 · Databases

Handling MySQL Auto Increment ID Exhaustion: Strategies and Solutions

This article explains why MySQL auto‑increment IDs can run out in large or special‑case tables and presents six practical solutions—including changing the column type to BIGINT, using UUIDs, segmenting IDs, composite keys, adjusting auto‑increment steps, and sharding databases—to ensure scalable and unique primary keys.

ID ExhaustionMySQLSharding
0 likes · 6 min read
Handling MySQL Auto Increment ID Exhaustion: Strategies and Solutions
Tencent Cloud Developer
Tencent Cloud Developer
Oct 15, 2024 · Industry Insights

Why Write Latency Drives Multi‑Active Distributed Architecture Design

This article analyzes how write latency, write volume, isolation, and data replication strategies influence the design of multi‑active distributed systems, offering practical guidance on sharding, synchronous and asynchronous replication, routing, and architecture selection for high availability and performance across regions.

Data ReplicationHigh AvailabilityMulti-Active
0 likes · 23 min read
Why Write Latency Drives Multi‑Active Distributed Architecture Design
Programmer XiaoFu
Programmer XiaoFu
Oct 10, 2024 · Databases

Three Strategies Big Companies Use to Manage Massive MySQL Tables

The article walks through how to assess MySQL table size, explains why large tables slow down due to B+‑tree index overhead, and compares three practical solutions—table partitioning, database sharding (horizontal, vertical, hash‑range), and hot‑cold archiving—while outlining their trade‑offs and selection criteria.

Database ScalingHot Cold ArchivingMySQL
0 likes · 16 min read
Three Strategies Big Companies Use to Manage Massive MySQL Tables
21CTO
21CTO
Oct 3, 2024 · Databases

MongoDB 8.0 Unveiled: Massive Performance Gains and New Vector Support

MongoDB 8.0 launches with up to 36% higher read throughput, 56% faster batch writes, 200% faster time‑series processing, 50‑fold faster sharding distribution, quantized vector search, and queryable encryption, delivering significant performance, cost, and security improvements for modern workloads.

DatabaseMongoDBSharding
0 likes · 4 min read
MongoDB 8.0 Unveiled: Massive Performance Gains and New Vector Support
Architect
Architect
Oct 3, 2024 · Databases

How to Tame Massive MySQL Tables: Partitioning, Sharding, and Archiving Strategies

This article walks through evaluating massive MySQL tables, explains why large row counts slow queries, and compares three practical solutions—table partitioning, database sharding, and hot/cold archiving—while highlighting their trade‑offs and offering guidance on selecting the right approach for a given workload.

Hot Cold ArchivingLarge TablesMySQL
0 likes · 18 min read
How to Tame Massive MySQL Tables: Partitioning, Sharding, and Archiving Strategies
Programmer1970
Programmer1970
Oct 2, 2024 · Databases

A Deep Dive into MongoDB’s Technical Architecture

This article outlines MongoDB’s core features, its layered technical architecture—including storage, data model, query language, indexing, replication, sharding, and transactions—and explains the three cluster deployment modes with detailed roles and request flow for sharded clusters.

Database ArchitectureIndexingMongoDB
0 likes · 10 min read
A Deep Dive into MongoDB’s Technical Architecture
Architect's Guide
Architect's Guide
Sep 29, 2024 · Databases

Advantages of NewSQL Databases Over Middleware‑Based Sharding: Architecture, Transactions, HA, and Scaling

This article objectively compares NewSQL distributed databases with traditional middleware‑based sharding, examining their architectural advantages, distributed transaction handling, high‑availability mechanisms, scaling and sharding strategies, storage engine differences, ecosystem maturity, and provides guidance on selecting the appropriate solution for various workloads.

Database ArchitectureNewSQLPaxos
0 likes · 21 min read
Advantages of NewSQL Databases Over Middleware‑Based Sharding: Architecture, Transactions, HA, and Scaling
Programmer1970
Programmer1970
Sep 25, 2024 · Backend Development

How to Use XXL‑JOB in SpringBoot for Flexible Sharding of Massive Data

This article explains how to configure XXL‑JOB with a sharding‑broadcast strategy in a SpringBoot application, use ID modulo hashing to partition large tables, implement the task with XxlJobHelper, write MyBatis sharding SQL, and verify correct parallel processing through detailed logs.

Batch ProcessingMyBatisSharding
0 likes · 7 min read
How to Use XXL‑JOB in SpringBoot for Flexible Sharding of Massive Data
Architecture Digest
Architecture Digest
Sep 23, 2024 · Databases

Evaluating and Optimizing Large MySQL Tables: Partitioning, Sharding, and Hot‑Cold Archiving Strategies

This article explains how to assess MySQL table size using table capacity, disk usage, and instance resources, describes why large tables degrade performance, and presents three practical solutions—partitioning, sharding (horizontal/vertical), and hot‑cold data archiving—along with their trade‑offs and implementation tips.

B+TreeHotColdArchivingMySQL
0 likes · 15 min read
Evaluating and Optimizing Large MySQL Tables: Partitioning, Sharding, and Hot‑Cold Archiving Strategies
Programmer1970
Programmer1970
Sep 22, 2024 · Backend Development

15 Proven Techniques to Optimize API Performance

This article enumerates fifteen practical techniques—including local and distributed caching, parallelism, asynchrony, pooling, sharding, SQL tuning, pre‑computation, transaction trimming, bulk I/O, lock granularity, context propagation, collection sizing, and query optimization—to systematically improve the response time and throughput of service interfaces.

APIAsynchronousSQL
0 likes · 14 min read
15 Proven Techniques to Optimize API Performance
Java High-Performance Architecture
Java High-Performance Architecture
Sep 21, 2024 · Backend Development

Master MyBatis-Mate: Advanced Features for Secure, Sharded Java Backends

This article introduces MyBatis‑Mate, an enterprise‑grade MyBatis extension that provides sharding, data auditing, sensitive‑word filtering, field encryption, dictionary binding, dynamic DDL maintenance, data‑scope permissions and performance monitoring, complete with practical code examples and configuration guidance.

DatabaseMyBatisSharding
0 likes · 15 min read
Master MyBatis-Mate: Advanced Features for Secure, Sharded Java Backends
Top Architect
Top Architect
Sep 17, 2024 · Databases

Design and Implementation of Database Table Sharding for Loan and Repayment Applications Using ShardingSphere and Spring Boot

This technical article describes how a senior architect designed, configured, and migrated loan and repayment application tables to a sharded MySQL setup using ShardingSphere, Spring Boot, and MyBatis‑Plus, covering schema design, historical data sync, dual‑write switches, code integration, and common pitfalls.

Data MigrationShardingbackend
0 likes · 19 min read
Design and Implementation of Database Table Sharding for Loan and Repayment Applications Using ShardingSphere and Spring Boot
21CTO
21CTO
Aug 31, 2024 · Databases

MongoDB Claims Edge Over PostgreSQL with Atlas Wins and Strong Q2 Growth

MongoDB reported a 13% revenue rise to $478.1 million in Q2 2025, highlighted a major win over PostgreSQL at Fanatics Betting & Gaming, and emphasized its flexible JSON‑like architecture, horizontal scaling, and sharding advantages while noting ongoing competition and a revised growth outlook.

ATLASMongoDBNoSQL
0 likes · 5 min read
MongoDB Claims Edge Over PostgreSQL with Atlas Wins and Strong Q2 Growth
Architecture Digest
Architecture Digest
Aug 29, 2024 · Backend Development

mybatis-mate – Enterprise‑level MyBatis Extension for Sharding, Auditing, Encryption, and Data Permissions

The article introduces mybatis-mate, an enterprise‑grade MyBatis module that supports sharding, multi‑datasource, data auditing, field encryption, sensitive‑data masking, automatic DDL maintenance, performance logging, and fine‑grained data‑scope control, providing code examples and configuration guidelines for Spring Boot projects.

Data AuditingMyBatisSharding
0 likes · 15 min read
mybatis-mate – Enterprise‑level MyBatis Extension for Sharding, Auditing, Encryption, and Data Permissions
Java High-Performance Architecture
Java High-Performance Architecture
Aug 28, 2024 · Databases

NewSQL vs Sharding: Which Database Architecture Truly Wins?

This article objectively compares NewSQL distributed databases with traditional middleware‑based sharding solutions, examining their architectural differences, transaction support, scalability, HA, storage engines, and ecosystem maturity, to help readers decide which approach best fits their performance, consistency, and operational needs.

HANewSQLSharding
0 likes · 18 min read
NewSQL vs Sharding: Which Database Architecture Truly Wins?
macrozheng
macrozheng
Aug 23, 2024 · Databases

NewSQL vs Middleware Sharding: Which Architecture Truly Wins?

This article objectively compares NewSQL databases with middleware‑based sharding solutions, examining architecture, distributed transactions, CAP constraints, high availability, scaling, SQL support, storage engines, and maturity to help readers choose the right approach for their workloads.

CAP theoremDistributed DatabasesHigh Availability
0 likes · 19 min read
NewSQL vs Middleware Sharding: Which Architecture Truly Wins?
Top Architect
Top Architect
Aug 18, 2024 · Databases

Design and Implementation of Database Table Partitioning for Loan and Repayment Systems

This article details the design, planning, and step‑by‑step implementation of a database table‑partitioning solution for loan and repayment request tables, covering schema decisions, sharding strategy, historical data migration, dynamic synchronization, code examples with ShardingSphere, SpringBoot, and operational considerations for seamless migration.

Data MigrationShardingShardingSphere
0 likes · 18 min read
Design and Implementation of Database Table Partitioning for Loan and Repayment Systems
Architecture & Thinking
Architecture & Thinking
Aug 15, 2024 · Databases

How to Evolve Your Database Architecture for Massive Scale: From Monolith to Sharding

This article outlines the progressive evolution of database architectures—from simple monolithic designs to cold‑hot separation, read‑write splitting, and horizontal sharding—explaining each pattern, its components, and the performance and scalability benefits they bring to large‑scale systems.

Cold Hot SeparationDatabase ArchitectureMySQL
0 likes · 8 min read
How to Evolve Your Database Architecture for Massive Scale: From Monolith to Sharding
FunTester
FunTester
Aug 15, 2024 · Backend Development

9 Proven Techniques to Supercharge Service Performance

This article outlines nine practical methods—caching, parallelization, batch processing, data compression, lock‑free design, sharding, request avoidance, pooling, and asynchronous handling—demonstrating how each can be applied to backend services to dramatically reduce latency and improve throughput.

AsynchronousBatch ProcessingLock-Free
0 likes · 25 min read
9 Proven Techniques to Supercharge Service Performance
Su San Talks Tech
Su San Talks Tech
Aug 5, 2024 · Fundamentals

Mastering Distributed Systems: Common Pitfalls and How to Avoid Them

This article explains the core concepts of distributed systems—including the CAP theorem, BASE theory, message‑queue challenges, Redis sentinel issues, sharding strategies, unique ID generation, and distributed transaction patterns—while offering practical guidance to prevent common pitfalls and improve reliability.

CAP theoremMessage QueueSharding
0 likes · 27 min read
Mastering Distributed Systems: Common Pitfalls and How to Avoid Them
Architect
Architect
Jul 29, 2024 · Databases

NewSQL vs Middleware Sharding: Which Architecture Truly Wins?

This article objectively compares NewSQL distributed databases with traditional middleware‑based sharding solutions, examining their architectures, transaction models, scalability, high‑availability mechanisms, storage engines, maturity, and offering a decision framework to help engineers choose the most suitable approach for their workloads.

CAP theoremDatabase ArchitectureDistributed Databases
0 likes · 20 min read
NewSQL vs Middleware Sharding: Which Architecture Truly Wins?
Java Tech Enthusiast
Java Tech Enthusiast
Jul 21, 2024 · Backend Development

Interface Performance Optimization Techniques for Backend Development

The article outlines practical backend interface performance optimizations—including proper indexing, SQL tuning, parallel remote calls, batch queries, asynchronous processing, scoped transactions, fine-grained locking, pagination batching, multi-level caching, sharding, and monitoring tools—to dramatically reduce latency and improve throughput.

IndexingSQL OptimizationSharding
0 likes · 25 min read
Interface Performance Optimization Techniques for Backend Development
Selected Java Interview Questions
Selected Java Interview Questions
Jul 20, 2024 · Backend Development

Introduction and Usage Guide for MyBatis-Mate: Enterprise Features for MyBatis-Plus

This article introduces MyBatis-Mate, an official MyBatis‑Plus extension offering enterprise‑level capabilities such as dictionary binding, field encryption, data masking, dynamic DDL maintenance, sharding, multi‑datasource switching, performance logging, and data permission control, with detailed Spring Boot integration examples.

MyBatisShardingSpring Boot
0 likes · 15 min read
Introduction and Usage Guide for MyBatis-Mate: Enterprise Features for MyBatis-Plus
Programmer XiaoFu
Programmer XiaoFu
Jul 18, 2024 · Backend Development

11 Proven Techniques to Boost API Performance by 100×

This article walks through eleven practical methods—such as adding proper indexes, optimizing SQL, parallelizing remote calls, eliminating duplicate queries, using async processing, shrinking transaction scope, refining lock granularity, applying pagination, leveraging caching layers, and employing sharding and monitoring tools—to dramatically improve backend interface performance.

API performanceMonitoringSQL Tuning
0 likes · 27 min read
11 Proven Techniques to Boost API Performance by 100×
Architect
Architect
Jul 17, 2024 · Databases

Design and Implementation of Table Sharding for Cash Repayment System Using ShardingSphere and SpringBoot

The article describes how a one‑year‑old loan/repayment service was refactored by introducing a 50‑table sharding scheme with ShardingSphere, detailing the design decisions, historical data migration, code changes, configuration files, pitfalls, and runtime synchronization strategies to achieve efficient query performance on billions of rows.

Data MigrationMySQLSharding
0 likes · 23 min read
Design and Implementation of Table Sharding for Cash Repayment System Using ShardingSphere and SpringBoot
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Jul 5, 2024 · Backend Development

Nine High-Performance Optimization Techniques for Large-Scale Backend Architecture

This article presents nine comprehensive performance‑optimization strategies—including load balancing, sharding, read/write separation, caching, indexing, CDN, asynchronous processing, code refinement, and algorithm improvement—aimed at boosting the efficiency and scalability of large‑scale backend systems.

Load BalancingShardingcaching
0 likes · 7 min read
Nine High-Performance Optimization Techniques for Large-Scale Backend Architecture
dbaplus Community
dbaplus Community
Jun 23, 2024 · Databases

How Vitess Scales MySQL for YouTube: Architecture and Lessons

This article explains how Vitess was created to overcome MySQL leader‑follower replication limits at YouTube, detailing its sidecar VTTablet, stateless VTGate router, topology key‑value store, and scaling strategies that enable billions of users to be served reliably.

Database ArchitectureMySQL scalingSharding
0 likes · 7 min read
How Vitess Scales MySQL for YouTube: Architecture and Lessons
Top Architect
Top Architect
Jun 19, 2024 · Databases

Choosing Between NewSQL Databases and Middleware‑Based Sharding: A Comparative Analysis

This article objectively compares NewSQL databases with middleware‑based sharding solutions, examining their architectures, distributed transaction handling, scalability, high‑availability mechanisms, storage engines, and suitability for various workloads, and provides guidance on selecting the appropriate approach based on specific system requirements.

CAP theoremNewSQLSharding
0 likes · 19 min read
Choosing Between NewSQL Databases and Middleware‑Based Sharding: A Comparative Analysis
Zhuanzhuan Tech
Zhuanzhuan Tech
Jun 12, 2024 · Databases

Designing a Scalable Persistence Layer: Sharding, Query Optimization, and Read/Write Splitting

This article shares a practical case study on how a coupon‑distribution system evolved from a single‑table design to a sharded, multi‑database architecture with query optimization, ElasticSearch integration, and read/write splitting to handle growing data volume, complex queries, and increasing request traffic.

DatabasesElasticsearchSharding
0 likes · 12 min read
Designing a Scalable Persistence Layer: Sharding, Query Optimization, and Read/Write Splitting
Lobster Programming
Lobster Programming
Jun 11, 2024 · Fundamentals

Why Consistent Hashing Matters: Solving Cache Distribution and Scaling Issues

Consistent hashing replaces simple modulo‑based distribution to efficiently locate cached data across changing numbers of servers, using a hash ring and virtual nodes to ensure balanced load, minimize data movement, and improve reliability in distributed caching, load balancing, and database sharding scenarios.

Shardingconsistent hashingdistributed caching
0 likes · 6 min read
Why Consistent Hashing Matters: Solving Cache Distribution and Scaling Issues
Top Architect
Top Architect
Jun 7, 2024 · Backend Development

Deep Dive into XXL-JOB: Time Wheel Scheduling, Consistent Hash Routing, and Sharding Implementation

This article examines the open‑source XXL-JOB distributed task scheduler, explaining its lightweight architecture, time‑wheel based scheduling, consistent‑hash routing, sharding mechanisms, and includes annotated source code snippets to illustrate key implementation details for backend developers.

Consistent HashShardingTime Wheel
0 likes · 12 min read
Deep Dive into XXL-JOB: Time Wheel Scheduling, Consistent Hash Routing, and Sharding Implementation
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
May 27, 2024 · Databases

Database Sharding (分库分表): Concepts, Strategies, and Implementation with ShardingSphere

This article explains database sharding concepts, reasons for using it, various sharding strategies, and demonstrates implementation with ShardingSphere, including configuration and Java code examples, while also offering additional resources for advanced architecture and interview preparation.

ShardingShardingSphereperformance
0 likes · 6 min read
Database Sharding (分库分表): Concepts, Strategies, and Implementation with ShardingSphere
21CTO
21CTO
May 25, 2024 · Databases

How Pinterest Scaled Its Architecture: From Clusters to Sharding

This article examines Pinterest's evolution through four development phases, the core technologies it adopted, and how it transitioned from complex clustering to a simpler sharding architecture to achieve scalable, high‑availability data storage.

ClusteringDatabase ScalingMySQL
0 likes · 11 min read
How Pinterest Scaled Its Architecture: From Clusters to Sharding
Alibaba Cloud Developer
Alibaba Cloud Developer
May 24, 2024 · Backend Development

Scalable Architecture for Community & E‑Commerce: Sharding, Caching & Distributed Transactions

This article outlines a comprehensive backend architecture for community platforms and billion‑level e‑commerce systems, covering microservice decomposition, DDD modeling, caching strategies, Redis clustering, message‑queue decoupling, database sharding, read‑write separation, distributed transaction approaches, multithreaded data migration, and massive counting techniques.

Shardingcachingdistributed transactions
0 likes · 14 min read
Scalable Architecture for Community & E‑Commerce: Sharding, Caching & Distributed Transactions
Laravel Tech Community
Laravel Tech Community
May 21, 2024 · Databases

MongoDB Replication Set and Sharding Configuration Guide

This article provides a comprehensive step‑by‑step guide to setting up MongoDB replica sets and sharded clusters, explaining the architecture, member roles, configuration files, initialization commands, and operational procedures for ensuring data redundancy, high availability, and horizontal scaling.

ClusterDatabaseHigh Availability
0 likes · 29 min read
MongoDB Replication Set and Sharding Configuration Guide
Java Architect Essentials
Java Architect Essentials
May 19, 2024 · Databases

Choosing Between NewSQL Databases and Middleware‑Based Sharding: A Comparative Analysis

This article objectively compares NewSQL distributed databases with traditional middleware‑based sharding solutions, examining their architectures, distributed transaction support, high availability, scaling, storage engines, SQL capabilities, maturity, and suitability for various workloads, to help architects decide which approach best fits their needs.

Database ArchitectureNewSQLSharding
0 likes · 20 min read
Choosing Between NewSQL Databases and Middleware‑Based Sharding: A Comparative Analysis
Java High-Performance Architecture
Java High-Performance Architecture
May 12, 2024 · Databases

NewSQL vs Sharding: Which Database Architecture Truly Wins?

This article objectively compares NewSQL databases with middleware‑based sharding solutions, examining their architectural differences, transaction handling, scalability, high‑availability mechanisms, storage engines, and maturity to help readers decide which approach best fits their specific workload and business requirements.

CAP theoremNewSQLSharding
0 likes · 18 min read
NewSQL vs Sharding: Which Database Architecture Truly Wins?
Selected Java Interview Questions
Selected Java Interview Questions
May 10, 2024 · Databases

Comparing NewSQL Databases with Middleware‑Based Sharding: Advantages, Limitations, and Practical Guidance

This article objectively compares NewSQL databases and middleware‑plus‑sharding architectures, examining their core principles, distributed transaction handling, high‑availability mechanisms, scaling and sharding strategies, SQL support, storage engines, and maturity to help engineers decide which solution fits their workload.

Database ArchitectureHigh AvailabilityNewSQL
0 likes · 18 min read
Comparing NewSQL Databases with Middleware‑Based Sharding: Advantages, Limitations, and Practical Guidance
Programmer XiaoFu
Programmer XiaoFu
May 10, 2024 · Databases

From Single Node to Tank: 20 Diagrams of Redis Architecture Evolution

This article walks through Redis's architectural journey—from a lone instance to a high‑availability, high‑performance cluster—covering persistence (RDB, AOF, hybrid), master‑slave replication, Sentinel automatic failover, sharding strategies, and the modern Redis Cluster design.

ClusterHigh AvailabilityPersistence
0 likes · 19 min read
From Single Node to Tank: 20 Diagrams of Redis Architecture Evolution
Sanyou's Java Diary
Sanyou's Java Diary
May 9, 2024 · Databases

From Single Node to Cluster: Mastering Redis Architecture Evolution

This article walks you through Redis’s architectural journey—from a simple single‑node setup, through persistence mechanisms, master‑slave replication, Sentinel‑driven automatic failover, and finally sharding with Redis Cluster—explaining each component’s purpose, trade‑offs, and how they collectively boost performance and reliability.

ClusterHigh AvailabilityPersistence
0 likes · 18 min read
From Single Node to Cluster: Mastering Redis Architecture Evolution
Java Captain
Java Captain
Apr 26, 2024 · Databases

Choosing Between Sharding Middleware and NewSQL Distributed Databases: Advantages, Trade‑offs, and Use Cases

This article objectively compares middleware‑based sharding with modern NewSQL distributed databases, examining their architectural differences, performance, transaction support, scalability, high‑availability, and operational considerations, to help practitioners decide which approach best fits their workload and organizational constraints.

Database ArchitectureHigh AvailabilityNewSQL
0 likes · 20 min read
Choosing Between Sharding Middleware and NewSQL Distributed Databases: Advantages, Trade‑offs, and Use Cases
Full-Stack Internet Architecture
Full-Stack Internet Architecture
Apr 25, 2024 · Databases

Redis Cluster: Architecture, Setup, Testing, and High Availability

This article explains Redis Cluster's sharding architecture, demonstrates how to configure multiple Redis nodes on different ports, shows commands for creating and testing the cluster, and illustrates failover behavior, highlighting its scalability and high‑availability advantages over Sentinel mode for large‑scale data workloads.

ClusterDatabaseRedis
0 likes · 11 min read
Redis Cluster: Architecture, Setup, Testing, and High Availability