BestHub
Sign in
Tagged articles

partitioning

209 articles · Page 1 of 3
Java Tech Workshop
Java Tech Workshop
Apr 28, 2026 · Backend Development

Ensuring Message Order in SpringBoot: Partitioning and Sequential Consumption

This article examines why message ordering is critical in distributed systems, explains how partition mechanisms in Kafka, RocketMQ, and RabbitMQ enable ordered consumption, and provides detailed SpringBoot implementations, best‑practice guidelines, partition‑key design principles, concurrency settings, idempotency, and real‑world case studies to ensure reliable sequential processing.

Backend DevelopmentMessage OrderingRabbitMQ
0 likes · 28 min read
Ensuring Message Order in SpringBoot: Partitioning and Sequential Consumption
Spring Full-Stack Practical Cases
Spring Full-Stack Practical Cases
Apr 3, 2026 · Databases

10 Must‑Know MySQL Advanced Techniques to Supercharge Performance

This article presents ten practical MySQL advanced techniques—including EXPLAIN analysis, composite and covering indexes, index condition push‑down, prefix indexes, window functions, CTEs, JSON handling, partition tables, join optimizations, variables, online DDL, and generated columns—each illustrated with real‑world SQL examples and execution‑plan screenshots to boost efficiency in high‑volume, high‑concurrency environments.

Advanced SQLCTEIndex Optimization
0 likes · 16 min read
10 Must‑Know MySQL Advanced Techniques to Supercharge Performance
mikechen
mikechen
Mar 12, 2026 · Big Data

How Kafka Handles Million‑Message Concurrency: Architecture Deep Dive

This article explains how Kafka’s sequential disk writes, zero‑copy data path, partition‑based parallelism, and configurable broker and partition settings enable linear‑scale throughput that can reach millions of transactions per second in large‑scale streaming systems.

ThroughputZero‑copydistributed systems
0 likes · 5 min read
How Kafka Handles Million‑Message Concurrency: Architecture Deep Dive
SpringMeng
SpringMeng
Mar 9, 2026 · Databases

10 Advanced MySQL Techniques to Supercharge Query Performance

This article walks through ten high‑level MySQL tricks—including EXPLAIN analysis, advanced indexing, window functions, CTEs, JSON handling, partitioning, join optimizations, user‑defined variables, online DDL, and generated columns—showing concrete examples, performance trade‑offs, and step‑by‑step guidance for turning slow queries into lightning‑fast operations.

CTEIndexingSQL
0 likes · 15 min read
10 Advanced MySQL Techniques to Supercharge Query Performance
Senior Xiao Ying
Senior Xiao Ying
Feb 19, 2026 · Databases

Designing MySQL Partition Strategies for Billion‑Row Tables: A Complete Guide

This article explains MySQL partition types (RANGE, LIST, HASH, KEY), demonstrates partition pruning, details maintenance commands, presents real‑world use cases such as time‑series and massive tables, and offers best‑practice recommendations for key selection, partition count, automation, and query optimization.

Large TablesSQLmysql
0 likes · 12 min read
Designing MySQL Partition Strategies for Billion‑Row Tables: A Complete Guide
Ubuntu
Ubuntu
Feb 15, 2026 · Operations

Step‑by‑Step Guide: Install Ubuntu 24.04 LTS on Real Hardware

This comprehensive tutorial walks you through preparing the boot USB, configuring BIOS/UEFI, selecting installation options—including dual‑boot, full‑disk erase, or manual partitioning—setting language, network, and drivers, and completing post‑install updates and driver checks to ensure a smooth Ubuntu 24.04 LTS setup.

BIOSDual BootGuide
0 likes · 9 min read
Step‑by‑Step Guide: Install Ubuntu 24.04 LTS on Real Hardware
Tech Musings
Tech Musings
Feb 12, 2026 · Databases

From MySQL to Apache Doris: Key Design Shifts for OLAP Migration

This article explains how backend engineers should rethink table design, indexing, partitioning, and key strategies when migrating attendance data from MySQL's OLTP model to Apache Doris 2.1.7's OLAP architecture, providing concrete DDL examples and practical tips.

Apache DorisIndexingOLAP
0 likes · 12 min read
From MySQL to Apache Doris: Key Design Shifts for OLAP Migration
Su San Talks Tech
Su San Talks Tech
Feb 10, 2026 · Databases

10 Must‑Know MySQL Advanced Techniques to Supercharge Performance

This article presents eleven practical MySQL advanced techniques—including EXPLAIN analysis, sophisticated indexing, window functions, CTEs, JSON handling, partitioning, join optimizations, user variables, online DDL, and generated columns—to help developers dramatically improve query performance and scalability in high‑load environments.

Advanced QueriesIndexingWindow Functions
0 likes · 18 min read
10 Must‑Know MySQL Advanced Techniques to Supercharge Performance
Architect Chen
Architect Chen
Feb 6, 2026 · Backend Development

How Kafka Achieves Billion‑Message Throughput with Sequential I/O, Zero‑Copy, and Partitioning

This article explains how Kafka leverages sequential disk I/O, zero‑copy data transfer, OS page cache, and partition‑based parallelism to deliver massive throughput, detailing the underlying mechanisms, performance numbers, and a practical formula for estimating total system capacity.

Backend PerformanceHigh concurrencySequential I/O
0 likes · 5 min read
How Kafka Achieves Billion‑Message Throughput with Sequential I/O, Zero‑Copy, and Partitioning
DevOps Coach
DevOps Coach
Jan 31, 2026 · Databases

7 Overlooked PostgreSQL Architecture Mistakes That Kill Performance

The article reveals seven common PostgreSQL architectural oversights—such as neglecting vacuum, misusing UUID primary keys, treating the database as a queue, missing indexes, over‑relying on ORMs, ignoring write‑side scaling, and failing to partition large tables—that silently degrade performance and reliability, and provides concrete fixes and best‑practice configurations.

IndexingORMPostgreSQL
0 likes · 7 min read
7 Overlooked PostgreSQL Architecture Mistakes That Kill Performance
IT Services Circle
IT Services Circle
Dec 29, 2025 · Backend Development

Mastering Sharding: When, How, and How Much to Split Your Database

This guide walks senior backend engineers through the strategic reasoning, capacity estimation, and step‑by‑step migration techniques for sharding databases, covering when to split, choosing between partitioning, read/write splitting, or full sharding, and how to plan safe expansions.

Data MigrationRead‑Write Splittingcapacity planning
0 likes · 19 min read
Mastering Sharding: When, How, and How Much to Split Your Database
Big Data Tech Team
Big Data Tech Team
Dec 28, 2025 · Big Data

When to Use Hive Partitioning vs Bucketing: A Practical Guide

This article explains Hive's partitioning and bucketing techniques, compares their purposes, advantages, and pitfalls, and shows how to combine them with concrete SQL examples to improve query performance, reduce I/O, and optimize joins and sampling in large data warehouses.

BucketingData WarehouseHive
0 likes · 7 min read
When to Use Hive Partitioning vs Bucketing: A Practical Guide
DevOps Coach
DevOps Coach
Dec 26, 2025 · Databases

Achieving Sub‑Second Queries on 1.2 B‑Row PostgreSQL Using BRIN, pg_cron & Query Folding

The article recounts how a PostgreSQL instance on a modest 2‑CPU, 4 GB VM handling 1.2 billion rows was dramatically accelerated by adding BRIN indexes, scheduling maintenance with pg_cron, applying query folding and tuning memory and parallel settings, achieving sub‑second query times without additional hardware.

BRIN IndexPerformance TuningPostgreSQL
0 likes · 9 min read
Achieving Sub‑Second Queries on 1.2 B‑Row PostgreSQL Using BRIN, pg_cron & Query Folding
Raymond Ops
Raymond Ops
Dec 17, 2025 · Operations

Zero‑to‑Hero Linux Installation: Mastering Partition Strategies

This guide walks you through why proper Linux partitioning matters, presents production‑grade and scenario‑specific layouts, covers hardware checks, BIOS/UEFI settings, step‑by‑step partitioning with fdisk/GPT and LVM, filesystem choices, security hardening, automation scripts, monitoring, and best‑practice tips for a flawless system install.

AutomationFilesystemInstallation
0 likes · 16 min read
Zero‑to‑Hero Linux Installation: Mastering Partition Strategies
Senior Brother's Insights
Senior Brother's Insights
Dec 3, 2025 · Databases

Can MySQL Partition Tables Work When created_at Is NULL? Understanding Default Timestamp Partitioning

This article explains why MySQL partitions based on a timestamp column still function correctly even when the application does not explicitly set the column, by detailing the default CURRENT_TIMESTAMP behavior, partition key evaluation, and practical examples with table definitions and insert statements.

DEFAULT CURRENT_TIMESTAMPSQLmysql
0 likes · 6 min read
Can MySQL Partition Tables Work When created_at Is NULL? Understanding Default Timestamp Partitioning
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Dec 3, 2025 · Big Data

Kafka High‑Throughput Tricks: Sequential Writes, Zero‑Copy, Partitioning

The article explains how Kafka achieves high throughput by writing messages sequentially to disk, leveraging OS page cache and zero‑copy system calls, using partitioned topics for parallelism, batching and compressing records on both producer and broker sides, and employing asynchronous replication with configurable persistence strategies.

BatchingZero‑copycompression
0 likes · 4 min read
Kafka High‑Throughput Tricks: Sequential Writes, Zero‑Copy, Partitioning
Senior Brother's Insights
Senior Brother's Insights
Oct 29, 2025 · Databases

Sharding vs Partitioning: When and How to Scale Your Database

This article explains the concepts, differences, advantages, and disadvantages of database sharding and partitioning, provides practical MySQL examples for horizontal sharding by region and range partitioning, and offers guidance on choosing the right approach for scalability and maintainability.

Data DistributionShardingdatabase scaling
0 likes · 10 min read
Sharding vs Partitioning: When and How to Scale Your Database
Su San Talks Tech
Su San Talks Tech
Oct 29, 2025 · Databases

How to Safely Delete Millions of Rows from a Large Table: Proven Strategies & Code

This article explains why deleting tens of millions of rows is challenging, analyzes transaction, lock, resource, and business impacts, and presents five practical solutions—including batch deletion, table rebuild, partition dropping, online migration, and professional tools—along with best‑practice guidelines and a decision matrix.

Batch Processingdata deletiondatabase optimization
0 likes · 28 min read
How to Safely Delete Millions of Rows from a Large Table: Proven Strategies & Code
Senior Brother's Insights
Senior Brother's Insights
Oct 15, 2025 · Databases

Master MySQL Partitioning: When, How, and Why It Beats Sharding

This guide explains MySQL partitioning fundamentals, shows how to create RANGE, LIST, and KEY partitions with concrete CREATE TABLE examples, compares the performance benefits and maintenance trade‑offs, and clarifies the differences between partitioning and sharding to help you decide when to use each technique.

KEYListSharding
0 likes · 7 min read
Master MySQL Partitioning: When, How, and Why It Beats Sharding
Liangxu Linux
Liangxu Linux
Oct 7, 2025 · Operations

Master Linux Installation: Zero‑Error Partitioning Strategies

This guide walks you through why careful partition planning is critical, presents production, desktop, development, and cloud server layouts, details hardware checks, BIOS settings, command‑line and LVM partitioning steps, compares filesystems, and offers security hardening, automation scripts, and monitoring tips for a flawless Linux installation.

FilesystemLVMLinux
0 likes · 15 min read
Master Linux Installation: Zero‑Error Partitioning Strategies
IT Services Circle
IT Services Circle
Oct 4, 2025 · Backend Development

How to Solve Message Queue Backlog in High‑Traffic Scenarios: Interview‑Ready Strategies

This article explains why consumer instances cannot scale indefinitely, how to plan partition numbers, fast‑track solutions for message backlog, consumer‑side performance tweaks, and an advanced asynchronous consumption model, providing a complete, interview‑friendly framework for handling MQ congestion.

Message Queueasynchronous consumptionconsumer backlog
0 likes · 22 min read
How to Solve Message Queue Backlog in High‑Traffic Scenarios: Interview‑Ready Strategies
Su San Talks Tech
Su San Talks Tech
Sep 16, 2025 · Backend Development

Mastering Message Order in Distributed Queues: From Basics to Advanced Strategies

This article explores the fundamentals of message ordering in distributed message queues, explains why ordering is determined by broker arrival, compares global and partial ordering, and presents practical solutions—from single-partition designs to multi-partition hashing, handling data skew, and safe expansion—plus interview tips.

Consistent HashingOrderingdistributed systems
0 likes · 24 min read
Mastering Message Order in Distributed Queues: From Basics to Advanced Strategies
Alibaba Cloud Big Data AI Platform
Alibaba Cloud Big Data AI Platform
Aug 15, 2025 · Big Data

How MaxCompute Extends SQL to Match BigQuery – New Features & Compatibility

This article details the challenges of migrating 100,000+ SQL statements from BigQuery to MaxCompute for a leading Southeast Asian tech group, explains the new MaxCompute SQL syntax, auto‑partition and ingestion‑time tables, enhanced built‑in functions, and the BigQuery compatibility mode that ensures seamless query behavior.

BigQueryData MigrationMaxCompute
0 likes · 15 min read
How MaxCompute Extends SQL to Match BigQuery – New Features & Compatibility
Raymond Ops
Raymond Ops
Aug 14, 2025 · Operations

Master Linux Storage: Device Discovery, Mounting, Partitioning & Quotas

This guide covers essential Linux storage management techniques, including how to list devices, mount and unmount filesystems, use the find command for file searches, create and manage disk partitions, configure swap space, and set up disk quotas for user limits.

Disk QuotaStorage Managementmounting
0 likes · 10 min read
Master Linux Storage: Device Discovery, Mounting, Partitioning & Quotas
Spring Full-Stack Practical Cases
Spring Full-Stack Practical Cases
Aug 4, 2025 · Backend Development

Master Spring Batch Partitioning in Spring Boot 3 to Process Millions of Records Efficiently

This article demonstrates how to use Spring Batch partitioning in Spring Boot 3, covering the architecture of manager and worker steps, custom partitioner implementation, job and step configuration, essential beans, and a complete runnable example that processes millions of records with parallel threads.

Spring Batchdata processingpartitioning
0 likes · 10 min read
Master Spring Batch Partitioning in Spring Boot 3 to Process Millions of Records Efficiently
Tencent Cloud Middleware
Tencent Cloud Middleware
Aug 1, 2025 · Backend Development

Mastering TDMQ CKafka Production: Configurations, Partitioning, and Reliable Messaging

This guide explains how to efficiently produce messages with TDMQ CKafka, covering topic creation, partition sizing, retry policies, asynchronous sending, key/value handling, batch tuning, sticky partitioning, idempotence, ACK settings, code examples, and strategies to avoid data skew and ensure high‑throughput, low‑latency messaging.

BatchingCKafkaIdempotence
0 likes · 18 min read
Mastering TDMQ CKafka Production: Configurations, Partitioning, and Reliable Messaging
MaGe Linux Operations
MaGe Linux Operations
Jul 31, 2025 · Operations

Zero‑Failure Linux Installation: Master Partitioning Strategies from a Veteran Ops Engineer

Drawing on a decade of operations experience, this guide walks you through zero‑incident Linux system installation and optimal partitioning, covering hardware checks, BIOS settings, multiple partition schemes for servers, desktops, cloud, LVM automation, filesystem choices, security hardening, performance tuning, troubleshooting, and post‑install monitoring.

LVMSystem Installationpartitioning
0 likes · 17 min read
Zero‑Failure Linux Installation: Master Partitioning Strategies from a Veteran Ops Engineer
Tech Freedom Circle
Tech Freedom Circle
Jul 15, 2025 · Databases

One‑Order Four‑Split: Solving MySQL Deadlocks in Million‑Row Imports

During massive daily imports of up to one million reconciliation records, MySQL deadlocks can cripple performance; this article dissects the four classic deadlock conditions, then presents a systematic “One‑Order Four‑Split” strategy—ordered writes, transaction splitting, index management, partitioning, and gap‑lock removal—backed by benchmarks, code samples, and configuration tweaks that cut lock rates by up to 90% and reduce batch times to under ten minutes.

Batch InsertDeadlockIndex Optimization
0 likes · 27 min read
One‑Order Four‑Split: Solving MySQL Deadlocks in Million‑Row Imports
Selected Java Interview Questions
Selected Java Interview Questions
Jul 8, 2025 · Databases

How to Scale 1 Billion Orders with MySQL Sharding: Strategies, Gene IDs, and Migration

This article explores the performance bottlenecks of a 700‑million‑row MySQL orders table and presents a comprehensive sharding solution—including vertical and horizontal partitioning, gene‑based shard keys, routing logic, global secondary indexes, data migration strategies, and common pitfalls—demonstrating how query latency can drop from seconds to milliseconds while supporting billions of records.

Data MigrationShardingdatabase scaling
0 likes · 9 min read
How to Scale 1 Billion Orders with MySQL Sharding: Strategies, Gene IDs, and Migration
Senior Tony
Senior Tony
Jun 8, 2025 · Backend Development

Why Kafka Handles Millions of Messages Per Second: Batch, Partition, Zero‑Copy, and Compression Explained

This article breaks down the core techniques that give Kafka its high‑throughput capability, including producer batch settings (batch.size, linger.ms), broker append‑only writes, consumer poll configuration, partition distribution, zero‑copy data transfer, dual‑thread processing, and configurable compression algorithms.

Batch ProcessingZero‑copycompression
0 likes · 8 min read
Why Kafka Handles Millions of Messages Per Second: Batch, Partition, Zero‑Copy, and Compression Explained
Java Tech Enthusiast
Java Tech Enthusiast
May 10, 2025 · Databases

Efficient Strategies for Deleting Massive Data from Large Database Tables

The article outlines why deleting tens of millions of rows in one statement harms performance and reliability, then presents safe, efficient tactics—such as indexed batch deletes, partition drops, creating a new table, or using TRUNCATE—plus pre‑deletion planning, post‑delete validation, monitoring, and space reclamation.

SQLbackupbulk delete
0 likes · 11 min read
Efficient Strategies for Deleting Massive Data from Large Database Tables
IT Services Circle
IT Services Circle
May 6, 2025 · Databases

Efficient Strategies for Deleting Massive Data from Large Tables

This article explains the risks of bulk deletions on million‑row tables and provides a step‑by‑step guide—including pre‑deletion planning, indexing, backup, batch deletion, partitioning, table recreation, TRUNCATE, and post‑deletion verification—to safely and quickly remove large volumes of data while minimizing lock contention, log growth, CPU spikes, replication lag, and rollback complications.

Batch DeleteDatabasesSQL
0 likes · 11 min read
Efficient Strategies for Deleting Massive Data from Large Tables
Programmer XiaoFu
Programmer XiaoFu
Mar 5, 2025 · Databases

From 12 s to 200 ms: 6 MySQL Deep‑Pagination Tricks for 20 M Orders

When a MySQL order table with 20 million rows is paginated using a large OFFSET, query latency can explode from seconds to minutes, but applying six concrete techniques—cursor pagination, deferred join, covering index, table partitioning, precomputed pages, and Elasticsearch integration—can shrink response time to a few hundred milliseconds.

Covering IndexCursor PaginationElasticsearch
0 likes · 9 min read
From 12 s to 200 ms: 6 MySQL Deep‑Pagination Tricks for 20 M Orders
Liangxu Linux
Liangxu Linux
Feb 7, 2025 · Fundamentals

Master Linux Disk Management: Devices, Partitions, RAID & LVM

This guide explains Linux disk management fundamentals, covering device files, disk types, RAID levels, partitioning methods, filesystem choices, mounting techniques, useful command‑line tools, real‑world case studies, and step‑by‑step LVM creation and expansion.

FilesystemLVMLinux
0 likes · 30 min read
Master Linux Disk Management: Devices, Partitions, RAID & LVM
MaGe Linux Operations
MaGe Linux Operations
Feb 4, 2025 · Operations

Master Linux Storage Management: Devices, Mounting, Partitioning & Quotas

This guide walks through Linux storage fundamentals—including device identification, viewing, mounting (temporary and permanent), file searching, disk partitioning methods, swap setup, device removal, and disk quota configuration—providing practical commands and best‑practice tips for system administrators.

Disk QuotaStorage Managementmounting
0 likes · 10 min read
Master Linux Storage Management: Devices, Mounting, Partitioning & Quotas
BirdNest Tech Talk
BirdNest Tech Talk
Dec 29, 2024 · Fundamentals

Unlocking Distributed System Design: 20 Core Patterns Explained

This article distills the key design patterns behind distributed systems—covering replication, partitioning, consensus, and fault‑tolerance—by presenting each pattern’s problem statement, concrete solution, trade‑offs, and technical considerations, all illustrated with real‑world examples from projects like Kafka and Cassandra.

Consensusdesign patternsdistributed systems
0 likes · 18 min read
Unlocking Distributed System Design: 20 Core Patterns Explained
IT Services Circle
IT Services Circle
Oct 1, 2024 · Databases

Effectiveness of Adding an Index on a Status Column in a Tens‑of‑Millions Row MySQL Table

This article explains how adding an index to a status column in a tens‑of‑millions‑row MySQL table affects query performance, covering the basic index lookup process, extreme cases where full scans are chosen, selectivity, covering indexes, composite indexes, partitioning, and using EXPLAIN to verify execution plans.

EXPLAINQuery OptimizationSQL
0 likes · 9 min read
Effectiveness of Adding an Index on a Status Column in a Tens‑of‑Millions Row MySQL Table
ITPUB
ITPUB
Sep 23, 2024 · Databases

Master PostgreSQL Table Partitioning: Types, Commands, and Best Practices

This guide explains PostgreSQL table partitioning, covering its definition, benefits, drawbacks, the three partition types (RANGE, LIST, HASH), step‑by‑step SQL commands for creating partitions, indexes, and managing them, and concludes with practical recommendations for effective use.

HashListPostgreSQL
0 likes · 10 min read
Master PostgreSQL Table Partitioning: Types, Commands, and Best Practices
Architecture Digest
Architecture Digest
Sep 23, 2024 · Databases

Evaluating and Optimizing Large MySQL Tables: Partitioning, Sharding, and Hot‑Cold Archiving Strategies

This article explains how to assess MySQL table size using table capacity, disk usage, and instance resources, describes why large tables degrade performance, and presents three practical solutions—partitioning, sharding (horizontal/vertical), and hot‑cold data archiving—along with their trade‑offs and implementation tips.

B+TreeHotColdArchivingSharding
0 likes · 15 min read
Evaluating and Optimizing Large MySQL Tables: Partitioning, Sharding, and Hot‑Cold Archiving Strategies
ITPUB
ITPUB
May 5, 2024 · Databases

What GitLab’s 34k‑Line Postgres Schema Reveals About Scalable Database Design

This article examines GitLab’s extensive PostgreSQL schema, explaining why the project uses a pure‑SQL structure file, how primary‑key choices, internal versus external IDs, naming conventions, timestamps, text types, partitioning, GIN indexes, JSONB columns, foreign‑key strategies and other data‑type decisions impact performance, maintainability, and scalability.

IndexesJSONBPostgreSQL
0 likes · 25 min read
What GitLab’s 34k‑Line Postgres Schema Reveals About Scalable Database Design
Alibaba Cloud Developer
Alibaba Cloud Developer
Apr 17, 2024 · Databases

Why MySQL’s TDDL Optimizer Picks the Wrong Index and How to Fix It

This article investigates a slow‑SQL incident caused by the TDDL optimizer reordering a MySQL query, analyzes why the optimizer chose an inefficient index, demonstrates how to verify and force the correct index, and proposes better index and partition‑key designs for optimal performance.

Index OptimizationTDDLmysql
0 likes · 11 min read
Why MySQL’s TDDL Optimizer Picks the Wrong Index and How to Fix It
Liangxu Linux
Liangxu Linux
Apr 4, 2024 · Operations

Master Linux Disk Management: Partition, Format, and Mount Like a Pro

This guide walks you through Linux disk management fundamentals, covering how to partition disks with fdisk and gdisk, create filesystems using mkfs, format partitions, mount and unmount devices, configure automatic mounts via /etc/fstab, and handle special cases such as ISO and Windows share mounts.

Linuxdisk managementformatting
0 likes · 38 min read
Master Linux Disk Management: Partition, Format, and Mount Like a Pro
MaGe Linux Operations
MaGe Linux Operations
Apr 2, 2024 · Fundamentals

Understanding Mechanical Hard Drives: Structure, Partitioning, and Linux Management

This guide explains the physical structure of mechanical hard drives, how data is organized into sectors, tracks, and cylinders, compares old and new zoning methods, and provides detailed Linux commands for calculating capacity, creating partitions with MBR or GPT, formatting filesystems, and managing mounts and swap space.

Linuxhard drivepartitioning
0 likes · 51 min read
Understanding Mechanical Hard Drives: Structure, Partitioning, and Linux Management
Architect
Architect
Mar 29, 2024 · Databases

Understanding Database Sharding and Partitioning with ShardingSphere

This article explains the fundamentals of database sharding and partitioning, why they are needed for massive data volumes, various vertical and horizontal splitting strategies, routing algorithms, client vs. proxy deployment models, and the practical challenges and solutions when using ShardingSphere 5.x.

ShardingSpheredatabase shardinghorizontal-splitting
0 likes · 17 min read
Understanding Database Sharding and Partitioning with ShardingSphere
StarRocks
StarRocks
Mar 13, 2024 · Databases

Master StarRocks: Simplify Partitioning, Data Import, and Table Optimization

This guide walks you through using StarRocks—covering effortless expression‑based partitioning, streamlined data loading with INSERT FROM FILES and PIPE, powerful in‑flight data transformation using SELECT/JOIN/UNNEST, and flexible table structure tweaks via ALTER TABLE to boost query performance.

PipeSQLStarRocks
0 likes · 16 min read
Master StarRocks: Simplify Partitioning, Data Import, and Table Optimization
Open Source Linux
Open Source Linux
Mar 12, 2024 · Operations

Master Linux Disk Management: Partition, Format, and Mount Like a Pro

This comprehensive guide explains Linux disk management fundamentals, covering how to partition disks with fdisk and gdisk, format them using mkfs, mount and unmount filesystems, configure automatic mounts via /etc/fstab, and handle special cases such as ISO images and Windows shares.

Linuxdisk managementmounting
0 likes · 40 min read
Master Linux Disk Management: Partition, Format, and Mount Like a Pro
Su San Talks Tech
Su San Talks Tech
Feb 21, 2024 · Databases

Mastering MySQL Sharding: Strategies, Pitfalls, and Best Practices

This comprehensive guide explains why MySQL sharding (分库分表) is essential for handling massive data growth, outlines various horizontal and vertical partitioning strategies, discusses common challenges such as hot‑spot, cross‑database queries, pagination, and transaction handling, and compares popular middleware solutions.

database scalingmysqlpartitioning
0 likes · 28 min read
Mastering MySQL Sharding: Strategies, Pitfalls, and Best Practices
Architect's Guide
Architect's Guide
Feb 4, 2024 · Databases

Evaluating Large MySQL Table Size and Optimizing Query Performance with Partitioning, Sharding, and Archiving

This article explains how to assess MySQL table size through table capacity, disk usage, and instance resources, illustrates the impact of B+‑tree index depth on query speed, and presents three practical solutions—table partitioning, database sharding, and hot‑cold data archiving—to improve performance.

Data ArchivingShardingdatabase optimization
0 likes · 15 min read
Evaluating Large MySQL Table Size and Optimizing Query Performance with Partitioning, Sharding, and Archiving
Aikesheng Open Source Community
Aikesheng Open Source Community
Oct 25, 2023 · Databases

Understanding and Optimizing Indexes in OceanBase

This article explains the concepts of local and global indexes in OceanBase, demonstrates how to create tables and indexes, compares six practical scenarios—including no index, local index, partitioned queries, unique and global indexes—and provides performance metrics to guide database optimization.

IndexingLocal IndexOceanBase
0 likes · 8 min read
Understanding and Optimizing Indexes in OceanBase
Weimob Technology Center
Weimob Technology Center
Oct 13, 2023 · Big Data

Optimizing StarRocks Tables: Design Tips, Real‑World Cases and Monitoring Strategies

This article explains how to design efficient StarRocks tables with proper field types, partitioning and bucketing, compares update and primary‑key models, presents real‑world cases of memory and tablet issues, provides a complete table‑creation example, and outlines comprehensive monitoring metrics to keep the analytical data warehouse performant and stable.

AnalyticsPrimary KeyStarRocks
0 likes · 25 min read
Optimizing StarRocks Tables: Design Tips, Real‑World Cases and Monitoring Strategies
ITPUB
ITPUB
Oct 9, 2023 · Databases

Boost MySQL Performance on Large Tables: Proven Strategies and Tools

This article examines why MySQL can still handle massive tables and presents practical techniques—including proper primary keys, eliminating redundant indexes, choosing optimal data types, compression, archiving, partitioning, sharding, query optimization, and safe schema changes—to keep large‑table performance high.

Indexingcompressionmysql
0 likes · 16 min read
Boost MySQL Performance on Large Tables: Proven Strategies and Tools
Big Data Technology & Architecture
Big Data Technology & Architecture
Oct 7, 2023 · Big Data

Comprehensive Guide to OLAP Optimization and ClickHouse Performance Tuning

This article explains how to optimize OLAP workloads by balancing normalization and denormalization, applying data sharding, replication, indexing, partitioning, materialized views, columnar storage, compression, and lifecycle management, and provides practical ClickHouse SQL examples for index creation, partitioning, and query plan analysis.

ClickHouseIndexingOLAP
0 likes · 15 min read
Comprehensive Guide to OLAP Optimization and ClickHouse Performance Tuning
dbaplus Community
dbaplus Community
Sep 10, 2023 · Big Data

Master Data Warehouse Architecture: Layers, Naming Rules, and Lifecycle Tips

This guide outlines a comprehensive data model architecture for data warehouses, detailing layer definitions (ODS, CDM, DWD, DWS, ADS), naming conventions, data type standards, partitioning strategies, redundancy rules, and lifecycle management policies to ensure consistency, performance, and maintainability across big‑data environments.

Data WarehouseNaming Conventionspartitioning
0 likes · 28 min read
Master Data Warehouse Architecture: Layers, Naming Rules, and Lifecycle Tips
Sanyou's Java Diary
Sanyou's Java Diary
Jun 26, 2023 · Big Data

Master Kafka Interview Questions: Architecture, Partitioning, and Reliability Explained

This article provides a comprehensive overview of Kafka, covering its core architecture, message queue models, communication process, partition selection, consumer groups, rebalancing strategies, partition assignment algorithms, reliability guarantees, replica synchronization, and reasons for removing Zookeeper in newer versions.

Consumer GroupReliabilityStreaming
0 likes · 20 min read
Master Kafka Interview Questions: Architecture, Partitioning, and Reliability Explained
ITPUB
ITPUB
May 21, 2023 · Databases

Mastering Database Sharding: When, How, and Pitfalls of Partitioning

This article explains the concept, motivations, design methods, common patterns, and trade‑offs of database sharding (both table and database level), offering practical code examples, scalability criteria, and guidance on avoiding over‑engineering while considering alternatives like TiDB.

Hash ShardingRange ShardingTiDB
0 likes · 19 min read
Mastering Database Sharding: When, How, and Pitfalls of Partitioning
Architect
Architect
Jan 18, 2023 · Databases

Design and Architecture of Bilibili's High‑Performance KV Storage System

This article presents the background, overall architecture, partitioning strategies, raft‑based replication, binlog support, multi‑active deployment, bulk‑load mechanisms, storage‑engine optimizations, load‑balancing policies, and failure‑detection & recovery techniques of a high‑reliability, high‑throughput key‑value store used at Bilibili.

KV storageRaftbulk load
0 likes · 22 min read
Design and Architecture of Bilibili's High‑Performance KV Storage System
Architecture Digest
Architecture Digest
Jan 2, 2023 · Databases

Database Sharding and Partitioning Strategy for High‑Volume Order Systems

This article explains how to classify massive order data into hot and cold segments, store them in MySQL, Elasticsearch and Hive respectively, and implement sharding and partitioning at both table and database levels using modulo and hash calculations to achieve scalable performance for billions of orders.

Hivearchitecturepartitioning
0 likes · 8 min read
Database Sharding and Partitioning Strategy for High‑Volume Order Systems
dbaplus Community
dbaplus Community
Nov 18, 2022 · Databases

When and How to Shard Databases: A Complete Guide to Partitioning, Algorithms, and Tools

This article explains why and when to apply database sharding, distinguishes between sharding databases, tables, or both, describes horizontal and vertical splitting, guides key selection and sharding algorithms, covers global ID generation, lists open‑source tools, and discusses the operational challenges introduced by sharding.

ID Generationhorizontal-splittingopen-source
0 likes · 17 min read
When and How to Shard Databases: A Complete Guide to Partitioning, Algorithms, and Tools
Architecture Digest
Architecture Digest
Oct 30, 2022 · Backend Development

High‑Availability Architecture for a Large‑Scale Membership System

This article details the design and implementation of a high‑availability, high‑performance membership system that serves billions of users across multiple platforms, covering Elasticsearch dual‑center clusters, traffic‑isolated three‑cluster setups, Redis caching strategies, MySQL dual‑center partitioning, and advanced flow‑control and degradation mechanisms.

ElasticsearchHigh Availabilitypartitioning
0 likes · 18 min read
High‑Availability Architecture for a Large‑Scale Membership System
JavaEdge
JavaEdge
Aug 28, 2022 · Databases

Why Serializability Matters: Single‑Threaded Transactions and Stored Procedures

The article explains how RC and snapshot isolation fail to prevent write skew and phantom reads, advocates serializable isolation, and examines three implementation techniques—strict serial execution, two‑phase locking, and optimistic concurrency—detailing their trade‑offs, stored‑procedure usage, partitioning, and practical constraints for achieving true serializable isolation.

DatabasesStored ProceduresTransaction Isolation
0 likes · 11 min read
Why Serializability Matters: Single‑Threaded Transactions and Stored Procedures
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Aug 28, 2022 · Backend Development

Why Is Kafka So Fast? Uncover the 4 Core Performance Secrets

This article explains the four key techniques—page‑cache usage, sequential disk writes, zero‑copy transfers, and partitioned segment indexing—that enable Kafka to achieve exceptionally high write performance, detailing how each mechanism reduces latency and maximizes throughput.

Backend DevelopmentSequential WriteZero‑copy
0 likes · 5 min read
Why Is Kafka So Fast? Uncover the 4 Core Performance Secrets
Wukong Talks Architecture
Wukong Talks Architecture
Aug 9, 2022 · Big Data

Kafka Basics: 15 Key Questions and In‑Depth Answers

This comprehensive guide covers Kafka’s core concepts, architecture, Zookeeper role, producer sending modes, partitioning strategies, replica types, message deletion, performance optimizations, consumer models, offset management, and best‑practice recommendations for scaling and ensuring ordered delivery in distributed streaming systems.

StreamingZookeeperbig-data
0 likes · 31 min read
Kafka Basics: 15 Key Questions and In‑Depth Answers
IT Architects Alliance
IT Architects Alliance
Aug 3, 2022 · Big Data

Kafka Producer: Overview, Detailed Process, Key Challenges, and Configuration

This article provides a comprehensive English guide to Kafka's producer component, covering its end‑to‑end workflow, detailed steps such as record creation, serialization, partitioning, batching, sender thread, custom data structures, send modes, common exceptions, and essential configuration parameters for optimal performance.

Avropartitioningserialization
0 likes · 16 min read
Kafka Producer: Overview, Detailed Process, Key Challenges, and Configuration
Top Architect
Top Architect
Jul 27, 2022 · Big Data

Understanding Kafka Architecture: Topics, Partitions, Replication, Zero‑Copy, and Zookeeper Integration

This article explains Kafka's core architecture, covering the logical concept of topics, physical partitioning and replication, leader‑follower mechanics, consumer groups, log segmentation with zero‑copy I/O, and how Zookeeper manages broker registration, topic metadata, consumer coordination, and offset tracking, while also discussing producer and consumer load‑balancing strategies.

distributed-systemskafkapartitioning
0 likes · 10 min read
Understanding Kafka Architecture: Topics, Partitions, Replication, Zero‑Copy, and Zookeeper Integration
IT Architects Alliance
IT Architects Alliance
Jul 21, 2022 · Big Data

Explanation of Kafka Components and Architecture

This article provides a comprehensive overview of Kafka’s core components—including brokers, topics, partitions, producers, and consumers—explaining their roles, log structures, replication mechanisms, and the system’s architecture, supplemented with practical visual illustrative diagrams.

distributed systemsmessagingpartitioning
0 likes · 8 min read
Explanation of Kafka Components and Architecture
JavaEdge
JavaEdge
Jul 10, 2022 · Databases

How to Partition Massive Datasets: Strategies, Trade‑offs, and Hotspot Prevention

This article explains why large‑scale systems need data partitioning beyond replication, defines partitions and sharding, compares key‑range and hash‑based partitioning methods, discusses load‑balancing and hotspot avoidance, and shows how replication works together with partitioning to improve scalability.

HashingShardingdatabase
0 likes · 12 min read
How to Partition Massive Datasets: Strategies, Trade‑offs, and Hotspot Prevention
Shepherd Advanced Notes
Shepherd Advanced Notes
Jul 6, 2022 · Databases

Understanding Apache Doris: Real‑Time Analytical Database Architecture and Data Modeling

This article introduces Apache Doris, a high‑performance, real‑time analytical database built on an MPP architecture, covering its simple FE/BE design, three data‑model types (Aggregate, Unique, Duplicate), partitioning and bucketing strategies, rollup tables, and the limitations and best practices of row‑level updates.

Apache DorisMPPdata modeling
0 likes · 28 min read
Understanding Apache Doris: Real‑Time Analytical Database Architecture and Data Modeling
Aikesheng Open Source Community
Aikesheng Open Source Community
Jun 29, 2022 · Databases

Understanding Multi-Column Partition Tables in MySQL: Implementation, Performance Comparison, and Best Practices

This article explains MySQL's support for multi‑column partition tables, provides a stored‑procedure to create them, compares performance against single‑column partitions under various query conditions, discusses partition column order impact, and offers practical guidance for when and how to use multi‑column partitions.

Multi-Columndatabasemysql
0 likes · 10 min read
Understanding Multi-Column Partition Tables in MySQL: Implementation, Performance Comparison, and Best Practices
Architecture & Thinking
Architecture & Thinking
Jun 25, 2022 · Databases

Why and How to Split Databases: Vertical & Horizontal Sharding Explained

This article explains the reasons for database sharding, compares vertical (scale‑up) and horizontal (scale‑out) partitioning methods, illustrates vertical database/table splits and horizontal partition strategies such as HASH, RANGE, LIST, KEY and composite, and discusses the challenges like distributed transactions, cross‑database joins and increased management overhead.

Horizontal PartitioningVertical Partitioningdatabase sharding
0 likes · 14 min read
Why and How to Split Databases: Vertical & Horizontal Sharding Explained
Big Data Technology & Architecture
Big Data Technology & Architecture
May 24, 2022 · Databases

Apache Doris Basics: Creating Databases, Tables, Partitioning, Data Import, and Rollup

This article provides a comprehensive guide to Apache Doris, covering how to create databases and tables with single and composite partitions, import data via broker and routine loads, understand its aggregate, uniq, and duplicate data models, and leverage rollup and prefix index features for optimized querying.

Apache DorisSQLdata modeling
0 likes · 20 min read
Apache Doris Basics: Creating Databases, Tables, Partitioning, Data Import, and Rollup
Code Ape Tech Column
Code Ape Tech Column
Apr 23, 2022 · Big Data

Kafka Fundamentals: Architecture, Replication, Partitioning, and Performance

This article provides a comprehensive overview of Kafka, covering its role as a message middleware, core concepts, architecture, replication management, partition handling, producer sending modes, partition assignment strategies, load balancing, reliability mechanisms, consumer models, controller election, and factors that affect its high throughput and potential message loss scenarios.

Message Queuedistributed systemskafka
0 likes · 29 min read
Kafka Fundamentals: Architecture, Replication, Partitioning, and Performance
dbaplus Community
dbaplus Community
Apr 10, 2022 · Databases

Designing a High‑Performance Distributed KV Store for B‑Station

This article details the background, architecture, core features, and operational practices of a custom high‑reliability, high‑throughput key‑value storage system that combines Raft replication, flexible partitioning, binlog support, bulk loading, and multi‑active deployment for B‑Station's diverse data workloads.

BinlogFailure RecoveryRaft replication
0 likes · 22 min read
Designing a High‑Performance Distributed KV Store for B‑Station
Big Data Technology & Architecture
Big Data Technology & Architecture
Mar 23, 2022 · Databases

ClickHouse SQL Fundamentals: CREATE, TABLE, Views, ALTER, Partitioning, Import/Export, and Mutation Operations

This article provides a comprehensive guide to ClickHouse SQL, covering database creation, table definitions, column defaults, temporary and partitioned tables, view types, DDL commands, data import/export formats, and mutation‑based update and delete operations with practical code examples.

ClickHouseMutationSQL
0 likes · 17 min read
ClickHouse SQL Fundamentals: CREATE, TABLE, Views, ALTER, Partitioning, Import/Export, and Mutation Operations
Bilibili Tech
Bilibili Tech
Mar 11, 2022 · Databases

Design and Architecture of Bilibili's High‑Performance Distributed KV Store

Bilibili’s high‑performance distributed KV store combines hash and range partitioning, Raft‑based multi‑replica consistency, and a Metaserver‑managed topology of pools, zones, nodes, tables, shards and replicas, offering features such as partition splitting, binlog streaming, multi‑active replication, bulk loading, KV‑storage separation, and automated load, leader and health balancing for reliable, scalable data services.

Distributed storageFailure RecoveryRaft consensus
0 likes · 22 min read
Design and Architecture of Bilibili's High‑Performance Distributed KV Store
Top Architect
Top Architect
Feb 22, 2022 · Databases

Database Sharding: Concepts, Global ID Strategies, Partitioning Schemes, and Expansion Solutions

This article explains database sharding fundamentals, including terminology, when to adopt sharding, various global ID generation methods such as auto‑increment, UUID, COMB, and Snowflake, different partitioning strategies, challenges like distributed transactions, and practical expansion and migration solutions.

ID GenerationShardingpartitioning
0 likes · 14 min read
Database Sharding: Concepts, Global ID Strategies, Partitioning Schemes, and Expansion Solutions