BestHub
Sign in
Tagged articles

Partitioning

220 articles · Page 2 of 3
Architecture & Thinking
Architecture & Thinking
Jun 25, 2022 · Databases

Why and How to Split Databases: Vertical & Horizontal Sharding Explained

This article explains the reasons for database sharding, compares vertical (scale‑up) and horizontal (scale‑out) partitioning methods, illustrates vertical database/table splits and horizontal partition strategies such as HASH, RANGE, LIST, KEY and composite, and discusses the challenges like distributed transactions, cross‑database joins and increased management overhead.

Horizontal PartitioningMySQLPartitioning
0 likes · 14 min read
Why and How to Split Databases: Vertical & Horizontal Sharding Explained
Big Data Technology & Architecture
Big Data Technology & Architecture
May 24, 2022 · Databases

Apache Doris Basics: Creating Databases, Tables, Partitioning, Data Import, and Rollup

This article provides a comprehensive guide to Apache Doris, covering how to create databases and tables with single and composite partitions, import data via broker and routine loads, understand its aggregate, uniq, and duplicate data models, and leverage rollup and prefix index features for optimized querying.

Apache DorisData ModelingPartitioning
0 likes · 20 min read
Apache Doris Basics: Creating Databases, Tables, Partitioning, Data Import, and Rollup
Code Ape Tech Column
Code Ape Tech Column
Apr 23, 2022 · Big Data

Kafka Fundamentals: Architecture, Replication, Partitioning, and Performance

This article provides a comprehensive overview of Kafka, covering its role as a message middleware, core concepts, architecture, replication management, partition handling, producer sending modes, partition assignment strategies, load balancing, reliability mechanisms, consumer models, controller election, and factors that affect its high throughput and potential message loss scenarios.

KafkaPartitioningReplication
0 likes · 29 min read
Kafka Fundamentals: Architecture, Replication, Partitioning, and Performance
dbaplus Community
dbaplus Community
Apr 10, 2022 · Databases

Designing a High‑Performance Distributed KV Store for B‑Station

This article details the background, architecture, core features, and operational practices of a custom high‑reliability, high‑throughput key‑value storage system that combines Raft replication, flexible partitioning, binlog support, bulk loading, and multi‑active deployment for B‑Station's diverse data workloads.

BinlogFailure RecoveryPartitioning
0 likes · 22 min read
Designing a High‑Performance Distributed KV Store for B‑Station
Big Data Technology & Architecture
Big Data Technology & Architecture
Mar 23, 2022 · Databases

ClickHouse SQL Fundamentals: CREATE, TABLE, Views, ALTER, Partitioning, Import/Export, and Mutation Operations

This article provides a comprehensive guide to ClickHouse SQL, covering database creation, table definitions, column defaults, temporary and partitioned tables, view types, DDL commands, data import/export formats, and mutation‑based update and delete operations with practical code examples.

ClickHouseMutationPartitioning
0 likes · 17 min read
ClickHouse SQL Fundamentals: CREATE, TABLE, Views, ALTER, Partitioning, Import/Export, and Mutation Operations
Hacker Afternoon Tea
Hacker Afternoon Tea
Mar 17, 2022 · Databases

Building Scalable Time‑Series with Distributed PostgreSQL (Citus) – Official Example

This guide shows how to combine PostgreSQL declarative time‑range partitioning with Citus distributed sharding to create a scalable time‑series database, covering table creation, distribution, automatic monthly partition generation, columnar archiving, maintenance with pg_cron, and performance benefits such as faster inserts, queries, and data expiration.

CitusPartitioningPostgreSQL
0 likes · 11 min read
Building Scalable Time‑Series with Distributed PostgreSQL (Citus) – Official Example
Bilibili Tech
Bilibili Tech
Mar 11, 2022 · Databases

Design and Architecture of Bilibili's High‑Performance Distributed KV Store

Bilibili’s high‑performance distributed KV store combines hash and range partitioning, Raft‑based multi‑replica consistency, and a Metaserver‑managed topology of pools, zones, nodes, tables, shards and replicas, offering features such as partition splitting, binlog streaming, multi‑active replication, bulk loading, KV‑storage separation, and automated load, leader and health balancing for reliable, scalable data services.

Failure RecoveryPartitioningRaft consensus
0 likes · 22 min read
Design and Architecture of Bilibili's High‑Performance Distributed KV Store
Top Architect
Top Architect
Feb 22, 2022 · Databases

Database Sharding: Concepts, Global ID Strategies, Partitioning Schemes, and Expansion Solutions

This article explains database sharding fundamentals, including terminology, when to adopt sharding, various global ID generation methods such as auto‑increment, UUID, COMB, and Snowflake, different partitioning strategies, challenges like distributed transactions, and practical expansion and migration solutions.

Partitioningid generationscalability
0 likes · 14 min read
Database Sharding: Concepts, Global ID Strategies, Partitioning Schemes, and Expansion Solutions
IT Architects Alliance
IT Architects Alliance
Dec 3, 2021 · Big Data

Comprehensive Overview of Apache Kafka Architecture and Core Concepts

This article provides an in‑depth technical guide to Apache Kafka, covering its distributed streaming architecture, core concepts such as topics, partitions, brokers, producers and consumers, reliability guarantees, storage mechanisms, configuration parameters, and consumer assignment strategies, supplemented with Java code examples.

Apache KafkaConsumerDistributed Streaming
0 likes · 24 min read
Comprehensive Overview of Apache Kafka Architecture and Core Concepts
Tongcheng Travel Technology Center
Tongcheng Travel Technology Center
Nov 19, 2021 · Big Data

Real‑Time Data Warehouse Practices with Apache Kudu: Architecture, Partitioning, and Platformization

This article reviews the challenges of building a real‑time data warehouse, compares Lambda and Kappa architectures, introduces Apache Kudu’s master‑tablet design, storage model and partition strategies, and shares practical experiences and future directions for a Kudu‑based streaming analytics platform.

Apache KuduKappa ArchitectureLambda Architecture
0 likes · 8 min read
Real‑Time Data Warehouse Practices with Apache Kudu: Architecture, Partitioning, and Platformization
Liangxu Linux
Liangxu Linux
Nov 17, 2021 · Fundamentals

Unlocking Linux File System Secrets: Disk Structure, Inodes, and IO Performance

This article explores Linux file system fundamentals, covering physical disk layout, partition strategies, how the OS reduces random‑read latency, the real space cost of empty files and directories, inode usage, block allocation, and practical tips for improving disk I/O performance.

File SystemFilesystem FundamentalsLinux
0 likes · 16 min read
Unlocking Linux File System Secrets: Disk Structure, Inodes, and IO Performance
Laravel Tech Community
Laravel Tech Community
Oct 19, 2021 · Backend Development

Redis Scaling Strategies: Partitioning, Master‑Slave Replication, Sentinel, and Cluster

This article introduces various Redis scaling solutions—including basic partitioning, master‑slave replication, Sentinel high‑availability, and Redis Cluster—explaining their concepts, typical usage patterns, configuration commands, advantages, and drawbacks to help developers choose the right approach for high‑traffic environments.

PartitioningRedisReplication
0 likes · 12 min read
Redis Scaling Strategies: Partitioning, Master‑Slave Replication, Sentinel, and Cluster
Top Architect
Top Architect
Oct 16, 2021 · Databases

Redis Scaling Strategies: Partitioning, Master‑Slave, Sentinel, and Cluster

This article explains the main Redis scaling approaches—including simple partitioning, master‑slave replication, Sentinel high‑availability, and native Cluster—detailing their concepts, usage methods, advantages, drawbacks, and providing practical configuration commands and code examples.

Master‑SlavePartitioningRedis
0 likes · 12 min read
Redis Scaling Strategies: Partitioning, Master‑Slave, Sentinel, and Cluster
IT Architects Alliance
IT Architects Alliance
Sep 9, 2021 · Databases

Redis Scaling Strategies: Partitioning, Master‑Slave, Sentinel, and Cluster

This article introduces the main Redis scaling solutions—including simple partitioning, master‑slave replication, Sentinel high‑availability, and Redis Cluster—explaining their concepts, usage patterns, advantages, and drawbacks to help developers choose the appropriate architecture for high‑traffic environments.

Master‑SlavePartitioningRedis
0 likes · 12 min read
Redis Scaling Strategies: Partitioning, Master‑Slave, Sentinel, and Cluster
Architect
Architect
Sep 8, 2021 · Databases

Redis Scaling Solutions: Partitioning, Master‑Slave, Sentinel, and Cluster

This article explains how to extend Redis beyond a single instance by covering partitioning, master‑slave replication, Sentinel automatic failover, and Redis Cluster, describing their usage methods, advantages, and drawbacks for high‑traffic, high‑availability scenarios.

PartitioningRedisSentinel
0 likes · 11 min read
Redis Scaling Solutions: Partitioning, Master‑Slave, Sentinel, and Cluster
Full-Stack Internet Architecture
Full-Stack Internet Architecture
Aug 19, 2021 · Backend Development

Ensuring Ordered Message Processing in RocketMQ: Partitioning, Routing, and Consumer Coordination

This article explains how to achieve high‑throughput, high‑availability ordered messaging in RocketMQ by using multi‑partition designs, routing strategies, and consumer locking mechanisms to guarantee local order while handling scaling, failures, and consumer concurrency.

Consumer CoordinationOrdered MessagingPartitioning
0 likes · 7 min read
Ensuring Ordered Message Processing in RocketMQ: Partitioning, Routing, and Consumer Coordination
Java Architect Essentials
Java Architect Essentials
Jul 27, 2021 · Backend Development

Kafka Overview: Architecture, Core Features, and Operational Details

This article provides a comprehensive technical overview of Apache Kafka, covering its distributed messaging architecture, key features such as high‑throughput read/write, replication, partitioning, consumer group mechanics, offset management, rebalance processes, and practical code examples for synchronous and asynchronous offset commits.

Consumer OffsetsDistributed MessagingKafka
0 likes · 22 min read
Kafka Overview: Architecture, Core Features, and Operational Details
ITPUB
ITPUB
May 24, 2021 · Databases

Master MySQL Performance: From Single‑Table Tuning to Horizontal Sharding

This comprehensive guide covers MySQL single‑table optimization, field and index design, query best practices, engine choices, system parameters, hardware scaling, read/write splitting, caching strategies, partitioning, vertical and horizontal sharding, client vs. proxy architectures, and alternative databases, providing actionable steps for high‑performance data handling.

MySQLPartitioning
0 likes · 26 min read
Master MySQL Performance: From Single‑Table Tuning to Horizontal Sharding
Code Ape Tech Column
Code Ape Tech Column
May 10, 2021 · Industry Insights

Why Kafka Beats Redis List: A Deep Dive into Modern Messaging Middleware

This article compares Redis list, Kafka, and Pulsar as messaging middleware, explaining their architectures, strengths, and weaknesses—including queue fundamentals, partitioning, cursor management, consumer groups, high‑availability mechanisms, storage strategies, and consumption models—to help readers choose the right solution for large‑scale systems.

KafkaPartitioningPulsar
0 likes · 30 min read
Why Kafka Beats Redis List: A Deep Dive into Modern Messaging Middleware
Architect's Tech Stack
Architect's Tech Stack
Mar 15, 2021 · Databases

Optimizing a 20‑Million‑Row MySQL Table: Design, Indexing, Partitioning, and Migration Strategies

This article describes how to improve the performance of a massive MySQL 5.6 user‑log table by redesigning schema, applying proper indexes, using partitioning, considering table sharding, and evaluating upgrade paths to compatible cloud or big‑data databases, with concrete SQL examples and cost analysis.

Database OptimizationIndexingMySQL
0 likes · 16 min read
Optimizing a 20‑Million‑Row MySQL Table: Design, Indexing, Partitioning, and Migration Strategies
vivo Internet Technology
vivo Internet Technology
Jan 13, 2021 · Big Data

An Introduction to Apache Kafka: Architecture, Concepts, and Operations

Apache Kafka is a distributed, fault‑tolerant streaming platform that uses broker clusters, topic partitions, and replicated logs to provide publish/subscribe and queue messaging, configurable retention, strong ordering within partitions, producer acknowledgments, consumer groups, and performance optimizations such as batching and zero‑copy for real‑time data pipelines.

Apache KafkaConsumerDistributed Streaming
0 likes · 26 min read
An Introduction to Apache Kafka: Architecture, Concepts, and Operations
Top Architect
Top Architect
Jan 13, 2021 · Big Data

Migrating Over 2 Billion MySQL Records to the Cloud with Kafka and BigQuery

Facing a MySQL table with over 2 billion continuously growing rows, the team designed a cloud‑based solution using Kafka to stream data into Google BigQuery, applied partitioned schemas and data cleaning to improve query performance, reduce storage costs, and avoid downtime.

BigQueryPartitioningdata cleaning
0 likes · 8 min read
Migrating Over 2 Billion MySQL Records to the Cloud with Kafka and BigQuery
Java Architect Essentials
Java Architect Essentials
Dec 31, 2020 · Databases

MySQL Single Table Optimization, Sharding, Partitioning, and Scaling Techniques

When a MySQL table grows large, performance degrades sharply, so this guide explains single‑table tuning, proper field choices, index strategies, query best practices, engine differences, system parameters, hardware upgrades, read‑write splitting, caching layers, table partitioning, vertical and horizontal sharding, and how to choose suitable sharding solutions.

DatabaseScalingIndexingMySQL
0 likes · 25 min read
MySQL Single Table Optimization, Sharding, Partitioning, and Scaling Techniques
Laravel Tech Community
Laravel Tech Community
Dec 10, 2020 · Databases

MySQL Single‑Table Optimization, Partitioning, Sharding and Scaling Strategies

This article provides a comprehensive guide to improving MySQL performance by optimizing single‑table schemas, indexes, queries, engine settings, system parameters, and then discusses read/write separation, caching layers, table partitioning, vertical and horizontal sharding, compatible scalable databases and when to consider NoSQL alternatives.

Database OptimizationMySQLPartitioning
0 likes · 21 min read
MySQL Single‑Table Optimization, Partitioning, Sharding and Scaling Strategies
JD Tech Talk
JD Tech Talk
Nov 30, 2020 · Big Data

Scalable Time Series Similarity Search in Big Data: Partitioning, Dimensionality Reduction, and LSH Approaches

This article examines the challenges of performing time‑series similarity queries on massive datasets and presents three scalable solutions—partition‑based indexing, dimensionality‑reduction using MinHash, and a combined approach with Locality Sensitive Hashing—to reduce computation while preserving similarity accuracy.

LSHMinhashPartitioning
0 likes · 10 min read
Scalable Time Series Similarity Search in Big Data: Partitioning, Dimensionality Reduction, and LSH Approaches
ITPUB
ITPUB
Nov 12, 2020 · Databases

How Vivo Scaled to Billions of Records: Sharding and InnoDB Compression Strategies

This article details how Vivo's cloud service tackled explosive data growth by applying horizontal and vertical sharding, routing‑table based dynamic expansion, and MySQL InnoDB compression, providing step‑by‑step guidance, performance results, and practical recommendations for large‑scale database deployments.

Database ScalingInnoDBMySQL
0 likes · 16 min read
How Vivo Scaled to Billions of Records: Sharding and InnoDB Compression Strategies
Fulu Network R&D Team
Fulu Network R&D Team
Nov 2, 2020 · Databases

Optimizing Large MySQL Tables on Alibaba Cloud: Design, Indexing, PolarDB Migration, X‑Engine Partitioning, and Parallel Query

This article details how to improve performance of massive MySQL tables on Alibaba Cloud by redesigning schemas and indexes, migrating to PolarDB with read‑write separation, partitioning historical data using the X‑Engine storage engine, and leveraging PolarDB's parallel query capabilities for faster analytics.

Database OptimizationIndexingMySQL
0 likes · 12 min read
Optimizing Large MySQL Tables on Alibaba Cloud: Design, Indexing, PolarDB Migration, X‑Engine Partitioning, and Parallel Query
Java Captain
Java Captain
Oct 29, 2020 · Databases

MySQL Single‑Table Optimization, Partitioning, Sharding and Scaling Strategies

When a MySQL table grows large, CRUD performance degrades, so this article presents comprehensive optimization techniques—including field design, indexing, query rewriting, engine selection, system parameters, hardware upgrades, read‑write separation, caching, partitioning, vertical and horizontal splitting, sharding architectures, and recommendations for compatible scalable databases—to help maintain high performance and scalability.

Database OptimizationMySQLPartitioning
0 likes · 26 min read
MySQL Single‑Table Optimization, Partitioning, Sharding and Scaling Strategies
StarRing Big Data Open Lab
StarRing Big Data Open Lab
Oct 14, 2020 · Databases

How In-Memory Databases Organize Data and Indexes: Techniques and System Comparisons

This article explains how modern in‑memory database systems structure data into fixed‑ and variable‑length blocks, choose partitioned or non‑partitioned architectures, handle row versus column storage, and implement cache‑aware index structures such as CSB+-Tree, PB+-Tree, Bw‑Tree, Adaptive Radix, OLFIT and Skiplists, illustrated with Hekaton, H‑Store/VoltDB, HyPer and SAP HANA examples.

Partitioningdata organizationdatabase systems
0 likes · 22 min read
How In-Memory Databases Organize Data and Indexes: Techniques and System Comparisons
Architecture Digest
Architecture Digest
Sep 28, 2020 · Databases

Database Bottlenecks and Sharding Strategies: IO, CPU, and Partitioning Solutions

This article analyzes common database performance bottlenecks such as I/O and CPU limits, explains horizontal and vertical sharding techniques, reviews available sharding tools, discusses the challenges of distributed transactions, cross‑node joins, pagination, and global primary‑key generation, and offers practical mitigation strategies.

PartitioningPerformancedistributed-systems
0 likes · 15 min read
Database Bottlenecks and Sharding Strategies: IO, CPU, and Partitioning Solutions
Big Data Technology & Architecture
Big Data Technology & Architecture
Sep 24, 2020 · Big Data

HiveSQL Classic Optimization Cases: Partitioning, Subset Decomposition, and Percentile Approximation Improvements

This article presents three HiveSQL optimization case studies—restructuring a large‑scale query with partitioned tables, breaking a complex window‑function query into smaller subsets with joins, and refactoring excessive PERCENTILE_APPROX calls—demonstrating how each change reduces execution time from hours to minutes and improves overall performance.

HiveHiveSQLOptimization
0 likes · 9 min read
HiveSQL Classic Optimization Cases: Partitioning, Subset Decomposition, and Percentile Approximation Improvements
Top Architect
Top Architect
Sep 19, 2020 · Databases

MySQL Performance Optimization: Design, Indexing, Partitioning, and Migration Strategies

This article analyzes a severe MySQL performance issue caused by massive user‑access logs, then presents three solution paths—optimizing the existing database, migrating to a compatible high‑performance database, and adopting big‑data technologies—detailing design best practices, indexing, partitioning, sharding, and cloud‑native options to restore query speed and scalability.

Cloud DatabaseDatabase OptimizationIndexing
0 likes · 18 min read
MySQL Performance Optimization: Design, Indexing, Partitioning, and Migration Strategies
Architecture Digest
Architecture Digest
Sep 18, 2020 · Databases

Optimizing MySQL RDS for Large‑Scale User Activity Logs: Design, Indexing, Partitioning, and Migration Strategies

This article analyzes the performance problems of a massive MySQL 5.6 RDS table storing billions of user‑activity records and presents three practical solutions—optimizing the existing schema and queries, migrating to a MySQL‑compatible high‑performance database, and adopting a big‑data platform—detailing design, indexing, partitioning, sharding, and cloud‑native options.

Cloud DatabaseIndexingMySQL
0 likes · 15 min read
Optimizing MySQL RDS for Large‑Scale User Activity Logs: Design, Indexing, Partitioning, and Migration Strategies
ITPUB
ITPUB
Sep 9, 2020 · Databases

How to Speed Up Massive MySQL User‑Log Tables: Partitioning, Indexing, and Migration Strategies

This article examines performance problems with a 20‑million‑row MySQL user‑log table on Alibaba Cloud RDS, outlines three solution paths—optimizing the existing database, migrating to a MySQL‑compatible high‑performance service, and adopting a big‑data engine—and provides detailed guidance on schema design, indexing, partitioning, and practical SQL tweaks.

Database OptimizationIndexingMySQL
0 likes · 17 min read
How to Speed Up Massive MySQL User‑Log Tables: Partitioning, Indexing, and Migration Strategies
JavaEdge
JavaEdge
Sep 8, 2020 · Databases

Understanding Redis Sharding: Benefits, Strategies, and Implementation

This article explains Redis sharding, covering its purpose, various partitioning methods such as range and hash partitioning, implementation approaches like client‑side and proxy‑assisted sharding, their drawbacks, and practical steps for migration and using Redis Cluster or Twemproxy.

PartitioningProxyRedis
0 likes · 9 min read
Understanding Redis Sharding: Benefits, Strategies, and Implementation
Programmer DD
Programmer DD
Sep 7, 2020 · Databases

How to Speed Up Massive MySQL Tables: Practical Optimization Strategies

This article examines why a MySQL 5.6 RDS table with tens of millions of rows becomes unbearably slow, then presents three concrete approaches—optimizing the existing database, migrating to a MySQL‑compatible service, and adopting a big‑data engine—detailing design, indexing, partitioning, sharding, and cloud options to restore performance.

Database OptimizationIndexingMySQL
0 likes · 18 min read
How to Speed Up Massive MySQL Tables: Practical Optimization Strategies
Architects Research Society
Architects Research Society
Sep 2, 2020 · Databases

Scaling PostgreSQL for Multi‑Terabyte Databases: Indexes, Partitioning, Tablespaces, Parallelism, and Replication

This article explains how to extract maximum performance and scalability from PostgreSQL for multi‑terabyte workloads by leveraging specialized indexes, declarative partitioning, tablespaces, parallel query execution, read‑only replica load‑balancing, and foreign‑table sharding techniques.

Database ScalingParallelismPartitioning
0 likes · 10 min read
Scaling PostgreSQL for Multi‑Terabyte Databases: Indexes, Partitioning, Tablespaces, Parallelism, and Replication
Programmer DD
Programmer DD
Aug 17, 2020 · Databases

Understanding ClickHouse MergeTree Partitioning and Merging Rules

This article explains how ClickHouse's MergeTree engine creates partition directories, the naming convention for partitions, and the automatic and manual merging processes that consolidate parts while tracking block numbers and merge levels.

ClickHouseMergeTreePartitioning
0 likes · 7 min read
Understanding ClickHouse MergeTree Partitioning and Merging Rules
Architect
Architect
Aug 16, 2020 · Databases

Database Bottlenecks and Sharding: Strategies, Tools, and Implementation Steps

This article explains common I/O and CPU bottlenecks in databases, introduces horizontal and vertical sharding concepts, compares sharding tools, outlines practical sharding steps, discusses typical sharding issues such as non‑partition queries and expansion, and provides a concise summary and example implementation.

PartitioningPerformancescaling
0 likes · 10 min read
Database Bottlenecks and Sharding: Strategies, Tools, and Implementation Steps
Laravel Tech Community
Laravel Tech Community
Aug 2, 2020 · Databases

MySQL Database Design and SQL Writing Standards

This document outlines comprehensive MySQL database design standards and SQL writing guidelines, covering naming conventions, character sets, table and column specifications, index design, partitioning, data types, transaction handling, and prohibited statements to ensure stable, performant, and maintainable production databases.

IndexingMySQLPartitioning
0 likes · 18 min read
MySQL Database Design and SQL Writing Standards
Full-Stack Internet Architecture
Full-Stack Internet Architecture
Jul 31, 2020 · Backend Development

Why Kafka Is Fast: Partition Parallelism, Sequential Disk Writes, Page Cache, Zero‑Copy, Batching and Compression

The article explains how Kafka achieves high throughput by using partition‑level parallelism, sequential disk writes with segment files, extensive use of the OS page cache, zero‑copy data paths, request batching and optional compression, while also discussing the underlying disk I/O principles.

BackendKafkaPartitioning
0 likes · 14 min read
Why Kafka Is Fast: Partition Parallelism, Sequential Disk Writes, Page Cache, Zero‑Copy, Batching and Compression
Big Data Technology & Architecture
Big Data Technology & Architecture
Jul 23, 2020 · Big Data

Comprehensive Kafka FAQ: Uses, Architecture, Offsets, and Partition Management

This article provides an extensive overview of Apache Kafka, covering its use cases, key concepts such as ISR, AR, HW, LEO, and LW, message ordering, the roles of partitioners, serializers and interceptors, producer and consumer client architecture, offset handling, multithreaded consumption, and topic partition management.

KafkaPartitioningbig data
0 likes · 16 min read
Comprehensive Kafka FAQ: Uses, Architecture, Offsets, and Partition Management
Full-Stack Internet Architecture
Full-Stack Internet Architecture
Jul 16, 2020 · Databases

Comprehensive Guide to MySQL Optimization: Table Design, Indexes, Partitioning, and Read/Write Splitting

This article provides a thorough overview of MySQL performance tuning, covering proper table design according to the three normal forms, various index types and their creation, partitioning strategies, read/write separation, storage engine selection, and practical SQL tips for improving query efficiency.

Database OptimizationIndexingMySQL
0 likes · 14 min read
Comprehensive Guide to MySQL Optimization: Table Design, Indexes, Partitioning, and Read/Write Splitting
Java Captain
Java Captain
Jun 14, 2020 · Databases

MySQL Single‑Table Optimization, Indexing, Partitioning, and Sharding Strategies

This article presents comprehensive techniques for improving MySQL performance on large single tables, covering column choices, index design, query writing, engine selection, system parameters, hardware upgrades, read‑write splitting, caching, table partitioning, vertical and horizontal sharding, and guidance on selecting mature sharding solutions.

Database OptimizationMySQLPartitioning
0 likes · 27 min read
MySQL Single‑Table Optimization, Indexing, Partitioning, and Sharding Strategies
Architecture Digest
Architecture Digest
Jun 10, 2020 · Databases

Comprehensive MySQL Single‑Table Optimization and Scaling Strategies

This article presents a thorough guide to improving MySQL performance for large single tables, covering field choices, indexing best practices, query tuning, engine selection, system parameters, hardware upgrades, read/write separation, caching layers, partitioning, vertical and horizontal sharding, and compatible scalable database alternatives.

Database OptimizationIndexingMySQL
0 likes · 21 min read
Comprehensive MySQL Single‑Table Optimization and Scaling Strategies
DataFunTalk
DataFunTalk
Jun 7, 2020 · Databases

ByteKV: Design and Implementation of a Strongly Consistent Range-Partitioned KV Store

ByteKV is a C++-based, strongly consistent, range-partitioned key‑value storage system built by ByteDance, featuring a multi‑Raft consensus layer, custom storage engines (RocksDB and BlockDB), automatic partition splitting/merging, load balancing, distributed transactions, and a SQL table layer for rich data models.

ConsensusPartitioningSQL
0 likes · 47 min read
ByteKV: Design and Implementation of a Strongly Consistent Range-Partitioned KV Store
Top Architect
Top Architect
May 29, 2020 · Databases

Redis Scaling Strategies: Partitioning, Master‑Slave Replication, Sentinel, and Cluster

This article explains how to extend Redis beyond single‑node limits by using partitioning, master‑slave replication, Sentinel for automatic failover, and Redis Cluster with hash slots, detailing their usage, advantages, drawbacks, and configuration examples for building high‑availability and scalable in‑memory data stores.

PartitioningRedisReplication
0 likes · 11 min read
Redis Scaling Strategies: Partitioning, Master‑Slave Replication, Sentinel, and Cluster
Java Backend Technology
Java Backend Technology
Apr 26, 2020 · Databases

When to Shard Your Database? A Practical Guide to Partitioning Strategies

This article explains database bottlenecks caused by IO and CPU limits, introduces horizontal and vertical sharding for databases and tables, compares popular sharding tools, discusses challenges such as distributed transactions, cross‑node joins, pagination and global ID generation, and offers guidance on when and how to apply sharding in real‑world systems.

Partitioningdatabasedistributed systems
0 likes · 14 min read
When to Shard Your Database? A Practical Guide to Partitioning Strategies
ITPUB
ITPUB
Apr 22, 2020 · Databases

Master Oracle Database Essentials: Startup, Users, Tablespaces, Indexes & More

This guide walks through essential Oracle Database administration tasks, covering how to start and shut down the database and listener, create and manage tablespaces, users, and permissions, control transactions, build and maintain indexes, views, materialized views, sequences, synonyms, and partitioned tables, with command examples and best‑practice tips.

Database AdministrationOraclePartitioning
0 likes · 21 min read
Master Oracle Database Essentials: Startup, Users, Tablespaces, Indexes & More
Big Data Technology Architecture
Big Data Technology Architecture
Apr 13, 2020 · Backend Development

Understanding Kafka Producer: Architecture, Data Structures, Serialization, Partitioning, and Buffering

This article provides a comprehensive overview of Kafka's Producer side, covering its architecture, the ProducerRecord data structure, serialization mechanisms, partitioning logic, and the accumulator buffer, while comparing old and new Producer clients and illustrating key configurations with code examples.

AccumulatorKafkaPartitioning
0 likes · 9 min read
Understanding Kafka Producer: Architecture, Data Structures, Serialization, Partitioning, and Buffering
Java Captain
Java Captain
Mar 31, 2020 · Databases

Optimizing Large MySQL Tables: Indexes, Partitioning, Sharding, and System Tuning

When a MySQL table grows to millions of rows, query, insert, update and delete performance degrades sharply, but by applying proper field design, index strategies, SQL best practices, engine selection, system parameters, caching, partitioning, vertical and horizontal sharding, and choosing suitable client‑side or proxy‑side architectures, the database can efficiently handle tens of millions of records.

MySQLOptimizationPartitioning
0 likes · 24 min read
Optimizing Large MySQL Tables: Indexes, Partitioning, Sharding, and System Tuning
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Mar 6, 2020 · Big Data

Mastering Kafka: Build Producers, Consumers, and Custom Partitioners

This extensive tutorial walks through Kafka producer and consumer fundamentals, demonstrates how to configure key‑based and custom partitioning, explains offset management, consumer group coordination, and essential configuration parameters, and includes complete Java code examples for real‑world e‑commerce scenarios.

ConsumerKafkaPartitioning
0 likes · 24 min read
Mastering Kafka: Build Producers, Consumers, and Custom Partitioners
Tencent Cloud Developer
Tencent Cloud Developer
Feb 18, 2020 · Backend Development

Technical Overview of Tencent Cloud CKafka for High-Scale Online Classroom Messaging

Tencent Cloud CKafka powers Tencent Classroom’s pandemic‑era online teaching by replacing a custom queue with a high‑performance, highly available, partition‑based message bus that scales to millions of real‑time interactions, offers configurable replication and tuning for reliability, and integrates with big‑data and streaming tools for analytics.

Backend DevelopmentCKafkaKafka
0 likes · 15 min read
Technical Overview of Tencent Cloud CKafka for High-Scale Online Classroom Messaging
Architecture Digest
Architecture Digest
Feb 13, 2020 · Databases

Understanding InnoDB Logical Storage Structure and MySQL Partitioning

This article explains the InnoDB logical storage architecture—including segments, extents, and pages—then introduces MySQL table partitioning concepts, details the main partition types (RANGE, LIST, HASH, KEY) with SQL examples, and discusses their performance implications for OLTP and OLAP workloads.

InnoDBMySQLPartitioning
0 likes · 8 min read
Understanding InnoDB Logical Storage Structure and MySQL Partitioning
Big Data Technology & Architecture
Big Data Technology & Architecture
Dec 25, 2019 · Big Data

Understanding Flink StreamPartitioner and Its Implementations

Flink’s StreamPartitioner abstracts data routing in DataStream, offering eight built‑in partitioners—including Global, Shuffle, Rebalance, KeyGroup, Broadcast, Rescale, Forward, and Custom—each with distinct channel selection logic, illustrated with source code snippets and explanations of their runtime behavior.

DataStreamFlinkJava
0 likes · 8 min read
Understanding Flink StreamPartitioner and Its Implementations
vivo Internet Technology
vivo Internet Technology
Dec 25, 2019 · Big Data

Understanding and Mitigating Data Skew in Spark and Hadoop

Data skew in Spark and Hadoop occurs when a few keys dominate shuffle traffic, causing slow tasks, OOM errors, and job failures; the article describes how to detect skew via UI metrics or sampling and offers mitigation tactics such as filtering keys, increasing shuffle partitions, custom partitioners, broadcast joins, salted keys, and Hadoop‑specific settings.

Data SkewPartitioningPerformance Optimization
0 likes · 18 min read
Understanding and Mitigating Data Skew in Spark and Hadoop
Architects Research Society
Architects Research Society
Oct 22, 2019 · Big Data

Continuous Delivery of Event Streaming Pipelines with Spring Cloud Data Flow

This article explains how to build, deploy, and continuously update event streaming pipelines using Spring Cloud Data Flow and Apache Kafka, covering common topologies, named destinations, parallel and partitioned streams, function composition, multiple input/output bindings, and practical shell commands for registration and management.

Continuous DeploymentEvent StreamingKafka
0 likes · 19 min read
Continuous Delivery of Event Streaming Pipelines with Spring Cloud Data Flow
ITPUB
ITPUB
Oct 9, 2019 · Databases

What’s New in PostgreSQL 12? Performance Boosts and Advanced Features

PostgreSQL 12 brings major enhancements—including up to 40% faster queries on large datasets, improved indexing, partitioning, JSON path queries, generated columns, pluggable table access methods, stronger authentication, and built‑in JIT compilation—while also adding better localization, multi‑constraint authorization, and convenient maintenance commands.

IndexingJSONPartitioning
0 likes · 6 min read
What’s New in PostgreSQL 12? Performance Boosts and Advanced Features
Java Captain
Java Captain
Sep 5, 2019 · Databases

MySQL Optimization and Best Practices: Indexing, Partitioning, Replication, and Performance Tuning

This article provides a comprehensive guide to MySQL performance optimization, covering database design principles, field type selection, indexing strategies, storage engine choices, query caching, partitioning methods, master‑slave replication setup, read/write splitting, load balancing, and profiling tools for diagnosing slow queries.

IndexingMySQLOptimization
0 likes · 43 min read
MySQL Optimization and Best Practices: Indexing, Partitioning, Replication, and Performance Tuning
ITPUB
ITPUB
Jul 19, 2019 · Operations

How to Shrink /home and Expand Root on CentOS 7

This guide walks through shrinking the /home partition, extending the root filesystem on a CentOS 7 server, handling busy mount issues, installing missing tools like fuser and xfsdump, resizing XFS, and safely remounting the partitions.

CentOSLinuxPartitioning
0 likes · 2 min read
How to Shrink /home and Expand Root on CentOS 7
Aikesheng Open Source Community
Aikesheng Open Source Community
Jun 12, 2019 · Databases

DBLE Sharding Algorithm Overview and Configuration Guide

This article explains DBLE's hash‑based sharding algorithm, detailing how logical shards are derived via modulo, mapped to physical shards through a rule.xml configuration and mapFile, compares it with MyCat, and provides development, operational, and configuration best practices along with code examples.

DBLEPartitioningmapFile
0 likes · 6 min read
DBLE Sharding Algorithm Overview and Configuration Guide
21CTO
21CTO
Apr 15, 2019 · Big Data

Mastering High‑Concurrency Big Data: Sharding, Partitioning, and Index Strategies

This article explores practical techniques for handling massive, high‑concurrency data workloads, covering relational database limits, read/write separation, vertical and horizontal sharding, key selection, archival to NoSQL stores, and the use of heterogeneous index tables to maintain performance.

Database ScalingHigh ConcurrencyPartitioning
0 likes · 6 min read
Mastering High‑Concurrency Big Data: Sharding, Partitioning, and Index Strategies
Youzan Coder
Youzan Coder
Mar 15, 2019 · Backend Development

How to Build a Youzan‑Custom NSQ Client with Partitioning, Lookup, and Message Tracing

This guide explains how to construct a client library for Youzan's customized NSQ, covering workflow configuration, nsqd discovery via nsqlookupd, connection establishment, message publishing and consumption, ordered consumption, and the implementation of message tracing with concrete command syntax and response handling.

Client LibraryMessage TracingNSQ
0 likes · 15 min read
How to Build a Youzan‑Custom NSQ Client with Partitioning, Lookup, and Message Tracing
Programmer DD
Programmer DD
Feb 11, 2019 · Databases

How Partitioned Tables Supercharge MySQL Performance and Their Limits

This article demonstrates how MySQL partition tables dramatically improve query speed for large datasets, compares partition-key, secondary-index, and full‑scan queries, outlines the practical performance gains, and details the numerous limitations and trade‑offs that influence when to adopt partitioning versus sharding.

MySQLPartitioningdatabase
0 likes · 15 min read
How Partitioned Tables Supercharge MySQL Performance and Their Limits
Big Data Technology & Architecture
Big Data Technology & Architecture
Jan 2, 2019 · Big Data

Optimizing Spark Direct Kafka Consumption: Subpartition Concurrency and Repartition Strategies

To address the long processing time caused by uneven Spark partitions when reading Kafka via the Direct approach, this article explains the SPARK‑22056 solution that modifies KafkaRDD.getPartitions to support a configurable 'topic.partition.subconcurrency' parameter, discusses its trade‑offs, and presents alternative repartition and multithreading techniques.

PartitioningScalaSpark
0 likes · 6 min read
Optimizing Spark Direct Kafka Consumption: Subpartition Concurrency and Repartition Strategies
AntTech
AntTech
Dec 26, 2018 · Databases

OceanBase 2.0 Partitioning and Performance Optimizations for Ant Group’s Double‑11 Peak

This article explains how OceanBase 2.0’s partitioning architecture, partition‑group feature, and a series of performance optimizations enabled Ant Group’s payment core to handle millions of transactions per second during the Double‑11 shopping festival, while also describing the system’s advantages, results, and future capabilities.

Ant GroupDouble 11OceanBase
0 likes · 10 min read
OceanBase 2.0 Partitioning and Performance Optimizations for Ant Group’s Double‑11 Peak
Qunar Tech Salon
Qunar Tech Salon
Nov 20, 2018 · Databases

Strategies for Massive Data Storage and Sharding in High‑Scale Systems

This article examines various approaches to storing massive data, including table partitioning, NoSQL/NewSQL, and database sharding, analyzes their advantages and drawbacks, and presents practical sharding designs for user, forum, and order databases, with considerations for caching, redundancy, and query routing.

CachingPartitioningsharding
0 likes · 19 min read
Strategies for Massive Data Storage and Sharding in High‑Scale Systems
Efficient Ops
Efficient Ops
Jul 16, 2018 · Databases

Master MySQL Advanced Features: Partitioning, Views, Triggers, and More

This article explores MySQL’s advanced capabilities—including partition tables, views, stored procedures, triggers, foreign‑key constraints, bind variables, user‑defined functions, and character‑set considerations—explaining their principles, usage patterns, performance impacts, and best‑practice recommendations for handling large‑scale data.

Foreign KeysMySQLPartitioning
0 likes · 34 min read
Master MySQL Advanced Features: Partitioning, Views, Triggers, and More
Java Captain
Java Captain
Jul 7, 2018 · Databases

Understanding MySQL Indexes, B+Tree vs Hash, Partitioning, and Redis Fundamentals

This article explains why auto‑increment primary keys are preferred in MySQL, how B+Tree and hash indexes differ, the benefits of composite and partitioned indexes, isolation levels, MVCC, row‑level locking, triggers, stored procedures, optimization tips, MyISAM vs InnoDB, table design guidelines, and also covers Redis architecture, persistence, replication, and eviction policies.

B+ TreeMySQLPartitioning
0 likes · 28 min read
Understanding MySQL Indexes, B+Tree vs Hash, Partitioning, and Redis Fundamentals
ITPUB
ITPUB
Jun 14, 2018 · Databases

Mastering Oracle Partitioning: Concepts, History, Best Practices & New Features

This article presents a comprehensive overview of Oracle partitioning, covering basic concepts, the evolution of partition features across versions, practical best‑practice scenarios, and the latest enhancements in Oracle 12c, 12.2, and 18c that improve performance, manageability, and availability.

Best PracticesOraclePartitioning
0 likes · 17 min read
Mastering Oracle Partitioning: Concepts, History, Best Practices & New Features
Java Backend Technology
Java Backend Technology
Apr 8, 2018 · Databases

Master MySQL: Table Design, Slow Query Fixes, and Recovery Strategies

This article reviews common MySQL pitfalls—including engine choice, table partitioning, index design, slow‑query causes and optimizations, diagnostic tools, and how to recover from accidental data loss—offering practical guidance for developers to improve performance and reliability.

Index OptimizationMySQLPartitioning
0 likes · 10 min read
Master MySQL: Table Design, Slow Query Fixes, and Recovery Strategies
ITPUB
ITPUB
Jan 31, 2018 · Databases

Master MySQL: Table Design, Slow Query Fixes, and Recovery Strategies

This article explains how to choose MySQL storage engines, apply effective table partitioning and indexing, identify common causes of slow queries, adopt practical SQL optimization tips, use diagnostic tools, and handle data loss or bugs with proper recovery procedures.

DiagnosticsMySQLPartitioning
0 likes · 12 min read
Master MySQL: Table Design, Slow Query Fixes, and Recovery Strategies
ITPUB
ITPUB
Nov 25, 2017 · Databases

How to Handle Billions of Rows in SQL Server: Bulk Insert, Partitioning, and Index Tuning

This article walks through a real‑world monitoring project that must store over 400 million rows per day in SQL Server 2012, detailing the performance bottlenecks, bulk‑copy tuning, schema redesign, partitioning strategies, index removal and creation, and query‑optimisation techniques that finally achieve sub‑second query times and meet strict latency requirements.

Bulk InsertIndex OptimizationPartitioning
0 likes · 16 min read
How to Handle Billions of Rows in SQL Server: Bulk Insert, Partitioning, and Index Tuning
Architecture Digest
Architecture Digest
Nov 3, 2017 · Databases

Understanding Database Sharding: Why It Matters and How to Implement It

Database sharding, or splitting databases into multiple instances and tables, is essential for scaling backend systems; this article explains its importance, historical context, core concepts, storage stretching techniques, index and consistency handling, and provides a practical design example for transaction systems.

Database ScalingIndexingPartitioning
0 likes · 11 min read
Understanding Database Sharding: Why It Matters and How to Implement It
dbaplus Community
dbaplus Community
Oct 11, 2017 · Databases

Master MySQL Advanced Features: Partitioning, Views, Stored Procedures, and More

This article explores MySQL’s advanced features—including partition tables, views, stored procedures, triggers, foreign key constraints, bind variables, user‑defined functions, and character set considerations—explaining their principles, usage patterns, performance implications, and practical tips for large‑scale data scenarios.

MySQLPartitioningStored Procedures
0 likes · 35 min read
Master MySQL Advanced Features: Partitioning, Views, Stored Procedures, and More
ITPUB
ITPUB
Oct 7, 2017 · Databases

Essential Oracle SQL Commands: From Connecting to Advanced Tablespace Management

This guide provides a comprehensive collection of Oracle SQL commands covering connection, query execution, variable handling, formatting, tablespace creation, modification, and external table operations, presented step‑by‑step for database administrators and developers.

Database AdministrationExternal TableOracle
0 likes · 11 min read
Essential Oracle SQL Commands: From Connecting to Advanced Tablespace Management
ITPUB
ITPUB
May 9, 2017 · Databases

Master MySQL Single-Table Optimization and Scaling Strategies

Learn how to boost MySQL performance for massive tables by optimizing schema design, indexing, query patterns, engine selection, system parameters, hardware upgrades, read/write splitting, caching layers, partitioning, vertical and horizontal sharding, and choosing appropriate client or proxy sharding solutions.

MySQLOptimizationPartitioning
0 likes · 25 min read
Master MySQL Single-Table Optimization and Scaling Strategies