This article explains how to manually assemble a Kubernetes 1.36 cluster using binary files, detailing the overall architecture, core components, and the roles of master and worker nodes, while providing a free 55‑page PDF with step‑by‑step instructions.
This article outlines the essential principles for building a high‑availability system, covering cluster and distributed designs, fault‑tolerance, reliable hardware, disaster recovery, monitoring, security, capacity planning, and automated scaling to achieve optimal performance and resilience.
This article explains how to keep MySQL and Elasticsearch data in sync using synchronous calls, asynchronous notifications, or binlog listeners, and dives deep into Elasticsearch cluster design, node roles, distributed storage, query phases, split‑brain handling, and fault‑tolerance mechanisms.
This article explains MongoDB's three cluster architectures—deprecated Master‑Slave replication, the highly available Replica Set with automatic failover, and horizontal scaling Sharding—detailing their roles, data flow, advantages, limitations, and operational considerations for building reliable, scalable databases.
This article provides a comprehensive technical overview of Elasticsearch, covering core concepts such as indexes, types, documents, fields, mappings, shards and replicas, the cluster architecture, routing, write and read workflows, segment handling, translog, refresh and flush mechanisms, as well as practical optimization recommendations for hardware, shard strategy, routing, write performance and memory configuration.
This article provides a concise, concept‑level overview of Kubernetes architecture, explaining the roles of master and worker nodes, Pods, networking, scheduling, storage, services, scaling, and controller interactions, illustrated with diagrams to help beginners grasp the system’s complexity.
This article provides a diagram‑driven walkthrough of Kafka’s fundamental concepts—including topics, partitions, producers, consumers, consumer groups, and cluster architecture—explaining how messages flow, are stored, and achieve reliability and ordering within a distributed streaming system.
This article provides a comprehensive overview of ClickHouse, covering its core use cases, detailed explanations of the various table engines, recommended schema and deployment patterns, performance‑tuning parameters, tooling choices, and operational guidelines for building and maintaining high‑availability OLAP clusters.
This article explains Elasticsearch’s distributed cluster design, covering core concepts such as nodes, indices, shards, and replicas, compares mixed and tiered deployment models, examines data‑layer storage options, and discusses two typical distributed system architectures with their trade‑offs.
This article explains Elasticsearch's cluster architecture, including nodes, indices, shards, and replicas, compares mixed and tiered deployment models, discusses the data storage layer and replication trade‑offs, and presents two typical distributed data system designs with their advantages and drawbacks.
This guide provides a comprehensive overview of ClickHouse, covering its core use cases, detailed table‑engine choices, cluster design, Zookeeper integration, query and data‑loading best practices, client tools, and key configuration parameters to ensure high performance and reliability in OLAP workloads.
This comprehensive guide introduces ClickHouse as a high‑performance columnar DBMS, outlines its typical OLAP use cases, explains the four families of table engines with detailed MergeTree variants, provides SQL creation syntax, shares development and write best practices, and describes cluster architecture, Zookeeper role, chproxy, client tools, availability considerations, and key configuration parameters.
This article explains Elasticsearch’s distributed cluster design, covering nodes, indices, shards, replicas, deployment models, data storage options, and the trade‑offs of different distributed system architectures for search and analytics workloads.
This article explains Elasticsearch’s cluster and data layer architecture, covering nodes, indices, shards, replicas, mixed and tiered deployment models, storage options, and the trade‑offs of different distributed system designs for scalable systems.
This article explains Kafka’s core design concepts—including the role of a message system, topics, partitions, producers, consumers, consumer groups, replica management, controller coordination with Zookeeper, performance optimizations like sequential writes and zero‑copy, and its network thread model—illustrated with diagrams and code snippets.
This article introduces Elasticsearch as a distributed, high‑performance search engine, explains its cluster architecture, node roles, shard and replica mechanisms, write and refresh processes, and outlines search handling techniques such as pagination, scroll and search‑after with practical curl examples.
This article explains the core concepts of Elasticsearch—including nodes, indices, shards, and replicas—covers its cluster and data‑layer architectures, compares mixed and tiered deployment models, and discusses the advantages and drawbacks of replica‑based distributed storage systems.
This article explains the architecture of Elasticsearch clusters, covering node roles, index, shard and replica concepts, deployment models, data storage mechanisms, and compares two distributed system designs—local‑file‑system and shared‑file‑system—highlighting their advantages and trade‑offs.
This article analyzes Elasticsearch’s distributed architecture, explaining core concepts such as nodes, indices, shards and replicas, comparing mixed and layered deployment models, and discussing data‑layer design choices and their trade‑offs in modern distributed storage and search systems.
This article provides a concise, question‑driven overview of Kubernetes, explaining the roles of master and node components, pod networking, scheduling, storage, service exposure, scaling, and controller interactions, while summarizing the essential objects and processes that power a cloud‑native cluster.
This article describes how to design and build a native distributed cluster called MyCluster without using any existing frameworks, covering master‑slave architecture, leader election, split‑brain handling, centralized configuration management, custom communication protocols, state transitions, and client interfaces.
This article details Vivo's journey in building a company‑wide comment platform, explaining why MongoDB was chosen over MySQL, how its sharded cluster is structured, the challenges of shard‑key selection, and the practical steps taken to scale and maintain the system.
This article explains Pika’s native distributed cluster architecture, detailing its deployment structure, data distribution with tables and slots, request processing flow, both non‑consistent and Raft‑based log replication, and the enhanced metadata management that enables scalable, high‑availability storage beyond single‑node Redis limitations.
The article explains the background, architecture, data distribution, processing flow, replication mechanisms, and management features of Pika's native distributed cluster, detailing how Etcd, LVS, and RocksDB are used to achieve scalable, persistent Redis-compatible storage with table isolation and flexible slot replication.
The article details how JD.com's order center migrated its massive order query workload from MySQL to Elasticsearch, iteratively improving cluster isolation, node deployment, replica tuning, master‑slave redundancy, version upgrades, and data synchronization while addressing performance pitfalls such as deep pagination and FieldData usage.
This article details how JD.com’s order center migrated its Elasticsearch cluster from a basic, default‑configured setup to a highly available, dual‑cluster architecture with increased replicas, dedicated hardware, version upgrades, and optimized shard and pagination strategies to handle billions of documents and hundreds of millions of daily queries.
This article details how JD Daojia's order center migrated from a simple MySQL‑backed system to a high‑throughput Elasticsearch cluster, describing each architectural phase, performance tuning measures, dual‑cluster real‑time backup, version upgrades, data synchronization strategies, and the key pitfalls encountered such as deep pagination and FieldData memory issues.
This article details how JD.com's order center migrated its Elasticsearch cluster from a simple, default‑configured setup to a highly available, multi‑replica, dual‑cluster architecture with version upgrades, data synchronization strategies, and performance optimizations to support billions of documents and hundreds of millions of daily queries.
This article details the progressive evolution of JD.com’s order center Elasticsearch cluster—from its initial default setup through isolation, replica optimization, master‑slave adjustments, and real‑time dual‑cluster backup—highlighting architectural decisions, scaling strategies, synchronization methods, and operational challenges encountered.
This article details the step‑by‑step evolution of the JD Daojia order‑center Elasticsearch cluster—from an initial loosely configured deployment to a real‑time dual‑cluster architecture with replica tuning, master‑slave adjustments, data‑sync strategies, and lessons learned about pagination, fielddata, and doc values—highlighting how each phase improved query throughput, stability, and scalability for billions of documents.
This article details JD.com’s journey of evolving the Elasticsearch cluster for its order system—from a chaotic initial deployment to a real‑time dual‑cluster architecture—highlighting scaling strategies, data sync methods, and the performance pitfalls they overcame.
This article provides a comprehensive introduction to Kafka, covering its role as a messaging system, core concepts such as topics, partitions, producers, consumers, and messages, and then delves into the cluster architecture including replicas, consumer groups, controller coordination with Zookeeper, performance optimizations, log segmentation, and network design.
This article details how JD.com’s order center evolved its Elasticsearch deployment from a single, default‑configured cluster to a real‑time, dual‑cluster architecture with replica tuning, master‑slave failover, version upgrades, and optimized data synchronization to handle billions of documents and hundreds of millions of daily queries.
This article details how JD.com’s order center migrated its Elasticsearch cluster through multiple architectural stages—initial deployment, isolation, replica tuning, master‑slave adjustments, and real‑time dual‑cluster backup—while addressing data synchronization, scaling, and performance pitfalls to achieve high availability and query stability.
This article explains the architecture of a Flink cluster, detailing the startup procedures for JobManager and TaskManager, the three deployment modes, and the end‑to‑end flow of a Flink job from client code through StreamGraph, JobGraph, ExecutionGraph to the physical execution on TaskManagers.
This article introduces the fundamentals of time‑series databases, explains why InfluxDB was chosen, describes the TSM storage engine and shard concepts, outlines the internal 360 InfluxDB‑HA architecture, compares its performance with a single node, and provides integration and future‑development guidelines.
This article details how JD.com’s order center migrated its Elasticsearch cluster from a basic, mixed‑cloud deployment to a real‑time dual‑cluster backup solution, covering each architectural stage, scaling decisions, data‑sync strategies, and the performance pitfalls encountered along the way.
This article details how JD Daojia's order center migrated from MySQL to Elasticsearch, iteratively refined its ES cluster architecture across five stages, tackled scalability and reliability challenges, and implemented robust data synchronization and optimization techniques to support billions of documents and hundreds of millions of daily queries.
This article details how JD Daojia's order center migrated its Elasticsearch cluster through multiple architectural stages—from an initial loosely configured setup to a real‑time dual‑cluster solution—addressing scalability, high availability, data synchronization, and performance optimization for billions of documents and hundreds of millions of daily queries.
This article details how JD.com’s order center migrated its Elasticsearch cluster from a basic, mixed‑node setup to a real‑time, dual‑cluster architecture with increased replicas, physical isolation, version upgrades, and a robust data‑sync strategy to handle billions of documents and hundreds of millions of daily queries.
JD Daojia’s order center faced massive query loads, prompting a shift from MySQL to Elasticsearch and a multi‑stage evolution of its ES cluster—from an initial loosely configured setup, through isolation, replica tuning, master‑slave adjustments, to a real‑time dual‑cluster architecture—enhancing stability, throughput, and scalability.
This article explains Elasticsearch's cluster architecture, including nodes, indices, shards, replicas, deployment models, and data layer storage, and compares two types of distributed data system designs—local file‑system based and shared‑storage based—highlighting their advantages and trade‑offs.
This article details JD.com’s order center journey from a simple Elasticsearch deployment to a highly available, dual‑cluster architecture, covering isolation, replica tuning, hot‑cold data separation, version upgrades, and practical lessons on pagination, field data, and doc values.
This article explains the architecture of Elasticsearch clusters, covering core concepts such as nodes, indices, shards, replicas, indexing flow, mixed and tiered deployment models, data‑layer storage, and compares these designs with other distributed data system architectures.
This article details how JD Daojia’s order center migrated from MySQL‑only reads to a multi‑stage Elasticsearch architecture, describing each evolution step, data‑sync strategies, performance pitfalls, and the final real‑time active‑passive cluster that ensures high availability for billions of daily queries.
This article details JD Daojia's order center Elasticsearch architecture evolution—from a chaotic initial deployment to a real‑time dual‑cluster backup—covering scaling strategies, data synchronization methods, and the practical pitfalls encountered along the way.
This article details how JD.com’s order center migrated its Elasticsearch cluster through multiple stages—from an initial unoptimized deployment to a real‑time dual‑cluster backup solution—addressing scalability, reliability, shard tuning, version upgrades, and data synchronization strategies to support billions of documents and hundreds of millions of daily queries.
This article details how JD Daojia's order center migrated from MySQL to a multi‑stage Elasticsearch cluster—covering initial deployment, isolation, replica tuning, primary‑secondary setup, real‑time dual‑cluster upgrades, data synchronization methods, and key pitfalls—to achieve massive scalability, high availability, and performance for billions of orders.
This article explains Elasticsearch’s cluster architecture, detailing node roles, index, shard, and replica concepts, compares mixed and tiered deployment models, examines data storage mechanisms, and discusses the advantages and trade‑offs of different distributed system designs.
The article describes the motivation, architecture, FullNat forwarding process, high‑availability mechanisms, and performance‑optimizing techniques of TVS, a custom four‑layer load‑balancing platform built to overcome LVS limitations and meet the company’s demanding traffic and reliability requirements.
This article explains the components stored in each K-DB lock—including data block address, instance ID, lock mode, block state, and role—and describes how locks are created, used, and released, with a focus on the CWLS module that manages lock granting and conversion in a clustered environment.
