BestHub
Sign in
Tagged articles
1273 articles
Page 6 of 13
Selected Java Interview Questions
Selected Java Interview Questions
Aug 27, 2022 · Backend Development

Deploying a Cost‑Effective ClickHouse‑Based Backend Data Platform: Comparison with Elasticsearch and Step‑by‑Step Setup Guide

This article compares Elasticsearch and ClickHouse for log analytics, presents cost analysis, and provides detailed deployment instructions for Zookeeper, Kafka, Filebeat, and ClickHouse to build a private, high‑performance backend data platform for SaaS services.

ElasticsearchFilebeatKafka
0 likes · 12 min read
Deploying a Cost‑Effective ClickHouse‑Based Backend Data Platform: Comparison with Elasticsearch and Step‑by‑Step Setup Guide
Meituan Technology Team
Meituan Technology Team
Aug 25, 2022 · Databases

Transactions, Consistency, and Consensus in Distributed Systems

This article explains how transactions, consistency, and consensus interrelate in distributed systems, reviews replication models, details ACID properties, internal consistency models, consensus algorithms, and Kafka’s configuration and transaction support, and highlights verification tools like TLA+ and Jepsen for ensuring correctness.

ACIDConsensusConsistency
0 likes · 46 min read
Transactions, Consistency, and Consensus in Distributed Systems
Meituan Technology Team
Meituan Technology Team
Aug 25, 2022 · Databases

Data Replication in Distributed Systems – Part 1: Models, Challenges, and Design Considerations

The article surveys three data‑replication models—master‑slave, multi‑master, and leaderless—explains how they enable scalability and fault‑tolerance, and examines core distributed‑system challenges such as partial node failures, unreliable networks, and unsynchronized clocks, while stressing safety‑liveness trade‑offs and design techniques like quorum and timeouts.

ConsistencyDistributed SystemsKafka
0 likes · 37 min read
Data Replication in Distributed Systems – Part 1: Models, Challenges, and Design Considerations
dbaplus Community
dbaplus Community
Aug 23, 2022 · Big Data

How Meituan Scaled Kafka to 15k Nodes and Overcame Latency & Management Challenges

Meituan’s data platform uses Kafka as a unified cache and distribution layer, and as the cluster grew beyond 15,000 brokers and daily traffic exceeded 30 PB, the team tackled slow nodes, read/write latency, large‑scale cluster management, full‑link monitoring, service lifecycle, and future cloud‑native deployment challenges.

KafkaLarge-Scale ClusterRead‑Write Latency
0 likes · 22 min read
How Meituan Scaled Kafka to 15k Nodes and Overcame Latency & Management Challenges
Big Data Technology & Architecture
Big Data Technology & Architecture
Aug 23, 2022 · Big Data

Using Flink Broadcast State for Dynamic Configuration Updates and Real‑Time Data Enrichment

This article explains how Flink's Broadcast State feature can be used to dynamically update processing rules and enrich streaming events with user information from MySQL, showing configuration, code examples, key considerations, and runtime results that demonstrate real‑time adaptability without restarting the job.

Broadcast StateDynamic ConfigurationFlink
0 likes · 15 min read
Using Flink Broadcast State for Dynamic Configuration Updates and Real‑Time Data Enrichment
Tencent Cloud Developer
Tencent Cloud Developer
Aug 19, 2022 · Backend Development

Message Queue Basics, Use Cases, and Selection Guide for Kafka, RabbitMQ, Pulsar, and RocketMQ

The article explains fundamental concepts, common use cases, and selection criteria for four popular message‑queue platforms—Kafka, RabbitMQ, Pulsar, and RocketMQ—detailing each system’s architecture, strengths, and drawbacks so readers can match workload, scalability, ordering, latency, and operational needs to the most suitable middleware.

Distributed SystemsKafkaMessage Queue
0 likes · 16 min read
Message Queue Basics, Use Cases, and Selection Guide for Kafka, RabbitMQ, Pulsar, and RocketMQ
Ops Development Stories
Ops Development Stories
Aug 19, 2022 · Big Data

Master Kafka: From Core Concepts to Advanced Operations and Performance Tuning

This comprehensive guide explains Kafka’s origins, core architecture, data structures, write and read workflows, operational commands for topic and consumer‑group management, and practical performance‑tuning tips such as disk layout, JVM settings, flush policies, and log retention, providing a complete reference for engineers working with distributed streaming platforms.

Data StreamingDistributed MessagingKafka
0 likes · 32 min read
Master Kafka: From Core Concepts to Advanced Operations and Performance Tuning
IT Architects Alliance
IT Architects Alliance
Aug 13, 2022 · Operations

Why ClickHouse Beats Elasticsearch: Performance, Cost, and Deployment Guide

This article compares ClickHouse and Elasticsearch, analyzes cost savings, and provides step‑by‑step deployment instructions for Zookeeper, Kafka, Filebeat, and ClickHouse clusters, including configuration details, troubleshooting tips, and practical code snippets for building a scalable analytics pipeline.

DeploymentElasticsearchFilebeat
0 likes · 13 min read
Why ClickHouse Beats Elasticsearch: Performance, Cost, and Deployment Guide
IT Architects Alliance
IT Architects Alliance
Aug 6, 2022 · Backend Development

Practical Coding Tips: Kafka Transaction Commit, Redis Distributed Lock Simplification, AOP Lock Annotation, and Business Log Decoupling

This article shares practical development techniques, including how to safely commit Kafka messages within transactions, simplify Redis distributed locks with Redisson, implement an AOP‑based lock annotation for method-level locking, and decouple business logging using asynchronous thread pools and helper utilities.

Kafkaaopdistributed-lock
0 likes · 14 min read
Practical Coding Tips: Kafka Transaction Commit, Redis Distributed Lock Simplification, AOP Lock Annotation, and Business Log Decoupling
IT Architects Alliance
IT Architects Alliance
Aug 4, 2022 · Big Data

Explanation of Kafka Components and Architecture

This article provides a comprehensive overview of Kafka's core components—including brokers, topics, partitions, producers, and consumers—explaining their roles, how they interact within a Kafka cluster, and illustrating the architecture with diagrams while also noting recent version changes and related disclaimer information.

BrokerConsumerKafka
0 likes · 7 min read
Explanation of Kafka Components and Architecture
Meituan Technology Team
Meituan Technology Team
Aug 4, 2022 · Big Data

Optimizing Kafka for Large-Scale Data Platforms at Meituan

The article details Meituan's massive Kafka deployment—over 15,000 machines handling more than 30 PB of daily data—its performance and management challenges, and the comprehensive application‑layer, system‑layer, and hybrid‑layer optimizations Meituan implemented to reduce read/write latency and improve large‑scale cluster reliability.

Cluster ManagementData PlatformFull‑Link Monitoring
0 likes · 25 min read
Optimizing Kafka for Large-Scale Data Platforms at Meituan
Sohu Tech Products
Sohu Tech Products
Aug 3, 2022 · Fundamentals

Common Message Queues: RabbitMQ, RocketMQ, and Kafka – Components, Features, and Best Practices

This article introduces the core concepts, components, exchange types, reliability mechanisms, ordering, delay, transaction, high‑availability, and load‑balancing strategies of three popular message‑queue systems—RabbitMQ, RocketMQ, and Kafka—while also discussing common challenges such as message ordering, delayed consumption, reliability, idempotence, and backlog handling.

Distributed SystemsKafkaMessage Queue
0 likes · 33 min read
Common Message Queues: RabbitMQ, RocketMQ, and Kafka – Components, Features, and Best Practices
IT Architects Alliance
IT Architects Alliance
Aug 3, 2022 · Big Data

Understanding Kafka Architecture: Topics, Partitions, Replication, Log Segmentation, Zero‑Copy, and Zookeeper Integration

This article explains Kafka's core concepts—including topics, partitions and replicas, log segment storage, leader‑follower mechanics, consumer groups, network threading model, zero‑copy I/O, and the essential role of Zookeeper for broker, topic, consumer, and offset management—providing a comprehensive overview for developers and architects.

Big DataKafkaStreaming
0 likes · 10 min read
Understanding Kafka Architecture: Topics, Partitions, Replication, Log Segmentation, Zero‑Copy, and Zookeeper Integration
IT Architects Alliance
IT Architects Alliance
Aug 1, 2022 · Backend Development

Overview of RabbitMQ, RocketMQ, and Kafka Components and Mechanisms

This article provides a comprehensive overview of the core components, exchange types, TTL, confirmation mechanisms, dead‑letter queues, and reliability features of RabbitMQ, RocketMQ, and Kafka, along with practical guidance on ordering, delayed consumption, transaction handling, high availability, load balancing, and message deduplication in distributed messaging systems.

KafkaRabbitMQRocketMQ
0 likes · 33 min read
Overview of RabbitMQ, RocketMQ, and Kafka Components and Mechanisms
37 Interactive Technology Team
37 Interactive Technology Team
Aug 1, 2022 · Backend Development

Understanding Kafka Consumer Groups, Polling, and Offset Commit Strategies with kafka-go

The article demonstrates how kafka-go implements consumer groups by creating readers that spawn poll goroutines per partition, explains heartbeat‑driven rebalancing, and compares automatic versus manual offset‑commit strategies, highlighting their impact on throughput, reliability, and message‑processing semantics.

GoKafkaconsumer-group
0 likes · 15 min read
Understanding Kafka Consumer Groups, Polling, and Offset Commit Strategies with kafka-go
MaGe Linux Operations
MaGe Linux Operations
Jul 31, 2022 · Backend Development

Mastering Message Queues: Key Concepts of RabbitMQ, RocketMQ, and Kafka

This article provides a comprehensive overview of modern message‑queue systems, detailing RabbitMQ components and exchange types, RocketMQ’s core architecture and high‑availability features, and Kafka’s fundamental concepts, controller mechanisms, and consumer rebalance strategies, while also covering reliability, ordering, and dead‑letter handling techniques.

Distributed SystemsKafkaMessage Queue
0 likes · 31 min read
Mastering Message Queues: Key Concepts of RabbitMQ, RocketMQ, and Kafka
Su San Talks Tech
Su San Talks Tech
Jul 31, 2022 · Fundamentals

Mastering Message Queues: Core Concepts of RabbitMQ, RocketMQ, and Kafka

This article explains the fundamental components and mechanisms of RabbitMQ, RocketMQ, and Kafka—including exchanges, TTL, confirm and return listeners, consumer ACK/NACK, dead‑letter queues, delay and ordered messages, transactional flows, high‑availability setups, load balancing, partitioning, leader election, offset handling, and practical solutions to common messaging challenges.

Distributed SystemsKafkaMessage Queue
0 likes · 31 min read
Mastering Message Queues: Core Concepts of RabbitMQ, RocketMQ, and Kafka
AI Illustrated Series
AI Illustrated Series
Jul 28, 2022 · Backend Development

How to Ensure Distributed Transaction Consistency Using Message Queues

This article analyzes the challenges of distributed transactions in multi‑node systems and compares various solutions—including 2PC, 3PC, TCC, Saga, local message tables, and MQ‑based approaches—detailing their mechanisms, trade‑offs, and practical implementations with concrete examples from order and cart services.

2PCDistributed TransactionsKafka
0 likes · 19 min read
How to Ensure Distributed Transaction Consistency Using Message Queues
Top Architect
Top Architect
Jul 27, 2022 · Big Data

Understanding Kafka Architecture: Topics, Partitions, Replication, Zero‑Copy, and Zookeeper Integration

This article explains Kafka's core architecture, covering the logical concept of topics, physical partitioning and replication, leader‑follower mechanics, consumer groups, log segmentation with zero‑copy I/O, and how Zookeeper manages broker registration, topic metadata, consumer coordination, and offset tracking, while also discussing producer and consumer load‑balancing strategies.

KafkaPartitioningdistributed-systems
0 likes · 10 min read
Understanding Kafka Architecture: Topics, Partitions, Replication, Zero‑Copy, and Zookeeper Integration
DataFunTalk
DataFunTalk
Jul 26, 2022 · Big Data

Feature Platform Architecture and Stream‑Batch Integrated Solutions

This talk presents Shuhe Technology’s feature platform, detailing its four‑layer architecture, feature storage services, stream‑batch integrated processing, event‑center design, consistency models, and four model‑strategy invocation schemes, illustrating data flows from MySQL through Sqoop, Kafka, Flink, HBase and ClickHouse.

Big DataFlinkHBase
0 likes · 17 min read
Feature Platform Architecture and Stream‑Batch Integrated Solutions
Sanyou's Java Diary
Sanyou's Java Diary
Jul 21, 2022 · Fundamentals

How to Choose the Right Message Queue: Kafka vs RabbitMQ vs RocketMQ

This article explains the fundamentals of message queues, compares Kafka, RabbitMQ, and RocketMQ in terms of architecture, key concepts, advantages, and drawbacks, and provides practical guidance on selecting the most suitable queue for different application scenarios such as high‑throughput logging, financial transactions, or small‑scale services.

KafkaRabbitMQRocketMQ
0 likes · 16 min read
How to Choose the Right Message Queue: Kafka vs RabbitMQ vs RocketMQ
Top Architect
Top Architect
Jul 21, 2022 · Big Data

Kafka Components and Architecture Explained

This article provides a comprehensive overview of Kafka's core components—including brokers, topics, partitions, producers, and consumers—explaining their roles, relationships, replication strategies, and how they work together within a Kafka cluster to ensure reliable, scalable message streaming.

BrokerConsumerDistributed
0 likes · 8 min read
Kafka Components and Architecture Explained
dbaplus Community
dbaplus Community
Jul 19, 2022 · Cloud Native

How to Build a Scalable Kubernetes Log System with ClickHouse and Fluent‑Bit

This article explains why Stone Docs switched from SLS/ES to ClickHouse for Kubernetes log storage, outlines the four‑stage architecture (collection, transmission, storage, management), compares DaemonSet, network, and SideCar collection methods, and provides concrete ClickHouse table definitions and Fluent‑Bit configurations for a production‑grade logging pipeline.

Cloud NativeFluent BitKafka
0 likes · 14 min read
How to Build a Scalable Kubernetes Log System with ClickHouse and Fluent‑Bit
Java High-Performance Architecture
Java High-Performance Architecture
Jul 19, 2022 · Backend Development

Build a Dynamic Microservice Flow Orchestrator with Nacos, Kafka, and SpringBoot

This article explains how to create a lightweight, plug‑in‑style microservice flow orchestration system using Docker‑installed Nacos for configuration, three SpringBoot services with Kafka integration, and dynamic topic management to enable hot‑swapable data pipelines without heavy frameworks.

Dynamic ConfigurationFlow OrchestrationKafka
0 likes · 10 min read
Build a Dynamic Microservice Flow Orchestrator with Nacos, Kafka, and SpringBoot
Open Source Linux
Open Source Linux
Jul 19, 2022 · Backend Development

Master Kafka Basics: Visual Guide to Topics, Partitions, and Architecture

This article visually explains Kafka's core concepts—including producers, consumers, topics, partitions, consumer groups, and cluster architecture—so readers can clearly understand how messages flow, are stored, and remain fault‑tolerant within a distributed streaming system.

BackendKafkaMessage Queue
0 likes · 6 min read
Master Kafka Basics: Visual Guide to Topics, Partitions, and Architecture
StarRocks
StarRocks
Jul 18, 2022 · Big Data

How Songguo Mobility Built a Real‑Time OLAP Platform with StarRocks: From 1.0 to 3.0

Songguo Mobility’s data‑center team migrated from a fragmented Impala‑Kudu‑ClickHouse stack to a unified StarRocks‑based real‑time OLAP architecture, iterating through three versions to solve scalability, latency, and maintenance challenges while supporting minute‑level dashboards for orders and vehicle analytics.

FlinkKafkaReal-time OLAP
0 likes · 19 min read
How Songguo Mobility Built a Real‑Time OLAP Platform with StarRocks: From 1.0 to 3.0
Architecture & Thinking
Architecture & Thinking
Jul 14, 2022 · Fundamentals

Understanding Message Queues: Execution Principles and Choosing the Right MQ

This article explains the fundamentals of message middleware, covering its architecture, components, communication models, advantages, common protocols, major open‑source solutions, detailed feature comparisons, and practical guidance for selecting the most suitable message queue based on system scale, business needs, performance, high availability, and operational considerations.

Distributed SystemsKafkaMQ
0 likes · 21 min read
Understanding Message Queues: Execution Principles and Choosing the Right MQ
Alibaba Cloud Native
Alibaba Cloud Native
Jul 12, 2022 · Big Data

How to Troubleshoot Kafka Message Loss with the Managed Retrieval Component

This article explains common Kafka message‑loss and duplicate‑consumption issues, introduces Alibaba Cloud's fully managed Kafka Retrieval Component, and provides step‑by‑step guidance—including enabling the service, using Tablestore for multi‑index and SQL searches—to help engineers quickly locate and verify missing or duplicated messages.

Big DataCloud NativeKafka
0 likes · 7 min read
How to Troubleshoot Kafka Message Loss with the Managed Retrieval Component
Selected Java Interview Questions
Selected Java Interview Questions
Jul 7, 2022 · Backend Development

Overview of RabbitMQ, RocketMQ, and Kafka: Components, Mechanisms, and Best Practices

This article provides a comprehensive introduction to three major message‑queue systems—RabbitMQ, RocketMQ, and Kafka—explaining their core components, exchange types, delivery guarantees, confirmation mechanisms, dead‑letter handling, high‑availability designs, load‑balancing strategies, and common troubleshooting techniques.

KafkaRocketMQ
0 likes · 32 min read
Overview of RabbitMQ, RocketMQ, and Kafka: Components, Mechanisms, and Best Practices
Code Ape Tech Column
Code Ape Tech Column
Jun 28, 2022 · Backend Development

Implementing Distributed WebSocket Messaging with Spring Cloud Alibaba, Redis, and Kafka

This article explains how to solve cross‑server WebSocket communication in a distributed Java application by using a message queue (Redis or Kafka) to broadcast messages, storing user‑node mappings, and providing complete Spring configuration, code examples, and a demo page for end‑to‑end testing.

BackendDistributed MessagingKafka
0 likes · 18 min read
Implementing Distributed WebSocket Messaging with Spring Cloud Alibaba, Redis, and Kafka
Su San Talks Tech
Su San Talks Tech
Jun 25, 2022 · Backend Development

How Kafka Broker Handles High‑Throughput Requests: Deep Dive into Its Network Architecture

This article thoroughly examines Kafka Broker's network architecture and request‑handling pipeline, exploring sequential, multi‑threaded, and Reactor‑based designs, detailing Acceptor and Processor threads, NIO mechanisms, and offering practical tuning tips to achieve high concurrency and performance.

KafkaReactor Patternnetwork architecture
0 likes · 21 min read
How Kafka Broker Handles High‑Throughput Requests: Deep Dive into Its Network Architecture
ShiZhen AI
ShiZhen AI
Jun 23, 2022 · Backend Development

How Kafka Clients Choose the Correct Listener in Multi‑Network Environments

This article explains how a Kafka client determines which broker listener to use when a cluster has multiple listeners, detailing the metadata update flow, server‑side filtering by listener name, and the impact of mixed listener names in the bootstrap.servers list.

Bootstrap ServersKafkaListeners
0 likes · 9 min read
How Kafka Clients Choose the Correct Listener in Multi‑Network Environments
MaGe Linux Operations
MaGe Linux Operations
Jun 19, 2022 · Big Data

Visualizing Kafka: Core Concepts Explained with Diagrams

This article provides a diagram‑driven walkthrough of Kafka’s fundamental concepts—including topics, partitions, producers, consumers, consumer groups, and cluster architecture—explaining how messages flow, are stored, and achieve reliability and ordering within a distributed streaming system.

Cluster ArchitectureKafkaPartitions
0 likes · 6 min read
Visualizing Kafka: Core Concepts Explained with Diagrams
IT Architects Alliance
IT Architects Alliance
Jun 13, 2022 · Backend Development

Understanding Message Queues (MQ): Concepts, Benefits, Common Implementations, and Advanced RabbitMQ Features

This article explains what a Message Queue (MQ) is, why it is used for traffic shaping, system decoupling, and asynchronous processing, reviews popular MQ products such as Kafka, ActiveMQ, RocketMQ and RabbitMQ, and details RabbitMQ's core concepts and advanced features like dead‑letter, delayed, idempotent, priority, and lazy queues.

Dead Letter QueueKafkaRabbitMQ
0 likes · 13 min read
Understanding Message Queues (MQ): Concepts, Benefits, Common Implementations, and Advanced RabbitMQ Features
IT Architects Alliance
IT Architects Alliance
Jun 7, 2022 · Backend Development

Improving Kafka Consumer Design: From Poll‑Then‑Process Loop to a Better Model

This article analyzes the shortcomings of the traditional poll‑then‑process loop in Kafka consumers, explains configuration pitfalls such as max.poll.interval.ms, and proposes a more robust architecture using work queues, a poller, an executor, and an offset manager to achieve at‑most‑once, at‑least‑once, and exactly‑once processing guarantees.

ConsumerKafkaPolling
0 likes · 15 min read
Improving Kafka Consumer Design: From Poll‑Then‑Process Loop to a Better Model
Efficient Ops
Efficient Ops
Jun 7, 2022 · Big Data

Visualizing Kafka: Core Concepts Explained with Diagrams

This article visually breaks down Kafka’s fundamental concepts—including topics, partitions, producers, consumers, consumer groups, and cluster architecture—so readers can grasp how messages flow, are stored, and achieve load balancing and ordering within a distributed streaming platform.

Distributed SystemsKafkaMessage Queue
0 likes · 6 min read
Visualizing Kafka: Core Concepts Explained with Diagrams
Zuoyebang Tech Team
Zuoyebang Tech Team
Jun 7, 2022 · Big Data

How Doris Powered Zuoyebang’s Real‑Time Data Warehouse for Faster Insights

Zuoyebang’s data team replaced fragmented, slow query solutions with Apache Doris, building a unified real‑time data warehouse that dramatically cut query latency from hours to seconds, streamlined data modeling, and improved reliability across diverse business scenarios, while integrating with Flink, Kafka, and ES via a unified API.

Apache DorisElasticsearchFlink
0 likes · 20 min read
How Doris Powered Zuoyebang’s Real‑Time Data Warehouse for Faster Insights
Architect's Journey
Architect's Journey
Jun 1, 2022 · Operations

How We Resolved a Kafka Consumer Production Outage Step by Step

The article recounts a production incident where a Kafka‑based consumer in a finance microservice hit thread‑pool exhaustion and slow‑query alerts, analyzes the root causes of async processing and bulk message bursts, and outlines a three‑phase remediation that includes data repair, switching to synchronous consumption, and request‑level batching to prevent future failures.

KafkaMessage QueueMicroservices
0 likes · 6 min read
How We Resolved a Kafka Consumer Production Outage Step by Step
vivo Internet Technology
vivo Internet Technology
May 31, 2022 · Big Data

Kafka Load Balancing and Cruise Control: Concepts, Manual Migration, and Deployment

Kafka’s server‑side load imbalance, caused by static replica placement on broker disks, makes manual replica migration infeasible at scale, but Cruise Control automates metric collection, analysis, and execution of fine‑grained rebalance plans—including broker de‑commissioning and leader dispersion—allowing large clusters to expand and operate efficiently.

Big DataCluster ManagementCruise Control
0 likes · 21 min read
Kafka Load Balancing and Cruise Control: Concepts, Manual Migration, and Deployment
macrozheng
macrozheng
May 31, 2022 · Backend Development

Can Redis Streams Replace Kafka? A Practical Comparison

This article compares Redis Streams with Kafka, highlighting Redis's simplicity, performance, and built‑in features like consumer groups, while discussing when each solution is appropriate for different project scales and requirements.

KafkaStreambackend-development
0 likes · 7 min read
Can Redis Streams Replace Kafka? A Practical Comparison
Code Ape Tech Column
Code Ape Tech Column
May 27, 2022 · Backend Development

Comparison of Four Popular Open-Source Distributed Message Queues: Kafka, ActiveMQ, RabbitMQ, and RocketMQ

This article introduces and compares four widely used open-source distributed message-queue systems—Kafka, ActiveMQ, RabbitMQ, and RocketMQ—detailing their architectures, deployment requirements, features, high-availability solutions, advantages, disadvantages, and provides guidance for selecting the appropriate middleware in large-scale applications.

ActiveMQDistributed SystemsKafka
0 likes · 38 min read
Comparison of Four Popular Open-Source Distributed Message Queues: Kafka, ActiveMQ, RabbitMQ, and RocketMQ
dbaplus Community
dbaplus Community
May 24, 2022 · Big Data

How Vipshop Replaced ELK with ClickHouse for a Scalable, Low‑Cost Log System

Vipshop’s Dragonfly log platform evolved from a costly 260‑node Elasticsearch cluster to a ClickHouse‑based architecture that uses a unified JSON format, vfilebeat ingestion, Flink parsing, and MergeTree storage to achieve high‑throughput writes, fast vectorized queries, flexible TTL management, and dramatically lower operational expenses.

EFKFlinkKafka
0 likes · 20 min read
How Vipshop Replaced ELK with ClickHouse for a Scalable, Low‑Cost Log System
Java High-Performance Architecture
Java High-Performance Architecture
May 23, 2022 · Big Data

Visualizing Kafka: Core Concepts, Topics, Partitions, and Architecture Explained

This article visually breaks down Kafka’s core concepts—including topics, partitions, producers, consumers, and cluster architecture—explaining how messages flow, how consumer groups achieve load balancing, and how ZooKeeper coordinates leaders and followers, providing a clear, illustrated guide for developers working with distributed streaming systems.

KafkaMessage QueuePartitions
0 likes · 6 min read
Visualizing Kafka: Core Concepts, Topics, Partitions, and Architecture Explained
G7 EasyFlow Tech Circle
G7 EasyFlow Tech Circle
May 20, 2022 · Backend Development

Securing Public‑Facing Kafka: Authentication, Configuration, and Scaling Strategies

This article shares G7 Tech’s practical experience of exposing Kafka to the public internet, covering encryption, AAA, three authentication schemes, listener configuration, scaling for massive topics with Kubernetes, storage optimization, and integration with the gmq management platform and Kafka‑REST.

AuthenticationKafkaKubernetes
0 likes · 10 min read
Securing Public‑Facing Kafka: Authentication, Configuration, and Scaling Strategies
vivo Internet Technology
vivo Internet Technology
May 18, 2022 · Backend Development

Kafka Cluster Fault Analysis: Root Cause and Cascading Failure Mechanism

A Kafka cluster at vivo suffered a total traffic drop across a resource group when a broker’s disk failed, because the default producer partitioner still hashed keys to the failed partition, exhausting client buffers and blocking all healthy partitions, prompting recommendations to avoid keys or use custom partitioners.

Distributed SystemsKafkafault-analysis
0 likes · 9 min read
Kafka Cluster Fault Analysis: Root Cause and Cascading Failure Mechanism
dbaplus Community
dbaplus Community
May 17, 2022 · Backend Development

How to Size a Kafka Cluster for Over 1 Billion Daily Requests

This article walks through a scenario‑driven capacity assessment for a production‑grade Kafka cluster, covering QPS calculations, storage needs, physical machine count, disk choices, memory, CPU, network bandwidth, deployment steps, and a final resource summary.

Cluster SizingKafkabackend-development
0 likes · 13 min read
How to Size a Kafka Cluster for Over 1 Billion Daily Requests
JavaEdge
JavaEdge
May 14, 2022 · Backend Development

Unveiling Kafka’s Triple‑High Architecture: Availability, Performance, and Concurrency

This article breaks down Kafka’s high‑availability, high‑performance, and high‑concurrency design, covering controller and leader election, replica and ISR mechanisms, ACK settings, the Reactor NIO model, zero‑copy I/O, compression, producer batching, memory‑pooling, and the multi‑layer network threading architecture.

KafkaNetwork ConcurrencyProducer Batching
0 likes · 21 min read
Unveiling Kafka’s Triple‑High Architecture: Availability, Performance, and Concurrency
DeWu Technology
DeWu Technology
May 9, 2022 · Backend Development

Common Issues and Solutions for Message Queue Middleware

Message‑queue middleware such as RabbitMQ, RocketMQ, ActiveMQ, and Kafka introduces challenges like ordering, loss, duplication, back‑pressure and delayed delivery, which can be mitigated by using single‑consumer queues or partitioning, enabling acknowledgments and replication, applying idempotent identifiers, scaling consumers, and employing dead‑letter or scheduling mechanisms.

KafkaMQMessage Queue
0 likes · 21 min read
Common Issues and Solutions for Message Queue Middleware
macrozheng
macrozheng
May 5, 2022 · Backend Development

Choosing the Right Message Queue: Kafka vs RabbitMQ vs RocketMQ vs ActiveMQ Explained

An in‑depth guide compares the four major message‑queue middleware—Kafka, RabbitMQ, RocketMQ and ActiveMQ—detailing their architectures, modes, evaluation criteria, strengths, weaknesses, and selection considerations to help engineers and interviewees answer critical MQ interview questions and make informed technology choices.

ActiveMQKafkaMessage Queue
0 likes · 47 min read
Choosing the Right Message Queue: Kafka vs RabbitMQ vs RocketMQ vs ActiveMQ Explained
Programmer DD
Programmer DD
May 3, 2022 · Backend Development

Mastering Spring Kafka Listeners: Containers, Concurrency, and @KafkaListener

This article explains how Spring Kafka’s KafkaMessageListenerContainer and ConcurrentMessageListenerContainer work, details their start-up process and internal consumer threads, shows how @KafkaListener bridges business logic, and provides configuration examples for single‑message and batch processing in Spring Boot.

KafkaMessage Listenerbackend-development
0 likes · 11 min read
Mastering Spring Kafka Listeners: Containers, Concurrency, and @KafkaListener
Code Ape Tech Column
Code Ape Tech Column
Apr 23, 2022 · Big Data

Kafka Fundamentals: Architecture, Replication, Partitioning, and Performance

This article provides a comprehensive overview of Kafka, covering its role as a message middleware, core concepts, architecture, replication management, partition handling, producer sending modes, partition assignment strategies, load balancing, reliability mechanisms, consumer models, controller election, and factors that affect its high throughput and potential message loss scenarios.

Distributed SystemsKafkaMessage Queue
0 likes · 29 min read
Kafka Fundamentals: Architecture, Replication, Partitioning, and Performance
Architecture Digest
Architecture Digest
Apr 18, 2022 · Operations

Zero‑Downtime Migration of 2000 Microservices to a Multi‑Cluster Managed Kafka at Wix

This article details how Wix migrated its 2000 microservices from a self‑hosted Kafka cluster to a multi‑cluster Confluent Cloud setup with zero downtime, describing the design decisions, partitioning strategy, Greyhound SDK usage, replication service, migration orchestrator, and best‑practice recommendations.

Confluent CloudGreyhound SDKKafka
0 likes · 17 min read
Zero‑Downtime Migration of 2000 Microservices to a Multi‑Cluster Managed Kafka at Wix
Tencent Cloud Developer
Tencent Cloud Developer
Apr 14, 2022 · Big Data

Understanding Kafka's Transition from ZooKeeper to KRaft: Architecture, Installation, Raft Algorithm, and Common Issues

The article explains how Kafka 3.x replaces ZooKeeper with the internal KRaft consensus layer, detailing the Raft‑based metadata storage, step‑by‑step KRaft cluster installation and configuration, and covering related concepts such as leader election, consumer‑group rebalancing, reliability settings, and performance optimizations.

InstallationKRaftKafka
0 likes · 41 min read
Understanding Kafka's Transition from ZooKeeper to KRaft: Architecture, Installation, Raft Algorithm, and Common Issues
Big Data Technology & Architecture
Big Data Technology & Architecture
Apr 11, 2022 · Big Data

Real-Time Data Warehouse Construction: Background, Objectives, Architecture, and Case Studies

This article explains the growing demand for real‑time data warehouses, outlines their objectives and layered architecture, and presents detailed case studies from Didi, Kuaishou, Tencent, Youzan and others, illustrating design choices, implementation challenges, and best practices for building scalable streaming data platforms.

FlinkKafkabig-data
0 likes · 48 min read
Real-Time Data Warehouse Construction: Background, Objectives, Architecture, and Case Studies
MaGe Linux Operations
MaGe Linux Operations
Apr 7, 2022 · Operations

Master ELK Stack: From Basics to Full‑Scale Log Management

This article introduces the ELK stack components, explains why centralized logging is essential, outlines core log‑system features, compares three ELK architectures, provides step‑by‑step installation and configuration for Filebeat, Logstash, Elasticsearch and Kibana, and demonstrates a complete pipeline using Kafka with code examples and diagrams.

ELKElasticsearchFilebeat
0 likes · 25 min read
Master ELK Stack: From Basics to Full‑Scale Log Management
JavaEdge
JavaEdge
Apr 6, 2022 · Big Data

When Does Kafka Lose Data? Proven Strategies to Prevent Message Loss

This article explains Kafka's message delivery semantics, identifies the exact scenarios where data can be lost in producer, broker, and consumer components, and provides concrete configuration and coding practices to ensure reliable, at‑least‑once or exactly‑once delivery in production environments.

BrokerConsumerData Reliability
0 likes · 18 min read
When Does Kafka Lose Data? Proven Strategies to Prevent Message Loss
21CTO
21CTO
Apr 4, 2022 · Backend Development

Choosing the Right Message Broker: RabbitMQ vs Kafka vs Redis for Microservices

This article compares three popular open‑source message brokers—RabbitMQ, Kafka, and Redis—examining their scalability, persistence, consumer models, and ideal use cases to help developers select the most suitable asynchronous communication solution for their microservice architecture.

BackendKafkaMicroservices
0 likes · 8 min read
Choosing the Right Message Broker: RabbitMQ vs Kafka vs Redis for Microservices
Ziru Technology
Ziru Technology
Apr 1, 2022 · Fundamentals

How Kafka’s Hierarchical Timing Wheel Optimizes Task Scheduling

This article explains the time‑wheel algorithm, from its basic circular‑buffer principle to simple, round‑based, and hierarchical variants, and shows how Kafka implements a multi‑level timing wheel to achieve efficient, low‑memory delayed task execution.

Kafkabackend algorithmshierarchical timer
0 likes · 13 min read
How Kafka’s Hierarchical Timing Wheel Optimizes Task Scheduling
IT Architects Alliance
IT Architects Alliance
Mar 30, 2022 · Backend Development

Designing a Scalable Event-Driven Microservices Architecture for E‑Commerce

This article guides readers through the evolution from a monolithic e‑commerce application to a highly available, scalable, low‑latency event‑driven microservices architecture, covering design patterns, principles, API gateway, BFF, service aggregation, CQRS, event sourcing, and technology stack choices such as Kafka and Cassandra.

CQRSEvent-drivenKafka
0 likes · 21 min read
Designing a Scalable Event-Driven Microservices Architecture for E‑Commerce
dbaplus Community
dbaplus Community
Mar 27, 2022 · Backend Development

When to Choose Kafka Over RabbitMQ: A Six‑Scenario Comparison

This article compares Kafka and RabbitMQ across six real‑world scenarios—message ordering, routing, timeout handling, persistence, error handling, and throughput—to help engineers decide which message queue best fits their system requirements.

Backend ArchitectureKafkaMessage Ordering
0 likes · 12 min read
When to Choose Kafka Over RabbitMQ: A Six‑Scenario Comparison
Su San Talks Tech
Su San Talks Tech
Mar 27, 2022 · Big Data

Top 10 Advanced Kafka Interview Questions with In‑Depth Answers

This article provides a comprehensive collection of advanced Kafka interview questions covering core architecture, storage mechanisms, replication, leader election, controller responsibilities, consumer group rebalancing, message semantics, partition assignment strategies, performance tuning, and practical tips for handling large message backlogs.

ConsumerGroupKafkaReplication
0 likes · 33 min read
Top 10 Advanced Kafka Interview Questions with In‑Depth Answers
StarRocks
StarRocks
Mar 23, 2022 · Databases

Accelerating Zepp Health’s Analytics with StarRocks: An OLAP Case Study

Facing inflexible point‑lookup limits and slow query times on HBase, Zepp Health redesigned its massive event‑tracking data pipeline—migrating ingestion through Kafka, Flink, and Hudi to a StarRocks‑based OLAP layer—achieving sub‑100 ms average query latency, 20 % storage savings, and dramatically faster multi‑dimensional analytics.

Big DataFlinkHudi
0 likes · 9 min read
Accelerating Zepp Health’s Analytics with StarRocks: An OLAP Case Study
dbaplus Community
dbaplus Community
Mar 22, 2022 · Operations

How Meituan Scales Kafka to 7,500 Nodes: Real-World Optimizations and Lessons

Meituan’s data platform runs Kafka on over 7,500 machines, handling daily traffic exceeding 21 PB, and tackles latency, slow nodes, and massive cluster management through layered optimizations—including disk balancing, pipeline acceleration, fetcher isolation, cgroup isolation, SSD caching, isolation strategies, full‑link monitoring, lifecycle management, and TOR disaster recovery.

Data PlatformKafkaLarge-scale systems
0 likes · 23 min read
How Meituan Scales Kafka to 7,500 Nodes: Real-World Optimizations and Lessons
Top Architect
Top Architect
Mar 15, 2022 · Backend Development

Understanding Apache Kafka Replication Mechanism and Its Design Principles

This article explains Apache Kafka's replication mechanism, covering the benefits of data redundancy, the roles of leader and follower replicas, the In‑Sync Replica (ISR) concept, unclean leader election, and how these design choices affect availability, consistency, and scalability in distributed systems.

ISRKafkaReplication
0 likes · 11 min read
Understanding Apache Kafka Replication Mechanism and Its Design Principles
IT Architects Alliance
IT Architects Alliance
Mar 14, 2022 · Big Data

Comprehensive Guide to Kafka Architecture, Core Concepts, and Production Deployment

This article provides an in‑depth overview of Kafka, covering why messaging systems are needed, core concepts, cluster architecture, performance optimizations such as sequential disk writes and zero‑copy, hardware sizing, replication, consumer groups, offset management, rebalance strategies, and practical deployment and operational guidelines.

Big DataCluster DeploymentDistributed Messaging
0 likes · 35 min read
Comprehensive Guide to Kafka Architecture, Core Concepts, and Production Deployment
Alibaba Cloud Developer
Alibaba Cloud Developer
Mar 7, 2022 · Big Data

How China Mobile’s Real‑Time Computing Platform Scales Billions of Events with Flink

This article details China Mobile (Suzhou) Software Technology's evolution from Storm to Flink for real‑time computing, its multi‑version engine and log‑retrieval designs, signal‑business data pipeline optimizations, stability practices around ZooKeeper, and future directions in resource scaling and data‑lake integration.

FlinkKafkaReal-Time
0 likes · 12 min read
How China Mobile’s Real‑Time Computing Platform Scales Billions of Events with Flink
dbaplus Community
dbaplus Community
Mar 2, 2022 · Big Data

How Real‑Time Data Warehouses Power Modern Business: Architecture, Cases, and Best Practices

This article explores the growing demand for real‑time data warehouses, compares them with traditional offline warehouses, and presents detailed architectures, layer designs, naming conventions, and case studies from companies like Didi, Kuaishou, Tencent, and Youzan, highlighting challenges, solutions, and performance optimizations.

Big Data ArchitectureFlinkIceberg
0 likes · 47 min read
How Real‑Time Data Warehouses Power Modern Business: Architecture, Cases, and Best Practices
Architect's Journey
Architect's Journey
Mar 2, 2022 · Backend Development

Interview Basics: How to Guarantee Message Reliability in MQ

The article explains how to achieve 100% message delivery and consumption in MQ systems by covering producer acknowledgments, broker persistence mechanisms, and consumer idempotency, with detailed comparisons of RabbitMQ and Kafka implementations and configuration tips.

ACKKafkaMessage Queue
0 likes · 13 min read
Interview Basics: How to Guarantee Message Reliability in MQ
Efficient Ops
Efficient Ops
Feb 23, 2022 · Operations

Why a Single Kafka Broker Crash Can Halt All Consumers – The HA Explained

An in‑depth look at Kafka’s high‑availability architecture reveals how multi‑replica redundancy, ISR mechanisms, and the configuration of the __consumer_offset topic interact, explaining why a single broker failure can render the entire cluster unusable and how to properly configure replication and ack settings to prevent it.

ACKConsumer OffsetISR
0 likes · 10 min read
Why a Single Kafka Broker Crash Can Halt All Consumers – The HA Explained
Top Architect
Top Architect
Feb 20, 2022 · Big Data

Understanding Kafka’s Core Design: Topics, Partitions, Replicas, Consumer Groups, and Performance Optimizations

This article explains Kafka’s fundamental architecture—including topics, partitions, replication, consumer groups, cluster coordination, and performance techniques such as sequential writes, zero‑copy, and log segmentation—to help readers improve their design and coding skills for large‑scale messaging systems.

Kafkaperformance optimization
0 likes · 12 min read
Understanding Kafka’s Core Design: Topics, Partitions, Replicas, Consumer Groups, and Performance Optimizations
Tencent Cloud Developer
Tencent Cloud Developer
Feb 15, 2022 · Backend Development

Message Queue Middleware Selection Guide: Kafka vs Pulsar vs RocketMQ vs RabbitMQ vs NSQ

This guide compares Kafka, Pulsar, RocketMQ, RabbitMQ, and NSQ across architecture, consumption models, delay and dead‑letter support, persistence, ordering, multi‑tenant isolation, and performance, recommending Kafka for high‑throughput log processing, Pulsar for transactional delayed messaging and disaster recovery, and NSQ for Go‑centric custom solutions.

Distributed SystemsKafkaMessage Queue
0 likes · 26 min read
Message Queue Middleware Selection Guide: Kafka vs Pulsar vs RocketMQ vs RabbitMQ vs NSQ