BestHub
Sign in
Tagged articles
71 articles
Page 1 of 1
Architect Chen
Architect Chen
Nov 4, 2025 · Backend Development

Mastering RocketMQ: Architecture, Workflow, and Real‑World Use Cases

This article provides a comprehensive overview of RocketMQ, covering its core components (Producer, Broker, Consumer, NameServer), detailed architecture diagrams, step‑by‑step message production and consumption processes, and typical scenarios such as asynchronous decoupling, traffic shaping, transactional messaging, and event‑driven logging.

Backend ArchitectureBrokerConsumer
0 likes · 6 min read
Mastering RocketMQ: Architecture, Workflow, and Real‑World Use Cases
Cognitive Technology Team
Cognitive Technology Team
Apr 8, 2025 · Backend Development

Design and Implementation of RocketMQ NameServer: Core Functions, Architecture, and Optimization Strategies

The article explains RocketMQ NameServer's lightweight, stateless design, its core routing and metadata management functions, AP‑oriented architecture, fault‑tolerant mechanisms, scalability features, and practical optimization techniques for high availability and low operational cost.

Distributed MessagingNameServerRocketMQ
0 likes · 6 min read
Design and Implementation of RocketMQ NameServer: Core Functions, Architecture, and Optimization Strategies
Linux Ops Smart Journey
Linux Ops Smart Journey
Feb 27, 2025 · Information Security

Build Military‑Grade Kafka Security in Four Easy Steps

This guide walks you through four progressive stages—basic configuration, SSL encryption, SCRAM authentication, and combined SSL+SASL—showing how to harden Kafka with certificates, keystores, and client settings to achieve financial‑level protection.

Distributed MessagingKafkaSASL
0 likes · 9 min read
Build Military‑Grade Kafka Security in Four Easy Steps
Sanyou's Java Diary
Sanyou's Java Diary
Jan 23, 2025 · Backend Development

How RocketMQ Achieves High Performance and Scalability with Queues, Brokers, and mmap

This article explains how RocketMQ tackles synchronous registration bottlenecks, tight coupling, and traffic‑burst risks by introducing an intermediate queue layer, designing a durable high‑availability broker, leveraging page cache and mmap for zero‑copy I/O, and using a nameserver for automatic routing, ultimately delivering a high‑throughput, low‑latency messaging system.

Distributed MessagingRocketMQbroker design
0 likes · 27 min read
How RocketMQ Achieves High Performance and Scalability with Queues, Brokers, and mmap
Architecture & Thinking
Architecture & Thinking
Dec 18, 2024 · Backend Development

Mastering RocketMQ: Core Concepts of Topics, Queues, and Messaging

This article explains RocketMQ’s fundamental architecture, detailing the definitions, roles, and interactions of topics, message queues, messages, producers, consumers, consumer groups, and subscription relationships, and highlights how these components work together to achieve high performance, availability, and scalability.

Backend DevelopmentDistributed MessagingMessage Queue
0 likes · 7 min read
Mastering RocketMQ: Core Concepts of Topics, Queues, and Messaging
Architect
Architect
Sep 7, 2024 · Backend Development

How to Reset RocketMQ Consumer Offsets Using Message Backtracking – A Step‑by‑Step Guide

When a SMS service fails and messages need to be re‑sent, this article shows how to use RocketMQ's message backtracking feature to reset consumer offsets, verify the reset, and explore the internal workflow and alternative pull‑consumer strategies with concrete code examples and detailed analysis.

Distributed MessagingJavaMessage Backtracking
0 likes · 21 min read
How to Reset RocketMQ Consumer Offsets Using Message Backtracking – A Step‑by‑Step Guide
360 Tech Engineering
360 Tech Engineering
May 10, 2024 · Cloud Native

Practical Experience of Deploying and Optimizing Apache Pulsar on Kubernetes at 360

This article shares the architecture design, cluster deployment, storage selection, multi‑region mode, service discovery, performance tuning, monitoring, alerting, and future plans of a production‑grade Apache Pulsar platform running on Kubernetes, providing valuable insights for engineers adopting Pulsar.

Apache PulsarCloud NativeDistributed Messaging
0 likes · 17 min read
Practical Experience of Deploying and Optimizing Apache Pulsar on Kubernetes at 360
360 Smart Cloud
360 Smart Cloud
May 9, 2024 · Cloud Native

Practical Experience of Deploying and Optimizing Apache Pulsar on Kubernetes at 360

This article shares 360's practical experience deploying Apache Pulsar on Kubernetes, covering architecture design, storage choices, multi‑cluster modes, service discovery, performance tuning, monitoring, alerting, and future plans, offering valuable guidance for engineers building cloud‑native messaging platforms.

Apache PulsarCloud NativeDistributed Messaging
0 likes · 19 min read
Practical Experience of Deploying and Optimizing Apache Pulsar on Kubernetes at 360
JD Retail Technology
JD Retail Technology
Nov 1, 2023 · Big Data

Kafka: Usage Scenarios, Core Concepts, Installation, Configuration, and Practical Operations

This article provides a comprehensive guide to Apache Kafka, covering its typical use cases, fundamental terminology, step‑by‑step installation and environment preparation, detailed configuration of brokers and topics, cluster deployment, producer‑consumer commands, and a Java client example for sending messages.

Distributed MessagingInstallationJava Client
0 likes · 21 min read
Kafka: Usage Scenarios, Core Concepts, Installation, Configuration, and Practical Operations
Tencent Cloud Middleware
Tencent Cloud Middleware
Jul 21, 2023 · Backend Development

How Pulsar GEO Replication Keeps Subscription State in Sync Across Clusters

This article explains how Apache Pulsar's GEO Replication synchronizes subscription state between primary and backup clusters, detailing the challenges of ledger mismatches, the two‑step mapping process for MessageIds, and the mechanisms that ensure seamless failover without duplicate consumption.

Apache PulsarDistributed MessagingGEO Replication
0 likes · 8 min read
How Pulsar GEO Replication Keeps Subscription State in Sync Across Clusters
Code Ape Tech Column
Code Ape Tech Column
Jul 21, 2023 · Backend Development

Implementing Distributed WebSocket Messaging with Redis and Kafka in Spring

This article explains how to enable cross‑node WebSocket communication in a distributed Spring application by publishing messages to a Redis or Kafka topic, tracking user connections, and routing messages to the appropriate server instance, complete with full code examples and configuration details.

Backend DevelopmentDistributed MessagingKafka
0 likes · 16 min read
Implementing Distributed WebSocket Messaging with Redis and Kafka in Spring
FunTester
FunTester
May 5, 2023 · Big Data

Kafka Client API Performance Testing with Producer and Consumer Examples

This article introduces Kafka, a high‑performance distributed messaging system, and provides step‑by‑step Java/Groovy examples for configuring a producer and consumer, demonstrating how to benchmark their throughput using the FunQpsConcurrent framework, along with necessary Gradle dependencies and server setup instructions.

ConsumerDistributed MessagingJava
0 likes · 8 min read
Kafka Client API Performance Testing with Producer and Consumer Examples
MaGe Linux Operations
MaGe Linux Operations
Dec 30, 2022 · Big Data

Mastering Kafka: Core Concepts, Architecture, and Performance Optimizations

This comprehensive guide explores Kafka as a distributed messaging middleware, detailing its core concepts, architecture, producer and consumer mechanisms, configuration options, Zookeeper integration, controller responsibilities, network model, performance optimizations such as zero‑copy, page‑cache usage, batching, compression, and partition concurrency.

Distributed MessagingKafkaZooKeeper
0 likes · 41 min read
Mastering Kafka: Core Concepts, Architecture, and Performance Optimizations
Tencent Cloud Developer
Tencent Cloud Developer
Nov 24, 2022 · Backend Development

Kafka Stability Best Practices: Prevention, Monitoring, and Fault Resolution

This guide outlines Kafka stability best practices across three phases—pre‑prevention with tuning, producer/consumer guidelines, and cluster configuration; runtime monitoring using white‑box and black‑box metrics and alerts; and fault resolution strategies for backlogs, consumption blocks, and message loss, plus cost control and idempotence techniques.

Backend DevelopmentDistributed MessagingKafka
0 likes · 29 min read
Kafka Stability Best Practices: Prevention, Monitoring, and Fault Resolution
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Oct 27, 2022 · Backend Development

Understanding Kafka: Architecture, Principles, Features, and Use Cases

This article explains Kafka's distributed publish‑subscribe architecture, detailing its core components, underlying mechanisms with Zookeeper coordination, key features such as high throughput and fault tolerance, and common application scenarios like log collection, user activity tracking, and stream processing.

Backend ArchitectureDistributed MessagingKafka
0 likes · 5 min read
Understanding Kafka: Architecture, Principles, Features, and Use Cases
IT Architects Alliance
IT Architects Alliance
Aug 30, 2022 · Big Data

Understanding Kafka: Architecture, Topics, Partitions, Producers, Consumers, Offsets, Transactions, and Configuration

This article provides a comprehensive overview of Apache Kafka, explaining its distributed message‑queue architecture, the role of topics and partitions, producer and consumer workflows, leader election, offset management, consumer‑group rebalancing, delivery semantics, transaction processing, file organization, and key configuration settings.

Big DataDistributed MessagingKafka
0 likes · 17 min read
Understanding Kafka: Architecture, Topics, Partitions, Producers, Consumers, Offsets, Transactions, and Configuration
Top Architect
Top Architect
Aug 28, 2022 · Big Data

Understanding Kafka Architecture: Topics, Partitions, Producers, Consumers, Offsets, and Transactions

Kafka is a high‑performance distributed message queue that uses topics, partitions, and replicas for scalability, with producers writing messages, consumers reading them via consumer groups, and advanced features such as offset management, delivery semantics, rebalancing, and exactly‑once transactional processing, all coordinated by Zookeeper.

ConsumersDistributed MessagingExactly-Once
0 likes · 17 min read
Understanding Kafka Architecture: Topics, Partitions, Producers, Consumers, Offsets, and Transactions
Ops Development Stories
Ops Development Stories
Aug 19, 2022 · Big Data

Master Kafka: From Core Concepts to Advanced Operations and Performance Tuning

This comprehensive guide explains Kafka’s origins, core architecture, data structures, write and read workflows, operational commands for topic and consumer‑group management, and practical performance‑tuning tips such as disk layout, JVM settings, flush policies, and log retention, providing a complete reference for engineers working with distributed streaming platforms.

Data StreamingDistributed MessagingKafka
0 likes · 32 min read
Master Kafka: From Core Concepts to Advanced Operations and Performance Tuning
Code Ape Tech Column
Code Ape Tech Column
Jun 28, 2022 · Backend Development

Implementing Distributed WebSocket Messaging with Spring Cloud Alibaba, Redis, and Kafka

This article explains how to solve cross‑server WebSocket communication in a distributed Java application by using a message queue (Redis or Kafka) to broadcast messages, storing user‑node mappings, and providing complete Spring configuration, code examples, and a demo page for end‑to‑end testing.

BackendDistributed MessagingJava
0 likes · 18 min read
Implementing Distributed WebSocket Messaging with Spring Cloud Alibaba, Redis, and Kafka
IT Architects Alliance
IT Architects Alliance
Mar 14, 2022 · Big Data

Comprehensive Guide to Kafka Architecture, Core Concepts, and Production Deployment

This article provides an in‑depth overview of Kafka, covering why messaging systems are needed, core concepts, cluster architecture, performance optimizations such as sequential disk writes and zero‑copy, hardware sizing, replication, consumer groups, offset management, rebalance strategies, and practical deployment and operational guidelines.

Big DataCluster DeploymentDistributed Messaging
0 likes · 35 min read
Comprehensive Guide to Kafka Architecture, Core Concepts, and Production Deployment
Tencent Cloud Developer
Tencent Cloud Developer
Jan 11, 2022 · Cloud Native

Apache Pulsar: Architecture, Topics, Producers, Consumers, and Storage

Apache Pulsar is a cloud‑native distributed messaging system that combines stateless brokers, a BookKeeper storage cluster, and ZooKeeper metadata to route messages from producers to consumers across scalable partitioned or non‑partitioned topics, supporting multiple subscription models, producer access modes, routing strategies, batching, acknowledgments, delayed delivery, and deduplication.

Apache PulsarBookKeeperCloud Native
0 likes · 26 min read
Apache Pulsar: Architecture, Topics, Producers, Consumers, and Storage
Su San Talks Tech
Su San Talks Tech
Dec 28, 2021 · Big Data

What Makes Kafka the Backbone of Real‑Time Big Data Processing?

This article provides a comprehensive overview of Apache Kafka, covering its distributed architecture, key advantages and drawbacks, the role of ZooKeeper, message delivery semantics, partitioning strategies, storage mechanisms, and performance optimizations such as zero‑copy and batch processing, all essential for high‑throughput real‑time data pipelines.

Big DataDistributed MessagingStreaming
0 likes · 23 min read
What Makes Kafka the Backbone of Real‑Time Big Data Processing?
Tencent Cloud Developer
Tencent Cloud Developer
Dec 24, 2021 · Cloud Native

Deep Dive into Apache Pulsar Message Acknowledgment Mechanism

Apache Pulsar ensures reliable delivery by letting consumers acknowledge messages individually, cumulatively, or within batches, while also supporting negative acknowledgments and sophisticated tracking mechanisms that batch ack requests, use BitSet for batch indexes, and employ time‑wheel‑based redelivery for unacknowledged messages.

Apache PulsarConsumer TrackerDistributed Messaging
0 likes · 10 min read
Deep Dive into Apache Pulsar Message Acknowledgment Mechanism
Java Architect Essentials
Java Architect Essentials
Oct 14, 2021 · Backend Development

Analysis of Alipay Android Wallet System Architecture and MetaQ Distributed Messaging Middleware

This article presents a detailed overview of Alipay's Android wallet system architecture, illustrating its core components such as settlement, customer service, fund processing, and accounting, and further examines the open‑source MetaQ (Metamorphosis) distributed messaging middleware, highlighting its features, advantages over Kafka, and suitable application scenarios.

AlipayAndroidBackend
0 likes · 5 min read
Analysis of Alipay Android Wallet System Architecture and MetaQ Distributed Messaging Middleware
Java Architect Essentials
Java Architect Essentials
Jul 27, 2021 · Backend Development

Kafka Overview: Architecture, Core Features, and Operational Details

This article provides a comprehensive technical overview of Apache Kafka, covering its distributed messaging architecture, key features such as high‑throughput read/write, replication, partitioning, consumer group mechanics, offset management, rebalance processes, and practical code examples for synchronous and asynchronous offset commits.

Consumer OffsetsDistributed MessagingKafka
0 likes · 22 min read
Kafka Overview: Architecture, Core Features, and Operational Details
Big Data Technology & Architecture
Big Data Technology & Architecture
Jul 27, 2021 · Big Data

An Introduction to Apache Pulsar: Core Concepts, Architecture, and Key Features

Apache Pulsar is a cloud‑native distributed messaging platform that combines messaging, storage, and lightweight compute, featuring multi‑tenant support, geo‑replication, and high throughput, and this article introduces its core concepts, architecture components such as brokers, BookKeeper, ZooKeeper, and key design features.

Apache PulsarBookKeeperCloud Native
0 likes · 13 min read
An Introduction to Apache Pulsar: Core Concepts, Architecture, and Key Features
MaGe Linux Operations
MaGe Linux Operations
Jun 7, 2021 · Backend Development

Understanding Apache RocketMQ: Architecture, Components, and Key Concepts

This article explains Apache RocketMQ's high‑performance distributed messaging architecture, detailing its core components—Producer, Broker, Consumer, and Namesrv—along with routing metadata, message ordering, priority handling, parallelism, and essential concepts such as topics, groups, tags, and reliability mechanisms.

Apache RocketMQBackend ArchitectureDistributed Messaging
0 likes · 14 min read
Understanding Apache RocketMQ: Architecture, Components, and Key Concepts
ITFLY8 Architecture Home
ITFLY8 Architecture Home
May 31, 2021 · Backend Development

Why Apache Pulsar Beats Kafka and RocketMQ for Scalable Messaging Platforms

This article details how Lakala built a distributed, cloud‑native messaging platform using Apache Pulsar, covering functional requirements, architectural advantages, performance testing, and real‑world integration scenarios such as OGG adapters, TiDB pipelines, OpenMessaging, custom sources, functions, Flink connectors, and future plans.

Apache PulsarBackend ArchitectureDistributed Messaging
0 likes · 18 min read
Why Apache Pulsar Beats Kafka and RocketMQ for Scalable Messaging Platforms
Tencent Cloud Developer
Tencent Cloud Developer
May 27, 2021 · Big Data

An Introduction to Kafka: Architecture, Core Components, Service Governance, Performance Optimizations, and Installation Guide

Kafka is a high‑throughput distributed publish‑subscribe system that uses brokers, topics, partitions, offsets, producers, consumers, and Zookeeper for metadata and leader election, offering fast sequential disk writes, page‑cache zero‑copy transfers, ISR‑based replication, and includes step‑by‑step installation of JDK, Zookeeper, and Kafka.

Big DataDistributed MessagingInstallation
0 likes · 11 min read
An Introduction to Kafka: Architecture, Core Components, Service Governance, Performance Optimizations, and Installation Guide
Full-Stack Internet Architecture
Full-Stack Internet Architecture
Apr 8, 2021 · Backend Development

Kafka Interview Guide: Concepts, Architecture, Configuration, and Performance

This article provides a comprehensive overview of Kafka, covering its role as a distributed messaging middleware, core concepts, architecture components, common interview questions, command‑line tools, producer and consumer configurations, high‑availability mechanisms, delivery semantics, and performance optimizations for backend developers.

ConsumerDistributed MessagingKafka
0 likes · 20 min read
Kafka Interview Guide: Concepts, Architecture, Configuration, and Performance
Tencent Cloud Middleware
Tencent Cloud Middleware
Mar 29, 2021 · Cloud Native

Mastering Apache Pulsar Geo‑Replication: Modes, Configs, and Common Pitfalls

Apache Pulsar’s built‑in Geo‑Replication lets multiple clusters across different regions synchronize data, offering both synchronous and asynchronous modes; this guide explains the three asynchronous patterns—full‑mesh, unidirectional, and failover—detailing required configurations, operational principles, and current limitations.

Apache PulsarCloud NativeCluster Configuration
0 likes · 10 min read
Mastering Apache Pulsar Geo‑Replication: Modes, Configs, and Common Pitfalls
Programmer DD
Programmer DD
Mar 29, 2021 · Big Data

Mastering Kafka: High‑Throughput Distributed Messaging Explained

This comprehensive guide introduces Kafka as a high‑throughput, distributed, publish‑subscribe messaging system, detailing its core concepts, architecture, features, replication, log management, reliability guarantees, and typical use cases such as log collection, real‑time analytics, and cross‑cluster mirroring.

Big DataDistributed MessagingKafka
0 likes · 15 min read
Mastering Kafka: High‑Throughput Distributed Messaging Explained
Sohu Tech Products
Sohu Tech Products
Jan 13, 2021 · Operations

Introducing MQCloud: A One‑Stop Service Platform for RocketMQ Monitoring, Management, and Operations

This article explains RocketMQ’s core features, the operational challenges of using its console, and how the MQCloud platform was designed to separate business and admin roles, provide comprehensive monitoring, automated deployment, security hardening, and a customized client, ultimately turning operational pain into a scalable, open‑source solution.

Distributed MessagingMQCloudRocketMQ
0 likes · 15 min read
Introducing MQCloud: A One‑Stop Service Platform for RocketMQ Monitoring, Management, and Operations
Laravel Tech Community
Laravel Tech Community
Nov 19, 2020 · Backend Development

Apache Pulsar Go Client 0.3.0 Release Highlights

The Apache Pulsar Go client, a Go‑language SDK for the cloud‑native distributed messaging platform, has been updated to version 0.3.0, adding Key Shared support, schema capabilities for producers and consumers, and configurable reconnection retry attempts.

Apache PulsarBackendCloud Native
0 likes · 2 min read
Apache Pulsar Go Client 0.3.0 Release Highlights
Top Architect
Top Architect
Nov 13, 2020 · Backend Development

Alipay System Architecture Overview and the Open‑Source Distributed Message Middleware Metamorphosis (MetaQ)

This article presents a detailed overview of Alipay's multi‑layer system architecture, explains the design and capabilities of its open‑source Java‑based distributed message middleware Metamorphosis (MetaQ), and discusses why it was created as an alternative to Kafka and the scenarios where it excels.

AlipayDistributed MessagingJava
0 likes · 6 min read
Alipay System Architecture Overview and the Open‑Source Distributed Message Middleware Metamorphosis (MetaQ)
Programmer DD
Programmer DD
Oct 29, 2020 · Backend Development

Master Kafka Interview Questions: Architecture, Configurations, and Best Practices

This article provides a comprehensive overview of Kafka as a distributed messaging middleware, covering its core concepts, architecture, producer and consumer mechanics, common interview questions, configuration options, high‑availability guarantees, and performance optimizations for backend developers.

Backend DevelopmentConsumerDistributed Messaging
0 likes · 20 min read
Master Kafka Interview Questions: Architecture, Configurations, and Best Practices
21CTO
21CTO
Oct 20, 2020 · Backend Development

How RocketMQ Transformed Cainiao’s Logistics System for Double‑11 Peaks

The article examines the scaling challenges of Cainiao's logistics platform during Alibaba's Double‑11 shopping festivals and explains how adopting RocketMQ for asynchronous decoupling dramatically reduced costs, stabilized the transaction‑logistics chain, and enabled sustainable high‑throughput processing.

AlibabaBackend ArchitectureDistributed Messaging
0 likes · 6 min read
How RocketMQ Transformed Cainiao’s Logistics System for Double‑11 Peaks
ITPUB
ITPUB
Jul 10, 2020 · Backend Development

How RocketMQ Cut Costs and Stabilized Alibaba Logistics During Double‑11

Facing soaring order peaks each Double‑11, Alibaba's logistics arm re‑engineered its architecture with RocketMQ, using asynchronous decoupling to handle payment‑order traffic, dramatically reduce hardware spend, and keep logistics QPS stable without sacrificing user experience.

AlibabaBackend ArchitectureCost reduction
0 likes · 7 min read
How RocketMQ Cut Costs and Stabilized Alibaba Logistics During Double‑11
21CTO
21CTO
Jun 13, 2020 · Backend Development

How RocketMQ Transformed Alibaba’s Logistics System for Double‑11 Peaks

This article examines the logistical challenges faced by Alibaba's Cainiao platform during Double‑11, explains why traditional databases and Redis fall short, and shows how adopting RocketMQ for asynchronous decoupling dramatically reduced costs and stabilized high‑volume order processing.

Backend ArchitectureDistributed MessagingRocketMQ
0 likes · 6 min read
How RocketMQ Transformed Alibaba’s Logistics System for Double‑11 Peaks
Top Architect
Top Architect
Mar 10, 2020 · Backend Development

Overview of Alipay System Architecture and the Open‑Source Distributed Messaging Middleware Metamorphosis (MetaQ)

This article presents an overview of Alipay’s comprehensive system architecture and introduces MetaQ (Metamorphosis), an open‑source Java‑based distributed messaging middleware, detailing its features, advantages over Kafka, and suitable high‑throughput, reliable messaging scenarios.

AlipayBackendDistributed Messaging
0 likes · 5 min read
Overview of Alipay System Architecture and the Open‑Source Distributed Messaging Middleware Metamorphosis (MetaQ)
Big Data Technology Architecture
Big Data Technology Architecture
Feb 26, 2020 · Big Data

Comprehensive Guide to Kafka Architecture, Messaging Mechanisms, Replication, Controllers, and Consumer Rebalance

This article provides an in‑depth yet approachable overview of Kafka's core concepts—including its architecture, terminology, message‑sending pipeline, replication strategy, controller role, and consumer group rebalance mechanisms—helping readers quickly grasp how Kafka works as a high‑throughput distributed messaging and streaming platform.

Consumer RebalanceDistributed MessagingKafka
0 likes · 21 min read
Comprehensive Guide to Kafka Architecture, Messaging Mechanisms, Replication, Controllers, and Consumer Rebalance
MaGe Linux Operations
MaGe Linux Operations
Dec 6, 2019 · Backend Development

How Xiaomi’s Talos Redefined Distributed Messaging for Massive Scale

Xiaomi’s Talos, a self‑developed distributed message queue, tackles the limitations of Kafka by separating storage and compute on HDFS, introducing stateless scaling, advanced consistency, partition delay allocation, and extensive performance and resource optimizations to support trillions of daily messages and multi‑tenant workloads.

Distributed MessagingPerformance OptimizationStorage Compute Separation
0 likes · 16 min read
How Xiaomi’s Talos Redefined Distributed Messaging for Massive Scale
ITPUB
ITPUB
Dec 2, 2019 · Backend Development

How Xiaomi Built Talos: A Scalable, Stateless Message Queue for Billions of Events

This article details Xiaomi's journey from Kafka 0.8 to the home‑grown Talos system, covering business motivations, storage‑compute separation architecture, key challenges such as tail‑read and consistency, and extensive performance, resource, and platform optimizations that enable a high‑throughput, multi‑tenant messaging service.

Distributed MessagingHDFSMessage Queue
0 likes · 16 min read
How Xiaomi Built Talos: A Scalable, Stateless Message Queue for Billions of Events
360 Quality & Efficiency
360 Quality & Efficiency
Aug 8, 2019 · Big Data

An Introduction to Kafka: Architecture, Design Principles, and Common Issues

This article introduces Kafka, covering its definition, core concepts such as topics, partitions, offsets, producers and consumers, typical use cases, underlying design principles including message‑partition allocation and retention policies, processing mechanisms, and common troubleshooting questions for real‑world deployments.

Big DataDistributed MessagingKafka
0 likes · 7 min read
An Introduction to Kafka: Architecture, Design Principles, and Common Issues
Big Data Technology & Architecture
Big Data Technology & Architecture
Jun 11, 2019 · Backend Development

Introduction to Apache Pulsar: Architecture, Core Concepts, and Consumption Modes

This article provides an overview of Apache Pulsar, detailing its enterprise‑grade features, architectural components such as brokers, BookKeeper and ZooKeeper, core concepts like topics and metadata, and the supported consumption modes, illustrating why it is considered a next‑generation distributed messaging system.

Apache PulsarBackendDistributed Messaging
0 likes · 9 min read
Introduction to Apache Pulsar: Architecture, Core Concepts, and Consumption Modes
Architecture Digest
Architecture Digest
May 5, 2019 · Big Data

Kafka Architecture Overview: Topics, Partitions, Producers, Consumers, Replication, Leader Election, Offsets, Rebalance, Delivery Semantics, and Transactions

This article provides a comprehensive overview of Kafka's architecture, covering topics, partitions, producer and consumer workflows, replication and leader election, offset management, consumer group coordination, rebalance processes, delivery semantics (at‑most‑once, at‑least‑once, exactly‑once), transactional messaging, and underlying file and configuration details.

Distributed MessagingExactly-OnceKafka
0 likes · 16 min read
Kafka Architecture Overview: Topics, Partitions, Producers, Consumers, Replication, Leader Election, Offsets, Rebalance, Delivery Semantics, and Transactions
Didi Tech
Didi Tech
Jan 15, 2019 · Backend Development

DDMQ: Didi's Open‑Source Distributed Message Queue

Didi Mobility has open‑sourced DDMQ, an enterprise‑grade distributed message queue that delivers millisecond‑level latency and millions of messages per second, supports real‑time, delayed and transactional messages via RocketMQ/Kafka and RocksDB, offers multi‑language SDKs, flexible consumption, server‑side filtering, a unified console, and achieves five‑nine availability across Didi’s ride‑hailing, finance and delivery platforms.

DDMQDistributed MessagingMessage Queue
0 likes · 7 min read
DDMQ: Didi's Open‑Source Distributed Message Queue
Architects' Tech Alliance
Architects' Tech Alliance
Nov 7, 2018 · Backend Development

An Overview of Apache RocketMQ: Origin, Concept Model, Storage, Deployment, and Best Practices

This article introduces Apache RocketMQ by covering its origin, core concepts such as topics, producers and consumers, storage architecture with CommitLog and ConsumeQueue, deployment components like brokers and name servers, and practical best‑practice guidance for handling duplicates, ordering, and message replay.

ApacheBackend DevelopmentDistributed Messaging
0 likes · 8 min read
An Overview of Apache RocketMQ: Origin, Concept Model, Storage, Deployment, and Best Practices
Java Captain
Java Captain
Sep 15, 2018 · Backend Development

Overview of Alipay System Architecture and the Open‑Source Distributed Messaging Middleware Metamorphosis (MetaQ)

This article presents a comprehensive overview of Alipay’s system architecture, detailing its core components such as payment processing, accounting, settlement, and transaction modules, and introduces the open‑source Java‑based distributed messaging middleware Metamorphosis (MetaQ), highlighting its features, advantages over Kafka, and suitable use cases.

AlipayBackend DevelopmentDistributed Messaging
0 likes · 6 min read
Overview of Alipay System Architecture and the Open‑Source Distributed Messaging Middleware Metamorphosis (MetaQ)
Architecture Digest
Architecture Digest
Aug 7, 2018 · Big Data

Apache Kafka Overview, Architecture, and Sample Producer/Consumer Code

This article provides a comprehensive overview of Apache Kafka, comparing it with ActiveMQ, explaining its distributed architecture, topics, partitions, consumption models, high‑availability mechanisms, exactly‑once semantics, and includes detailed Java producer and consumer code examples for practical implementation.

Big DataConsumerDistributed Messaging
0 likes · 22 min read
Apache Kafka Overview, Architecture, and Sample Producer/Consumer Code
Alibaba Cloud Developer
Alibaba Cloud Developer
Oct 19, 2017 · Cloud Native

Why OpenMessaging Could Redefine Cloud‑Native Messaging Standards

The OpenMessaging project, launched by Alibaba with partners like Yahoo and Didi, aims to create a vendor‑neutral, cloud‑native messaging standard, detailing its value, current implementation in RocketMQ, and a three‑year roadmap covering open‑source releases, cloud platform integration, and ecosystem expansion.

Apache RocketMQCloud NativeDistributed Messaging
0 likes · 7 min read
Why OpenMessaging Could Redefine Cloud‑Native Messaging Standards
Architecture Digest
Architecture Digest
Aug 10, 2017 · Backend Development

Scalable Architecture for Large Websites: Modular Design, Distributed Messaging, Service Platforms, and Open APIs

The article explains how to build a highly scalable large‑website architecture by adopting modularization, reducing coupling through distributed message queues and services, and exposing functionality via an open platform, covering core concepts, typical implementations like ActiveMQ and Dubbo, and essential design considerations.

Distributed MessagingOpen PlatformScalability
0 likes · 8 min read
Scalable Architecture for Large Websites: Modular Design, Distributed Messaging, Service Platforms, and Open APIs
21CTO
21CTO
Jun 11, 2017 · Big Data

How Kafka Guarantees High Reliability – Architecture, Replication & Benchmarks

This article explains Kafka's distributed architecture, topic‑partition model, replication and ISR mechanisms, data durability settings, delivery guarantees, deduplication strategies, and presents benchmark results that illustrate how configuration choices affect throughput and latency in real‑world deployments.

Distributed MessagingKafkaReplication
0 likes · 33 min read
How Kafka Guarantees High Reliability – Architecture, Replication & Benchmarks
MaGe Linux Operations
MaGe Linux Operations
May 28, 2017 · Backend Development

Understanding Kafka’s Architecture: Topics, Partitions, and Reliability

This article explains Kafka’s core architecture—including brokers, topics, partitions, offsets, producer and consumer mechanics, replication, availability, consistency, persistence, performance optimizations, and Zookeeper integration—providing a comprehensive guide for building reliable distributed messaging systems.

Distributed MessagingKafkaOFFSET
0 likes · 15 min read
Understanding Kafka’s Architecture: Topics, Partitions, and Reliability
Tencent Cloud Developer
Tencent Cloud Developer
Mar 13, 2017 · Cloud Computing

Technical Overview of Tencent Cloud CMQ: Architecture, Reliability, Consistency, and Scalability

Tencent Cloud CMQ is a Raft‑based distributed message queue that delivers high reliability, strong consistency, and horizontal scalability for finance‑grade workloads, using multi‑node broker sets with majority‑acknowledged writes, automatic leader election, unlimited buffering, full‑path tracing, while requiring application‑level idempotency and offering limited strict ordering.

CMQDistributed MessagingRaft
0 likes · 11 min read
Technical Overview of Tencent Cloud CMQ: Architecture, Reliability, Consistency, and Scalability
Architecture Digest
Architecture Digest
Mar 15, 2016 · Backend Development

Overview of Alipay System Architecture and the Metamorphosis Distributed Message Middleware

This article presents a detailed overview of Alipay's system architecture—including its processing flow, fund handling platform, accounting, settlement, and transaction modules—and introduces Metamorphosis, an open‑source high‑performance distributed messaging middleware developed by Alibaba, highlighting its features, design motivations, and suitable use cases.

AlipayBackendDistributed Messaging
0 likes · 6 min read
Overview of Alipay System Architecture and the Metamorphosis Distributed Message Middleware
21CTO
21CTO
Feb 23, 2016 · Big Data

Why Kafka Dominates Modern Data Pipelines: Architecture, Benefits, and Guarantees

Kafka, the open‑source distributed messaging system from LinkedIn, offers O(1) persistence, high throughput, partitioned topics, and flexible delivery guarantees, making it a cornerstone for modern big‑data pipelines and real‑time processing alongside Hadoop, Spark, and Storm.

Big DataConsumerDelivery Guarantees
0 likes · 21 min read
Why Kafka Dominates Modern Data Pipelines: Architecture, Benefits, and Guarantees
21CTO
21CTO
Nov 21, 2015 · Big Data

Why Build a Kafka System? Core Use Cases and Design Principles

This article explains why Kafka is essential for activity and operational data pipelines, outlines key use cases such as news feeds, relevance ranking, security, monitoring, and reporting, and details its deployment topology, design decisions, and message persistence strategies.

Distributed MessagingKafkaReal-time Processing
0 likes · 14 min read
Why Build a Kafka System? Core Use Cases and Design Principles
21CTO
21CTO
Sep 14, 2015 · Backend Development

Why Apache Kafka Beats Traditional Message Queues: Architecture, Code, and Performance

This article explains Apache Kafka's distributed publish‑subscribe design, core components, storage model, ZooKeeper coordination, performance benchmarks against ActiveMQ and RabbitMQ, and provides Java producer and consumer code examples for building high‑throughput messaging applications.

Apache KafkaDistributed MessagingMessage Queue
0 likes · 16 min read
Why Apache Kafka Beats Traditional Message Queues: Architecture, Code, and Performance