Tagged articles
broker
52 articles · Page 1 of 1
The smart-mqtt 1.8.0 release removes built-in monitoring storage and extracts configuration management, adopting Prometheus-standard metrics and a silent config module to reduce operational weight for containerized deployments.
Configuration ManagementMQTTPrometheus
0 likes · 5 min read

This guide walks you through the entire lifecycle of a RocketMQ message, identifies where loss can occur, provides step‑by‑step diagnostics using TrackType and timestamps, and offers concrete configuration, command‑line and recovery strategies for producers, brokers and consumers.
ConfigurationConsumerMessage Loss
0 likes · 11 min read

The article breaks down RocketMQ’s end‑to‑end reliability design into three checkpoints—producer to broker, broker persistence, and consumer acknowledgment—explaining each mechanism, configuration options, common pitfalls, and practical recommendations for production deployments.
ConsumerMessage QueueReplication
0 likes · 16 min read

This article provides a comprehensive overview of RocketMQ, covering its core components (Producer, Broker, Consumer, NameServer), detailed architecture diagrams, step‑by‑step message production and consumption processes, and typical scenarios such as asynchronous decoupling, traffic shaping, transactional messaging, and event‑driven logging.
ConsumerDistributed MessagingMessage Queue
0 likes · 6 min read

This article explains RabbitMQ's system architecture, covering producers, brokers, consumers, exchanges, routing keys, bindings, queue storage, network protocols, and reliability features, using clear analogies and diagrams to illustrate core concepts.
AMQPExchangeMessage Queue
0 likes · 14 min read

During a gray‑scale upgrade, a Vivo Pulsar deployment using the KoP protocol experienced a sudden drop in partition consumption rates because the consumer group name vanished from Zookeeper nodes, prompting an in‑depth analysis of the root cause, replication steps, and three progressively refined remediation strategies.
Consumer MetricsKoPPulsar
0 likes · 13 min read

smart-mqtt v1.2.0 introduces async batch processing, smart resource management, and a refactored subscription model, and a three‑million‑reconnection stress test shows the broker remains stable under extreme network churn, with no packet loss, service interruption, or resource spikes.
DockerJavaMQTT
0 likes · 5 min read

This article examines the typical reasons Kafka messages are lost across producers, brokers, and consumers, and provides detailed configuration recommendations and best‑practice solutions to significantly reduce the risk of data loss in distributed streaming systems.
ConfigurationConsumerKafka
0 likes · 15 min read

This article explains how RocketMQ implements long‑polling for message consumption, detailing the pull‑based model, the broker and consumer timeout settings, the internal suspension of pull requests, and the processing loop that resumes suspended requests to improve efficiency.
BackendConsumerJava
0 likes · 7 min read

The article thoroughly explains RocketMQ’s architecture—including stateless NameServers for routing, master‑slave Brokers with sequential CommitLog storage, stateless Producers and pull‑or‑push Consumers, message flow, indexing, and synchronous double‑write reliability—showing how these components deliver high‑throughput, scalable, fault‑tolerant messaging for distributed systems.
ConsumerMessage QueueRocketMQ
0 likes · 20 min read

This article analyzes the challenges of scaling automated quality‑inspection hardware, introduces IoT fundamentals, evaluates application‑layer protocols, MQTT broker options, QoS levels, and compares cloud, local, and edge deployment architectures to recommend a low‑latency, high‑reliability solution for the smart inspection platform.
Edge ComputingIoTMQTT
0 likes · 22 min read

This article breaks down how Kafka achieves up to 20 million messages per second and 600 MB/s throughput by examining producer batching and compression, broker PageCache and zero‑copy techniques, and consumer parallelism with group coordination.
CompressionConsumerKafka
0 likes · 6 min read

This article presents a detailed case study of RocketMQ's fast‑failure mechanism, explains the root causes of a massive message‑write surge, analyzes broker thread models, and proposes topic‑level rate‑limiting solutions with concrete implementation and code examples.
Fast-FailureRocketMQTopic
0 likes · 16 min read

This article explains how Kafka achieves extremely high throughput—up to 20 million messages and 600 MB per second per node—by optimizing the producer, broker, and consumer components through batch sending, custom protocols, page‑cache usage, zero‑copy transfers, and efficient compression algorithms.
CompressionConsumerKafka
0 likes · 7 min read

This article explains why introducing a message queue like Kafka helps decouple systems and control traffic, then dives into the three key questions of detecting, locating, and preventing message loss, offering concrete monitoring methods, configuration settings, and troubleshooting steps for producers, brokers, and consumers.
ConfigurationConsumerData Consistency
0 likes · 13 min read

This article explains Kafka's message delivery semantics, analyzes potential data‑loss scenarios across producer, broker, and consumer components, and provides concrete configuration and coding practices—such as idempotent producers, proper ACK settings, replication factors, and manual offset commits—to maximize message durability and reliability.
ConsumerKafkaMessage Semantics
0 likes · 18 min read

This article provides a comprehensive overview of Kafka’s core components—including brokers, topics, partitions, producers, and consumers—explaining their roles, relationships, replication, leader/follower dynamics, and architectural diagrams, while also highlighting configuration details and operational considerations for building reliable distributed streaming systems.
ConsumerTopicbroker
0 likes · 9 min read

This article provides a detailed overview of message queue fundamentals, covering producers, consumers, brokers, queue and topic models, delivery guarantees, acknowledgment mechanisms, ordering, persistence, high availability, and selection criteria, helping readers quickly grasp essential MQ concepts for distributed systems.
ConsumerMessage Queuebroker
0 likes · 9 min read

This article explains how RocketMQ reduces message loss and duplicate consumption through synchronous sending, retry mechanisms, multi‑master brokers, synchronous disk flushing, master‑slave replication, at‑least‑once delivery, and idempotent processing techniques.
ConsumerIdempotencyMessage Reliability
0 likes · 10 min read

This article provides a comprehensive overview of Kafka's core components—including brokers, topics, partitions, producers, and consumers—explaining their roles, how they interact within a Kafka cluster, and illustrating the architecture with diagrams while also noting recent version changes and related disclaimer information.
ConsumerKafkaTopic
0 likes · 7 min read

This article provides a comprehensive overview of Kafka's core components—including brokers, topics, partitions, producers, and consumers—explaining their roles, relationships, replication strategies, and how they work together within a Kafka cluster to ensure reliable, scalable message streaming.
ConsumerKafkaReplication
0 likes · 8 min read

The Apache Pulsar 2.10.0 release introduces automatic cluster failover, lazy‑loading producers, new TableView support, enhanced broker interceptors, enriched client authentication, Etcd metadata storage, and numerous bug fixes, offering developers and operators a more flexible and performant messaging platform.
Apache PulsarClientFailover
0 likes · 7 min read

This article explains how Kafka achieves producer idempotence by assigning unique producer IDs and sequence numbers, describes the broker’s validation process, and walks through the relevant producer‑side and broker‑side code paths, highlighting configuration considerations and limitations.
IdempotenceKafkaMessage Queue
0 likes · 13 min read

This article explains Kafka's message delivery semantics, identifies the exact scenarios where data can be lost in producer, broker, and consumer components, and provides concrete configuration and coding practices to ensure reliable, at‑least‑once or exactly‑once delivery in production environments.
ConsumerData ReliabilityKafka
0 likes · 18 min read

mica‑mqtt, built on t‑io, offers a simple, low‑latency, high‑performance open‑source MQTT component that integrates easily into existing services, supports multiple protocols and deployment scenarios, and includes Spring Boot starter, GraalVM compilation, and Redis‑based clustering for robust IoT solutions.
BackendGraalVMJava
0 likes · 6 min read

This article explains how Apache Pulsar brokers track consumer acknowledgment using cursors, describes persistent and non‑persistent subscription differences, details cursor metadata stored in ZooKeeper and BookKeeper, and outlines optimizations for handling message gaps, including RangeSet improvements and a new LRU‑based storage design.
Apache PulsarBookKeeperCursor Management
0 likes · 12 min read

This article explains how Apache Pulsar brokers track consumer acknowledgments using cursors, store metadata in ZooKeeper and BookKeeper, handle acknowledgment gaps with range sets, and optimize memory usage through LRU and segmented storage to ensure reliable message delivery.
Apache PulsarBookKeeperCursor
0 likes · 15 min read

This article explains why message acknowledgment is crucial in Apache Pulsar, describes the four ack modes, shows which subscription types support each mode, and details the internal implementation, batching, negative ack handling, and unacknowledged‑message processing for reliable broker‑consumer communication.
Ack ModesApache PulsarConsumer
0 likes · 12 min read

This article explains the complete Kafka topic deletion workflow, from the client’s deleteTopics request through Zookeeper node creation, controller coordination, broker StopReplica handling, log renaming, delayed file removal, and final cleanup, while providing code excerpts and practical Q&A for common pitfalls.
KafkaLog CleanupScala
0 likes · 17 min read

RSocket is a binary, back‑pressure‑aware protocol that supports four interaction models—fire‑and‑forget, request‑response, request‑stream, and request‑channel—using multiplexed frames with odd/even stream IDs for true bidirectional communication, and provides a reactive, resumable API suitable for microservices, IoT, and broker‑based architectures.
BackendReactive Streamsbroker
0 likes · 8 min read

This article provides a comprehensive overview of Knative Eventing's Broker and Trigger model, detailing background concepts, event routing patterns, data‑plane and control‑plane workflows, and includes practical YAML and command‑line examples for deploying and managing the components in a Kubernetes environment.
KnativeKubernetesTrigger
0 likes · 11 min read

This article explains why Kafka can lose messages at the broker, producer, and consumer levels, describes the underlying mechanisms such as page cache flushing and acknowledgment settings, and provides practical code examples and mitigation strategies to improve reliability.
ACKConsumerJava
0 likes · 12 min read

This article explains the three places where Kafka can lose messages—Broker, Producer, and Consumer—detailing the underlying mechanisms, the impact of flush and ack settings, and practical configuration and coding strategies to minimize data loss.
Ack SettingsConfigurationConsumer
0 likes · 15 min read

This article explains the three places where Kafka can lose messages—broker, producer, and consumer—detailing the underlying causes such as page cache flushing, acknowledgment settings, buffer handling, and offset commits, and provides practical configuration and design strategies to minimize data loss and improve reliability.
ConsumerMessage Lossacks
0 likes · 13 min read

The article thoroughly explains Kafka’s message‑queue reliability design and implementation, covering use‑case scenarios, core concepts, storage format, producer acknowledgment settings, broker replication mechanisms (ISR, HW, LEO), consumer delivery semantics, the epoch solution for synchronization, and practical configuration guidelines for various consistency and availability requirements.
ConsistencyConsumerKafka
0 likes · 15 min read

This guide walks you through setting up a fault‑tolerant RocketMQ cluster on two CentOS 7 machines, covering deployment options, environment preparation, Java installation, NameServer and Broker configuration, startup procedures, troubleshooting, and optional console installation.
ClusterConfigurationHigh Availability
0 likes · 22 min read

This guide walks through setting up a fault‑tolerant RocketMQ cluster on two CentOS 7 servers, covering installation of Java, NameServer, multiple Master‑Slave brokers in async and sync modes, configuration tweaks, port opening, verification, and optional console deployment.
Cluster DeploymentHigh AvailabilityJava
0 likes · 22 min read

This article documents the supported features, deprecated services, and specific limitations of a range of MQTT brokers, providing a concise reference for developers to understand each broker's capabilities and constraints within IoT and backend messaging systems.
IoTMQTTbroker
0 likes · 2 min read

The article examines a recurring RocketMQ error (TIMEOUT_CLEAN_QUEUE) caused by the broker’s fast‑failure mechanism, explains how the broker queues and thread pools operate, reveals that SYSTEM_BUSY is mistakenly excluded from retry logic—a bug—and proposes both a temporary configuration tweak and a permanent code fix via a PR.
Fast-FailureMessage QueueTimeout
0 likes · 8 min read

This article explains the end‑to‑end process of a Kafka producer sending a message to a broker, covering client components, the internal cache storage model, partition selection logic, and the workings of the sender thread and network client.
brokermessagingnetwork
0 likes · 7 min read

This article introduces Kafka as a distributed messaging and stream‑processing platform, then enumerates the most important broker‑side configuration parameters—such as broker.id, log.dirs, zookeeper.connect, listeners, and retention settings—explaining their purpose and providing practical tuning advice for production environments.
Backendbrokerdistributed-systems
0 likes · 7 min read

This article explains how Zookeeper functions as an essential part of Kafka by managing broker status, controller election, quotas, ISR tracking, node and topic registration, as well as consumer offset storage and registration, providing a comprehensive overview for interview preparation.
ConsumerKafkaMessage Queue
0 likes · 4 min read

This tutorial walks through setting up Knative Eventing to capture Kubernetes events, covering the Broker/Trigger model, required prerequisites, creating service accounts, defining an ApiServerSource, deploying a simple event‑display service, configuring a Broker and Trigger, and verifying event delivery.
KnativeKubernetesTrigger
0 likes · 10 min read

This article introduces RocketMQ, compares it with ActiveMQ, explains its producer/consumer group model, describes various broker deployment modes, and provides a step‑by‑step dual‑master installation guide with configuration, log, and console setup instructions.
Message QueueNameServerRocketMQ
0 likes · 7 min read

This article introduces the MQTT protocol, compares several broker implementations, and details functional and performance testing methods—including connection, publishing, subscription tests, and throughput measurements—to evaluate MQTT broker suitability for IoT platforms.
IoTMQTTbroker
0 likes · 5 min read

This article explains the core concepts, components, design considerations, and common models of message queues—including point‑to‑point and publish/subscribe—while covering ordering, acknowledgment, transaction support, persistence, high availability, and practical selection guidance for high‑concurrency systems.
BackendConsumerMessage Queue
0 likes · 8 min read


JD TechApr 2, 2018 · Backend Development
This article introduces the MQTT protocol, outlines its key characteristics, explains the challenges it poses for IoT messaging, and details how JMQ addresses these challenges through a layered proxy‑broker architecture, subscription management, quality‑of‑service handling, and real‑world JD deployment.
IoTJMQMQTT
0 likes · 8 min read

This article explains the internal workflow of RocketMQ transactional messages, covering how producers send half messages, how brokers process commit or rollback requests, the storage mechanisms for transaction state, the periodic transaction check logic, and the differences between file‑system and database implementations.
JavaMessage QueueRocketMQ
0 likes · 22 min read

This tutorial provides a step‑by‑step walkthrough for setting up Apache RocketMQ 4.2 on a 64‑bit Linux system, covering environment prerequisites, downloading source, building binaries, starting NameServer and Broker, sending and receiving messages, and shutting down the services.
JavaLinuxMaven
0 likes · 6 min read

Message queues act as middleware in distributed systems, enabling decoupling, asynchronous processing, traffic shaping, and reliable communication between producers and consumers, while supporting high concurrency, scalability, and eventual consistency, and come in brokered and broker‑less varieties such as RabbitMQ and ZeroMQ.
RabbitMQZeroMQbroker
0 likes · 5 min read

This article provides a comprehensive overview of Kafka's core configuration options, detailing default values and descriptions for broker, producer, consumer, and topic‑level settings to help administrators fine‑tune performance, reliability, and resource usage.
ConfigurationConsumerKafka
0 likes · 23 min read

The article presents a detailed overview of Ctrip's Hermes asynchronous messaging system, describing its architectural evolution from a simple Mongo‑based queue to a broker‑centric design with MySQL and Kafka back‑ends, and explains optimization techniques for single‑node performance, clustering, lease‑based management, and reliable delivery.
CtripHermesKafka
0 likes · 22 min read
