0 views collected around this technical thread.
This article details Vivo's end‑to‑end Pulsar observability solution, covering the challenges of Prometheus‑based monitoring, the architecture of the alerting pipeline, adaptor development, metric optimizations for subscription backlog and bundle load, and fixes for kop lag reporting issues.
This article explains the three message delivery semantics, the common causes of duplicate messages in queue systems, and presents concrete producer‑side, broker‑side, and consumer‑side deduplication techniques for Kafka and Pulsar, including code samples and best‑practice recommendations.
Message queues can deliver duplicates under at‑least‑once semantics, so to protect idempotent business logic such as orders and payments you should combine producer‑side idempotence (e.g., Kafka’s enable.idempotence), broker‑side deduplication (e.g., Pulsar), and a consumer‑side guard using unique IDs stored in a database or Redis.
Message queues can cause duplicate messages that affect idempotent business processes, so this article explains the three delivery semantics (At Least Once, Exactly Once, At Most Once), the causes of duplication, and practical deduplication techniques for producers, brokers (Kafka, Pulsar), and consumers using code examples.
This comprehensive guide compares five popular message queue systems—Kafka, Pulsar, RabbitMQ, RocketMQ, and NSQ—covering their architectures, core concepts, selection criteria, feature differences such as ordering, latency, durability, scaling, and operational considerations, to help engineers select the most suitable solution for their use cases.
The article surveys major message‑queue systems—Kafka, Pulsar, RocketMQ, RabbitMQ, and NSQ—detailing their architectures, key features, selection criteria such as ordering, scalability and latency, performance benchmarks, operational considerations, and typical use cases, and advises which platform best fits high‑throughput, reliability, low‑latency, or lightweight Go‑centric scenarios.
Tencent Cloud TDMQ for Apache Pulsar achieves multi‑AZ high availability by containerizing ZooKeeper, BookKeeper and Brokers, using managed ZK, persistent cloud disks and elastic NICs, enforcing quorum and rack‑aware replicas, and planning cross‑region bidirectional replication to ensure seamless disaster recovery and continuous messaging.
This technical presentation details Vivo's end‑to‑end big‑data architecture, the evolution from Kafka to Pulsar for massive message processing, deployment strategies, high‑availability mechanisms, observability practices, and future plans for cloud‑native, containerized messaging middleware.
The article walks through building a custom OpenTelemetry Java‑agent extension for Pulsar client metrics—migrating from SkyWalking, setting up a Gradle project, using ByteBuddy to instrument methods with advice, registering gauge metrics, packaging the jar, handling common class‑loader pitfalls, and configuring deployment via the OpenTelemetry operator.
This article traces the two‑decade evolution of message queues—from early decoupling solutions like ActiveMQ, through high‑throughput designs such as Kafka, to modern platformized systems like RocketMQ and Pulsar—while explaining fundamental concepts, partitioning, and storage architectures that underpin today’s distributed messaging platforms.
This article introduces five widely used message middleware systems—Kafka, RabbitMQ, ActiveMQ, RocketMQ, and Pulsar—explaining their core architectures, key components, typical use cases, and advantages for building reliable, high‑throughput backend solutions.
This article explains seven typical message‑queue use cases—including ordinary, ordered, delayed, transactional, trace, dead‑letter, and priority messages—detailing their business motivations, implementation challenges, and concrete code examples for Kafka, RocketMQ, Pulsar, and RabbitMQ.
From the early days of ActiveMQ and RabbitMQ to the high‑throughput designs of Kafka, RocketMQ, and Pulsar, this article traces the development stages of message queues, explains core concepts such as topics, producers, consumers, and partitions, and compares their architectural and storage designs.
To give the business team a global, application‑level view of Pulsar performance, the team built a SkyWalking Java‑Agent plugin that automatically collects producer and consumer metrics from the Pulsar client, exposing latency, backlog and failure counts via Prometheus without modifying the client code.
The article traces two decades of message‑queue evolution—from early decoupling tools like ActiveMQ, through Kafka’s high‑throughput log model, to Pulsar’s layered, cloud‑native architecture—explaining core concepts, storage designs, and trade‑offs that guide choosing the right MQ for modern workloads.
vivo’s Internet Storage team details how it operates RocketMQ for low‑latency online services and Kafka for massive big‑data pipelines, outlines resource isolation, traffic balancing, intelligent throttling, and governance practices, and describes its migration from RabbitMQ and planned shift from Kafka to cloud‑native Pulsar.
The article explains fundamental concepts, common use cases, and selection criteria for four popular message‑queue platforms—Kafka, RabbitMQ, Pulsar, and RocketMQ—detailing each system’s architecture, strengths, and drawbacks so readers can match workload, scalability, ordering, latency, and operational needs to the most suitable middleware.
This article surveys common delayed‑message implementations—including external storage, RocksDB, Redis, and open‑source MQs like RocketMQ, Pulsar, and QMQ—detailing their architectures, advantages, disadvantages, and practical considerations for distributed asynchronous messaging systems.
This article surveys common delayed‑message solutions in distributed asynchronous messaging, evaluates implementations based on external storage, databases, RocksDB, Redis, and open‑source MQs like RocketMQ, Pulsar and QMQ, and discusses their advantages, drawbacks, and practical design considerations.
This article provides a comprehensive technical overview of Apache BookKeeper, covering its role as a distributed append‑only log service, core concepts, high‑availability mechanisms, storage‑media evolution, comparisons with Raft, and community resources, while illustrating its use in Pulsar and large‑scale data platforms.