A Gentle Introduction to Apache Pulsar: Architecture, Features, and Use Cases

Traditional messaging systems suffer from tight coupling of storage and service, making scaling and operations difficult, and have fixed consumption models and limited multi‑tenant or multi‑datacenter features.

Apache Pulsar addresses these issues with a layered architecture that separates storage (Apache BookKeeper) from brokers, providing high scalability, independent broker and storage scaling, and easier containerization of ZooKeeper, brokers, and bookies.

Since its large‑scale deployment at Yahoo in 2015 and adoption by companies such as Zhaopin, Pulsar has supported thousands of topics and billions of messages, offering features like cross‑region replication, multi‑tenant isolation, zero data loss, zero rebalancing time, a unified queue‑and‑stream model, built‑in load balancer, service discovery, functions, and a proxy for external access.

The architecture includes a global ZooKeeper for configuration, a Pulsar proxy for authentication and authorization, and supports exclusive, failover, and shared subscription types to realize both streaming and queuing semantics.

Overall, Pulsar’s cloud‑native, highly extensible design makes it suitable for enterprise‑grade messaging workloads.