At ten‑million QPS scale, a Kafka topic becomes a cross‑team contract rather than a simple name, requiring standardized naming, granularity, partitioning, schema registry, lifecycle management, and governance platforms to transform ad‑hoc topics into machine‑verifiable infrastructure assets.
This article visually breaks down Kafka’s fundamental concepts—including producers, consumers, topics, consumer groups, partitions, and cluster architecture—explaining how messages flow, are stored, and achieve load balancing and ordering within a distributed streaming system.
This guide compares Kafka, Pulsar, RocketMQ, RabbitMQ, and NSQ across architecture, consumption models, delay and dead‑letter support, persistence, ordering, multi‑tenant isolation, and performance, recommending Kafka for high‑throughput log processing, Pulsar for transactional delayed messaging and disaster recovery, and NSQ for Go‑centric custom solutions.
This article provides a visual walkthrough of Kafka's fundamental concepts—including its basic architecture, producer‑consumer model, topics, partitions, and cluster design—offering clear explanations and diagrams to help readers grasp how Kafka enables reliable, scalable message streaming in modern systems.
This article explains how Wix engineers built a robust, event‑driven messaging infrastructure on Kafka that serves over 1,400 microservices, detailing six key patterns—including consumption‑projection, end‑to‑end event flows, in‑memory KV stores, scheduling, transactional events, and aggregation—that improve scalability, resilience, and operational simplicity.
This article provides a visual walkthrough of Kafka's fundamental concepts—including producers, consumers, topics, partitions, and cluster architecture—illustrated with diagrams to help readers clearly understand how messages flow and are stored in a distributed streaming system.
This article compares Apache Pulsar and Apache Kafka, outlines Kafka's operational pain points, details Pulsar's architecture and features, provides step‑by‑step installation and code examples for Pulsar clients, and discusses when to choose Pulsar over Kafka.
