Tagged articles
distributed-streaming
19 articles · Page 1 of 1
This article introduces Apache Kafka as a distributed streaming platform, detailing its core advantages—high throughput via sequential writes and zero-copy, horizontal scalability through partitions, durability with 7-day retention and replication—and key use cases including log aggregation, event-driven architectures, real-time analytics, and big data integration with tools like Flink and Spark Streaming.
KafkaKafka Streamsbig-data-integration
0 likes · 5 min read

This article outlines Kafka's role as a leading MQ middleware, highlighting its key traits of decoupling, asynchronous processing, and throttling, compares it with traditional messaging systems, and provides a structured list of basic, intermediate, and advanced topics plus interview questions for in‑depth study.
KafkaMessage Queuedata integration
0 likes · 5 min read

This comprehensive guide introduces Apache Kafka's core concepts—topics, partitions, producers, consumers, and APIs—covers common use cases, walks through downloading, installing, and configuring a single‑node broker, demonstrates multi‑broker clustering, and explains how to use Kafka Connect for data import and export.
Producer-Consumerdistributed-streaminginstallation
0 likes · 26 min read

When Kafka consumers can't keep up with producers, messages pile up, storage pressure rises, latency spikes, and downstream services suffer, but increasing consumer threads, batching, and asynchronous processing can alleviate these issues and restore system stability.
Consumer LagKafkadistributed-streaming
0 likes · 4 min read

Kafka 4.0 marks a milestone release that replaces ZooKeeper with the KRaft consensus engine, improves scalability and performance, introduces a server‑side consumer‑group protocol, adds shared‑group queue capabilities, and updates Java requirements and documentation, delivering a more robust and flexible streaming platform.
Java11KRaftKafka
0 likes · 6 min read

This article analyzes the limitations of native Apache Kafka rate‑limiting, introduces JDQ's multi‑dimensional, dynamic, priority‑aware bandwidth control framework, and demonstrates its effectiveness through partition‑level, single‑machine, and hierarchical elastic throttling experiments for large‑scale e‑commerce data streams.
Elastic ThrottlingKafkabandwidth management
0 likes · 18 min read

Kafka is an open‑source, distributed streaming platform that uses a publish/subscribe message queue architecture to provide high‑throughput, fault‑tolerant real‑time data processing, featuring topics, partitions, replicas, consumer groups, and multiple APIs for producers, consumers, streams, connectors, and administration.
KafkaMessage Queuebig data
0 likes · 20 min read

This article provides a detailed overview of Apache Kafka, covering its distributed streaming architecture, storage mechanisms, replication, consumer groups, compression techniques, exactly‑once semantics, configuration tips, and performance optimizations for building reliable high‑throughput data pipelines.
KafkaReplicationbig data
0 likes · 19 min read

This article introduces Apache Kafka and Apache Pulsar, compares their core features such as publish/subscribe messaging, storage, real‑time pipelines, and stream processing, outlines key characteristics like high throughput, scalability and fault tolerance, and explains fundamental concepts and architecture components unique to each platform.
KafkaPulsarbig data
0 likes · 14 min read

This article provides an in‑depth introduction to Apache Kafka, covering its distributed streaming platform fundamentals, message‑queue models, topic and partition design, broker and cluster roles, producer partitioning logic, reliability guarantees, consumer group assignors, offset management, and performance optimizations such as sequential disk writes and zero‑copy techniques.
Apache Kafkaconsumer groupsdistributed-streaming
0 likes · 25 min read

This article provides an in‑depth technical guide to Apache Kafka, covering its distributed streaming architecture, core concepts such as topics, partitions, brokers, producers and consumers, reliability guarantees, storage mechanisms, configuration parameters, and consumer assignment strategies, supplemented with Java code examples.
Apache KafkaConsumerPartitioning
0 likes · 24 min read

This article provides a thorough overview of Apache Kafka, covering its core streaming concepts, key components such as topics, partitions, producers and consumers, common use cases, step‑by‑step installation and multi‑broker configuration, fault‑tolerance testing, and an introduction to Kafka Connect for data import/export.
ConsumerKafkaKafka Connect
0 likes · 24 min read

This article provides a comprehensive guide to Apache Kafka, covering its core concepts, architecture, key APIs, topics and partitions, deployment steps, multi‑broker clustering, fault tolerance, and data integration using Kafka Connect, with detailed command‑line examples.
ConsumerKafkaKafka Connect
0 likes · 26 min read

Apache Kafka is a distributed, fault‑tolerant streaming platform that uses broker clusters, topic partitions, and replicated logs to provide publish/subscribe and queue messaging, configurable retention, strong ordering within partitions, producer acknowledgments, consumer groups, and performance optimizations such as batching and zero‑copy for real‑time data pipelines.
Apache KafkaConsumerMessage Queue
0 likes · 26 min read

Kafka is an Apache‑hosted distributed streaming platform that provides high‑throughput, durable, publish‑subscribe messaging, originally developed by LinkedIn; this article explains its core concepts, message system classifications, architecture components, APIs, replication, consumer groups, and guarantees, comparing it with other messaging solutions.
KafkaPublish‑Subscribebig data
0 likes · 17 min read

This article explains why Apache Kafka has become a cornerstone of modern big‑data pipelines by detailing its high‑throughput, fault‑tolerant publish‑subscribe architecture, real‑time processing capabilities, extensive language support, scalability mechanisms, and the wide range of use cases adopted by leading enterprises.
KafkaMessage Queuedistributed-streaming
0 likes · 9 min read

This article introduces Apache Kafka as a distributed streaming platform, explaining its three core capabilities, key concepts such as producers, topics, brokers, partitions and consumers, and detailing how messages are delivered, stored in partitions, and consumed by consumer groups.
KafkaMessage QueuePartitioning
0 likes · 8 min read

This article explains Kafka’s core concepts—including producers, consumers, topics, partitions, brokers, and consumer groups—describes its distributed architecture with leader‑follower replication, and provides three hands‑on kafka‑python experiments that demonstrate basic messaging, fault‑tolerant consumer groups, and offset management for reliable consumption.
Consumer GroupKafkaProducer-Consumer
0 likes · 9 min read

This article provides a comprehensive overview of Apache Kafka, explaining its role in real‑time data pipelines and stream processing, describing key concepts such as topics, partitions, logs, producers, consumers, replication, guarantees, and how Kafka functions as both a messaging and storage system.
Consumer APIKafkaProducer API
0 likes · 13 min read
