BestHub
Sign in
Tagged articles
39 articles
Page 1 of 1
Laravel Tech Community
Laravel Tech Community
Mar 23, 2025 · Big Data

Apache Kafka 4.0 Released: First Version Without ZooKeeper and New Features

Apache Kafka 4.0 has been officially released as the first major version that runs entirely without Apache ZooKeeper, introducing KRaft mode, a new consumer group protocol (KIP‑848), early‑access queue support (KIP‑932), updated Java requirements, and other enhancements aimed at improving scalability, operability, and messaging versatility.

Apache KafkaKIP-848KIP-932
0 likes · 3 min read
Apache Kafka 4.0 Released: First Version Without ZooKeeper and New Features
Tencent Cloud Developer
Tencent Cloud Developer
Jul 16, 2024 · Big Data

In‑Depth Exploration of Apache Kafka: Architecture, High Reliability, and High Performance

Apache Kafka achieves high‑throughput, fault‑tolerant messaging by combining a partitioned log architecture with leader‑follower replication, asynchronous producer pipelines, configurable acknowledgments, page‑cache‑based sequential writes, zero‑copy transfers, batching, compression, and a multi‑reactor network model that together ensure scalability, reliability, and performance.

Apache KafkaReliabilityStreaming
0 likes · 30 min read
In‑Depth Exploration of Apache Kafka: Architecture, High Reliability, and High Performance
FunTester
FunTester
Jan 5, 2024 · Big Data

An Overview of Apache Kafka and Kafka Streams Technical Features

This article introduces Apache Kafka as a high‑throughput, scalable, fault‑tolerant distributed streaming platform, explains why it is chosen for real‑time data pipelines, and details key Kafka Streams concepts such as stream processing, interactive queries, stateful processing, windowing, serialization, and testing.

Apache KafkaBig DataStreaming
0 likes · 13 min read
An Overview of Apache Kafka and Kafka Streams Technical Features
dbaplus Community
dbaplus Community
Nov 19, 2023 · Big Data

How Agoda Scales Apache Kafka: Two‑Step Logging, Monitoring, and Cost Attribution

This article details Agoda's evolution of Apache Kafka usage—from a two‑step logging architecture that separates developer concerns, through cluster layout, scaling metrics, monitoring and audit pipelines, to cost attribution, authentication, ACLs, and automation tools—highlighting trade‑offs and operational lessons learned.

Apache KafkaCost ManagementScalability
0 likes · 17 min read
How Agoda Scales Apache Kafka: Two‑Step Logging, Monitoring, and Cost Attribution
Java Architecture Diary
Java Architecture Diary
Jul 11, 2023 · Big Data

Redpanda vs Apache Kafka with KRaft: Why Redpanda Is Up to 10× Faster

This article presents a detailed benchmark comparing Redpanda 23.1 and Apache Kafka 3.4.0 (with and without KRaft) across multiple AWS instance types, showing how Redpanda consistently delivers higher throughput and dramatically lower end‑to‑end latency, often outperforming Kafka by 4‑20× even with extra hardware.

Apache KafkaBig DataKRaft
0 likes · 12 min read
Redpanda vs Apache Kafka with KRaft: Why Redpanda Is Up to 10× Faster
Architects Research Society
Architects Research Society
Jun 4, 2023 · Big Data

Understanding Transactions in Apache Kafka

This article explains the design, semantics, and practical usage of Apache Kafka's transaction API, covering why transactions are needed for exactly‑once processing, the underlying atomic multi‑partition writes, zombie fencing, consumer guarantees, Java API details, performance considerations, and operational best practices.

Apache KafkaDistributed SystemsExactly-Once
0 likes · 19 min read
Understanding Transactions in Apache Kafka
vivo Internet Technology
vivo Internet Technology
Feb 23, 2022 · Big Data

Kafka-based Real-Time Data Warehouse: Architecture and Practice for Search

The article explains how Kafka serves as the core of a real‑time data warehouse for search, detailing its advantages over traditional databases, integration with Flink for low‑latency stream processing, architectural patterns such as Lambda/Kappa, scaling challenges, and comprehensive monitoring using Kafka Eagle.

Apache KafkaData IntegrationFlink
0 likes · 15 min read
Kafka-based Real-Time Data Warehouse: Architecture and Practice for Search
IT Architects Alliance
IT Architects Alliance
Feb 13, 2022 · Big Data

Comprehensive Overview of Apache Kafka Architecture and Core Concepts

This article provides an in‑depth introduction to Apache Kafka, covering its distributed streaming platform fundamentals, message‑queue models, topic and partition design, broker and cluster roles, producer partitioning logic, reliability guarantees, consumer group assignors, offset management, and performance optimizations such as sequential disk writes and zero‑copy techniques.

Apache KafkaDistributed StreamingReliability
0 likes · 25 min read
Comprehensive Overview of Apache Kafka Architecture and Core Concepts
Java Architect Essentials
Java Architect Essentials
Dec 7, 2021 · Big Data

Apache Kafka 3.0 Release Highlights and New Features

The article provides a comprehensive overview of Apache Kafka 3.0, detailing its core APIs, two main use‑cases, major feature additions, deprecations, KRaft consensus improvements, enhanced producer guarantees, and numerous KIP‑driven changes across the broker, client, Connect, Streams, and MirrorMaker components.

Apache KafkaEvent StreamingKIP
0 likes · 14 min read
Apache Kafka 3.0 Release Highlights and New Features
IT Architects Alliance
IT Architects Alliance
Dec 3, 2021 · Big Data

Comprehensive Overview of Apache Kafka Architecture and Core Concepts

This article provides an in‑depth technical guide to Apache Kafka, covering its distributed streaming architecture, core concepts such as topics, partitions, brokers, producers and consumers, reliability guarantees, storage mechanisms, configuration parameters, and consumer assignment strategies, supplemented with Java code examples.

Apache KafkaConsumerDistributed Streaming
0 likes · 24 min read
Comprehensive Overview of Apache Kafka Architecture and Core Concepts
21CTO
21CTO
Sep 27, 2021 · Big Data

Tech Highlights: China Crypto Ban, Huawei’s New Language, Kafka 3.0

A roundup of recent tech news covering China's crackdown on cryptocurrency, Huawei's upcoming programming language, the release of Apache Kafka 3.0, and other major developments in China's digital economy and industry leadership.

Apache KafkaBig DataDigital Economy
0 likes · 8 min read
Tech Highlights: China Crypto Ban, Huawei’s New Language, Kafka 3.0
Programmer DD
Programmer DD
Sep 26, 2021 · Big Data

What’s New in Apache Kafka 3.0? Key Features and Improvements Explained

Apache Kafka 3.0.0 introduces a host of enhancements—including deprecated Java 8/Scala 2.12 support, Raft metadata snapshots, stronger producer guarantees, MirrorMaker 2 upgrades, and Kafka Streams improvements—while continuing to serve real‑time data pipelines and streaming applications.

Apache KafkaBig DataKafka 3.0
0 likes · 3 min read
What’s New in Apache Kafka 3.0? Key Features and Improvements Explained
IT Architects Alliance
IT Architects Alliance
Sep 25, 2021 · Big Data

Apache Kafka 3.0.0 Release: New Features, API Changes, and KRaft Improvements

Apache Kafka 3.0.0 introduces numerous enhancements including deprecation of Java 8 and Scala 2.12 support, KRaft metadata snapshots, stronger default producer delivery guarantees, expanded Connect and Streams APIs, updated MirrorMaker 2 configuration, and many KIP-driven feature and API changes for improved streaming and event processing.

Apache KafkaEvent ProcessingKIP
0 likes · 15 min read
Apache Kafka 3.0.0 Release: New Features, API Changes, and KRaft Improvements
Laravel Tech Community
Laravel Tech Community
Apr 22, 2021 · Big Data

Apache Kafka 2.8.0 Release Highlights and New Features

Apache Kafka 2.8.0 introduces several significant enhancements, including a new group API, mutual TLS authentication for SASL_SSL listeners, JSON request/response logging, broker connection rate limiting, topic identifiers, self‑managed quorum replacing ZooKeeper, and numerous improvements to Streams and Connect APIs for more reliable real‑time data pipelines.

Apache KafkaBig DataDistributed Systems
0 likes · 2 min read
Apache Kafka 2.8.0 Release Highlights and New Features
Big Data Technology & Architecture
Big Data Technology & Architecture
Apr 9, 2021 · Backend Development

Kafka 2.8 Introduces KRaft: Running Without ZooKeeper

Kafka 2.8 replaces ZooKeeper with an internal Quorum controller (KRaft), allowing users to run Kafka without external coordination, reducing resource usage, improving performance, and supporting larger clusters, while noting that some features like ACLs and partition reassignment are still pending.

Apache KafkaKRaftZooKeeper
0 likes · 4 min read
Kafka 2.8 Introduces KRaft: Running Without ZooKeeper
vivo Internet Technology
vivo Internet Technology
Jan 13, 2021 · Big Data

An Introduction to Apache Kafka: Architecture, Concepts, and Operations

Apache Kafka is a distributed, fault‑tolerant streaming platform that uses broker clusters, topic partitions, and replicated logs to provide publish/subscribe and queue messaging, configurable retention, strong ordering within partitions, producer acknowledgments, consumer groups, and performance optimizations such as batching and zero‑copy for real‑time data pipelines.

Apache KafkaConsumerDistributed Streaming
0 likes · 26 min read
An Introduction to Apache Kafka: Architecture, Concepts, and Operations
Architects Research Society
Architects Research Society
Dec 8, 2020 · Backend Development

Using Apache Kafka with Spring Boot: Error Handling, Deserialization, Multi‑Listener, and Transactions

This article explains how to integrate Apache Kafka with Spring Boot, covering error recovery with SeekToCurrentErrorHandler, deserialization handling, domain‑object conversion, multiple listener routing, and transaction support, while providing complete code examples for each feature.

Apache KafkaError HandlingMessage Conversion
0 likes · 10 min read
Using Apache Kafka with Spring Boot: Error Handling, Deserialization, Multi‑Listener, and Transactions
Architects Research Society
Architects Research Society
Jul 29, 2020 · Big Data

Static Members and Incremental Cooperative Rebalancing in Apache Kafka

Apache Kafka 2.3 introduced static members and incremental cooperative rebalancing to reduce disruptive global rebalances, allowing workers to retain assignments during failures, schedule delayed rebalances, and improve scalability for Kafka Connect clusters, balancing availability and fault tolerance.

Apache KafkaDistributed SystemsIncremental Rebalancing
0 likes · 12 min read
Static Members and Incremental Cooperative Rebalancing in Apache Kafka
Big Data Technology & Architecture
Big Data Technology & Architecture
Dec 19, 2019 · Big Data

Apache Kafka 2.4.0 Release: New Features and Improvements

Apache Kafka 2.4.0 introduces a range of new capabilities—including consumer replica fetching, incremental cooperative rebalancing, MirrorMaker 2.0, a new Java authorization API, KTable non‑key joins, administrative replica reassignment, protected REST endpoints, and offset deletion—along with numerous performance and stability improvements.

Apache KafkaBig DataDistributed Systems
0 likes · 3 min read
Apache Kafka 2.4.0 Release: New Features and Improvements
Architects Research Society
Architects Research Society
Oct 20, 2019 · Big Data

Spring Cloud Data Flow: Building and Deploying Event Stream Pipelines with Apache Kafka (Part 3)

This article explains how Spring Cloud Data Flow, together with Spring Cloud Skipper, enables developers to design, deploy, and manage event‑stream pipelines on Apache Kafka, covering ecosystem overview, pipeline components, Docker‑based local setup, stream creation, debugging, monitoring, and integration of Kafka Streams applications.

Apache KafkaEvent StreamingKubernetes
0 likes · 15 min read
Spring Cloud Data Flow: Building and Deploying Event Stream Pipelines with Apache Kafka (Part 3)
Architects Research Society
Architects Research Society
Oct 16, 2019 · Backend Development

Deep Dive into Using Apache Kafka with Spring Boot: Error Handling, Message Conversion, and Transaction Support

This article explains how to integrate Apache Kafka with Spring Boot, covering error handling with SeekToCurrentErrorHandler, deserialization error handling, type‑inferred message conversion, multi‑listener routing, and transactional message processing, providing code examples and configuration details for each feature.

Apache KafkaError HandlingJava
0 likes · 10 min read
Deep Dive into Using Apache Kafka with Spring Boot: Error Handling, Message Conversion, and Transaction Support
DataFunTalk
DataFunTalk
May 27, 2019 · Big Data

Practical Applications and Ecosystem Integration of Apache Kafka

This article explores Apache Kafka’s evolution, core messaging and stream processing capabilities, typical use cases, internal storage mechanisms, API choices, and best practices for deploying Kafka on Kubernetes, providing readers with comprehensive guidance to assess suitability and implement effective Kafka solutions.

Apache KafkaKafka APIsKubernetes
0 likes · 16 min read
Practical Applications and Ecosystem Integration of Apache Kafka
Big Data Technology & Architecture
Big Data Technology & Architecture
May 19, 2019 · Big Data

Implementing End-to-End Exactly-Once Semantics in Apache Flink with Apache Kafka Using Two-Phase Commit Sink

This article explains how Apache Flink’s TwoPhaseCommitSinkFunction, introduced in version 1.4, enables end-to-end exactly-once semantics when integrated with Apache Kafka, detailing the checkpoint mechanism and the two-phase commit protocol that ensures reliable data processing.

Apache FlinkApache KafkaBig Data
0 likes · 4 min read
Implementing End-to-End Exactly-Once Semantics in Apache Flink with Apache Kafka Using Two-Phase Commit Sink
Programmer DD
Programmer DD
May 5, 2019 · Operations

20 Proven Kafka Best Practices to Scale High‑Throughput Streams

This article presents New Relic’s 20 best‑practice recommendations for Apache Kafka, covering partitions, consumers, producers, and brokers, to help engineers design, configure, and monitor high‑throughput, reliable streaming pipelines at scale.

Apache KafkaHigh Throughput
0 likes · 13 min read
20 Proven Kafka Best Practices to Scale High‑Throughput Streams
Qunar Tech Salon
Qunar Tech Salon
Nov 14, 2018 · Big Data

Comparing Apache Pulsar and Apache Kafka: Message Models, Consumption, Acknowledgment, Retention, and Architecture

This article provides a detailed comparison between Apache Pulsar and Apache Kafka, covering their message consumption models (queue vs. stream), subscription types, acknowledgment mechanisms, retention policies, and underlying layered architecture, highlighting Pulsar's unified API and segment‑based storage advantages.

Apache KafkaApache PulsarDistributed Systems
0 likes · 21 min read
Comparing Apache Pulsar and Apache Kafka: Message Models, Consumption, Acknowledgment, Retention, and Architecture
21CTO
21CTO
Nov 7, 2018 · Big Data

Why Data Streams Are the Backbone of Real-Time Big Data Analytics

Data streams, akin to endless rivers, enable continuous, real-time processing of diverse sources such as IoT telemetry, web logs, and e-commerce events, offering advantages over batch processing, while presenting challenges like scalability and fault tolerance, and are supported by tools like Kinesis, Kafka, Flink, and Storm.

Amazon KinesisApache KafkaBig Data
0 likes · 6 min read
Why Data Streams Are the Backbone of Real-Time Big Data Analytics
21CTO
21CTO
Jul 11, 2018 · Backend Development

Understanding Data Streams: From Node.js to Java, Kafka, and Kinesis

This article explains what data streams are, how they differ from arrays, the types of streams in Node.js, demonstrates Java Stream operations, and introduces popular streaming platforms like Apache Kafka and Amazon Kinesis, highlighting their core features and real‑time processing capabilities.

Amazon KinesisApache KafkaBackend Development
0 likes · 7 min read
Understanding Data Streams: From Node.js to Java, Kafka, and Kinesis
Java High-Performance Architecture
Java High-Performance Architecture
May 22, 2018 · Big Data

Is Apache Kafka Right for You? Core Features, Stream Processing, and Use Cases

This article explains Apache Kafka’s evolution and adoption by Fortune‑500 firms, outlines its two core capabilities—messaging (queue and publish/subscribe) and stream processing via the Java Stream API—provides example code, typical use cases, and guidance on scenarios where Kafka may not be the best solution.

Apache KafkaUse Casesstream processing
0 likes · 5 min read
Is Apache Kafka Right for You? Core Features, Stream Processing, and Use Cases
Art of Distributed System Architecture Design
Art of Distributed System Architecture Design
Mar 30, 2016 · Big Data

The Growing Role of Apache Kafka in Modern Big Data Architectures

The article explains how Apache Kafka has become a pivotal, high‑scalable publish‑subscribe system in the big‑data ecosystem, addressing the limitations of traditional databases, enabling real‑time data integration across specialized distributed systems, and shaping future data‑governance practices.

Apache KafkaData IntegrationStreaming
0 likes · 7 min read
The Growing Role of Apache Kafka in Modern Big Data Architectures
21CTO
21CTO
Sep 14, 2015 · Backend Development

Why Apache Kafka Beats Traditional Message Queues: Architecture, Code, and Performance

This article explains Apache Kafka's distributed publish‑subscribe design, core components, storage model, ZooKeeper coordination, performance benchmarks against ActiveMQ and RabbitMQ, and provides Java producer and consumer code examples for building high‑throughput messaging applications.

Apache KafkaDistributed MessagingMessage Queue
0 likes · 16 min read
Why Apache Kafka Beats Traditional Message Queues: Architecture, Code, and Performance
Art of Distributed System Architecture Design
Art of Distributed System Architecture Design
Apr 15, 2015 · Big Data

Understanding Stream Processing, Event Sourcing, and Complex Event Processing

The article explains the fundamentals of stream processing, event sourcing, and complex event processing, comparing raw event storage with aggregated results, illustrating architectures with Kafka, Samza, and other frameworks, and highlighting benefits such as scalability, flexibility, and decoupling for modern data‑driven systems.

Apache KafkaApache SamzaBig Data
0 likes · 11 min read
Understanding Stream Processing, Event Sourcing, and Complex Event Processing