Offset Management

Top Architect

Jan 19, 2025 · Backend Development

Understanding Kafka Consumer: Offset Management, Rebalance, Partition Assignment, and Thread Safety

This article provides a comprehensive technical walkthrough of KafkaConsumer, covering Java configuration code, delivery semantics (at‑most‑once, at‑least‑once, exactly‑once), offset commit strategies, rebalance mechanisms, partition assignment algorithms, thread‑safety concerns, and internal poll implementation, followed by unrelated promotional content.

ConsumerKafkaOffset Management

0 likes · 16 min read

Understanding Kafka Consumer: Offset Management, Rebalance, Partition Assignment, and Thread Safety

Sanyou's Java Diary

Oct 10, 2024 · Big Data

Why Kafka Handles Millions of Messages Per Second: Inside Its High‑Performance Architecture

This article explains how Kafka achieves ultra‑high throughput and low latency despite being disk‑based, covering its Reactor I/O network model, zero‑copy techniques, partitioning strategies, segment logs with sparse indexes, sequential disk writes, page cache usage, compression, batch processing, and lock‑free offset management.

Big DataKafkaOffset Management

0 likes · 27 min read

Why Kafka Handles Millions of Messages Per Second: Inside Its High‑Performance Architecture

Architect's Guide

Apr 28, 2024 · Backend Development

Kafka Consumer Usage Example and Deep Dive into Offset Management, Rebalance, and Thread Safety

This article presents a Java Kafka consumer example, explains offset semantics (at‑most‑once, at‑least‑once, exactly‑once), details consumer rebalance mechanisms, partition assignment strategies, thread‑safety considerations, and showcases core poll, heartbeat, and auto‑commit implementations with accompanying code snippets.

ConsumerJavaKafka

0 likes · 14 min read

Kafka Consumer Usage Example and Deep Dive into Offset Management, Rebalance, and Thread Safety

Architecture Digest

Mar 25, 2021 · Big Data

Uber's Multi-Region Kafka Architecture and Disaster Recovery

This article explains how Uber built a multi‑region Kafka infrastructure with disaster‑recovery capabilities, detailing its replication topology, active/active and active/passive consumption modes, offset‑management service, and the challenges of ensuring reliable, low‑latency data streaming across regions.

KafkaOffset Managementdata streaming

0 likes · 9 min read

Uber's Multi-Region Kafka Architecture and Disaster Recovery

Architecture Digest

Aug 8, 2019 · Big Data

Kafka Practical Guide: Concepts, Architecture, Configuration, Monitoring, and Management

This article provides a comprehensive overview of Kafka, covering its basic concepts, architecture, deployment, configuration, monitoring, producer and consumer settings, offset management, high availability, replication, leader election, and practical tips for deployment, tuning, and troubleshooting in production environments.

High AvailabilityKafkaMessage Queue

0 likes · 37 min read

Kafka Practical Guide: Concepts, Architecture, Configuration, Monitoring, and Management