BestHub
Sign in
Tagged articles

Producer

90 articles · Page 1 of 1
Cloud Architecture
Cloud Architecture
Jun 5, 2026 · Backend Development

RocketMQ 4.x Deep Dive: Send Mechanics, Thread Model & High‑Concurrency

This article explains why a RocketMQ producer is more than a simple send API, detailing its role in system throughput, consistency and fault isolation, and walks through sending principles, thread models, high‑concurrency design, retry strategies, back‑pressure mechanisms, outbox integration and production‑grade monitoring.

Message QueueProducerRocketMQ
0 likes · 54 min read
RocketMQ 4.x Deep Dive: Send Mechanics, Thread Model & High‑Concurrency
Programmer1970
Programmer1970
May 24, 2026 · Operations

Full-Stack Guide to Tracing RocketMQ Message Loss

This guide walks you through the entire lifecycle of a RocketMQ message, identifies where loss can occur, provides step‑by‑step diagnostics using TrackType and timestamps, and offers concrete configuration, command‑line and recovery strategies for producers, brokers and consumers.

BrokerConfigurationMessage Loss
0 likes · 11 min read
Full-Stack Guide to Tracing RocketMQ Message Loss
samdeepthink
samdeepthink
May 8, 2026 · Backend Development

Understanding RocketMQ’s Three Reliability Checkpoints

The article breaks down RocketMQ’s end‑to‑end reliability design into three checkpoints—producer to broker, broker persistence, and consumer acknowledgment—explaining each mechanism, configuration options, common pitfalls, and practical recommendations for production deployments.

BrokerMessage QueueProducer
0 likes · 16 min read
Understanding RocketMQ’s Three Reliability Checkpoints
Java Tech Workshop
Java Tech Workshop
Apr 27, 2026 · Backend Development

How to Integrate Kafka with SpringBoot for High‑Performance Messaging

This article walks through Kafka’s core architecture, explains why it achieves massive throughput, and provides a step‑by‑step SpringBoot integration—including environment setup, Maven dependencies, configuration, producer and consumer code, advanced features like transactions and dead‑letter queues, plus performance monitoring and tuning tips.

Dead Letter QueueKafkaMessage Queue
0 likes · 11 min read
How to Integrate Kafka with SpringBoot for High‑Performance Messaging
Java Tech Workshop
Java Tech Workshop
Apr 26, 2026 · Backend Development

Integrating Spring Boot with RocketMQ for Message Production and Consumption

This tutorial explains why RocketMQ is chosen for high‑throughput messaging, outlines its core components and typical scenarios, guides environment setup via Docker or local installation, shows Maven integration, provides Spring Boot configuration, and presents complete producer and consumer code with testing steps, extensions for sync/async messages, and common troubleshooting tips.

DockerMavenMessage Queue
0 likes · 16 min read
Integrating Spring Boot with RocketMQ for Message Production and Consumption
Architect Chen
Architect Chen
Nov 4, 2025 · Backend Development

Mastering RocketMQ: Architecture, Workflow, and Real‑World Use Cases

This article provides a comprehensive overview of RocketMQ, covering its core components (Producer, Broker, Consumer, NameServer), detailed architecture diagrams, step‑by‑step message production and consumption processes, and typical scenarios such as asynchronous decoupling, traffic shaping, transactional messaging, and event‑driven logging.

Backend ArchitectureBrokerDistributed Messaging
0 likes · 6 min read
Mastering RocketMQ: Architecture, Workflow, and Real‑World Use Cases
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Jul 14, 2025 · Backend Development

Kafka Producer Idempotency: PID, Sequence Numbers, and Broker Deduplication

Kafka ensures that a producer’s repeated message sends, caused by network glitches or broker failures, result in only one persisted record per partition by using a unique Producer ID, monotonically increasing sequence numbers, and broker-side tracking of the latest committed sequence for each PID‑partition pair.

KafkaProducerbackend
0 likes · 5 min read
Kafka Producer Idempotency: PID, Sequence Numbers, and Broker Deduplication
Senior Tony
Senior Tony
Jun 8, 2025 · Backend Development

Why Kafka Handles Millions of Messages Per Second: Batch, Partition, Zero‑Copy, and Compression Explained

This article breaks down the core techniques that give Kafka its high‑throughput capability, including producer batch settings (batch.size, linger.ms), broker append‑only writes, consumer poll configuration, partition distribution, zero‑copy data transfer, dual‑thread processing, and configurable compression algorithms.

Batch ProcessingHigh ThroughputPartitioning
0 likes · 8 min read
Why Kafka Handles Millions of Messages Per Second: Batch, Partition, Zero‑Copy, and Compression Explained
vivo Internet Technology
vivo Internet Technology
Jun 4, 2025 · Cloud Native

Deep Dive into Apache Pulsar Producer: Architecture, Parameters, and Performance Tuning

This article provides a comprehensive analysis of Apache Pulsar's Producer component, detailing its message-sending workflow, key design principles, configuration parameters, and practical performance tuning techniques to improve throughput, reduce latency, and lower resource consumption in large‑scale cloud‑native messaging systems.

Apache PulsarCloud NativeProducer
0 likes · 26 min read
Deep Dive into Apache Pulsar Producer: Architecture, Parameters, and Performance Tuning
Java Captain
Java Captain
May 23, 2025 · Backend Development

Common Causes of Kafka Message Loss and Mitigation Strategies

This article examines the typical reasons Kafka messages are lost across producers, brokers, and consumers, and provides detailed configuration recommendations and best‑practice solutions to significantly reduce the risk of data loss in distributed streaming systems.

BrokerConfigurationKafka
0 likes · 15 min read
Common Causes of Kafka Message Loss and Mitigation Strategies
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
May 22, 2025 · Big Data

Optimizing Kafka Producer for High Throughput: Batching, Asynchronous Sending, Compression, and Concurrency

This article details practical techniques for boosting Kafka producer throughput, covering batch sending optimization, linger.ms tuning, asynchronous send handling, compression options, and concurrent sending strategies, while providing code examples and performance considerations for high‑throughput streaming applications.

AsynchronousHigh ThroughputKafka
0 likes · 4 min read
Optimizing Kafka Producer for High Throughput: Batching, Asynchronous Sending, Compression, and Concurrency
Xuanwu Backend Tech Stack
Xuanwu Backend Tech Stack
Apr 8, 2025 · Backend Development

How to Prevent Message Loss in RabbitMQ: Producer, Server, and Consumer Strategies

This article explains why messages can be lost in RabbitMQ at the producer, broker, or consumer stages and provides concrete techniques—including confirm mode, transaction handling, persistence settings, manual acknowledgments, and retry mechanisms with dead‑letter queues—to ensure reliable delivery.

Message QueueMessage reliabilityProducer
0 likes · 11 min read
How to Prevent Message Loss in RabbitMQ: Producer, Server, and Consumer Strategies
DaTaobao Tech
DaTaobao Tech
Aug 14, 2024 · Backend Development

Deep Dive into RocketMQ Architecture and Core Components

The article thoroughly explains RocketMQ’s architecture—including stateless NameServers for routing, master‑slave Brokers with sequential CommitLog storage, stateless Producers and pull‑or‑push Consumers, message flow, indexing, and synchronous double‑write reliability—showing how these components deliver high‑throughput, scalable, fault‑tolerant messaging for distributed systems.

BrokerMessage QueueProducer
0 likes · 20 min read
Deep Dive into RocketMQ Architecture and Core Components
Rare Earth Juejin Tech Community
Rare Earth Juejin Tech Community
Jul 22, 2024 · Big Data

Comprehensive Guide to Kafka: Architecture, Core Concepts, and Configuration

This article provides an in‑depth overview of Apache Kafka, covering its use cases, comparison with other message queues, versioning, performance mechanisms, core concepts such as topics, partitions, offsets, consumer groups, rebalancing, replication, leader election, idempotence, transactions, compression, interceptors, request handling, and practical configuration tips for reliable streaming applications.

Big DataKafkaMessage Queue
0 likes · 25 min read
Comprehensive Guide to Kafka: Architecture, Core Concepts, and Configuration
MaGe Linux Operations
MaGe Linux Operations
Nov 12, 2023 · Backend Development

Why RocketMQ Producers Need Load Balancing and How It Works

This article explains why load balancing is crucial for RocketMQ producers, details the internal selection algorithm using ThreadLocal indexes and round‑robin logic, and provides Java code examples illustrating how messages are distributed across queues.

Load BalancingMessage QueueProducer
0 likes · 5 min read
Why RocketMQ Producers Need Load Balancing and How It Works
Code Ape Tech Column
Code Ape Tech Column
Jul 4, 2023 · Backend Development

Guide to Using Apache Pulsar Java Client with Spring Boot

This tutorial explains how to deploy a single‑node Pulsar cluster with Docker, configure the Java client in a Spring Boot application, and create producers and consumers with detailed code examples and parameter explanations.

Apache PulsarProducerSpring Boot
0 likes · 11 min read
Guide to Using Apache Pulsar Java Client with Spring Boot
Architect's Guide
Architect's Guide
Jun 6, 2023 · Backend Development

Kafka Core Concepts, Architecture, Performance Tuning, and Cluster Capacity Planning

This article provides a comprehensive overview of Kafka, covering its core value for decoupling and asynchronous processing, fundamental concepts such as producers, consumers, topics, partitions and replication, high‑performance mechanisms like zero‑copy and OS cache, detailed resource evaluation for CPU, memory, disk and network, operational tools, consumer‑group rebalance strategies, LEO/HW offsets, controller management, and delayed‑task scheduling.

Cluster PlanningKafkaMessage Queue
0 likes · 29 min read
Kafka Core Concepts, Architecture, Performance Tuning, and Cluster Capacity Planning
FunTester
FunTester
May 5, 2023 · Big Data

Kafka Client API Performance Testing with Producer and Consumer Examples

This article introduces Kafka, a high‑performance distributed messaging system, and provides step‑by‑step Java/Groovy examples for configuring a producer and consumer, demonstrating how to benchmark their throughput using the FunQpsConcurrent framework, along with necessary Gradle dependencies and server setup instructions.

Distributed MessagingKafkaProducer
0 likes · 8 min read
Kafka Client API Performance Testing with Producer and Consumer Examples
Top Architect
Top Architect
Mar 30, 2023 · Backend Development

Understanding Kafka Idempotent Producer and How to Prevent Message Duplicates

This article explains why message duplication occurs in Kafka, describes the three delivery semantics, and provides practical solutions—including idempotent producers, transactions, and consumer-side idempotence—along with configuration tips and code examples to achieve exactly‑once delivery.

ConfigurationKafkaMessage Duplication
0 likes · 11 min read
Understanding Kafka Idempotent Producer and How to Prevent Message Duplicates
Top Architect
Top Architect
Mar 27, 2023 · Big Data

Kafka Architecture, Performance Optimization, and Production Deployment Guide

This article provides a comprehensive overview of Kafka’s core concepts, high‑performance design, cluster planning, resource evaluation, deployment steps, producer and consumer configurations, fault‑tolerance mechanisms, and operational tools, offering practical guidance for building and managing a high‑throughput Kafka production environment.

Cluster DeploymentKafkaProducer
0 likes · 31 min read
Kafka Architecture, Performance Optimization, and Production Deployment Guide
Architecture Digest
Architecture Digest
Feb 9, 2023 · Big Data

Understanding Kafka Messages, Topics, Partitions, and Consumers

This article explains Kafka's core concepts—including messages as byte arrays, optional keys for partition control, topic and partition organization, producer and consumer roles, offsets, consumer groups, and broker clusters—providing a concise technical overview for developers learning Kafka.

KafkaMessageProducer
0 likes · 6 min read
Understanding Kafka Messages, Topics, Partitions, and Consumers
Java High-Performance Architecture
Java High-Performance Architecture
Nov 5, 2022 · Big Data

Why Can Kafka Process 20 Million Messages per Second? Inside Its High‑Performance Architecture

This article explains how Kafka achieves extremely high throughput—up to 20 million messages and 600 MB per second per node—by optimizing the producer, broker, and consumer components through batch sending, custom protocols, page‑cache usage, zero‑copy transfers, and efficient compression algorithms.

BrokerKafkaProducer
0 likes · 7 min read
Why Can Kafka Process 20 Million Messages per Second? Inside Its High‑Performance Architecture
Tencent Cloud Developer
Tencent Cloud Developer
Sep 26, 2022 · Big Data

Kafka Architecture Overview and Core Concepts

Kafka’s architecture consists of brokers forming clusters, producers publishing to topics split into partitions with replicas, consumers organized in groups pulling messages by offset, ZooKeeper managing metadata, and log‑based storage using append‑only files, indexes, and zero‑copy, while configurable acknowledgment, batching, and replication ensure high throughput and fault‑tolerant reliability.

KafkaProducerconsumer
0 likes · 18 min read
Kafka Architecture Overview and Core Concepts
IT Architects Alliance
IT Architects Alliance
Sep 6, 2022 · Operations

How to Guarantee Zero Message Loss with Kafka: Best Practices and Configurations

This article explains why introducing a message queue like Kafka helps decouple systems and control traffic, then dives into the three key questions of detecting, locating, and preventing message loss, offering concrete monitoring methods, configuration settings, and troubleshooting steps for producers, brokers, and consumers.

BrokerConfigurationData Consistency
0 likes · 13 min read
How to Guarantee Zero Message Loss with Kafka: Best Practices and Configurations
Wukong Talks Architecture
Wukong Talks Architecture
Sep 2, 2022 · Big Data

Preventing Data Loss in Kafka: Message Semantics, Failure Scenarios, and Reliability Solutions

This article explains Kafka's message delivery semantics, analyzes potential data‑loss scenarios across producer, broker, and consumer components, and provides concrete configuration and coding practices—such as idempotent producers, proper ACK settings, replication factors, and manual offset commits—to maximize message durability and reliability.

BrokerData lossKafka
0 likes · 18 min read
Preventing Data Loss in Kafka: Message Semantics, Failure Scenarios, and Reliability Solutions
Top Architect
Top Architect
Aug 31, 2022 · Big Data

Kafka Components and Architecture Overview

This article provides a comprehensive overview of Kafka’s core components—including brokers, topics, partitions, producers, and consumers—explaining their roles, relationships, replication, leader/follower dynamics, and architectural diagrams, while also highlighting configuration details and operational considerations for building reliable distributed streaming systems.

BrokerProducerTopic
0 likes · 9 min read
Kafka Components and Architecture Overview
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Aug 21, 2022 · Backend Development

12 Core Principles of Message Queues (MQ) – A Comprehensive Summary

This article provides a detailed overview of message queue fundamentals, covering producers, consumers, brokers, queue and topic models, delivery guarantees, acknowledgment mechanisms, ordering, persistence, high availability, and selection criteria, helping readers quickly grasp essential MQ concepts for distributed systems.

BrokerMessage QueuePersistence
0 likes · 9 min read
12 Core Principles of Message Queues (MQ) – A Comprehensive Summary
IT Architects Alliance
IT Architects Alliance
Aug 4, 2022 · Big Data

Explanation of Kafka Components and Architecture

This article provides a comprehensive overview of Kafka's core components—including brokers, topics, partitions, producers, and consumers—explaining their roles, how they interact within a Kafka cluster, and illustrating the architecture with diagrams while also noting recent version changes and related disclaimer information.

BrokerKafkaProducer
0 likes · 7 min read
Explanation of Kafka Components and Architecture
Top Architect
Top Architect
Jul 21, 2022 · Big Data

Kafka Components and Architecture Explained

This article provides a comprehensive overview of Kafka's core components—including brokers, topics, partitions, producers, and consumers—explaining their roles, relationships, replication strategies, and how they work together within a Kafka cluster to ensure reliable, scalable message streaming.

BrokerKafkaProducer
0 likes · 8 min read
Kafka Components and Architecture Explained
Architecture Digest
Architecture Digest
Jun 15, 2022 · Backend Development

Understanding RocketMQ Producer and Consumer Load Balancing Mechanisms

This article explains RocketMQ’s overall architecture and details how producers and consumers achieve load balancing through routing synchronization, queue selection, and rebalance strategies, illustrated with code examples and a case study of assigning consumption to a specific machine.

ProducerRocketMQconsumer
0 likes · 20 min read
Understanding RocketMQ Producer and Consumer Load Balancing Mechanisms
JavaEdge
JavaEdge
Apr 6, 2022 · Big Data

When Does Kafka Lose Data? Proven Strategies to Prevent Message Loss

This article explains Kafka's message delivery semantics, identifies the exact scenarios where data can be lost in producer, broker, and consumer components, and provides concrete configuration and coding practices to ensure reliable, at‑least‑once or exactly‑once delivery in production environments.

BrokerData ReliabilityKafka
0 likes · 18 min read
When Does Kafka Lose Data? Proven Strategies to Prevent Message Loss
vivo Internet Technology
vivo Internet Technology
Mar 30, 2022 · Backend Development

Load Balancing Mechanism in RocketMQ Producer and Consumer

The article explains RocketMQ’s load‑balancing for producers and consumers, detailing how producers obtain routing info from the NameServer, select MessageQueues via round‑robin while avoiding failed brokers, and how consumers periodically rebalance queue allocation using strategies such as AllocateMessageQueueAveragely to ensure even distribution and customizable assignment.

ProducerRocketMQconsumer
0 likes · 22 min read
Load Balancing Mechanism in RocketMQ Producer and Consumer
Architect
Architect
Feb 12, 2022 · Big Data

In-Depth Overview of Apache Kafka Architecture and Core Concepts

This article provides a comprehensive introduction to Apache Kafka, covering its distributed streaming platform features, message queue patterns, topic and partition design, broker and cluster roles, producer and consumer mechanics, partition assignment strategies, data storage, reliability guarantees, and performance optimizations such as zero‑copy and batch processing.

Message QueueProducerStreaming
0 likes · 23 min read
In-Depth Overview of Apache Kafka Architecture and Core Concepts
Tencent Cloud Developer
Tencent Cloud Developer
Jan 11, 2022 · Cloud Native

Apache Pulsar: Architecture, Topics, Producers, Consumers, and Storage

Apache Pulsar is a cloud‑native distributed messaging system that combines stateless brokers, a BookKeeper storage cluster, and ZooKeeper metadata to route messages from producers to consumers across scalable partitioned or non‑partitioned topics, supporting multiple subscription models, producer access modes, routing strategies, batching, acknowledgments, delayed delivery, and deduplication.

Apache PulsarBookKeeperCloud Native
0 likes · 26 min read
Apache Pulsar: Architecture, Topics, Producers, Consumers, and Storage
IT Architects Alliance
IT Architects Alliance
Dec 3, 2021 · Big Data

Comprehensive Overview of Apache Kafka Architecture and Core Concepts

This article provides an in‑depth technical guide to Apache Kafka, covering its distributed streaming architecture, core concepts such as topics, partitions, brokers, producers and consumers, reliability guarantees, storage mechanisms, configuration parameters, and consumer assignment strategies, supplemented with Java code examples.

Apache KafkaDistributed StreamingPartitioning
0 likes · 24 min read
Comprehensive Overview of Apache Kafka Architecture and Core Concepts
Big Data Technology & Architecture
Big Data Technology & Architecture
Aug 21, 2021 · Big Data

Kafka Overview: Background, Core Concepts, Producer/Consumer Configuration, Core Principles, Operations, and Stream Processing

This article provides a comprehensive beginner-friendly guide to Apache Kafka, covering its background, core concepts, producer and consumer settings with code examples, underlying architecture, operational monitoring, integration with Spark and Flink, and an introduction to Kafka Streams.

ProducerStreamingconsumer
0 likes · 19 min read
Kafka Overview: Background, Core Concepts, Producer/Consumer Configuration, Core Principles, Operations, and Stream Processing
360 Quality & Efficiency
360 Quality & Efficiency
Jul 23, 2021 · Big Data

Introduction to Apache Kafka: Architecture, APIs, Code Examples, and Optimization

This article provides a comprehensive overview of Apache Kafka, covering its definition, architecture, command‑line API usage, Java producer and consumer examples, core principles such as high availability and message ordering, configuration tuning, and a summary of its advantages as a high‑throughput, fault‑tolerant streaming platform.

ConfigurationKafkaMessage Queue
0 likes · 12 min read
Introduction to Apache Kafka: Architecture, APIs, Code Examples, and Optimization
Selected Java Interview Questions
Selected Java Interview Questions
Jul 19, 2021 · Backend Development

Message Queue Concepts and Consumption Scenarios

This article explains the core concepts of message queues—including producers, consumers, messages, brokers, and push/pull delivery—and analyzes three consumption scenarios: at‑most‑once, at‑least‑once, and exactly‑once, detailing the required producer, broker, and consumer behaviors for each.

Message BrokerMessage QueueProducer
0 likes · 6 min read
Message Queue Concepts and Consumption Scenarios
Java Interview Crash Guide
Java Interview Crash Guide
Jun 25, 2021 · Backend Development

Understanding Kafka Transactions: TC Service, Producer Flow, and Code Walkthrough

This article explains how Kafka implements transactions, detailing the role of the Transaction Coordinator (TC) service, the transaction flow diagram, producer initialization, partition handling, offset commits, commit and abort processes, and includes a complete Java code example with client‑side and server‑side components.

KafkaProducerTransactions
0 likes · 22 min read
Understanding Kafka Transactions: TC Service, Producer Flow, and Code Walkthrough
Java Architect Essentials
Java Architect Essentials
Jun 15, 2021 · Big Data

Comprehensive Guide to Apache Kafka: Concepts, Installation, Configuration, and Usage

This article provides a thorough overview of Apache Kafka, covering its core streaming concepts, key components such as topics, partitions, producers and consumers, common use cases, step‑by‑step installation and multi‑broker configuration, fault‑tolerance testing, and an introduction to Kafka Connect for data import/export.

Distributed StreamingInstallationKafka
0 likes · 24 min read
Comprehensive Guide to Apache Kafka: Concepts, Installation, Configuration, and Usage
Java High-Performance Architecture
Java High-Performance Architecture
May 15, 2021 · Backend Development

How to Ensure Ordered Messaging with RabbitMQ and Kafka

This article explains how to achieve ordered message processing by coordinating both producers and consumers, covering the differences between RabbitMQ's simple queue ordering and Kafka's partition-based approach, and offering practical techniques for global and partial ordering.

KafkaOrderingProducer
0 likes · 5 min read
How to Ensure Ordered Messaging with RabbitMQ and Kafka
Spring Full-Stack Practical Cases
Spring Full-Stack Practical Cases
Apr 12, 2021 · Big Data

Master Kafka’s Delivery Guarantees: At‑Most, At‑Least, and Exactly‑Once Explained

This article explains Kafka’s three delivery semantics—At most once, At least once, and Exactly once—from both producer and consumer perspectives, details the required configuration settings, and discusses how Kafka ensures idempotence, transaction support, and prevents data loss, duplication, and ordering issues.

Delivery GuaranteesKafkaProducer
0 likes · 23 min read
Master Kafka’s Delivery Guarantees: At‑Most, At‑Least, and Exactly‑Once Explained
Full-Stack Internet Architecture
Full-Stack Internet Architecture
Apr 8, 2021 · Backend Development

Kafka Interview Guide: Concepts, Architecture, Configuration, and Performance

This article provides a comprehensive overview of Kafka, covering its role as a distributed messaging middleware, core concepts, architecture components, common interview questions, command‑line tools, producer and consumer configurations, high‑availability mechanisms, delivery semantics, and performance optimizations for backend developers.

Distributed MessagingKafkaProducer
0 likes · 20 min read
Kafka Interview Guide: Concepts, Architecture, Configuration, and Performance
Code Ape Tech Column
Code Ape Tech Column
Feb 26, 2021 · Backend Development

Why Kafka Messages Get Lost and How to Prevent It

This article explains the three places where Kafka can lose messages—Broker, Producer, and Consumer—detailing the underlying mechanisms, the impact of flush and ack settings, and practical configuration and coding strategies to minimize data loss.

Ack SettingsBrokerConfiguration
0 likes · 15 min read
Why Kafka Messages Get Lost and How to Prevent It
vivo Internet Technology
vivo Internet Technology
Jan 13, 2021 · Big Data

An Introduction to Apache Kafka: Architecture, Concepts, and Operations

Apache Kafka is a distributed, fault‑tolerant streaming platform that uses broker clusters, topic partitions, and replicated logs to provide publish/subscribe and queue messaging, configurable retention, strong ordering within partitions, producer acknowledgments, consumer groups, and performance optimizations such as batching and zero‑copy for real‑time data pipelines.

Apache KafkaDistributed StreamingMessage Queue
0 likes · 26 min read
An Introduction to Apache Kafka: Architecture, Concepts, and Operations
Programmer DD
Programmer DD
Jan 5, 2021 · Backend Development

Understanding Kafka Partition Assignment: Strategies and Code Walkthrough

This article explains how Kafka determines which partition a producer sends a record to, how partition counts are configured, and how consumer groups assign partitions using the default, range, and round‑robin strategies, complemented by detailed Java code examples.

KafkaPartition AssignmentProducer
0 likes · 21 min read
Understanding Kafka Partition Assignment: Strategies and Code Walkthrough
21CTO
21CTO
Dec 1, 2020 · Big Data

How Kafka Implements Transactions: Inside the TC Service and Producer Workflow

This article provides a comprehensive walkthrough of Kafka's transaction mechanism, covering the transaction coordinator, producer initialization, partition handling, commit and abort processes, state management, high‑availability design, timeout handling, and relevant source code snippets.

KafkaProducerTransaction Coordinator
0 likes · 22 min read
How Kafka Implements Transactions: Inside the TC Service and Producer Workflow
Tencent Cloud Developer
Tencent Cloud Developer
Nov 19, 2020 · Backend Development

Kafka Message Queue Reliability Design and Implementation

The article thoroughly explains Kafka’s message‑queue reliability design and implementation, covering use‑case scenarios, core concepts, storage format, producer acknowledgment settings, broker replication mechanisms (ISR, HW, LEO), consumer delivery semantics, the epoch solution for synchronization, and practical configuration guidelines for various consistency and availability requirements.

BrokerFault ToleranceKafka
0 likes · 15 min read
Kafka Message Queue Reliability Design and Implementation
Programmer DD
Programmer DD
Oct 29, 2020 · Backend Development

Master Kafka Interview Questions: Architecture, Configurations, and Best Practices

This article provides a comprehensive overview of Kafka as a distributed messaging middleware, covering its core concepts, architecture, producer and consumer mechanics, common interview questions, configuration options, high‑availability guarantees, and performance optimizations for backend developers.

Distributed MessagingKafkaMessage Queue
0 likes · 20 min read
Master Kafka Interview Questions: Architecture, Configurations, and Best Practices
JavaEdge
JavaEdge
Aug 13, 2020 · Backend Development

How RocketMQ Producer Works: A Deep Dive into Init, Startup, and Message Sending

This article examines the RocketMQ client’s Producer implementation using the latest release‑4.7.1 source and unit tests, explaining the facade pattern, startup sequence, state management, and the three message‑sending modes with detailed code walkthroughs and design insights.

ProducerRocketMQjava
0 likes · 13 min read
How RocketMQ Producer Works: A Deep Dive into Init, Startup, and Message Sending
Big Data Technology & Architecture
Big Data Technology & Architecture
Jul 23, 2020 · Big Data

Comprehensive Kafka FAQ: Uses, Architecture, Offsets, and Partition Management

This article provides an extensive overview of Apache Kafka, covering its use cases, key concepts such as ISR, AR, HW, LEO, and LW, message ordering, the roles of partitioners, serializers and interceptors, producer and consumer client architecture, offset handling, multithreaded consumption, and topic partition management.

Big DataKafkaMessage Queue
0 likes · 16 min read
Comprehensive Kafka FAQ: Uses, Architecture, Offsets, and Partition Management
Big Data Technology & Architecture
Big Data Technology & Architecture
Jul 22, 2020 · Big Data

Kafka Architecture and Core Concepts: Producers, Brokers, and Consumers

This article explains Kafka's fundamental architecture, including the roles of producers, brokers, and consumers, key concepts such as topics, partitions, replicas, ISR, and controller, as well as detailed mechanisms of producer client structure, interceptors, serializers, partitioners, and consumer group rebalancing strategies.

Big DataKafkaProducer
0 likes · 22 min read
Kafka Architecture and Core Concepts: Producers, Brokers, and Consumers
Architecture Digest
Architecture Digest
May 16, 2020 · Backend Development

RocketMQ Producer and Consumer Best Practices and Configuration Guide

This article provides a comprehensive guide to using RocketMQ, covering producer and consumer message handling, tag and key usage, logging, failure retry strategies, oneway sending, broker roles, name server addressing, client configuration, JVM and Linux kernel tuning, and practical code examples.

ConfigurationMessage QueueProducer
0 likes · 19 min read
RocketMQ Producer and Consumer Best Practices and Configuration Guide
Top Architect
Top Architect
May 14, 2020 · Big Data

Kafka Overview, Architecture, Installation, and Operational Guide

This article provides a comprehensive introduction to Kafka, covering its definition, message queue concepts, architecture components, installation steps, configuration details, startup procedures, operational commands, producer and consumer mechanisms, reliability guarantees, partition strategies, offset management, and performance optimizations.

Big DataInstallationKafka
0 likes · 22 min read
Kafka Overview, Architecture, Installation, and Operational Guide
Architecture Digest
Architecture Digest
May 3, 2020 · Big Data

Kafka Concept Overview

This article provides a comprehensive introduction to Kafka, covering its definition, message‑queue models, architecture components, installation steps, configuration details, producer and consumer mechanisms, reliability guarantees, partition assignment strategies, offset management, and high‑performance read/write techniques.

Big DataKafkaMessage Queue
0 likes · 20 min read
Kafka Concept Overview
Big Data Technology Architecture
Big Data Technology Architecture
Apr 22, 2020 · Big Data

Key Kafka Producer Configuration Parameters and Tuning Recommendations

This article explains the most important Kafka producer configuration parameters—such as acks, max.request.size, retries, compression.type, buffer.memory, batch.size, linger.ms, request.timeout.ms, and max.in.fight.requests.per.connection—provides practical tuning advice, and presents a recommended setup to achieve high throughput without message loss.

ConfigurationKafkaMessage reliability
0 likes · 9 min read
Key Kafka Producer Configuration Parameters and Tuning Recommendations
Big Data Technology Architecture
Big Data Technology Architecture
Apr 13, 2020 · Backend Development

Understanding Kafka Producer: Architecture, Data Structures, Serialization, Partitioning, and Buffering

This article provides a comprehensive overview of Kafka's Producer side, covering its architecture, the ProducerRecord data structure, serialization mechanisms, partitioning logic, and the accumulator buffer, while comparing old and new Producer clients and illustrating key configurations with code examples.

AccumulatorKafkaPartitioning
0 likes · 9 min read
Understanding Kafka Producer: Architecture, Data Structures, Serialization, Partitioning, and Buffering
ITFLY8 Architecture Home
ITFLY8 Architecture Home
Mar 6, 2020 · Big Data

Mastering Kafka: Build Producers, Consumers, and Custom Partitioners

This extensive tutorial walks through Kafka producer and consumer fundamentals, demonstrates how to configure key‑based and custom partitioning, explains offset management, consumer group coordination, and essential configuration parameters, and includes complete Java code examples for real‑world e‑commerce scenarios.

KafkaPartitioningProducer
0 likes · 24 min read
Mastering Kafka: Build Producers, Consumers, and Custom Partitioners
Big Data Technology & Architecture
Big Data Technology & Architecture
Nov 24, 2019 · Big Data

Common Apache Kafka Exceptions and Their Causes

This article lists frequent Apache Kafka exceptions such as UnknownTopicOrPartitionException, LEADER_NOT_AVAILABLE, NotLeaderForPartitionException, TimeoutException, RecordTooLargeException, and others, explaining each error message, typical reasons, and practical troubleshooting steps for producers and consumers.

Big DataExceptionsKafka
0 likes · 5 min read
Common Apache Kafka Exceptions and Their Causes
Architect's Tech Stack
Architect's Tech Stack
May 31, 2019 · Big Data

Kafka Architecture Overview: Producers, Consumers, Partitions, Replication, and Transactions

This article provides a comprehensive overview of Apache Kafka's architecture, covering topics such as producer and consumer workflows, partition and replica management, leader election, offset handling, message delivery semantics, transaction support, and file organization, illustrating how Kafka achieves high performance and scalability.

KafkaMessage QueueProducer
0 likes · 18 min read
Kafka Architecture Overview: Producers, Consumers, Partitions, Replication, and Transactions
Big Data Technology & Architecture
Big Data Technology & Architecture
May 17, 2019 · Backend Development

Understanding Kafka Producer Idempotence: PID, Sequence Numbers, and Implementation Details

This article explains how Apache Kafka implements producer idempotence by introducing Producer IDs (PID) and sequence numbers, describes the request‑response flow for PID allocation, details server‑side PID management, shows the exact‑once guarantee mechanism, and answers common configuration questions with code examples.

KafkaPiDProducer
0 likes · 32 min read
Understanding Kafka Producer Idempotence: PID, Sequence Numbers, and Implementation Details
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Dec 27, 2018 · Backend Development

Fundamentals of Message Queues: Producers, Consumers, Brokers, and Messaging Models

This article explains the core concepts, components, design considerations, and common models of message queues—including point‑to‑point and publish/subscribe—while covering ordering, acknowledgment, transaction support, persistence, high availability, and practical selection guidance for high‑concurrency systems.

BrokerMessage QueueProducer
0 likes · 8 min read
Fundamentals of Message Queues: Producers, Consumers, Brokers, and Messaging Models
Architecture Digest
Architecture Digest
Aug 7, 2018 · Big Data

Apache Kafka Overview, Architecture, and Sample Producer/Consumer Code

This article provides a comprehensive overview of Apache Kafka, comparing it with ActiveMQ, explaining its distributed architecture, topics, partitions, consumption models, high‑availability mechanisms, exactly‑once semantics, and includes detailed Java producer and consumer code examples for practical implementation.

Big DataDistributed MessagingKafka
0 likes · 22 min read
Apache Kafka Overview, Architecture, and Sample Producer/Consumer Code
Programmer DD
Programmer DD
Feb 10, 2018 · Backend Development

How RocketMQ Handles Transactional Messages: From Producer to Broker

This article explains the internal workflow of RocketMQ transactional messages, covering how producers send half messages, how brokers process commit or rollback requests, the storage mechanisms for transaction state, the periodic transaction check logic, and the differences between file‑system and database implementations.

BrokerMessage QueueProducer
0 likes · 22 min read
How RocketMQ Handles Transactional Messages: From Producer to Broker
21CTO
21CTO
Feb 23, 2016 · Big Data

Why Kafka Dominates Modern Data Pipelines: Architecture, Benefits, and Guarantees

Kafka, the open‑source distributed messaging system from LinkedIn, offers O(1) persistence, high throughput, partitioned topics, and flexible delivery guarantees, making it a cornerstone for modern big‑data pipelines and real‑time processing alongside Hadoop, Spark, and Storm.

Big DataDelivery GuaranteesDistributed Messaging
0 likes · 21 min read
Why Kafka Dominates Modern Data Pipelines: Architecture, Benefits, and Guarantees