Tagged articles

29 articles

Page 1 of 1

Jan 22, 2026 · Big Data

Unlock Kafka’s Power: Core Concepts, High‑Performance Architecture & Real‑World Scaling Tips

This comprehensive guide explores Kafka’s core value as a message queue, explains producers, consumers, topics, partitions, and replication, dives into cluster architecture, zero‑copy I/O, resource planning for disks, memory, CPU and network, and provides practical configuration, consumer‑group management, and operational tooling tips for building high‑throughput, highly available Kafka deployments.

Distributed SystemsKafkaMessage Queue

0 likes · 31 min read

Unlock Kafka’s Power: Core Concepts, High‑Performance Architecture & Real‑World Scaling Tips

Open Source Tech Hub

Jul 31, 2025 · Backend Development

Mastering Redis Streams: From Basics to Consumer Groups

This comprehensive guide explains Redis Streams—its data model, entry IDs, range queries, blocking reads, consumer groups, pending message handling, trimming, persistence, and performance characteristics—providing practical commands, examples, and best‑practice tips for building reliable streaming applications.

StreamsXADDXCLAIM

0 likes · 41 min read

Mastering Redis Streams: From Basics to Consumer Groups

Big Data Technology Architecture

Mar 25, 2025 · Big Data

Kafka 4.0 Release: KRaft Architecture, Consumer Group Optimizations, and New Queue Features

Kafka 4.0 marks a milestone release that replaces ZooKeeper with the KRaft consensus engine, improves scalability and performance, introduces a server‑side consumer‑group protocol, adds shared‑group queue capabilities, and updates Java requirements and documentation, delivering a more robust and flexible streaming platform.

Big DataDistributed StreamingJava11

0 likes · 6 min read

Kafka 4.0 Release: KRaft Architecture, Consumer Group Optimizations, and New Queue Features

Big Data Technology & Architecture

Mar 24, 2025 · Big Data

Apache Kafka 4.0: Major New Features – KRaft Architecture, Consumer Group Protocol, Queue Mode, Java Upgrade, API Simplifications and More

Apache Kafka 4.0 introduces a groundbreaking KRaft architecture that removes ZooKeeper, a revamped consumer‑group protocol that dramatically speeds up rebalancing, a new queue mode for point‑to‑point messaging, upgraded Java requirements, streamlined APIs, and numerous performance and security enhancements, reshaping both development and operations for large‑scale streaming workloads.

Java 11KRaftKafka

0 likes · 12 min read

Apache Kafka 4.0: Major New Features – KRaft Architecture, Consumer Group Protocol, Queue Mode, Java Upgrade, API Simplifications and More

FunTester

Jun 19, 2023 · Big Data

Kafka Architecture and Core Concepts: Brokers, Producers, Consumers, Topics, Partitions, Replicas, and Reliability

This article provides a comprehensive overview of Kafka's architecture and fundamental concepts, covering its overall structure, key components such as brokers, producers, consumers, topics, partitions, replicas, leader‑follower synchronization, offset handling, message storage at both logical and physical layers, as well as producer and consumer workflows, partition assignment strategies, rebalancing, log management, zero‑copy I/O, and reliability mechanisms.

Distributed SystemsKafkaLog Management

0 likes · 22 min read

Kafka Architecture and Core Concepts: Brokers, Producers, Consumers, Topics, Partitions, Replicas, and Reliability

Top Architect

May 15, 2023 · Backend Development

Comprehensive Guide to Kafka: Architecture, Performance Tuning, and Operational Practices

This article provides an in-depth overview of Kafka, covering its core value as a message queue, fundamental concepts, cluster architecture, producer and consumer configurations, scaling strategies, monitoring tools, and practical operational commands for building and maintaining high‑throughput, highly available streaming systems.

BackendKafkaMessage Queue

0 likes · 31 min read

Comprehensive Guide to Kafka: Architecture, Performance Tuning, and Operational Practices

Top Architect

Apr 30, 2023 · Backend Development

Kafka Core Concepts, Architecture, Performance, and Operational Practices

This article provides a comprehensive overview of Kafka, covering its core value as a message queue, fundamental concepts, cluster architecture, log storage mechanisms, zero‑copy data transfer, high‑throughput and high‑availability design, consumer group behavior, rebalance strategies, and practical operational commands for managing topics, partitions, and offsets.

BackendDistributed SystemsKafka

0 likes · 31 min read

Kafka Core Concepts, Architecture, Performance, and Operational Practices

Architecture Digest

Apr 23, 2023 · Backend Development

Kafka Core Concepts, Architecture, Performance Optimizations, and Production Deployment Guide

This article provides a comprehensive technical overview of Kafka, covering its core message‑queue value, architecture components such as producers, consumers, topics, partitions and replication, high‑performance mechanisms like zero‑copy and OS cache, resource planning for disks, memory, CPU and network, operational tools and commands, consumer‑group management, rebalance strategies, and internal scheduling mechanisms such as the time‑wheel.

Backend ArchitectureDistributed SystemsKafka

0 likes · 30 min read

Kafka Core Concepts, Architecture, Performance Optimizations, and Production Deployment Guide

IT Architects Alliance

Mar 28, 2023 · Big Data

Kafka Core Concepts, Architecture, Performance Optimization, and Production Deployment Guide

This comprehensive guide explains Kafka's core value as a message queue, its fundamental concepts, cluster architecture, high‑performance data handling, resource planning for large‑scale deployments, operational tools, consumer‑group mechanics, offset management, rebalance strategies, and custom partitioner implementation.

DeploymentReplicationStreaming

0 likes · 29 min read

Kafka Core Concepts, Architecture, Performance Optimization, and Production Deployment Guide

IT Architects Alliance

Aug 30, 2022 · Big Data

Understanding Kafka: Architecture, Topics, Partitions, Producers, Consumers, Offsets, Transactions, and Configuration

This article provides a comprehensive overview of Apache Kafka, explaining its distributed message‑queue architecture, the role of topics and partitions, producer and consumer workflows, leader election, offset management, consumer‑group rebalancing, delivery semantics, transaction processing, file organization, and key configuration settings.

Big DataDistributed MessagingKafka

0 likes · 17 min read

Architect

Aug 26, 2022 · Big Data

Deep Dive into Kafka Architecture: Topics, Partitions, Replication, Consumers, and Transactions

This article provides a comprehensive overview of Kafka's architecture, covering topics, partitions, replication, producer and consumer workflows, offset management, rebalancing, delivery semantics, exactly‑once guarantees, transaction handling, file organization, and key configuration settings.

Distributed MessagingExactly-OnceReplication

0 likes · 17 min read

Deep Dive into Kafka Architecture: Topics, Partitions, Replication, Consumers, and Transactions

Efficient Ops

Jun 7, 2022 · Big Data

Visualizing Kafka: Core Concepts Explained with Diagrams

This article visually breaks down Kafka’s fundamental concepts—including topics, partitions, producers, consumers, consumer groups, and cluster architecture—so readers can grasp how messages flow, are stored, and achieve load balancing and ordering within a distributed streaming platform.

Distributed SystemsKafkaMessage Queue

0 likes · 6 min read

Visualizing Kafka: Core Concepts Explained with Diagrams

Top Architect

Jun 1, 2022 · Big Data

Understanding Kafka Core Concepts: Topics, Partitions, Producers, Consumers, and Architecture

This article explains the fundamental concepts of Apache Kafka, including producers, consumers, topics, partitions, consumer groups, and the role of ZooKeeper in a clustered architecture, while illustrating how messages are stored, routed, and ordered for reliable, scalable stream processing.

KafkaPartitionsbig-data

0 likes · 6 min read

Understanding Kafka Core Concepts: Topics, Partitions, Producers, Consumers, and Architecture

Java High-Performance Architecture

May 23, 2022 · Big Data

Visualizing Kafka: Core Concepts, Topics, Partitions, and Architecture Explained

This article visually breaks down Kafka’s core concepts—including topics, partitions, producers, consumers, and cluster architecture—explaining how messages flow, how consumer groups achieve load balancing, and how ZooKeeper coordinates leaders and followers, providing a clear, illustrated guide for developers working with distributed streaming systems.

KafkaMessage QueuePartitions

0 likes · 6 min read

Visualizing Kafka: Core Concepts, Topics, Partitions, and Architecture Explained

IT Architects Alliance

Feb 13, 2022 · Big Data

Comprehensive Overview of Apache Kafka Architecture and Core Concepts

This article provides an in‑depth introduction to Apache Kafka, covering its distributed streaming platform fundamentals, message‑queue models, topic and partition design, broker and cluster roles, producer partitioning logic, reliability guarantees, consumer group assignors, offset management, and performance optimizations such as sequential disk writes and zero‑copy techniques.

Apache KafkaDistributed StreamingReliability

0 likes · 25 min read

Comprehensive Overview of Apache Kafka Architecture and Core Concepts

Wukong Talks Architecture

Dec 8, 2021 · Big Data

Understanding Kafka Core Concepts: Architecture, Messaging Models, Partitioning, Consumer Groups, and Reliability

This article provides a comprehensive overview of Kafka, covering its layered architecture with Zookeeper, core concepts such as topics, partitions and consumer groups, communication workflow, partition selection strategies, rebalancing mechanisms, reliability configurations, replica synchronization, and reasons for moving away from Zookeeper, all explained in clear English.

Distributed SystemsKafkaReliability

0 likes · 19 min read

Understanding Kafka Core Concepts: Architecture, Messaging Models, Partitioning, Consumer Groups, and Reliability

Top Architect

Nov 19, 2021 · Big Data

Understanding Kafka’s Design: Topics, Partitions, Consumer Groups, and Cluster Architecture

This article explains Kafka’s core design concepts—including the role of a message system, topics, partitions, producers, consumers, consumer groups, replica management, controller coordination with Zookeeper, performance optimizations like sequential writes and zero‑copy, and its network thread model—illustrated with diagrams and code snippets.

Cluster ArchitectureKafkaconsumer groups

0 likes · 12 min read

Understanding Kafka’s Design: Topics, Partitions, Consumer Groups, and Cluster Architecture

MaGe Linux Operations

May 12, 2021 · Big Data

Visualizing Kafka: Core Concepts Explained with Diagrams

This article visually breaks down Kafka’s core concepts—including producers, consumers, topics, partitions, consumer groups, and cluster architecture—explaining how messages flow, are stored, and replicated across partitions and nodes, while highlighting the role of ZooKeeper in managing metadata.

Distributed SystemsKafkaMessaging

0 likes · 5 min read

Java High-Performance Architecture

May 7, 2021 · Big Data

Mastering Kafka Partitions: Boost Scalability, Fault Tolerance, and Ordering

Kafka partitions are the fundamental storage units that enable topics to scale horizontally, maintain message order within each partition, provide fault‑tolerance through replication, and support parallel consumption, with various write strategies such as key‑based, round‑robin, and custom rules.

KafkaMessage OrderingPartitions

0 likes · 9 min read

Mastering Kafka Partitions: Boost Scalability, Fault Tolerance, and Ordering

IT Architects Alliance

Apr 28, 2021 · Industry Insights

Why Kafka Beats Redis List: A Deep Dive into Modern Message Queue Architecture

This article compares the fundamentals and limitations of Redis list‑based queues with Kafka and Pulsar, explaining how partitions, cursors, replication, segment storage, and consumption models overcome Redis’s drawbacks and enable high‑performance, scalable, and highly available messaging systems.

Distributed SystemsKafkaMessage Queue

0 likes · 30 min read

Why Kafka Beats Redis List: A Deep Dive into Modern Message Queue Architecture

Java High-Performance Architecture

Apr 26, 2021 · Fundamentals

Visualizing Kafka: Core Concepts Explained with Diagrams

This article provides a visual walkthrough of Kafka's fundamental concepts—including producers, consumers, topics, partitions, and cluster architecture—illustrated with diagrams to help readers clearly understand how messages flow and are stored in a distributed streaming system.

Distributed SystemsKafkaPartitions

0 likes · 6 min read

Programmer DD

Jan 5, 2021 · Backend Development

Understanding Kafka Partition Assignment: Strategies and Code Walkthrough

This article explains how Kafka determines which partition a producer sends a record to, how partition counts are configured, and how consumer groups assign partitions using the default, range, and round‑robin strategies, complemented by detailed Java code examples.

JavaKafkaPartition Assignment

0 likes · 21 min read

Understanding Kafka Partition Assignment: Strategies and Code Walkthrough

Big Data Technology & Architecture

Sep 17, 2020 · Big Data

Monitoring Kafka Consumer Groups with kafka-consumer-groups and Kafka Manager

This article explains how to monitor Kafka consumer groups using the built‑in kafka‑consumer‑groups tool and the Kafka Manager UI, providing commands, field explanations, and setup steps to ensure real‑time data availability for downstream services such as MongoDB or Elasticsearch.

Big DataKafkaKafka Manager

0 likes · 4 min read

Monitoring Kafka Consumer Groups with kafka-consumer-groups and Kafka Manager

Java High-Performance Architecture

Nov 12, 2019 · Backend Development

How Kafka Consumer Groups Boost Performance and Fault Tolerance

Kafka consumer groups enable multiple consumers to share partition workloads, ensuring exclusive consumption within a group, flexible consumption patterns like broadcast and unicast, and automatic fault‑tolerance through rebalancing, ultimately improving throughput, scalability, and resilience of streaming applications.

Backend DevelopmentKafkaconsumer groups

0 likes · 4 min read

How Kafka Consumer Groups Boost Performance and Fault Tolerance

Big Data Technology Architecture

Aug 26, 2019 · Big Data

Kafka Architecture and File Storage Mechanism: Design, Performance, and Operational Practices

This article provides a comprehensive overview of Kafka, covering its core features, use‑case scenarios, partition and replica design, file storage structure, consumer‑group coordination, delivery guarantees, performance optimizations, and the role of Zookeeper in managing the cluster.

Distributed MessagingKafkaReplication

0 likes · 54 min read

Kafka Architecture and File Storage Mechanism: Design, Performance, and Operational Practices

Architecture Digest

Aug 14, 2019 · Big Data

Kafka Overview: Architecture, Storage Mechanism, Replication, and Consumer/Producer Model

Kafka is a distributed, partitioned, replicated messaging system originally developed by LinkedIn, offering high throughput, low latency, fault tolerance, and scalability; this article explains its core concepts, file storage design, partition replication, leader election, consumer groups, delivery guarantees, and operational considerations for big‑data pipelines.

Big DataDistributed SystemsKafka

0 likes · 56 min read

Kafka Overview: Architecture, Storage Mechanism, Replication, and Consumer/Producer Model

Architecture Digest

May 5, 2019 · Big Data

Kafka Architecture Overview: Topics, Partitions, Producers, Consumers, Replication, Leader Election, Offsets, Rebalance, Delivery Semantics, and Transactions

This article provides a comprehensive overview of Kafka's architecture, covering topics, partitions, producer and consumer workflows, replication and leader election, offset management, consumer group coordination, rebalance processes, delivery semantics (at‑most‑once, at‑least‑once, exactly‑once), transactional messaging, and underlying file and configuration details.

Distributed MessagingExactly-OnceKafka

0 likes · 16 min read

Kafka Architecture Overview: Topics, Partitions, Producers, Consumers, Replication, Leader Election, Offsets, Rebalance, Delivery Semantics, and Transactions

System Architect Go

Apr 11, 2019 · Big Data

Introduction to Apache Kafka: Core Concepts, Message Delivery, Partition Storage, and Consumption

This article introduces Apache Kafka as a distributed streaming platform, explaining its three core capabilities, key concepts such as producers, topics, brokers, partitions and consumers, and detailing how messages are delivered, stored in partitions, and consumed by consumer groups.

Big DataDistributed StreamingKafka

0 likes · 8 min read

Introduction to Apache Kafka: Core Concepts, Message Delivery, Partition Storage, and Consumption

Java High-Performance Architecture

Jul 21, 2018 · Databases

Master Redis Streams: From Simple Adds to Powerful Consumer Groups

This guide walks through Redis 5's stream feature, showing how to set up a Docker Redis instance, add entries, perform range queries, use blocking reads, create and manage consumer groups, handle pending messages, claim ownership, inspect stream metadata, delete entries, and limit stream length for efficient message queue implementations.

BackendMessage QueueStreams

0 likes · 10 min read

Master Redis Streams: From Simple Adds to Powerful Consumer Groups