BestHub
Sign in
Tagged articles

kafka

1310 articles · Page 1 of 14
Java Architect Handbook
Java Architect Handbook
Jul 4, 2026 · Backend Development

How to Build a Clean SpringBoot Scaffold Quickly

The article walks through creating a SpringBoot project from scratch, covering IDE setup pain points, project initialization, version compatibility between SpringBoot, Spring Cloud and Kafka, Maven dependency management, core scaffold classes such as global exception handling, logging aspect, CORS and Swagger configuration, and recommends useful tools like embedded Redis, MariaDB, Hutool, MyBatis‑Plus, MapStruct and Redisson.

embedded-rediskafkamapstruct
0 likes · 11 min read
How to Build a Clean SpringBoot Scaffold Quickly
Alibaba Cloud Native
Alibaba Cloud Native
Jun 26, 2026 · Cloud Native

One-Click Real-Time Stream Ingestion: Alibaba Cloud Kafka’s Native Data Lake Integration

Alibaba Cloud Message Queue for Kafka introduces a native message‑to‑lake capability that integrates Apache Iceberg with OSS Table Bucket, eliminating Spark/Flink/Kafka Connect, providing exactly‑once semantics, automatic schema management, dual write modes, smart partitioning, and up to ten‑fold performance gains across diverse real‑time analytics scenarios.

Apache IcebergCloud NativeData Lake
0 likes · 12 min read
One-Click Real-Time Stream Ingestion: Alibaba Cloud Kafka’s Native Data Lake Integration
Programmer XiaoFu
Programmer XiaoFu
Jun 23, 2026 · Backend Development

Why Kafka Still Delivers Out‑of‑Order Messages Even When Using the Same Key

Even though Kafka guarantees that messages with the same key land in the same partition, the article explains how producer retries, multithreaded consumer processing, and partition expansion can break ordering, and provides concrete techniques such as idempotent producers and single‑threaded consumption to preserve order.

Consumer ConcurrencyMessage OrderingPartition Expansion
0 likes · 10 min read
Why Kafka Still Delivers Out‑of‑Order Messages Even When Using the Same Key
ZhiKe AI
ZhiKe AI
Jun 22, 2026 · Fundamentals

Message Queues: Power When Correct, Disaster When Wrong – 3 Scenarios & Tips

The article explains how message queues can dramatically improve response time, decouple services, and smooth traffic spikes, outlines seven advantages and eight drawbacks, and provides concrete guidelines on when to adopt them, how to prevent loss, duplication, and ordering issues, and how to ensure end‑to‑end reliability.

Message QueueRabbitMQReliability
0 likes · 15 min read
Message Queues: Power When Correct, Disaster When Wrong – 3 Scenarios & Tips
Architect Chen
Architect Chen
Jun 19, 2026 · Operations

Comprehensive Guide to Kafka Commands (2026 Edition)

This article provides a step‑by‑step reference for Kafka command‑line tools, covering version checks, topic creation, listing, describing, deletion, partition alteration, consumer‑group inspection, offset resets, producer/consumer testing, broker API version queries, cluster metadata, leader election, and log‑directory monitoring, with concrete examples and expected outputs.

broker monitoringcommand-lineconsumer groups
0 likes · 6 min read
Comprehensive Guide to Kafka Commands (2026 Edition)
Alibaba Cloud Native
Alibaba Cloud Native
Jun 19, 2026 · Big Data

Why Real-Time Data Lake Ingestion Is Dropping ETL in the AI Era: Architecture Simplification from Kafka to Iceberg

In the AI‑driven era, enterprises need a data foundation that supports both real‑time consumption and long‑term historical analysis, and the emerging "zero‑ETL" trend moves generic ingestion capabilities from external Flink/Spark jobs into a streamlined Kafka‑to‑Iceberg pipeline, reducing complexity while preserving low latency, consistency, schema evolution, CDC semantics and open‑ecosystem compatibility.

Data LakeIcebergStreaming
0 likes · 25 min read
Why Real-Time Data Lake Ingestion Is Dropping ETL in the AI Era: Architecture Simplification from Kafka to Iceberg
Alibaba Cloud Developer
Alibaba Cloud Developer
Jun 18, 2026 · Big Data

How AI-Driven Real-Time Data Lakes Are Ditching ETL: A Kafka‑to‑Iceberg Architecture Simplification

In the AI era, enterprises need a data foundation that supports both low‑latency streaming and long‑term analytics, and the combination of Kafka, Iceberg and object storage is emerging as a preferred solution; by moving ingestion capabilities closer to the message layer and eliminating external ETL jobs, a "zero‑ETL" approach reduces architectural complexity, improves consistency, and streamlines schema evolution and small‑file management.

CDCData LakeIceberg
0 likes · 27 min read
How AI-Driven Real-Time Data Lakes Are Ditching ETL: A Kafka‑to‑Iceberg Architecture Simplification
Linyb Geek Road
Linyb Geek Road
Jun 14, 2026 · Big Data

How to Solve Data Ordering Issues in Apache Kafka

This article explains how Kafka maintains order within partitions using keys and offsets, why ordering across partitions can break, and how to preserve strict sequencing through producer configuration, idempotent producers, and exactly‑once transactional processing.

Exactly-oncedata orderingidempotent producer
0 likes · 9 min read
How to Solve Data Ordering Issues in Apache Kafka
Architect Chen
Architect Chen
Jun 9, 2026 · Big Data

How Kafka Prevents Duplicate Consumption: Three Main Solutions

The article explains why Kafka does not guarantee exactly‑once delivery and presents three practical approaches—business‑level idempotence, manual offset management, and Kafka’s transaction/EOS features—to reliably avoid duplicate message processing.

Exactly-onceIdempotenceOffset Management
0 likes · 4 min read
How Kafka Prevents Duplicate Consumption: Three Main Solutions
Java Tech Enthusiast
Java Tech Enthusiast
Jun 6, 2026 · Backend Development

Why Kafka Marks a Live Consumer as Dead and Forces Rebalance

Even when a consumer process runs and logs normally, Kafka may deem it dead and trigger a rebalance because the poll interval exceeds max.poll.interval.ms, a situation known as ‘false dead’; this article explains the root cause and practical ways to prevent it.

Rebalanceconsumerheartbeat
0 likes · 8 min read
Why Kafka Marks a Live Consumer as Dead and Forces Rebalance
Coder Trainee
Coder Trainee
Jun 3, 2026 · Cloud Native

Unified Messaging with Spring Cloud Stream: One Codebase for Multiple MQs

This article demonstrates how Spring Cloud Stream provides a unified programming model that decouples business logic from the underlying message broker, allowing a single codebase to work with RocketMQ, Kafka, or RabbitMQ by only changing configuration, and walks through project structure, implementation, conditional routing, MQ switching, testing, and common pitfalls.

JavaMessage-drivenMicroservices
0 likes · 19 min read
Unified Messaging with Spring Cloud Stream: One Codebase for Multiple MQs
StarRocks
StarRocks
May 28, 2026 · Industry Insights

How Fresha Built a Modern Real‑Time Analytics Stack with AutoMQ and StarRocks

Fresha replaced its Postgres‑Snowflake‑MSK pipeline with an AutoMQ‑based Diskless Kafka message layer and StarRocks for real‑time analytics, cutting storage costs 17‑20×, dropping query latency from seconds to sub‑second, and migrating ~1,000 topics in a week with zero downtime.

AutoMQCloud MigrationPostgres
0 likes · 24 min read
How Fresha Built a Modern Real‑Time Analytics Stack with AutoMQ and StarRocks
LuTiao Programming
LuTiao Programming
May 21, 2026 · Backend Development

Stop Fighting Microservice Calls—Why Experts Prefer Event‑Driven Architecture for Decoupling Distributed Systems

The article explains how traditional synchronous microservice calls create tight coupling, cascading failures, scaling bottlenecks, and high latency, and demonstrates that adopting an event‑driven architecture with producers, consumers, and a message broker such as Kafka can fully decouple services, improve scalability, and enable patterns like event sourcing and CQRS.

CQRSEvent SourcingEvent-Driven Architecture
0 likes · 14 min read
Stop Fighting Microservice Calls—Why Experts Prefer Event‑Driven Architecture for Decoupling Distributed Systems
Su San Talks Tech
Su San Talks Tech
May 19, 2026 · Interview Experience

Designing a Hundred‑Billion‑Scale Message Queue: A ByteDance Interview Walkthrough

This article walks through the interview question of designing a message queue that handles billions of messages daily and peaks at millions of QPS, covering traffic calculations, core roles, storage and throughput techniques, scalability, high availability, observability, framework comparisons, a real‑world case study, and key follow‑up interview topics.

Message QueuePulsarRocketMQ
0 likes · 12 min read
Designing a Hundred‑Billion‑Scale Message Queue: A ByteDance Interview Walkthrough
Lobster Programming
Lobster Programming
May 6, 2026 · Backend Development

How to Choose the Right MQ: RabbitMQ vs RocketMQ vs Kafka

This article compares RabbitMQ, RocketMQ, and Kafka on throughput, latency, scalability, and reliability, outlining each system's core features and recommending suitable scenarios such as reliable messaging, high‑performance streaming, and large‑scale real‑time data processing.

LatencyMessage QueueRabbitMQ
0 likes · 6 min read
How to Choose the Right MQ: RabbitMQ vs RocketMQ vs Kafka
Woodpecker Software Testing
Woodpecker Software Testing
Apr 30, 2026 · Databases

Datafaker: A Powerful Tool for Bulk Test Data Generation

Datafaker is a Python‑compatible utility that creates large volumes of synthetic test data for databases, streams, files, and messaging systems, offering flexible metadata rules, multi‑backend support, and command‑line options for quick data provisioning.

ElasticsearchMetadataPython
0 likes · 14 min read
Datafaker: A Powerful Tool for Bulk Test Data Generation
Java Tech Workshop
Java Tech Workshop
Apr 29, 2026 · Backend Development

How to Diagnose and Scale SpringBoot Message Backlog with Monitoring

The article explains why message backlog occurs in SpringBoot applications, outlines systematic troubleshooting steps, proposes comprehensive monitoring across producer, broker, and consumer layers, and presents scaling tactics such as instance expansion, concurrency tuning, batch consumption, and long‑term capacity planning.

BacklogMessage QueueMonitoring
0 likes · 16 min read
How to Diagnose and Scale SpringBoot Message Backlog with Monitoring
Java Tech Workshop
Java Tech Workshop
Apr 29, 2026 · Backend Development

Mastering SpringBoot Transactional Messaging for Distributed Consistency

This article explains how SpringBoot handles transactional messaging to achieve distributed data consistency, covering the concept of transaction messages, CAP theory, final consistency, and three practical implementations using a local message table, Kafka transactions, and RocketMQ, plus idempotency and compensation strategies.

CAP theoremRocketMQTransactional Messaging
0 likes · 17 min read
Mastering SpringBoot Transactional Messaging for Distributed Consistency
Java Tech Workshop
Java Tech Workshop
Apr 28, 2026 · Backend Development

Ensuring Message Order in SpringBoot: Partitioning and Sequential Consumption

This article examines why message ordering is critical in distributed systems, explains how partition mechanisms in Kafka, RocketMQ, and RabbitMQ enable ordered consumption, and provides detailed SpringBoot implementations, best‑practice guidelines, partition‑key design principles, concurrency settings, idempotency, and real‑world case studies to ensure reliable sequential processing.

Backend DevelopmentMessage OrderingRabbitMQ
0 likes · 28 min read
Ensuring Message Order in SpringBoot: Partitioning and Sequential Consumption
Golang Shines
Golang Shines
Apr 28, 2026 · Backend Development

Essential Go Packages for Production Environments

This article compiles a curated list of production‑ready Go packages covering testing, logging, error handling, caching, databases, HTTP routing, HTTP clients, fault tolerance, Kafka, and various utility libraries, explaining their key features, concrete code examples, and why they are preferred in real‑world services.

CachingGoHTTP
0 likes · 15 min read
Essential Go Packages for Production Environments
LuTiao Programming
LuTiao Programming
Apr 28, 2026 · Backend Development

How I Built a High‑Performance Java Price‑Comparison Engine from Scratch

Starting from a simple sequential Java price‑aggregator, the article walks through successive architectural upgrades—concurrent calls with CompletableFuture, timeout and fallback handling, Spring Boot service exposure, caching, bulkhead isolation, microservice split, and Kafka‑driven event processing—showing how latency drops from 1500 ms to under 20 ms.

CachingJavaMicroservices
0 likes · 9 min read
How I Built a High‑Performance Java Price‑Comparison Engine from Scratch
Java Tech Workshop
Java Tech Workshop
Apr 27, 2026 · Backend Development

How to Integrate Kafka with SpringBoot for High‑Performance Messaging

This article walks through Kafka’s core architecture, explains why it achieves massive throughput, and provides a step‑by‑step SpringBoot integration—including environment setup, Maven dependencies, configuration, producer and consumer code, advanced features like transactions and dead‑letter queues, plus performance monitoring and tuning tips.

Dead‑Letter QueueJavaMessage Queue
0 likes · 11 min read
How to Integrate Kafka with SpringBoot for High‑Performance Messaging
DevOps Coach
DevOps Coach
Apr 26, 2026 · Backend Development

Forget Kafka: A Lightweight Go Queue Achieves 2 Million Messages per Second

The article analyzes how replacing Kafka with a simple in‑memory Go queue reduced architectural complexity, boosted throughput from 240‑330 K to 1.8‑2.0 M messages per second, and clarified debugging, while still acknowledging scenarios where Kafka remains the better choice.

Backend PerformanceGoIn‑Memory Ring Buffer
0 likes · 8 min read
Forget Kafka: A Lightweight Go Queue Achieves 2 Million Messages per Second
LuTiao Programming
LuTiao Programming
Apr 22, 2026 · Backend Development

Kafka Deep Dive: Core Concepts Every Architect Must Master to Prevent Outages

The article explains why merely “knowing how to use” Kafka is insufficient, detailing how offset commits, consumer acknowledgments, producer acks, and rebalance behavior affect reliability, and provides concrete code examples, risk scenarios, and configuration recommendations to prevent message loss and duplicate processing in production systems.

Consumer offsetIdempotenceMessage reliability
0 likes · 7 min read
Kafka Deep Dive: Core Concepts Every Architect Must Master to Prevent Outages
ITPUB
ITPUB
Apr 17, 2026 · Industry Insights

Why LinkedIn Dumped Kafka for Its Own ‘Northguard’ Streaming Engine

LinkedIn, the original home of Apache Kafka, abandoned the platform for a home‑grown system called Northguard, redesigning log storage, decentralizing metadata, and adding a virtualized Xinfra layer to handle trillions of daily events, while still acknowledging Kafka’s relevance for most companies.

LinkedInNorthguardStreaming
0 likes · 7 min read
Why LinkedIn Dumped Kafka for Its Own ‘Northguard’ Streaming Engine
Architect Chen
Architect Chen
Apr 16, 2026 · Big Data

Supercharge Kafka Consumer Performance: Parallelism, Batching, and Multithreading

This guide explains practical techniques to dramatically increase Kafka consumer throughput, including scaling consumer instances or partitions, tuning fetch and poll parameters, and implementing a multithreaded consumer model, while also covering hardware, JVM, and OS optimizations and monitoring recommendations.

Batch FetchConsumer ParallelismMonitoring
0 likes · 5 min read
Supercharge Kafka Consumer Performance: Parallelism, Batching, and Multithreading
LuTiao Programming
LuTiao Programming
Apr 8, 2026 · Backend Development

From Chaos to Production: Building a Real Food-Delivery Backend with Spring Boot

The article chronicles the step‑by‑step evolution of a small team’s chaotic food‑delivery backend into a production‑ready system, detailing how they introduced layering, transactions, security, caching, async processing, messaging, observability, scalability, resilience, containerization, and testing using Spring Boot, Kafka, Redis, JWT, and Docker.

DockerMicroservicesRedis
0 likes · 10 min read
From Chaos to Production: Building a Real Food-Delivery Backend with Spring Boot
Ray's Galactic Tech
Ray's Galactic Tech
Apr 4, 2026 · Backend Development

How to Build a High‑Concurrency Story Creation Platform with AgentScope Java

This article presents a step‑by‑step engineering guide for constructing a production‑grade, high‑throughput story generation platform using AgentScope Java, Spring Boot, Kafka, Redis, PostgreSQL, and Kubernetes, covering architecture, task modeling, DAG orchestration, code organization, scalability, observability, and deployment best practices.

High concurrencyJavaSpring Boot
0 likes · 39 min read
How to Build a High‑Concurrency Story Creation Platform with AgentScope Java
Alibaba Cloud Native
Alibaba Cloud Native
Mar 30, 2026 · Industry Insights

How Haier’s AIoT Platform Scaled to Billions of Messages with Kafka Serverless on Alibaba Cloud

The article details how Haier Smart Home’s AIoT platform tackled massive device messaging demands by migrating its self‑built Kafka clusters to Alibaba Cloud’s Kafka Serverless, outlining the technical challenges, step‑by‑step migration plan, custom performance tuning, risk‑co‑governance, and the resulting improvements in stability, throughput, and operational efficiency.

AIoTAlibaba CloudCloud Migration
0 likes · 11 min read
How Haier’s AIoT Platform Scaled to Billions of Messages with Kafka Serverless on Alibaba Cloud
360 Zhihui Cloud Developer
360 Zhihui Cloud Developer
Mar 27, 2026 · Cloud Native

How AutoMQ Transforms Kafka into a Cloud‑Native, Elastic Messaging Service

This article examines the limitations of traditional Kafka in large‑scale deployments and presents AutoMQ’s cloud‑native redesign—detailing its stateless architecture, storage separation, automatic scaling, read/write isolation, performance benchmarks, and real‑world migration case studies that demonstrate reduced latency, higher throughput, and lower resource costs.

AutoMQCloud NativeElastic Scaling
0 likes · 13 min read
How AutoMQ Transforms Kafka into a Cloud‑Native, Elastic Messaging Service
Alibaba Cloud Native
Alibaba Cloud Native
Mar 20, 2026 · Cloud Native

How a Gaming Platform Scaled to Millions with RocketMQ & Kafka: A Cloud‑Native Success Story

Facing explosive growth, the game‑service platform 悠悠有品 rebuilt its architecture on Alibaba Cloud, using RocketMQ for core transaction messaging and Kafka for data synchronization, achieving elastic scaling, high availability, cost reduction, and reliable high‑concurrency processing across its trading and analytics pipelines.

RocketMQcloud-nativecost-optimization
0 likes · 8 min read
How a Gaming Platform Scaled to Millions with RocketMQ & Kafka: A Cloud‑Native Success Story
Cognitive Technology Team
Cognitive Technology Team
Mar 9, 2026 · Operations

Mastering Kafka ISR: How In‑Sync Replicas Ensure Consistency and High Availability

This article explains Kafka's In‑Sync Replicas (ISR) mechanism, detailing its definitions, dynamic scaling, interaction with High Watermark, extreme unclean leader election scenarios, and practical tuning and troubleshooting tips for maintaining strong consistency and high availability in production clusters.

High AvailabilityISRPerformance Tuning
0 likes · 15 min read
Mastering Kafka ISR: How In‑Sync Replicas Ensure Consistency and High Availability
dbaplus Community
dbaplus Community
Mar 5, 2026 · Backend Development

How to Ensure Message Order in Kafka: From Basics to Advanced Solutions

This article explains the concept of message ordering in distributed systems, details how Kafka stores messages in partitions, compares global and partial ordering, evaluates single‑partition, asynchronous, and multi‑partition solutions—including handling data skew and partition expansion—and provides a practical interview guide.

Message Orderingbackenddistributed systems
0 likes · 22 min read
How to Ensure Message Order in Kafka: From Basics to Advanced Solutions
Architect Chen
Architect Chen
Mar 3, 2026 · Backend Development

Preventing Kafka Duplicate Consumption with Idempotent Design

This article explains practical strategies to avoid duplicate message consumption in Kafka, covering business idempotency with unique IDs, database or Redis deduplication tables, enabling producer idempotence, consumer-side checks, and Kafka's transaction-based exactly‑once semantics, along with their trade‑offs and suitable scenarios.

Exactly-onceMessage Deduplicationbackend
0 likes · 4 min read
Preventing Kafka Duplicate Consumption with Idempotent Design
Golang Shines
Golang Shines
Feb 25, 2026 · Big Data

Essential Kafka Core Concepts – A Comprehensive Cheat Sheet (PDF Included)

This article outlines Kafka's role as a leading MQ middleware, highlighting its key traits of decoupling, asynchronous processing, and throttling, compares it with traditional messaging systems, and provides a structured list of basic, intermediate, and advanced topics plus interview questions for in‑depth study.

Data IntegrationDistributed StreamingMessage Queue
0 likes · 5 min read
Essential Kafka Core Concepts – A Comprehensive Cheat Sheet (PDF Included)
LuTiao Programming
LuTiao Programming
Feb 21, 2026 · Backend Development

Stop Building Chaotic Payment Systems: A Complete Design Methodology for Payment Domains

The article presents a production‑grade, modular design methodology for payment domain systems, detailing five core modules, essential principles such as idempotency and ACID, concrete implementation examples with Kafka, Redis, PostgreSQL, and guidance on risk, compliance, and high‑availability engineering.

PCI-DSSPostgreSQLRisk Management
0 likes · 8 min read
Stop Building Chaotic Payment Systems: A Complete Design Methodology for Payment Domains
ITPUB
ITPUB
Feb 11, 2026 · Backend Development

How to Guarantee Zero Message Loss in MQ Systems: A Full‑Lifecycle Design

This guide explains why guaranteeing 100% message reliability in MQ is a critical system‑design interview topic and presents a three‑layer architecture—production, storage, and consumption—detailing ACK settings, local message tables, broker replication, leader election safeguards, manual offset commits, and idempotent processing to prevent any message loss.

AcknowledgmentMQMessage reliability
0 likes · 11 min read
How to Guarantee Zero Message Loss in MQ Systems: A Full‑Lifecycle Design
ITPUB
ITPUB
Feb 9, 2026 · Databases

ClickHouse vs Doris vs Redis: Real‑World Query Performance Test with Flink

Using a 600k‑record IP range dataset, we built identical tables in ClickHouse and Doris, and a Redis skip‑list store, then ran three Flink‑Kafka streaming jobs to compare query latency across the three databases under varying traffic rates, revealing Redis as fastest, ClickHouse second, Doris slowest.

ClickHouseDatabase PerformanceDoris
0 likes · 8 min read
ClickHouse vs Doris vs Redis: Real‑World Query Performance Test with Flink
Architect Chen
Architect Chen
Feb 6, 2026 · Backend Development

How Kafka Achieves Billion‑Message Throughput with Sequential I/O, Zero‑Copy, and Partitioning

This article explains how Kafka leverages sequential disk I/O, zero‑copy data transfer, OS page cache, and partition‑based parallelism to deliver massive throughput, detailing the underlying mechanisms, performance numbers, and a practical formula for estimating total system capacity.

Backend PerformanceHigh concurrencySequential I/O
0 likes · 5 min read
How Kafka Achieves Billion‑Message Throughput with Sequential I/O, Zero‑Copy, and Partitioning
Top Architect
Top Architect
Feb 4, 2026 · Backend Development

Build a Robust Asynchronous Processing SDK with Spring, Kafka and MySQL

This article introduces a generic asynchronous processing SDK for Java back‑ends, explaining its design principles, advantages, component architecture, database schema, configuration via Apollo, usage steps, and practical demonstrations, while providing complete code snippets and a GitHub repository for reference.

AsynchronousJavaSpring
0 likes · 12 min read
Build a Robust Asynchronous Processing SDK with Spring, Kafka and MySQL
Tencent Cloud Developer
Tencent Cloud Developer
Feb 4, 2026 · Backend Development

How We Cut Server Costs by 82%: Refactoring a High‑Concurrency QQ Game Service from C++ to Go with Kafka

This article details the redesign of a core QQ game achievement service that suffered from low resource utilization and heavy CAS contention, describing how moving from a synchronous C++ implementation to an asynchronous Go‑Kafka pipeline eliminated lock conflicts, reduced server count by 82%, and dramatically improved latency and stability.

High concurrencykafkarefactoring
0 likes · 11 min read
How We Cut Server Costs by 82%: Refactoring a High‑Concurrency QQ Game Service from C++ to Go with Kafka
Java Tech Enthusiast
Java Tech Enthusiast
Feb 3, 2026 · Backend Development

Spring Boot 4.0.2: Critical Kafka Fixes, Dependency Upgrades & What to Watch

Spring Boot 4.0.2 is a maintenance release that focuses on fixing over 20 bugs—including a critical Kafka transaction auto‑configuration issue—upgrading more than 40 core dependencies, improving documentation, and introducing minor breaking changes, making it essential for Java developers to upgrade promptly.

Bug Fixdependency-upgradekafka
0 likes · 8 min read
Spring Boot 4.0.2: Critical Kafka Fixes, Dependency Upgrades & What to Watch
LuTiao Programming
LuTiao Programming
Jan 30, 2026 · Backend Development

Spring Boot 4.0.2 Released: Over 20 Critical Bugs Fixed and Kafka Transaction Issue Resolved

Spring Boot 4.0.2, launched on January 22, brings more than 20 critical bug fixes, restores Kafka transaction auto‑configuration, improves startup performance by up to 15%, stabilizes native‑image support, refines test behavior, upgrades core dependencies, and introduces a few breaking changes, making it a must‑upgrade for production users.

Native Imagedependency-upgradekafka
0 likes · 8 min read
Spring Boot 4.0.2 Released: Over 20 Critical Bugs Fixed and Kafka Transaction Issue Resolved
Ray's Galactic Tech
Ray's Galactic Tech
Jan 30, 2026 · Cloud Native

Scale a Monolithic Article Interaction Service with Kubernetes Microservices

This article walks through converting a single‑service article interaction module—handling likes, favorites, and reads—into independent microservices deployed on Kubernetes, detailing architecture goals, service separation, Redis‑based high‑concurrency handling, Kafka async persistence, deployment configurations, auto‑scaling, and real‑world performance results.

Rediskafka
0 likes · 8 min read
Scale a Monolithic Article Interaction Service with Kubernetes Microservices
Su San Talks Tech
Su San Talks Tech
Jan 30, 2026 · Backend Development

Why Kafka Rebalance Causes Backlog, Duplicates, and Data Loss—and How to Fix It

Kafka consumer group rebalances can trigger message backlogs, duplicate processing, and data loss; this article explains common rebalance triggers, their impact on consumption, and practical configuration and coding strategies—such as tuning timeout parameters, using manual offset commits, and sticky partition assignment—to minimize disruptions.

Consumer GroupData lossMessage Duplication
0 likes · 12 min read
Why Kafka Rebalance Causes Backlog, Duplicates, and Data Loss—and How to Fix It
LuTiao Programming
LuTiao Programming
Jan 27, 2026 · Big Data

Why LinkedIn Is Replacing Kafka with Its Own Next‑Gen Streaming System

LinkedIn, facing planetary‑scale data volumes, found Kafka’s architecture hitting fundamental limits and built Northguard—a decentralized, log‑striped streaming platform with Raft‑based metadata and an Xinfra migration layer—to gradually replace Kafka’s core responsibilities while maintaining compatibility.

Data ArchitectureLinkedInNorthguard
0 likes · 8 min read
Why LinkedIn Is Replacing Kafka with Its Own Next‑Gen Streaming System
DevOps Coach
DevOps Coach
Jan 27, 2026 · Backend Development

7 Essential Kafka Design Patterns Every Engineer Should Master

This guide presents seven practical Kafka design patterns—single‑key single‑write, log compaction, multi‑consumer‑group fan‑out, retry and dead‑letter topics, exactly‑once processing with Streams, schema evolution with Avro, and choreography vs orchestration—detailing when to use each, core principles, code examples, tips, common pitfalls, and final recommendations for building reliable, observable, and maintainable event‑driven systems.

Reliabilitydesign patternsevent streaming
0 likes · 9 min read
7 Essential Kafka Design Patterns Every Engineer Should Master
Ray's Galactic Tech
Ray's Galactic Tech
Jan 23, 2026 · Backend Development

How to Build a Kafka‑Level High‑Performance Message Queue from Scratch

This article presents a step‑by‑step guide to designing and implementing a Kafka‑class distributed log‑based message queue kernel, covering architecture, sequential writes, sparse indexing, zero‑copy I/O, partitioning, replication, consumer‑group metadata, batch pipelines, crash recovery, and performance benchmarks.

IndexingMessage QueueZero‑copy
0 likes · 7 min read
How to Build a Kafka‑Level High‑Performance Message Queue from Scratch
Architect's Guide
Architect's Guide
Jan 22, 2026 · Big Data

Unlock Kafka’s Power: Core Concepts, High‑Performance Architecture & Real‑World Scaling Tips

This comprehensive guide explores Kafka’s core value as a message queue, explains producers, consumers, topics, partitions, and replication, dives into cluster architecture, zero‑copy I/O, resource planning for disks, memory, CPU and network, and provides practical configuration, consumer‑group management, and operational tooling tips for building high‑throughput, highly available Kafka deployments.

Message QueuePerformance Tuningcluster scaling
0 likes · 31 min read
Unlock Kafka’s Power: Core Concepts, High‑Performance Architecture & Real‑World Scaling Tips
ITPUB
ITPUB
Jan 21, 2026 · Interview Experience

How to Design a Billion‑User Real‑Time Step Leaderboard for Interviews

This article breaks down the interview‑level system design of a WeChat‑style step leaderboard that must support over a billion users, handling massive write spikes, low‑latency friend ranking queries, storage scaling, and relationship complexity with a three‑part architecture using MQ, Redis, and MySQL.

High concurrencyLeaderboardRedis
0 likes · 8 min read
How to Design a Billion‑User Real‑Time Step Leaderboard for Interviews
Tech Freedom Circle
Tech Freedom Circle
Jan 15, 2026 · Backend Development

Kafka Rebalance Storm Crushed 120k QPS in JD Interview – How to Understand and Fix

In a JD senior Java architect interview, a Kafka consumer‑group rebalance storm caused QPS to drop from 120k to zero, triggering massive message loss and latency spikes, and the article walks through the rebalance fundamentals, failure causes, impact analysis, cooperative sticky assignor migration, and comprehensive monitoring and mitigation strategies.

Consumer GroupMonitoringRebalance
0 likes · 28 min read
Kafka Rebalance Storm Crushed 120k QPS in JD Interview – How to Understand and Fix
Code Wrench
Code Wrench
Jan 14, 2026 · Backend Development

When to Choose NATS Over Kafka for Go Microservices: A Practical Guide

This article compares Kafka, RabbitMQ, and NATS for Go microservices, explains why Kafka is often over‑engineered for internal communication, and shows how NATS provides a lightweight, event‑driven alternative with concrete code examples and a clear selection matrix.

GoMessage QueueMicroservices
0 likes · 9 min read
When to Choose NATS Over Kafka for Go Microservices: A Practical Guide
JD Retail Technology
JD Retail Technology
Jan 13, 2026 · Backend Development

Deep Dive into Kafka, RocketMQ, and JMQ Storage Architectures

This article compares the storage models, data organization, indexing, read/write processes, and performance trade‑offs of three major message queues—Kafka, RocketMQ, and JMQ—providing detailed technical insights for architects and engineers making storage‑related design decisions.

Backend EngineeringJMQMessage Queue
0 likes · 16 min read
Deep Dive into Kafka, RocketMQ, and JMQ Storage Architectures
Top Architect
Top Architect
Jan 12, 2026 · Backend Development

How to Build a Robust Asynchronous Processing SDK with Spring, Kafka, and XXL‑Job

This article explains the design and implementation of a generic asynchronous processing SDK for Java, covering its purpose, advantages, core principles, component choices, design patterns, configuration via Apollo, usage steps, safety considerations, and provides complete SQL and Spring configuration examples along with a GitHub repository link.

AsynchronousJavaSpring
0 likes · 11 min read
How to Build a Robust Asynchronous Processing SDK with Spring, Kafka, and XXL‑Job
iQIYI Technical Product Team
iQIYI Technical Product Team
Jan 8, 2026 · Big Data

How iQIYI Cut Stream Data Costs by 70%: From Private‑Cloud Kafka to AutoMQ

This article details iQIYI's evolution from a tightly coupled private‑cloud Kafka setup to a cloud‑native AutoMQ architecture, describing the challenges of scaling, the development of the Stream platform and Stream‑SDK, the migration to hybrid and public‑cloud Kafka, and the resulting cost and elasticity improvements.

AutoMQCloud MigrationData Architecture
0 likes · 12 min read
How iQIYI Cut Stream Data Costs by 70%: From Private‑Cloud Kafka to AutoMQ
Ray's Galactic Tech
Ray's Galactic Tech
Jan 6, 2026 · Backend Development

Scalable GPS Data Backend: SpringBoot, Kafka, MongoDB & Redis Design

This guide outlines a complete backend architecture for high‑volume GPS data, detailing the overall system flow, technology stack choices, Maven dependencies, data models, Kafka producer/consumer configurations, SpringBoot controllers, asynchronous processing, Redis caching, health checks, Docker deployment, and performance tuning recommendations to ensure stability and scalability.

DockerMongoDBRedis
0 likes · 11 min read
Scalable GPS Data Backend: SpringBoot, Kafka, MongoDB & Redis Design
Tech Freedom Circle
Tech Freedom Circle
Jan 6, 2026 · Backend Development

Why Choose RocketMQ Over Kafka? The Real Reasons Behind the 90% Mistake

This article dissects a common interview question about Kafka's higher throughput versus RocketMQ's richer features, explains the underlying design philosophies, storage models, I/O paths, scaling limits, real‑world use cases such as transaction, delayed and ordered messages, and provides concrete optimization steps and code samples to help engineers make an informed messaging platform choice.

JavaMessage QueueRocketMQ
0 likes · 42 min read
Why Choose RocketMQ Over Kafka? The Real Reasons Behind the 90% Mistake
JakartaEE China Community
JakartaEE China Community
Jan 6, 2026 · Backend Development

Run a WildFly Application with JBang in Minutes

This tutorial shows how to use JBang scripts to launch a WildFly server, create a simple Jakarta REST endpoint, and extend the server with WildFly Glow add‑ons such as Kafka, providing step‑by‑step commands, code examples, and Docker integration for rapid prototyping.

JBangJava scriptingWildFly Glow
0 likes · 9 min read
Run a WildFly Application with JBang in Minutes
Architect Chen
Architect Chen
Jan 2, 2026 · Backend Development

Preventing Duplicate Consumption in Kafka: Design, Idempotence, and Configuration Strategies

This guide explains how to avoid duplicate message consumption in Kafka by designing unique identifiers, implementing consumer-side idempotence with deduplication tables, leveraging Kafka’s transactional features, and establishing system-level safeguards and monitoring to ensure reliable, exactly‑once processing.

Backend DevelopmentExactly-onceIdempotence
0 likes · 4 min read
Preventing Duplicate Consumption in Kafka: Design, Idempotence, and Configuration Strategies
dbaplus Community
dbaplus Community
Dec 30, 2025 · Backend Development

How to Tackle Massive Message Queue Backlogs in High‑Traffic Scenarios

During peak traffic like Double‑11, a message queue can accumulate millions of messages, and simply adding consumer instances only offers temporary relief; this article explains the partition model limits, how to calculate proper partition numbers, fast remediation tactics, and deep consumer‑side optimizations for robust, scalable processing.

BacklogMessage Queueconsumer optimization
0 likes · 20 min read
How to Tackle Massive Message Queue Backlogs in High‑Traffic Scenarios
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Dec 25, 2025 · Backend Development

How to Resolve Kafka Backlog Under High Load: Practical Tips

This article explains why Kafka experiences message backlog in high‑load environments, identifies producer‑consumer speed mismatches, I/O and resource bottlenecks, and offers concrete strategies such as scaling consumers, tuning hardware, and adjusting Kafka configurations to eliminate the backlog.

BacklogPerformance Tuningdistributed systems
0 likes · 4 min read
How to Resolve Kafka Backlog Under High Load: Practical Tips
dbaplus Community
dbaplus Community
Dec 23, 2025 · Backend Development

Is Redis a Viable Message Queue? List, Pub/Sub, and Stream Compared to Kafka & RabbitMQ

This article examines whether Redis can serve as a reliable message queue by comparing its List, Pub/Sub, and Stream features against professional solutions like Kafka and RabbitMQ, covering usage patterns, code examples, performance trade‑offs, persistence, and handling of message loss and backlog.

Backend DevelopmentMessage QueuePub-Sub
0 likes · 16 min read
Is Redis a Viable Message Queue? List, Pub/Sub, and Stream Compared to Kafka & RabbitMQ
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Dec 22, 2025 · Operations

How to Diagnose and Resolve Kafka Consumer Lag Quickly

When Kafka consumers fall behind, this guide walks you through confirming the backlog, pinpointing bottlenecks in production, consumption, or brokers, and applying concrete steps—such as checking offsets, comparing TPS, inspecting consumer logic, and adjusting partitions—to efficiently eliminate lag.

Consumer Lagkafka
0 likes · 4 min read
How to Diagnose and Resolve Kafka Consumer Lag Quickly
Architect Chen
Architect Chen
Dec 21, 2025 · Backend Development

How to Resolve Kafka Backlog: Boost Consumer Throughput and Optimize Partitions

This guide explains why Kafka backlog occurs when production outpaces consumption and provides practical steps—such as increasing consumer instances, optimizing processing, expanding partitions, applying flow‑control, and managing message capacity—to eliminate the backlog and keep the cluster healthy.

BacklogFlow ControlPartition Management
0 likes · 4 min read
How to Resolve Kafka Backlog: Boost Consumer Throughput and Optimize Partitions
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Dec 9, 2025 · Backend Development

Boost Kafka to Over 1 Million Messages per Second: Metrics and Tuning Tips

This article explains what high concurrency means for Kafka, outlines key performance metrics such as QPS, TPS, throughput and latency, and provides concrete configuration and architectural techniques—including broker optimization, horizontal scaling, network batching, and zero‑copy—to achieve write rates exceeding one million records per second.

High concurrencyPerformance Tuningbackend
0 likes · 4 min read
Boost Kafka to Over 1 Million Messages per Second: Metrics and Tuning Tips
IT Services Circle
IT Services Circle
Dec 9, 2025 · Backend Development

Mastering Kafka Rebalance: Prevent Backlog, Duplicates, and Data Loss

When Kafka consumer groups rebalance, partitions are reassigned, often causing message backlog, duplicate processing, or loss; understanding the triggers, impact, and optimization techniques—like tuning timeouts, managing offset commits, and using sticky assignors—can keep your streaming pipelines reliable.

Consumer GroupOffset ManagementPerformance Tuning
0 likes · 13 min read
Mastering Kafka Rebalance: Prevent Backlog, Duplicates, and Data Loss
dbaplus Community
dbaplus Community
Dec 8, 2025 · Databases

Which Database Wins IP Range Lookups? ClickHouse vs Doris vs Redis Benchmarks

This article presents a systematic benchmark comparing ClickHouse, Doris, and Redis for IP‑range dimension lookups using Flink‑Kafka pipelines, detailing test design, result table schema, query interfaces, and performance results across varying data rates, concluding that Redis offers the fastest and most stable query latency.

ClickHouseDatabase BenchmarkDoris
0 likes · 7 min read
Which Database Wins IP Range Lookups? ClickHouse vs Doris vs Redis Benchmarks
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Dec 3, 2025 · Big Data

Kafka High‑Throughput Tricks: Sequential Writes, Zero‑Copy, Partitioning

The article explains how Kafka achieves high throughput by writing messages sequentially to disk, leveraging OS page cache and zero‑copy system calls, using partitioned topics for parallelism, batching and compressing records on both producer and broker sides, and employing asynchronous replication with configurable persistence strategies.

BatchingZero‑copycompression
0 likes · 4 min read
Kafka High‑Throughput Tricks: Sequential Writes, Zero‑Copy, Partitioning
Code Wrench
Code Wrench
Nov 22, 2025 · Backend Development

Build a Production-Ready Rule Engine with Gray Release Using Go, Kafka, and Redis

Learn how to design and implement a ready-to-use rule engine combined with a gray release system using Golang, Kafka, Redis, and CEL, complete with Docker‑compose deployment, edge execution, token‑bucket throttling, and webhook actions, plus full source code for a production‑grade marketing strategy platform.

CELGoMicroservices
0 likes · 9 min read
Build a Production-Ready Rule Engine with Gray Release Using Go, Kafka, and Redis
dbaplus Community
dbaplus Community
Nov 18, 2025 · Backend Development

How to Guarantee 100% No Message Loss in Distributed MQ Systems

Ensuring that messages never disappear in a distributed MQ system requires a three‑pronged strategy covering production, storage, and consumption, with proper ACK configurations, local message tables, replication settings, and manual offset commits to achieve reliable, at‑least‑once processing without data loss.

MQReliabilitySystem Design
0 likes · 11 min read
How to Guarantee 100% No Message Loss in Distributed MQ Systems
MaGe Linux Operations
MaGe Linux Operations
Nov 18, 2025 · Big Data

Zero‑Data‑Loss Kafka Cluster Scaling: Complete Step‑by‑Step Guide

This comprehensive guide explains how to safely expand a Kafka cluster without data loss by covering applicable scenarios, pre‑conditions, anti‑pattern warnings, environment matrices, a detailed checklist, step‑by‑step Linux commands for broker preparation, partition‑rebalancing plan generation, throttled execution, real‑time monitoring, verification, rollback procedures, backup strategies, performance testing, common troubleshooting, FAQs and best‑practice scripts, all illustrated with code snippets and practical examples.

LinuxPartition RebalancingShell Scripts
0 likes · 47 min read
Zero‑Data‑Loss Kafka Cluster Scaling: Complete Step‑by‑Step Guide
Ray's Galactic Tech
Ray's Galactic Tech
Nov 17, 2025 · Backend Development

Kafka vs RabbitMQ vs RocketMQ: Which Message Broker Fits Your Use Case?

This article provides an in‑depth, multi‑dimensional comparison of Apache Kafka, RabbitMQ, and Apache RocketMQ—covering design philosophy, performance, reliability, features, ecosystem, operations, and typical scenarios—to help you choose the right message middleware for your architecture.

RabbitMQRocketMQkafka
0 likes · 6 min read
Kafka vs RabbitMQ vs RocketMQ: Which Message Broker Fits Your Use Case?
Tech Freedom Circle
Tech Freedom Circle
Nov 15, 2025 · Databases

How to Prevent Order Loss in a 100k TPS Flash Sale When the Master DB Crashes – 5 Practical Solutions

The article dissects a high‑traffic flash‑sale interview question—how to guarantee zero order loss at 100,000 TPS when the master MySQL instance fails—by explaining the underlying performance‑consistency conflict, the three skills interviewers assess, and presenting five concrete, code‑driven solutions ranging from MySQL parameter tuning to semi‑sync replication, local message tables, group replication, and Redis‑Kafka traffic shaping.

Data ConsistencyFlash SaleGroup Replication
0 likes · 28 min read
How to Prevent Order Loss in a 100k TPS Flash Sale When the Master DB Crashes – 5 Practical Solutions
mikechen
mikechen
Nov 13, 2025 · Backend Development

How to Diagnose and Resolve Kafka Message Backlog Issues

This article explains what Kafka message backlog is, outlines the main reasons it occurs—such as producer speed outpacing consumers, slow consumer processing, and downstream bottlenecks—and provides practical steps for producer throttling, consumer scaling and logic improvements, and Kafka cluster enhancements to eliminate the backlog.

Backend Developmentkafkamessage backlog
0 likes · 6 min read
How to Diagnose and Resolve Kafka Message Backlog Issues
Ray's Galactic Tech
Ray's Galactic Tech
Nov 9, 2025 · Backend Development

Hybrid Push‑Pull Timeline Architecture: Scaling Social Feeds for Billions

To serve billions of users with real‑time timelines, modern social platforms combine push‑based delivery for regular users and pull‑based retrieval for high‑profile accounts, employing hot‑cold separation, Kafka fan‑out, Redis caching, and scalable storage strategies to balance write and read loads.

RedisScalable ArchitectureSocial Media
0 likes · 9 min read
Hybrid Push‑Pull Timeline Architecture: Scaling Social Feeds for Billions
Ray's Galactic Tech
Ray's Galactic Tech
Nov 8, 2025 · Backend Development

Mastering Kafka Message Ordering: Guarantees, Pitfalls, and Practical Configurations

This guide explains how Kafka guarantees ordering within a single partition, why global ordering isn’t provided, and offers concrete producer and consumer configurations, key‑based partitioning, troubleshooting tips, verification methods, and multiple strategies for achieving global order when needed.

Message OrderingProducer Configurationconsumer
0 likes · 11 min read
Mastering Kafka Message Ordering: Guarantees, Pitfalls, and Practical Configurations
Ops Community
Ops Community
Nov 6, 2025 · Big Data

Zero Data Loss Kafka Cluster Scaling: From 3 to 10 Nodes – A Complete Guide

This comprehensive guide walks you through expanding or shrinking a production‑grade Kafka cluster—covering prerequisites, anti‑pattern warnings, environment matrices, step‑by‑step expansion and contraction procedures, partition rebalancing principles, monitoring, best practices, and troubleshooting—to ensure zero data loss during scaling.

Big DataPartition RebalancingZero Data Loss
0 likes · 27 min read
Zero Data Loss Kafka Cluster Scaling: From 3 to 10 Nodes – A Complete Guide
DataFunTalk
DataFunTalk
Nov 6, 2025 · Cloud Native

How Tencent Music Cut Kafka Costs by 50% with Cloud‑Native AutoMQ

Tencent Music migrated its massive Kafka streaming infrastructure to the cloud‑native AutoMQ platform, slashing operational costs by over half, achieving second‑level partition migration, and dramatically improving scaling efficiency while maintaining high‑throughput, low‑latency data processing for its music services.

AutoMQData StreamingOperations
0 likes · 16 min read
How Tencent Music Cut Kafka Costs by 50% with Cloud‑Native AutoMQ
Top Architect
Top Architect
Oct 31, 2025 · Backend Development

Mastering Message Queues: A Deep Dive into RabbitMQ, RocketMQ, and Kafka

This comprehensive guide explains the core components, exchange types, TTL, confirm mechanisms, consumer ACK/NACK, dead‑letter queues, and high‑availability features of RabbitMQ, RocketMQ, and Kafka, while also covering load balancing, ordering, transaction handling, and best practices for reliable message delivery.

Backend DevelopmentMessage QueueRabbitMQ
0 likes · 32 min read
Mastering Message Queues: A Deep Dive into RabbitMQ, RocketMQ, and Kafka
Big Data Technology & Architecture
Big Data Technology & Architecture
Oct 30, 2025 · Backend Development

What’s New in Apache Kafka 4.1? Core Features and Architecture Changes Explained

Apache Kafka 4.1.0 introduces native queue semantics, a new Streams rebalancing protocol, multi‑version Connect plugins, a revamped consumer‑group protocol, enhanced transaction safety, and numerous client, monitoring, and security improvements, offering a comprehensive upgrade over the 4.0 release.

Streamingdistributed-systemskafka
0 likes · 6 min read
What’s New in Apache Kafka 4.1? Core Features and Architecture Changes Explained
Su San Talks Tech
Su San Talks Tech
Oct 28, 2025 · Backend Development

How to Prevent MQ Message Loss: 5 Proven Strategies for Reliable Messaging

Discover the three stages where MQ messages can be lost and explore five practical solutions—including producer confirmations, message persistence, consumer acknowledgments, transactional messaging, and retry with dead‑letter queues—complete with code examples and guidance on selecting the right approach for different scenarios.

Dead‑Letter QueueMessage QueueRabbitMQ
0 likes · 14 min read
How to Prevent MQ Message Loss: 5 Proven Strategies for Reliable Messaging
Shepherd Advanced Notes
Shepherd Advanced Notes
Oct 24, 2025 · Backend Development

Why Choose Spring Boot + DelayQueue for a Custom Distributed Delayed-Task Queue?

The article systematically analyzes common distributed delayed‑task implementations—Redis ZSet scanning, message‑queue delay features, and Redis key‑expiration listeners—highlighting their pros, cons, and suitable scenarios, then proposes a Spring Boot + DelayQueue component to achieve precise timing, dynamic delays, and robust coordination.

DelayQueueDelayed TasksRedis
0 likes · 11 min read
Why Choose Spring Boot + DelayQueue for a Custom Distributed Delayed-Task Queue?