BestHub
Sign in
Tagged articles

Kafka

1415 articles · Page 5 of 15
Java Tech Enthusiast
Java Tech Enthusiast
Apr 30, 2024 · Backend Development

How to Prevent Message Loss and Duplication in Message Queues

This article examines why messages can be lost or duplicated in typical queue systems, explains the failure points from producer to broker to consumer, and provides practical techniques such as synchronous flushing, broker clustering, database unique constraints, and Redis deduplication to achieve reliable, idempotent processing.

BackendKafkaMessage Duplication
0 likes · 7 min read
How to Prevent Message Loss and Duplication in Message Queues
Lobster Programming
Lobster Programming
Apr 29, 2024 · Backend Development

Step-by-Step Guide to Installing and Running Kafka on Linux

This article explains what Kafka is, outlines the prerequisites, provides detailed Linux installation steps, configuration tweaks, startup commands, testing procedures, and troubleshooting tips, enabling readers to set up a functional Kafka cluster and produce‑consume messages.

InstallationKafkaLinux
0 likes · 5 min read
Step-by-Step Guide to Installing and Running Kafka on Linux
Architect's Guide
Architect's Guide
Apr 28, 2024 · Backend Development

Kafka Consumer Usage Example and Deep Dive into Offset Management, Rebalance, and Thread Safety

This article presents a Java Kafka consumer example, explains offset semantics (at‑most‑once, at‑least‑once, exactly‑once), details consumer rebalance mechanisms, partition assignment strategies, thread‑safety considerations, and showcases core poll, heartbeat, and auto‑commit implementations with accompanying code snippets.

ConsumerJavaKafka
0 likes · 14 min read
Kafka Consumer Usage Example and Deep Dive into Offset Management, Rebalance, and Thread Safety
Programmer XiaoFu
Programmer XiaoFu
Apr 26, 2024 · Backend Development

8 Code Patterns That Can Spike Your CPU to 100%

The article enumerates eight common coding practices—such as fetching massive data at once, using Kafka auto‑acknowledgement, infinite loops, aggressive multithreaded imports, bulk file synchronization, deadlocks, poorly designed regexes, and heavy real‑time calculations—that can cause Java services to hit 100% CPU usage, explaining the underlying mechanisms like full GC, thread‑context switching, and NFA backtracking.

CPU usageFull GCJava performance
0 likes · 9 min read
8 Code Patterns That Can Spike Your CPU to 100%
Su San Talks Tech
Su San Talks Tech
Apr 25, 2024 · Backend Development

Choosing the Right Message Queue: Kafka, RabbitMQ, RocketMQ, and ActiveMQ Compared

This article provides a comprehensive overview of four major message queue systems—Kafka, RabbitMQ, RocketMQ, and ActiveMQ—covering their fundamentals, architectures, key concepts, advantages, disadvantages, and guidance for selecting the most suitable solution based on throughput, reliability, scalability, and use‑case requirements.

BackendKafkaRabbitMQ
0 likes · 18 min read
Choosing the Right Message Queue: Kafka, RabbitMQ, RocketMQ, and ActiveMQ Compared
21CTO
21CTO
Apr 22, 2024 · Big Data

Inside Uber’s Real‑Time Data Infrastructure: How They Scale Streaming at Massive Scale

This article explores Uber’s sophisticated real‑time data infrastructure, detailing how the company leverages open‑source technologies such as Apache Kafka, Flink, Pinot, and Presto, and describing the architectural components, scaling challenges, multi‑region resilience, data back‑filling, and operational practices that enable low‑latency analytics for millions of daily rides and deliveries.

FlinkKafkaPinot
0 likes · 25 min read
Inside Uber’s Real‑Time Data Infrastructure: How They Scale Streaming at Massive Scale
37 Interactive Technology Team
37 Interactive Technology Team
Apr 15, 2024 · Backend Development

High-Performance Data Collection and Persistence Using OpenResty, Lua, and Kafka

The article describes a high‑throughput, fault‑tolerant data‑collection pipeline built with OpenResty, Lua, and Kafka that writes incoming records to disk using a cross‑worker mutex and cached file handles, rotates files via Nginx reload, and achieves over 13× performance versus typical online methods.

DataIngestionFileIOHighPerformance
0 likes · 7 min read
High-Performance Data Collection and Persistence Using OpenResty, Lua, and Kafka
Rare Earth Juejin Tech Community
Rare Earth Juejin Tech Community
Apr 6, 2024 · Big Data

Deep Dive into Kafka’s Underlying Mechanisms: Sequential Writes, Sparse Indexing, Segment Storage, and Replication

This article explores Apache Kafka’s core storage architecture, explaining how sequential append‑only writes, sparse indexing, segmented log files, and a leader‑based replication mechanism together enable high‑throughput, reliable, and scalable event streaming for massive data workloads.

Event StreamingKafkaReplication
0 likes · 11 min read
Deep Dive into Kafka’s Underlying Mechanisms: Sequential Writes, Sparse Indexing, Segment Storage, and Replication
Sanyou's Java Diary
Sanyou's Java Diary
Apr 1, 2024 · Backend Development

7 Real-World Message Queue Patterns Every Backend Engineer Should Know

This article shares seven classic use cases of message queues—including asynchronous decoupling, traffic smoothing, bus architecture, delayed tasks, broadcast consumption, distributed transactions, and data hub integration—illustrated with real-world experiences and code examples to help engineers design robust high‑concurrency systems.

Broadcast ConsumptionKafkaRocketMQ
0 likes · 12 min read
7 Real-World Message Queue Patterns Every Backend Engineer Should Know
Architect
Architect
Mar 30, 2024 · Backend Development

11 Ways to Auto-Close Expired Orders: From Passive to Distributed Queues

The article systematically compares eleven technical approaches for automatically closing expired e‑commerce orders, detailing each method’s implementation steps, trade‑offs, performance characteristics, and ideal deployment scenarios, and finally ranks them to guide practical selection.

KafkaRabbitMQRedis
0 likes · 19 min read
11 Ways to Auto-Close Expired Orders: From Passive to Distributed Queues
Su San Talks Tech
Su San Talks Tech
Mar 30, 2024 · Backend Development

7 Real-World Message Queue Patterns Every Backend Engineer Should Know

From asynchronous decoupling to distributed transactions, this article explores seven classic message‑queue use cases—such as peak‑shaving, event buses, delayed tasks, broadcast consumption, and data hub integration—illustrated with real‑world examples from e‑commerce, ride‑hailing, and lottery systems, plus RocketMQ code snippets.

KafkaRocketMQasynchronous-processing
0 likes · 13 min read
7 Real-World Message Queue Patterns Every Backend Engineer Should Know
HelloTech
HelloTech
Mar 21, 2024 · Big Data

Streaming Prediction System Construction and Real‑time Feature Templatization

The article describes how a Flink‑based streaming prediction platform was built to flatten peak request loads, reduce latency and memory use, and improve stability by deduplicating SDK calls, incrementally loading Hive features, partitioned caching, and comprehensive monitoring, while a templating system automates feature definition, SQL generation and stress testing, enabling real‑time supply‑demand forecasting that outperforms offline methods.

AIFlinkKafka
0 likes · 8 min read
Streaming Prediction System Construction and Real‑time Feature Templatization
Java Tech Enthusiast
Java Tech Enthusiast
Mar 16, 2024 · Backend Development

Common Causes of 100% CPU Usage in Java Backend Services

Typical reasons for Java backend services hitting 100 % CPU include fetching massive data sets, misusing Kafka auto‑acknowledge, infinite loops or recursion, thread‑pool overload causing context‑switch thrashing, bulk file synchronization, lock‑order deadlocks, poorly designed regular expressions, and intensive real‑time calculations under high concurrency.

CPUJavaKafka
0 likes · 8 min read
Common Causes of 100% CPU Usage in Java Backend Services
IT Services Circle
IT Services Circle
Mar 9, 2024 · Backend Development

Java Backend Interview Guide: Networks, Data Structures, MySQL Logs, Elasticsearch, Kafka, Microservices, and Spring Essentials

This article compiles a comprehensive Java backend interview guide covering network protocols, HTTP characteristics, core data structures and algorithms, MySQL logging mechanisms, Elasticsearch full‑text search, Kafka reliability and deduplication, common micro‑service components, load‑balancing algorithms, and essential Spring concepts such as IoC, AOP, transactions, and MVC workflow.

BackendElasticsearchJava
0 likes · 22 min read
Java Backend Interview Guide: Networks, Data Structures, MySQL Logs, Elasticsearch, Kafka, Microservices, and Spring Essentials
Architect's Guide
Architect's Guide
Mar 2, 2024 · Fundamentals

RabbitMQ vs Kafka: Core Differences and When to Use Each

This article compares RabbitMQ and Apache Kafka across architecture, message ordering, routing, timing, retention, fault handling, scalability, and consumer complexity, and provides guidance on which platform suits specific use‑cases such as flexible routing, strict ordering, long‑term retention, or high throughput.

KafkaRabbitMQStreaming
0 likes · 19 min read
RabbitMQ vs Kafka: Core Differences and When to Use Each
Airbnb Technology Team
Airbnb Technology Team
Mar 1, 2024 · Big Data

Riverbed: A Scalable Data Framework for Real‑time and Batch Processing at Airbnb

Airbnb’s Riverbed framework unifies streaming CDC events and batch Spark jobs behind a GraphQL‑based declarative API to automatically build and maintain distributed materialized views, using Kafka‑partitioned ordering and version control to deliver billions of daily updates with low‑latency reads for features such as payments and search.

AirbnbApache SparkKafka
0 likes · 8 min read
Riverbed: A Scalable Data Framework for Real‑time and Batch Processing at Airbnb
IT Services Circle
IT Services Circle
Feb 27, 2024 · Backend Development

Common Causes of 100% CPU Usage in Java Backend Systems and Their Solutions

This article analyzes typical reasons why Java backend services experience sustained 100% CPU usage—such as bulk data retrieval, Kafka auto‑acknowledgement, infinite loops, excessive multithreading, massive file synchronization, deadlocks, inefficient regular expressions, and heavy real‑time calculations—and offers practical mitigation strategies.

CPUJavaKafka
0 likes · 9 min read
Common Causes of 100% CPU Usage in Java Backend Systems and Their Solutions
Su San Talks Tech
Su San Talks Tech
Feb 26, 2024 · Backend Development

Why Does My Java Service Hit 100% CPU? 8 Real-World Causes and Fixes

This article examines eight common reasons why Java backend services experience 100% CPU usage—such as fetching excessive data, automatic Kafka acknowledgments, infinite loops, multithreaded imports, massive file syncs, deadlocks, inefficient regular expressions, and heavy real‑time calculations—and provides practical mitigation strategies.

CPUJavaKafka
0 likes · 9 min read
Why Does My Java Service Hit 100% CPU? 8 Real-World Causes and Fixes
MaGe Linux Operations
MaGe Linux Operations
Feb 20, 2024 · Big Data

Redis Streams vs Kafka: Which Is Better for Real‑Time Event Processing?

This article compares Redis Streams and Kafka, examining their architectures, ordering guarantees, consumer group models, scalability, and trade‑offs, and shows how Redis can emulate Kafka‑like semantics using the Runnel library, while highlighting memory‑speed benefits versus Kafka’s durable, unlimited log storage.

Event ProcessingKafkaRedis
0 likes · 9 min read
Redis Streams vs Kafka: Which Is Better for Real‑Time Event Processing?
Alibaba Cloud Developer
Alibaba Cloud Developer
Feb 19, 2024 · Big Data

Master Kafka: Core Concepts, Architecture, and Interview Essentials

This comprehensive guide explains why Kafka is the industry standard for real‑time data pipelines, compares it with RabbitMQ and RocketMQ, details its architecture, APIs, leader election, offset management, idempotence, rebalance handling, and provides practical interview questions and code examples for engineers.

Kafka
0 likes · 19 min read
Master Kafka: Core Concepts, Architecture, and Interview Essentials
Programmer DD
Programmer DD
Feb 17, 2024 · Backend Development

How to Seamlessly Upgrade Spring Boot 2.x to 3.x: Step‑by‑Step Guide

This guide explains why upgrading from Spring Boot 2.x to 3.x is essential, outlines the required JDK 17 migration, Spring Boot version bump, configuration property changes, Jakarta EE transition, and updates to Spring Security, Kafka, RestTemplate, and OpenAPI integration, providing code examples for each step.

Jakarta EEJava 17Kafka
0 likes · 10 min read
How to Seamlessly Upgrade Spring Boot 2.x to 3.x: Step‑by‑Step Guide
Sanyou's Java Diary
Sanyou's Java Diary
Feb 1, 2024 · Backend Development

Message Queues Unveiled: From Decoupling to Platformization and Core Architectures

This article traces the two‑decade evolution of message queues—from early decoupling solutions like ActiveMQ, through high‑throughput designs such as Kafka, to modern platformized systems like RocketMQ and Pulsar—while explaining fundamental concepts, partitioning, and storage architectures that underpin today’s distributed messaging platforms.

KafkaPulsarRocketMQ
0 likes · 17 min read
Message Queues Unveiled: From Decoupling to Platformization and Core Architectures
Efficient Ops
Efficient Ops
Jan 31, 2024 · Databases

Why ClickHouse Beats Elasticsearch for High‑Performance Log Analytics

Facing data security and cost challenges in SaaS, the author evaluates ClickHouse versus Elasticsearch, highlighting ClickHouse’s superior write throughput, query speed, lower storage and CPU usage, and provides detailed deployment guides for Zookeeper, Kafka, FileBeat, and ClickHouse to build a cost‑effective private analytics platform.

ClickHouseDatabase deploymentElasticsearch
0 likes · 8 min read
Why ClickHouse Beats Elasticsearch for High‑Performance Log Analytics
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Jan 27, 2024 · Backend Development

Understanding Message Queues: Concepts, Use Cases, Types, and Common Implementations

This article introduces message queues, explaining their role in high‑traffic systems, key characteristics such as asynchronous communication, decoupling, reliability and buffering, compares point‑to‑point and publish‑subscribe models, and reviews popular implementations like RabbitMQ, Kafka, ActiveMQ, RocketMQ and Pulsar.

Backend DevelopmentKafkaRabbitMQ
0 likes · 8 min read
Understanding Message Queues: Concepts, Use Cases, Types, and Common Implementations
MaGe Linux Operations
MaGe Linux Operations
Jan 21, 2024 · Big Data

Master Kafka: Core Concepts, Metrics, and Troubleshooting Guide

This article explains Kafka's fundamental components, version evolution, key monitoring metrics for producers, brokers, consumers and Zookeeper, and provides step‑by‑step troubleshooting methods for common issues such as slow topic throughput and message backlog.

KafkaStreamingTroubleshooting
0 likes · 8 min read
Master Kafka: Core Concepts, Metrics, and Troubleshooting Guide
Sanyou's Java Diary
Sanyou's Java Diary
Jan 18, 2024 · Big Data

Inside Kafka’s Network Layer: How Messages Are Sent and Received

This article walks through Kafka’s network‑layer internals, detailing how the client prepares, pre‑sends, and actually sends messages, then reads and parses responses, with code snippets that illustrate the Selector, KafkaChannel, and related classes in Java NIO.

Java NIOKafkaMessage Receiving
0 likes · 22 min read
Inside Kafka’s Network Layer: How Messages Are Sent and Received
Rare Earth Juejin Tech Community
Rare Earth Juejin Tech Community
Jan 13, 2024 · Big Data

What Is Kafka? Overview, Architecture, Features, Deployment, and Sample Code

Kafka, an Apache‑developed distributed publish/subscribe messaging system, provides reliable, high‑throughput real‑time data streaming with producers, consumers, brokers, streams, and connectors, and the article explains its core concepts, architecture, advantages, deployment methods, use cases, and includes Java code examples for producers and consumers.

JavaKafkaStreaming
0 likes · 8 min read
What Is Kafka? Overview, Architecture, Features, Deployment, and Sample Code
Architect
Architect
Jan 4, 2024 · Backend Development

RabbitMQ vs Kafka: How to Choose the Right Messaging System

This article explains asynchronous messaging patterns, compares RabbitMQ and Apache Kafka in depth, and provides a step‑by‑step decision guide that highlights their architectural differences, strengths, weaknesses, and suitable use‑cases for modern software systems.

KafkaRabbitMQTechnology Comparison
0 likes · 11 min read
RabbitMQ vs Kafka: How to Choose the Right Messaging System
Programmer DD
Programmer DD
Jan 2, 2024 · Backend Development

How to Seamlessly Upgrade Spring Boot 2.x to 3.x – Step-by-Step Guide

This guide walks developers through upgrading from Spring Boot 2.x to 3.x, covering JDK 17 migration, pom.xml changes, configuration property updates, Jakarta EE transition, security reconfiguration, Kafka template adjustments, and OpenAPI integration, with code examples for each step.

Jakarta EEJavaKafka
0 likes · 9 min read
How to Seamlessly Upgrade Spring Boot 2.x to 3.x – Step-by-Step Guide
Architect
Architect
Dec 30, 2023 · Big Data

Designing a Scalable Log Collection Agent: Lessons from Vivo’s Bees‑Agent

This article details the end‑to‑end design of Vivo’s custom log‑collection agent, covering file discovery with inotify, unique file identification using inode and content hash, real‑time reading via RandomAccessFile, checkpointing, Kafka integration, offline HDFS ingestion, resource throttling, and platform‑wide management, while comparing it with open‑source alternatives.

Agent designKafkaLog Collection
0 likes · 26 min read
Designing a Scalable Log Collection Agent: Lessons from Vivo’s Bees‑Agent
Cloud Native Technology Community
Cloud Native Technology Community
Dec 29, 2023 · Cloud Native

Mastering Strimzi Kafka Operator: Architecture, Deployment & Tuning on K8s

This article provides an in‑depth analysis of the Strimzi Kafka Operator, covering its core features, multi‑layer architecture, detailed installation steps on Kubernetes, Kafka cluster creation, production/consumption workflows, and the internal reconciliation mechanisms that enable automated scaling, storage tuning, and fault‑recovery.

Cloud NativeKafkaKubernetes
0 likes · 11 min read
Mastering Strimzi Kafka Operator: Architecture, Deployment & Tuning on K8s
Alibaba Cloud Native
Alibaba Cloud Native
Dec 28, 2023 · Cloud Computing

How to Set Up No‑Code Data Dump from Alibaba Cloud Kafka to OSS

This guide explains how to use Alibaba Cloud Message Queue Kafka's no‑code, fully managed, serverless dump feature to transfer data to OSS, covering its benefits, typical scenarios, required prerequisites, step‑by‑step configuration, testing, and verification of the resulting objects.

Alibaba CloudData IntegrationKafka
0 likes · 9 min read
How to Set Up No‑Code Data Dump from Alibaba Cloud Kafka to OSS
Efficient Ops
Efficient Ops
Dec 27, 2023 · Big Data

Why ClickHouse Beats Elasticsearch for Log Analytics – Performance, Cost & Deployment

This article compares ClickHouse and Elasticsearch for log analytics, highlighting ClickHouse’s superior write throughput, query speed, and lower server costs, then details a cost‑effective deployment architecture—including Zookeeper, Kafka, FileBeat, and ClickHouse setup—and shares optimization tips and visualization using ClickVisual.

ClickHouseElasticsearchKafka
0 likes · 13 min read
Why ClickHouse Beats Elasticsearch for Log Analytics – Performance, Cost & Deployment
ITPUB
ITPUB
Dec 24, 2023 · Backend Development

Why Kafka Is the Backbone of Modern Messaging, Streaming, and Data Pipelines

This article explains how Kafka serves as a high‑throughput, durable messaging system, a reliable storage layer, a log‑aggregation hub, a stream‑processing engine, and a core component for CDC, system migration, monitoring, and event‑sourcing architectures.

CDCEvent SourcingKafka
0 likes · 9 min read
Why Kafka Is the Backbone of Modern Messaging, Streaming, and Data Pipelines
dbaplus Community
dbaplus Community
Dec 20, 2023 · Operations

Scaling Kafka to 1000+ Nodes: Governance, Auto‑Balancing & Tiered Storage

This article outlines how a large‑scale Kafka deployment of over a thousand machines across dozens of clusters was engineered for stability and efficiency through a custom Guardian controller that adds partition‑level throttling, automatic balancing, multi‑tenant isolation, cross‑IDC management, tiered storage, audit capabilities, and fully automated operational workflows.

KafkaOperationsauto-balancing
0 likes · 21 min read
Scaling Kafka to 1000+ Nodes: Governance, Auto‑Balancing & Tiered Storage
MaGe Linux Operations
MaGe Linux Operations
Dec 18, 2023 · Backend Development

How to Prevent Kafka Message Loss in Critical Transaction Systems

This article explains why Kafka can lose messages in production, broker, and consumer stages, analyzes root causes such as asynchronous batch sends, JVM crashes, and network failures, and provides practical solutions including callbacks, retry mechanisms, replication settings, and manual offset commits to ensure reliable delivery.

Backend DevelopmentKafkaMessage reliability
0 likes · 10 min read
How to Prevent Kafka Message Loss in Critical Transaction Systems
IT Xianyu
IT Xianyu
Dec 16, 2023 · Backend Development

Understanding CQRS and Event Sourcing with Spring Microservices

This article explains the CQRS pattern, its origins, benefits, and pitfalls, then details how to implement CQRS and event sourcing in Spring‑based microservices using Axon and Kafka, while discussing architectural considerations, scalability, consistency, and tooling.

ArchitectureAxonCQRS
0 likes · 12 min read
Understanding CQRS and Event Sourcing with Spring Microservices
Tencent Cloud Middleware
Tencent Cloud Middleware
Dec 12, 2023 · Cloud Native

How Tencent Cloud Implements Tiered Storage for Kafka: Architecture, Challenges, and Evolution

This article examines the challenges of Kafka's traditional architecture, explains why local‑state heavy deployments cause operational difficulty and resource waste, and details Tencent Cloud's elastic, storage‑compute‑separated designs—including tiered storage, segment state machines, offset constraints, and performance optimizations—while sharing practical implementation insights and future directions.

Cloud NativeData LifecycleElastic Architecture
0 likes · 17 min read
How Tencent Cloud Implements Tiered Storage for Kafka: Architecture, Challenges, and Evolution
Architect's Guide
Architect's Guide
Dec 12, 2023 · Backend Development

Understanding Kafka Consumer: Delivery Guarantees, Rebalance Mechanisms, Partition Assignment, and Thread Safety

This article provides a comprehensive guide to KafkaConsumer, covering message delivery semantics (at‑most‑once, at‑least‑once, exactly‑once), practical exactly‑once implementations, consumer rebalance triggers and strategies, partition assignment algorithms, thread‑safety considerations, and detailed Java code examples of the consumer workflow.

ConsumerJavaKafka
0 likes · 14 min read
Understanding Kafka Consumer: Delivery Guarantees, Rebalance Mechanisms, Partition Assignment, and Thread Safety
Nullbody Notes
Nullbody Notes
Dec 9, 2023 · Backend Development

Designing Load Balancing and Keep‑Alive Strategies in Go

This article walks through implementing three load‑balancing algorithms (random, round‑robin, weighted round‑robin) in Go, explains the weighted selection math with step‑by‑step examples, and presents two service‑liveness solutions—heartbeat via Docker/Kafka and HTTP health checks—complete with code snippets and deployment commands.

DockerGoKafka
0 likes · 9 min read
Designing Load Balancing and Keep‑Alive Strategies in Go
ITPUB
ITPUB
Dec 2, 2023 · Backend Development

Why Did My Flink Kafka Job Lose Data? Uncovering Misconfigured Bootstrap Servers

A Flink job that reads from Kafka and writes to Elasticsearch was losing data because the bootstrap.servers list mixed production and pre‑release clusters, causing random server selection, partition discovery failures, and offset mismatches, which were resolved by correcting the server configuration.

Bootstrap ServersData lossFlink
0 likes · 8 min read
Why Did My Flink Kafka Job Lose Data? Uncovering Misconfigured Bootstrap Servers
JavaEdge
JavaEdge
Nov 24, 2023 · Backend Development

Why Kafka Is the Ultimate Backbone for Modern Backend Systems

This article explores how Kafka serves as a versatile backbone for messaging, durable storage, log aggregation, monitoring, commit logs, recommendation pipelines, stream processing, CDC, system migration, and event sourcing, highlighting its performance, reliability, and practical deployment patterns.

BackendKafkaStreaming
0 likes · 10 min read
Why Kafka Is the Ultimate Backbone for Modern Backend Systems
Programmer DD
Programmer DD
Nov 24, 2023 · Backend Development

What’s New in Spring Boot 3.2? Explore Java 21 Features and Virtual Threads

Spring Boot 3.2, released shortly after Java 21, brings a host of enhancements such as virtual thread support, CRaC checkpoint restore, SSL bundle reloading, improved observability, new RestClient and JdbcClient, Jetty 12, Pulsar, Kafka and RabbitMQ SSL, redesigned nested JAR handling, Docker image build upgrades, and a comprehensive video walkthrough by Josh Long.

Backend DevelopmentDockerJava 21
0 likes · 7 min read
What’s New in Spring Boot 3.2? Explore Java 21 Features and Virtual Threads
Selected Java Interview Questions
Selected Java Interview Questions
Nov 21, 2023 · Backend Development

Design and Implementation of a Generic Asynchronous Processing SDK for Spring Applications

This article introduces a generic asynchronous processing SDK for Spring-based backend systems, explaining its purpose, advantages, underlying principles, components such as Kafka, XXL‑Job, MySQL, and Vue, design patterns employed, database schema, configuration via Apollo, usage examples, and deployment details.

KafkaSDKSpring
0 likes · 8 min read
Design and Implementation of a Generic Asynchronous Processing SDK for Spring Applications
IT Architects Alliance
IT Architects Alliance
Nov 17, 2023 · Backend Development

Design and Implementation of a Generic Asynchronous Processing SDK

This article introduces a generic asynchronous processing SDK for Java Spring applications, covering its purpose, advantages, underlying principles, component choices such as Kafka, XXL‑Job, MySQL, design patterns, database schema, configuration via Apollo, usage instructions, safety considerations, and provides code examples and a GitHub repository link.

@AsyncAOPKafka
0 likes · 9 min read
Design and Implementation of a Generic Asynchronous Processing SDK
DeWu Technology
DeWu Technology
Nov 15, 2023 · Backend Development

Thread Profiling: Design and Implementation of Client‑Server Performance Analysis

Thread profiling uses threshold‑triggered tasks on business threads to capture stack snapshots, which a dedicated profiler thread sends via high‑performance gRPC to a server that queues them in Kafka, enriches and stores them in ClickHouse, correlates with OpenTelemetry traces, and provides metrics that let developers quickly pinpoint latency bottlenecks and improve system stability.

GoJavaKafka
0 likes · 11 min read
Thread Profiling: Design and Implementation of Client‑Server Performance Analysis
Architect
Architect
Nov 13, 2023 · Backend Development

Designing a Robust Asynchronous Processing SDK with Spring, Kafka, and MySQL

This article explains why asynchronous processing is needed in evolving systems, outlines the goals of guaranteeing execution without blocking the main flow, and walks through a complete SDK design that uses Spring transaction events, Kafka, XXL‑Job, MySQL, and a Vue UI, including configuration, code snippets, and deployment details.

BackendDesignKafka
0 likes · 10 min read
Designing a Robust Asynchronous Processing SDK with Spring, Kafka, and MySQL
Selected Java Interview Questions
Selected Java Interview Questions
Nov 10, 2023 · Backend Development

Design and Implementation of a Generic Asynchronous Processing SDK for Java Backend Systems

This article introduces a generic asynchronous processing SDK for Java back‑ends, explaining its purpose, advantages, underlying principles, components, design patterns, database schema, configuration, usage, and operational considerations, and provides complete code and configuration examples.

Backend DevelopmentJavaKafka
0 likes · 9 min read
Design and Implementation of a Generic Asynchronous Processing SDK for Java Backend Systems
macrozheng
macrozheng
Nov 9, 2023 · Big Data

7 Real-World Kafka Use Cases Every Engineer Should Know

This article explains Kafka's core components and features, then details seven practical scenarios—including log processing, recommendation streams, monitoring, CDC, system migration, event sourcing, and message queuing—showing how Kafka powers modern distributed systems.

KafkaStreamingbig data
0 likes · 12 min read
7 Real-World Kafka Use Cases Every Engineer Should Know
ITPUB
ITPUB
Nov 7, 2023 · Big Data

7 Real-World Kafka Use Cases That Power Modern Distributed Systems

This article introduces Apache Kafka’s core components and key features, then details seven practical use cases—including log processing, recommendation streams, monitoring, CDC, system migration, event sourcing, and message queuing—illustrated with diagrams and step‑by‑step workflows for distributed systems.

KafkaStreamingbig data
0 likes · 10 min read
7 Real-World Kafka Use Cases That Power Modern Distributed Systems
Selected Java Interview Questions
Selected Java Interview Questions
Nov 5, 2023 · Backend Development

Design and Implementation of a High‑Performance Distributed Reconciliation System for Large‑Scale Payment Orders

This article presents a comprehensive design of a distributed reconciliation system that handles tens of millions of daily payment orders by using a six‑module architecture, Kafka for decoupled state transitions, Hive for large‑scale data processing, and Java‑based plug‑in patterns to achieve six‑nine accuracy and significant operational cost savings.

HiveJavaKafka
0 likes · 15 min read
Design and Implementation of a High‑Performance Distributed Reconciliation System for Large‑Scale Payment Orders
Smart Era Software Development
Smart Era Software Development
Nov 3, 2023 · Operations

Inside Bilibili’s Kafka: Challenges, Guardian Federation, and Future Automation

The article details how Bilibili operates over 1,000 Kafka nodes across 20+ clusters, outlines the scalability and stability challenges they faced, and explains the design and implementation of their self‑built Guardian federation controller, partition‑level throttling, automatic balancing, multi‑tenant isolation, tiered storage, audit, and automated ops workflows.

AuditCluster GovernanceKafka
0 likes · 21 min read
Inside Bilibili’s Kafka: Challenges, Guardian Federation, and Future Automation
Bilibili Tech
Bilibili Tech
Nov 3, 2023 · Big Data

Comprehensive Governance and Optimization Strategies for Large‑Scale Kafka Clusters

To tame a petabyte‑scale Kafka deployment of over 1,000 brokers, the team built a Raft‑based federation controller (Guardian) that adds per‑partition I/O throttling, disk‑aware automatic balancing, multi‑tenant isolation, cross‑IDC migration, request‑queue splitting, tiered storage, auditing, and fully automated rolling upgrades, enabling stable, self‑healing operations.

Cluster GovernanceKafkaPerformance Optimization
0 likes · 21 min read
Comprehensive Governance and Optimization Strategies for Large‑Scale Kafka Clusters
Top Architect
Top Architect
Nov 2, 2023 · Big Data

Understanding Distributed Systems and Kafka: Concepts, Architecture, and Ensuring Ordered Message Consumption

This article introduces the fundamentals of distributed systems, provides an overview of Apache Kafka’s architecture and core components, explains how Kafka ensures message ordering within partitions, and outlines Java‑based strategies to guarantee ordered consumption, including single‑partition consumption, partition assignment, and key‑based partitioning.

JavaKafkabig data
0 likes · 10 min read
Understanding Distributed Systems and Kafka: Concepts, Architecture, and Ensuring Ordered Message Consumption
HelloTech
HelloTech
Oct 31, 2023 · Big Data

Investigation of Data Loss in a Flink Kafka Consumer Caused by Mixed Kafka Cluster Configuration

The data loss in a Flink‑Kafka job was caused by a mis‑configured bootstrap.servers list that mixed production and pre‑release Kafka clusters, leading different subtasks to connect to different clusters, resulting in inconsistent partition discovery and offset fetching, which omitted several partitions until the list was corrected.

Cluster ConfigurationData lossElasticsearch
0 likes · 8 min read
Investigation of Data Loss in a Flink Kafka Consumer Caused by Mixed Kafka Cluster Configuration
JD Cloud Developers
JD Cloud Developers
Oct 25, 2023 · Backend Development

Master Kafka: Core Concepts, Architecture, and Practical Tips

This article explains Kafka's fundamentals, including topics, partitions, brokers, replication, producer‑consumer workflow, consumer groups, offset management, and common exception handling, while providing code examples and diagrams to help developers understand and effectively use this distributed messaging system.

JavaKafkaReplication
0 likes · 21 min read
Master Kafka: Core Concepts, Architecture, and Practical Tips
Architect
Architect
Oct 19, 2023 · Industry Insights

How Vivo Built a Highly Available Push System: Multi‑Region Architecture, Real‑Time Traffic Scheduling, and Disaster‑Recovery Strategies

This article analyzes the design of Vivo's push notification platform, detailing its high‑concurrency requirements, three‑region long‑connection deployment, traffic‑scheduling bypass layer, and layered storage disaster‑recovery solutions, while explaining the trade‑offs and performance metrics behind each architectural decision.

Cloud NativeKafkaRedis
0 likes · 14 min read
How Vivo Built a Highly Available Push System: Multi‑Region Architecture, Real‑Time Traffic Scheduling, and Disaster‑Recovery Strategies
Wukong Talks Architecture
Wukong Talks Architecture
Oct 13, 2023 · Backend Development

7 Common Message Queue Scenarios and Their Implementations

This article explains seven typical message‑queue use cases—including ordinary, ordered, delayed, transactional, trace, dead‑letter, and priority messages—detailing their business motivations, implementation challenges, and concrete code examples for Kafka, RocketMQ, Pulsar, and RabbitMQ.

KafkaPulsarRabbitMQ
0 likes · 11 min read
7 Common Message Queue Scenarios and Their Implementations
MaGe Linux Operations
MaGe Linux Operations
Oct 8, 2023 · Big Data

Understanding Kafka: Core Concepts, Architecture, and Performance Secrets

This article explains Kafka’s fundamental role as a message system, detailing topics, partitions, producers, consumers, replica management, consumer groups, the controller, Zookeeper coordination, and performance optimizations such as sequential writes, zero‑copy, log segmentation, and network design, providing a comprehensive overview for big‑data practitioners.

Consumer GroupKafkaPerformance Optimization
0 likes · 11 min read
Understanding Kafka: Core Concepts, Architecture, and Performance Secrets
Efficient Ops
Efficient Ops
Oct 7, 2023 · Big Data

Master Kafka Basics: Topics, Partitions, Producers, and Cluster Architecture

This article explains Kafka's role as a messaging system, covering core concepts such as topics, partitions, producers, consumers, messages, cluster architecture, replicas, consumer groups, controller coordination with Zookeeper, and performance optimizations like sequential writes and zero‑copy networking.

Consumer GroupKafkaTopic
0 likes · 11 min read
Master Kafka Basics: Topics, Partitions, Producers, and Cluster Architecture
Sanyou's Java Diary
Sanyou's Java Diary
Oct 6, 2023 · Backend Development

Inside Kafka Broker: How Its Network Architecture Handles Millions of Requests

This article deeply dissects Kafka Broker's network architecture and request‑processing pipeline, covering sequential, multithreaded, and event‑driven designs, the Reactor pattern, Acceptor and Processor threads, core request flow, and practical tuning parameters for high‑throughput, low‑latency deployments.

Backend DevelopmentKafkaNetwork Architecture
0 likes · 22 min read
Inside Kafka Broker: How Its Network Architecture Handles Millions of Requests
Java High-Performance Architecture
Java High-Performance Architecture
Sep 28, 2023 · Databases

How to Use Debezium for MySQL CDC in Spring Boot Without Adding Extra Middleware

Learn how to capture MySQL data changes using Debezium's CDC capabilities within a Spring Boot application, avoiding heavyweight message brokers by leveraging binlog monitoring, configuring connectors, handling snapshots, and processing change events for use cases like cache invalidation, data integration, and simplifying monolithic architectures.

CDCData IntegrationDebezium
0 likes · 24 min read
How to Use Debezium for MySQL CDC in Spring Boot Without Adding Extra Middleware
HomeTech
HomeTech
Sep 27, 2023 · Backend Development

Design and Evolution of a High‑Availability SMS Platform at AutoHome

This article details the architectural evolution, high‑availability strategies, fault‑monitoring mechanisms, and performance optimizations of AutoHome's enterprise SMS platform, covering its migration from .Net to Java, service decomposition with Kafka, multi‑datacenter deployment, and operational safeguards for large‑scale events.

BackendDistributed ArchitectureKafka
0 likes · 9 min read
Design and Evolution of a High‑Availability SMS Platform at AutoHome
Top Architect
Top Architect
Sep 25, 2023 · Backend Development

RabbitMQ vs Kafka: Detailed Comparison and When to Use Each

This article provides an in‑depth technical comparison of RabbitMQ and Apache Kafka, covering their core architectural differences, message ordering, routing, timing, retention, fault handling, scalability, consumer complexity, and offers guidance on selecting the appropriate platform for various backend scenarios.

KafkaRabbitMQStreaming
0 likes · 18 min read
RabbitMQ vs Kafka: Detailed Comparison and When to Use Each
Efficient Ops
Efficient Ops
Sep 24, 2023 · Big Data

Mastering Kafka: From Basics to Advanced Operations and Performance Tuning

This article provides a comprehensive overview of Apache Kafka, covering its architecture, core concepts such as topics, partitions, and replicas, common operational commands, and practical performance‑tuning tips for high‑throughput, low‑latency streaming workloads.

KafkaOperationsdistributed systems
0 likes · 23 min read
Mastering Kafka: From Basics to Advanced Operations and Performance Tuning
Sanyou's Java Diary
Sanyou's Java Diary
Sep 21, 2023 · Big Data

Understanding Kafka: Core Concepts, Architecture, and Reliability Explained

This article provides a comprehensive overview of Kafka, covering its overall architecture, key components such as brokers, producers, consumers, topics, partitions, replicas, and ZooKeeper, as well as logical and physical storage mechanisms, producer and consumer workflows, configuration parameters, partition assignment strategies, rebalancing, and the replication model that ensures data reliability.

Data StreamingKafkadistributed systems
0 likes · 18 min read
Understanding Kafka: Core Concepts, Architecture, and Reliability Explained
Wukong Talks Architecture
Wukong Talks Architecture
Sep 21, 2023 · Backend Development

Detecting and Preventing Message Loss in Kafka Message Queues

This article explains how to detect, diagnose, and prevent message loss in Kafka-based message queue systems by covering system decoupling, traffic control, data consistency challenges, producer, broker, and consumer issues, and offering configuration, monitoring, and operational best‑practice solutions.

Kafkadata consistencydistributed systems
0 likes · 12 min read
Detecting and Preventing Message Loss in Kafka Message Queues
ITPUB
ITPUB
Sep 15, 2023 · Databases

Importing Billions of Kafka Rows into Doris and Benchmarking Against ClickHouse

This article explains Doris's various data import methods, focuses on the routine load approach for Kafka streams, describes how to handle mixed‑schema topics using the max_error_number parameter, and compares query performance of a 130 million‑row dataset against ClickHouse, highlighting each system's strengths and limitations.

ClickHouseDorisKafka
0 likes · 10 min read
Importing Billions of Kafka Rows into Doris and Benchmarking Against ClickHouse
ITPUB
ITPUB
Sep 13, 2023 · Backend Development

Why Is Kafka So Fast? 7 Core Techniques Behind Its High Throughput

This article explains how Kafka achieves million‑message‑per‑second throughput by leveraging zero‑copy I/O, an append‑only log, batch processing, compression, consumer pull optimization, unflushed memory buffers, and JVM garbage‑collection tuning, detailing each mechanism and its impact on performance.

Batch ProcessingCompressionGC optimization
0 likes · 14 min read
Why Is Kafka So Fast? 7 Core Techniques Behind Its High Throughput
Sanyou's Java Diary
Sanyou's Java Diary
Sep 7, 2023 · Operations

How to Keep Kafka Stable: Proven Practices for Prevention, Monitoring, and Recovery

This comprehensive guide explains how to ensure Kafka stability by applying proactive prevention, continuous runtime monitoring, and effective fault‑resolution strategies, covering producer and consumer tuning, cluster configuration, performance optimization, alerting, and idempotent consumption to prevent message loss and service disruption.

Kafkafault-recoveryperformance tuning
0 likes · 30 min read
How to Keep Kafka Stable: Proven Practices for Prevention, Monitoring, and Recovery
Mike Chen's Internet Architecture
Mike Chen's Internet Architecture
Sep 7, 2023 · Backend Development

Comprehensive Overview of Message Queues: Types, Core Concepts, and Comparison of Kafka, RocketMQ, and RabbitMQ

This article provides a detailed overview of popular message queue systems, explains their core concepts such as decoupling and eventual consistency, and compares the advantages and disadvantages of Kafka, RocketMQ, RabbitMQ, and other notable MQ solutions for high‑concurrency scenarios.

KafkaRabbitMQRocketMQ
0 likes · 7 min read
Comprehensive Overview of Message Queues: Types, Core Concepts, and Comparison of Kafka, RocketMQ, and RabbitMQ
Shepherd Advanced Notes
Shepherd Advanced Notes
Sep 1, 2023 · Big Data

Visual Guide to Kafka’s Architecture and Core Principles

This article explains Kafka’s architecture in depth, covering components such as producers, consumers, topics, partitions, replication, Zookeeper coordination, the Controller’s election and responsibilities, state machines, and the NIO‑based network model, using clear diagrams and analogies.

ArchitectureControllerKafka
0 likes · 14 min read
Visual Guide to Kafka’s Architecture and Core Principles
ITPUB
ITPUB
Aug 26, 2023 · Operations

When to Choose Kafka Over RabbitMQ? A Detailed Comparison

This article compares Kafka and RabbitMQ across scalability, durability, latency, data flow, ordering, reliability, persistence, extensibility, and complexity, then outlines ideal use cases for each and offers practical guidance on selecting the right message queue for a project.

KafkaRabbitMQcomparison
0 likes · 8 min read
When to Choose Kafka Over RabbitMQ? A Detailed Comparison
Bilibili Tech
Bilibili Tech
Aug 11, 2023 · Backend Development

Designing a High‑Performance Asynchronous Event System for Bilibili’s Like Service

Bilibili’s railgun platform transforms its high‑traffic like service into a scalable, fault‑tolerant system by moving writes to an asynchronous, Kafka‑driven pipeline, applying CQRS, partitioned processing, idempotency, hot‑key isolation, rate‑limiting, and unified SDKs, dramatically reducing database load and achieving ten‑fold throughput gains.

BackendCQRSKafka
0 likes · 21 min read
Designing a High‑Performance Asynchronous Event System for Bilibili’s Like Service