BestHub
Sign in
Tagged articles

Kafka

1415 articles · Page 1 of 15
Ray's Galactic Tech
Ray's Galactic Tech
Sep 29, 2026 · Backend Development

Production-Grade LLM Annotation Pipeline: Pydantic, PostgreSQL & Kafka Patterns

The article details how a SaaS team evolved a fragile LLM ticket-classification script into a production-grade pipeline using strict Pydantic contracts, PostgreSQL state machines with atomic leasing, Kafka outbox patterns, immutable training snapshots, and metric-gated rollouts to handle model drift, replay, and human review safely.

KafkaLLM data pipelinePostgreSQL
0 likes · 27 min read
Production-Grade LLM Annotation Pipeline: Pydantic, PostgreSQL & Kafka Patterns
liandk
liandk
Sep 25, 2026 · Backend Development

MQ Production Failure Troubleshooting: Message Loss, Duplicates, Backlog & Dead Letters

This comprehensive guide covers the five critical MQ production failures — message loss, duplicate consumption, massive backlog, consumer hangs, and dead letter queue blocking — with root cause analysis, emergency mitigation steps, and long-term architectural fixes for RocketMQ, Kafka, and RabbitMQ.

KafkaMQRabbitMQ
0 likes · 14 min read
MQ Production Failure Troubleshooting: Message Loss, Duplicates, Backlog & Dead Letters
Cloud Architecture
Cloud Architecture
Sep 24, 2026 · Backend Development

Scaling Spring Boot Sign-In to 100M Users: Distributed Architecture Evolution & Production Hardening

This article details the evolution of a Spring Boot daily sign-in system from 10K to 100M users, covering Redis bitmap sharding, Lua atomic operations, command outbox pattern, Kafka async reward processing, idempotency guarantees, fault tolerance strategies, and production-grade observability with real incident postmortems.

BitmapCommand OutboxKafka
0 likes · 36 min read
Scaling Spring Boot Sign-In to 100M Users: Distributed Architecture Evolution & Production Hardening
DevOps Operations Practice
DevOps Operations Practice
Sep 9, 2026 · Operations

Deploy Kafka 3.8.1 KRaft Cluster in 10 Minutes with Docker Compose

This guide walks through deploying a three-node Kafka 3.8.1 cluster using KRaft mode and Docker Compose, covering machine preparation, kernel tuning, cluster ID generation, node-specific configuration, startup, and verification steps including topic creation and message production/consumption.

Cluster SetupDockerDocker Compose
0 likes · 9 min read
Deploy Kafka 3.8.1 KRaft Cluster in 10 Minutes with Docker Compose
Xiaolin Talks Programming
Xiaolin Talks Programming
Sep 5, 2026 · Backend Development

Migrating Log Platform from Elasticsearch to ClickHouse: Table Design, Materialized Views & Tuning Lessons

The author details migrating a high-volume logging platform from Elasticsearch to ClickHouse, covering schema design with ReplicatedMergeTree, Spring Boot integration via JDBC and MyBatis, materialized views for pre-aggregation, and optimization techniques like skip indexes, batch inserts, and capacity planning.

ClickHouseElasticsearchKafka
0 likes · 21 min read
Migrating Log Platform from Elasticsearch to ClickHouse: Table Design, Materialized Views & Tuning Lessons
Spring Full-Stack Practical Cases
Spring Full-Stack Practical Cases
Sep 4, 2026 · Backend Development

10 Message Queue Patterns for Spring Boot: Decoupling, Async, Traffic Shaping & More

This article details 10 practical message queue scenarios for Spring Boot microservices, covering system decoupling, asynchronous processing, traffic shaping for flash sales, data synchronization, centralized logging, broadcast configuration updates, ordered message processing, delayed messages for timeouts, retry mechanisms with dead-letter queues, and transactional messaging for distributed consistency, with code examples for RabbitMQ, Kafka, and RocketMQ.

KafkaMicroservicesRabbitMQ
0 likes · 25 min read
10 Message Queue Patterns for Spring Boot: Decoupling, Async, Traffic Shaping & More
LuTiao Programming
LuTiao Programming
Sep 3, 2026 · Backend Development

Kafka 4.2 Share Groups: Can Kafka Replace RabbitMQ for Task Queues?

The author tests Kafka 4.2 Share Groups (Kafka Queues) by migrating a report-generation task queue from RabbitMQ, showing how Share Groups enable elastic consumer scaling beyond partition limits, manual acknowledgment modes (ACCEPT/RELEASE/REJECT/RENEW), and long-task handling with renew(), while noting RabbitMQ's routing strengths remain relevant.

KafkaKafka QueuesRabbitMQ
0 likes · 16 min read
Kafka 4.2 Share Groups: Can Kafka Replace RabbitMQ for Task Queues?
Data Integration and Governance
Data Integration and Governance
Sep 1, 2026 · Big Data

Building a Real-Time Data Warehouse: End-to-End Pipeline from Kafka to BI

This article details the end-to-end architecture of a real-time data warehouse, covering data ingestion via CDC and Kafka, stream processing challenges like deduplication and event-time handling, layered modeling (ODS/DWD/DWS/ADS), unified metric definitions, and critical production monitoring for latency, lag, data reconciliation, and consistency with offline results.

CDCFlinkKafka
0 likes · 20 min read
Building a Real-Time Data Warehouse: End-to-End Pipeline from Kafka to BI
Niu Liu
Niu Liu
Aug 31, 2026 · R&D Management

Enterprise Codex Governance: Trustworthy Collection, Secure Access, Auditable Lifecycle

This article details the architecture of an enterprise governance platform for Codex AI coding assistants, covering local encrypted collection with 90ms hooks, server-side parsing with tenant isolation, tiered content access (L0–L3) requiring multi-party approval, and operational use cases from deployment health monitoring to employee offboarding with zero-residue verification.

AI coding assistantCodexDesensitization
0 likes · 17 min read
Enterprise Codex Governance: Trustworthy Collection, Secure Access, Auditable Lifecycle
Alibaba Cloud Native
Alibaba Cloud Native
Aug 28, 2026 · Industry Insights

When Model Power Is Plenty, Real‑Time Data Pipelines Become the AI Production Bottleneck

As AI models become sufficiently capable, the primary obstacle to production shifts from model selection to the real‑time data link, requiring new load handling, sub‑second context availability, and tighter trust boundaries, prompting a redesign of streaming platforms, consumption models, and security mechanisms.

AICloud NativeEvent-driven AI
0 likes · 16 min read
When Model Power Is Plenty, Real‑Time Data Pipelines Become the AI Production Bottleneck
Woodpecker Software Testing
Woodpecker Software Testing
Aug 28, 2026 · Backend Development

Mastering Concurrent User Testing: Deep Dive into Performance Optimization

The article explains that true concurrent user testing must emulate realistic user sessions with think time and asynchronous actions, identifies three typical bottlenecks—application thread blocking, database connection pool exhaustion, and middleware resource contention—and demonstrates proactive performance‑left‑shift practices and emerging AI‑driven adaptive testing techniques.

AIConcurrent TestingHikariCP
0 likes · 7 min read
Mastering Concurrent User Testing: Deep Dive into Performance Optimization
Cloud Architecture
Cloud Architecture
Aug 27, 2026 · Backend Development

Reliable Delivery for Billion-Scale LongConn Gateways: Tackling ACK Storms

The article explains why a successful Netty write does not guarantee message delivery, and presents a production‑grade solution for billion‑scale long‑connection gateways that combines application‑level ACKs, timeout‑driven retries, offline compensation, state‑machine design, back‑pressure handling, and observability to prevent ACK storms and retry avalanches.

ACK StormBackend DevelopmentKafka
0 likes · 29 min read
Reliable Delivery for Billion-Scale LongConn Gateways: Tackling ACK Storms
Ray's Galactic Tech
Ray's Galactic Tech
Aug 26, 2026 · Big Data

Layered Real‑Time Data Warehouse with Flink CDC, Kafka & Doris

The article explains why real‑time data‑warehouse projects often fail in production and presents a complete, production‑ready solution that layers ODS, DWD, DWS and ADS using Flink CDC to capture MySQL changes, Kafka for buffering and replay, and Doris for OLAP storage, with detailed guidance on architecture, state handling, fault‑tolerance and operations.

CDCDorisFlink
0 likes · 35 min read
Layered Real‑Time Data Warehouse with Flink CDC, Kafka & Doris
Architect Chen
Architect Chen
Aug 21, 2026 · Operations

All Kafka Development Commands Explained – 2026 Edition

This guide walks through the essential Kafka command‑line tools for creating, listing, describing, and altering topics, producing and consuming messages, and managing consumer groups, providing concrete examples and key parameters for each operation.

ConsumerConsumer GroupKafka
0 likes · 5 min read
All Kafka Development Commands Explained – 2026 Edition
Coder Trainee
Coder Trainee
Aug 20, 2026 · Backend Development

How I Resolved a 1‑Million‑Message Kafka Consumer Backlog

When a Kafka topic’s consumer lag suddenly spiked to over one million messages, the author traced the issue to improper acknowledgment handling, implemented emergency scaling, corrected exception processing, added a dead‑letter queue, and created a reusable checklist to prevent future backlogs.

AcknowledgmentConsumer LagKafka
0 likes · 7 min read
How I Resolved a 1‑Million‑Message Kafka Consumer Backlog
Ray's Galactic Tech
Ray's Galactic Tech
Aug 19, 2026 · Backend Development

Designing a 12‑State Payment System State Machine from Pending to Completed

This article presents a complete design for a 12‑state payment order state machine that handles high‑concurrency scenarios such as payment callbacks arriving after an order has been cancelled, using explicit state transitions, database CAS updates, Redis + DB idempotency, Outbox pattern and Kafka‑driven asynchronous processing to achieve reliable, auditable and compensatable order lifecycle management.

KafkaMySQLOutbox
0 likes · 39 min read
Designing a 12‑State Payment System State Machine from Pending to Completed
Cloud Architecture
Cloud Architecture
Aug 18, 2026 · Backend Development

Scalable Enterprise Real‑Time Push with Spring Boot, WebFlux, Kafka & Redis

This guide walks through why traditional polling or WebSocket solutions quickly break at scale, explains the Server‑Sent Events protocol, and presents a production‑grade architecture that combines Spring Boot 3, WebFlux, Kafka, Redis, and Kubernetes to deliver reliable, ordered, and observable one‑way push notifications for millions of concurrent users.

KafkaRedisSSE
0 likes · 43 min read
Scalable Enterprise Real‑Time Push with Spring Boot, WebFlux, Kafka & Redis
Ray's Galactic Tech
Ray's Galactic Tech
Aug 14, 2026 · Backend Development

Payment Callback After Order Cancellation? 3 Consistency Challenges and Practical Engineering Solutions

The article examines why payment callbacks that arrive later than order cancellations cause money‑in‑order‑canceled anomalies, outlines the three core eventual‑consistency problems—message ordering, duplicate consumption, and message loss—and presents a complete engineering solution using business idempotency keys, a transactional outbox, state‑machine handling, and reconciliation compensation.

Kafkadistributed systemseventual consistency
0 likes · 35 min read
Payment Callback After Order Cancellation? 3 Consistency Challenges and Practical Engineering Solutions
Ray's Galactic Tech
Ray's Galactic Tech
Aug 12, 2026 · Backend Development

Message Queue Showdown: When to Use Kafka, RabbitMQ, or Pulsar

This article dissects common MQ mis‑selections, categorises four message types, explains the five architectural roles of a message broker, compares Kafka, RabbitMQ and Pulsar on capabilities and trade‑offs, and provides a step‑by‑step selection guide with real‑world code snippets and best‑practice patterns.

KafkaOutbox PatternPulsar
0 likes · 26 min read
Message Queue Showdown: When to Use Kafka, RabbitMQ, or Pulsar
Coder Life Journal
Coder Life Journal
Aug 11, 2026 · Backend Development

Spring Cloud + Kafka: 6 Common Pitfalls and How to Avoid Them

The article walks through six real‑world pitfalls when integrating Spring Cloud with Kafka—message loss, duplicate processing, out‑of‑order events, massive consumer lag, serialization mismatches, and misuse of Kafka transactions—and provides concrete configuration tweaks, code examples, and operational safeguards to prevent each issue.

KafkaSpring Clouddistributed systems
0 likes · 9 min read
Spring Cloud + Kafka: 6 Common Pitfalls and How to Avoid Them
ITPUB
ITPUB
Aug 9, 2026 · Big Data

Why a CK+Kafka+Filebeat Stack Beats ELK for Log Analytics

The article compares Elasticsearch and ClickHouse for log storage, analyzes cost and performance, and provides a step‑by‑step guide to deploying a Zookeeper‑Kafka‑Filebeat‑ClickHouse pipeline, including common pitfalls and their solutions, showing how the combo reduces server costs by up to half.

ClickHouseElasticsearchFilebeat
0 likes · 15 min read
Why a CK+Kafka+Filebeat Stack Beats ELK for Log Analytics
Java Tech Enthusiast
Java Tech Enthusiast
Aug 8, 2026 · Backend Development

Why Does Kafka Re‑Consume Tens of Thousands of Messages After a Successful Offset Commit?

The article explains why Kafka consumers can reprocess tens of thousands of messages after a restart despite successful offset commits, detailing the roles of automatic commit timing, rebalance triggers, GC pauses, and the limitations of manual commits, and offers practical configuration and idempotent processing solutions.

ConsumerJavaKafka
0 likes · 8 min read
Why Does Kafka Re‑Consume Tens of Thousands of Messages After a Successful Offset Commit?
Cloud Architecture
Cloud Architecture
Aug 7, 2026 · Backend Development

Building a Trillion‑Message Queue: Kafka‑Level Architecture and Implementation

This article explains why and how to build a Kafka‑grade message‑queue kernel from scratch, detailing functional and non‑functional goals, core design principles, storage layout, replication, consumer‑group coordination, performance optimizations, deployment on Kubernetes, and a step‑by‑step roadmap to production‑grade reliability.

Distributed LogJavaKafka
0 likes · 40 min read
Building a Trillion‑Message Queue: Kafka‑Level Architecture and Implementation
Cloud Architecture
Cloud Architecture
Aug 6, 2026 · Big Data

Exporting 10 Billion Elasticsearch Records: From Simple Script to Enterprise Offline Platform

The article analyses why exporting billions of Elasticsearch documents requires a full‑stack platform rather than a one‑off script, detailing the pitfalls of naive pagination, the benefits of PIT + search_after + slicing, and a complete architecture with Kafka, Redis, MySQL, Kubernetes and observability for reliable, scalable offline data export.

Data ExportElasticsearchJava
0 likes · 40 min read
Exporting 10 Billion Elasticsearch Records: From Simple Script to Enterprise Offline Platform
Code Farming
Code Farming
Aug 4, 2026 · Backend Development

Why 50,000 Simultaneous Registrations Crashed the System—and How Isolation Prevents It

When an education company faced 50,000 concurrent re‑registration requests, its monolithic internal‑external architecture collapsed, but by physically separating networks, enforcing gateway rate‑limiting and circuit‑breaking, pre‑loading data into Redis, and using one‑way Kafka streams, the system remained stable.

Circuit BreakingKafkaRedis
0 likes · 6 min read
Why 50,000 Simultaneous Registrations Crashed the System—and How Isolation Prevents It
YiSu Grain
YiSu Grain
Jul 28, 2026 · Big Data

Day 39: Big Data Architecture – Distributed Storage, Batch vs Stream Processing, and Compute‑Storage Separation

This lesson explains why a single database cannot scale for massive e‑commerce data, introduces the three core pillars of distributed storage—sharding, replication, and horizontal scaling—covers batch and stream processing differences with Spark, Hive, Flink and Storm, compares compute‑storage integration versus separation, and shows how Kafka, HDFS, Spark and Flink fit together in a real‑time data platform.

Batch ProcessingCompute-Storage SeparationFlink
0 likes · 25 min read
Day 39: Big Data Architecture – Distributed Storage, Batch vs Stream Processing, and Compute‑Storage Separation
Smart Sea Tide
Smart Sea Tide
Jul 27, 2026 · Big Data

Comprehensive Overview of Big Data Open‑Source Frameworks

This article provides a detailed, structured survey of the most widely used open‑source big‑data technologies—including Hadoop ecosystems, storage systems, processing engines, query tools, data ingestion, exchange, messaging, scheduling, governance, visualization, mining, and cloud platforms—highlighting each project's origins, core features, typical use cases, and notable strengths or limitations to aid technology selection and system design.

FlinkHBaseHadoop
0 likes · 59 min read
Comprehensive Overview of Big Data Open‑Source Frameworks
Ray's Galactic Tech
Ray's Galactic Tech
Jul 17, 2026 · Backend Development

Three Critical Guarantees for Message Queues: No Loss, No Duplicates, No Disorder – Deep Dive into Production‑Grade Solutions

This article dissects why modern systems must enforce three reliability guarantees—no message loss, no duplicate processing, and no out‑of‑order delivery—by examining real‑world order flows, outbox patterns, idempotent keys, partitioning strategies, consumer acknowledgments, and operational safeguards such as replay, dead‑letter handling, and monitoring.

KafkaOutbox Patterndistributed systems
0 likes · 28 min read
Three Critical Guarantees for Message Queues: No Loss, No Duplicates, No Disorder – Deep Dive into Production‑Grade Solutions
Ray's Galactic Tech
Ray's Galactic Tech
Jul 16, 2026 · Backend Development

Building a Scalable Smart Tag System for 100k QPS with AI‑Generated Code

The article explains that while many teams focus on model accuracy, the real challenges of an AI‑powered tagging system are write spikes, timeouts, duplicate tagging, cost overruns and state inconsistency, and it proposes a five‑layer architecture, async fallback, governance and validation practices to reliably achieve a 100 k QPS target.

AI taggingKafkaLLM Integration
0 likes · 30 min read
Building a Scalable Smart Tag System for 100k QPS with AI‑Generated Code
Ray's Galactic Tech
Ray's Galactic Tech
Jul 16, 2026 · Artificial Intelligence

K8s, Kafka, Nacos Agent Platform to Prevent Token Bankruptcy and Skill Avalanches

The article details how a production‑grade Agent platform built on Kubernetes, Kafka, and Nacos addresses token budget overruns, uncontrolled skill execution, and RAG hallucinations by introducing a four‑layer runtime architecture, token pre‑allocation, explicit state management, dynamic governance policies, and robust skill specifications.

KafkaKubernetesLLM Agents
0 likes · 32 min read
K8s, Kafka, Nacos Agent Platform to Prevent Token Bankruptcy and Skill Avalanches
Xiaolin Talks Programming
Xiaolin Talks Programming
Jul 16, 2026 · Backend Development

CDC + Outbox Pattern: Production-Grade Cross-Database Sync with Spring Boot & Kafka

This article details a production-ready cross-database synchronization architecture using CDC and Outbox pattern with Spring Boot, Debezium, and Kafka, covering schema design, connector configuration, transaction boundaries, ordering guarantees, idempotent consumption, dead-letter queues, and monitoring strategies.

CDCCross-Database SyncDebezium
0 likes · 20 min read
CDC + Outbox Pattern: Production-Grade Cross-Database Sync with Spring Boot & Kafka
Ray's Galactic Tech
Ray's Galactic Tech
Jul 14, 2026 · Cloud Native

Spring Boot + Netty MQTT Gateway for Million Connections & Millisecond Push

To support millions of persistent MQTT connections with sub‑millisecond latency, the article walks through a Spring Boot + Netty cloud‑native gateway design that separates connection, event, state and governance planes, details async authentication, back‑pressure handling, command state machines, and loss‑less Kubernetes roll‑outs.

Cloud NativeKafkaKubernetes
0 likes · 38 min read
Spring Boot + Netty MQTT Gateway for Million Connections & Millisecond Push
dbaplus Community
dbaplus Community
Jul 13, 2026 · Interview Experience

How to Process a 10 GB CSV File with Only 512 MB RAM?

The article walks through an interview scenario where a candidate must handle a 10 GB CSV using just 512 MB of memory, critiques naive answers, and presents a step‑by‑step streaming, chunked, and checkpoint‑based solution with optional Kafka/Flink considerations.

CSVFlinkJava
0 likes · 8 min read
How to Process a 10 GB CSV File with Only 512 MB RAM?
Xiaolin Talks Programming
Xiaolin Talks Programming
Jul 13, 2026 · Backend Development

CQRS & Event Sourcing in Spring Boot: Read/Write Separation, Snapshots & Compensation

This guide details implementing CQRS and Event Sourcing in Spring Boot, covering command/query separation, event bus choices (Outbox pattern, Kafka), projection idempotency, state snapshots for performance, Saga-based compensation for distributed transactions, and practical tuning tips for production workloads.

CQRSEvent BusEvent Sourcing
0 likes · 14 min read
CQRS & Event Sourcing in Spring Boot: Read/Write Separation, Snapshots & Compensation
Niu Liu
Niu Liu
Jul 13, 2026 · Artificial Intelligence

Building a Real‑Time Recommendation Engine with Flink: A Complete Example Project

The article walks through constructing a full‑stack real‑time recommendation system—from user‑behavior collection via Kafka, through Flink streaming jobs for hot‑list, user and item profiling, to storage in Redis, HBase and Elasticsearch, and finally a React/Ant Design console that visualizes the pipeline and enables debugging.

ElasticsearchFlinkHBase
0 likes · 12 min read
Building a Real‑Time Recommendation Engine with Flink: A Complete Example Project
TechVision Expert Circle
TechVision Expert Circle
Jul 12, 2026 · Backend Development

Designing a Millisecond‑Level Real‑Time Data Processing System

The article outlines practical engineering steps to build a sub‑200 ms end‑to‑end real‑time data pipeline, covering latency budgeting, data ingestion with gRPC/Kafka, stream engine selection (Flink 2.x, RisingWave), state backend tuning, output sink choices, and monitoring with backpressure and OpenTelemetry.

Apache FlinkKafkaOpenTelemetry
0 likes · 13 min read
Designing a Millisecond‑Level Real‑Time Data Processing System
Cloud Architecture
Cloud Architecture
Jul 9, 2026 · Backend Development

How Spring Boot, Kafka, Redis, and MongoDB Power Real‑Time GPS Tracking for Millions of Vehicles

This article walks through a production‑grade architecture that uses Spring Boot, Kafka, Redis, and MongoDB to ingest, buffer, order, and store high‑frequency vehicle GPS data from hundreds of thousands of devices while guaranteeing low latency, scalability, fault‑tolerance, and accurate replay capabilities.

KafkaMicroservicesMongoDB
0 likes · 38 min read
How Spring Boot, Kafka, Redis, and MongoDB Power Real‑Time GPS Tracking for Millions of Vehicles
YiSu Grain
YiSu Grain
Jul 7, 2026 · Backend Development

Stop Memorizing Isolated Terms: Visualize a User Request Flow in a Microservice Architecture

This article walks through a complete e‑commerce request path, illustrating how an API gateway, load balancer, service registry, Redis cache, Kafka queue and the database cooperate, and explains where CAP/BASE, distributed transaction patterns and service‑governance mechanisms fit in the overall diagram.

API GatewayCAP theoremDistributed Transactions
0 likes · 13 min read
Stop Memorizing Isolated Terms: Visualize a User Request Flow in a Microservice Architecture
YiSu Grain
YiSu Grain
Jul 7, 2026 · Fundamentals

Why You Don't Need to Send an SMS Immediately After an Order – Understanding Kafka and Message Queues

The article explains how using a message queue such as Kafka lets an order service return quickly by handling non‑critical tasks like SMS, points, and logging asynchronously, thereby improving latency, decoupling services, and smoothing traffic spikes while also covering Kafka's core concepts and trade‑offs.

KafkaPeak Shavingasynchronous-processing
0 likes · 11 min read
Why You Don't Need to Send an SMS Immediately After an Order – Understanding Kafka and Message Queues
Su San Talks Tech
Su San Talks Tech
Jul 7, 2026 · Backend Development

How to Choose Among Four Popular Message Queues?

This article compares Kafka, RocketMQ, RabbitMQ, and ActiveMQ, explaining their architectures, core concepts, strengths and weaknesses, and provides practical guidance on selecting the most suitable queue based on throughput, reliability, scalability, and use‑case requirements.

Backend DevelopmentKafkaRabbitMQ
0 likes · 20 min read
How to Choose Among Four Popular Message Queues?
Architect Chen
Architect Chen
Jul 6, 2026 · Big Data

Comprehensive Guide to Kafka Core Commands (2026 Edition)

This article walks through the essential Kafka command‑line tools, covering how to check the Kafka version, create, list, describe, alter, and delete topics, manage partitions, produce and consume messages, inspect consumer groups, reset offsets, query broker metadata, view log directories, and run built‑in performance tests for both producers and consumers.

Consumer GroupKafkaTopic Management
0 likes · 6 min read
Comprehensive Guide to Kafka Core Commands (2026 Edition)
Lobster Programming
Lobster Programming
Jul 6, 2026 · Backend Development

Why Kafka Achieves Such High Throughput

Kafka’s exceptionally high throughput stems from a combination of design choices—including sequential disk writes, zero‑copy data transfer, batch processing, partition‑level parallelism, OS page‑cache utilization, and producer‑side compression—that together minimize I/O overhead, CPU usage, and latency.

CompressionKafkaPartitioning
0 likes · 6 min read
Why Kafka Achieves Such High Throughput
Java Architect Handbook
Java Architect Handbook
Jul 4, 2026 · Backend Development

How to Build a Clean SpringBoot Scaffold Quickly

The article walks through creating a SpringBoot project from scratch, covering IDE setup pain points, project initialization, version compatibility between SpringBoot, Spring Cloud and Kafka, Maven dependency management, core scaffold classes such as global exception handling, logging aspect, CORS and Swagger configuration, and recommends useful tools like embedded Redis, MariaDB, Hutool, MyBatis‑Plus, MapStruct and Redisson.

KafkaMapStructMaven
0 likes · 11 min read
How to Build a Clean SpringBoot Scaffold Quickly
Random Bulletin
Random Bulletin
Jul 1, 2026 · Operations

Evolving Message Expiration for 10 Million QPS: From No TTL to Smart Policies

When a high‑traffic system processes billions of messages per day, stale “zombie” messages can corrupt business state; this article walks through the evolution from never‑expiring queues to uniform TTL, then per‑topic and per‑message TTL, and finally to smart, context‑aware expiration, detailing the engineering trade‑offs, implementation patterns, and operational checklist needed for reliable 10 M‑QPS message pipelines.

KafkaRocketMQdistributed systems
0 likes · 19 min read
Evolving Message Expiration for 10 Million QPS: From No TTL to Smart Policies
Java Baker
Java Baker
Jun 29, 2026 · Backend Development

How to Diagnose Uneven CPU Usage in Java Services Using Kafka

This article walks through the symptoms, root cause analysis, and step‑by‑step solutions for uneven CPU usage across Java service instances, highlighting how mismatched Kafka partition counts and thread or GC issues can lead to load imbalance and how to resolve them.

CPUJavaKafka
0 likes · 8 min read
How to Diagnose Uneven CPU Usage in Java Services Using Kafka
Cloud Architecture
Cloud Architecture
Jun 28, 2026 · Backend Development

Building a Production‑Grade Kafka Reverse Proxy with Nginx + Lua for Scalable, Auditable Message Access

The article explains why a simple TCP proxy is insufficient for Kafka, and presents a production‑ready reverse‑proxy architecture built on Nginx stream and Lua that adds metadata rewriting, multi‑tenant quota, TLS termination, dynamic routing, observability, and graceful degradation to achieve scalable, auditable message ingestion.

KafkaLuaMessage Governance
0 likes · 33 min read
Building a Production‑Grade Kafka Reverse Proxy with Nginx + Lua for Scalable, Auditable Message Access
Random Bulletin
Random Bulletin
Jun 28, 2026 · Operations

Backlog Monitoring at 10 Million QPS: From Manual Checks to SLO‑Driven Automation

At 10 million QPS, message backlog can silently grow for hours, turning minutes of lag into hundreds of millions of undelivered messages; this article walks through a six‑stage evolution—from manual command‑line checks to SLO‑driven predictive monitoring—detailing metrics, pitfalls, and tool choices for a robust, multi‑dimensional alert system.

AlertingKafkaSLO
0 likes · 21 min read
Backlog Monitoring at 10 Million QPS: From Manual Checks to SLO‑Driven Automation
Cloud Architecture
Cloud Architecture
Jun 27, 2026 · Backend Development

Enterprise Messaging System Deep Dive: Core Engine to High‑Concurrency

This guide explains how enterprise messaging systems reliably propagate state changes across distributed services, covering core concepts, outbox patterns, topic/tag modeling, consumer idempotency, dead‑letter handling, high‑concurrency engineering, monitoring, and Kubernetes deployment to build a production‑grade, observable platform.

KafkaOutboxevent-driven
0 likes · 44 min read
Enterprise Messaging System Deep Dive: Core Engine to High‑Concurrency
DeepNoMind
DeepNoMind
Jun 27, 2026 · Backend Development

Designing a Production‑Grade Distributed Logging and Metrics Platform

This article presents an end‑to‑end design of a production‑grade observability platform that ingests millions of real‑time logs, metrics, and events, detailing functional and non‑functional requirements, capacity planning, component choices such as Kafka, Flink, Elasticsearch, object‑storage data lakes, and the trade‑offs involved.

Data LakeElasticsearchFlink
0 likes · 21 min read
Designing a Production‑Grade Distributed Logging and Metrics Platform
Alibaba Cloud Native
Alibaba Cloud Native
Jun 26, 2026 · Cloud Native

One-Click Real-Time Stream Ingestion: Alibaba Cloud Kafka’s Native Data Lake Integration

Alibaba Cloud Message Queue for Kafka introduces a native message‑to‑lake capability that integrates Apache Iceberg with OSS Table Bucket, eliminating Spark/Flink/Kafka Connect, providing exactly‑once semantics, automatic schema management, dual write modes, smart partitioning, and up to ten‑fold performance gains across diverse real‑time analytics scenarios.

Apache IcebergCloud NativeData Lake
0 likes · 12 min read
One-Click Real-Time Stream Ingestion: Alibaba Cloud Kafka’s Native Data Lake Integration
Random Bulletin
Random Bulletin
Jun 24, 2026 · Operations

Rack Awareness: From Zero to High‑QPS – Boost Availability, Cut Cross‑AZ Traffic

A real‑world rack‑power outage showed that three‑replica Kafka clusters can lose all replicas when brokers share a failure domain, prompting a deep dive into rack awareness—how fault‑domain tags are injected, replica‑placement and leader‑distribution algorithms, consumer‑proximity reads, bandwidth costs, failure scenarios, and the stepwise evolution from hundred‑thousand to ten‑million QPS deployments.

KafkaPulsarRack Awareness
0 likes · 26 min read
Rack Awareness: From Zero to High‑QPS – Boost Availability, Cut Cross‑AZ Traffic
Random Bulletin
Random Bulletin
Jun 23, 2026 · Backend Development

Choosing the Right Replication Factor for 10M QPS Message Queues: From Dual to Multi‑Replica

The article walks through why a single replica is insufficient for million‑scale message queues, explains the latency and data‑loss trade‑offs of dual‑replica sync and async modes, shows how three‑replica majority voting becomes the sweet spot, and then details the engineering considerations for scaling to five or more replicas across availability zones and regions.

AZ awarenessKafkaPulsar
0 likes · 18 min read
Choosing the Right Replication Factor for 10M QPS Message Queues: From Dual to Multi‑Replica
Programmer XiaoFu
Programmer XiaoFu
Jun 23, 2026 · Backend Development

Why Kafka Still Delivers Out‑of‑Order Messages Even When Using the Same Key

Even though Kafka guarantees that messages with the same key land in the same partition, the article explains how producer retries, multithreaded consumer processing, and partition expansion can break ordering, and provides concrete techniques such as idempotent producers and single‑threaded consumption to preserve order.

Consumer ConcurrencyKafkaPartition Expansion
0 likes · 10 min read
Why Kafka Still Delivers Out‑of‑Order Messages Even When Using the Same Key
ZhiKe AI
ZhiKe AI
Jun 22, 2026 · Fundamentals

Message Queues: Power When Correct, Disaster When Wrong – 3 Scenarios & Tips

The article explains how message queues can dramatically improve response time, decouple services, and smooth traffic spikes, outlines seven advantages and eight drawbacks, and provides concrete guidelines on when to adopt them, how to prevent loss, duplication, and ordering issues, and how to ensure end‑to‑end reliability.

KafkaRabbitMQSystem Decoupling
0 likes · 15 min read
Message Queues: Power When Correct, Disaster When Wrong – 3 Scenarios & Tips
Random Bulletin
Random Bulletin
Jun 20, 2026 · Backend Development

Message Ordering at 10M+ QPS: From Global to Partition to Business-Key

The article explains that message ordering is a spectrum—from strict global ordering to partition-level and finally business-key ordering—and examines the trade‑offs, pitfalls, and engineering techniques needed to maintain order while scaling to tens of millions of QPS.

KafkaPartitioningbusiness key
0 likes · 24 min read
Message Ordering at 10M+ QPS: From Global to Partition to Business-Key
Niu Liu
Niu Liu
Jun 19, 2026 · Big Data

Building a Real‑Time Risk Control Engine with Flink 2.2.1, CEP, and Aviator

This article details a real‑time risk control system for e‑commerce and finance built on Apache Flink 2.2.1 and CEP, featuring a dynamic Aviator rule engine, three Kafka event streams, multi‑channel output to Redis, MySQL and Kafka, a Spring Boot‑React management UI, and step‑by‑step deployment instructions.

AviatorCEPFlink
0 likes · 11 min read
Building a Real‑Time Risk Control Engine with Flink 2.2.1, CEP, and Aviator
Architect Chen
Architect Chen
Jun 19, 2026 · Operations

Comprehensive Guide to Kafka Commands (2026 Edition)

This article provides a step‑by‑step reference for Kafka command‑line tools, covering version checks, topic creation, listing, describing, deletion, partition alteration, consumer‑group inspection, offset resets, producer/consumer testing, broker API version queries, cluster metadata, leader election, and log‑directory monitoring, with concrete examples and expected outputs.

KafkaTopic Managementbroker monitoring
0 likes · 6 min read
Comprehensive Guide to Kafka Commands (2026 Edition)
Alibaba Cloud Native
Alibaba Cloud Native
Jun 19, 2026 · Big Data

Why Real-Time Data Lake Ingestion Is Dropping ETL in the AI Era: Architecture Simplification from Kafka to Iceberg

In the AI‑driven era, enterprises need a data foundation that supports both real‑time consumption and long‑term historical analysis, and the emerging "zero‑ETL" trend moves generic ingestion capabilities from external Flink/Spark jobs into a streamlined Kafka‑to‑Iceberg pipeline, reducing complexity while preserving low latency, consistency, schema evolution, CDC semantics and open‑ecosystem compatibility.

Data LakeKafkaStreaming
0 likes · 25 min read
Why Real-Time Data Lake Ingestion Is Dropping ETL in the AI Era: Architecture Simplification from Kafka to Iceberg
Alibaba Cloud Developer
Alibaba Cloud Developer
Jun 18, 2026 · Big Data

How AI-Driven Real-Time Data Lakes Are Ditching ETL: A Kafka‑to‑Iceberg Architecture Simplification

In the AI era, enterprises need a data foundation that supports both low‑latency streaming and long‑term analytics, and the combination of Kafka, Iceberg and object storage is emerging as a preferred solution; by moving ingestion capabilities closer to the message layer and eliminating external ETL jobs, a "zero‑ETL" approach reduces architectural complexity, improves consistency, and streamlines schema evolution and small‑file management.

CDCData LakeKafka
0 likes · 27 min read
How AI-Driven Real-Time Data Lakes Are Ditching ETL: A Kafka‑to‑Iceberg Architecture Simplification
Random Bulletin
Random Bulletin
Jun 17, 2026 · Backend Development

Rebalance Optimization for Million‑QPS Kafka Clusters: From Default to Custom Assignors

At a 5,000‑instance, 20,000‑partition Kafka consumer cluster, the default eager Rebalance caused a 90‑second stop‑the‑world pause, illustrating how a single GC‑induced heartbeat miss can flood brokers with tens of millions of messages; the article dissects eager vs cooperative protocols, built‑in assignor limits, and four custom assignor patterns to scale Rebalance from 100 K to tens of millions of QPS.

AssignorCooperative ProtocolCustom Scheduler
0 likes · 28 min read
Rebalance Optimization for Million‑QPS Kafka Clusters: From Default to Custom Assignors
Random Bulletin
Random Bulletin
Jun 16, 2026 · Backend Development

Consumer Rate Limiting: From Zero to Full Control in Million‑QPS Architectures

The article explains why consumer‑side rate limiting is essential in million‑QPS systems, detailing how unchecked consumers can overwhelm downstream services, and presents practical strategies—including pause/resume, token‑bucket algorithms, adaptive thresholds, and global coordination—to safely throttle consumption without dropping messages.

Kafkaconsumer optimizationdistributed systems
0 likes · 16 min read
Consumer Rate Limiting: From Zero to Full Control in Million‑QPS Architectures
Random Bulletin
Random Bulletin
Jun 15, 2026 · Backend Development

From Auto to Manual: Mastering Consumer Offsets for Ten‑Million QPS Systems

The article examines a real payment loss incident caused by Kafka's default automatic offset commit, explains why automatic commits become a hidden trap at high traffic, and provides a step‑by‑step guide to switching to manual commits with async‑first, sync‑fallback, idempotency, dead‑letter handling, batch strategies, and concurrency controls for reliable ten‑million QPS consumption.

Kafkadead-letter queuehigh throughput
0 likes · 15 min read
From Auto to Manual: Mastering Consumer Offsets for Ten‑Million QPS Systems
Cloud Architecture
Cloud Architecture
Jun 15, 2026 · Backend Development

Trillion‑Message Engine Showdown: RabbitMQ vs Kafka Architecture, Performance and Cloud‑Native Pitfalls

An experienced architect compares RabbitMQ and Kafka across core protocols, storage, replication, consumption semantics, and real‑world production designs, offering Java 17/Spring Boot code, cloud‑native deployment tips, observability, and a decision framework that matches messaging patterns to business requirements.

Cloud NativeEvent StreamingJava
0 likes · 45 min read
Trillion‑Message Engine Showdown: RabbitMQ vs Kafka Architecture, Performance and Cloud‑Native Pitfalls
Random Bulletin
Random Bulletin
Jun 14, 2026 · Backend Development

Scaling to Millions of QPS: How Batch Consumption Beats Single-Message Processing

The article explains why single‑message consumption stalls under high QPS due to fixed per‑message overhead, and how merging pull, processing, and commit into batch operations dramatically boosts throughput while introducing trade‑offs in latency, memory, and failure handling, with practical guidelines for batch size selection and dynamic tuning.

KafkaPerformance Optimizationbatch consumption
0 likes · 16 min read
Scaling to Millions of QPS: How Batch Consumption Beats Single-Message Processing
Linyb Geek Road
Linyb Geek Road
Jun 14, 2026 · Big Data

How to Solve Data Ordering Issues in Apache Kafka

This article explains how Kafka maintains order within partitions using keys and offsets, why ordering across partitions can break, and how to preserve strict sequencing through producer configuration, idempotent producers, and exactly‑once transactional processing.

Exactly-OnceKafkaTransactions
0 likes · 9 min read
How to Solve Data Ordering Issues in Apache Kafka
Random Bulletin
Random Bulletin
Jun 12, 2026 · Backend Development

From Random to Intelligent: Partition Selection Strategies for 10 Million QPS Systems

The article walks through four generations of partition‑selection strategies—from naive random and round‑robin to key‑hash, sticky, and finally intelligent load‑aware routing—explaining how hotspot keys, batch inefficiency, consumer skew, and fault amplification threaten stability at 10 M QPS and offering concrete engineering actions to design, test, monitor, and switch strategies in large‑scale message‑queue deployments.

KafkaPartitioninghigh‑throughput
0 likes · 20 min read
From Random to Intelligent: Partition Selection Strategies for 10 Million QPS Systems
Random Bulletin
Random Bulletin
Jun 11, 2026 · Backend Development

From Fixed to Exponential Backoff: Preventing Producer Stampedes in 10 Million QPS Kafka Clusters

The article explains why fixed‑interval retries cause thousands of producers to synchronize their retries and overload brokers, and how exponential backoff, full jitter, retry budgeting, and differentiated policies per topic can safely handle send failures even at tens of millions of QPS.

ExponentialBackoffHighThroughputIdempotence
0 likes · 26 min read
From Fixed to Exponential Backoff: Preventing Producer Stampedes in 10 Million QPS Kafka Clusters
Random Bulletin
Random Bulletin
Jun 10, 2026 · Backend Development

Message Compression at Scale: From No Compression to Selective Strategies

A midnight bandwidth alarm triggers a deep dive into message compression, revealing how algorithm choice, compression placement, batch handling, and selective policies evolve from simple no‑compression setups to multi‑layered strategies that balance CPU, bandwidth, disk costs, and latency in million‑QPS systems.

KafkaZstdbatch compression
0 likes · 20 min read
Message Compression at Scale: From No Compression to Selective Strategies
Random Bulletin
Random Bulletin
Jun 9, 2026 · Backend Development

From Synchronous to Asynchronous Sending: Achieving 10 Million QPS with Kafka Producer

The article analyses how synchronous Kafka producer calls block business threads on ACK, limiting throughput, and shows that moving to asynchronous sending with the Accumulator decouples submission from confirmation, unlocking batch, compression, and back‑pressure mechanisms that enable stable operation at tens of millions of QPS.

Kafkaasynchronousbackpressure
0 likes · 21 min read
From Synchronous to Asynchronous Sending: Achieving 10 Million QPS with Kafka Producer
Architect Chen
Architect Chen
Jun 9, 2026 · Big Data

How Kafka Prevents Duplicate Consumption: Three Main Solutions

The article explains why Kafka does not guarantee exactly‑once delivery and presents three practical approaches—business‑level idempotence, manual offset management, and Kafka’s transaction/EOS features—to reliably avoid duplicate message processing.

ConsumerExactly-OnceIdempotence
0 likes · 4 min read
How Kafka Prevents Duplicate Consumption: Three Main Solutions
Java Tech Enthusiast
Java Tech Enthusiast
Jun 6, 2026 · Backend Development

Why Kafka Marks a Live Consumer as Dead and Forces Rebalance

Even when a consumer process runs and logs normally, Kafka may deem it dead and trigger a rebalance because the poll interval exceeds max.poll.interval.ms, a situation known as ‘false dead’; this article explains the root cause and practical ways to prevent it.

ConsumerKafkaRebalance
0 likes · 8 min read
Why Kafka Marks a Live Consumer as Dead and Forces Rebalance
Architectural Methodology
Architectural Methodology
Jun 6, 2026 · Interview Experience

How to Plan a Phased Migration and Ensure Data Consistency for Legacy System Refactoring

The article outlines a step‑by‑step architecture redesign for a 20‑year‑old monolithic production system, detailing problem analysis, a phased microservice migration using the "killer" pattern, data‑sync strategies, distributed transaction handling, observability, rollback mechanisms, and practical interview‑style insights.

CanalCloud NativeDDD
0 likes · 15 min read
How to Plan a Phased Migration and Ensure Data Consistency for Legacy System Refactoring
Cloud Architecture
Cloud Architecture
Jun 4, 2026 · Backend Development

Kafka Backlog Mastery: Root Causes, Emergency Fixes, and Production‑Grade Governance

This comprehensive guide explains why Kafka message backlog occurs, how to diagnose its root causes, and provides a step‑by‑step 5‑minute emergency response and production‑grade consumer architecture, including back‑pressure control, idempotent processing, capacity planning, observability, and cloud‑native deployment strategies.

BacklogConsumerKafka
0 likes · 48 min read
Kafka Backlog Mastery: Root Causes, Emergency Fixes, and Production‑Grade Governance
Random Bulletin
Random Bulletin
Jun 3, 2026 · Operations

Designing Partitions for Millions of QPS: From Default Settings to Precise Capacity Planning

The article explains why partition count is not a simple integer but the solution of a multi‑constraint capacity‑planning equation, walks through three design stages—from naïve defaults, through CPU‑core‑based heuristics, to a precise engineering model that balances throughput, ordering, scaling, fault domains and storage costs for million‑plus QPS workloads.

KafkaPartition Designcapacity planning
0 likes · 32 min read
Designing Partitions for Millions of QPS: From Default Settings to Precise Capacity Planning
Coder Trainee
Coder Trainee
Jun 3, 2026 · Cloud Native

Unified Messaging with Spring Cloud Stream: One Codebase for Multiple MQs

This article demonstrates how Spring Cloud Stream provides a unified programming model that decouples business logic from the underlying message broker, allowing a single codebase to work with RocketMQ, Kafka, or RabbitMQ by only changing configuration, and walks through project structure, implementation, conditional routing, MQ switching, testing, and common pitfalls.

JavaKafkaMessage-driven
0 likes · 19 min read
Unified Messaging with Spring Cloud Stream: One Codebase for Multiple MQs
TechVision Expert Circle
TechVision Expert Circle
Jun 2, 2026 · Artificial Intelligence

How to Build a System with True Self‑Decision Capability

This article details a CTO‑driven initiative to give core business systems genuine self‑decision ability by defining the concept, presenting a four‑layer architecture, and sharing concrete design choices, safety mechanisms, governance practices, and real‑world lessons learned from an e‑commerce fulfillment use case.

KafkaLLM AgentsMulti-Agent Architecture
0 likes · 14 min read
How to Build a System with True Self‑Decision Capability
Random Bulletin
Random Bulletin
Jun 2, 2026 · Backend Development

Queue Selection at Ten‑Million QPS: From General‑Purpose to Specialized Queues

In systems handling ten‑million QPS, generic message queues like Kafka hit scalability and latency limits, so the article breaks down a three‑question framework—loss tolerance, latency sensitivity, and ordering needs—to match five specialized queue types and guide a transition from a single‑queue approach to a queue matrix.

KafkaRocketMQdistributed systems
0 likes · 18 min read
Queue Selection at Ten‑Million QPS: From General‑Purpose to Specialized Queues
StarRocks
StarRocks
May 28, 2026 · Industry Insights

How Fresha Built a Modern Real‑Time Analytics Stack with AutoMQ and StarRocks

Fresha replaced its Postgres‑Snowflake‑MSK pipeline with an AutoMQ‑based Diskless Kafka message layer and StarRocks for real‑time analytics, cutting storage costs 17‑20×, dropping query latency from seconds to sub‑second, and migrating ~1,000 topics in a week with zero downtime.

AutoMQKafkaPostgres
0 likes · 24 min read
How Fresha Built a Modern Real‑Time Analytics Stack with AutoMQ and StarRocks
Cloud Architecture
Cloud Architecture
May 22, 2026 · Operations

Kafka vs Pulsar: Choosing the Right Messaging System for Architecture and Production

Kafka and Pulsar are both mature distributed messaging platforms, but they differ fundamentally in architecture, performance, scalability, and operational complexity; this article analyzes their core designs, benchmarks, engineering trade‑offs, and real‑world scenarios to guide architects in making a reliable technology selection.

ArchitectureKafkaKubernetes
0 likes · 36 min read
Kafka vs Pulsar: Choosing the Right Messaging System for Architecture and Production
LuTiao Programming
LuTiao Programming
May 21, 2026 · Backend Development

Stop Fighting Microservice Calls—Why Experts Prefer Event‑Driven Architecture for Decoupling Distributed Systems

The article explains how traditional synchronous microservice calls create tight coupling, cascading failures, scaling bottlenecks, and high latency, and demonstrates that adopting an event‑driven architecture with producers, consumers, and a message broker such as Kafka can fully decouple services, improve scalability, and enable patterns like event sourcing and CQRS.

CQRSEvent SourcingKafka
0 likes · 14 min read
Stop Fighting Microservice Calls—Why Experts Prefer Event‑Driven Architecture for Decoupling Distributed Systems
Su San Talks Tech
Su San Talks Tech
May 19, 2026 · Interview Experience

Designing a Hundred‑Billion‑Scale Message Queue: A ByteDance Interview Walkthrough

This article walks through the interview question of designing a message queue that handles billions of messages daily and peaks at millions of QPS, covering traffic calculations, core roles, storage and throughput techniques, scalability, high availability, observability, framework comparisons, a real‑world case study, and key follow‑up interview topics.

KafkaPulsarRocketMQ
0 likes · 12 min read
Designing a Hundred‑Billion‑Scale Message Queue: A ByteDance Interview Walkthrough
Cloud Architecture
Cloud Architecture
May 9, 2026 · Backend Development

High‑Concurrency Flash‑Sale Blueprint: Redis Atomic Stock and Kafka Throttling

The article presents a production‑grade, scalable flash‑sale architecture that combines Redis atomic inventory deduction, Kafka asynchronous peak‑shaving, and careful database finalization, detailing each layer’s goals, pre‑filtering techniques, Lua scripting, idempotency, capacity planning, monitoring, and compensation strategies to prevent overselling and ensure reliability.

JavaKafkaRedis
0 likes · 31 min read
High‑Concurrency Flash‑Sale Blueprint: Redis Atomic Stock and Kafka Throttling
Cloud Architecture
Cloud Architecture
May 8, 2026 · Databases

Slow Queries Causing Outages? Build a Cloud‑Native Distributed MySQL Slow‑Log Platform from Scratch

This article walks through the design and implementation of a production‑grade, cloud‑native MySQL slow‑log collection and analysis platform, covering everything from MySQL slow‑log fundamentals and multi‑node ingestion to Kafka buffering, Go‑based parsing, SQL fingerprinting, Elasticsearch and ClickHouse storage, alerting, APM integration, and a phased rollout roadmap.

Cloud NativeKafkaKubernetes
0 likes · 35 min read
Slow Queries Causing Outages? Build a Cloud‑Native Distributed MySQL Slow‑Log Platform from Scratch
Cloud Architecture
Cloud Architecture
May 6, 2026 · Backend Development

Why Polling Breaks at Scale and How Spring Boot Webhooks Enable High‑Concurrency Event‑Driven Architecture

The article explains why traditional polling becomes unsustainable under high load, contrasts polling with webhook‑based event delivery, and provides a complete Spring Boot implementation—including outbox pattern, Kafka integration, retry logic, security, observability, and operational best practices—to build a production‑grade, scalable webhook platform.

KafkaOutboxSpring Boot
0 likes · 36 min read
Why Polling Breaks at Scale and How Spring Boot Webhooks Enable High‑Concurrency Event‑Driven Architecture
Lobster Programming
Lobster Programming
May 6, 2026 · Backend Development

How to Choose the Right MQ: RabbitMQ vs RocketMQ vs Kafka

This article compares RabbitMQ, RocketMQ, and Kafka on throughput, latency, scalability, and reliability, outlining each system's core features and recommending suitable scenarios such as reliable messaging, high‑performance streaming, and large‑scale real‑time data processing.

KafkaRabbitMQRocketMQ
0 likes · 6 min read
How to Choose the Right MQ: RabbitMQ vs RocketMQ vs Kafka