Kafka Streams is a client library that enables low‑latency, fault‑tolerant real‑time processing of Kafka data through configurable topologies, time semantics, and state stores, and the article explains its architecture, essential configurations, monitoring‑focused ETL example, performance tuning, and strategies for handling partition skew.
This article introduces Apache Kafka as a high‑throughput, scalable, fault‑tolerant distributed streaming platform, explains why it is chosen for real‑time data pipelines, and details key Kafka Streams concepts such as stream processing, interactive queries, stateful processing, windowing, serialization, and testing.
Event sourcing models state changes as immutable logs, and when combined with CQRS and Kafka Streams, it enables scalable, fault‑tolerant architectures where write and read paths are decoupled, supporting local or external state stores, interactive queries, and zero‑downtime upgrades.
This article explains how event sourcing models state changes as immutable logs, discusses the trade‑offs of the pattern, and shows how Apache Kafka and Kafka Streams enable CQRS and interactive queries for building scalable, resilient backend applications.
This article explores how stream computing systems achieve correct results by addressing data integrity, distinguishing consistency from correctness, formalizing integrity inference, and comparing implementations across major engines such as Flink, Kafka Streams, MillWheel, and Spark Structured Streaming.
The Apache Kafka 3.3 release introduces production‑ready KRaft mode and a series of KIP enhancements across brokers, streams, and connect, detailing new metrics, API changes, and operational improvements while encouraging users to review the full release notes and try the updated platform.
This article describes the design, implementation, and performance of ShouQianBa's advertising billing system, detailing the migration from Apache Druid to Kafka Streams, the architecture for real‑time event processing, data aggregation, persistence, fault tolerance, and the achieved low‑latency, high‑throughput metrics.
This article explains how to design and implement a terabyte‑level log monitoring platform using ELK Stack, FileBeat, Elastic APM, Kafka Streams, Prometheus, and Grafana, covering data collection, filtering, visualization, and resource‑efficient processing for large‑scale microservice environments.
This article explains how to build a terabyte‑level log monitoring platform for micro‑service environments by unifying log collection with FileBeat, enriching observability through Elastic APM, processing streams via Kafka Streams, and visualizing metrics with Grafana and Kibana, while addressing cost‑effective filtering and retention strategies.
This article explains Kafka's evolution—highlighting the introduction of Kafka Streams, the shift to a full distributed stream processing platform, practical learning paths, source‑code reading tips, common pitfalls, and the major new features introduced in Kafka 3.0.
This article explains how to design and implement a TB‑scale log monitoring solution using the ELK stack, FileBeat, Elastic APM, Kafka Streams, Prometheus and Grafana, detailing architecture, data collection, filtering, visualization, and the trade‑offs of resource usage in large‑scale microservice environments.
This article explains how to design and implement a TB‑level log monitoring system for microservice environments using the ELK stack, detailing log collection with FileBeat, tracing via Elastic APM, resource‑efficient processing with Kafka Streams, and visualization through Grafana and Kibana.
This article explains how to design and implement a TB‑level log monitoring platform for micro‑service environments using ELK Stack, Filebeat, Elastic APM, Kafka Streams, Prometheus, and Grafana, covering data collection, filtering, storage, and visualization while addressing cost and resource constraints.
This guide walks through setting up Kafka Streams with Spring Boot 2.3, covering environment configuration, core concepts, topology design, and multiple practical examples—including message sending, listening, transformations, aggregations, filtering, branching, and multi‑field grouping—complete with full code snippets and execution results.
This article explains how event sourcing models state changes as immutable logs, discusses its advantages and drawbacks, shows how CQRS separates command and query responsibilities, and demonstrates how Apache Kafka and Kafka Streams enable scalable, fault‑tolerant implementations with real‑world examples.
In large‑scale microservice environments, centralized log collection, filtering, and visualization using Filebeat, Elastic APM, Kafka Streams, Grafana and Prometheus can turn scattered logs into actionable operational data while controlling resource costs.
This article provides a comprehensive guide to upgrading Apache Kafka across multiple versions, detailing rolling upgrade procedures, configuration adjustments, protocol changes, new features, deprecations, and performance considerations for Kafka brokers, producers, consumers, and Kafka Streams applications.
The article describes an enterprise‑grade log monitoring solution that unifies log collection, filtering, cleaning, and visualization across hundreds of microservices using tools such as Filebeat, Elastic APM, Kafka Streams, Prometheus, Grafana and Kibana to improve troubleshooting, performance analysis, and operational efficiency.
