1 views collected around this technical thread.
This article explains how Kafka's Schema Registry manages schema evolution and compatibility types—backward, forward, transitive, and full—using Avro schemas, demonstrates the impact of field additions or deletions on producers and consumers, and shows how to change a topic's compatibility setting via REST API.
This article provides a comprehensive English guide to Kafka's producer component, covering its end‑to‑end workflow, detailed steps such as record creation, serialization, partitioning, batching, sender thread, custom data structures, send modes, common exceptions, and essential configuration parameters for optimal performance.
This article chronicles the four‑year evolution of Chehaoduo Group’s Kafka ecosystem—from its initial role as a simple data‑ingestion layer to becoming the core of the company’s large‑scale data pipeline—detailing cluster management, upgrade strategies, multi‑cluster deployment, AVRO schema handling, SDK development, and operational lessons learned.
Apache Avro is a high‑performance binary data serialization system originating from Hadoop that uses JSON‑defined schemas to enable compact storage, efficient network transfer, container file formats for MapReduce, and RPC communication without requiring code generation or explicit field numbers.