Kafka 4.0 replaces ZooKeeper with the native KRaft (Kafka Raft) consensus layer, simplifying deployment, boosting scalability, cutting latency, and introducing a new incremental consumer rebalance protocol and share‑group model that together reshape how distributed streaming applications are built and operated.
This article explains how Huawei's terminal cloud adopts the Apache Iceberg table format to efficiently manage large-scale datasets, detailing its architecture, feature engineering, merge operations, LSM-based storage, schema versioning, AB testing support, catalog enhancements, and future roadmap for full lifecycle data governance.
Learn how to overcome memory and speed bottlenecks when exporting massive datasets to Excel by using streaming APIs like SXSSFWorkbook and EasyExcel, optimizing database pagination, tuning JVM and connection pools, and applying asynchronous shard processing to achieve stable sub‑200 MB memory usage for millions of rows.
This article introduces Spring Framework's ResponseBodyEmitter, explains its role in asynchronous HTTP responses, outlines typical use cases such as long polling and real‑time log streaming, provides a complete Spring Boot example with code, and compares it with SSE and raw OutputStream approaches.
The Kuaishou‑hosted Apache Hudi Asia technical salon gathered over 230 attendees and featured seven experts from Kuaishou, Meituan, TikTok, Huawei, JD and others, who shared best practices, architecture designs, and performance optimizations for large‑scale data lake applications across AI, BI, and real‑time workloads.
This article explains the purpose, core methods, and practical scenarios of Spring Framework’s ResponseBodyEmitter, compares it with SSE and raw streaming, and provides a complete Spring Boot controller example that demonstrates how to implement real‑time log streaming and other asynchronous HTTP responses.
The Apache Hudi Asia technical salon held in Beijing on March 29 gathered over 230 on‑site participants and 16,000 online viewers, featuring expert talks from leading Chinese tech companies that showcased real‑world Hudi implementations, performance optimizations, and future roadmap for data‑lake technologies.
This article provides a comprehensive overview of Flink CDC 3.3, detailing its CDC fundamentals, new connectors, Transform module enhancements, asynchronous snapshot splitting, community adoption, and upcoming roadmap for broader ecosystem support and batch‑mode execution.
iQIYI transformed its real‑time data warehouse by replacing a costly Kafka‑based Lambda stack with a unified stream‑batch Iceberg lake, cutting storage expenses by 90%, halving compute costs, extending data retention, and delivering minute‑level freshness for 90% of use cases while preserving second‑level processing where needed.
Apache Kafka 4.0 eliminates Zookeeper with the KRaft consensus protocol, upgrades Java requirements to 17, introduces a next‑generation consumer rebalance protocol, adds Share Group queue semantics, and bundles numerous performance, security, and API improvements for modern streaming workloads.
Chronon is an open‑source ML feature platform that lets engineers declaratively define, compute, and serve both batch and real‑time features with built‑in observability, data‑quality checks, and a low‑latency retrieval API, ensuring online‑offline consistency while simplifying pipeline management and enabling future automation.
This article explains how Spring Framework's ResponseBodyEmitter enables real‑time, chunked HTTP responses for use cases such as log streaming, progress updates, chat, and AI output, detailing its advantages over SSE, usage scenarios, core methods, a complete controller example, and best‑practice considerations.
Apache Kafka 4.0 introduces a groundbreaking KRaft architecture that removes ZooKeeper, a revamped consumer‑group protocol that dramatically speeds up rebalancing, a new queue mode for point‑to‑point messaging, upgraded Java requirements, streamlined APIs, and numerous performance and security enhancements, reshaping both development and operations for large‑scale streaming workloads.
Apache Kafka 4.0 has been officially released as the first major version that runs entirely without Apache ZooKeeper, introducing KRaft mode, a new consumer group protocol (KIP‑848), early‑access queue support (KIP‑932), updated Java requirements, and other enhancements aimed at improving scalability, operability, and messaging versatility.
This article presents a comprehensive overview of Flink CDC + Hologres high‑performance data synchronization, detailing write and consumption optimizations, architectural principles, and future directions to achieve low latency and high throughput in real‑time data pipelines.
Alibaba Mama’s advertising platform migrated from a monolithic Flink‑Kafka pipeline to a layered Paimon lakehouse, adding DWS upsert support and multi‑layer storage, which delivers minute‑level data freshness, cuts latency by 2.5 hours, reduces resource use over 40 %, halves development effort and achieves ≥99.9 % availability.
gRPC offers superior performance, efficient Protobuf encoding, strong API contracts, seamless streaming, cross‑language support, and HTTP/2 advantages, making it a compelling alternative to REST for modern web services, while tools like gRPC‑Gateway, ConnectRPC, and Buf streamline migration and ecosystem integration.
The article introduces DeepSeek4j 1.4, a Java Spring Boot library that overcomes existing framework limitations by preserving DeepSeek's chain‑of‑thought capabilities, adding full reactive streaming, and providing a simple one‑line API along with quick‑start instructions and code examples.
This article introduces Spring Framework's ResponseBodyEmitter, explains its advantages over SSE for asynchronous HTTP responses, demonstrates practical usage with a real‑time log‑streaming example, outlines core methods, working principles, best‑practice considerations, and compares it with traditional Streaming and Server‑Sent Events.
This article examines how Apache Iceberg’s snapshot‑based ACID transactions, logical‑physical partition evolution, and COW/MOR update modes enable efficient real‑time data lake ingestion, and demonstrates AutoMQ’s Kafka‑to‑Iceberg Table Topic solution that simplifies schema management, reduces latency, and cuts operational costs.
Processing large files efficiently in Python requires techniques such as line-by-line iteration, chunked reads, generators, buffered I/O, and streaming, which help avoid memory errors, improve speed, and optimize resources for tasks like log analysis, data scraping, and real-time API handling.
This article introduces Alibaba's LangEngine, a pure Java AI application framework, detailing its high‑availability gateway architecture, communication protocols, streaming and non‑streaming output, multi‑level metadata caching, asynchronous and serverless designs, and future open‑source roadmap, offering practical guidance for building robust AI services.
This guide explains how to install, configure, and use the Grok‑PHP client to call Grok AI models from PHP, covering basic usage, advanced options, available model enums, and streaming responses with code examples.
The article introduces DeepSeek4j 1.4, a Java‑based framework that overcomes Spring AI’s limitations by fully preserving DeepSeek’s chain‑of‑thought and billing features, adding reactive streaming, providing Spring Boot starter integration, and offering quick‑start code samples and configuration guidance.
deepseek4j 1.3 introduces web‑search capability, streaming responses, system prompts, expanded multi‑platform support, enhanced SSE debugging, and upcoming features like API‑key rotation and resilience, enabling Java developers to integrate DeepSeek models effortlessly while focusing on business logic.
This article presents a comprehensive overview of Alibaba Cloud's modern data stack built on Flink CDC, detailing its core concepts, extended capabilities, typical application scenarios, performance optimizations, a live demo, and future development plans for large‑scale streaming data integration.
This article introduces FastExcel, an upgraded Java library for high‑performance Excel read/write, outlines its key features, provides step‑by‑step code examples for entity creation, event listeners, writing, reading, PDF conversion, compares it with EasyExcel, and concludes with its suitability for large‑scale data processing.
FastExcel is a Java library that builds on EasyExcel to provide higher performance, low‑memory streaming, and additional features such as PDF conversion, offering simple APIs, full compatibility, and detailed code examples for creating entity classes, listeners, and read/write operations.
This article summarizes a talk by Alibaba Cloud senior engineer and Flink Committer Luo Yuxia on the challenges of separating lake and stream storage, introduces the Fluss lake‑stream unified architecture, explains its technical benefits such as second‑level data freshness, unified metadata, efficient changelog generation, and outlines future plans for broader ecosystem integration.
This article describes the challenges of data quality in Ctrip’s hybrid‑cloud FinOps billing system and presents the design, implementation, and high‑availability deployment of a custom Kafka Gatekeeper proxy that performs pre‑validation, configurable rules, self‑service dashboards, and automated alerts to improve coverage, timeliness, and responsibility attribution.
This article provides a comprehensive overview of Apache Paimon, covering its real‑time lake ingestion, unified stream‑batch processing, table types (primary‑key and append‑only), LSM‑tree storage, bucket mechanisms, merge‑engine options, compaction strategies, concurrency control, consumption methods, tag management, data cleanup, and system tables for big‑data workloads.
The article recaps Zhihu's Cloud Native Innovation Forum where experts from AutoMQ, OceanBase, and Flashcat shared practical solutions on streaming data ingestion, unified database architectures, and AI‑driven observability, highlighting real‑world deployments, performance optimizations, and cost‑saving strategies.
When processing massive MySQL result sets, loading all rows into JVM memory can cause OOM and slow performance, so this guide compares three approaches—regular pagination, streaming queries using server-side cursors, and cursor‑based fetchSize control—detailing their implementations, MyBatis configurations, and trade‑offs.
This comprehensive guide explains Kafka’s architecture, core APIs, topics and partitions, deployment steps, multi‑broker clustering, and practical use cases such as messaging, log aggregation, stream processing, and data import/export with Kafka Connect, providing a hands‑on tutorial for developers and engineers.
This article explains how Apache Paimon can be used as a real‑time state store for Flink, detailing its low‑cost, scalable storage, lookup‑join design, table schema, bucket configuration, memory tuning, and practical use cases such as handling refund‑adjusted order tags and cumulative metrics.
Tencent's advertising team replaced a traditional HDFS‑Hive warehouse with an Apache Iceberg‑based data lake, adding primary‑key tables, multi‑stream merging, adaptive compaction, and Spark SPJ optimizations to achieve minute‑level feature update latency, 10× back‑fill speed, and up to 60% storage savings.
This guide explains how to optimize Paimon primary‑key and Append‑Scalable tables in Flink by adjusting sink and source parallelism, checkpoint intervals, making small‑file merges fully asynchronous, changing file formats, and applying ordering strategies to improve both write and read performance.
Bilibili migrated its massive user‑behavior, commercial AI training, and database synchronization pipelines from Hive and Kafka to an Iceberg‑based streaming‑batch architecture, using Flink and the Magnus optimizer to achieve minute‑level freshness, reduce CPU and memory usage by about 20‑22 %, save roughly 3.55 M CNY annually, and dramatically improve query latency and join performance.
This article presents Bilibili's end‑to‑end exploration of a streaming‑batch unified data pipeline built on Apache Iceberg, detailing the original and iterated architectures for massive user behavior transmission, online AI training, DB synchronization, and dimension‑join, along with performance gains, cost savings, and future plans.
This guide outlines why audio‑video technology is booming, how mastering FFmpeg provides a rapid entry point, and offers a structured learning path—including core concepts, streaming tools, API usage, and community contribution—to help developers quickly become proficient in media processing.
This article chronicles the birth, technical innovations, open‑source journey, and community impact of Alibaba’s EasyExcel library, explains its memory‑optimized streaming design with a concise code example, and announces the forthcoming EasyExcel‑Plus project along with a free book giveaway.
This article explains how to configure key OpenAI chat completion parameters—such as temperature, top_p, streaming, response format, and tool selection—and walks through the structure of the API's JSON response, highlighting fields like id, model, choices, finish_reason, and usage for better control and cost estimation.
This article compares Kafka and RabbitMQ across design philosophy, performance, data models, delivery semantics, ecosystem, and typical use cases, helping developers choose the right messaging middleware for high‑throughput streaming or reliable task queues.
This article explains how a gRPC bidirectional streaming read method was added to Ozone to reduce chunk‑by‑chunk request gaps, describes the client‑side implementation, presents single‑ and multi‑threaded performance tests showing roughly 30% faster reads, and outlines future enhancements such as pre‑fetching.
This article introduces Apache Paimon, an open‑source table format that supports batch and streaming reads and writes, explains its architecture, unified storage model, and core concepts such as file layout, snapshots, manifests, data files, partitions, and consistency guarantees.
This article introduces Apache Hudi Streamer, a versatile Spark‑based data ingestion tool likened to a Swiss Army knife, detailing its core options—including table configuration, continuous mode, source classes, transformers, table services, catalog synchronization, and advanced features—while guiding users on practical pipeline setup.
This article details Ele.me's journey in evolving its real‑time data warehouse, covering the original 1.0 architecture, the 2.0 lakehouse redesign with Paimon and StarRocks, performance evaluations of lake formats and query engines, and the roadmap toward a 3.0 streaming lakehouse solution.
This article presents the design goals, architecture, technology selection, and component details of a unified video review system that leverages FFmpeg for frame extraction, stream‑based parallel processing, and flexible synchronous/asynchronous workflows to achieve low latency and high scalability.
A wave of shutdowns affecting popular free video and anime piracy platforms such as RARBG and Animeflix reveals how pandemic costs, legal pressures, court rulings, and coordinated anti‑piracy actions by industry alliances are reshaping the digital media landscape and pushing users toward legitimate services.
This article details the major memory model changes in Flink 1.11 for JobManager and TaskManager, compares them with Flink 1.9, provides concrete JVM command examples, explains the relationship between memory settings and parallelism, and introduces fine‑grained resource management for streaming workloads.
This article shares a personal, experience‑driven overview of Apache Paimon, highlighting its design simplicity, key capabilities such as schema evolution, stream‑batch unified processing, primary‑key support, and closed‑loop data handling, while discussing when its features are appropriate for production environments.
Learn how to compile Nginx with the MP4 module and set up a server block to stream MP4 files directly, including required commands, configuration snippets, and restart procedures, enabling seamless offline video playback without downloading.
The article details Dewu’s custom Flink scheduler, DwScheduler, which adds JSON‑based resource specifications, per‑TaskManager slot sharing for balanced CPU use, hot TaskManager migration callbacks, and a new TmRestart strategy for rapid pod‑process recovery, offering practical techniques to enhance real‑time stream processing stability and performance.
This article analyzes the role, application scenarios, stability challenges, and comprehensive governance framework of Apache Kafka at Soul, covering deployment, configuration, monitoring, standard controls, common misuse, and future directions toward cloud‑native solutions.
This article evaluates MySQL backup strategies using Percona XtraBackup, comparing local, remote, and streaming modes in terms of backup duration, primary server load impact, and backup file size, and provides practical recommendations based on detailed test results.
This article provides an in‑depth overview of Apache Kafka, covering its use cases, comparison with other message queues, versioning, performance mechanisms, core concepts such as topics, partitions, offsets, consumer groups, rebalancing, replication, leader election, idempotence, transactions, compression, interceptors, request handling, and practical configuration tips for reliable streaming applications.
This guide walks you through creating a Kafka‑enabled EMR Serverless Spark cluster, configuring network connections and security groups, uploading JARs and Python resources, and finally launching and monitoring a PySpark streaming application.
Apache Kafka achieves high‑throughput, fault‑tolerant messaging by combining a partitioned log architecture with leader‑follower replication, asynchronous producer pipelines, configurable acknowledgments, page‑cache‑based sequential writes, zero‑copy transfers, batching, compression, and a multi‑reactor network model that together ensure scalability, reliability, and performance.
This article breaks down Netflix’s comprehensive technology stack—from mobile and web front‑ends to microservices, data storage, streaming pipelines, and CI/CD tools—showcasing how the platform delivers seamless, high‑performance video experiences to billions of users worldwide.
This guide shows how to deploy Apache Flink 1.17 in Docker, configure off‑heap memory, connect it to Pulsar via the 4.1.0‑1.17 connector, run example jobs that copy topics and perform windowed word‑count, and provides Maven dependencies, custom serialization tips, batching settings, and version‑specific best‑practice notes.
This article introduces the JavaScript WritableStream API, explains its core methods and construction, demonstrates practical scenarios such as file uploads, logging, data transformation, and media handling, and discusses advanced considerations like chunk sizing, error recovery, concurrency control, and performance optimization.
This article introduces the concept of streams, explains the Web Streams API and its ReadableStream component, details constructors, methods, queuing strategies, back‑pressure handling, BYOB and byte streams, and provides practical code examples and usage scenarios for modern web development.
This article presents a comprehensive overview of the evolution from traditional Lambda‑based real‑time data warehouse solutions to a data‑lake‑integrated architecture, detailing the shortcomings of legacy designs, the iterative improvements made at JD Technology, and the technical and operational challenges encountered during implementation.
This article summarizes Wang Feng's presentation on the next‑generation Lakehouse architecture, explaining how Apache Paimon provides a unified, real‑time data lake format that bridges batch and streaming workloads, enabling low‑latency analytics and AI integration for modern big‑data applications.
This article explains Kafka's role as a high‑throughput distributed message queue, detailing its core components, topic‑partition model, consumer groups, storage mechanisms, fault‑tolerance features, delivery guarantees, ZooKeeper coordination, and scalability strategies for building reliable real‑time data pipelines.
This article provides a comprehensive overview of Apache Kafka, covering its role as a message queue, core components, topic and partition design, consumer groups, storage mechanisms, high‑availability features, delivery guarantees, ZooKeeper coordination, and scalability strategies for building robust real‑time data pipelines.
This article presents UCloud's USDP‑based streaming data lake warehouse solution that leverages Flink for real‑time processing and Paimon for lake storage, detailing its architecture, advantages, practical scenarios, and providing complete SQL and Flink CDC code snippets for end‑to‑end implementation.
OpenAI’s open‑source openai‑assistants‑quickstart project shows how to integrate the Assistants API into a Next.js app, offering streaming chat, code‑interpreter, file‑search, and function‑calling tools, and provides step‑by‑step setup instructions so developers can quickly build and customize AI assistants.
This article details Xiaomi's sales data warehouse development, covering its history, architecture, dimensional modeling, layer design, streaming‑batch integration, governance, security, and future directions, while also addressing practical Q&A on implementation challenges and best practices.
This article provides a detailed overview of Apache Kafka, covering its core characteristics, distributed architecture, key components such as topics, partitions, brokers, producers, consumers, ZooKeeper, and common application scenarios like log collection, event‑driven architecture, real‑time analytics, and monitoring.
This article provides a comprehensive overview of Apache Kafka, covering its core features, architectural components, message flow, and common application scenarios such as log collection, decoupled messaging, activity tracking, operational monitoring, and stream processing.
Twitter processes up to 400 billion events per day, moving from a Lambda‑style architecture with Scalding, Heron, and TSAR to a hybrid Twitter‑Data‑Center and Google Cloud pipeline that delivers sub‑10 ms latency, higher throughput, and lower operational cost while simplifying real‑time aggregation.
The article reviews Apache Paimon's graduation to an Apache Top-Level Project, outlines the essential capabilities of modern lakehouse frameworks—including streaming and batch I/O, multi‑engine integration, and advanced features—and discusses the problems they solve and the promising direction of the lakehouse ecosystem.
On World Intellectual Property Day, Bilibili honored ten breakthrough patents that together enable billion‑scale video duplicate detection, AI‑driven story generation, synchronized live rhythm‑games, automatic OTT casting, knowledge‑graph‑based content moderation, glitch‑free multi‑audio streaming, modular playback integration, neural‑network resolution encoding, AV1 reference‑frame pruning, and fine‑grained GPU isolation.
This article explores Uber’s sophisticated real‑time data infrastructure, detailing how the company leverages open‑source technologies such as Apache Kafka, Flink, Pinot, and Presto, and describing the architectural components, scaling challenges, multi‑region resilience, data back‑filling, and operational practices that enable low‑latency analytics for millions of daily rides and deliveries.
This article presents a comprehensive overview of JD Tech's real‑time data warehouse evolution, detailing the legacy Lambda‑based design, its shortcomings, the transition to a data‑lake‑integrated architecture, iterative improvements, encountered technical and non‑technical issues, and future outlooks.
The article details a Golang‑based high‑throughput long‑connection service for live‑streaming, describing its five‑layer architecture, multi‑protocol support, load‑balancing, message‑queue decoupling, aggregation with brotli compression, multi‑region deployment, priority channels, and future enhancements for observability and intelligent endpoint selection.
In large‑scale Flink double‑stream joins, terabyte‑sized RocksDB state caused severe compaction latency and CPU spikes, but enabling RocksDB BlobDB KV‑separation (and an inner‑compaction patch) dramatically shrank SST files, reduced read/write latencies to sub‑millisecond levels, and cut CPU spikes by about half.
This article presents Li Auto’s end‑to‑end data integration journey, detailing the evolution of its data platform, the challenges of heterogeneous sources, and how a unified Flink‑on‑K8s solution with cloud‑native architecture, operator management, monitoring, and checkpointing addresses batch‑stream convergence and future scalability.
The article presents the background, full‑scenario construction, core architecture, device‑pool strategy, remote iOS control via WebDriverAgent, screen‑sync using ffmpeg, streaming pipeline, data collection, and practical lessons of the Cloud Touch platform that enables unified remote testing and customer‑support workflows for mobile applications.
The article explains Flink’s metrics architecture—core concepts, reporter interfaces, built‑in and custom metric types, elastic plugin design, and scheduled reporting—illustrated with a consumption‑latency example, and shows how Didi uses these metrics for real‑time UI curves, alerts, and intelligent task diagnosis.
This article compares RabbitMQ and Apache Kafka across architecture, message ordering, routing, timing, retention, fault handling, scalability, and consumer complexity, and provides guidance on which platform suits specific use‑cases such as flexible routing, strict ordering, long‑term retention, or high throughput.
Airbnb’s Riverbed framework unifies streaming CDC events and batch Spark jobs behind a GraphQL‑based declarative API to automatically build and maintain distributed materialized views, using Kafka‑partitioned ordering and version control to deliver billions of daily updates with low‑latency reads for features such as payments and search.
This article compares Redis Streams and Kafka, examining their architectures, ordering guarantees, consumer group models, scalability, and trade‑offs, and shows how Redis can emulate Kafka‑like semantics using the Runnel library, while highlighting memory‑speed benefits versus Kafka’s durable, unlimited log storage.
This article explains Kafka as a distributed publish/subscribe messaging system, detailing its core functions, architecture, advantages, deployment methods, common use cases, and provides Java consumer and producer code examples for real‑time data processing.
This guide shows how to install the OpenAI PHP async client and implement streaming and non‑streaming chat, image generation, audio speech, and embedding features using Workerman and Webman, including Azure OpenAI support, with complete code examples.
This article explains how Apache InLong provides automatic, secure, high‑performance real‑time data transfer to StarRocks, detailing the transactional Stream Load API, the two‑phase commit process, Flink‑based ingestion architecture, exactly‑once guarantees, and performance test results across different parallelism levels.
This article explains Kafka's fundamental components, version evolution, key monitoring metrics for producers, brokers, consumers and Zookeeper, and provides step‑by‑step troubleshooting methods for common issues such as slow topic throughput and message backlog.
This tutorial demonstrates how to use Spring WebFlux’s reactive streaming to create a ChatGPT‑like typing effect, covering backend setup, SSE integration, frontend Axios handling, and a comparison between Flux and traditional Server‑Sent Events.
Kafka, an Apache‑developed distributed publish/subscribe messaging system, provides reliable, high‑throughput real‑time data streaming with producers, consumers, brokers, streams, and connectors, and the article explains its core concepts, architecture, advantages, deployment methods, use cases, and includes Java code examples for producers and consumers.
This article introduces Apache Kafka as a high‑throughput, scalable, fault‑tolerant distributed streaming platform, explains why it is chosen for real‑time data pipelines, and details key Kafka Streams concepts such as stream processing, interactive queries, stateful processing, windowing, serialization, and testing.
Li Auto built a cloud‑native data‑integration platform by deploying Flink on Kubernetes, unifying batch and streaming workloads with a storage layer (JuiceFS + BOS) and Flink Operator, enabling simple source‑sink pipelines, elastic scaling, automated checkpointing, and centralized monitoring while addressing earlier fragmentation and resource inefficiencies.
This article explains the limitations of using JDBC for true streaming reads in Spark and Flink, demonstrates failed attempts with MySQL, shows workarounds that revert to batch processing, and recommends Flink CDC as the practical solution for incremental MySQL ingestion.
This article explains how Kafka serves as a high‑throughput, durable messaging system, a reliable storage layer, a log‑aggregation hub, a stream‑processing engine, and a core component for CDC, system migration, monitoring, and event‑sourcing architectures.
The Flink Forward Asia 2023 conference showcased major updates to Apache Flink (versions 1.17 and 1.18), introduced the Apache Paimon lakehouse project, announced Flink CDC 3.0, and highlighted community growth, cloud‑native deployments, and real‑time data‑warehouse use cases across industry leaders.
This step‑by‑step guide shows how to install and configure a single‑node Hadoop 3.2.0 environment on CentOS 7, set up Python 3.7, write MapReduce mapper and reducer scripts in Python, and run a word‑count job using Hadoop streaming, illustrating core Hadoop concepts and their relevance today.
This tutorial walks you through Hadoop’s core components, sets up a single‑node Hadoop cluster on CentOS 7, installs Python 3, writes mapper and reducer scripts in Python, and runs a Hadoop‑Streaming word‑count job to demonstrate classic big‑data processing techniques.
The Flink Forward Asia 2023 conference recap highlights opening remarks, a keynote on Flink’s dominance in streaming compute, detailed 2023 technical advancements, case studies, the launch of Flink CDC 3.0, and a preview of Flink 2.0, along with links to photos and video recordings.
This article presents a comprehensive overview of Zhihu's Bridge platform, detailing its content‑operation architecture—including content pool, management, analysis, monitoring, and intervention modules—explaining the underlying streaming and batch technologies such as Flink, Doris, and Elasticsearch, and outlining future automation and AI‑driven workflow directions.
A Flink job that reads from Kafka and writes to Elasticsearch was losing data because the bootstrap.servers list mixed production and pre‑release clusters, causing random server selection, partition discovery failures, and offset mismatches, which were resolved by correcting the server configuration.
