CVTE’s Journey of Stream Computing Adoption: Architecture, Applications, and Lessons Learned

Introduction

The article introduces the business background for CVTE’s adoption of stream computing, the problems encountered during its application, and future plans for stream computing usage.

1. CVTE Overview

Guangzhou CVTE Electronic Technology Co., Ltd., founded in December 2005, employs over 6,000 staff (about 60% technical) and focuses on LCD controller boards and interactive smart panels. The company has built well‑known brands such as Seewo and MAXHUB and has received numerous national innovation and employer awards.

2. Pre‑Stream Computing Data Landscape

CVTE, as a manufacturing enterprise, faced increasing numbers of systems, growing data volumes, and complex inter‑system data interactions, leading to scalability and latency issues.

3. Why Adopt Stream Computing

Early solutions using CDC‑based real‑time table sync and scheduled view queries could no longer meet business needs as system count and data size grew. Stream computing was introduced to achieve SQL compatibility, view‑less calculations, incremental materialized views, real‑time writes, and join capabilities.

4. Stream Computing Application Process

4.1 Data Integration Platform Construction

The integration platform combines data extraction (CDC) and stream computing. CDC synchronizes various databases to Kafka, the stream engine consumes Kafka messages, computes results, and writes them back to target databases or Kafka, decoupling write processes and enabling unified data tracing.

4.1.1 Integration Platform Architecture Diagram

4.1.2 Data Extraction Scheme

4.2 Stream Computing Exploration

Four technology iterations were explored: PipelineDB, ksqlDB, Materialize, and RisingWave, each addressing limitations of the previous solution.

4.2.1 PipelineDB Application

In 2019 CVTE piloted real‑time production capacity calculation using PipelineDB on PostgreSQL, but it lacked support for complex stream‑stream joins, limiting large‑scale adoption.

4.2.2 ksqlDB Application

ksqlDB was evaluated but a critical defect prevented production use; however, the exploration yielded a solution for generating tombstone messages for Oracle delete events.

4.2.3 Materialize Application

Materialize, adopted in October 2020, satisfied many requirements and resolved prior shortcomings, yet some limitations still hindered massive deployment.

4.2.4 RisingWave Application

Starting in early 2023, CVTE migrated to RisingWave, finding its feature set and reliability superior to Materialize, and began testing larger scenarios such as real‑time material calculation platforms and master‑data synchronization.

5. Stream Computing Summary

After nearly five years of exploration, CVTE has achieved real‑time processing of massive data streams, supporting analytics, quality alerts, and anomaly detection, while reducing costs and enhancing decision‑making; the company continues to expand stream computing across manufacturing and supply‑chain domains.