Real-Time Computing Solutions with Flink and HBase: Architecture, Market Analysis, and Use Cases

At the 10th HBase Meetup in Hangzhou, Alibaba senior product engineer Gao Yang introduced the background of real‑time computing and presented a scenario‑driven solution built on Flink + HBase, highlighting typical applications in online education, city brain, and real‑time fraud detection.

Speaker Introduction : Gao Yang (nickname: Wu Yu), senior real‑time computing product expert in Alibaba Cloud Computing Platform.

Alibaba Cloud’s real‑time computing team questions whether a single product can meet increasingly complex customer scenarios, leading to the Flink + HBase solution.

Real‑time Computing Market Competition – Traditional Vendors : Companies like IBM and Oracle entered early, offering rich 2B/2C solutions but at high cost and with on‑premise deployment; their cloud transition has been weak.

Real‑time Computing Market Competition – Cloud Vendors : Google (Dataflow, 2014), Microsoft Azure (2015), AWS (2016), Alibaba Cloud and Huawei (2017), and Tencent Cloud (2018) entered later, offering lower‑cost services with large growth potential.

Open‑Source Technology Ecosystem : The stream processing landscape evolved from Storm (1st gen) to Spark (2nd gen) to Flink (3rd gen). Flink’s stateful processing makes it suitable for event‑driven scenarios and micro‑service architectures, but it still requires upstream Kafka for ingestion and downstream storage such as HBase.

Flink Containerized Solution – Architecture : Built on Google Kubernetes Engine, the solution integrates Flink, Kafka, and HBase as containers, allowing additional services to be added via Kubernetes orchestration. A SaaS layer on top provides industry‑specific services such as rule engines for security, decision engines for finance, and AI/ML services for video analysis.

Ecosystem Partners : Alibaba Cloud seeks vertical partners to contribute SaaS‑level solutions, creating an end‑to‑end closed‑loop ecosystem that combines Flink, HBase, and Kafka.

Typical Flink Real‑time Scenarios : Since its launch in Oct 2018, the solution has been applied to data analysis (real‑time dashboards), event‑driven monitoring and fraud detection, and strong‑compute data processing such as city traffic management and online education video analysis.

Online Education – Real‑time Video Analysis : Alibaba Cloud collaborated with leading online‑education unicorns to apply Flink + HBase + Kafka for video quality assessment and course progress monitoring.

Online Education – Real‑time Prediction : Using machine‑learning on streaming data, the platform predicts the number of courses that can be opened in the next 30 minutes, demonstrating Flink’s role in online ML workloads.

City Brain – Real‑time Video Analysis : Deployed in multiple Chinese cities, the solution processes massive video streams from traffic cameras, using Flink for real‑time analytics and HBase for storage, enabling intelligent traffic control.

Real‑time Fraud Detection (Risk Control) : Events from apps or web logs are queued, then processed by risk models and rule engines in Flink to generate real‑time alerts for financial and marketing fraud scenarios.