How Alibaba’s Blink Testing Platform Guarantees Real‑Time Big Data Reliability

Introduction

Apache Flink is a distributed open‑source framework for stream and batch processing. In 2016 Alibaba adopted Flink and evolved it into Blink. By 2017 Blink powered thousands of real‑time jobs during Double 11, processing up to 470 million logs per second, making reliability a critical concern.

Blink Testing Platform Overview

The Blink testing team built a dedicated testing platform (see image below) that covers three stages: code‑quality verification, integration testing, and pre‑release testing.

Code‑Quality Verification Stage

This stage includes static code scanning, unit tests, and minicluster‑based integration tests. Only when all three pass can code be committed to the Blink Git repository.

Functional Testing

Blink uses the defender framework (derived from pytest) to run end‑to‑end scenario tests. It supports IDE and Jenkins triggers, three execution modes (yarn_job, yarn_session, local), and provides detailed result reports via web UI or email. Key benefits include unified case scheduling, fine‑grained resource configuration, and support for both batch and streaming jobs.

Performance Testing

Performance tests focus on operator, SQL, and runtime metrics. Operator tests monitor individual operator performance and combinations. SQL tests use TPCH and TPCDS benchmarks. Runtime tests include end‑to‑end and module‑level measurements (network, scheduler, failover). Future plans aim to integrate E2E, module, and parameter‑tuning tests for root‑cause analysis.

Stability Testing

Stability tests simulate “black‑monkey” (environment failures) and “white‑monkey” (job‑level failures) scenarios using shell commands and Byteman injection. Tests run in iterative cycles: monkey injection, job submission, failover verification, and checkpoint/resource checks. The architecture consists of component, action, execution, and WebUI layers.

Pre‑Release Testing

Pre‑release testing clones real online jobs and data to run complex business logic at scale. It includes simulation testing (environment cloning, adaptation, and execution) and compatibility testing (static checks of execution plans and dynamic run‑time verification).

Simulation Testing

Simulation tests compare new Blink versions against current production versions using cloned tasks and sampled data, evaluating functionality, performance, and stability.

Compatibility Testing

Static checks analyze execution‑plan correctness and generation time across engine versions. Dynamic run‑time tests execute cloned jobs on the new engine; successful runs allow automatic version upgrades.

Outlook

After more than a year of effort, Blink’s overall quality has improved significantly, and its testing tools are maturing. The team plans to open source these tools, provide additional quality‑assurance utilities, and further boost developer productivity as Blink’s user base grows.