Alibaba’s RDC: Integrated Testing and Continuous Delivery to Defeat the ‘No Silver Bullet’

1. Alibaba R&D Collaboration Platform Overview

Alibaba's R&D Collaboration Platform (AONE), now offered as Alibaba Cloud RDC, provides an end‑to‑end continuous delivery service covering requirement → coding → testing → release → feedback, addressing cross‑role, cross‑organization, and cross‑region collaboration challenges.

The platform follows a micro‑service architecture with three layers: the infrastructure layer (IDC, network, OS, Docker, databases), the platform services layer (middleware, scheduling, application servers), and the RDC layer (project, code, application, test, delivery, operation management).

2. Quality Assurance Strategy

The strategy combines “prevention”, “control”, and “post‑mortem improvement”, focusing on three practices:

Test‑driven continuous delivery : each CI/CD cycle runs automated unit, API, integration, UI, mobile UI, load, and smoke tests.

Online quality monitoring : machine and business metric alarms, log aggregation, SLA data visualization, 24/7 monitoring and fault‑drill exercises.

R&D quality improvement : code review and corporate rule scanning to enforce coding standards.

3. Continuous Delivery Process

The test‑driven CD workflow includes change submission, unit testing, isolated test environment deployment, pre‑release integration testing, beta traffic replay, and final production release with smoke testing. RDC’s release system and lab integration enforce test gates before advancing stages.

4. Testing Techniques and Methods

4.1 Layered Automated Testing

Adopts a testing pyramid (unit 70%, integration/API 20%, UI 10%) to allocate effort across development stages.

4.2 Test Framework

After evaluating tools such as Fitness, RobotFramework, JMeter, Postman, and SoapUI, Alibaba selected TestNG for its flexibility, listener support, and Java ecosystem, delegating other functionalities to RDC.

The framework consists of four layers: basic test libraries (Selenium, Macaca, SSH, HttpClient, CLI), service layer (business‑level wrappers), test‑case logic layer, and test‑scenario layer.

4.3 Unit Testing and Dependency Isolation

Uses Mock and Stub techniques, with PowerMock+Mock for constructor, static, and enum mocking, complemented by Mockito for most cases.

4.4 Large‑scale Integration Testing

API‑based integration tests run in pre‑release environments, using HttpClient wrappers and ensuring test gates via RDC.

4.5 UI Automation on Cloud Browsers

UI tests focus on critical user flows, implemented with Selenium PageObject pattern and CSS selectors for speed; a Selenium‑grid cloud ensures cross‑browser stability.

4.6 Full‑site UI Validation

Combines element extraction via Selenium crawlers with image comparison (Sikuli) to verify page rendering.

4.7 Core API Load Testing

JMeter scripts execute 10‑thread, 8‑minute loads; baseline response times are stored and compared against new runs, with deviations >10% flagged.

4.8 Mobile Real‑Device Testing

Macaca drives Android and iOS real devices in a Jenkins‑managed cluster, integrating results back into RDC.

4.9 AOP Recording and Replay for API Testing

ASM bytecode instrumentation records production traffic, replays it in a beta environment, and uses mocks for write operations, with RDC orchestrating the test gates.

4.10 Test Reporting

Reports are layered: real‑time alerts via DingTalk, single‑run detailed reports with logs, and statistical analysis dashboards for pass rates and failure trends.

5. Online Quality Monitoring

Machine metrics (CPU, memory, TPS, JVM) trigger alarms; business metrics are logged, aggregated, and visualized via Alibaba Cloud SLS, reducing daily error volume from hundreds of thousands to thousands.

SLA‑driven data quantifies user experience, combining database extraction and log‑based metrics to identify bottlenecks.

6. R&D Quality Standards and Improvement

Code review, corporate rule scanning (Sonar + Alibaba standards), and RDC’s efficiency dashboards track demand, defect, release, rollback, and code churn metrics for continuous improvement.

7. Summary and Outlook

Future work aims to enhance UI test cost‑effectiveness, expand crawler‑based replay tools, and develop predictive fault detection through massive log mining and self‑learning alert systems.