How Alibaba’s CloudDBA Transforms Database Optimization with AI and Automation

At the 2017 China Application Performance Management Conference, Alibaba senior database expert Qiao Honglin presented the exploration and practice of an intelligent database optimization system.

Business Demands

Rapid growth of database scale and fast‑changing workloads require moving beyond manual DBA support to productized services that provide service productization, global scale optimization, proactive diagnosis, intelligent anomaly detection, and capacity forecasting.

User Demands

Developers need transparent, self‑service optimization, continuous performance tuning, quantitative tracking with closed‑loop processes, and product‑based solutions rather than relying on DBA intervention.

CloudDBA Key Technologies

CloudDBA consists of four layers: a data collection layer gathering per‑second metrics, a computation layer performing real‑time and offline analysis, a diagnostic service layer offering expert SQL, space, and configuration advice, and an access layer (iDB) exposing services to developers.

SQL Optimization

The system identifies high‑impact SQL statements, generates candidate indexes using a cost‑based optimizer, evaluates alternatives with dynamic sampling, and provides what‑if optimization suggestions, enabling developers to apply and verify improvements in a closed loop.

Space Optimization

CloudDBA analyzes storage usage, recommends compression or engine migration (e.g., InnoDB to RocksDB), reclaims unused objects and duplicate indexes, and migrates cold data to cheaper storage, offering one‑click space‑saving actions.

Data‑Driven Approach

All metrics are collected at second granularity via AliSQL, streamed to Kafka, processed by JStorm for real‑time analytics, and stored for offline analysis, supporting global performance cost models, capacity forecasting, and large‑scale optimization decisions.

Intelligent Optimization Exploration

Research directions include second‑level anomaly detection and correlation analysis, proactive pre‑warning, accurate capacity forecasting using historical growth data, and future self‑diagnosis/self‑optimization where models automatically select and apply optimal fixes.

Conclusion

CloudDBA addresses both business and developer needs by providing automated, data‑driven, and increasingly intelligent database optimization services, with ongoing efforts to enhance AI capabilities for proactive and autonomous performance management.