Unlocking Intelligent Database Operations: Inside Zyb’s Multi‑Cloud Platform

Business Background

With diversified business scenarios, the team manages many standard databases such as MySQL, Redis, TiDB, Elasticsearch, and MongoDB, and the cluster size keeps growing.

Unified Multi‑Cloud Architecture

The company has built a cloud‑native multi‑cloud architecture. A unified Proxy layer is containerized, and standard database versions are self‑built to eliminate differences between clouds.

Self‑Developed Proxy

The DBA team open‑sourced a Redis‑cluster proxy called "recuffer", which improves traffic management and routing.

Database Platform – "Journey"

The "journey" platform provides intelligent lifecycle management for all supported databases, enhancing efficiency and stability.

Intelligent Operations Benefits

Reduces DBA manual work, freeing them for innovation.

Prevents human errors through safety checks, operation windows, and multi‑person review.

Detects unknown risks via alerts, log collection, and regular inspections.

Standardizes versions, configurations, and permissions, ensuring auditability.

Implementation Approach

Combines DBA expertise, comprehensive real‑time metrics, and algorithmic models to automate resource allocation and fault handling.

Platform Architecture

The platform is divided into five layers: User, Infrastructure, Database Service, Middleware, and Tools.

User Layer : Role‑based permission control.

Infrastructure Layer : Configuration management, logging, task/work‑order system, CMDB, monitoring & alert integration.

Database Service Layer : Management of each database type, including scaling, configuration, monitoring, backup, and audit.

Middleware Layer : Containerized Proxy management (configuration, K8s clusters, traffic control).

Tool Layer : Independent services such as DTS, data validation, inspection, and audit tools.

Key Components

Task System : Supports customizable, extensible, complex workflow orchestration with Ansible playbooks, platform tasks, and agent tasks, plus task chains and groups.

Monitoring System : Uses Prometheus, custom exporters, AlertManager, and Grafana for visualization.

Resource Management : Mix‑deployment of different database clusters on shared machines, guided by affinity, usage, and policy parameters.

Backup System : Automated backup pipeline with local and remote storage, verification, and expiration cleanup.

Failure Handling

Includes automated machine replacement, data recovery via DTS, and self‑healing mechanisms for common issues.

Multi‑Cloud Control Strategies

Supports cross‑cloud master‑slave replication and unit‑based architecture with bidirectional DTS sync, along with containerized Proxy control, cloud‑switching, and fault‑drill exercises.

Deployment Model

Multi‑region and multi‑cloud deployment isolates environments and ensures high availability.

Open‑Source Plans

The platform’s components will be open‑sourced at https://github.com/zyb-dba .