Insights from the 58 Group Technical Salon: Database Operations Platform Construction and Practices

Background

On January 8, 2019, the 58 Group Technical Salon (Session 7 – "Database Operations Platform Construction") was held at the Beijing headquarters, organized by the 58 Group Technical Engineering Platform and the HR Magic Academy. Participants included database teams from Tujia.com, Kingsoft Cloud, 58 Group TEG, and related R&D staff. Speakers Zhang Wenjing (Tujia.com), Yu Shaozai (Kingsoft Cloud), and Yu Bowei (58 TEG) shared their teams' experiences and practices.

Key Takeaways

1. Tujia.com Block‑Based DB Automation System Development

The DBOOP system, designed by Tujia.com’s database team, addresses DBA automation and daily developer needs such as work orders and reports. Highlights include:

Forms and Processes: A block‑based development mode allows rapid creation of new work‑order types and workflows by editing an XML configuration, eliminating the need for code and ensuring security through DBA approval.

Report System: Supports sub‑reports, nested reports, inter‑report navigation, snapshots, email/WeChat subscriptions, fine‑grained permission management, connection string handling, data source grouping, and sharding.

Database Performance Model: Uses a health‑scoring model and incorporates Facebook Prophet for time‑series forecasting of key metrics, enabling comparison of current and historical performance.

Block‑Based Development Mode: Enables rapid task, workflow, and report creation via XML without writing code, boosting DBA productivity.

2. Kingsoft Cloud Database Cloud Platform Construction and Practice

Kingsoft Cloud, a leading global cloud service provider, presented its database cloud platform architecture and best practices.

Service High Availability: Cross‑city active‑active deployment and disaster recovery.

Diverse Scenarios: Supports SQL/NoSQL, cross‑cloud disaster recovery, big‑data analysis, SaaS applications.

Elastic Scaling: Scale‑up and scale‑out capabilities, including read‑only instance addition and sharding.

Data Security: VPC, security groups, intrusion detection, and data reliability guarantees.

The technical implementation is based on OpenStack with LXC virtualization and includes four subsystems:

KTrove: Unified control system with modular design, supporting multiple database types (MySQL, MongoDB, Redis, PostgreSQL, SQLServer) and virtualization technologies.

KHA: High‑availability system with cross‑TOR/ AZ deployment, active‑passive health checks, dual‑active mode, and circuit‑breaker mechanisms.

Beacon: Operations system providing health dashboards and automated actions such as failover, migration, throttling, and repair.

Homels: Data verification system ensuring strong consistency across replicas and real‑time backup verification.

Best‑practice use cases shared include game data backup and rollback, financial two‑city‑three‑center high‑availability, video service instance upgrades, and music service read‑write separation via KingProxy.

3. 58 Cloud DB Platform Exploration and Practice

The 58 Cloud DB platform, built by the 58 Group DBA team, offers high‑reliability, high‑performance MySQL, Redis, and MongoDB cloud services to various subsidiaries.

Platform Overview: Supports MySQL, Redis, MongoDB, ES, TiDB; handles over 400 billion daily requests across 4 000+ clusters.

Evolution Stages: Standardization, Automation, Platformization, and Self‑service, each improving efficiency and reducing manual effort.

Service System: Unified high‑availability and high‑performance services with features such as automatic failover, NVM‑SSD storage, real‑time backup, and elastic scaling.

Core Functions: Database resource requests via work orders, SQL auto‑review, WeChat approval, and >99% automation; self‑service operations including slow‑log reporting, real‑time monitoring, and query tools.

Intelligent Operations: Machine‑learning based server recommendation for deployments and alarm SMS merging to reduce noise.

Summary

The salon facilitated deep discussions on database operations platform challenges across different companies, highlighting common problems, shared solutions, and future directions for intelligent database operations.

Next Salon Preview

Details of the upcoming session are shown in the following images.