Design and Implementation of a Unified Data Service Platform for Reducing Development Cost and Enhancing Efficiency

Author : Xiaofeng, Ctrip R&D Director, specializes in distributed databases, real‑time computing, and big‑data system architecture.

Background : Rapid growth of hotel data and personalized demands caused lengthy, non‑standardized API development cycles (often 2+ days), high maintenance costs for over 700 interfaces (500+ active), heterogeneous tech stacks across teams, and increasing business data requests.

To address these issues, a unified data service platform was designed to accelerate personalized demand support, lower R&D costs, avoid siloed development, and improve data reuse.

Platform Overview : Built on the company’s SOA foundation, the platform provides a unified technical solution that reduces operational costs, improves API stability, and enables configuration‑driven deployment without Java developer involvement. It offers a visual interface for data‑warehouse teams to expose Hive tables as APIs, ensures consistent data sources, and supplies standard request templates for better communication.

Implementation Details :

2.1 Platform Consolidation : Reduces API output teams and abstracts underlying storage (ClickHouse, ES, Redis, MySQL, StarRocks, etc.) based on workload characteristics.

2.2 Data Utilization : Reuses synchronized tables across scenarios via configurable SQL, leveraging lineage monitoring to avoid duplicate offline sync.

2.3 API Security : Uses appid‑token authorization generated by the SOA framework to protect unauthorized access.

2.4 Rate‑Limiting : Integrates Sentinel for token‑bucket throttling, per‑API isolation, and automatic circuit‑breaking.

2.5 Data Caching : Employs active and passive caches for frequently accessed configuration data, reducing server load.

2.6 Service Contract Unification : Standardizes request/response structures with head and params sections; dynamic JSON parsing matches parameters to SQL placeholders.

2.7 SQL Mapping : Uses ? or {index} placeholders; response fields are mapped to desired output via configuration.

2.8 Contract Documentation Generation : Automatically creates detailed API docs and emails them to requesters, reducing explanation effort.

2.9 Service Monitoring : Utilizes Clog and CK logging frameworks to record request/response traces, parameters, and latency for each API.

2.10 Production Impact : Since December 2021, the platform serves 10 appids, over 100 APIs, handling >3.9 million daily requests; 91.49 % of responses are under 10 ms, 99.99 % under 100 ms, with deployment cycles reduced to half a day.

Future Outlook : Plans include tiered cluster deployment for high‑priority callers, test‑environment support, and further cost‑effective isolation.