Evolution and Architecture of Ctrip's System: Operations, Frameworks, and Big Data

Author Introduction Zhou Yuan, senior R&D manager at Ctrip Technology Center, with over 10 years of software development experience, shares insights from a frontline perspective.

Preface Ctrip's architecture has undergone multiple iterations, each solving previous pain points while introducing new challenges. This article first outlines the overall architecture, then examines release systems, configuration management, and SOA as case studies, and finally highlights the UserProfile project.

1. Architecture Composition

The architecture consists of three layers: Operations, Framework, and Application.

1.1 Operations

Key operational highlights include:

1.1.1 Cluster Management – SLB directs traffic based on health checks, automatically adding or removing nodes.

1.1.2 FullDR Mechanism – Web, DB, and Redis clusters have long‑term disaster‑recovery plans, regularly exercised.

1.1.3 DBA Strategy – High‑availability data storage using M‑S mechanism, FullDR, and migration from MSSQL to MySQL.

1.1.4 NOC Mechanism – 24/7 monitoring with dashboards for orders and exceptions, ensuring rapid response.

1.2 Framework

Key framework components include:

1.2.1 SOA & Gateway – Service governance platform with a long history.

1.2.2 Release System – Features such as brake, rollback, version switching, shared DLL packaging, and POM checks; evolved through several generations.

1.2.3 Message Queue – A custom solution combining strengths of open‑source queues, offering ordered partitions, asynchronous compensation, and lifecycle tracking.

1.2.4 Configuration Management – Evolved to provide convenient, high‑performance configuration handling.

1.3 Application

Common techniques include PreLoading, LayerLoading, Sharding, circuit breaking, rate limiting, and degradation, which significantly improve website and app stability.

2. Architectural Evolution

2.1 Release System Four generations: ITSM, CITSM, CRoller (ROP), and Tars (CD). Each iteration improved developer‑release separation, added B/S architecture, integrated configuration, introduced All‑In‑One, ConfigGen, and finally remote backup for disaster resilience.

2.2 Configuration Management Four generations: simple web‑config wrapper, config integrated into releases, service‑driven runtime config with listener, and JSON‑based open‑source solution.

2.3 SOA Evolution from ESB‑based bus (first generation) to direct service connections (second generation) and then to advanced features such as circuit breaking, rate limiting, dynamic routing, and Gateway for H5/APP interactions.

3. UserProfile Project

The UserProfile system is a core big‑data component comprising registration, collection, computation, storage, query, and monitoring. Data sources include personal info, travel history, contacts, behavior, and orders.

Data is ingested via batch and streaming pipelines; streaming uses Kafka + Storm and Ctrip's Hermes platform.

Storage spans Hive, MySQL, Redis, all protected by FullDR + M‑S design, holding over 10 billion records. Average service response time stays around 10 ms, supported by circuit breaking, rate limiting, degradation, and sharding.

Recommended Reading

Graphic Captcha Practices at Ctrip

Undisclosed Anti‑Scraping Techniques

Ctrip User Data Collection and Analysis System

Ctrip Hotel Browsing Customer Churn Prediction