Design and Implementation of Ctrip Fast Log (CFL) – A MySQL Time‑Series Storage Engine

Jiang Yuxiang, a senior researcher at Ctrip Technology Assurance Center, has over ten years of experience in database development, especially MySQL source‑code research and modification.

While using MySQL, the flexibility of multiple storage engines (e.g., InnoDB, Blackhole) is appreciated, yet domestic development of MySQL storage engines remains scarce, prompting the need for a home‑grown solution.

During a routine discussion on suitable time‑series databases for Ctrip, the idea emerged to build a MySQL‑based time‑series storage engine that would retain SQL access, avoid external deployment complexities, and leverage existing MySQL tooling.

The article first reviews OpenTSDB, whose backend relies on HBase and requires a dedicated front‑end for data collection and dashboards, illustrating the lack of a universal architecture for time‑series databases.

The proposed architecture aims to be transparent to MySQL users: all familiar HA, replication, backup, and CRUD operations remain unchanged, allowing applications to interact via standard SQL.

The resulting experimental product, the CFL (Ctrip Fast Log) engine, records data using a fast‑log mechanism and fulfills the previously defined requirements.

MySQL storage engines consist of two layers: a functional layer that implements actual data operations (insert, delete, update) and an interface layer that maps MySQL handler calls to the functional layer. This separation is illustrated in the accompanying diagram.

The functional layer of CFL is designed for maximum insert throughput. Parallel sessions insert data into a table object, which buffers rows; once the buffer is full, it is queued for disk writes handled by dedicated writer threads, enabling concurrent disk I/O.

For storage, CFL adopts a strict sequential‑write strategy to maximize insertion speed and facilitate rapid failure recovery by writing data, index, and control information in order, while storing data and index in separate files for simplicity.

The interface layer inherits MySQL’s handler base class and implements required methods such as ha_open and ha_close. The article uses ha_cfl as an example to demonstrate how SQL operations are translated into calls to the functional layer.

Performance tests show that CFL’s insert‑per‑second (IPS) rates significantly exceed those of InnoDB and MyISAM across 1, 3, and 6 insert threads, confirming its superior write performance for time‑series workloads.

In conclusion, although the project is exploratory and has limitations (e.g., timestamp column constraints, lack of multi‑column indexes, limited update/delete flexibility), it successfully delivers a complete concept and implementation, providing valuable experience for the Chinese MySQL community.