How WCDB Evolved: Multi‑Language Support, Powerful SQL Builder, and Ultra‑Fast Backup

1. Introduction

WCDB is an open‑source, SQLite‑based client database released by the WeChat team in June 2017. It has gained wide industry recognition, released more than ten versions, and has continuously maintained backward compatibility while refining existing interfaces and adding new features.

2. Challenges

WeChat handles hundreds of different business databases with message counts reaching millions or tens of millions. This massive data volume and diverse use cases create constantly evolving requirements that the original code framework can no longer meet.

3. Major Upgrade

Since 2019 we abandoned strict backward compatibility to build a stronger WCDB. After multiple iterations, the interface layer and core logic have been comprehensively improved, and many new features have been added. The new version introduces:

Richer language support: added C++, full Java and Kotlin ORM support, covering more client platforms.

More powerful SQL expression capability: Winq has been rewritten and strengthened.

Safer data storage: new backup and repair schemes.

More flexible data extension: migration, compression, etc.

Finer performance optimizations: FTS5 optimization, interruptible transactions.

4. Richer Language Support

WCDB 1.0 supported Objective‑C, Swift, and Java, each with separate code bases. New capabilities were mainly added on the ObjC side, leaving Swift and Java lagging. The core logic is implemented in C++, allowing the interface layer for each language to bridge to the same C++ core. WCDB now also supports C++ and Kotlin, covering all mainstream client languages.

Code Framework

With this architecture, different language bindings can be integrated into the same project, reducing code size and avoiding interface conflicts.

ORM Implementation Example

Below is a simple C++ object used for ORM:

class Sample {
    public:
        int id;
        std::string content;
        WCDB_CPP_ORM_DECLARATION(Sample)
};

The object can be inserted and queried directly:

// INSERT INTO myTable(id, content) VALUES(1, 'text')
 database.insertObjects<Sample>(Sample(1, "text"), myTable);
// SELECT id, content FROM myTable WHERE id > 0
 auto objects = database.getAllObjects<Sample>(myTable, WCDB_FIELD(Sample::id) > 0);

WCDB_FIELD maps both column names and object members, enabling seamless serialization.

Class Member Pointer ORM

C++ cannot obtain class metadata at runtime, but compile‑time member pointers contain type information. By casting member pointers to void* we can use them as keys for column mapping:

int Sample::* memberPointer = &Sample::id;
Sample obj;
obj.*memberPointer = 1;
int id = obj.*memberPointer;

A template function removes the type from the member pointer:

template<typename T, typename O>
void* castMemberPointer(T O::*memberPointer) {
    union { T O::*memberp; void* voidp; } u;
    u.memberp = memberPointer;
    return u.voidp;
}

This pointer serves as a unique key for column‑member mapping.

5. More Powerful SQL Expression

WCDB 1.0 provided Winq, a C++ abstraction of SQLite syntax that avoided string concatenation. However, it had limitations such as strict call order, lack of token state storage, and missing support for Java, Kotlin, and Swift. The new Winq separates an interface layer and a core layer; the core stores token states and validates syntax, while the interface provides a high‑level, chainable API.

Example of a SELECT built with the new Winq:

WCDB::StatementSelect().select(WCTSequence.seq)
    .from("sqlite_sequence")
    .offset(100)
    .limit(10)
    .order(WCTSequence.seq)
    .where(WCTSequence.seq > 1000);

The new Winq supports all 26 SQLite statements and 23 token types, eliminating SQL injection risks.

6. Safer Data Storage

WCDB introduces a new backup and repair scheme. By mapping table names to leaf‑page numbers and CRC values, it can recover data even when large portions of the file are corrupted. Incremental backup further reduces backup time: a 500 MB DB backs up in ~63 ms, and a 10 GB DB in under 0.9 s, with a repair rate above 99%.

WCDB also minimizes the window for external logic to corrupt the DB by opening the main DB file read‑only during normal operations and only opening it writable during checkpoint.

7. Flexible Data Extension

WCDB adds data migration, compression, and automatic column addition capabilities.

Data Migration

WCDB creates a temporary view that merges old and new tables, allowing developers to work as if migration were already complete. It preprocesses CRUD statements to route operations to both tables when necessary, handling INSERT, UPDATE, DELETE, and SELECT transparently.

Data Compression

WCDB uses Zstd dictionary compression to shrink large serialized fields (XML/JSON/PB). Compression and decompression are performed automatically during CRUD operations, and existing data can be compressed asynchronously without blocking the UI.

Automatic Column Addition

When a query fails due to an unknown column, WCDB checks the corresponding ORM class for a newly defined column, adds it to the table, and retries the operation, eliminating manual schema updates.

8. Extreme Performance Optimizations

FTS5 Optimization

WCDB fully supports SQLite FTS5, adding custom tokenizers and pinyin search, improving full‑text search performance.

Interruptible Transactions

Long transactions are broken into short loops that commit periodically or when the main thread is blocked, preventing UI stalls while preserving most of the transaction’s atomicity.

WAL Header Update Optimization

WCDB defers WAL header fsync to checkpoint time when possible, reducing UI‑visible latency and cutting iOS WeChat’s stutter by 5‑10%.

9. Open Source

The new WCDB is available on GitHub: https://github.com/Tencent/wcdb . Stars are welcome!

10. Summary

WCDB now supports C++, Java, Kotlin, Swift, and Objective‑C across Android, iOS, Windows, macOS, and Linux. It offers powerful multi‑language ORM, robust backup, migration, compression, and performance features, making it a comprehensive client‑side database solution.