How WeChat’s XLog Achieves High‑Performance, Secure Logging on Mobile

Source: WeChat client development team

Preface

Mars is WeChat's official terminal foundation component written in C++, platform‑independent and business‑agnostic, already integrated into Android, iOS, Mac, Windows, WP clients and now being open‑sourced. It includes four parts: comm (common library), xlog (logging module), sdt (network diagnostics), and stn (signal distribution).

Introduction

For mobile developers, the biggest pain point is debugging issues reported by users when no logs are available. A reliable logging solution must work throughout the app lifecycle without degrading performance.

Conventional Approach

Solution: encrypt each log line and write to a file.

Implementing this in Java on Android causes heavy GC and intensive I/O, leading to noticeable UI jank. In release builds the log is often disabled, forcing developers to rebuild the app to collect logs, which is inefficient.

Writing directly to a file stalls the app because data is first written to the system cache (dirty page) and later flushed to disk under conditions such as timed write‑back, memory pressure, or low memory, causing two copies of data and frequent kernel‑user space switches.

The main drawback is reduced smoothness —the app becomes less responsive, which is unacceptable for user experience.

Further Considerations

To avoid heavy I/O, logs can be buffered in memory, compressed, then encrypted before being written to disk. Using C++ avoids Java GC and enables a cross‑platform module.

Solution: buffer logs in memory, compress and encrypt when a size threshold is reached, then write to a file.

This approach improves release‑time logging but introduces the risk of lost logs if the app crashes before the buffer is flushed.

Android can use shared memory as a buffer, but newer Android versions restrict this, and other platforms lack a comparable mechanism.

Thus a logging system must ensure smoothness , integrity , and fault tolerance throughout the app lifecycle.

Mars' XLog Module

XLog combines the advantages of in‑memory buffering and reliable file storage using mmap.

mmap

mmap maps a file directly into the process address space, eliminating extra data copies and reducing kernel‑user switches.

Performance test (512 Byte writes, 150 KB buffer, 1 M iterations) shows mmap performance is comparable to pure memory writes while preserving reliability.

mmap flushes occur on memory pressure, process crash, explicit msync / munmap, or periodic sync (30‑60 s on FreeBSD).

However, mmap still requires handling CPU spikes from batch compression and ensuring log security.

Compression

We use phrase‑based compression (LZ77) followed by Huffman coding, achieving up to 86.3 % reduction. Stream‑oriented compression limits the impact of corrupted blocks, keeping overall log integrity.

Four compression schemes were evaluated; the two stream‑oriented methods reached ~83.7 % compression while allowing partial recovery after data loss.

Benchmarks on three Android devices show the per‑line overhead of stream compression is only a few microseconds, far less than the UI‑visible threshold, and it smooths CPU usage compared to bulk compression.

XLog Summary

Log each line: compress → encrypt → write into an mmap‑backed buffer.

This achieves high smoothness , integrity , and fault tolerance with an acceptable 83.7 % compression ratio, and only fails when the storage medium is unavailable.

Platform‑specific pitfalls include iOS file‑protection settings requiring NSFileProtectionNone, and sparse‑file mmap on Android that can raise SIGBUS when storage is exhausted (solved by pre‑filling the file with zeros).

Additional strategies: log cleanup on startup, configurable cache directories for removable storage, and multiple formatting APIs (type‑safe, indexed, and lazy matching).

Conclusion

Effective mobile logging must balance smoothness , integrity , fault tolerance , and security . XLog meets these goals by using memory‑mapped files, stream compression, and encryption, delivering high performance without sacrificing reliability.