How Mars xlog Delivers High‑Performance, Secure Logging on Mobile

Introduction

Mars is a platform‑agnostic C++ component used by WeChat on Android, iOS, macOS, Windows, and WP. It includes independent libraries such as comm (socket, thread, message queue), xlog (logging), sdt (network diagnostics), and stn (signalling).

Why Logging Matters for Mobile Developers

When users report crashes, the lack of logs makes root‑cause analysis impossible. Therefore a logging solution must be active throughout the app lifecycle without degrading performance.

Conventional Approach

Write each log line to an encrypted file.

This method incurs heavy GC and I/O, causing noticeable UI jank. In release builds developers often disable logging, forcing them to rebuild the app with logging enabled for each crash, which is inefficient.

Direct file writes also block the thread because the OS first writes to a dirty page cache and later flushes it to disk under conditions such as periodic write‑back, memory pressure, or explicit sync calls. This uncontrolled flush leads to performance bottlenecks.

Rethinking the Design

To avoid I/O spikes, logs can be buffered in memory, compressed, then encrypted before being written to disk. Using C++ avoids Java‑side GC and enables a truly cross‑platform module.

Store logs in an in‑memory buffer, compress and encrypt them, and write to a file when the buffer reaches a threshold.

While this improves performance, it introduces the risk of lost logs if the process crashes before the buffer is flushed.

Introducing mmap

Memory‑mapped files let the program treat a file as memory, eliminating extra copies between user and kernel space. Tests writing 512 B blocks 1 000 000 times show mmap performance is comparable to pure memory writes.

mmap also provides controllable write‑back conditions: low memory, process crash, explicit msync / munmap, or periodic sync (e.g., 30‑60 s on FreeBSD).

Compression Strategy

The chosen scheme uses a two‑stage process: first LZ77‑style phrase compression, then Huffman coding. This achieves up to 86.3 % reduction. Four variants were benchmarked; the best two reach about 83.7 % compression while preserving stream‑wise decompression, so a single corrupted block does not affect earlier logs.

Micro‑second‑level per‑log overhead makes the impact on UI smoothness negligible, while spreading CPU usage evenly across the app’s lifetime.

xlog Solution Summary

Compress each log line on the fly, encrypt it, and write it into an mmap‑backed buffer.

This approach satisfies smoothness, completeness, and fault tolerance with an acceptable 83.7 % compression ratio. Logs are retained unless the storage device fails or runs out of space.

Platform‑Specific Pitfalls

iOS: after device lock the file may become read‑only; set NSFileProtectionNone to allow writes.

Boost‑based mmap on some platforms creates sparse files; when storage is exhausted, SIGBUS can occur. Pre‑filling the file with zeros avoids the crash.

Additional Strategies

Clear logs on app start to limit disk usage.

Support a cache directory for SD‑card‑less devices and flush to external storage when available.

Logging API Features

Type‑safe formatting: %s %d (e.g., xinfo("%s %d", "test", 1)).

Positional placeholders: %0 %1.

Lazy smart matching: %_.

Conclusion

A robust mobile logging module must guarantee four qualities: security, smoothness, completeness, and fault tolerance. Direct file writes break smoothness; pure memory buffers risk loss. By combining mmap with stream‑compression and encryption, Mars xlog achieves high performance, high compression, and reliable log retention without sacrificing user experience.