Design and Implementation of a Mobile Log Retrieval Component

Logs are essential for recording user behavior, network diagnostics, and device status, but mobile logs often reside only on the device and cannot be retrieved remotely when users report obscure issues. To improve troubleshooting efficiency, a foundational mobile log retrieval component was developed.

Requirements

Security and stability: logs must be protected from sensitive data leakage and must not affect app performance.

Completeness: logs should survive crashes, kills, and other extreme scenarios.

Accurate retrieval: enable precise log collection across multiple users and devices.

Efficient storage: retain 15–30 days of logs while minimizing device storage usage.

Technical Solution

Overall Design

The system collects internal mobile logs, writes them locally, and when a user reports an issue, the management platform tags the user by phone number or identifier. The SDK then uploads the relevant logs for analysis. The SDK supports both iOS and Android.

Log Writing

The writing module is implemented in C/C++ for cross‑platform use, employing a private log protocol with asynchronous multi‑file writes, log levels, automatic timestamp/thread/method injection, and Android multi‑process support.

It uses mmap to map a file into memory; if mapping fails, an in‑memory buffer is used. Data is streamed through compression and encryption before being flushed to disk when the cache threshold is reached or when explicitly requested.

Why mmap? mmap eliminates a copy between user and kernel space, reduces context switches, and automatically writes back to disk on memory pressure, process crash, msync / munmap , or after a timeout, thus preventing log loss.

Why stream compression? Full‑file compression yields higher ratios but spikes CPU usage and storage. Stream compression compresses logs in small chunks (≈5 KB) as they are written, lowering peak CPU load and storage impact, at the cost of slightly lower compression ratios.

Multi‑process handling

Instead of a slow ContentProvider, the design adopts a per‑process file strategy to avoid performance bottlenecks while ensuring data consistency across processes.

Log Retrieval

The retrieval mechanism primarily uses short‑lived connections triggered on app launch or background entry, supplemented by long‑lived silent push (WPush) for real‑time needs. All network traffic is encrypted with asymmetric keys protecting the symmetric session key.

Two retrieval models are discussed:

User‑centric : tags users and pushes log upload commands, but struggles with multi‑device login scenarios.

Device‑centric : maps users to device fingerprints (device + BundleID + timestamp) to uniquely identify logs, allowing accurate retrieval even when multiple users share a device or a user uses multiple devices.

Management Platform

The platform provides a simple UI for marking, querying, and downloading user logs, displaying multi‑device mappings, and supporting filters by package name, BundleID, platform, and time range.

Conclusion

The article described the implementation of log writing, retrieval, and management components for the 58 client, addressing remote log collection challenges and outlining areas for future improvement.