Root Cause Analysis and Resolution of Periodic Crashes Caused by File Handle Leak in Didi Android Delivery Merchant App

Didi Internationalized Android delivery merchant client experienced sudden large‑scale crash alerts after a stable release period. Initial investigation showed crashes originated from the native libpush.so library. After upgrading the underlying .so version, only sporadic crashes remained, which later escalated into a second massive crash wave.

The crash volume peaked at about 50 incidents per day during the first wave and up to 173 incidents in the second wave, with a one‑to‑one correlation between crashes and affected users. All incidents were concentrated on three Huawei device models after roughly 14 days of runtime.

Analysis : The crash address was consistently 0x000000000007ce08 , pointing to a specific code block in libpush.so . Using IDA, the problematic code was identified as a call to fopen that returned null, followed by an unchecked fwrite operation. Since the system API itself is reliable, the failure was attributed to an abnormal scenario where the file could not be opened.

The underlying issue was a file‑handle (FD) leak. Each device opened about 19 handles every 6 hours without proper closure, eventually exceeding the per‑process limit (1024 on the affected custom devices, lower than typical Android limits of 32768). When the limit was reached, the app crashed with errors such as "Could not allocate dup blob fd" and "Too many open files".

To reproduce the issue, the team shortened the 6‑hour polling interval to 2 minutes, causing the FD count to hit the limit quickly and generating identical crash logs.

Code used to list open file descriptors:

private void listFd() {
    String tag = "FD_TAG";
    File fdFile = new File("/proc/" + android.os.Process.myPid() + "/fd");
    File[] files = fdFile.listFiles();
    int length = files.length;
    MerchantLogUtils.logFd(tag, "fd length: " + length);
    StringBuilder sb = new StringBuilder();
    for (int i = 0; i < length; i++) {
        File f = files[i];
        String strFile = Os.readlink(f.getAbsolutePath());
        sb.append(strFile + "\n");
    }
}

Mitigation & Prevention : The fix involved correcting the null‑check before fwrite and ensuring proper handle closure. Recommendations include:

Audit all handle‑related code (HandlerThread, IO streams, SQLite cursors, InputChannel, Bitmap IPC) and enforce release in finally blocks.

Integrate handle‑leak checks into code review checklists and QA test cycles.

Monitor FD counts in production (e.g., adb shell ulimit -n ) and set alert thresholds.

Automate long‑running tests that record FD growth.

By combining crash‑platform data, IDA analysis, version diffing, and systematic FD monitoring, the team pinpointed the root cause, applied the fix, and verified stability in subsequent releases.