Mastering Android OOM: Thread, File, and Memory Leak Solutions

OOM Problem Classification

Online OOM issues can be roughly divided into three categories: excessive thread count, too many open files, and insufficient memory.

1. Excessive Thread Count

1.1 Error Message

pthread_create (1040KB stack) failed: Out of memory

This typical error occurs when creating a new thread fails.

Android limits the maximum number of threads per process via /proc/sys/kernel/threads-max. When the number of threads exceeds this limit, OOM is triggered.

1.2 Source Analysis

void Thread::CreateNativeThread(JNIEnv* env, jobject java_peer, size_t stack_size, bool is_daemon) {
    ...
    pthread_create_result = pthread_create(...);
    if (pthread_create_result == 0) {
        return;
    }
    // creation failed
    {
        std::string msg(...);
        ScopedObjectAccess soa(env);
        soa.Self()->ThrowOutOfMemoryError(msg.c_str());
    }
}

1.3 Thread Optimization

Traditional solution: forbid new Thread and enforce thread‑pool usage. However, legacy code and third‑party libraries cannot be changed directly.

1.3.1 Non‑intrusive new Thread Replacement

Define a ShadowThread subclass that overrides start() to submit tasks to a custom thread pool. Use bytecode instrumentation (ASM) to replace every new Thread with new ShadowThread at compile time.

public class ShadowThread extends Thread {
    @Override
    public synchronized void start() {
        Log.i("ShadowThread", "start,name=" + getName());
        CustomThreadPool.THREAD_POOL_EXECUTOR.execute(new MyRunnable(getName()));
    }

    class MyRunnable implements Runnable {
        String name;
        public MyRunnable(String name) { this.name = name; }
        @Override
        public void run() {
            try {
                ShadowThread.this.run();
                Log.d("ShadowThread", "run name=" + name);
            } catch (Exception e) {
                Log.w("ShadowThread", "name=" + name + ",exception:" + e.getMessage());
                RuntimeException exception = new RuntimeException("threadName=" + name + ",exception:" + e.getMessage());
                exception.setStackTrace(e.getStackTrace());
                throw exception;
            }
        }
    }
}

After instrumentation, the original new Thread calls are replaced, reducing the thread‑count peak.

1.3.2 Thread‑Pool Parameter Tuning

Key parameters of ThreadPoolExecutor:

corePoolSize : core threads, not released unless allowCoreThreadTimeOut is true.

maximumPoolSize : maximum threads.

keepAliveTime : idle thread lifetime.

…

Typical optimizations:

Set a small keepAliveTime (1‑3 s).

Enable allowCoreThreadTimeOut(true).

executor.allowCoreThreadTimeOut(true)

1.4 Thread Leak Monitoring

Hook native thread lifecycle functions ( pthread_create, pthread_detach, pthread_join, pthread_exit) to record creation, stack trace, and detect joinable threads that exit without detach/join.

In Linux, a joinable thread retains its stack and descriptor until pthread_join is called.

2. Too Many Open Files

2.1 Error Message

E/art: ashmem_create_region failed for 'indirect ref table': Too many open files
java.lang.OutOfMemoryError: Could not allocate JNI Env

Android inherits Linux file‑descriptor limits. /proc/pid/limits shows Max open files, often 1024 on low‑end devices. Use ulimit -n to view the limit.

2.2 File‑Descriptor Monitoring

When the descriptor count exceeds 1000 or grows continuously by 50, collect the paths and report.

private fun dumpFd() {
    val fdNames = runCatching { File("/proc/self/fd").listFiles() }
        .getOrElse { return emptyArray() }
        ?.map { runCatching { Os.readlink(it.path) }.getOrElse { "failed to read link ${it.path}" } }
        ?: emptyList()
    Log.d("TAG", "dumpFd: size=${fdNames.size},fdNames=$fdNames")
}

2.3 IO Monitoring

Monitor open, read, write, close calls.

open : record file name, fd, size, stack, thread.

read/write : record type, count, total size, buffer size, latency.

close : total time, max continuous read/write time.

2.3.1 Java Hook

In Android 6.0,

FileInputStream → IoBridge.open → Libcore.os.open → BlockGuardOs.open → Posix.open

. Hook Libcore.io.Libcore.os via reflection and dynamic proxy to intercept IO methods.

// Reflect Libcore.os
Class<?> clibcore = Class.forName("libcore.io.Libcore");
Field fos = clibcore.getDeclaredField("os");
// Create proxy
Proxy.newProxyInstance(cPosix.getClassLoader(), getAllInterfaces(cPosix), this);

Drawbacks: performance overhead, cannot monitor native code, version compatibility issues.

2.3.2 Native Hook

Hook functions in libc.so such as open, read, write, close. Use frameworks like xhook or bhook. Example target libraries: libjavacore.so, libopenjdkjvm.so.

int open(const char *pathname, int flags, mode_t mode);
ssize_t read(int fd, void *buf, size_t size);
ssize_t write(int fd, const void *buf, size_t size);
int close(int fd);

3. Insufficient Memory

3.1 Heap OOM

Java heap exhaustion is common on Android 7.0 devices. Large arrays and Bitmap pixel data (stored in the heap on Android 3.0‑7.0) are typical culprits.

3.2 JVM Memory Layout

Method area

Program counter

Java stack

Native stack

Heap

3.3 Image Loading Optimization

3.3.1 Conventional Techniques

Use soft references, onLowMemory, and control Bitmap pixel storage.

3.3.2 Non‑intrusive Automatic Compression

During Gradle’s mergeResourcesTask, add a task that scans all resources, compresses image files, and replaces originals if the size shrinks.

3.4 Big‑Image Monitoring

Hook image‑loading frameworks (Glide, Picasso, Fresco, Coil) to register listeners that detect when a loaded bitmap exceeds the view size. Alternatively, replace ImageView with a custom subclass that overrides setImageDrawable, setImageBitmap, etc., and performs size checks on the UI thread’s idle handler.

4. Memory Leak Monitoring

4.1 LeakCanary (Debug)

Add

debugImplementation 'com.squareup.leakcanary:leakcanary-android:2.8.1'

to detect leaks in debug builds. It dumps an HPROF snapshot, which freezes the app for several seconds.

4.2 ResourceCanary (WeChat)

Separates detection and analysis; detection runs on‑device, analysis runs on a server.

4.3 KOOM (Online)

Periodically (every 5 s) checks memory usage. When usage exceeds a threshold (e.g., 80 % of max heap) or rises sharply, it forks a child process, calls Debug.dumpHprofData, and resumes the parent. The child writes the heap dump, then exits. The parent waits via waitpid.

// Simplified flow
int pid = suspendAndFork(); // parent forks child
if (pid == 0) {
    Debug.dumpHprofData(path);
    exitProcess();
} else if (pid > 0) {
    resumeAndWait(pid);
}

The dump is analyzed in a separate service using the shark library to build a HeapGraph, filter leaking objects, find paths to GC roots, and generate a JSON report.

5. Native Memory Leak Monitoring

Hook allocation functions ( malloc, realloc, calloc, memalign, posix_memalign) and free. Record stack trace, size, address, and thread in a map on allocation; remove the entry on free. Periodically run a mark‑and‑sweep to find unreachable native blocks and retrieve their allocation metadata.

Conclusion

The article classifies OOM into thread‑count, file‑descriptor, and heap‑memory categories, and presents mature open‑source solutions from large companies for each case, including non‑intrusive thread replacement, thread‑pool tuning, file‑descriptor and I/O monitoring, automatic image compression, big‑image detection, Java and native memory‑leak monitoring, and online heap‑dump analysis.