Debug Linux Core Dumps with dmesg, addr2line, gdb, and strace

What is a core dump?

A core dump is a file generated when a Linux process terminates abnormally (e.g., segmentation fault). It contains a snapshot of the process memory, register state, stack pointer, and call stack, enabling post‑mortem analysis of the crash.

Enabling core dumps

By default many distributions limit core file size to zero. Enable unlimited size for the current shell: ulimit -c unlimited Optionally, ensure the kernel’s core pattern allows writing files (e.g., echo "core.%p" > /proc/sys/kernel/core_pattern).

Generating a core dump – a minimal example

Source code that crashes:

#include <stdio.h>
int main(){
    int *p = NULL;
    *p = 0;               // dereference null pointer → SIGSEGV
    printf("bad
");
    return 0;
}

Compile with debugging symbols and run:

g++ -g -o crash main.cpp
./crash
# Segmentation fault (core dumped)
ls -l
# crash  core.1234  main.cpp

Finding the faulting address with dmesg and mapping it with addr2line

After the crash, the kernel logs the instruction pointer (IP) that caused the fault: dmesg | grep crash Typical output contains a line such as:

[ 212.33] crash[1234]: segfault at 0 ip 0000000000400571 sp 00007ffdf0aafbb0 error 6 in crash[400000+1000]

Extract the IP address ( 0000000000400571) and translate it to a source location: addr2line -e crash 0000000000400571 Result: /path/to/main.cpp:6 This points directly to the line that dereferenced the null pointer.

Debugging the core file with gdb

Load the executable and the core file together: gdb crash core.1234 Useful gdb commands: bt – display the backtrace (stack frames). info locals – show local variables of the current frame. info args – show function arguments. p var – print the value of a variable (replace var with the actual name). info registers – inspect register contents. quit – exit gdb.

Running bt immediately after loading the core shows the offending line, e.g.:

#0  0x0000000000400571 in main () at main.cpp:6

Using strace together with addr2line

strace

records system calls and signals. Execute the program under strace to capture the SIGSEGV address: strace -i ./crash Typical fragment of the output:

[0000000000400571] --- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=NULL} ---

Translate the address to source code: addr2line -e crash 0000000000400571 Result again points to main.cpp:6. This method is handy when you also need to see the sequence of system calls leading to the crash.

Key points

Enable core dumps ( ulimit -c unlimited) before running potentially unstable code.

Compile with -g to retain symbol information.

Use dmesg or strace to obtain the faulting instruction address.

Map the address to source with addr2line -e <em>executable</em> <em>address</em>.

Load the core file into gdb for interactive inspection (backtrace, locals, registers, etc.).