Mastering kdump: From Theory to Hands‑On Linux Crash Dump Configuration

Kernel crashes are critical failures that can halt services and make troubleshooting difficult; kdump is a built‑in Linux tool that captures a full memory dump at the moment of a crash, providing a "black box" for post‑mortem analysis.

What is kdump?

Kdump reserves a portion of physical memory for a secondary "capture" kernel that loads when the primary kernel crashes. The capture kernel saves the entire memory state—including kernel data structures, process stacks, memory mappings, and registers—to a core dump file (usually named vmcore), which can later be examined to pinpoint the cause of the crash.

How kdump Works

Dual‑kernel cooperation

Two kernels cooperate: the production kernel runs normally, while the capture kernel stays dormant in reserved memory. When a fatal error occurs, control switches instantly to the capture kernel, which preserves the memory image before the system reboots.

Role of kexec

Kexec allows loading and booting a new kernel without a full BIOS reboot, enabling a rapid hand‑off to the capture kernel and preventing loss of crash data.

Memory reservation

The crashkernel boot parameter defines the size of the reserved area (e.g., crashkernel=256M). Dynamic reservations such as crashkernel=256M-2G:256M,2G-:512M adapt to total system memory.

Practical kdump Setup

1. Install kexec‑tools

RHEL/CentOS: sudo yum install kexec-tools Debian/Ubuntu:

sudo apt-get install kexec-tools

2. Configure kernel parameters

Edit /etc/default/grub and add a crashkernel entry, then rebuild the GRUB config:

GRUB_CMDLINE_LINUX="crashkernel=256M quiet"
# For BIOS systems
sudo grub2-mkconfig -o /boot/grub2/grub.cfg
# For UEFI systems
sudo grub2-mkconfig -o /boot/efi/EFI/redhat/grub.cfg

3. Edit /etc/kdump.conf

path /var/crash

– where vmcore is stored. core_collector makedumpfile -c -l -message-level 1 -d 31 – compresses and filters the dump. default reboot – automatically reboots after dump creation.

4. Enable and start the service

sudo systemctl start kdump.service
sudo systemctl enable kdump.service

Verify status with systemctl status kdump.service.

5. Verify the configuration

Trigger a crash in a test environment:

echo 1 > /proc/sys/kernel/sysrq
echo c > /proc/sysrq-trigger

After reboot, check /var/crash for a timestamped directory containing vmcore.

Analyzing vmcore Files

Use the crash utility (e.g.,

crash /usr/lib/debug/lib/modules/$(uname -r)/vmlinux /var/crash/vmcore

) to inspect stack traces ( bt), process lists ( ps), and memory maps ( vm). GDB can also be employed for deeper inspection, though crash is specialized for kernel dumps.

Case Study

A production server crashed unexpectedly. After kdump captured vmcore, the team used crash to run bt, revealing a null‑pointer dereference in some_function of the business application. Further ps analysis confirmed the affected process. The root cause was a logic error that passed a null pointer under specific conditions; fixing the code restored service.

This example demonstrates how kdump, combined with analysis tools, enables rapid, accurate fault isolation and improves system reliability.