Understanding Linux printk: From Kernel Ring Buffer to User‑Space Log Retrieval

Kernel‑space implementation of printk

In the Linux kernel, printk is the primary logging function, similar to printf in C. Developers typically use its wrapper macros such as pr_err, pr_info, and pr_debug, which add log levels and module information before ultimately calling printk. printk writes each log entry into a dedicated kernel ring buffer, an array‑based circular queue. When the buffer becomes full, new entries overwrite the oldest ones. The size of this ring buffer can be adjusted via kernel parameters.

After placing a message in the ring buffer, the kernel also forwards it to the system console through console‑related mechanisms, though the details of that path are omitted here.

User‑space methods to view kernel logs

Linux provides several ways to read the messages stored by printk from user space:

dmesg : By default reads /dev/kmsg. When invoked, it opens the device, obtains a private seq pointer to the next unread entry in the ring buffer, and repeatedly calls read, advancing seq after each line until all available logs are consumed.

cat /proc/kmsg : Similar to dmesg but reads /proc/kmsg. The key difference is that /proc/kmsg uses a single global syslog_seq variable, so concurrent readers can interfere with each other, leading to incomplete logs. This is why dmesg prefers /dev/kmsg.

klogctl : A glibc wrapper around the syslog system call. By passing SYSLOG_ACTION_READ, it mimics the behavior of cat /proc/kmsg. The underlying kernel code for klogctl also uses syslog_seq, inheriting the same concurrency limitation.

System console : The kernel can also push logs directly to the active console. A separate console_seq tracks the next message to display. Console output respects the kernel's log level filter (default level 7, showing messages of priority debug and higher). The log level can be changed via the loglevel kernel parameter.

Implications for developers and system administrators

Understanding printk and how its output can be accessed is crucial for kernel developers and anyone troubleshooting low‑level system issues. For example, when the OOM killer terminates a process, the kernel logs a line with pr_err. By examining dmesg output, one can confirm that a process was killed due to memory exhaustion.

Beyond fatal events, many error‑level and higher logs provide early warnings of hardware failures, driver problems, or misbehaving modules. Monitoring these logs enables timely intervention before a problem escalates.

All related diagrams, source code snippets, and further explanations are available in the accompanying GitHub repository:

https://github.com/wangyuntao/linux-kernel-illustrated

In summary, printk stores kernel messages in a ring buffer, which can be read via dmesg, /proc/kmsg, klogctl, or the system console; each method has its own semantics and limitations, and choosing the right tool helps developers diagnose and resolve kernel‑level issues effectively.