Unlock the Secrets of Kernel Module Makefiles: 3 Debugging Tricks Revealed

1. Familiar Yet Unfamiliar Kernel Module Makefile

When beginners write a Hello World kernel module, they encounter a Makefile that most developers think they know well. Using a small example from Git, we revisit this familiar Makefile.

$ git clone https://gitee.com/os-newbie/module_makefile.git

The repository contains libbie.c, libbie.h, Makefile, and newbie.c. The Makefile content is:

ifneq ($(KERNELRELEASE),)
obj-m+=newbs.o
newbs-objs:=newbie.o libbie.o
else
KERNELVER:=$(shell uname -r)
KERNELDIR:=/usr/src/kernels/$(KERNELVER)/
PWD:=$(shell pwd)
default:
	$(MAKE) -C $(KERNELDIR) M=$(PWD) modules
clean:
	$(MAKE) -C $(KERNELDIR) M=$(PWD) clean
endif

Running make in the directory compiles the module and produces newbs.ko. The build output includes lines like CC, LD, Entering directory, Leaving directory, Building modules, and MODPOST 1 modules, whose meanings are often unclear even to experienced developers.

2. Makefile Debugging Techniques

2.1 Small Example

We prepare a minimal Makefile to illustrate three debugging tricks.

$ git clone https://gitee.com/os-newbie/makefile_debug.git
$ cd makefile_debug/
$ cat Makefile
aa:=11
bb:=$(aa)
cc:=$(bb)
bb:=9999
export bb
include cfg_makefile
default: prqst
	@echo "Makefile default start"
	@echo "bb is $(bb)"
	@echo "cccc is $(cccc) in main makefile before"
	make -f sub_makefile
	@echo "cccc is $(cccc) in main makefile after"
	@echo "Makefile default end"
prqst:
	@echo "zz is $(zz)"
$ cat cfg_makefile
xx:=4444
yy:=$(xx)
zz:=$(yy)
$ cat sub_makefile
aaaa:=222222
bbbb:=$(aaaa)
cccc:=$(bbbb)
all:
	@echo "sub_makefile start"
	@echo "cccc is $(cccc)"
	@echo "bb is $(bb) in sub_makefile"
	@echo "sub_makefile end"

Running make shows the expected output. The key points are:

The make command implicitly uses Makefile (equivalent to make -f Makefile).

The include directive and recursive make -f sub_makefile connect the three Makefiles.

2.2 Recursive vs. Non‑Recursive Make

Large projects contain thousands of Makefiles. For the Linux kernel, find /usr/src/kernels/... -name Makefile | wc -l yields 1806 files. Recursive make calls itself with make -f sub_makefile, while non‑recursive includes files with include cfg_makefile. Linux uses recursive make; Android uses non‑recursive.

2.3 Debugging Trick 1 – Make Options

Use make --debug with various flags to get detailed logs. Example commands:

$ make --debug=a,m SHELL="bash -x" > make.log 2>&1   # Full debug
$ make --debug=v,m SHELL="bash -x" > make.log 2>&1   # Recommended
$ make --debug=v > make.log 2>&1   # Less verbose
$ make --debug=b > make.log 2>&1   # Minimal

Running with --debug=v,m and grepping adds line numbers for easier analysis.

2.4 Debugging Trick 2 – Variable Tracing

Extract final variable values with:

$ make -p 2>&1 | grep -A 1 '^# makefile' | grep -v '^--' | awk '/# makefile/&&/line/{getline n;print $0,";",n}' | LC_COLLATE=C sort -k4 -k6n | uniq > var.log

The resulting var.log shows each variable, its source file, line number, and final value (e.g., bb := 9999 from line 4).

2.5 Debugging Trick 3 – Intermediate Values

Insert $(warning $(var123)) between variable assignments to print intermediate values during the make process.

3. Kernel Module Makefile Analysis

3.1 Extracting Key Information

Using trick 1, we capture the overall build process:

$ cd ../module_makefile
$ echo '0:+ make' > make.log
$ make --debug=v,m SHELL="bash -x" 2>&1 | grep -n . >> make.log

Key sections include reading makefiles, updating makefiles, goal targets, considering targets, and invoking recipes. Summarized logs show four make invocations during the build.

3.2 Detailed Steps

The build proceeds as follows:

Run make in the module directory, which reads the module’s Makefile.

The -C /usr/src/kernels/… option switches to the kernel source directory, passing M=$(PWD) to build the module.

Inside the kernel source, the default target modules triggers make -f scripts/Makefile.build and make -f scripts/Makefile.modpost. Makefile.build compiles newbie.c and libbie.c into newbie.o and libbie.o, then links them into newbs.o. Makefile.modpost links newbs.o into the final newbs.ko module.

3.3 Understanding obj‑m and newbs‑objs

When KERNELRELEASE is set (inside the kernel source), the Makefile executes the ifneq branch, adding newbs.o to obj‑m and defining newbs‑objs as newbie.o libbie.o. The kernel’s scripts/Makefile.lib processes these variables to build composite objects.

Key Snippets from scripts/Makefile.lib

multi-used-y := $(sort $(foreach m,$(obj-y),$(if $(strip $($$(m:.o=-objs) $$(m:.o=-y))),$(m))))
multi-used-m := $(sort $(foreach m,$(obj-m),$(if $(strip $($$(m:.o=-objs) $$(m:.o=-y))),$(m))))
multi-used   := $(multi-used-y) $(multi-used-m)
multi-objs-y := $(foreach m,$(multi-used-y),$($$(m:.o=-objs)) $($$(m:.o=-y)))
multi-objs-m := $(foreach m,$(multi-used-m),$($$(m:.o=-objs)) $($$(m:.o=-y)))
multi-objs    := $(multi-objs-y) $(multi-objs-m)

These lines expand newbs‑objs to the actual source objects, enabling the kernel build system to compile and link them correctly.

4. Practical Takeaways

By applying the three debugging tricks, developers can:

Obtain a complete picture of the make process, including recursive calls.

Trace variable values and understand how kernel‑specific variables like KERNELRELEASE affect the flow.

Inspect intermediate values to demystify complex Makefile logic.

These techniques help both newcomers and seasoned kernel developers gain deeper insight into the compilation of kernel modules.