Mastering nsenter: Debug Container Networks and Linux Namespaces

What is nsenter?

nsenter is a utility from the

util-linux

package that runs a specified program inside the namespace(s) of a target process.

Typical Use Cases

The most common use case is entering a container's network namespace to debug networking without relying on limited container tools like

ip address

,

ping

,

telnet

, or

tcpdump

. By using nsenter, you can leverage host tools to inspect the container's network stack.

nsenter can also enter other namespaces such as

mnt

,

uts

,

ipc

,

pid

, and

user

, and you can specify a root directory and working directory for the executed program.

Usage

Basic syntax:

<code>nsenter [options] [program [arguments]]</code>

Options include:

<code>-t, --target pid          Target process ID</code>
<code>-m, --mount[=file]       Enter mount namespace</code>
<code>-u, --uts[=file]         Enter UTS namespace</code>
<code>-i, --ipc[=file]         Enter IPC namespace</code>
<code>-n, --net[=file]         Enter network namespace</code>
<code>-p, --pid[=file]         Enter PID namespace</code>
<code>-U, --user[=file]        Enter user namespace</code>
<code>-G, --setgid gid         Set GID for the program</code>
<code>-S, --setuid uid         Set UID for the program</code>
<code>-r, --root[=directory]   Set root directory</code>
<code>-w, --wd[=directory]     Set working directory</code>
<code>If no program is given, $SHELL is executed.</code>

Example:

<code># Get the PID of an nginx container</code>
<code>docker inspect -f {{.State.Pid}} nginx</code>
<code># Enter its network namespace</code>
<code>nsenter -n -t5645</code>
<code># Verify network interfaces</code>
<code>ip addr</code>

In Kubernetes, obtain the container ID first, then retrieve its PID before using nsenter.

Linux Namespaces Overview

Namespaces isolate various resources for processes:

mount : Separate mount points (since Linux 2.4.19)

ipc : Separate IPC objects like message queues and shared memory (since 2.6.19)

uts : Separate hostname and domain name (since 2.6.19)

net : Separate network stack (since 2.6.24)

pid : Separate PID number space (since 2.6.24)

user : Separate user and group IDs (since 2.6.23, removed in 3.8)

cgroup : Separate control groups (since 4.6)

Each process’s namespaces are visible under

/proc/PID/ns

.

clone System Call

The

clone

system call creates a new process, optionally sharing resources and specifying namespaces via flags such as

CLONE_NEWNET

,

CLONE_NEWUTS

, etc.

<code>#define _GNU_SOURCE</code>
<code>#include <sched.h></code>
<code>int clone(int (*fn)(void *), void *child_stack, int flags, void *arg, ...);</code>

Example flag usage:

CLONE_NEWCGROUP – cgroup namespace

CLONE_NEWIPC – IPC namespace

CLONE_NEWNET – network namespace

CLONE_NEWNS – mount namespace

CLONE_NEWPID – PID namespace

CLONE_NEWUSER – user namespace

CLONE_NEWUTS – UTS namespace

<code>pid = clone(childFunc, stackTop, CLONE_NEWUTS | SIGCHLD, argv[1]);</code>

setns System Call

setns

attaches the calling thread to an existing namespace identified by a file descriptor.

<code>#define _GNU_SOURCE</code>
<code>#include <sched.h></code>
<code>int setns(int fd, int nstype);</code>

Typical usage involves opening

/proc/PID/ns/FILE

and calling

setns(fd, 0)

to join all namespaces, then executing a program with

execvp

.

<code>#define _GNU_SOURCE</code>
<code>#include <fcntl.h></code>
<code>#include <sched.h></code>
<code>#include <unistd.h></code>
<code>#include <stdlib.h></code>
<code>#include <stdio.h></code>
<code>int main(int argc, char *argv[]) {</code>
<code>    int fd;</code>
<code>    if (argc < 3) {</code>
<code>        fprintf(stderr, "%s /proc/PID/ns/FILE cmd args...\n", argv[0]);</code>
<code>        exit(EXIT_FAILURE);</code>
<code>    }</code>
<code>    fd = open(argv[1], O_RDONLY);</code>
<code>    if (fd == -1) perror("open");</code>
<code>    if (setns(fd, 0) == -1) perror("setns");</code>
<code>    execvp(argv[2], &amp;argv[2]);</code>
<code>    perror("execvp");</code>
<code>}</code>

Run it like:

<code>./ns_exec /proc/3550/ns/uts /bin/bash</code>

nsenter Wrapper

nsenter simplifies setns usage by allowing you to specify a target PID and desired namespaces directly, without manually handling file descriptors.

References

Container network debugging: https://tencentcloudcontainerteam.github.io/tke-handbook/skill/capture-packets-in-container.html

nsenter man page: http://www.man7.org/linux/man-pages/man1/nsenter.1.html

clone man page: http://www.man7.org/linux/man-pages/man2/clone.2.html

setns man page: http://www.man7.org/linux/man-pages/man2/setns.2.html