Mastering nsenter: Debug Container Networks and Linux Namespaces

What is nsenter?

nsenter is a utility from the util-linux package that runs a specified program inside the namespace(s) of a target process.

Typical Use Cases

The most common use case is entering a container's network namespace to debug networking without relying on limited container tools like ip address, ping, telnet, or tcpdump. By using nsenter, you can leverage host tools to inspect the container's network stack.

nsenter can also enter other namespaces such as mnt, uts, ipc, pid, and user, and you can specify a root directory and working directory for the executed program.

Usage

Basic syntax: nsenter [options] [program [arguments]] Options include:

-t, --target pid          Target process ID</code>
<code>-m, --mount[=file]       Enter mount namespace</code>
<code>-u, --uts[=file]         Enter UTS namespace</code>
<code>-i, --ipc[=file]         Enter IPC namespace</code>
<code>-n, --net[=file]         Enter network namespace</code>
<code>-p, --pid[=file]         Enter PID namespace</code>
<code>-U, --user[=file]        Enter user namespace</code>
<code>-G, --setgid gid         Set GID for the program</code>
<code>-S, --setuid uid         Set UID for the program</code>
<code>-r, --root[=directory]   Set root directory</code>
<code>-w, --wd[=directory]     Set working directory</code>
<code>If no program is given, $SHELL is executed.

Example:

# Get the PID of an nginx container</code>
<code>docker inspect -f {{.State.Pid}} nginx</code>
<code># Enter its network namespace</code>
<code>nsenter -n -t5645</code>
<code># Verify network interfaces</code>
<code>ip addr

In Kubernetes, obtain the container ID first, then retrieve its PID before using nsenter.

Linux Namespaces Overview

Namespaces isolate various resources for processes:

mount : Separate mount points (since Linux 2.4.19)

ipc : Separate IPC objects like message queues and shared memory (since 2.6.19)

uts : Separate hostname and domain name (since 2.6.19)

net : Separate network stack (since 2.6.24)

pid : Separate PID number space (since 2.6.24)

user : Separate user and group IDs (since 2.6.23, removed in 3.8)

cgroup : Separate control groups (since 4.6)

Each process’s namespaces are visible under /proc/PID/ns.

clone System Call

The clone system call creates a new process, optionally sharing resources and specifying namespaces via flags such as CLONE_NEWNET, CLONE_NEWUTS, etc.

#define _GNU_SOURCE</code>
<code>#include <sched.h></code>
<code>int clone(int (*fn)(void *), void *child_stack, int flags, void *arg, ...);

Example flag usage:

CLONE_NEWCGROUP – cgroup namespace

CLONE_NEWIPC – IPC namespace

CLONE_NEWNET – network namespace

CLONE_NEWNS – mount namespace

CLONE_NEWPID – PID namespace

CLONE_NEWUSER – user namespace

CLONE_NEWUTS – UTS namespace

pid = clone(childFunc, stackTop, CLONE_NEWUTS | SIGCHLD, argv[1]);

setns System Call

setns

attaches the calling thread to an existing namespace identified by a file descriptor.

#define _GNU_SOURCE</code>
<code>#include <sched.h></code>
<code>int setns(int fd, int nstype);

Typical usage involves opening /proc/PID/ns/FILE and calling setns(fd, 0) to join all namespaces, then executing a program with execvp.

#define _GNU_SOURCE</code>
<code>#include <fcntl.h></code>
<code>#include <sched.h></code>
<code>#include <unistd.h></code>
<code>#include <stdlib.h></code>
<code>#include <stdio.h></code>
<code>int main(int argc, char *argv[]) {</code>
<code>    int fd;</code>
<code>    if (argc < 3) {</code>
<code>        fprintf(stderr, "%s /proc/PID/ns/FILE cmd args...
", argv[0]);</code>
<code>        exit(EXIT_FAILURE);</code>
<code>    }</code>
<code>    fd = open(argv[1], O_RDONLY);</code>
<code>    if (fd == -1) perror("open");</code>
<code>    if (setns(fd, 0) == -1) perror("setns");</code>
<code>    execvp(argv[2], &argv[2]);</code>
<code>    perror("execvp");</code>
<code>}

Run it like:

./ns_exec /proc/3550/ns/uts /bin/bash

nsenter Wrapper

nsenter simplifies setns usage by allowing you to specify a target PID and desired namespaces directly, without manually handling file descriptors.

References

Container network debugging: https://tencentcloudcontainerteam.github.io/tke-handbook/skill/capture-packets-in-container.html

nsenter man page: http://www.man7.org/linux/man-pages/man1/nsenter.1.html

clone man page: http://www.man7.org/linux/man-pages/man2/clone.2.html

setns man page: http://www.man7.org/linux/man-pages/man2/setns.2.html