How to Use nsenter to Debug Container Networks and Enter Linux Namespaces

nsenter is a command from the util‑linux package that runs a specified program inside the namespaces of a given process.

Use Cases

A typical use case is entering a container’s network namespace to debug networking when the container lacks tools such as

ip address

,

ping

,

telnet

,

ss

, or

tcpdump

. Instead of using

docker inspect

to obtain the container IP, nsenter can directly attach to the container’s network namespace from the host.

nsenter can also enter other namespaces such as

mnt

,

uts

,

ipc

,

pid

, and

user

, and can set a root directory and working directory.

Usage

Syntax:

<code>nsenter [options] [program [arguments]]
options:
 -t, --target pid          target process PID
 -m, --mount[=file]        enter mount namespace
 -u, --uts[=file]          enter UTS namespace
 -i, --ipc[=file]          enter IPC namespace
 -n, --net[=file]          enter network namespace
 -p, --pid[=file]          enter PID namespace
 -U, --user[=file]         enter user namespace
 -G, --setgid gid          set GID for program
 -S, --setuid uid          set UID for program
 -r, --root[=directory]    set root directory
 -w, --wd[=directory]      set working directory
If no program is given, $SHELL is executed.</code>

Example: Find the PID of an nginx container and enter its network namespace.

<code># docker inspect -f {{.State.Pid}} nginx
5645
# nsenter -n -t5645
# ip addr
... (output omitted) ...</code>

In Kubernetes, obtain the container ID first:

<code># kubectl get pod test -oyaml | grep containerID
- containerID: docker://cf0873782d587dbca6aa32f49605229da3748600a9926e85b36916141597ec85
# kubectl get pod test -o template --template='{{range .status.containerStatuses}}{{.containerID}}{{end}}'
docker://cf0873782d587dbca6aa32f49605229da3748600a9926e85b36916141597ec85</code>

Principles

Namespaces

Linux namespaces isolate various resources for processes. The main types are mount, ipc, uts, net, pid, user, and cgroup.

mount : separate mount point hierarchy (since Linux 2.4.19)

ipc : separate IPC objects (since 2.6.19)

uts : separate hostname/domainname (since 2.6.19)

net : separate network stack (since 2.6.24)

pid : separate PID number space (since 2.6.24)

user : separate user and group IDs (since 2.6.23, removed in 3.8)

cgroup : separate cgroup hierarchy (since 4.6)

Namespace file descriptors are visible under

/proc/PID/ns

.

clone

The

clone

system call creates a new process and can place it in new namespaces using flags such as

CLONE_NEWNET

,

CLONE_NEWUTS

, etc.

<code>#define _GNU_SOURCE
#include <sched.h>
int clone(int (*fn)(void *), void *child_stack,
          int flags, void *arg, ...);
</code>

setns

setns

attaches the calling thread to an existing namespace identified by a file descriptor.

<code>#define _GNU_SOURCE
#include <sched.h>
int setns(int fd, int nstype);
</code>

Typical usage: open the namespace file (e.g.,

/proc/3550/ns/uts

) and call

setns(fd, 0)

, then exec the desired program.

<code>#define _GNU_SOURCE
#include <fcntl.h>
#include <sched.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>
int main(int argc, char *argv[]) {
    int fd;
    if (argc < 3) {
        fprintf(stderr, "%s /proc/PID/ns/FILE cmd args...\n", argv[0]);
        exit(EXIT_FAILURE);
    }
    fd = open(argv[1], O_RDONLY);
    if (fd == -1) perror("open");
    if (setns(fd, 0) == -1) perror("setns");
    execvp(argv[2], &argv[2]);
    perror("execvp");
}
</code>

Running the program:

<code>./ns_exec /proc/3550/ns/uts /bin/bash
</code>

References

Container network packet capture: https://tencentcloudcontainerteam.github.io/tke-handbook/skill/capture-packets-in-container.html

nsenter man page: http://www.man7.org/linux/man-pages/man1/nsenter.1.html

clone man page: http://www.man7.org/linux/man-pages/man2/clone.2.html

setns man page: http://www.man7.org/linux/man-pages/man2/setns.2.html