Master Container Debugging: How nsenter Lets You Enter Any Namespace

Hello everyone, I'm Feng.

Introduction

The nsenter command allows you to run a program inside the namespace of a specified process. It is part of the util-linux package.

Use Cases

A typical use case is entering a container's network namespace. Many lightweight containers omit basic networking tools like ip, ping, telnet, ss, or tcpdump, making network debugging difficult. By using nsenter, you can enter the container's network namespace and use host tools to diagnose connectivity.

Additionally, nsenter can enter other namespaces such as mnt, uts, ipc, pid, and user, as well as set a root directory and working directory.

Usage

The syntax of nsenter is:

nsenter [options] [program [arguments]]

Options

-t, --target pid          # Specify the PID of the target process
-m, --mount[=file]        # Enter mount namespace (optional file)
-u, --uts[=file]          # Enter UTS namespace (optional file)
-i, --ipc[=file]          # Enter IPC namespace (optional file)
-n, --net[=file]          # Enter network namespace (optional file)
-p, --pid[=file]          # Enter PID namespace (optional file)
-U, --user[=file]         # Enter user namespace (optional file)
-G, --setgid gid          # Set GID for the program
-S, --setuid uid          # Set UID for the program
-r, --root[=directory]    # Set root directory
-w, --wd[=directory]      # Set working directory

If no program is given, the default is $SHELL.

Example

Run an nginx container and get its PID:

[root@staight ~]# docker inspect -f {{.State.Pid}} nginx
5645

Enter the container's network namespace:

[root@staight ~]# nsenter -n -t5645
[root@staight ~]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 ...
18: eth0@if19: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 ...

In Kubernetes, obtain the container ID before getting its PID:

[root@node1 test]# kubectl get pod test -oyaml | grep containerID
  - containerID: docker://cf0873782d58...

[root@node1 test]# kubectl get pod test -o template --template='{{range .status.containerStatuses}}{{.containerID}}{{end}}'
 docker://cf0873782d58...

Principle

Namespace

A Linux namespace isolates a set of resources for a group of processes, allowing separate instances of system resources.

Linux currently supports the following namespaces:

mount:  # Isolates the mount point hierarchy (since 2.4.19)
ipc:    # Isolates System V IPC objects (since 2.6.19)
uts:    # Isolates hostname and domain name (since 2.6.19)
net:    # Isolates network stack (since 2.6.24)
pid:    # Isolates process IDs (since 2.6.24)
user:   # Isolates user and group IDs (since 2.6.23, removed in 3.8)
cgroup: # Isolates cgroup hierarchy (since 4.6)

Each process has a set of namespace entries visible under /proc/PID/ns.

/proc/1/ns$ ls -l
lrwxrwxrwx 1 root root 0 Sep 23 19:53 ipc -> ipc:[4026531839]
lrwxrwxrwx 1 root root 0 Sep 23 19:53 mnt -> mnt:[4026531840]
lrwxrwxrwx 1 root root 0 Sep 23 19:53 net -> net:[4026531956]
lrwxrwxrwx 1 root root 0 Sep 23 19:53 pid -> pid:[4026531836]
lrwxrwxrwx 1 root root 0 Sep 23 19:55 user -> user:[4026531837]
lrwxrwxrwx 1 root root 0 Sep 23 19:55 uts -> uts:[4026531838]

clone

The clone system call creates a new process and can specify which namespaces the child should share with the parent.

#define _GNU_SOURCE
#include <sched.h>

int clone(int (*fn)(void *), void *child_stack,
          int flags, void *arg, ...);

Relevant flags for namespaces:

CLONE_NEWCGROUP   # cgroup
CLONE_NEWIPC      # ipc
CLONE_NEWNET      # net
CLONE_NEWNS       # mount
CLONE_NEWPID      # pid
CLONE_NEWUSER     # user
CLONE_NEWUTS      # uts

pid = clone(childFunc, stackTop, CLONE_NEWUTS | SIGCHLD, argv[1]);

setns

The setns system call attaches the calling thread to an existing namespace identified by a file descriptor.

#define _GNU_SOURCE
#include <sched.h>

int setns(int fd, int nstype);

fd is a file descriptor referring to a namespace file under /proc/PID/ns/.

nstype specifies the namespace type; 0 means any type.

Typical usage:

Call setns(fd, 0) to join the namespace.

Call execvp() to run a program inside that namespace.

#define _GNU_SOURCE
#include <fcntl.h>
#include <sched.h>
#include <unistd.h>
#include <stdlib.h>
#include <stdio.h>

#define errExit(msg) do { perror(msg); exit(EXIT_FAILURE); } while (0)

int main(int argc, char *argv[])
{
    int fd;
    if (argc < 3) {
        fprintf(stderr, "%s /proc/PID/ns/FILE cmd args...
", argv[0]);
        exit(EXIT_FAILURE);
    }
    fd = open(argv[1], O_RDONLY);
    if (fd == -1)
        errExit("open");
    if (setns(fd, 0) == -1)
        errExit("setns");
    execvp(argv[2], &argv[2]);
    errExit("execvp");
}

./ns_exec /proc/3550/ns/uts /bin/bash

nsenter

The nsenter utility wraps the setns workflow, allowing you to specify a target PID instead of a file descriptor. It automatically resolves the appropriate namespace files under /proc/PID/ns/ and runs the given program inside them.