Mastering Linux Shared Memory: APIs, Structures, and Implementation Details

Linux gives each process its own virtual address space, so the same virtual address in different processes points to different physical memory. To enable inter‑process communication, Linux provides shared memory, which maps the same physical pages into the virtual spaces of multiple processes.

Using Shared Memory

1. Obtaining a Shared Memory Segment

Call shmget() to create or retrieve a segment. Its prototype is:

int shmget(key_t key, size_t size, int shmflg);

key

– usually generated by ftok() to uniquely identify the IPC resource. size – size of the segment in bytes. shmflg – flags such as IPC_CREAT (create if missing) and IPC_EXCL (fail if already exists).

On success, shmget() returns a non‑negative identifier; on failure it returns –1 and sets errno.

2. Attaching the Segment

Use shmat() to map the segment into the process’s virtual address space:

void *shmat(int shmid, const void *shmaddr, int shmflg);

shmid

– identifier returned by shmget(). shmaddr – desired address; if 0 the kernel chooses one. shmflg – SHM_RDONLY for read‑only mapping, otherwise read/write.

On success it returns the attached virtual address; on error it returns (void *)‑1.

3. Detaching the Segment

When a process no longer needs the segment, call shmdt(): int shmdt(const void *shmaddr); It removes the mapping; success returns 0, failure –1.

Example Programs

Two simple programs demonstrate the workflow. Process A creates a segment, attaches it, writes "Hello World", and exits. Process B attaches the same segment and reads the string.

#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
#define SHM_PATH "/tmp/shm"
#define SHM_SIZE 128
int main(int argc, char *argv[]) {
    int shmid;
    char *addr;
    key_t key = ftok(SHM_PATH, 0x6666);
    shmid = shmget(key, SHM_SIZE, IPC_CREAT|IPC_EXCL|0666);
    if (shmid < 0) { perror("shmget"); return -1; }
    addr = shmat(shmid, NULL, 0);
    if (addr == (void*)-1) { perror("shmat"); return -1; }
    sprintf(addr, "Hello World
");
    return 0;
}

#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
#define SHM_PATH "/tmp/shm"
#define SHM_SIZE 128
int main(int argc, char *argv[]) {
    int shmid;
    char *addr;
    char buf[128];
    key_t key = ftok(SHM_PATH, 0x6666);
    shmid = shmget(key, SHM_SIZE, IPC_CREAT);
    if (shmid < 0) { perror("shmget"); return -1; }
    addr = shmat(shmid, NULL, 0);
    if (addr == (void*)-1) { perror("shmat"); return -1; }
    strcpy(buf, addr);
    printf("%s", buf);
    return 0;
}

Running A then B prints “Hello World”, confirming successful sharing.

Kernel Implementation Overview

The kernel represents each shared memory segment with struct shmid_kernel, stored in a global array shm_segs[SHMMNI] (default limit 128). The inner struct shmid_ds holds metadata such as size, timestamps, creator PID, and attachment count.

struct shmid_ds {
    struct ipc_perm shm_perm;   /* permissions */
    int    shm_segsz;           /* size in bytes */
    __kernel_time_t shm_atime;  /* last attach */
    __kernel_time_t shm_dtime;  /* last detach */
    __kernel_time_t shm_ctime;  /* last change */
    __kernel_ipc_pid_t shm_cpid;/* creator pid */
    __kernel_ipc_pid_t shm_lpid;/* last operator pid */
    unsigned short shm_nattch;  /* current attaches */
    /* ... other fields ... */
};

struct shmid_kernel {
    struct shmid_ds u;          /* public info */
    unsigned long shm_npages;   /* number of pages */
    pte_t *shm_pages;           /* array of page table entries */
    struct vm_area_struct *attaches; /* list of vm areas */
};

The shmget() system call ( sys_shmget) validates arguments, looks up an existing key with findkey(), and either returns the existing identifier or creates a new struct shmid_kernel via newseg(). It also enforces limits such as shmmax and SHMMNI.

The shmat() implementation ( sys_shmat) performs several steps:

Validate shmid and retrieve the corresponding shmid_kernel from shm_segs.

Find a suitable virtual address (using get_unmapped_area() if none is supplied) and align it to SHMLBA.

Allocate a vm_area_struct to describe the mapping.

Initialize the vm_area_struct fields, especially vm_ops = &shm_vm_ops which defines callbacks for page faults.

Insert the new mapping into the segment’s attaches list and update timestamps.

Return the attached address to user space.

The shm_vm_ops structure provides callbacks such as shm_open, shm_close, and most importantly shm_nopage, which handles page‑faults.

Page‑Fault Handling with shm_nopage()

When a process accesses a part of the shared segment that has no physical page yet, the kernel invokes shm_nopage(). The function:

Computes the page index from the faulting address.

Locks shm_lock and checks whether the page table entry already exists.

If the entry is empty, it allocates a new high‑order page ( get_free_highpage()), clears it, and creates a new pte with PAGE_SHARED permissions.

Stores the new pte in shp->shm_pages[idx] so that all processes sharing the segment see the same physical page.

Updates accounting counters ( shm_rss) and returns the page to the fault handler.

This lazy allocation means physical memory is only consumed when actually accessed.

Key Takeaways

Shared memory in Linux is a kernel‑managed IPC mechanism that maps the same physical pages into multiple processes.

The user‑space API consists of shmget(), shmat(), and shmdt(), each with clear prototypes and flag options.

Internally the kernel tracks segments with struct shmid_kernel, stores metadata in struct shmid_ds, and uses a global array limited to 128 segments.

Mapping is performed lazily; physical pages are allocated on first access via the shm_nopage() page‑fault handler.

Understanding these structures and the flow of sys_shmget, sys_shmat, and the VM operations is essential for debugging IPC issues or extending the kernel.