Mastering Linux IPC: Pipes, Message Queues, Shared Memory, Semaphores, Signals & Sockets Explained

What Is Inter‑Process Communication (IPC)?

IPC is the set of kernel‑mediated mechanisms that allow independent processes to exchange data and coordinate their actions. Because each process has its own virtual address space, any data transfer must pass through the kernel.

Why IPC Is Needed

Process isolation improves stability and security, but many real‑world applications require collaboration (e.g., a web server that accepts connections in one process and processes requests in a pool of workers). IPC provides the conduit for data transfer, resource sharing, event notification and process control.

Six Core IPC Mechanisms

1. Pipes

Pipes are the oldest Unix IPC form. They are kernel buffers that provide a unidirectional byte stream (FIFO). Two variants exist:

Anonymous pipe – created with pipe(), lives only between a parent and its child.

Named pipe (FIFO) – created with mkfifo(), appears as a file in the filesystem and can be used by unrelated processes.

#include <stdio.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
int main(){
    int pipe_fd[2];
    pid_t pid;
    if(pipe(pipe_fd)==-1){ perror("pipe"); return 1; }
    pid=fork();
    if(pid==-1){ perror("fork"); return 1; }
    if(pid==0){ // child
        close(pipe_fd[0]);
        write(pipe_fd[1],"Hello, parent!",14);
        close(pipe_fd[1]);
    } else { // parent
        char buffer[1024];
        close(pipe_fd[1]);
        ssize_t n=read(pipe_fd[0],buffer,sizeof(buffer));
        if(n>0){ buffer[n]='\0'; printf("Parent received: %s
",buffer); }
        close(pipe_fd[0]);
    }
    return 0;
}

Anonymous pipes are simple and fast but support only one‑way communication and require a parent‑child relationship. Named pipes break the parent‑child restriction and can be used by any processes that know the FIFO pathname.

2. Message Queues

Message queues store user‑defined messages in kernel memory, allowing asynchronous communication. They support random access, message types and multiple readers/writers, which makes them suitable for producer‑consumer patterns.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/msg.h>
#define MSG_SIZE 128

typedef struct {
    long mtype;
    char mtext[MSG_SIZE];
} message_buf;

int main(){
    key_t key = ftok(".", 'a');
    int msgid = msgget(key, IPC_CREAT|0666);
    message_buf msg;
    msg.mtype = 1;
    strcpy(msg.mtext, "Hello, message queue!");
    msgsnd(msgid, &msg, strlen(msg.mtext)+1, 0);
    printf("Message sent: %s
", msg.mtext);
    msgrcv(msgid, &msg, MSG_SIZE, 1, 0);
    printf("Message received: %s
", msg.mtext);
    msgctl(msgid, IPC_RMID, NULL);
    return 0;
}

Message queues decouple sender and receiver, but each send/receive incurs a system‑call overhead and the queue size is limited.

3. Shared Memory

Shared memory maps the same physical memory region into the address spaces of multiple processes, enabling the fastest data exchange because after the segment is attached no further copying occurs.

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/shm.h>
#define SHM_SIZE 1024

int main(){
    key_t key = ftok(".", 'a');
    int shmid = shmget(key, SHM_SIZE, IPC_CREAT|0666);
    char *shmaddr = (char*)shmat(shmid, NULL, 0);
    strcpy(shmaddr, "Hello, shared memory!");
    printf("Data written: %s
", shmaddr);
    shmdt(shmaddr);
    shmctl(shmid, IPC_RMID, NULL);
    return 0;
}

Because many processes may access the segment concurrently, developers must combine shared memory with synchronization primitives (e.g., semaphores or mutexes) to avoid race conditions.

4. Semaphores and P/V Operations

Semaphores are integer counters used to enforce mutual exclusion and ordering. The classic operations are:

P (wait) – decrement; block if the result becomes negative.

V (signal) – increment; wake a blocked process if the previous value was ≤ 0.

Typical usage in C involves semget, semop and semctl.

#include <stdio.h>
#include <stdlib.h>
#include <sys/types.h>
#include <sys/ipc.h>
#include <sys/sem.h>
union semun { int val; struct semid_ds *buf; unsigned short *array; };

void P(int semid){
    struct sembuf op={0,-1,SEM_UNDO};
    if(semop(semid,&op,1)==-1){ perror("P"); exit(1); }
}

void V(int semid){
    struct sembuf op={0,1,SEM_UNDO};
    if(semop(semid,&op,1)==-1){ perror("V"); exit(1); }
}

int main(){
    key_t key=ftok(".",'a');
    int semid=semget(key,1,IPC_CREAT|0666);
    union semun arg; arg.val=1; semctl(semid,0,SETVAL,arg);
    P(semid); printf("Process 1 in critical section
"); sleep(1); V(semid);
    return 0;
}

Semaphores guarantee exclusive access to a shared resource but do not carry data themselves.

5. Signals

Signals are asynchronous notifications (e.g., SIGINT from Ctrl‑C) that interrupt a process, invoke a handler and then resume execution. They convey an event rather than a resource count.

SIGINT (2) – generated by Ctrl‑C; default action: terminate (catchable).

SIGKILL (9) – forced termination; default action: terminate (uncatchable).

SIGTERM (15) – normal termination request; default action: terminate (catchable).

SIGPIPE (13) – write to a closed pipe or socket; default action: terminate (catchable).

SIGCHLD (17) – child state change; default action: ignore (catchable).

6. Sockets

Sockets provide a universal IPC mechanism that works both locally and across the network. They are identified by an IP address and port (or a Unix‑domain path) and support full‑duplex communication.

Typical TCP server workflow: socket() → bind() → listen() → accept() → read()/write() → close().

// server.c
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/socket.h>
#include <netinet/in.h>
#define PORT 8080
#define BUF_SIZE 1024
int main(){
    int server_fd = socket(AF_INET,SOCK_STREAM,0);
    int opt=1; setsockopt(server_fd,SOL_SOCKET,SO_REUSEADDR|SO_REUSEPORT,&opt,sizeof(opt));
    struct sockaddr_in addr={0};
    addr.sin_family=AF_INET; addr.sin_addr.s_addr=INADDR_ANY; addr.sin_port=htons(PORT);
    bind(server_fd,(struct sockaddr*)&addr,sizeof(addr));
    listen(server_fd,3);
    int client = accept(server_fd,NULL,NULL);
    char buf[BUF_SIZE]; read(client,buf,BUF_SIZE);
    printf("Client: %s
",buf);
    const char *resp="Hello from Server!";
    send(client,resp,strlen(resp),0);
    close(client); close(server_fd);
    return 0;
}

// client.c
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#define PORT 8080
#define BUF_SIZE 1024
int main(){
    int sock = socket(AF_INET,SOCK_STREAM,0);
    struct sockaddr_in serv={0};
    serv.sin_family=AF_INET; serv.sin_port=htons(PORT);
    inet_pton(AF_INET,"127.0.0.1",&serv.sin_addr);
    connect(sock,(struct sockaddr*)&serv,sizeof(serv));
    const char *msg="Hello from Client!";
    send(sock,msg,strlen(msg),0);
    char buf[BUF_SIZE]; read(sock,buf,BUF_SIZE);
    printf("Server: %s
",buf);
    close(sock);
    return 0;
}

Sockets excel in distributed systems and web servers but incur higher overhead than pure local IPC mechanisms.

Typical Application Scenarios

Simple data transfer – use pipes (e.g., shell pipelines).

Large‑volume transfers – shared memory for databases or graphics pipelines.

Process synchronization – semaphores for file access or critical sections.

Asynchronous processing – message queues for order‑processing systems.

Networked services – sockets for HTTP servers, chat applications, distributed computing.

Web‑Server IPC Case Study

A typical web server has a master process that accepts client connections on port 8080 and a pool of worker processes that handle the HTTP requests. The master forwards the accepted client socket descriptor to a worker via a Unix‑domain socket (ancillary data). Workers then read the request, generate a response, and close the connection.

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <sys/socket.h>
#include <sys/un.h>
#include <sys/wait.h>
#include <errno.h>
#define SOCK_PATH "/tmp/web_server.sock"
#define WORKER_NUM 3

void worker_process(){
    int sockfd, connfd;
    struct sockaddr_un addr;
    socklen_t addr_len = sizeof(addr);
    if((sockfd = socket(AF_UNIX, SOCK_STREAM, 0))==-1){ perror("worker socket"); exit(EXIT_FAILURE); }
    memset(&addr,0,sizeof(addr));
    addr.sun_family = AF_UNIX;
    strncpy(addr.sun_path, SOCK_PATH, sizeof(addr.sun_path)-1);
    if(bind(sockfd,(struct sockaddr*)&addr,addr_len)==-1){ perror("worker bind"); close(sockfd); exit(EXIT_FAILURE); }
    if(listen(sockfd,5)==-1){ perror("worker listen"); close(sockfd); exit(EXIT_FAILURE); }
    printf("Worker %d waiting...
", getpid());
    while(1){
        if((connfd = accept(sockfd,(struct sockaddr*)&addr,&addr_len))==-1){ if(errno==EINTR) continue; perror("worker accept"); close(sockfd); exit(EXIT_FAILURE); }
        char buffer[1024];
        ssize_t n = read(connfd, buffer, sizeof(buffer)-1);
        if(n>0){ buffer[n]='\0'; printf("Worker %d got: %s
", getpid(), buffer); }
        const char *resp = "HTTP/1.1 200 OK
Content-Length: 12

Hello Client!";
        write(connfd, resp, strlen(resp));
        close(connfd);
    }
}

void master_process(){
    int listen_fd, client_fd, worker_fd;
    struct sockaddr_in client_addr;
    socklen_t client_len = sizeof(client_addr);
    // 1. TCP socket listening on 8080
    if((listen_fd = socket(AF_INET, SOCK_STREAM, 0))==-1){ perror("master socket"); exit(EXIT_FAILURE); }
    int opt=1; setsockopt(listen_fd, SOL_SOCKET, SO_REUSEADDR|SO_REUSEPORT, &opt, sizeof(opt));
    struct sockaddr_in listen_addr = {0};
    listen_addr.sin_family = AF_INET;
    listen_addr.sin_addr.s_addr = INADDR_ANY;
    listen_addr.sin_port = htons(8080);
    if(bind(listen_fd,(struct sockaddr*)&listen_addr,sizeof(listen_addr))==-1){ perror("master bind"); close(listen_fd); exit(EXIT_FAILURE); }
    if(listen(listen_fd,10)==-1){ perror("master listen"); close(listen_fd); exit(EXIT_FAILURE); }
    printf("Master listening on port 8080...
");
    // 2. Fork worker processes
    for(int i=0;i<WORKER_NUM;i++){
        pid_t pid = fork();
        if(pid==-1){ perror("fork"); exit(EXIT_FAILURE); }
        if(pid==0){ worker_process(); exit(EXIT_SUCCESS); }
    }
    // 3. Accept clients and pass fd to a worker via Unix domain socket
    while(1){
        if((client_fd = accept(listen_fd,(struct sockaddr*)&client_addr,&client_len))==-1){ perror("master accept"); continue; }
        printf("Master accepted client fd %d
", client_fd);
        if((worker_fd = socket(AF_UNIX, SOCK_STREAM, 0))==-1){ perror("master worker socket"); close(client_fd); continue; }
        struct sockaddr_un worker_addr = {0};
        worker_addr.sun_family = AF_UNIX;
        strncpy(worker_addr.sun_path, SOCK_PATH, sizeof(worker_addr.sun_path)-1);
        if(connect(worker_fd,(struct sockaddr*)&worker_addr,sizeof(worker_addr))==-1){ perror("master connect worker"); close(worker_fd); close(client_fd); continue; }
        // Pass the client fd using ancillary data (SCM_RIGHTS)
        struct msghdr msg = {0};
        struct iovec iov[1];
        char dummy='x';
        iov[0].iov_base=&dummy; iov[0].iov_len=1;
        msg.msg_iov=iov; msg.msg_iovlen=1;
        char cmsgbuf[CMSG_SPACE(sizeof(int))];
        msg.msg_control=cmsgbuf; msg.msg_controllen=sizeof(cmsgbuf);
        struct cmsghdr *cmsg = CMSG_FIRSTHDR(&msg);
        cmsg->cmsg_level = SOL_SOCKET;
        cmsg->cmsg_type = SCM_RIGHTS;
        cmsg->cmsg_len = CMSG_LEN(sizeof(int));
        *((int *)CMSG_DATA(cmsg)) = client_fd;
        if(sendmsg(worker_fd,&msg,0)==-1) perror("master sendmsg");
        close(client_fd);
        close(worker_fd);
    }
}

int main(){
    unlink(SOCK_PATH); // clean old socket file
    master_process();
    return 0;
}

This example mirrors real‑world servers such as Nginx: the master process accepts connections, then transfers the client socket descriptor to a worker via a Unix‑domain socket. The design provides high concurrency, clear separation of concerns and the ability to scale workers across machines.

Choosing the Right IPC Mechanism

Select an IPC method based on data size, latency requirements, synchronization needs and whether communication stays on a single host or spans a network:

Pipes/FIFOs – simple byte streams, ideal for small, linear data flow.

Message queues – asynchronous decoupling, support for multiple producers/consumers and message prioritisation.

Shared memory – maximal throughput for large data sets, but requires explicit synchronization.

Semaphores – provide mutual exclusion and ordering; often combined with shared memory.

Signals – lightweight event notification for asynchronous events.

Sockets – universal, network‑ready communication; suitable for distributed systems and services.