Unlocking Server Performance: The Four Hidden Killers You Must Eliminate

Introduction

This article shares the author’s long‑term experience with server development, defining a server as a program that processes massive discrete messages or requests per second. It focuses on architectural factors that limit performance rather than simple multithreading tricks.

Four Major Performance Killers

Data Copies

Context Switches

Memory Allocation

Lock Contention

If a server can avoid these four issues, its performance will be outstanding.

Data Copies

Data copies are widely recognized as harmful, yet they often hide deep inside libraries or drivers. To reduce copies, use buffer descriptors (or buffer chains) that contain a pointer to the buffer, its length, a pointer to the actual data, offsets, a doubly‑linked list of other buffers, and a reference count. Incrementing the reference count instead of copying data can be effective for large data blocks, but traversing descriptor chains may become more expensive than copying.

Avoid extreme zero‑copy solutions that introduce other overheads such as additional context switches or fragmented I/O.

Context Switches

Context switches are often more detrimental than data copies. Excessive active threads relative to CPU cores cause linear or exponential growth in switches, reducing useful work. Limiting active threads to the number of CPUs, using asynchronous I/O (select/poll, AIO, completion ports), and employing an event‑driven framework can mitigate this.

Designing a server with a request queue, listener threads, and worker threads should minimize the number of thread hops per request. Using a counted semaphore to cap concurrent active threads can further reduce unnecessary switches.

Memory Allocation

Frequent allocation and deallocation dominate runtime cost. Three recommendations are offered:

Pre‑allocate memory when possible, especially for known‑size structures, to reduce fragmentation and allocation overhead.

Use a look‑aside list (object cache) that recycles recently freed objects, avoiding repeated system calls.

Employ thread‑local or per‑CPU look‑aside lists to eliminate lock contention during allocation.

Periodically clean up unused objects to prevent unbounded growth.

Lock Contention

Locks can be too coarse (causing serialization) or too fine (causing high contention). The author suggests mapping locks onto a two‑dimensional grid where one axis represents code stages and the other represents data sets; locks should be evenly distributed across this grid.

Ensuring that two requests never compete for the same lock unless they share the same stage and data set dramatically reduces contention. Visualizing lock placement and measuring contention helps refine the design.

Other Considerations

Additional factors include storage subsystem characteristics, network protocol tuning (e.g., TCP_CORK, Nagle), support for scatter‑gather I/O, page and cache sizes, system‑call overhead, lock starvation, wake‑up storms, and “thundering herd” problems. Testing and profiling on target platforms are essential to uncover hidden costs.