Unlock Linux Performance: Master Memory Alignment and Struct Optimization

Introduction

Memory alignment is a often‑overlooked factor that directly limits program performance on Linux. Improper alignment forces the CPU to perform extra memory accesses and reduces cache‑hit rates, creating bottlenecks in kernel‑mode code, user‑space applications, and high‑concurrency scenarios.

1. Understanding Memory Alignment

1.1 What Is Memory Alignment?

Memory alignment means that a data object's start address must be a multiple of its size or a specific alignment value. For example, a struct containing char a; int b; short c; occupies more than the naïve 7 bytes because the compiler inserts padding to satisfy alignment rules.

1.2 Why Alignment Matters

(1) Hardware Access Mechanism: Modern CPUs read memory in fixed‑size blocks (e.g., 4 bytes on 32‑bit, 8 bytes on 64‑bit). When an int is placed at an address that is not a multiple of 4, the CPU must perform two separate reads and combine the results, doubling the access time.

(2) Performance Gains: Aligned data can be fetched in a single memory cycle, reducing CPU wait time. Aligned structures also improve cache‑line utilization, increasing cache‑hit rates and lowering latency.

(3) Comparative Example:

#include <stdio.h>
#include <time.h>
struct UnalignedStruct {
    char a;   // 1 byte
    int  b;   // 4 bytes
    short c;  // 2 bytes
};
int main(){
    struct UnalignedStruct arr[1000000];
    clock_t start = clock();
    for(int i=0;i<1000000;i++) arr[i].b = i;
    clock_t end = clock();
    double t = (double)(end-start)/CLOCKS_PER_SEC;
    printf("Unaligned access time: %f seconds
", t);
    return 0;
}

Running this code typically yields around 0.12 s. After reordering members to reduce padding:

#include <stdio.h>
#include <time.h>
struct AlignedStruct {
    int   b;   // 4 bytes
    short c;   // 2 bytes
    char  a;   // 1 byte
};
int main(){
    struct AlignedStruct arr[1000000];
    clock_t start = clock();
    for(int i=0;i<1000000;i++) arr[i].b = i;
    clock_t end = clock();
    double t = (double)(end-start)/CLOCKS_PER_SEC;
    printf("Aligned access time: %f seconds
", t);
    return 0;
}

The aligned version runs in roughly 0.08 s, a ~33 % improvement.

2. Linux Memory‑Alignment Rules

2.1 Basic Types

With GCC, alignment equals the type size: char → 1 byte, int → 4 bytes, double → 8 bytes. Variables are placed at addresses that are multiples of their alignment value.

char   ch = 'a';
int    num = 100;
double d   = 3.14;

In a struct, each member must start at an offset that is a multiple of its own alignment, causing the compiler to insert padding bytes.

2.2 Struct Alignment

(1) Member Offsets: The first member starts at offset 0. Subsequent members are placed at the next offset that satisfies their alignment, possibly inserting padding.

(2) Total Size: A struct’s size must be a multiple of the largest member’s alignment. This ensures that arrays of the struct keep each element properly aligned.

(3) Example:

struct Example {
    char   c;   // offset 0
    int    i;   // offset 4 (3 bytes padding after c)
    double d;   // offset 8 (already 8‑byte aligned)
};

The resulting size is 16 bytes (max alignment = 8).

3. Real‑World Impact

3.1 Struct Definition Optimization

Reordering members from smallest to largest reduces padding. For a network packet struct:

struct Packet {
    char   flag;      // 1 byte
    int    length;    // 4 bytes (3 bytes padding after flag)
    short  checksum; // 2 bytes (2 bytes padding after length)
}; // total 12 bytes

Optimized order:

struct OptimizedPacket {
    int    length;   // 4 bytes
    short  checksum; // 2 bytes
    char   flag;     // 1 byte
}; // total 8 bytes

3.2 Data Transmission & Storage

When data moves between systems with different alignment conventions (e.g., 32‑bit vs 64‑bit), misaligned layouts can cause parsing errors. Using a common on‑wire format and explicit padding avoids such issues.

3.3 Project Case Study

A Linux‑based distributed storage system stored file metadata in:

struct FileMeta {
    int    file_size;   // 4 bytes
    char   file_perm[4]; // 4 bytes
    time_t create_time; // 8 bytes, requires 8‑byte alignment
}; // size 24 bytes (4 bytes padding before create_time)

After reordering:

struct OptimizedFileMeta {
    time_t create_time; // 8 bytes, offset 0
    int    file_size;   // 4 bytes, offset 8
    char   file_perm[4]; // 4 bytes, offset 12
}; // size 16 bytes, better cache‑line usage

Benchmarks showed a noticeable reduction in memory‑access latency and overall throughput.

4. How to Achieve Proper Alignment

4.1 Compiler Directives

GCC supports #pragma pack(n) to set a maximum alignment for subsequent structs. Example:

#pragma pack(4)
struct Example {
    char  a; // padded to 4‑byte boundary
    int   b;
    short c;
};
#pragma pack() // restore default

Be cautious: non‑default packing may break ABI compatibility across platforms.

4.2 Coding Techniques

Arrange struct members from largest to smallest, or group same‑size members together, to minimise padding. For C++11 and later, alignas explicitly specifies alignment:

struct alignas(8) MyStruct {
    char a;
    int  b;
};

This forces the struct’s start address to be an 8‑byte multiple, improving access efficiency.

By applying these guidelines—understanding hardware constraints, following Linux alignment rules, and using compiler directives or careful member ordering—developers can eliminate hidden performance penalties and write faster, more memory‑efficient code.