Fundamentals 7 min read

How Can a Function Jump to an Uncalled Routine? Exploring Stack Tricks and Process Switching

The article explains how operating‑system multitasking and process switching share the same underlying mechanism as function calls, demonstrates a C program that overwrites a return address to jump to an unexpected function, and shows the resulting assembly to illustrate the similarity between buffer‑overflow attacks and legitimate context switches.

Liangxu Linux
Liangxu Linux
Liangxu Linux
How Can a Function Jump to an Uncalled Routine? Exploring Stack Tricks and Process Switching

Normal Function Call

A typical call sequence is A() calling B(). When B() finishes, the CPU executes a ret instruction that pops the saved return address from the stack and resumes execution in A().

void B() { /* ... */ }
void A() { /* ... */ }

Manipulating the Return Address

The following C program deliberately overwrites the return address stored in A 's stack frame so that, after funcB() returns, execution jumps to funcC() instead of back to funcA():

#include <stdio.h>
#include <stdlib.h>

void funcC() {
    printf("jump to funcC !!!
");
    exit(-1);
}

void funcB() {
    long *p = NULL;
    p = (long*)&p;               // point to the local variable itself
    *(p+2) = (long)funcC;         // overwrite saved return address
}

void funcA() {
    funcB();
}

int main() {
    funcA();
    return 0;
}

Running the program prints jump to funcC !!! even though funcC() is never called directly.

Why It Works

On the x86‑64 ABI, the return address is stored at rbp+8 (or, after the compiler’s prologue, at an offset of two 8‑byte slots from the base of the local variable area). The statement *(p+2) = (long)funcC; writes the address of funcC into that slot, replacing the original return address. When funcB executes its final ret, the CPU pops the forged address and jumps to funcC.

Generated Assembly (gcc 5.2.0, no optimizations)

0000000000400586 <funcC>:
  400586: 55                      push   %rbp
  400587: 48 89 e5                mov    %rsp,%rbp
  40058a: bf 74 06 40 00          mov    $0x400674,%edi
  40058f: e8 bc fe ff ff          callq  400450 <puts@plt>
  400594: bf ff ff ff ff          mov    $0xffffffff,%edi
  400599: e8 e2 fe ff ff          callq  400480 <exit@plt>

000000000040059e <funcB>:
  40059e: 55                      push   %rbp
  40059f: 48 89 e5                mov    %rsp,%rbp
  4005a2: 48 c7 45 f8 00 00 00 00 movq   $0x0,-0x8(%rbp)
  4005aa: 48 8d 45 f8            lea    -0x8(%rbp),%rax
  4005ae: 48 89 45 f8            mov    %rax,-0x8(%rbp)
  4005b2: 48 8b 45 f8            mov    -0x8(%rbp),%rax
  4005b6: 48 83 c0 10            add    $0x10,%rax   ; address of saved return slot
  4005ba: ba 86 05 40 00         mov    $0x400586,%edx   ; address of funcC
  4005bf: 48 89 10               mov    %rdx,(%rax)   ; overwrite return address
  4005c2: 90                     nop
  4005c3: 5d                     pop    %rbp
  4005c4: c3                     retq

Relation to Process Context Switching

The mechanism mirrors what an operating system does during a context switch: the kernel saves the current CPU state—including the return address and registers—in a process control block, then restores the saved state of another process. In a buffer‑overflow exploit the same overwrite is performed maliciously, whereas a legitimate scheduler performs it intentionally as part of the designed switch.

Original Source

Signed-in readers can open the original source through BestHub's protected redirect.

Sign in to view source
Republication Notice

This article has been distilled and summarized from source material, then republished for learning and reference. If you believe it infringes your rights, please contactadmin@besthub.devand we will review it promptly.

AssemblyC programmingfunction callbuffer overflowoperating system fundamentalsprocess switching
Liangxu Linux
Written by

Liangxu Linux

Liangxu, a self‑taught IT professional now working as a Linux development engineer at a Fortune 500 multinational, shares extensive Linux knowledge—fundamentals, applications, tools, plus Git, databases, Raspberry Pi, etc. (Reply “Linux” to receive essential resources.)

0 followers
Reader feedback

How this landed with the community

Sign in to like

Rate this article

Was this worth your time?

Sign in to rate
Discussion

0 Comments

Thoughtful readers leave field notes, pushback, and hard-won operational detail here.