Unlocking High-Concurrency in C/C++: A Deep Dive into Coroutines and Their Implementation

1. Introduction to Coroutines

In the vast world of C/C++ programming, high concurrency has long been the holy grail for developers. Traditional multithreaded or multiprocess models provide powerful concurrency but incur high resource overhead and complex synchronization. Imagine a massive server handling thousands of client requests; frequent thread switches and resource contention can become a performance bottleneck.

Coroutines emerge as a brilliant new star, offering a lightweight concurrency model that can achieve efficient multitasking within a single thread. Like a skilled dancer that pauses and resumes gracefully, a coroutine avoids resource contention and dramatically improves performance for I/O‑bound tasks.

In the following sections we will explore coroutines in C/C++ from zero to one, dissect their underlying principles, and demonstrate how to apply them to unlock high‑concurrency programming.

2. Coroutine (Coroutine) Overview

A coroutine, also called a micro‑thread or fiber, was proposed early but only gained widespread use in recent years (e.g., Lua).

Traditional sub‑routine calls follow a strict call‑stack hierarchy: function A calls B, B calls C, and each returns in order. The call stack is managed by the operating system, and a single thread executes one sub‑routine at a time.

Coroutines differ: they appear like sub‑routines but can be interrupted inside their body, yielding control to another coroutine, and later resume where they left off. This behavior resembles a CPU interrupt rather than a function call.

print '1'
print '2'
print '3'
def B():
    print 'x'
    print 'y'
    print 'z'

If coroutine A is interrupted to run coroutine B, the execution order could be:

1
2
x
y
3
z

Although A never explicitly calls B, the interleaved execution makes coroutine semantics harder to grasp.

Question:

Coroutine execution looks similar to multithreading, but what advantages do coroutines have over threads?

The biggest advantage is execution efficiency. Context switches are performed by the program itself rather than the kernel, eliminating thread‑switch overhead. As the number of threads grows, coroutine performance advantage becomes more pronounced.

Second, coroutines avoid the need for locks because only one thread manipulates shared data. Synchronization can be handled by simple state checks, further boosting performance.

Because coroutines run in a single thread, how can we exploit multiple CPU cores? The simplest method is to combine multiple processes with coroutines, leveraging both multi‑core parallelism and coroutine efficiency.

Python’s support for coroutines is limited; generators with yield can approximate coroutine behavior, offering substantial power despite incomplete support.

Example: Producer‑Consumer with Coroutines

Traditional producer‑consumer uses a thread for producing messages and another thread for consuming, protected by locks, which can easily deadlock.

Using coroutines, the producer yields the message directly to the consumer via yield, and the consumer returns the result, allowing the producer to continue. Example code:

def consumer():
    r = ''
    while True:
        n = yield r
        if not n:
            return
        print('[CONSUMER] Consuming %s...' % n)
        time.sleep(1)
        r = '200 OK'

def produce(c):
    c.next()
    n = 0
    while n < 5:
        n = n + 1
        print('[PRODUCER] Producing %s...' % n)
        r = c.send(n)
        print('[PRODUCER] Consumer return: %s' % r)
    c.close()

if __name__=='__main__':
    c = consumer()
    produce(c)

Execution result demonstrates seamless hand‑off between producer and consumer without any lock.

3. Coroutines in C/C++

C++ has been conservative; only with C++20 were coroutines standardized, and compiler support remains limited. Full adoption may wait for C++23.

Coroutines are generally lock‑free, but if the underlying implementation crosses threads, locks may still be required. Existing C++ coroutine libraries fall into two categories: assembly‑based context switching and OS‑provided APIs.

Assembly‑based: libco, Boost.context.

OS‑based: phxrpc (ucontext/Boost.context), libmill (setjmp/longjmp).

Assembly‑based switches are faster because they avoid kernel transitions, while OS‑based switches are more portable but incur higher overhead.

Other techniques include using setjmp / longjmp to save and restore execution contexts, or writing custom assembly to manipulate registers directly.

3.1 Differences Between Threads and Coroutines

Thread: Managed by the OS kernel, each thread has its own stack and registers. Scheduling is preemptive; the kernel decides when to switch threads, incurring context‑switch overhead.

Coroutine: User‑level lightweight thread, scheduled cooperatively by the program. Only the running coroutine yields control, resulting in minimal context‑switch cost and no need for locks.

Key differences summarized:

Scheduler: OS kernel vs. user program.

Context‑switch overhead: large vs. small.

Resource consumption: many vs. few.

Suitable scenarios: CPU‑bound vs. I/O‑bound.

3.2 Coroutine Execution Flow (C++20 Example)

#include <iostream>
#include <coroutine>

struct Task {
    struct promise_type;
    using handle_type = std::coroutine_handle<promise_type>;
    handle_type coro;

    Task(handle_type h) : coro(h) {}
    ~Task() { if (coro) coro.destroy(); }

    bool resume() {
        if (!coro.done()) {
            coro.resume();
            return true;
        }
        return false;
    }

    struct promise_type {
        Task get_return_object() {
            return Task{handle_type::from_promise(*this)};
        }
        std::suspend_never initial_suspend() { return {}; }
        std::suspend_never final_suspend() noexcept { return {}; }
        void return_void() {}
        void unhandled_exception() { std::terminate(); }
    };

    static Task simple_coroutine() {
        std::cout << "Coroutine started" << std::endl;
        co_await std::suspend_always{};
        std::cout << "Coroutine resumed" << std::endl;
    }
};

int main() {
    Task t = Task::simple_coroutine();
    std::cout << "Main function" << std::endl;
    t.resume();
    return 0;
}

The example shows a coroutine that starts, suspends at co_await, and later resumes, preserving local state.

4. Implementation Details of Coroutines

4.1 Origin and Motivation

High‑throughput servers need to handle massive I/O efficiently. Using epoll for event‑driven I/O, a synchronous handle() function performs recv, parsing, and send in the same loop, leading to poor performance. Asynchronous I/O with coroutines decouples I/O from the main loop, dramatically reducing response time.

By moving socket handling into coroutines and letting the scheduler manage epoll events, we achieve a lock‑free, high‑performance server architecture.

4.2 NtyCo Library API

Coroutine API:

int nty_coroutine_create(nty_coroutine **new_co, proc_coroutine func, void *arg)

void nty_schedule_run(void)

POSIX‑style asynchronous wrappers:

int nty_socket(int domain, int type, int protocol);
int nty_accept(int fd, struct sockaddr *addr, socklen_t *len);
int nty_recv(int fd, void *buf, int length);
int nty_send(int fd, const void *buf, int length);
int nty_close(int fd);

4.3 Core Primitive Operations

Three primitives: create, resume, yield. Creation registers the coroutine in the scheduler’s ready queue. yield saves the current context and transfers control back to the scheduler. resume restores a saved context and continues execution.

4.4 Context Switching Assembly

The low‑level switch saves registers of the current coroutine into a nty_cpu_ctx structure and restores registers from the target coroutine. The assembly routine _switch performs the save‑restore sequence and returns to the target’s saved instruction pointer.

4.5 Data Structures

Coroutine structure ( nty_coroutine) contains execution context, function pointer, arguments, stack information, status flags, and links for ready, sleep, and wait queues.

Scheduler ( nty_schedule) holds the global context, epoll file descriptor, event list, and three collections: ready queue, sleeping red‑black tree, and waiting red‑black tree.

4.6 Scheduling Algorithms

Two approaches:

Producer‑consumer model: ready coroutines are stored in a queue; the scheduler pops and resumes them.

Multi‑state model: ready, sleeping, and waiting coroutines are managed in separate containers; the scheduler checks timers and epoll events, then resumes eligible coroutines directly.

4.7 Performance Test

Test environment: one server (6 GB RAM, 4 CPU cores) and three clients (2 GB RAM, 2 CPU cores) on Ubuntu 14.04. The NtyCo server handled 1 million concurrent connections with 4 KB stacks per coroutine, demonstrating scalability and low latency compared to traditional thread‑per‑connection models.

5. Coroutine Creation and Execution with libco

libco limits a thread to 128 coroutines, tracked in stCoRoutineEnv_t. The environment stores a call stack, the current epoll instance, and pointers for shared‑stack handling.

When creating a coroutine, libco allocates a stCoRoutine_t structure, sets up its stack (either dedicated or shared), and registers it in the environment’s call stack.

int co_create(stCoRoutine_t **co, stCoRoutineEnv_t *env,
               void (*pfn)(void *), void *arg, int stack_size);

Resuming a coroutine pushes it onto the call stack and invokes co_swap, which saves the current context and jumps to the target coroutine’s entry point.

void co_resume(stCoRoutine_t *co) {
    stCoRoutineEnv_t *env = co->env;
    stCoRoutine_t *curr = env->pCallStack[env->iCallStackSize - 1];
    if (!co->cStart) {
        coctx_make(&co->ctx, (coctx_pfn_t)CoRoutineFunc, co, 0);
        co->cStart = 1;
    }
    env->pCallStack[env->iCallStackSize++] = co;
    co_swap(curr, co);
}

Yielding restores the previous coroutine:

void co_yield_env(stCoRoutineEnv_t *env) {
    stCoRoutine_t *last = env->pCallStack[env->iCallStackSize - 2];
    stCoRoutine_t *curr = env->pCallStack[env->iCallStackSize - 1];
    env->iCallStackSize--;
    co_swap(curr, last);
}

6. Coroutine Context Creation and Switching

The context structure coctx_t stores registers and a stack pointer. coctx_make prepares a new context by allocating space for two arguments, aligning the stack, and setting the instruction pointer ( EIP) to the coroutine entry function.

int coctx_make(coctx_t *ctx, coctx_pfn_t pfn, const void *s, const void *s1) {
    char *sp = ctx->ss_sp + ctx->ss_size - sizeof(coctx_param_t);
    sp = (char *)((unsigned long)sp & -16L);
    coctx_param_t param = (coctx_param_t)sp;
    param->s1 = s;
    param->s2 = s1;
    memset(ctx->regs, 0, sizeof(ctx->regs));
    ctx->regs[kESP] = (void *)sp - sizeof(void *);
    ctx->regs[kEIP] = (void *)pfn;
    return 0;
}

The assembly routine _switch saves the current registers into the “current” context and restores registers from the “new” context, finally executing ret to jump to the saved instruction pointer.

7. Using libco in an Echo Server

The example example_echosvr.cpp demonstrates how to build a high‑performance echo server with libco:

Create a non‑blocking listening socket on port 1024.

Spawn a pool of read‑write coroutines with readwrite_coroutine and start them with co_resume.

Create an accept coroutine that handles new connections.

Enter the event loop co_eventloop, which monitors epoll events and resumes the appropriate coroutine when I/O is ready.

Each read‑write coroutine registers its socket with co_poll, yields, and is later resumed when epoll signals readability or writability.

8. Managing Coroutines with Epoll

co_poll

registers an array of struct pollfd with libco’s epoll wrapper stCoEpoll_t. For each file descriptor, libco creates a stPollItem_t that stores the coroutine to be awakened. The poll operation adds the descriptors to epoll, sets a timeout if needed, and then yields the calling coroutine.

When epoll reports an event, the scheduler extracts the stored stPollItem_t, removes the descriptors from epoll, and resumes the associated coroutine via OnPollProcessEvent. Timeouts are handled by a timing‑wheel structure inside stCoEpoll_t, ensuring that coroutines waiting on a timeout are also resumed.

9. Timer Implementation

libco uses a timing‑wheel (array of buckets) to manage timeouts with O(1) insertion and removal. The wheel has 60 × 1000 slots, supporting timers up to 60 seconds (practically limited to 40 seconds). When a coroutine registers a timeout, the timer is placed in the appropriate bucket based on the expiration time. The event loop advances the wheel and moves expired items to the active list for processing.

10. EPOLL Event Loop

The main coroutine runs co_eventloop, which repeatedly calls co_epoll_wait to retrieve ready events. For each event, the stored stPollItem_t is added to the active list. Expired timers are also moved to the active list. The loop then iterates over the active list, invoking each item’s pfnProcess (typically OnPollProcessEvent) to resume the corresponding coroutine.

An optional user‑provided callback can be executed after each loop iteration; returning –1 terminates the event loop, allowing graceful shutdown or statistics collection.