Why Async in Python Still Struggles: GIL, Free Threads, and the Future of Concurrency

What is async?

Python 3.5 added the async and await keywords to run coroutines, which are most useful for I/O‑bound tasks such as HTTP requests. A coroutine is scheduled on an event loop that drives its execution.

Async vs. sync examples

def get_thing_sync():
    return http_client.get('/thing/which_takes?ages=1')

async def get_thing_async():
    return await http_client.get('/thing/which_takes?ages=1')

Calling the synchronous function or awaiting the asynchronous one takes the same amount of time for a single request. The benefit of async appears when many requests are started together: the event loop can issue all network operations at once and handle each response as it arrives, keeping the CPU busy only with bookkeeping.

Limitations of asyncio

File I/O is not asynchronous in the standard library; even asyncio cannot make open() non‑blocking.

Operations that block the Global Interpreter Lock (GIL) still serialize execution across threads.

Many third‑party libraries provide only synchronous APIs, forcing developers to offload blocking calls to thread pools via loop.run_in_executor().

Mixing sync and async code often leads to subtle bugs where developers forget to await a coroutine.

Some errors are caught by the compiler, but others (e.g., forgetting await ) only surface at runtime, making async code harder to reason about for developers without a networking background.

Free threads and sub‑interpreters (PEP 779, PEP 734)

Python 3.14 will introduce a “free‑thread” mode that removes the GIL and replaces it with finer‑grained locks, as well as a multiple‑interpreter executor. These changes aim to make true parallelism practical for CPU‑bound work and to simplify the implementation of async‑friendly APIs.

Comparison with C# async model

C# creates a task pool managed by the runtime, allowing tasks to run on any thread, while Python’s event loop is bound to the thread that created it. C# async functions are scheduled on the task pool, whereas Python async functions are scheduled on the event loop. This gives C# a higher‑level abstraction for parallelism.

API design challenges

Maintaining both synchronous and asynchronous versions of a library doubles the code base and testing effort. Certain language features, such as __init__ or other dunder methods, cannot be async, forcing developers to split initialization logic. Async properties are also problematic because they require explicit await on each access, leading to confusing APIs.

Operation

Async API

Description

Sleep asyncio.sleep() Asynchronously pause for a given duration.

TCP/UDP Streams asyncio.open_connection() Open a TCP/UDP connection.

HTTP aiohttp.ClientSession() Asynchronous HTTP client.

Run subprocess asyncio.subprocess Run a subprocess without blocking the event loop.

Queue asyncio.Queue Asynchronous queue implementation.

Conclusion

Asyncio’s primary value lies in network I/O, where Python’s sockets already release the GIL. For CPU‑bound or file‑I/O workloads, the GIL and lack of native async file APIs limit usefulness. The upcoming free‑thread and sub‑interpreter features in Python 3.14 could broaden concurrency use cases and reduce the need for duplicated sync/async APIs, but developers must still navigate integration complexities.