Concurrency

Concurrency#

Three concurrency models, each fit for a different kind of work.

Model	Right for
asyncio	I/O-bound work with thousands of in-flight ops (HTTP, DB, sockets)
threads	I/O-bound work with libraries that don’t speak async
processes	CPU-bound work (numerical, parsing, crypto); bypasses the GIL

The wrong pick costs you: threads on CPU-bound work hit the GIL; asyncio on CPU-bound work blocks the event loop; processes for chatty I/O drown in IPC overhead.

The GIL#

CPython has one Global Interpreter Lock: only one thread runs Python bytecode at a time. Implications.

CPU-bound Python code does not get faster with threads.
I/O-bound code does benefit, blocking system calls release the GIL.
C extensions that release the GIL (NumPy, hashlib, zlib) run in parallel under threads.

Python 3.13 ships an experimental free-threaded build (--disable-gil) that removes this; not yet default and many C extensions need recompilation.

asyncio#

A cooperative event loop; coroutines mark the points where they await I/O, and the loop runs other coroutines while one is blocked.

import asyncio, httpx

async def fetch(client, url):
    r = await client.get(url, timeout=5)
    return r.status_code

async def main(urls):
    async with httpx.AsyncClient() as c:
        return await asyncio.gather(*(fetch(c, u) for u in urls))

asyncio.run(main(urls))

Don’t mix sync-blocking calls into async code; offload them.

await asyncio.to_thread(blocking_call, arg)        # for short blocking
loop = asyncio.get_running_loop()
await loop.run_in_executor(None, work, arg)        # for CPU work via threads

TaskGroup (3.11+) for structured concurrency, cleanup is automatic, errors aggregate into an ExceptionGroup:

async with asyncio.TaskGroup() as tg:
    a = tg.create_task(fetch_a())
    b = tg.create_task(fetch_b())

Threads#

For I/O work via blocking libraries.

from concurrent.futures import ThreadPoolExecutor

with ThreadPoolExecutor(max_workers=16) as pool:
    results = list(pool.map(http_get, urls))

# or single tasks
with ThreadPoolExecutor() as pool:
    fut = pool.submit(work, x)
    result = fut.result()              # blocks for the answer

Synchronization primitives in threading: Lock, RLock, Event, Semaphore, Condition, Queue.

Always prefer Queue over hand-rolled lock dances; the queue handles producer / consumer hand-off correctly.

Processes#

For CPU-bound work that won’t release the GIL.

from concurrent.futures import ProcessPoolExecutor

with ProcessPoolExecutor() as pool:
    results = list(pool.map(crunch, chunks))

Caveats.

The function and its args must be picklable (top-level or importable; no closures over local state).
On macOS / Windows the default start method is spawn, the child re-imports your module. Wrap entry point in if __name__ == "__main__":.
Big arguments are pickled across the process boundary, use shared memory (multiprocessing.shared_memory) for large arrays.

Choosing#

You have…	Reach for
many HTTP calls	`asyncio` + `httpx.AsyncClient` (or threads if the lib is sync-only)
a slow CPU computation	`ProcessPoolExecutor`, one process per core
a single thing to retry	`tenacity` decorator, no concurrency
fan-out across machines	Job queue (Celery, RQ, Dramatiq)

Watching it run#

$ python -X dev -X importtime -c '...'    # dev mode, import timing
$ py-spy record -o flame.svg -- python script.py
$ py-spy top --pid <PID>
$ pyflame -r 0.001 -p <PID>                # legacy

In code.

import asyncio, sys
asyncio.run(main(), debug=True)            # extra checks + warnings
sys.settrace(...)                          # only for debuggers

Pitfalls#

GIL surprise, threads will not parallelise pure-Python loops. Profile first.
Blocking calls in asyncio silently freeze the event loop – even time.sleep instead of await asyncio.sleep.
Mutable shared state across threads / processes, use Queue or messages, not shared dicts.
Forking with threads / open sockets, wrap with multiprocessing.set_start_method("spawn") to dodge fork-after-thread footguns.
asyncio + signal handlers, use loop.add_signal_handler, not signal.signal.