The Tokio/Rayon Trap and Why Async/Await Fails Concurrency<br>--> The Tokio/Rayon Trap and Why Async/Await Fails Concurrency<br>Apr 22, 2026

Over the last decade, async/await won the concurrency wars because it is exceptionally easy. It allows developers to write asynchronous code that looks virtually identical to synchronous code.

But beneath that familiar syntax lies massive structural complexity. It hides control flow, obscures hardware realities, and ultimately pushes the burden of scheduling back onto the developer.

Rich Hickey articulated this perfectly in his talk Simple Made Easy: “Easy” is what is familiar and close at hand, while “Simple” is what is structurally untangled1. async/await is easy to write, but it is fiercely complex to operate.

Rob Pike talked about this architectural shift during his 2023 GopherConAU address:

Compared to goroutines, channels and select, async/await is easier and smaller for language implementers to build… But it pushes some of the complexity back on the programmer, often resulting in what Bob Nystrom has called ‘colored functions’. […] It’s important, though, whatever concurrency model you do provide, you do it exactly once, because an environment providing multiple concurrency implementations can be problematic.2

Pike’s remark about “multiple concurrency implementations” and async/await is exactly what is failing in production today.

The Production Trap: Confusing Asynchrony with Concurrency

The fundamental trap of async/await is that it conflates asynchrony (yielding while waiting for I/O) with concurrency (dealing with multiple things at once).

The syntax is a trap because it disguises interleaved state machines as isolated, sequential threads. Lulled by this illusion, a developer writes an async function exactly as they would blocking code — fetching a database record over the network, then immediately crunching the data. But what happens when that data crunching involves parsing a 10MB JSON payload, traversing a massive collection, or executing a compute-heavy cryptographic proof?

The cooperative executor halts.

EXPECTATION: Fast I/O<br>Tasks yield quickly. High throughput.<br>Incoming Network Traffic

Task Queue

(Stable / Low)

Executor (1 OS Thread)

I/O

yield

I/O

yield

REALITY: The Compute Trap<br>One CPU task stalls the executor. Queue explodes.<br>Incoming Network Traffic

Task Queue (Unbounded)

OOM<br>CRASH

Executor (1 OS Thread)

Heavy Compute Task<br>(bcrypt, parsing, etc.)

THREAD<br>HALTED

In a cooperative runtime like Rust’s Tokio or Node.js, the thread does not yield until it hits an await point. A 50-millisecond CPU-bound task in a function stalls the entire execution thread. Suddenly, thousands of unrelated network requests spike in latency and the system becomes unresponsive. Meanwhile the hardware is barely utilised.

The Broken Promise: Human in the loop Scheduler

When these latency spikes occur, the answer is always the same: separate your runtimes. Use Tokio for I/O, and send CPU-bound work to a dedicated thread pool like Rayon.

Recent postmortems highlight the resulting disaster. Engineering teams at PostHog3 and Meilisearch4 have documented the painful reality of untangling these complexities in production. Developers must carefully analyse every function to decide if it belongs in the “I/O pool” or the “Compute pool,” and then manually orchestrate the message-passing boundary between them.

If a developer must manually partition I/O and compute, strictly police the boundaries to prevent deadlocks, and ferry data between two different runtimes with two different mental models, the async abstraction has failed. The language feature promised to hide the complexity of concurrency. Instead, it turned the application developer into the human in the loop scheduler.

Unbounded by Default is OOM by Default

The second failure mode of async/await runtimes is how frictionless they make unbounded capacity.

Calling tokio::spawn(...) is cheap. When a downstream database slows down during a traffic spike, the ingress network loop happily continues accepting connections and spawning tasks. Because async tasks and memory allocations are typically unbounded by default in these ecosystems, the system does not push back.

In-flight tasks queue indefinitely. The application consumes RAM until the OS out-of-memory (OOM) killer violently terminates the process. Postmortems from major platforms consistently reveal the same root cause: queues do not fix overload, they simply delay the crash while making it catastrophic . Infinite capacity is a lie, and defaults that pretend otherwise are dangerous.

The Work-Stealing Myth

When systems hit these bottlenecks, developers often demand smarter, preemptive, work-stealing schedulers to distribute the load. The assumption is that if a core is idle, it should steal tasks from a busy core to guarantee fairness. But at massive scale, fairness is the enemy of throughput . Work-stealing destroys CPU cache locality.

When WhatsApp...