Async/Await Is a Trap: Why Rust Needs Thread-Per-Core

Async/await won the concurrency wars because it's easy to write. But as Rich Hickey and Rob Pike have pointed out, easy isn't simple. Under the hood, async/await hides state machines, obscures hardware realities, and pushes scheduling complexity onto developers. In production, this abstraction often breaks down catastrophically.

The Core Problem: Asynchrony vs. Concurrency

Async/await conflates asynchrony (yielding on I/O) with concurrency (handling multiple things at once). A developer writes an async function that looks like sequential code, but if that function performs a 50ms CPU-bound task—parsing a large JSON payload or running a cryptographic proof—the cooperative executor stalls. In runtimes like Tokio or Node.js, the thread doesn't yield until it hits an await point. One heavy compute task can spike latency for thousands of unrelated requests and make the system unresponsive.

The Human-in-the-Loop Scheduler

When these latency spikes hit, the standard fix is to separate runtimes: use Tokio for I/O and Rayon for CPU work. But this forces developers to manually partition every function into I/O or compute pools, orchestrate message passing between them, and police boundaries to prevent deadlocks. As postmortems from PostHog and Meilisearch show, this turns the developer into a human scheduler—exactly what async/await was supposed to eliminate. If you have to manually manage two runtimes with different mental models, the abstraction has failed.

Unbounded by Default Is OOM by Default

Calling tokio::spawn(...) is cheap, which is dangerous. When a downstream database slows down during a traffic spike, the ingress loop keeps accepting connections and spawning tasks. Async tasks and memory allocations are typically unbounded, so the system doesn't push back. In-flight tasks queue indefinitely until the OOM killer terminates the process. Queues don't fix overload; they just delay the crash while making it catastrophic.

The Work-Stealing Myth

When systems hit bottlenecks, developers often demand work-stealing schedulers for fairness. But at massive scale, fairness destroys throughput. Work-stealing moves state machines between CPU cores, abandoning L1/L2 cache and incurring 100+ nanosecond main memory fetch penalties. As WhatsApp discovered with the Erlang BEAM on 100+ core machines, idle threads fighting over global run queue locks can choke the system. If you're already forced to manually partition threads for I/O versus CPU, the generic work-stealing algorithm has failed you.

The Alternative: Thread-Per-Core

Enter Project Tina, an opinionated, shared-nothing, thread-per-core concurrency framework for Rust. Instead of hiding the state machine, Tina exposes it and gives developers better control primitives. Each core runs one OS thread with its own scheduler loop and isolates (e.g., TCP connections, HTTP handlers, workers). There's no work stealing, no mutexes, and strict cache locality. Cross-shard communication happens via lock-free SPSC rings. It's a return to the BEAM's fault-tolerance without the opaque garbage collection and global work-stealing.

Tina embraces strict constraints to guarantee massive throughput and reliability. It's a reminder that sometimes the simplest architecture—one thread per core, no shared memory—is the most robust.