CUDA DSLs Are Heading for Retirement — and That's Fine

Stanford's Hazy Research group has a provocative take: CUDA DSLs are on their way out. In a new blog post, they argue that AI agents are taking over the cognitive-offloading role that domain-specific languages have played for decades. The post is based on their experience building megakernels — first with a hand-crafted C++ abstraction layer, then with an agent that generated target-optimized code directly from a prompt.

Last year, the team built megakernels for LLM inference using a custom C++ framework. It worked, but it was miserable: complex data structures, inter-thread synchronization, and deeply nested control flow made the code hard to hold in your head. This year, they built a Mixture-of-Experts megakernel by deleting the abstraction entirely. An agent worked through the complexity directly, producing code that was just as fast — without the intermediate layer.

The pattern is clear: tasks that previously required abstractions (like writing a megakernel) are now manageable with agents, because the abstraction can live in the prompt as vague instructions rather than in carefully designed templates. The agent acts as a compiler that turns fuzzy intent into working code. The ideas behind the megakernel still matter, but expressing them is easier.

The abstraction's job is changing

The authors argue that the primary job of an abstraction — serving as a cognitive offloader — is being retired. Agents are taking that job. By induction, CUDA DSLs like ThunderKittens are next on the list, possibly within a year.

But they're careful to note that not all abstractions are created equal. An abstraction is also a shared surface where application, reuse, and review attach. Without it, you get a Cambrian explosion of verification challenges. Ten teams using ThunderKittens test the same tile semantics; ten teams generating bespoke megakernels have ten disjoint sets of problems.

There's also the question of the oracle. When they deleted the framework, they kept the reference implementations, numerics tolerances, and profiled expectations that served as the contract. You can only retire a layer once you have an oracle that outlives it.

What's left when the framework goes?

The post ends with a provocation: if a prompt can generate every layer of the stack, what is a codebase worth? The authors suggest that codebases are precise but brittle, tied to languages, frameworks, and conventions. Prompts are fuzzy but portable. With agents, there's a competition between an intelligent executor reading underspecified instructions and a dumb executor that needs every gap specified. DSLs are that specification, expressed as a codebase. If the executor stops being dumb, the DSL loses its ground.

They acknowledge the risks: no shared oracle, no knowledge transmission for newcomers, and a single biased sample. But the transformation seems inevitable. What we keep is the intent, the invariants, the tests, and the domain knowledge. The library may go, but the knowledge doesn't evaporate. Trust moves up a level — we scrutinize the spec and the oracle, not the diff. The implementation becomes disposable.

Abstractions retire. Ideas stay.