SIMD Isn't Scary: A Practical Guide for Everyday Engineers
Mitchell Hashimoto argues that SIMD is simpler than its reputation suggests, offering a five-step pattern that turns naive loops into vectorized code with dramatic speedups.
SIMD has long been viewed as arcane wizardry reserved for kernel hackers and database authors. Mitchell Hashimoto, creator of HashiCorp tools and the Ghostty terminal, disagrees. In a detailed walkthrough, he demonstrates that the common case of SIMD—processing N values at a time—follows a predictable five-step pattern that any engineer can learn.
The Five-Step Pattern
Hashimoto breaks vectorization into five steps:
- Broadcast constants — load your threshold or accumulator into every lane of a vector register.
- Loop over vector-width chunks — advance by the lane count instead of one element.
- Perform the SIMD operation — a single instruction compares or computes across all lanes in parallel.
- Reduce or store the result — collapse the vector result (e.g., check if all lanes passed).
- Scalar tail — handle the remaining elements that don't fill a full vector with the original scalar loop.
He illustrates this with a real-world example from Ghostty: scanning a slice of decoded codepoints for a C0 control character (value ≤ 0xF). The scalar loop is a single line; the SIMD version adds 12 lines but yields up to 16x throughput on AVX-512 hardware, and a measured 5x end-to-end speedup on an AVX2 Intel desktop.
Why Compilers Can't Always Do This
Hashimoto addresses the obvious question: why not let the compiler auto-vectorize? He notes that compilers are conservative—they must guarantee correctness for all inputs, including edge cases like misaligned data or variable-length loops. Hand-written SIMD, even following the simple pattern, often outperforms auto-vectorization because the programmer can assert alignment, lane counts, and tail handling that the compiler cannot safely assume.
The Takeaway
The post is a refreshingly grounded introduction. It doesn't promise that every loop should be vectorized, nor does it dive into the deep end of permute instructions or gather/scatter patterns. Instead, it gives working engineers a mental model and a concrete recipe for the 80% case. For anyone who regularly processes arrays of bytes, integers, or floats, this pattern is a tool worth having.
The common 'process N values at a time' SIMD code follows the same five steps. Once you learn the basics, writing SIMD is just about as easy as a for loop.