Google's HEIR Compiler Makes Homomorphic Encryption Practical for AI Inference

Google has open-sourced HEIR (Homomorphic Encryption Intermediate Representation), a compiler toolchain that converts pre-trained AI models to operate on encrypted inputs. The goal is to make homomorphic encryption (HE) a one-click solution for developers who want private AI inference without a PhD in cryptography.

Homomorphic encryption allows computations on ciphertexts, so a server can process encrypted data and return encrypted results without ever seeing the underlying plaintext. This flips the classic privacy/functionality trade-off: instead of choosing between protecting user data and providing features like spam detection or recommendations, you can have both—at a computational cost. And that cost is dropping fast.

Google's history in privacy tech includes differential privacy, private set membership, private information retrieval, and confidential computing on Google Cloud. HEIR is the newest addition, and unlike hardware-based secure enclaves, its guarantees are purely cryptographic.

HEIR has been in development since 2023 and has attracted partners building hardware accelerators for HE, including Belfort, Niobium, Cornami, and Optalysys. It's also become a research platform, with collaborations at Georgia Tech, Carnegie Mellon, UC Santa Barbara, and others. Four peer-reviewed papers have been built on HEIR so far.

To show HE is practical, Google shared four private inference applications compiled with HEIR, with single-threaded CPU latency numbers:

• Deep Learning Recommendation Model for private content recommendations (with Belfort Labs, LG, NYU)
• Credit card fraud detection (with Niobium and hardshell.ai)
• Kitsune network anomaly detection on encrypted traffic (with Niobium)
• Hotword detection for audio-triggered AI agents (with Belfort Labs)

The source code for all demos is on GitHub. The big remaining question is performance: HE has a nontrivial overhead, and while accelerators promise to help, Google hasn't yet published latency numbers for them.