lumabri: Run Huge MoE Models Across a Swarm of Peers, Pure C

lumabri is a new open-source project that turns a swarm of ordinary machines into a distributed inference cluster for huge mixture-of-experts (MoE) models. Built on the colibri engine and written in pure C with no dependencies, it lets one machine share a model while any other machine chats with it — no upfront download, no GPU required.

The core trick is an LD_PRELOAD shim (liblumabri.so) that intercepts libc calls like open, pread, and opendir. Files appear as sparse local mirrors of the true size, so the engine's normal file operations work. Missing blocks are fetched from peers on first use and cached locally, so subsequent reads hit local disk at full speed. No FUSE, no daemon on the read path.

For MoE models, the chatter keeps only the dense weights, router, and KV cache. The 4 KB activation for each routed expert is sent to the peer that holds those expert weights. Expert weights never reach the chatter. Both sides are built from the same engine source, so local and distributed runs produce byte-identical tokens.

Trust is handled carefully. Every maintainer computes a sha256 per MiB and sends it with registration. The origin signs that truth with an offline ed25519 key. A chatter verifies every block against a key it holds itself. For remote compute, LUMABRI_VERIFY=N reruns N percent of expert calls on a second replica and demands identical output. A disagreement stops the run.

Supported engines include OLMoE, GLM, Inkling, Kimi K3, and DeepSeek V4, each with a per-engine expert node. The project claims bit-for-bit identical tokens between local and distributed runs, proven by a test that caught a real bug in GLM's expert handling.

Deployment is straightforward: make, then lumabri serve --model /path/to/model on the server, and lumabri chat --tracker <server-ip>:7300 --engines-dir /path/to/colibri/c on clients. Disk and compute donors join with simple commands. Optional encryption via X25519/Ed25519 and ChaCha20-Poly1305 is available.