SwarmLLM

Running a large language model has always assumed one machine holds the whole thing, which locks out anyone whose single device, laptop, phone, or desktop, is not individually powerful enough. SwarmLLM, built by Nehanth Narendrula, questions that assumption by splitting a model's layers across multiple browser tabs on different devices, connected over WebRTC with no server and no installation required. The project's own framing captures the idea directly: every device brings a slice, together they run the whole model. A demo pairing a MacBook and an iPhone to run a 27B model is the clearest illustration of what the project is actually trying to prove.

The engineering is built from scratch rather than borrowed. SwarmLLM ships a custom WebGPU engine written in roughly 50 WGSL kernels, its own GGUF parser to load quantized weights, and a WebRTC mesh layer, room.js, that handles signaling and peer connections across networks. Each peer processes its slice of layers and only passes a small activation vector, around 10 KB, to the next device in the chain, rather than shipping raw weights or full hidden states. Four-bit quantization and speculative decoding with multi-token prediction reduce both memory and per-token latency, and the project gates every optimization on a golden-test suite intended to keep numerical results bit-exact.

The natural fit is technically curious users experimenting with pooling personal devices, a laptop plus a phone, or a household's collection of underused hardware, to run a model too large for any single one of them, entirely inside the browser with nothing to install. Because it needs no server, it also suits people who want to try distributed inference without standing up infrastructure. It is not yet aimed at production workloads; the room-based, ad hoc connection model (visit swarmllm.ai/room to start or join) reads as an experiment you run deliberately rather than an always-on service.

The caveats are substantial and mostly acknowledged by the project's own issue tracker. Only one release exists so far (v0.2.0), and the 2,048-token context window is explicitly flagged internally as needing to grow, with a hard 400-token answer cap still in place. Reliability depends on every participating device staying connected: open issues describe worker tabs dropping under memory pressure or when a browser tab is backgrounded, and data-channel congestion control adding real latency per hop. The throughput figures, 9 tokens per second without speculative decoding and 16 with it on a GB10, or 10.7 tokens per second in the MacBook-plus-iPhone demo, are the project's own benchmarks on its own hardware.

The verdict: SwarmLLM is a genuinely novel and well-engineered answer to a question most local-inference projects do not even ask, whether several personal devices can jointly run a model none of them could run alone. Adopt it as an experiment if you are comfortable in a pre-1.0, browser-based, ad hoc environment and want to see distributed inference working today. It is too early for anything resembling production use, given the single release, the openly documented context and reliability limits, and throughput numbers that remain unverified outside the project's own testing.