Slotstream

Mixture-of-experts models are getting large enough that even a well-specified Mac cannot hold them entirely in RAM, and a 105 GB, 125-billion-parameter model like Qwen3.8-Flash-Next is a clear example. Slotstream, built by Carlos Galarza, targets exactly that gap: Macs with 16 to 64 GB of unified memory that are far short of what the model needs if loaded whole. The pitch on the README is direct: run a 105 GB model on a 48 GB Mac by keeping most of it on SSD and loading only the parts you need. It is a native Swift program built on Apple's MLX and Metal, with no Python runtime anywhere in the stack.

The technical approach leans on the model's own mixture-of-experts structure. Shared weights stay resident in memory, while the experts a given token actually needs are read from SSD into a slot-based cache, with automatic memory planning and expert lookahead prediction trying to fetch the right experts before they are needed rather than after. The project exposes both OpenAI- and Ollama-compatible APIs, so existing tooling can point at it with minimal changes, and it runs fully offline with no Python dependency and no cloud account required. Distinctively, the project backs its numbers with published measurement and hardware compatibility documents rather than a single headline benchmark.

The fit is narrow but real: Apple Silicon owners in the 16 to 64 GB range who want to run large mixture-of-experts models that would otherwise require far more memory than they have. Because it speaks the Ollama and OpenAI protocols, it slots into existing local-LLM tooling rather than demanding a new client. The community-contributed hardware reports in the repository, covering machines from smaller MacBooks up to an M5 Pro with 48 GB, give prospective users a way to check whether their specific machine is likely to work before committing.

The honest caveats start with platform and maturity. Slotstream is Apple Silicon only, and the model weights it targets carry Qwen's own community license, separate from the project's MIT code license. Versioning is still in the 0.2.x range, and the issue tracker includes a real report of long re-prefills causing the server to exit silently after 15 or more minutes, the kind of failure mode that matters for anything unattended. Worth flagging too: the same author lists a commercial Mac app, Sevra, on the project homepage, so it is worth watching whether the open engine and the paid product diverge as the app matures.

The verdict: Slotstream is a serious, well-documented engine for a specific and growing problem, running large MoE models on memory-constrained Apple Silicon, backed by real measurement work rather than marketing claims alone. Adopt it if you have a 16 to 64 GB Mac and want to try a model that would not otherwise fit, and you are comfortable with 0.2.x-stage software. Hold off if you need non-Apple hardware, guaranteed uptime for long-running prefills, or want to wait until the relationship between the open project and its commercial sibling app is clearer.