LEANN

Anyone who has tried to build retrieval-augmented generation over their own files quickly hits a hidden tax: the vector index. Embedding a personal corpus of emails, PDFs, chat logs and source code produces gigabytes of dense vectors that have to live on disk, often dwarfing the original data. That cost is what keeps serious on-device, private RAG out of reach for most laptops. LEANN, from the StarTrail-org team and Berkeley's Sky Computing Lab, is a Python vector database built to remove that tax. It bills itself as RAG on everything, indexing diverse personal sources locally with zero telemetry. The headline claim is roughly 97% storage savings versus a conventional index like FAISS, while keeping search quality competitive. Whether that holds for your data is the question this analysis tries to frame honestly.

The distinctive idea is to not store most embeddings at all. Instead of persisting a vector for every chunk, LEANN keeps a pruned proximity graph and recomputes embeddings on demand only for the nodes its traversal actually visits during a query. A high-degree preserving pruning step keeps well-connected hub nodes while dropping redundant edges, and dynamic batching plus a two-level search keep recomputation efficient. The default HNSW backend maximizes storage savings by recomputing at search time, while an optional DiskANN backend trades some of that saving for speed using product quantization and reranking. The approach is documented in a peer-reviewed MLsys 2026 paper, LEANN: A Low-Storage Vector Index, which is a stronger signal of validity than most open-source projects can offer.

LEANN targets developers and technical users who want private retrieval over personal data without shipping it to a cloud service or paying the disk cost of a full vector store. Concrete sources it can index include PDFs, Markdown and Office documents, Apple Mail, Chrome history, WeChat, iMessage and Slack, exported ChatGPT and Claude conversations, code repositories with AST-aware chunking, and anything reachable through an MCP server. It runs as a native MCP service, so it can act as a drop-in semantic search layer for Claude Code, which otherwise leans on keyword search. Embeddings can come from local models via Ollama, LM Studio or llama.cpp, or from hosted providers, with separate choices for embedding and generation. The natural fit is a personal knowledge base you carry between machines.

The tradeoffs are real and worth weighing before adoption. Storage savings are paid back as compute: every query recomputes embeddings for visited nodes, so retrieval is slower than reading precomputed vectors and leans on a capable CPU or GPU. The project is at v0.x, which signals an unstable API, and recent commits skew toward fixes and CI rather than new features, so momentum is something to watch. Setup is not plug-and-play: it depends on compiled native libraries (libomp, Boost, Protobuf, ZeroMQ) and an embedding-model stack, which is friendlier on Linux than on Windows. Several conveniences, including some email and messaging connectors, are macOS-centric, with Windows support listed as coming. The strong recall and 97% figures are the project's own benchmarks, so verify on your corpus.

Against managed services like Pinecone or Weaviate, LEANN plays a different game. Those platforms optimize for low-latency retrieval at scale with operational polish, and they store full vectors on infrastructure you pay for and trust with your data. LEANN optimizes for the opposite corner: keep everything local, keep the index tiny, and accept recomputation latency as the price. For private on-device RAG over emails, chat logs and code, where storage and confidentiality matter more than millisecond response times, it is a credible and well-researched choice. It is too early where you need latency-sensitive production retrieval at scale, multi-user serving, or a stable API you can build a product on. For tinkerers and privacy-minded builders, it is worth a serious trial; for production teams, watch it mature.