turbovec

Vector search underpins most retrieval-augmented generation and semantic search systems, but storing embeddings at full precision is expensive. A 10 million document corpus of 1536-dimension float32 vectors needs roughly 31 GB of RAM, which pushes teams toward bigger machines or managed services. turbovec, a Rust library with Python bindings from solo developer Ryan Codrai, attacks this directly. It is a compressed vector index that holds the same corpus in about 4 GB, a 16x reduction, while keeping search fast. The project frames itself as an open implementation of TurboQuant, a quantization algorithm published by Google Research at ICLR 2026, packaged with hand-written SIMD kernels and drop-in adapters for common RAG frameworks so it can slot into existing pipelines without much rework.

TurboQuant is data-oblivious: it compresses vectors using the mathematical behavior of normalized, randomly rotated distributions rather than learning from your data. The pipeline extracts each vector length, applies an orthogonal rotation so coordinates follow predictable Beta distributions, then uses precomputed Lloyd-Max buckets (4 for 2-bit, 16 for 4-bit) and bit-packing, with a per-coordinate calibration step the authors call TQ+ to correct for finite-dimension drift. The notable consequence is that there is no training phase. You can ingest vectors online and persist the index to disk. Filtering is built into the SIMD kernel at 32-vector block granularity, so blocks with no allowed slots are skipped before any scoring, which keeps allowlist-restricted queries efficient. NEON and AVX-512 kernels target ARM and modern x86 respectively.

turbovec suits teams running fast semantic search over large corpora where storage or memory budgets are tight, especially the 10 million document range and above. Because there is no training step, it fits dynamic collections that grow continuously, such as a knowledge base or document store that ingests new content throughout the day, where retraining a quantizer would be awkward. The stable external ID support, with add_with_ids, remove, and allowlist filtering, makes it usable for multi-tenant setups or per-user document scoping inside a RAG application. The framework adapters for LangChain, LlamaIndex, Haystack, and Agno mean it can replace a heavier vector store in a prototype or self-hosted deployment, and the Rust core is also available directly via cargo for teams building outside Python.

The honest caveats are significant. turbovec is a single-developer project with no visible team or organizational backing, and the repository is young and beta in maturity, so production reliability at scale is unproven. The compression-versus-recall trade-offs rest largely on the project's own benchmarks: it reports beating FAISS by roughly 0.2 to 1.9 points at R@1 on OpenAI embeddings and winning on speed by 10 to 19 percent on Apple silicon, but these figures are not independently verified, and the authors openly note they trail FAISS on 2-bit x86 where its AVX-512 VBMI path wins. They also flag that the Beta distribution assumption is asymptotic and drifts at finite dimensions, especially for low-bit and word-vector embeddings, which is why TQ+ calibration exists. Quantization is inherently lossy, so maximum-recall cases will feel the cost.

Against managed offerings like Pinecone or Zilliz Cloud, turbovec is a different proposition: a library you embed and operate yourself, not a hosted service with an SLA, replication, or a support contract. That makes it attractive when you want control, low storage cost, and no per-vector pricing, and when your team is comfortable owning the operational side. It is worth adopting for fast semantic search over very large corpora where full float32 embeddings are too expensive, particularly on ARM hardware where its kernels look strongest. It is too early when you need guaranteed maximum recall regardless of storage, or a commercially supported vector database with uptime commitments. The peer-reviewed algorithm and clean API are genuine strengths, but treat it as promising infrastructure to pilot, benchmark on your own data, and validate before betting production on it.