LEANN: A New Open-Source Vector Index from UC Berkeley Reduces RAG Index Size Dramatically

28 October 2026 3 min read هدهد تك | Hudhud Tech

UC Berkeley’s Sky Computing Lab has developed LEANN, an open-source vector index that significantly reduces storage requirements by not storing embeddings. Instead, LEANN recomputes embeddings at query time, resulting in a dramatic reduction in size from 201 GB to just 6 GB for a 60 million chunk Wikipedia index. This innovation earned the Best Research Paper award at MLSys 2026, enabling semantic search and retrieval-augmented generation (RAG) over personal data on standard laptops.

TL;DR

Size: 6 GB for 60M chunks (repo); 4 GB in the paper’s 76 GB Wikipedia setup. Default embedder: Contriever (110M params). It is an index, not an LLM, so no context length applies.
Runs on: laptops and workstations. The paper tested an RTX 4090 box with 32 GB RAM and an M1 Ultra Mac. A CPU-only Linux install exists, with no published CPU latency.
Performance: matches HNSW downstream accuracy at 90% recall, using under 5% of raw-data storage.
Best: 25.5% NQ accuracy, identical to HNSW, using 4 GB instead of 188 GB.
Worst: HotpotQA retrieval takes 44.80 s on the Mac, 50.1% of total pipeline time.

What is LEANN?

LEANN is a low-storage approximate nearest neighbor (ANN) index built on top of FAISS. It addresses the issue of vector indexes often being larger than the data they index. The research paper indicates that a 76 GB corpus requires 173 GB of embeddings plus a 15 GB HNSW graph, effectively more than doubling the raw data size.

LEANN maintains a pruned graph, discarding full vectors and rebuilding them on demand. The code is available under the MIT license and can be installed with uv pip install leann.

How does LEANN recompute embeddings without stalling search?

Graph indexes like HNSW typically visit only a small subset of nodes per query, and LEANN takes advantage of this by storing the graph along with a compact product quantization (PQ) table. During a query, it re-embeds only the nodes along the search path.

Three techniques enhance this process:

Two-level search: inexpensive PQ distances rank neighbors initially, with only top candidates undergoing exact recomputation.
Dynamic batching: candidates from multiple hops are grouped, for instance, processing 64 at once to improve GPU efficiency.
High-degree preserving pruning: most nodes retain few edges, while the top 2% of “hub” nodes maintain more connections.

In ablation studies, the two-level search method yielded an average speedup of 1.4 times, while adding batching increased it to 1.8 times. However, embedding recomputation still accounts for approximately 76% of query latency.