annoy

What it is and what it does

Annoy is a nearest-neighbor search library built on C++ with Python bindings, designed to find points in space closest to a query point across many dimensions. It trades some accuracy for speed by building forests of random trees, allowing you to tune the tradeoff between search speed and precision at query time. The core innovation is that indexes are stored as static, memory-mapped files on disk—meaning you build an index once, save it, and then any number of processes can load and query it simultaneously without rebuilding or copying data.

The library supports multiple distance metrics (Euclidean, Manhattan, cosine, Hamming, and dot product) and is optimized for memory efficiency, making it practical for large datasets that would otherwise exhaust RAM. It was originally built at Spotify for music recommendation systems working with millions of high-dimensional vectors. You create an index by adding vectors with integer IDs, build a forest of trees to structure the search space, save the index to disk, and then load it (via mmap) in any process to perform fast approximate nearest-neighbor queries.

Use it for

• Build a recommendation engine that finds similar users or items by querying pre-built vector indexes across multiple application servers
• Index millions of embeddings from a machine learning model and serve nearest-neighbor queries in production without rebuilding indexes
• Share a single large nearest-neighbor index across many parallel processes or Hadoop jobs without duplicating data in memory
• Implement similarity search for music, images, or text embeddings where approximate results are acceptable and speed matters more than perfect accuracy
• Reduce memory footprint for high-dimensional vector search by using disk-based indexes that are memory-mapped on demand

Worth the install?

AI-flagged interpretation of the facts on this page. Verify before relying on it.

Install

annoy on PyPI

Before you install

Medium install friction due to compiled C++ components requiring a build step. The package is aging (last release 2023-06-14, 1157 days ago) but remains actively maintained with no recent commits blocked; the repository is not archived and has substantial community adoption (14285 stars).

Requires a C++ compiler and build tools to compile the native extension during installation.

License in practice

Licensed under Apache License 2.0, a permissive license that allows commercial and private use with minimal restrictions, making it safe for most production deployments.

Quickstart

pip install annoy

from annoy import AnnoyIndex
import random

f = 40
t = AnnoyIndex(f, 'angular')
for i in range(1000):
    v = [random.gauss(0, 1) for z in range(f)]
    t.add_item(i, v)

t.build(10)
t.save('test.ann')

u = AnnoyIndex(f, 'angular')
u.load('test.ann')
neighbors = u.get_nns_by_item(0, 100)

Package facts

Evidence: annoy-1.17.3-cp310-cp310-macosx_11_0_arm64.whl

Tags

More Scientific/Engineering packages

See also pynndescent · voyager · nmslib · usearch · scann · pyspark-hnsw · libcuvs-cu12 · simsimd · faiss-gpu · smmap