sparse-dot-topn

What it is and what it does

sparse_dot_topn is a Python package that accelerates a common operation in large-scale similarity matching: multiplying two sparse matrices and keeping only the top-n results per row. It wraps a C++ extension (with vendored OpenMP) to perform this combined operation in a single pass, avoiding the memory overhead of computing the full product and then filtering. The package supports CSR, CSC, and COO sparse matrix formats with 32- and 64-bit integer and float data types, and can parallelize across multiple threads.

The main use case is comparing very large feature vectors (e.g., TF-IDF matrices) and selecting the best matches efficiently. It offers optional threshold filtering to discard low-scoring results and density hints to reduce pre-allocated memory. The package is actively maintained, supports modern Python versions (3.9–3.12), and provides pre-built wheels for common platforms, though source installation is possible for architecture-specific optimization.

Use it for:

• Find top-10 most similar documents in a corpus by multiplying TF-IDF matrices and retaining only the highest-scoring matches per row.
• Distribute large-scale matrix multiplication across a cluster by splitting matrices into chunks, computing top-n products per chunk, and re-zipping results.
• Reduce memory footprint when comparing high-dimensional embeddings by filtering to top-n results during multiplication rather than storing the full product.
• Parallelize sparse matrix operations across multiple CPU cores to speed up similarity computations on multi-core systems.
• Apply threshold-based filtering during multiplication to discard low-confidence matches and further reduce output size.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Performs fast sparse matrix multiplication with integrated top-n result selection, reducing memory footprint and improving performance over separate multiplication and filtering steps.

Yes. The package solves a genuine performance and memory bottleneck in large-scale sparse matrix operations. It is actively maintained, has no known vulnerabilities, carries a permissive license, and provides convenient pre-built wheels. Install friction is moderate but manageable. Recommended for projects involving large sparse matrix similarity matching or top-n filtering.

Install

sparse-dot-topn on PyPI

pip install sparse-dot-topn

uv add sparse-dot-topn

poetry add sparse-dot-topn

Installing sparse-dot-topn

Before you install

Medium install friction due to C++ extension dependency, but wheels are pre-built for Python 3.9–3.12 on Windows, Linux, and macOS (x86 and ARM). Maintenance is active with recent releases and a healthy repository. Installation from wheels is straightforward; source builds require a C++17 compiler.

License in practice

Licensed under Apache Software License (permissive), imposing no significant restrictions on use, modification, or distribution in most contexts.

Quickstart

pip install sparse_dot_topn

import scipy.sparse as sparse
from sparse_dot_topn import sp_matmul_topn

A = sparse.random(1000, 100, density=0.1, format="csr")
B = sparse.random(100, 2000, density=0.1, format="csr")
C = sp_matmul_topn(A, B, top_n=10)

Requires Python 3.9 or later. C++ extension is vendored with OpenMP for parallelization; if OpenMP issues arise, omit n_threads argument or see INSTALLATION.md.

Verify before relying

• Whether the 6× speedup claim on Apple M2 Pro generalizes to other hardware platforms and matrix sizes.
• Performance characteristics when threshold and density parameters are tuned for specific use cases.
• Behavior and performance with negative-valued sparse matrices beyond the stated default change.

Package facts

Evidence: sparse_dot_topn-1.2.0-cp310-cp310-macosx_12_0_arm64.whl; sparse_dot_topn-1.2.0-cp310-cp310-macosx_12_0_x86_64.whl; sparse_dot_topn-1.2.0-cp310-cp310-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl; sparse_dot_topn-1.2.0-cp310-cp310-win_amd64.whl; sparse_dot_topn-1.2.0-cp311-cp311-macosx_12_0_arm64.whl; sparse_dot_topn-1.2.0-cp311-cp311-macosx_12_0_x86_64.whl; sparse_dot_topn-1.2.0-cp311-cp311-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl; sparse_dot_topn-1.2.0-cp311-cp311-win_amd64.whl; sparse_dot_topn-1.2.0-cp312-abi3-macosx_12_0_arm64.whl; sparse_dot_topn-1.2.0-cp312-abi3-macosx_12_0_x86_64.whl; sparse_dot_topn-1.2.0-cp312-abi3-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl; sparse_dot_topn-1.2.0-cp312-abi3-win_amd64.whl; sparse_dot_topn-1.2.0-cp39-cp39-macosx_12_0_arm64.whl; sparse_dot_topn-1.2.0-cp39-cp39-macosx_12_0_x86_64.whl; sparse_dot_topn-1.2.0-cp39-cp39-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl; sparse_dot_topn-1.2.0-cp39-cp39-win_amd64.whl

Tags

More Mathematics packages