rensa

What it is and what it does

Rensa is a Rust-based MinHash library that estimates Jaccard similarity between sets and identifies near-duplicates in large datasets. It implements two variants: R-MinHash (Rensa's own, optimized for speed) and C-MinHash (based on published research with formal variance bounds). The core algorithm applies k random hash functions to a set and keeps the minimum value from each; sets sharing many elements produce similar minimums, and the fraction of matching slots estimates the Jaccard index.

Rensa replaces the traditional modular reduction step with multiply-shift hashing, which is faster on modern CPUs and naturally produces 32-bit signatures (half the memory of standard implementations). It includes batch APIs for bulk operations, LSH (Locality Sensitive Hashing) indexing for efficient candidate retrieval, and a deduplicator for end-to-end workflows. The library has no runtime dependencies and supports Python 3.8 and later on CPython and PyPy.

Use it for:

• Find and remove near-duplicate documents or SQL queries in large datasets using LSH indexing and batch deduplication.
• Estimate Jaccard similarity between tokenized text sets without computing exact set intersection.
• Build a searchable index of document signatures to quickly retrieve candidates above a similarity threshold.
• Deduplicate synthetic or generated datasets where exact matching is too strict but similarity thresholds are appropriate.
• Batch-process thousands of token sets into MinHash signatures with minimal Python call overhead.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Rensa computes MinHash signatures for fast similarity estimation and deduplication of large datasets using Rust-backed algorithms with Python bindings.

Yes. Rensa is actively maintained, has no known vulnerabilities, carries a permissive MIT license, and offers substantial speed gains over comparable Python libraries (datasketch, FastSketch) with near-identical results. Install friction is moderate due to compiled bindings, but prebuilt wheels cover common platforms. Use it if you need to deduplicate or find similar items in large datasets and speed matters.

Install

rensa on PyPI

pip install rensa

uv add rensa

poetry add rensa

Installing rensa

Before you install

Medium install friction due to compiled Rust bindings, but wheels are prebuilt for common platforms (Linux, macOS, Windows) and Python versions. Active maintenance with recent releases; last commit 2026-08-04.

License in practice

MIT license is permissive; you can use, modify, and distribute this package with minimal restrictions.

Quickstart

pip install rensa

from rensa import RMinHash

m1 = RMinHash(num_perm=128, seed=42)
m1.update("the quick brown fox".split())

m2 = RMinHash(num_perm=128, seed=42)
m2.update("the quick brown cat".split())

print(m1.jaccard(m2))

Requires Python >= 3.8. Prebuilt wheels available for Linux, macOS, and Windows; other platforms may require building from source.

Verify before relying

• Whether the 608.52x speedup vs datasketch holds across all use cases or only the reference benchmark conditions.
• Whether C-MinHash's formal variance proofs provide practical advantages for specific deduplication thresholds.
• Performance characteristics when working with non-ASCII or very large token sets.

Package facts

Evidence: rensa-0.4.1-cp310-cp310-manylinux_2_17_i686.manylinux2014_i686.whl; rensa-0.4.1-cp310-cp310-manylinux_2_28_aarch64.whl; rensa-0.4.1-cp310-cp310-manylinux_2_28_armv7l.whl; rensa-0.4.1-cp310-cp310-manylinux_2_28_ppc64le.whl; rensa-0.4.1-cp310-cp310-manylinux_2_28_s390x.whl; rensa-0.4.1-cp310-cp310-manylinux_2_28_x86_64.whl; rensa-0.4.1-cp310-cp310-musllinux_1_2_aarch64.whl; rensa-0.4.1-cp310-cp310-musllinux_1_2_armv7l.whl; rensa-0.4.1-cp310-cp310-musllinux_1_2_i686.whl; rensa-0.4.1-cp310-cp310-musllinux_1_2_x86_64.whl; rensa-0.4.1-cp310-cp310-win_amd64.whl; rensa-0.4.1-cp311-cp311-macosx_10_12_x86_64.whl; rensa-0.4.1-cp311-cp311-macosx_11_0_arm64.whl; rensa-0.4.1-cp311-cp311-manylinux_2_17_i686.manylinux2014_i686.whl; rensa-0.4.1-cp311-cp311-manylinux_2_28_aarch64.whl; rensa-0.4.1-cp311-cp311-manylinux_2_28_armv7l.whl; rensa-0.4.1-cp311-cp311-manylinux_2_28_ppc64le.whl; rensa-0.4.1-cp311-cp311-manylinux_2_28_s390x.whl; rensa-0.4.1-cp311-cp311-manylinux_2_28_x86_64.whl; rensa-0.4.1-cp311-cp311-musllinux_1_2_aarch64.whl

Tags

More Information Analysis packages