pyprobables

What it is and what it does

pyprobables is a pure-Python library implementing common probabilistic data structures—Bloom filters, Count-Min sketches, Cuckoo filters, and Quotient filters. These structures trade a small, tunable false-positive rate for dramatic memory savings compared to exact data structures, making them useful when you need fast approximate answers about set membership or frequency counts without storing the full dataset.

The library is straightforward to use: instantiate a structure with your parameters (element count, false-positive tolerance, or capacity), then add items and query them. The documentation notes that C-compiled hashing algorithms can improve raw performance if needed. The package is actively maintained, supports modern Python versions, and carries no external dependencies.

Use it for:

• Implement a URL deduplicator for web crawlers to avoid revisiting pages without storing every URL in memory.
• Track which user IDs have already been processed in a stream-processing pipeline with bounded memory.
• Estimate word frequencies in large text corpora using Count-Min sketch without storing exact counts.
• Build a cache-miss detector or negative-lookup filter for database queries to avoid expensive lookups.
• Detect duplicate network packets or log entries in real-time monitoring systems with minimal overhead.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Provides pure-Python implementations of probabilistic data structures including Bloom filters, Count-Min sketches, Cuckoo filters, and Quotient filters for memory-efficient membership testing and frequency estimation.

Yes. pyprobables is a solid, actively maintained library with zero dependencies, permissive MIT licensing, and clean API design. Install it if you need any probabilistic data structure for approximate membership testing or frequency estimation. No security vulnerabilities reported.

Install

pyprobables on PyPI

pip install pyprobables

uv add pyprobables

poetry add pyprobables

Installing pyprobables

Before you install

Installs cleanly with no runtime dependencies. Actively maintained with a recent release and ongoing commits. Supports current Python versions (3.10–3.14+).

License in practice

MIT license is permissive and poses no restrictions on commercial or private use, modification, or redistribution.

Quickstart

pip install pyprobables

from probables import BloomFilter
blm = BloomFilter(est_elements=1000, false_positive_rate=0.05)
blm.add('google.com')
print(blm.check('google.com'))  # True

Requires Python 3.10 or later.

Verify before relying

• Whether the pure-Python implementation meets latency or throughput requirements without optional C-compiled hash libraries.
• Memory overhead and scalability limits for each data structure at production scale.
• Performance characteristics compared to other probabilistic data structure implementations.

Package facts

Evidence: pyprobables-0.7.0-py3-none-any.whl

Keywords: python, probabilistic, data-structure, bloom, filter, count-min, sketch, bloom-filter, count-min-sketch, cuckoo-filter, quotient-filter

Tags

More Libraries packages