simhash

What it is and what it does

Simhash is a Python implementation of Google's Simhash algorithm, which generates a compact fingerprint for text documents. Rather than comparing full documents, Simhash produces a fixed-size hash that allows fast detection of near-duplicate content—documents with small differences will have similar hashes. This is useful for deduplication in web crawlers, duplicate detection in databases, and content similarity analysis.

The package depends on numpy for numerical operations. It has been stable since its last release in 2022 but is no longer actively maintained. With 1038 GitHub stars and consistent monthly downloads, it remains a recognized tool for this specific task, though you should verify compatibility with your Python and numpy versions before relying on it in production.

Use it for:

• Detect near-duplicate web pages or documents in a crawl or corpus to avoid storing redundant content.
• Identify similar text submissions in bulk uploads to flag potential plagiarism or spam.
• Deduplicate large document collections by grouping content with similar Simhash values.
• Build a similarity index for fast approximate matching of new documents against a known set.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Computes Simhash fingerprints for text and documents to detect near-duplicate content with a single hash value.

Yes, if you need Simhash specifically and can verify it works with your Python and numpy versions. The algorithm is well-established and the package is stable, but abandonment means no fixes for future compatibility issues. Suitable for projects where you control dependencies and can test thoroughly before deployment.

Install

simhash on PyPI

pip install simhash

uv add simhash

poetry add simhash

Installing simhash

Before you install

Low friction install with only numpy as a runtime dependency. However, the package is abandoned—last release was 2022-03-03 and last commit 2022-03-24, with no active maintenance.

License in practice

MIT License permits free use, modification, and distribution with minimal restrictions, making it safe to incorporate into most projects without licensing concerns.

Quickstart

pip install simhash

from simhash import Simhash

hash1 = Simhash('This is a test document').value
hash2 = Simhash('This is a test document with minor changes').value

Verify before relying

• Whether the package works correctly with current numpy versions and modern Python releases
• Performance characteristics on large document collections or real-world deduplication workloads

Package facts

Evidence: simhash-2.1.2-py2-none-any.whl; simhash-2.1.2-py3-none-any.whl

Keywords: simhash

Tags

More Information Analysis packages