splink

What it is and what it does

Splink is a Python package for probabilistic record linkage that solves the problem of matching and deduplicating records when no unique identifier exists. It uses the Fellegi-Sunter statistical model to compute match probabilities between record pairs, supporting fuzzy matching, term frequency adjustments, and user-defined comparison logic. The package works by comparing multiple non-correlated columns (such as name, date of birth, and location for persons), estimating model parameters through unsupervised learning, and clustering pairwise predictions to generate estimated entity IDs.

The package is designed for datasets with multiple descriptive columns and runs on a local laptop via DuckDB or scales to 100+ million records on big-data backends like AWS Athena or Spark. It includes interactive visualizations to help diagnose model performance and is widely used in government, academia, and the private sector. Runtime dependencies include altair, duckdb, igraph, jinja2, numpy, pandas, and sqlglot.

Use it for:

• Deduplicate customer or patient records in databases lacking a master identifier.
• Link census or survey data across years or sources to track population changes.
• Match company records across datasets with different naming conventions or incomplete information.
• Resolve entity identity in fraud detection or compliance workflows where records may be partially obscured.
• Consolidate data from multiple administrative systems without a shared key.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Splink performs probabilistic record linkage and deduplication, matching records across datasets that lack unique identifiers by comparing multiple columns and assigning match probabilities.

Yes. Splink is actively maintained, has no known vulnerabilities, installs with low friction, and solves a specific and difficult problem (record linkage without unique identifiers) that has few mature alternatives in Python. The MIT license and strong maintenance signal (recent release, active repository) make it suitable for production use in government, academic, and commercial contexts. Install if you need to deduplicate or link records across datasets.

Install

splink on PyPI

pip install splink

uv add splink

poetry add splink

Installing splink

Before you install

Low friction install with a pure Python wheel. Actively maintained with a recent release (156 days ago) and 2338 repository stars. Supports current Python versions (3.9+).

License in practice

MIT license permits commercial and private use with minimal restrictions, making it suitable for government, academic, and private sector deployments.

Quickstart

pip install splink

import splink.comparison_library as cl
from splink import DuckDBAPI, Linker, SettingsCreator, block_on, splink_datasets

db_api = DuckDBAPI()
df = splink_datasets.fake_1000
settings = SettingsCreator(link_type="dedupe_only", comparisons=[cl.ExactMatch("name")])
linker = Linker(df, settings, db_api)
pairwise_predictions = linker.inference.predict()

Requires Python 3.9 or later; DuckDB is a runtime dependency for local linkage execution.

Verify before relying

• Whether optional backend installations (Spark, Athena, PostgreSQL) are commonly used or if DuckDB covers most use cases.
• Performance characteristics on datasets larger than the stated 'million records on a laptop in around a minute' benchmark.
• Whether the Fellegi-Sunter model's accuracy claims have been independently validated outside government case studies.

Package facts

Evidence: splink-4.0.16-py3-none-any.whl

Tags

More Information Analysis packages