datacompy

What it is and what it does

DataComPy is a DataFrame comparison tool that goes beyond simple equality checks by identifying and reporting row and column differences across Pandas, Polars, Spark, and Snowflake tables. It was originally designed as a Python equivalent to SAS's PROC COMPARE, offering structured output that can be rendered as text reports, HTML files, or JSON-serializable dictionaries for programmatic use.

The package accepts two DataFrames, specifies join columns for row matching, and produces detailed statistics on mismatches. It exposes a ReportData API that lets you access comparison results programmatically without relying on string parsing, making it suitable for dashboards, automated validation pipelines, and data quality checks. Version 1.0.4 is the current stable release; the v0.19.x line is no longer supported.

Use it for:

• Validate that ETL transformations produce expected output by comparing input and output DataFrames.
• Detect data drift or corruption by comparing snapshots of production tables at different points in time.
• Automate regression testing for data pipelines by comparing test results against baseline DataFrames.
• Generate audit reports showing which rows and columns differ between two database exports or Snowflake tables.
• Build data quality dashboards that consume structured comparison results via the ReportData API.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

DataComPy compares two DataFrames across Pandas, Polars, Spark, and Snowflake, reporting differences in rows and columns with configurable matching tolerance and structured output for programmatic access.

Yes. DataComPy is actively maintained, has no known vulnerabilities, low install friction, and solves a concrete problem for data engineers and QA teams. The permissive Apache license and support for multiple backends (Pandas, Polars, Spark, Snowflake) make it a practical choice for DataFrame validation and comparison workflows. Install it if you need to compare tabular data programmatically or generate detailed difference reports.

Install

datacompy on PyPI

pip install datacompy

uv add datacompy

poetry add datacompy

Installing datacompy

Before you install

Low install friction with a pure-wheel distribution. Active maintenance with a recent release (15 days ago) and ongoing commits. Runtime dependencies are well-established data libraries (pandas, polars, numpy, jinja2, ordered-set); optional extras for Spark and Snowflake are available but not required for core use.

License in practice

Apache Software License (permissive) places no significant restrictions on use, modification, or distribution in proprietary or open-source projects.

Quickstart

pip install datacompy

import pandas as pd
from datacompy import PandasCompare

df1 = pd.DataFrame({"id": [1, 2, 3], "val": [10, 20, 30]})
df2 = pd.DataFrame({"id": [1, 2, 3], "val": [10, 99, 30]})
compare = PandasCompare(df1, df2, join_columns="id")
print(compare.report())

Requires Python 3.10 or later.

Verify before relying

• Whether tolerance/matching configuration options are documented and how granular they are.
• Performance characteristics when comparing very large DataFrames across backends.
• Whether Spark and Snowflake backends require additional system dependencies beyond pip extras.

Package facts

Evidence: datacompy-1.0.4-py3-none-any.whl

Tags

More Quality Assurance packages