collate-data-diff

What it is and what it does

Collate-data-diff is a command-line tool and Python library forked from the archived datafold/data-diff project for comparing rows across SQL databases. It lets you identify differences between two datasets, whether they're in the same database or across different systems, without loading all data into memory. The tool is designed for data validation, synchronization checks, and quality assurance workflows where you need to verify that data matches between sources.

The package depends on dbt-core, click, pydantic, and several other libraries for database connectivity, data parsing, and terminal output formatting. It supports Python 3.10 through 3.14 and is classified as pre-alpha. The maintenance status is marked abandoned, though releases continue; the original repository is archived, so this fork's long-term viability depends on the maintaining organization.

Use it for:

• Validate that data migrated between databases matches the source after ETL operations.
• Compare production and staging database tables to detect unexpected divergence.
• Audit data consistency across federated or replicated database instances.
• Verify test data fixtures match expected state during integration testing.
• Identify row-level differences for reconciliation workflows in financial or compliance systems.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Compares rows across two SQL databases or datasets to identify differences, available as both a command-line tool and Python library.

Yes, if you need row-level database comparison and can tolerate the abandoned maintenance status. The package has low install friction, permissive licensing, and no known vulnerabilities. However, the abandoned status and archived upstream repository mean you should verify that the fork is actively maintained by the current publisher and plan for potential lack of future support or bug fixes.

Install

collate-data-diff on PyPI

pip install collate-data-diff

uv add collate-data-diff

poetry add collate-data-diff

Installing collate-data-diff

Before you install

Low install friction with a pure-Python wheel and permissive dependencies. However, the package is marked abandoned with no active maintenance, despite a recent release date; the underlying repository is archived and the last commit was in 2024-05-17.

License in practice

MIT license is permissive and imposes no restrictions on commercial or private use, modification, or redistribution.

Quickstart

pip install collate-data-diff

from collate_data_diff import compare
# Compare two database tables
results = compare(database1_conn, database2_conn, table_name)

Requires Python 3.10 or later; dbt-core is a runtime dependency with its own substantial dependency tree.

Verify before relying

• Whether the package is actively maintained despite the abandoned status and archived repository.
• Performance characteristics and scalability limits for large datasets.
• Specific SQL dialects and database systems supported beyond the general 'SQL databases' claim.
• Whether the fork is intended as a long-term replacement or temporary maintenance of the original.

Package facts

Evidence: collate_data_diff-0.11.15-py3-none-any.whl

Tags

More Database Engines/Servers packages