chispa
Pyspark test helper library
What it is and what it does
Chispa is a testing library that simplifies writing unit tests by providing assertion methods like `assert_column_equality` and `assert_df_equality` that compare data structures while generating human-readable error messages that highlight mismatches. When a test fails, the output uses color-coding and formatted tables to show exactly which rows or values differ, making debugging much faster than reading raw error output.
The library supports flexible comparison modes: you can ignore row order, column order, specific columns, nullable schema properties, and NaN equality. It integrates with pytest and is designed as a development dependency. The package is actively maintained, supports Python 3.10, 3.11, and 3.12, and carries no known security vulnerabilities.
Use it for:
- Write unit tests with clear assertion failures and readable error output.
- Compare actual and expected data structures while ignoring row or column order.
- Debug data quality issues by viewing formatted side-by-side comparisons of mismatched rows.
- Test column-level transformations with descriptive failure messages.
- Validate schema changes while ignoring nullable property differences.
Worth the install?
AI-flagged interpretation of the facts on this page — verify before relying
Provides assertion methods for testing PySpark code with descriptive error messages formatted for easy debugging.
Yes. Chispa is a lightweight, actively maintained testing utility with no security issues and low install friction. It directly addresses the pain point of making test failures readable and actionable. Install it as a development dependency if you write tests.
Install
chispa on PyPI
pip
pip install chispauv
uv add chispapoetry
poetry add chispaInstalling chispa
Before you install
Low friction install with a single runtime dependency (prettytable). Actively maintained with recent releases and no known vulnerabilities.
License in practice
MIT license permits unrestricted use, modification, and distribution in both open-source and commercial projects.
Quickstart
pip install chispa
from chispa.column_comparer import assert_column_equality
assert_column_equality(df, "actual_column", "expected_column")
Requires a working Spark environment; not declared as a runtime dependency but is the package's primary use case.
Verify before relying
- Whether Spark is an implicit dependency or must be installed separately.
- Performance characteristics when comparing very large DataFrames.
- Full list of assertion methods beyond column and DataFrame equality.
Package facts
| License | MIT (permissive) |
| Python support | supports the current Python release (<4.0,>=3.10) |
| Install friction | low — pure-Python wheel |
| Runtime dependencies | 1 — prettytable |
| Maintenance | actively maintained — 143 days since the last release |
| Last repo commit | |
| First released | |
| Downloads | 3,041,738/month — #2,777 on PyPI (30-day window, as of 2026-08-14) |
| Known vulnerabilities | none known (OSV.dev, checked 2026-08-14) |
Evidence: chispa-0.12.0-py3-none-any.whl
Keywords: apachespark, spark, pyspark, pytest
Tags
More Python Modules packages
Converts domain names between Unicode and…
permissive · top 100 on PyPI
setuptoolsSetuptools is a Python build backend and…
permissive · top 100 on PyPI
PyYAMLPyYAML parses and emits YAML 1.1 data format,…
permissive · top 100 on PyPI
pydanticPydantic validates Python data structures…
permissive · top 100 on PyPI
annotated-typesProvides reusable metadata objects for use with…
permissive · top 100 on PyPI
typing-inspectionProvides runtime tools to inspect and…
permissive · top 100 on PyPI
pyspark-testProvides a testing utility to assert equality…
permissive · top 15,000 on PyPI
quinnQuinn provides helper methods for PySpark…
permissive · top 15,000 on PyPI
databricks-testProvides a unit testing framework for…
permissive · top 15,000 on PyPI
cualleeCuallee provides a dataframe-agnostic API to…
permissive · top 15,000 on PyPI
sparkaidProvides utilities for working with Spark…
copyleft · top 15,000 on PyPI
datacompyDataComPy compares two DataFrames across…
permissive · top 5,000 on PyPI
dbldatagenGenerates synthetic data at scale within…
unclear · top 15,000 on PyPI
sparkdanticConverts Pydantic models to PySpark schemas…
unclear · top 5,000 on PyPI