{"categories":[{"label":"Quality Assurance","url":"https://skillfed.io/packages/category/software-development-quality-assurance/4"}],"enrichment":{"capability":"PyDDQ is a Python API for running data quality constraint checks on Spark DataFrames, validating row counts, uniqueness, nullability, foreign keys, and custom SQL expressions.","skillfed_tags":["spark-integration","data-validation","abandoned"],"use_cases":["Validate imported data meets row count and uniqueness constraints before ETL processing begins.","Write automated quality tests that inspect constraint results programmatically and fail data loads on violations.","Generate Markdown or console reports on data quality checks across multiple Spark DataFrames in a single run.","Enforce referential integrity by checking foreign key relationships between Spark tables.","Test that required columns are never null and custom SQL expressions hold true across the dataset."],"what_it_does":"PyDDQ is a Python wrapper around the Drunken Data Quality (DDQ) Scala library, enabling data quality validation on Spark DataFrames through a fluent constraint-checking API. It lets you define and run checks for row counts, unique keys, foreign keys, nullability, and custom SQL expressions, then report results to stdout or custom reporters in formats like Markdown or console output.\n\nThe package is designed for continuous data import pipelines where you need to assert data quality before processing. However, it requires a pre-configured PySpark environment with the DDQ jar file loaded via the driver class path\u2014it cannot be installed as a standalone Python package. The project has been abandoned since its last release on 2020-02-28, making it unsuitable for modern production environments without significant verification and potential maintenance work.","worth_installing":"No. The project is abandoned (last update 2020-02-28), requires complex pre-configuration of a PySpark environment with external jar files, and has received no maintenance. Modern alternatives and actively maintained data validation libraries are strongly preferred for any production or new development work."},"id":"pyddq","links":{"html":"https://skillfed.io/packages/pyddq","md":"https://skillfed.io/packages/pyddq.md","pypi":"https://pypi.org/project/pyddq/"},"maintenance":{"status":"abandoned"},"meta":{"latest_release":"2020-02-28","license_spdx":null,"license_treatment":"permissive","name":"pyddq","python_support":"unspecified","summary":"Python API for Drunken Data Quality"},"popularity":{"monthly_downloads":106337,"position":12656,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"5.0.0"}