{"categories":[{"label":"Quality Assurance","url":"https://skillfed.io/packages/category/software-development-quality-assurance"}],"enrichment":{"capability":"PyDeequ is a Python API for Apache Spark-based data quality measurement, providing metrics computation, constraint suggestion, constraint verification, and metrics repository persistence for large datasets.","skillfed_tags":["data-quality","spark","data-validation"],"use_cases":["Run data profiling on large datasets to compute completeness, uniqueness, and distribution metrics before loading into a data warehouse.","Define and verify data quality constraints on incoming data in a data pipeline to catch schema violations or unexpected nulls early.","Suggest validation rules automatically based on sample data, then persist and track constraint results over time to monitor data quality trends.","Identify and quarantine rows that fail specific quality checks (e.g., invalid email format, out-of-range values) for manual review or remediation.","Build data quality dashboards by storing verification results in a metrics repository and querying historical runs by tag and timestamp."],"what_it_does":"PyDeequ wraps Deequ, an AWS-built library on Apache Spark, to define and run data quality checks at scale. It lets you compute metrics on dataset columns (via Analyzers and Profiles), suggest validation constraints based on data patterns, verify datasets against those constraints, and persist quality metrics over time in a repository. The package is designed for teams building data pipelines who need to catch data quality issues before they propagate downstream\u2014treating data validation like unit tests for code.\n\nYou define checks declaratively (e.g., \"column X must be complete and unique\"), run them across large DataFrames, and get back pass/fail results at both aggregate and row levels. The row-level output lets you quarantine problematic records. It integrates with numpy and pandas as runtime dependencies.","worth_installing":"Yes. PyDeequ is actively maintained, has no known vulnerabilities, installs with low friction, and solves a concrete problem for data engineers. It is well-suited if you need declarative, scalable data quality checks. Caveat: you must have a Spark environment set up separately; PyDeequ is a wrapper, not a standalone tool."},"id":"pydeequ","links":{"html":"https://skillfed.io/packages/pydeequ","md":"https://skillfed.io/packages/pydeequ.md","pypi":"https://pypi.org/project/pydeequ/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2026-07-08","license_spdx":null,"license_treatment":"permissive","name":"pydeequ","python_support":"supports_current","summary":"PyDeequ - Unit Tests for Data"},"popularity":{"monthly_downloads":16519526,"position":1148,"tier":"top_5000"},"security":{"n_vulnerabilities":0},"version":"1.6.0"}