dbl-discoverx

What it is and what it does

DiscoverX is a Databricks Labs utility for automating administration across large numbers of Lakehouse assets. It lets you apply SQL templates or Python functions concurrently to multiple Delta tables selected by pattern matching, returning results as a unioned Spark DataFrame. Common use cases include bulk OPTIMIZE and VACUUM operations, deep cloning, tag management, owner changes, and semantic classification of columns for governance tasks like PII detection and GDPR compliance.

The package depends only on pyyaml and is designed to run in Databricks notebooks. It provides a fluent API (the `DX` class) for selecting tables, filtering by column presence, and executing templated operations with configurable concurrency. The fact sheet indicates the project is aging (469 days since last release) and carries no formal support SLA from Databricks—it is provided as-is for exploration.

Use it for:

• Run OPTIMIZE and VACUUM across all tables in a schema or catalog to reclaim storage and improve query performance.
• Deep clone entire schemas or catalogs for testing or disaster recovery without manual table-by-table operations.
• Detect and report on tables with too many small files or ineffective ZORDER clustering across the entire lakehouse.
• Scan all tables for PII (email, phone, IP) using semantic classification rules and extract or delete sensitive data for GDPR compliance.
• Bulk update table ownership or apply tags across hundreds of assets matching a selection pattern.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

DiscoverX automates bulk administration tasks across Lakehouse assets by executing SQL templates or Python functions concurrently against multiple Delta tables matching selection patterns.

Yes, if you are a Databricks user managing large Lakehouses and can tolerate the aging maintenance status and unclear license. The low install friction and no known vulnerabilities are positive signals. However, verify the license terms and test compatibility with your current Databricks and Python versions before production use, and understand that support is community-driven, not SLA-backed.

Install

dbl-discoverx on PyPI

pip install dbl-discoverx

uv add dbl-discoverx

poetry add dbl-discoverx

Installing dbl-discoverx

Before you install

Low install friction with a single runtime dependency (pyyaml). Package status is aging—last release was 469 days ago—and the project is provided without formal SLAs by Databricks Labs, meaning maintenance and support are exploratory only.

License in practice

License treatment is unclear; the classifier indicates 'Other/Proprietary License' but no SPDX identifier or raw license text is available. Verify the actual license terms before adopting in production or proprietary work.

Quickstart

pip install dbl-discoverx

from discoverx import DX
dx = DX(locale="US")
result = dx.from_tables("catalog.schema.*").with_sql("OPTIMIZE {full_table_name}").apply()

Requires a Databricks workspace with Unity Catalog enabled; designed to run in Databricks notebooks, not standalone Python environments.

Verify before relying

• Whether the package works with current Databricks SDK versions and recent Python releases
• Specific Python version requirements (requires_python is unspecified)
• Whether aging status reflects active maintenance or dormancy

Package facts

Evidence: dbl_discoverx-0.0.9-py3-none-any.whl

Tags

More Database packages