acryl-datahub-dagster-plugin

What it is and what it does

This package bridges Dagster and acryl-datahub by automatically extracting and forwarding pipeline metadata without manual instrumentation. It runs as a sensor that listens to job executions and captures asset definitions, upstream/downstream relationships, and run status, then pushes that lineage and metadata into acryl-datahub so your assets appear in the catalog alongside their execution history.

The plugin works with any acryl-datahub deployment—self-hosted or Cloud—and requires only configuration of the server endpoint and Dagster URL. Once registered in your Definitions, it operates transparently: every job run triggers metadata emission to acryl-datahub, building a continuous record of lineage and run outcomes without code changes to your existing Dagster jobs or assets.

Use it for:

• Track data lineage across Dagster pipelines in a centralized catalog for compliance and impact analysis.
• Monitor job execution history and task outcomes without adding instrumentation to each Dagster job.
• Map Dagster assets to external datasets to understand cross-system data dependencies.
• Integrate Dagster metadata into an existing acryl-datahub deployment for unified data governance.
• Automatically capture upstream/downstream relationships between Dagster assets for data discovery.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Automatically captures asset lineage, run history, and job metadata from Dagster pipelines and sends it to DataHub for catalog and lineage tracking.

Yes—if you run Dagster and use acryl-datahub for data catalog and lineage, this plugin eliminates manual metadata instrumentation and keeps lineage synchronized automatically. Low install friction, active maintenance, no known vulnerabilities, and permissive licensing make it a straightforward addition. Install only if you have both Dagster and acryl-datahub in your stack.

Install

acryl-datahub-dagster-plugin on PyPI

pip install acryl-datahub-dagster-plugin

uv add acryl-datahub-dagster-plugin

poetry add acryl-datahub-dagster-plugin

Installing acryl-datahub-dagster-plugin

Before you install

Low install friction with a pure-Python wheel. Active maintenance as of 2026-08-14 with no known vulnerabilities. Requires three runtime dependencies: dagster, dagit, and acryl-datahub.

License in practice

Licensed under Apache-2.0 (permissive), allowing use in commercial and private projects with minimal restrictions.

Quickstart

pip install acryl-datahub-dagster-plugin

from datahub.ingestion.graph.config import DatahubClientConfig
from datahub_dagster_plugin.sensors.datahub_sensors import DatahubDagsterSourceConfig, make_datahub_sensor

config = DatahubDagsterSourceConfig(
    datahub_client_config=DatahubClientConfig(server="http://localhost:8080"),
    dagster_url="http://localhost:3000",
)
datahub_sensor = make_datahub_sensor(config=config)

Requires Python 3.10 or later; requires a running Dagster instance and a DataHub deployment (self-hosted or Cloud).

Verify before relying

• Whether the sensor handles all Dagster asset types and external dataset mappings without gaps.
• Performance impact on Dagster job execution when the sensor is active.
• Compatibility with specific DataHub versions or deployment configurations.

Package facts

Evidence: acryl_datahub_dagster_plugin-1.7.0.4-py3-none-any.whl

Tags

More Software Development packages