acryl-datahub-airflow-plugin
DataHub Airflow plugin — automatically capture pipeline lineage, run history, and task metadata from Apache Airflow
Decision gist · record as of 2026-08-14
Yes, if you run Apache Airflow 3.0+ and use DataHub for metadata management. The plugin is actively maintained, has no known vulnerabilities, low install friction, and solves the common problem of capturing lineage from Airflow without manual instrumentation. The permissive Apache-2.0 license poses no restrictions. Install only if you have a DataHub instance to send metadata to; it is not useful standalone.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires Apache Airflow 3.0+ (version 1.6.0 or earlier for Airflow 2.x).
- Requires Python >= 3.10.
- Low install friction with a wheel distribution.
License · maintenance · safety
Apache-2.0 (permissive) — Licensed under Apache-2.0 (permissive), allowing commercial use, modification, and distribution with minimal restrictions.
last release 2026-08-11 (3 days) · last repo commit 2026-08-13 · 34 stars
0 known vulnerabilities (OSV.dev, 2026-08-14) · 771,401 downloads/mo, #5,101 on PyPI
Alternatives
Verify before relying
pip install acryl-datahub-airflow-plugin
# Add to airflow.cfg:
# [datahub]
# enabled = True
# conn_id = datahub_rest_default
# Set up connection:
# airflow connections add datahub_rest_default --conn-type HTTP --conn-host http://localhost:8080
# Plugin activates automatically on Airflow startup- Whether column-level lineage extraction works with all SQL dialects or only specific ones
- Performance impact when enable_multi_statement_sql_parsing is enabled on large DAGs
- Whether the plugin works with all Airflow executor types or has known limitations
What it is and what it does
This package is a plugin for Apache Airflow that automatically extracts and forwards pipeline metadata to DataHub. It captures DAG structure, task inputs and outputs, and execution history without requiring code changes to your existing DAGs. The plugin integrates with Airflow's OpenLineage framework and enhances it with DataHub's SQL parser to extract column-level lineage from SQL operators, providing richer metadata than standard OpenLineage extractors alone.
The plugin supports multiple emission paths (REST, Kafka, or file) and activates automatically once configured in Airflow's configuration file with a DataHub connection. It's designed for teams running Airflow who want centralized visibility into data pipelines and lineage without manual instrumentation, making it useful for data governance, impact analysis, and compliance tracking.
Use it for
- Automatically track dataset and column-level lineage from SQL-based Airflow tasks into a centralized DataHub catalog
- Monitor Airflow task execution status and duration to correlate failures with downstream data quality issues
- Enhance OpenLineage metadata with DataHub's advanced SQL parsing for more detailed lineage than standard extractors provide
- Stream Airflow pipeline metadata to Kafka for real-time consumption by other data governance tools
- Resolve temporary tables across multi-statement SQL tasks to capture complete lineage in complex ETL workflows
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes, if you run Apache Airflow 3.0+ and use DataHub for metadata management.
The plugin is actively maintained, has no known vulnerabilities, low install friction, and solves the common problem of capturing lineage from Airflow without manual instrumentation. The permissive Apache-2.0 license poses no restrictions. Install only if you have a DataHub instance to send metadata to; it is not useful standalone.
Install
acryl-datahub-airflow-plugin on PyPI
Before you install
Low install friction with a wheel distribution. Actively maintained—released 3 days ago with recent commits. Requires Apache Airflow and acryl-datahub as core dependencies, plus pydantic and apache-airflow-providers-openlineage.
Requires Apache Airflow 3.0+ (version 1.6.0 or earlier for Airflow 2.x). Requires Python >= 3.10.
License in practice
Licensed under Apache-2.0 (permissive), allowing commercial use, modification, and distribution with minimal restrictions.
Quickstart
pip install acryl-datahub-airflow-plugin
# Add to airflow.cfg:
# [datahub]
# enabled = True
# conn_id = datahub_rest_default
# Set up connection:
# airflow connections add datahub_rest_default --conn-type HTTP --conn-host http://localhost:8080
# Plugin activates automatically on Airflow startupVerify before relying
- Whether column-level lineage extraction works with all SQL dialects or only specific ones
- Performance impact when enable_multi_statement_sql_parsing is enabled on large DAGs
- Whether the plugin works with all Airflow executor types or has known limitations
Package facts
| License | Apache-2.0 permissive |
| Python support | Supports the current Python release >=3.10 |
| Install friction | Low. Pure-Python wheel |
| Runtime dependencies | 4 packagesacryl-datahubapache-airflow-providers-openlineageapache-airflowpydantic |
| Maintenance | Actively maintained 3 days since the last release |
| Last repo commit | |
| First released | |
| Downloads | 771,401 / month, #5,101 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
| Classifiers | Development Status :: 5 - Production/StableEnvironment :: ConsoleEnvironment :: MacOS XIntended Audience :: DevelopersIntended Audience :: Information TechnologyIntended Audience :: System AdministratorsOperating System :: POSIX :: LinuxOperating System :: UnixProgramming Language :: PythonProgramming Language :: Python :: 3Programming Language :: Python :: 3 :: OnlyProgramming Language :: Python :: 3.11Programming Language :: Python :: 3.12Topic :: Software Development |
Evidence: acryl_datahub_airflow_plugin-1.7.0.3-py3-none-any.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “datahub airflow integration”
- acryl-datahub-airflow-pluginAutomatically captures pipeline lineage, task metadata, and run…
- acryl-datahubDataHub's ingestion framework and CLI for pulling metadata from 50+…
- datahubThis is a redirect package that installs acryl-datahub when you run…
Give your agent the search over MCP, or paste the wish link into any chat.
More Software Development packages
Provides backported and experimental type hints for Python 3.9+, allowing use of newer typing features on older Python versions and enabling early experimentation with type system PEPs before they enter the standard library.
NumPy provides an N-dimensional array object and a comprehensive suite of mathematical, linear algebra, Fourier transform, and random number functions for scientific computing in Python.
FastAPI is a Python web framework for building REST APIs using type hints, with automatic request validation, serialization, and interactive API documentation.
Provides a way to document function parameters, class attributes, return types, and variables inline using Python's `Annotated` type hint syntax instead of traditional docstrings.
Typer builds command-line applications from Python functions using type hints, automatically generating help text, argument parsing, and shell completion.
Install it if you are building CLIs in Python.
Distlib provides low-level packaging utilities for building, distributing, and managing Python software—including metadata handling, version specifiers, wheel support, script installation, and dependency resolution.
See also acryl-datahub-dagster-plugin · openlineage-airflow · acryl-datahub · acryl-datahub-actions · acryl-executor · acryl-datahub-classify · astronomer-starship · mwaa-dr · apache-airflow-providers-openlineage · singer-sdk