openmetadata-ingestion
Ingestion Framework for OpenMetadata
Decision gist · record as of 2026-08-14
Yes, if you are already using OpenMetadata and need to ingest metadata from external systems. The framework is actively maintained, has low install friction, and provides a structured approach to metadata extraction. However, be aware of the Collate Community License restrictions: you cannot use it to offer competing metadata management services. If you need to evaluate whether the supported connectors cover your systems, consult the documentation.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires Python >= 3.9.
- The TopologyRunner (the standard execution model) expects a workflow configuration file; exact setup depends on your target metadata system and orchestration context.
- Low install friction with a pure-Python wheel.
License · maintenance · safety
(unclear) — Licensed under the Collate Community License Agreement Version 1.0, which permits use, modification, and distribution but prohibits using the software to offer competing SaaS, PaaS, or IaaS services. Redistribution requires prominent modification notices and license reproduction. This is a proprietary community license, not an open-source license.
last release 2026-08-12 (2 days) · last repo commit 2026-08-14 · 14,879 stars
0 known vulnerabilities (OSV.dev, 2026-08-14) · 519,315 downloads/mo, #6,218 on PyPI
Alternatives
Verify before relying
pip install openmetadata-ingestion
from openmetadata_ingestion.workflows.workflow import Workflow
workflow = Workflow(config_path='workflow_config.yaml')
workflow.execute()- Which specific metadata systems and data sources are supported by the included connectors
- Whether the framework can be used standalone or requires Apache Airflow or another orchestrator
- Performance characteristics and scalability limits for large metadata ingestion workloads
- Compatibility with specific versions of OpenMetadata server
What it is and what it does
OpenMetadata Ingestion is a Python framework for extracting metadata from external systems and pushing it into OpenMetadata's central metadata repository. It provides a pluggable connector architecture and a TopologyRunner execution engine that orchestrates the ingestion workflow, supporting both single-threaded and multi-threaded execution modes. The framework is designed to integrate with orchestration platforms like Apache Airflow, allowing metadata extraction to be scheduled and monitored as part of a broader data pipeline.
The package bundles 43 runtime dependencies covering cloud credential management (Azure Key Vault, Google Cloud Secrets), database connectivity (PyMySQL, SQLAlchemy), data transformation (Jinja2, Mako), and distributed execution (Kubernetes). It requires Python >= 3.9 and installs as a pure-Python wheel with low friction. The framework is actively maintained, though its licensing under the Collate Community License restricts use for competing SaaS or managed services.
Use it for
- Automatically extract metadata from databases, data warehouses, and data lakes and sync it to OpenMetadata on a schedule
- Build custom connectors for proprietary or internal systems to integrate their metadata into a centralized catalog
- Ingest data lineage and schema information from multiple sources in a single orchestrated workflow
- Populate OpenMetadata with metadata from cloud services (Azure, Google Cloud) using managed credential stores
- Monitor and profile metadata ingestion workflows using memory profiling and multi-threaded execution for large-scale operations
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes, if you are already using OpenMetadata and need to ingest metadata from external systems.
The framework is actively maintained, has low install friction, and provides a structured approach to metadata extraction. However, be aware of the Collate Community License restrictions: you cannot use it to offer competing metadata management services. If you need to evaluate whether the supported connectors cover your systems, consult the documentation.
Install
openmetadata-ingestion on PyPI
Before you install
Low install friction with a pure-Python wheel. Actively maintained with a recent release (2 days old) and strong repository signals (14879 stars, last commit 2026-08-14). Requires Python >= 3.9.
Requires Python >= 3.9. The TopologyRunner (the standard execution model) expects a workflow configuration file; exact setup depends on your target metadata system and orchestration context.
License in practice
Licensed under the Collate Community License Agreement Version 1.0, which permits use, modification, and distribution but prohibits using the software to offer competing SaaS, PaaS, or IaaS services. Redistribution requires prominent modification notices and license reproduction. This is a proprietary community license, not an open-source license.
Quickstart
pip install openmetadata-ingestion
from openmetadata_ingestion.workflows.workflow import Workflow
workflow = Workflow(config_path='workflow_config.yaml')
workflow.execute()Verify before relying
- Which specific metadata systems and data sources are supported by the included connectors
- Whether the framework can be used standalone or requires Apache Airflow or another orchestrator
- Performance characteristics and scalability limits for large metadata ingestion workloads
- Compatibility with specific versions of OpenMetadata server
Package facts
| License | Not declared unclear |
| Python support | Supports the current Python release >=3.9 |
| Install friction | Low. Pure-Python wheel |
| Runtime dependencies | 43 packagesazure-keyvault-secretscachetoolsrequestspydantic-settingsPyJWTgoogle-cloud-secret-managerlxmlpymysqlpython-dateutilsqlalchemytenacitysetuptoolstabulatepython-dotenvtyping-inspectshapelyPyYAMLcryptographyMakoJinja2email-validatorjsonpatchazure-identityantlr4-python3-runtimepackaginghttpxkubernetesmemory-profilerrequests-aws4authcollate-data-diff |
| Maintenance | Actively maintained 2 days since the last release |
| Last repo commit | |
| First released | |
| Downloads | 519,315 / month, #6,218 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
Evidence: openmetadata_ingestion-1.13.3.2-py3-none-any.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “metadata ingestion framework”
- openmetadata-ingestionOpenMetadata Ingestion is a framework for building connectors and…
- acryl-datahubDataHub's ingestion framework and CLI for pulling metadata from 50+…
- logdnaIntegrates Python's standard logging framework with LogDNA's cloud…
Give your agent the search over MCP, or paste the wish link into any chat.
More Database packages
psycopg2-binary is a PostgreSQL database adapter for Python that implements the DB API 2.0 specification, enabling Python applications to connect to and query PostgreSQL databases with thread-safe concurrent operations.
Python client library for connecting to and executing commands against Redis key-value stores, supporting both synchronous and asynchronous operations.
Install it if your application needs to interact with Redis; the only prerequisite is a running Redis server instance.
YDB Python SDK is the official client library for connecting to and querying YDB databases from Python applications.
Install it if you need to connect Python applications to YDB databases.
Connects Python applications to Snowflake data warehouses using the DB API 2.0 specification, enabling SQL queries, data transfers, and warehouse operations.
sqlparse tokenizes SQL text into a tree of statements, clauses, and expressions, and provides functions to split scripts, format queries, and inspect parsed tokens without validating dialect or syntax.
Install it if you need to manipulate, format, or analyze SQL text programmatically.
Provides base adapter protocols and shared functionality that database adapters use to integrate with dbt-core, handling connections, dialect translation, relation caching, and core interface management.
See also acryl-datahub · unstructured-ingest · azure-kusto-ingest · ingestr · apache-airflow-providers-openlineage · dagster-airbyte · openlineage-airflow · databricks-zerobus-ingest-sdk · axiom-py · humiolib