openmetadata-ingestion

What it is and what it does

OpenMetadata Ingestion is a Python framework for extracting metadata from external systems and pushing it into OpenMetadata's central metadata repository. It provides a pluggable connector architecture and a TopologyRunner execution engine that orchestrates the ingestion workflow, supporting both single-threaded and multi-threaded execution modes. The framework is designed to integrate with orchestration platforms like Apache Airflow, allowing metadata extraction to be scheduled and monitored as part of a broader data pipeline.

The package bundles 43 runtime dependencies covering cloud credential management (Azure Key Vault, Google Cloud Secrets), database connectivity (PyMySQL, SQLAlchemy), data transformation (Jinja2, Mako), and distributed execution (Kubernetes). It requires Python >= 3.9 and installs as a pure-Python wheel with low friction. The framework is actively maintained, though its licensing under the Collate Community License restricts use for competing SaaS or managed services.

Use it for:

• Automatically extract metadata from databases, data warehouses, and data lakes and sync it to OpenMetadata on a schedule
• Build custom connectors for proprietary or internal systems to integrate their metadata into a centralized catalog
• Ingest data lineage and schema information from multiple sources in a single orchestrated workflow
• Populate OpenMetadata with metadata from cloud services (Azure, Google Cloud) using managed credential stores
• Monitor and profile metadata ingestion workflows using memory profiling and multi-threaded execution for large-scale operations

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

OpenMetadata Ingestion is a framework for building connectors and ingesting metadata from various systems into OpenMetadata via its APIs, typically integrated into orchestration tools like Apache Airflow.

Yes, if you are already using OpenMetadata and need to ingest metadata from external systems. The framework is actively maintained, has low install friction, and provides a structured approach to metadata extraction. However, be aware of the Collate Community License restrictions: you cannot use it to offer competing metadata management services. If you need to evaluate whether the supported connectors cover your systems, consult the documentation.

Install

openmetadata-ingestion on PyPI

pip install openmetadata-ingestion

uv add openmetadata-ingestion

poetry add openmetadata-ingestion

Installing openmetadata-ingestion

Before you install

Low install friction with a pure-Python wheel. Actively maintained with a recent release (2 days old) and strong repository signals (14879 stars, last commit 2026-08-14). Requires Python >= 3.9.

License in practice

Licensed under the Collate Community License Agreement Version 1.0, which permits use, modification, and distribution but prohibits using the software to offer competing SaaS, PaaS, or IaaS services. Redistribution requires prominent modification notices and license reproduction. This is a proprietary community license, not an open-source license.

Quickstart

pip install openmetadata-ingestion

from openmetadata_ingestion.workflows.workflow import Workflow

workflow = Workflow(config_path='workflow_config.yaml')
workflow.execute()

Requires Python >= 3.9. The TopologyRunner (the standard execution model) expects a workflow configuration file; exact setup depends on your target metadata system and orchestration context.

Verify before relying

• Which specific metadata systems and data sources are supported by the included connectors
• Whether the framework can be used standalone or requires Apache Airflow or another orchestrator
• Performance characteristics and scalability limits for large metadata ingestion workloads
• Compatibility with specific versions of OpenMetadata server

Package facts

Evidence: openmetadata_ingestion-1.13.3.2-py3-none-any.whl

Tags

More Database packages