airflow-clickhouse-plugin

What it is and what it does

This is an Apache Airflow plugin that wraps clickhouse-driver to let you define ClickHouse queries as Airflow tasks. It offers two operator families: one based directly on clickhouse-driver's Client.execute (ClickHouseOperator, ClickHouseHook, ClickHouseSensor) for full native protocol support, and a second family (ClickHouseSQLExecuteQueryOperator and related operators) that implements the Python DB API 2.0 standard for compatibility with other SQL providers in Airflow.

The native-protocol operators support SQL templating, multiple queries per task, query logging, and custom connection parameters like compression and timeouts. The DB API 2.0 operators provide a standardized interface useful when migrating Airflow pipelines from other databases to ClickHouse. Both families are tested across multiple Python and Airflow versions. The plugin requires Airflow >=2.3.0 and Python >=3.10 for version 1.8.0.

Use it for:

• Define ClickHouse data transformations and loads as Airflow DAG tasks with SQL templating and XCom result passing.
• Migrate existing Airflow pipelines from MySQL, Postgres, or BigQuery to ClickHouse using DB API 2.0 operators.
• Monitor ClickHouse query results or table state changes using ClickHouseSensor or ClickHouseSqlSensor in task dependencies.
• Execute multi-statement ClickHouse scripts within a single task, with the final query result automatically pushed to XCom.
• Integrate ClickHouse operations with OpenLineage for data lineage tracking in Airflow 2.7.0+.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Provides Apache Airflow operators and hooks to execute SQL queries and commands against ClickHouse using the native ClickHouse TCP protocol via clickhouse-driver.

Yes. The plugin is actively maintained, has no known vulnerabilities, low install friction, and permissive licensing. It fills a clear need for ClickHouse integration in Airflow with two well-documented operator families. Install it if you run Airflow >=2.3.0 with Python >=3.10 and need to query or load ClickHouse from DAGs.

Install

airflow-clickhouse-plugin on PyPI

pip install airflow-clickhouse-plugin

uv add airflow-clickhouse-plugin

poetry add airflow-clickhouse-plugin

Installing airflow-clickhouse-plugin

Before you install

Low install friction; pure Python wheel with only two runtime dependencies (apache-airflow and clickhouse-driver). Active maintenance with a release 32 days ago and 184 repository stars.

License in practice

MIT License permits commercial and private use with minimal restrictions; no copyleft obligations or patent clauses.

Quickstart

pip install airflow-clickhouse-plugin

from airflow_clickhouse_plugin.operators.clickhouse import ClickHouseOperator

task = ClickHouseOperator(
    task_id='query_clickhouse',
    sql='SELECT count() FROM my_table',
    clickhouse_conn_id='clickhouse_default'
)

Requires an Airflow connection named 'clickhouse_default' (or custom conn_id) configured with ClickHouse host, port, user, and password; Airflow >=2.3.0 and Python >=3.10 for version 1.8.0.

Verify before relying

• Whether the plugin's OpenLineage integration (via optional openlineage extra) is production-ready and actively maintained.
• Performance characteristics when executing very large result sets or high-frequency query patterns.
• Compatibility with Airflow 3.4.0+ given the version table shows support only up to <3.4.0.

Package facts

Evidence: airflow_clickhouse_plugin-1.8.0-py3-none-any.whl

Keywords: clickhouse, airflow

Tags

More Database packages