apache-airflow-providers-elasticsearch

What it is and what it does

This is an Apache Airflow provider package that bridges Airflow workflows and Elasticsearch clusters. It supplies hooks, operators, and sensors that let you query Elasticsearch indices, index documents, and monitor cluster health as part of your Airflow DAGs. The package is built on top of the official elasticsearch client library and integrates with Airflow's connection and credential management system.

You install it alongside an existing Airflow deployment (>=2.11.0) to gain Elasticsearch-aware task types. It depends on apache-airflow, elasticsearch (8.10 or later, before 10.0), and two common Airflow provider packages for compatibility and SQL-like operations. The package supports Python 3.10 through 3.14 and includes an optional polars dependency for advanced data handling.

Use it for:

• Index log or event data into Elasticsearch from Airflow tasks as part of a data pipeline.
• Query Elasticsearch indices to fetch metrics or alerts and trigger downstream DAG logic based on results.
• Monitor Elasticsearch cluster health or index status and raise alerts via Airflow sensors.
• Build ETL workflows that read from Elasticsearch, transform data, and write back or to other systems.
• Orchestrate bulk indexing or reindexing operations across Elasticsearch clusters on a schedule.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Integrates Elasticsearch with Apache Airflow, providing operators, hooks, and sensors to query, index, and monitor Elasticsearch clusters within Airflow DAGs.

Yes, if you run Airflow and need to integrate Elasticsearch into your workflows. The package is actively maintained, carries no known vulnerabilities, has low install friction, and is part of the official Apache Airflow provider ecosystem. Install only if you have an Airflow instance >=2.11.0 and an Elasticsearch cluster >=8.10 to connect to.

Install

apache-airflow-providers-elasticsearch on PyPI

pip install apache-airflow-providers-elasticsearch

uv add apache-airflow-providers-elasticsearch

poetry add apache-airflow-providers-elasticsearch

Installing apache-airflow-providers-elasticsearch

Before you install

Low install friction with a pure-Python wheel. Actively maintained—released 6 days ago with a large upstream repository (46490 stars). Requires Airflow >=2.11.0 and elasticsearch >=8.10,<10.

License in practice

Apache-2.0 permissive license allows commercial and private use with minimal restrictions; typical for Apache Foundation projects.

Quickstart

pip install apache-airflow-providers-elasticsearch

from airflow.providers.elasticsearch.hooks.elasticsearch import ElasticsearchHook
hook = ElasticsearchHook(conn_id='elasticsearch_default')
results = hook.search(index='my_index', query={'match_all': {}})

Requires an existing Airflow installation (>=2.11.0) and an Elasticsearch cluster (>=8.10,<10) to connect to.

Verify before relying

• Specific operators and sensors available beyond hooks (e.g., ElasticsearchOperator, ElasticsearchSensor).
• Whether polars optional dependency adds data transformation capabilities or is only for export/import workflows.

Package facts

Evidence: apache_airflow_providers_elasticsearch-6.9.0-py3-none-any.whl

Keywords: airflow-provider, elasticsearch, airflow, integration

Tags

More Monitoring packages