apache-airflow-providers-apache-kafka
Provider package apache-airflow-providers-apache-kafka for Apache Airflow
Decision gist · record as of 2026-08-14
Yes. This is a stable, actively maintained provider package from the Apache Airflow project with no known vulnerabilities, low install friction, and a permissive Apache-2.0 license. Install it if you need to integrate Kafka into Airflow DAGs. The only prerequisite is an existing Airflow >=2.11.0 installation and a Kafka broker to connect to.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires an existing Apache Airflow installation (>=2.11.0) and a running Kafka broker accessible at the configured bootstrap.servers address.
- Low friction installation.
- Active maintenance with a release 6 days ago.
License · maintenance · safety
Apache-2.0 (permissive) — Apache-2.0 permissive license. You can use, modify, and distribute this package freely in commercial and private projects, provided you include a copy of the license and any NOTICE files.
last release 2026-08-08 (6 days) · last repo commit 2026-08-14 · 46,490 stars
0 known vulnerabilities (OSV.dev, 2026-08-14) · 1,928,602 downloads/mo, #3,424 on PyPI
Alternatives
Verify before relying
pip install apache-airflow-providers-apache-kafka
from airflow.providers.apache.kafka.operators.produce_to_topic import ProduceToTopicOperator
task = ProduceToTopicOperator(
task_id='produce_to_kafka',
topic='my_topic',
producer_config={'bootstrap.servers': 'kafka_broker'}
)- Specific operators and hooks available beyond produce/consume (e.g., consumer groups, offset management, schema registry integration).
- Whether optional extras (google, msk, common.messaging) are commonly needed or niche use cases.
- Example producer_config parameters and typical Kafka broker connection patterns.
What it is and what it does
This is an Apache Airflow provider package that extends Airflow with Kafka integration capabilities. It wraps the confluent-kafka library and exposes Kafka operators and hooks within Airflow's DAG framework, allowing you to define tasks that produce messages to or consume messages from Kafka topics as part of orchestrated workflows.
The package is maintained as part of the Apache Airflow project and is actively developed. It supports Python 3.10, 3.11, 3.12, 3.13, and 3.14, and requires Airflow >=2.11.0. Installation is straightforward via pip, and it has low dependency friction—the main runtime dependencies are apache-airflow, confluent-kafka, asgiref, and apache-airflow-providers-common-compat. No known security vulnerabilities are recorded.
Use it for
- Stream event data into Kafka topics as part of a scheduled Airflow pipeline.
- Consume Kafka messages in an Airflow task and process them (e.g., transform, validate, load into a data warehouse).
- Orchestrate multi-stage data pipelines where Kafka serves as the message broker between Airflow tasks.
- Integrate Kafka-based microservices with Airflow workflows for event-driven data processing.
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes.
This is a stable, actively maintained provider package from the Apache Airflow project with no known vulnerabilities, low install friction, and a permissive Apache-2.0 license. Install it if you need to integrate Kafka into Airflow DAGs. The only prerequisite is an existing Airflow >=2.11.0 installation and a Kafka broker to connect to.
Install
apache-airflow-providers-apache-kafka on PyPI
Before you install
Low friction installation. Active maintenance with a release 6 days ago. Requires Apache Airflow >=2.11.0 and confluent-kafka >=2.6.0 (or >=2.13.2 for Python 3.14+), both straightforward dependencies.
Requires an existing Apache Airflow installation (>=2.11.0) and a running Kafka broker accessible at the configured bootstrap.servers address.
License in practice
Apache-2.0 permissive license. You can use, modify, and distribute this package freely in commercial and private projects, provided you include a copy of the license and any NOTICE files.
Quickstart
pip install apache-airflow-providers-apache-kafka
from airflow.providers.apache.kafka.operators.produce_to_topic import ProduceToTopicOperator
task = ProduceToTopicOperator(
task_id='produce_to_kafka',
topic='my_topic',
producer_config={'bootstrap.servers': 'kafka_broker'}
)Verify before relying
- Specific operators and hooks available beyond produce/consume (e.g., consumer groups, offset management, schema registry integration).
- Whether optional extras (google, msk, common.messaging) are commonly needed or niche use cases.
- Example producer_config parameters and typical Kafka broker connection patterns.
Package facts
| License | Apache-2.0 permissive |
| Python support | Supports the current Python release >=3.10 |
| Install friction | Low. Pure-Python wheel |
| Runtime dependencies | 4 packagesapache-airflowapache-airflow-providers-common-compatasgirefconfluent-kafka |
| Maintenance | Actively maintained 6 days since the last release |
| Last repo commit | |
| First released | |
| Downloads | 1,928,602 / month, #3,424 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
| Classifiers | Development Status :: 5 - Production/StableEnvironment :: ConsoleEnvironment :: Web EnvironmentFramework :: Apache AirflowFramework :: Apache Airflow :: ProviderIntended Audience :: DevelopersIntended Audience :: System AdministratorsProgramming Language :: Python :: 3.10Programming Language :: Python :: 3.11Programming Language :: Python :: 3.12Programming Language :: Python :: 3.13Programming Language :: Python :: 3.14Topic :: System :: Monitoring |
Evidence: apache_airflow_providers_apache_kafka-1.15.1-py3-none-any.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “airflow kafka integration”
- apache-airflow-providers-apache-kafkaIntegrates Apache Kafka with Apache Airflow, providing operators and…
- apache-airflow-providers-common-messagingProvides common messaging abstractions and operators for Apache…
- acryl-datahub-airflow-pluginAutomatically captures pipeline lineage, task metadata, and run…
Give your agent the search over MCP, or paste the wish link into any chat.
More Monitoring packages
Wraps any iterable to display a real-time progress bar in the terminal or Jupyter notebook, showing iteration count, elapsed time, and estimated time remaining.
Provides generated Python code for OpenTelemetry semantic conventions, enabling standardized attribute naming and constant definitions for instrumentation and telemetry collection.
Install it if you are using OpenTelemetry and want to follow semantic conventions correctly.
Provides the reference implementation of the OpenTelemetry API for collecting and exporting traces, metrics, and logs from Python applications.
Provides the abstract API and interfaces for OpenTelemetry instrumentation in Python, defining how to emit traces, metrics, and logs without tying code to a specific SDK implementation.
Exports OpenTelemetry observability data to an OpenTelemetry Collector using Protobuf-encoded messages over HTTP.
Install it if you are using OpenTelemetry in Python and need to send data to a Collector over HTTP.
Provides automatic instrumentation commands and programmatic APIs to inject distributed tracing into Python applications without code changes, detecting and instrumenting packages used by your program.
Install it if you need distributed tracing without code changes and have compatible instrumented packages in your environment.
See also apache-airflow-providers-common-messaging · apache-airflow-providers-redis · apache-airflow-providers-microsoft-azure · apache-airflow-providers-cncf-kubernetes · apache-airflow-providers-http · apache-airflow-providers-standard · apache-airflow-providers-google · apache-airflow-providers-databricks · apache-airflow-providers-apache-cassandra · apache-airflow-providers-opensearch