apache-airflow-providers-apache-beam
Provider package apache-airflow-providers-apache-beam for Apache Airflow
Decision gist · record as of 2026-08-14
Yes, if you are already running Apache Airflow and need to orchestrate Apache Beam pipelines. The package is actively maintained, carries no known vulnerabilities, has low install friction, and integrates cleanly with Airflow's task model. Install only if you have both Airflow and Beam in your stack; it adds no value as a standalone library.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires an existing Apache Airflow installation (>=2.11.0) and Apache Beam (>=2.69.0) to be available in the environment.
- Low friction install as a pure Python wheel.
- Actively maintained with recent releases; requires Apache Airflow >=2.11.0 and Apache Beam >=2.69.0, plus pyarrow and numpy.
License · maintenance · safety
Apache-2.0 (permissive) — Apache-2.0 permissive license allows commercial and private use without restriction, typical for Apache Foundation projects.
last release 2026-03-02 (165 days) · last repo commit 2026-08-14 · 46,490 stars
0 known vulnerabilities (OSV.dev, 2026-08-14) · 287,421 downloads/mo, #8,032 on PyPI
Alternatives
Verify before relying
pip install apache-airflow-providers-apache-beam
from airflow.providers.apache.beam.operators.beam import BeamRunPythonPipelineOperator
task = BeamRunPythonPipelineOperator(
task_id='beam_job',
py_file='pipeline.py'
)- Whether the provider supports all Apache Beam runners (Direct, Dataflow, Spark, Flink) or a subset.
- Whether optional GCP integration (apache-beam[gcp]) is required for Dataflow or only for specific features.
- Performance characteristics when orchestrating large-scale or long-running Beam pipelines.
What it is and what it does
This is an Apache Airflow provider package that bridges Apache Beam into Airflow's workflow orchestration model. It supplies operators and hooks that allow you to define and schedule Beam pipelines as tasks within Airflow DAGs, treating data processing jobs as first-class workflow components. The package depends on apache-airflow, apache-beam, pyarrow, and numpy, and is designed for teams already running Airflow who want to orchestrate Beam workloads without leaving the Airflow ecosystem.
The package is actively maintained, supports Python 3.10–3.13, and requires Apache Airflow >=2.11.0 and Apache Beam >=2.69.0. It carries an Apache-2.0 license and integrates with optional Google Cloud dependencies for Dataflow support. Installation is straightforward via pip, and the package is positioned as a standard extension for Airflow users working with Beam-based data pipelines.
Use it for
- Schedule and monitor Apache Beam batch jobs as part of a larger Airflow data pipeline.
- Orchestrate Beam streaming pipelines with Airflow's scheduling and retry logic.
- Integrate Beam processing with other Airflow operators in a unified DAG.
- Run Beam jobs on Google Cloud Dataflow through Airflow task dependencies.
- Manage dependencies between Beam pipelines and upstream/downstream data tasks.
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes, if you are already running Apache Airflow and need to orchestrate Apache Beam pipelines.
The package is actively maintained, carries no known vulnerabilities, has low install friction, and integrates cleanly with Airflow's task model. Install only if you have both Airflow and Beam in your stack; it adds no value as a standalone library.
Install
apache-airflow-providers-apache-beam on PyPI
Before you install
Low friction install as a pure Python wheel. Actively maintained with recent releases; requires Apache Airflow >=2.11.0 and Apache Beam >=2.69.0, plus pyarrow and numpy. Supports Python 3.10–3.13.
Requires an existing Apache Airflow installation (>=2.11.0) and Apache Beam (>=2.69.0) to be available in the environment.
License in practice
Apache-2.0 permissive license allows commercial and private use without restriction, typical for Apache Foundation projects.
Quickstart
pip install apache-airflow-providers-apache-beam
from airflow.providers.apache.beam.operators.beam import BeamRunPythonPipelineOperator
task = BeamRunPythonPipelineOperator(
task_id='beam_job',
py_file='pipeline.py'
)Verify before relying
- Whether the provider supports all Apache Beam runners (Direct, Dataflow, Spark, Flink) or a subset.
- Whether optional GCP integration (apache-beam[gcp]) is required for Dataflow or only for specific features.
- Performance characteristics when orchestrating large-scale or long-running Beam pipelines.
Package facts
| License | Apache-2.0 permissive |
| Python support | Supports the current Python release >=3.10 |
| Install friction | Low. Pure-Python wheel |
| Runtime dependencies | 5 packagesapache-airflowapache-airflow-providers-common-compatapache-beampyarrownumpy |
| Maintenance | Actively maintained 165 days since the last release |
| Last repo commit | |
| First released | |
| Downloads | 287,421 / month, #8,032 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
| Classifiers | Development Status :: 5 - Production/StableEnvironment :: ConsoleEnvironment :: Web EnvironmentFramework :: Apache AirflowFramework :: Apache Airflow :: ProviderIntended Audience :: DevelopersIntended Audience :: System AdministratorsProgramming Language :: Python :: 3.10Programming Language :: Python :: 3.11Programming Language :: Python :: 3.12Programming Language :: Python :: 3.13Topic :: System :: Monitoring |
Evidence: apache_airflow_providers_apache_beam-6.2.3-py3-none-any.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “airflow apache beam integration”
- apache-airflow-providers-apache-beamIntegrates Apache Beam data processing pipelines into Apache Airflow…
- apache-airflow-providers-googleIntegrates Apache Airflow with Google Cloud services (BigQuery, Cloud…
- beam-nuggetsProvides Apache Beam transforms for reading and writing relational…
Give your agent the search over MCP, or paste the wish link into any chat.
More Monitoring packages
Wraps any iterable to display a real-time progress bar in the terminal or Jupyter notebook, showing iteration count, elapsed time, and estimated time remaining.
Provides generated Python code for OpenTelemetry semantic conventions, enabling standardized attribute naming and constant definitions for instrumentation and telemetry collection.
Install it if you are using OpenTelemetry and want to follow semantic conventions correctly.
Provides the reference implementation of the OpenTelemetry API for collecting and exporting traces, metrics, and logs from Python applications.
Provides the abstract API and interfaces for OpenTelemetry instrumentation in Python, defining how to emit traces, metrics, and logs without tying code to a specific SDK implementation.
Exports OpenTelemetry observability data to an OpenTelemetry Collector using Protobuf-encoded messages over HTTP.
Install it if you are using OpenTelemetry in Python and need to send data to a Collector over HTTP.
Provides automatic instrumentation commands and programmatic APIs to inject distributed tracing into Python applications without code changes, detecting and instrumenting packages used by your program.
Install it if you need distributed tracing without code changes and have compatible instrumented packages in your environment.
See also apache-airflow-providers-pinecone · apache-beam · apache-airflow-providers-zendesk · apache-airflow-providers-neo4j · apache-airflow-core · apache-airflow-providers-jenkins · apache-airflow-providers-git · apache-airflow-providers-databricks · apache-airflow · dag-factory