--- id: apache-airflow-providers-celery version: "3.23.1" license: Apache-2.0 license_treatment: permissive maintenance: active --- # apache-airflow-providers-celery — Provider package apache-airflow-providers-celery for Apache Airflow License: permissive · Maintenance: active · Downloads: 1.9M/mo ## What it is and what it does This is an Apache Airflow provider package that plugs Celery into Airflow as a distributed task executor. Instead of running all DAG tasks on a single Airflow scheduler, it routes them to remote Celery workers across a cluster, letting you scale horizontally by adding more worker nodes. The package wraps Celery's task queue mechanics and integrates them with Airflow's DAG scheduling, dependency tracking, and monitoring. You install it alongside an existing Airflow setup, configure it to point to a Celery broker (typically Redis or RabbitMQ), and then set Airflow to use CeleryExecutor. Airflow will then push tasks to the broker, workers pull and execute them, and results flow back. The package includes integration with flower, Celery's monitoring tool, so you can observe worker status and task progress from a web UI. Use it for: - Scale Airflow task execution across multiple machines by running workers on separate nodes and routing DAG tasks through Celery. - Process long-running or CPU-intensive tasks in parallel without blocking the Airflow scheduler. - Monitor distributed task execution and worker health using flower's web dashboard. - Run Airflow in a multi-tenant environment where different teams or projects need isolated worker pools. - Integrate Airflow with an existing Celery infrastructure already in use for other applications. ## Worth the install? AI-flagged interpretation of the facts above — verify before relying. Integrates Celery distributed task execution with Apache Airflow, enabling workers to process DAG tasks across a cluster using Celery as the task queue backend. Yes, if you are running Apache Airflow and need to scale task execution beyond a single machine. The package is actively maintained, has no known vulnerabilities, and is production-stable. Install only if you have a Celery broker already running or planned; it is not useful without one. The low install friction and permissive Apache-2.0 license make it a straightforward addition to an existing Airflow deployment. ## Install pip install apache-airflow-providers-celery uv add apache-airflow-providers-celery poetry add apache-airflow-providers-celery ## Installing apache-airflow-providers-celery Before you install: Low friction install as a pure Python wheel. Actively maintained with a release 6 days old. Requires apache-airflow >=2.11.0, apache-airflow-providers-common-compat >=1.15.0, celery[redis] >=5.5.0,<6, and flower >=1.0.0. License in practice: Apache-2.0 permissive license allows commercial and private use without restriction, typical for Apache Foundation projects. Quickstart: pip install apache-airflow-providers-celery from airflow.providers.celery.executors.celery_executor import CeleryExecutor # Configure in airflow.cfg or environment # executor = CeleryExecutor # celery_broker_url = redis://localhost/0 Requires an existing Airflow installation (>=2.11.0) and a Celery broker (e.g., Redis or RabbitMQ) to be running separately. Verify before relying: - Whether this provider supports Kubernetes pod operators or only traditional Celery workers - Performance characteristics and scaling limits for typical cluster sizes - Specific monitoring capabilities beyond what flower provides ## Package facts - License: Apache-2.0 (permissive) - Python support: supports_current - Install friction: low - Maintenance: active - Downloads: 1.9M/month (top 5,000 on PyPI) - Known vulnerabilities: none known ## Tags airflow celery executor, distributed task queue airflow, celery airflow integration, airflow worker scaling, celery provider airflow, airflow task distribution, airflow-executor, task-distribution, celery-integration [View on SkillFed](https://skillfed.io/packages/apache-airflow-providers-celery) · [View on PyPI](https://pypi.org/project/apache-airflow-providers-celery/)