--- id: apache-airflow-core version: "3.3.1" license: Apache-2.0 license_treatment: permissive maintenance: active --- # apache-airflow-core — Core packages for Apache Airflow, schedule and API server License: permissive · Maintenance: active · Downloads: 9.7M/mo ## What it is and what it does Apache Airflow Core is the runtime engine for Apache Airflow, a platform for authoring, scheduling, and monitoring workflows. It provides the scheduler that reads DAG definitions and triggers tasks on schedule, the REST API server for external interaction, the DAG file processor that parses workflow definitions, and the triggerer component for event-driven task execution. It is designed for data engineers and DevOps teams building complex, multi-step data pipelines that may span multiple systems and require visibility into execution state. The package depends on a large ecosystem of providers and utilities—fastapi for the web server, SQLAlchemy-based ORM layers for metadata storage, jinja2 for templating, cryptography for secure credential handling, and cron scheduling libraries. It targets modern Python versions and is classified as production-stable. Installation is straightforward from a wheel, though running Airflow in practice requires external infrastructure: a database to store DAG metadata and execution history, and typically a message broker or Kubernetes cluster for distributed task execution. Use it for: - Schedule and execute recurring ETL jobs that extract data from multiple sources, transform it, and load it into a data warehouse. - Orchestrate machine learning pipelines with dependencies between data preparation, model training, and evaluation stages. - Monitor and retry failed batch jobs across distributed systems with automatic alerting and logging. - Coordinate multi-team workflows where different teams own different pipeline stages and need visibility into upstream/downstream dependencies. - Build event-driven data ingestion pipelines that react to external triggers and fan out work across worker nodes. ## Worth the install? AI-flagged interpretation of the facts above — verify before relying. Apache Airflow's core runtime that schedules, executes, and monitors data pipelines defined as directed acyclic graphs (DAGs), with a REST API server and task triggering engine. Yes. Apache Airflow Core is the foundation of a mature, widely-adopted orchestration platform with 46489 repository stars and monthly downloads in the millions. It is actively maintained, permissively licensed, and production-stable. Install it if you need to build or run Airflow workflows; it is a prerequisite for any Airflow deployment. Be aware that it brings 65 dependencies and requires external infrastructure to operate at scale. ## Install pip install apache-airflow-core uv add apache-airflow-core poetry add apache-airflow-core ## Installing apache-airflow-core Before you install: Active maintenance with a release 2 days old and 46489 repository stars. Low install friction from a pure-wheel distribution. Depends on 65 runtime packages including fastapi, cryptography, and database/scheduling libraries—a substantial but standard stack for a production orchestration platform. License in practice: Apache License 2.0 (permissive) allows commercial use, modification, and redistribution with minimal restrictions, making it suitable for enterprise and proprietary workflows. Quickstart: pip install apache-airflow-core from airflow import DAG from airflow.operators.bash import BashOperator from datetime import datetime with DAG('example', start_date=datetime(2025, 1, 1)) as dag: task = BashOperator(task_id='hello', bash_command='echo hello') Requires Python 3.10 or later (3.15 excluded); a running database backend and a message broker for distributed execution are typical production requirements. Verify before relying: - Specific performance characteristics or throughput limits under typical workloads - Whether all 65 runtime dependencies are strictly required or some are optional for minimal setups - Compatibility details with specific database and message broker versions ## Package facts - License: Apache-2.0 (permissive) - Python support: supports_current - Install friction: low - Maintenance: active - Downloads: 9.7M/month (top 5,000 on PyPI) - Known vulnerabilities: none known ## Tags workflow orchestration, dag scheduler, data pipeline automation, task scheduling engine, airflow core runtime, distributed workflow management, cron-based job scheduling, workflow-orchestration, data-pipeline, distributed-computing [View on SkillFed](https://skillfed.io/packages/apache-airflow-core) · [View on PyPI](https://pypi.org/project/apache-airflow-core/)