apache-airflow-providers-papermill

What it is and what it does

This is an Apache Airflow provider package that bridges Jupyter notebooks and Airflow orchestration by wrapping the Papermill library. It lets you define notebook execution as native Airflow tasks, passing parameters into notebooks and capturing their outputs within your DAG. The package depends on Papermill for parameterized notebook execution, Scrapbook for output collection, and standard data-science libraries (pandas, ipykernel, nbconvert) to handle notebook rendering and kernel management.

You install it alongside an existing Airflow deployment (>=2.11.0) and use its PapermillOperator to run .ipynb files with injected parameters, making notebooks first-class citizens in Airflow workflows. It's designed for teams that want to orchestrate notebook-based analytics, ML experiments, or data processing pipelines without rewriting them as Python scripts.

Use it for:

• Schedule parameterized Jupyter notebooks as recurring Airflow tasks, injecting date ranges or configuration values at runtime.
• Orchestrate multi-step ML pipelines where each step is a notebook, capturing metrics and artifacts between steps.
• Run exploratory data analysis notebooks on a schedule with different input datasets, storing outputs for review.
• Integrate notebook-based reporting into Airflow DAGs, generating reports with dynamic parameters.
• Chain notebook execution with other Airflow operators to build hybrid workflows mixing notebooks and Python tasks.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Integrates Papermill notebook execution into Apache Airflow workflows, allowing you to parameterize and run Jupyter notebooks as Airflow tasks.

Yes, if you run Apache Airflow and want to execute Jupyter notebooks as orchestrated tasks. The package is actively maintained, has no known vulnerabilities, and low install friction. It's the standard way to integrate Papermill into Airflow. Install it only if you have an Airflow deployment already running and a genuine need to parameterize and schedule notebook execution; it adds dependencies (pandas, nbconvert, ipykernel) that are only useful in that context.

Install

apache-airflow-providers-papermill on PyPI

pip install apache-airflow-providers-papermill

uv add apache-airflow-providers-papermill

poetry add apache-airflow-providers-papermill

Installing apache-airflow-providers-papermill

Before you install

Low friction installation as a pure-Python wheel. Actively maintained with a release 6 days old; Airflow ecosystem packages are typically well-supported. Requires Apache Airflow >=2.11.0 and seven runtime dependencies including papermill, scrapbook, pandas, and nbconvert.

License in practice

Apache License 2.0 (permissive). You can use, modify, and distribute this package freely in commercial and private projects with minimal restrictions.

Quickstart

pip install apache-airflow-providers-papermill

from airflow.providers.papermill.operators.papermill import PapermillOperator

task = PapermillOperator(
    task_id='run_notebook',
    input_nb='path/to/notebook.ipynb',
    output_nb='path/to/output.ipynb',
    parameters={'param1': 'value1'}
)

Requires Apache Airflow >=2.11.0 and Python >=3.10; Jupyter notebook infrastructure must be available in your Airflow environment.

Verify before relying

• Whether the package supports execution in containerized or remote Airflow environments (e.g., Kubernetes executor).
• Performance characteristics when running large or long-running notebooks as Airflow tasks.
• How scrapbook integration works for capturing notebook outputs and metrics in Airflow context.

Package facts

Evidence: apache_airflow_providers_papermill-3.13.2-py3-none-any.whl

Keywords: airflow-provider, papermill, airflow, integration

Tags

More Monitoring packages