apache-airflow-providers-apache-pig

What it is and what it does

This is an Apache Airflow provider package that adds Pig integration to Airflow's orchestration framework. It supplies operators and hooks that allow you to define and run Apache Pig scripts as tasks within Airflow DAGs, enabling Pig-based data processing to be scheduled, monitored, and coordinated alongside other Airflow tasks.

The package is part of the official Apache Airflow ecosystem and is actively maintained. It depends on apache-airflow (>=2.11.0) and apache-airflow-providers-common-compat (>=1.10.1), and supports Python 3.10 through 3.14. Installation is straightforward via pip, with no compiled dependencies in the package itself, though your Airflow environment and cluster must have Pig available to execute scripts.

Use it for:

• Schedule and monitor Apache Pig scripts as part of larger Airflow data pipelines without custom orchestration code.
• Integrate Pig-based ETL jobs with other Airflow operators (SQL, Spark, Python) in a single DAG.
• Manage dependencies and retry logic for Pig scripts using Airflow's native task scheduling and error handling.
• Orchestrate Pig jobs on Hadoop clusters from a centralized Airflow control plane.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Integrates Apache Pig with Apache Airflow, providing operators and hooks to orchestrate Pig scripts as Airflow tasks within data pipelines.

Yes, if you run Pig scripts in production and use Airflow for orchestration. The package is actively maintained, has no known vulnerabilities, installs cleanly, and is permissively licensed. Install it only if you have Pig workloads to integrate; it adds no value for Airflow users who do not use Pig.

Install

apache-airflow-providers-apache-pig on PyPI

pip install apache-airflow-providers-apache-pig

uv add apache-airflow-providers-apache-pig

poetry add apache-airflow-providers-apache-pig

Installing apache-airflow-providers-apache-pig

Before you install

Low install friction with a pure-Python wheel. Actively maintained as part of the Apache Airflow ecosystem, with recent releases and no known vulnerabilities. Requires apache-airflow >=2.11.0 and apache-airflow-providers-common-compat >=1.10.1.

License in practice

Licensed under Apache License 2.0 (permissive). Safe for commercial and private use with minimal restrictions; you must retain license notices and can modify and distribute freely.

Quickstart

pip install apache-airflow-providers-apache-pig

from airflow.providers.apache.pig.operators.pig import PigOperator
from airflow import DAG

with DAG('pig_example') as dag:
    pig_task = PigOperator(
        task_id='run_pig_script',
        pig='script.pig'
    )

Requires an existing Airflow installation (>=2.11.0) and Python 3.10 or later. Pig itself must be available in your environment or cluster.

Verify before relying

• Whether Pig must be pre-installed on the target system or if the provider handles Pig runtime setup
• Supported Pig versions and any compatibility constraints with specific Hadoop distributions

Package facts

Evidence: apache_airflow_providers_apache_pig-4.8.5-py3-none-any.whl

Keywords: airflow-provider, apache.pig, airflow, integration

Tags

More Monitoring packages