apache-airflow-providers-common-ai

What it is and what it does

This is an Apache Airflow provider package that adds AI and LLM capabilities to Airflow pipelines by wrapping pydantic-ai. It supplies hooks and operators designed to let you orchestrate LLM-driven tasks as part of Airflow DAGs—allowing you to call language models, process their outputs, and integrate AI into data workflows alongside traditional data pipeline steps.

The package depends on Apache Airflow >=3.0.0, pydantic-ai-slim >=2.0.0, and Airflow's common-compat and standard providers. It supports Python 3.10 through 3.14. Optional extras unlock integrations with specific LLM providers (OpenAI, Anthropic, Bedrock, Google) and add-ons like code execution, safety shields, and data format support (Avro, Parquet, PDF, DOCX).

Use it for:

• Build Airflow DAGs that call LLMs to generate summaries, classifications, or transformations of data flowing through your pipeline.
• Orchestrate multi-step workflows where LLM outputs feed into downstream tasks, combining AI inference with traditional ETL logic.
• Integrate with multiple LLM providers (OpenAI, Anthropic, Bedrock, Google) through a unified Airflow operator interface.
• Add AI-powered data quality checks or anomaly detection as Airflow tasks using LLM reasoning.
• Process documents (PDF, DOCX) or query databases with AI assistance as part of scheduled Airflow jobs.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Provides AI/LLM hooks and operators for Apache Airflow pipelines, integrating pydantic-ai to enable LLM-driven workflow tasks.

Yes, if you run Apache Airflow >=3.0.0 and need to add LLM capabilities to your data pipelines. The package is actively maintained, has no known vulnerabilities, installs with low friction, and is licensed permissively. Start with the base install; add optional extras only for the LLM providers you actually use.

Install

apache-airflow-providers-common-ai on PyPI

pip install apache-airflow-providers-common-ai

uv add apache-airflow-providers-common-ai

poetry add apache-airflow-providers-common-ai

Installing apache-airflow-providers-common-ai

Before you install

Low friction installation as a pure Python wheel. Actively maintained with a release 6 days old. Requires Airflow >=3.0.0 and pydantic-ai-slim >=2.0.0; integrates cleanly into existing Airflow deployments.

License in practice

Apache-2.0 permissive license allows commercial use, modification, and redistribution with minimal restrictions—standard for Apache Foundation projects.

Quickstart

pip install apache-airflow-providers-common-ai

from airflow.providers.common.ai import ...
# Use AI operators/hooks in your Airflow DAG definitions

Requires Apache Airflow >=3.0.0 and Python >=3.10; pydantic-ai-slim >=2.0.0 must be installed.

Verify before relying

• Which specific AI operators and hooks are available in the common.ai provider package.
• Whether the package supports streaming or batch LLM inference modes.
• How error handling and retries are configured for LLM calls in Airflow tasks.
• Whether optional extras (anthropic, bedrock, google, openai, mcp) require separate API credentials or configuration.

Package facts

Evidence: apache_airflow_providers_common_ai-0.7.0-py3-none-any.whl

Keywords: airflow, airflow-provider, common.ai, integration

Tags

More Monitoring packages