dagster-azure

What it is and what it does

dagster-azure is a library that bridges Dagster—a cloud-native data pipeline orchestrator—with Microsoft Azure services. It provides pre-built components (ops and resources) for interacting with Azure Blob Storage, Data Lake Storage, and Azure Machine Learning, allowing you to declare data assets and pipelines in Python and have Dagster orchestrate their execution on Azure infrastructure.

The package is part of Dagster's integration ecosystem and is meant to be used alongside the core Dagster framework. It handles the plumbing between your Dagster asset definitions and Azure's SDKs, reducing boilerplate for authentication, connection management, and data movement. It's actively maintained and carries no known vulnerabilities.

Use it for:

• Build and orchestrate data pipelines that read from or write to Azure Blob Storage or Data Lake Storage.
• Integrate Azure Machine Learning model training and inference into Dagster-managed data workflows.
• Manage data asset lineage and observability across Azure-based data infrastructure from a single Dagster control plane.
• Develop and test Azure-dependent pipelines locally, then deploy the same code to production without modification.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Provides Azure-specific integrations for Dagster, enabling data pipelines to interact with Azure services like Blob Storage, Data Lake, and Machine Learning.

Yes, if you are already using Dagster and need to orchestrate workloads on Azure. The package is actively maintained, has no security vulnerabilities, and low install friction. It is a natural choice for teams standardizing on Dagster as their orchestration platform and Azure as their cloud provider. Not necessary if you are not using Dagster or do not have Azure infrastructure.

Install

dagster-azure on PyPI

pip install dagster-azure

uv add dagster-azure

poetry add dagster-azure

Installing dagster-azure

Before you install

Low friction install with a wheel distribution. Active maintenance—released 2026-08-14 with 15996 repository stars and no known vulnerabilities.

License in practice

Apache-2.0 permissive license allows commercial and private use with minimal restrictions.

Quickstart

pip install dagster-azure

import dagster as dg
from dagster_azure.adls2 import adls2_resource

@dg.asset
def my_asset() -> str:
    return "data"

Requires Python 3.10 or later (supports up to 3.14). Depends on dagster and six Azure SDK packages (azure-core, azure-identity, azure-storage-blob, azure-storage-file-datalake, azure-ai-ml).

Verify before relying

• Specific Azure resource types and operations supported beyond the six runtime dependencies listed.
• Whether this package is the recommended way to integrate Dagster with Azure or if core Dagster handles this natively.
• Performance characteristics and scalability limits when orchestrating large Azure workloads.

Package facts

Evidence: dagster_azure-0.29.18-py3-none-any.whl

Tags

More Distributed Computing packages