dagster-k8s

What it is and what it does

dagster-k8s is a Dagster integration that runs Dagster asset definitions and jobs on Kubernetes infrastructure. It bridges Dagster's declarative asset model—where you define data pipelines as Python functions—with Kubernetes' container orchestration, allowing you to scale asset execution across a k8s cluster. The package depends on dagster (the core orchestrator), google-auth (for credential handling), and kubernetes (the Python client library), and is designed for teams deploying Dagster in containerized environments.

The integration lets you configure how your assets run on k8s with resource specifications and executor settings. It is part of the broader Dagster ecosystem, which provides observability, lineage tracking, and a web UI for monitoring asset health and dependencies. This is most useful when your data infrastructure already runs on Kubernetes or when you need the scalability and isolation that k8s provides.

Use it for:

• Run Dagster asset jobs on Kubernetes clusters instead of local or single-machine executors.
• Scale data pipeline execution across multiple k8s nodes for parallel asset computation.
• Deploy Dagster in containerized environments where k8s is the standard orchestration layer.
• Isolate asset execution in separate k8s pods for resource control and fault isolation.
• Integrate Dagster with existing k8s-based data infrastructure and CI/CD pipelines.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Integrates Dagster data pipeline orchestration with Kubernetes, enabling declarative asset management and orchestration on k8s infrastructure.

Yes, if you are already using Dagster and deploying on Kubernetes. The package is actively maintained with no known vulnerabilities and low install friction. It is the standard way to run Dagster jobs on k8s. Install only if you have a running k8s cluster and need Dagster's orchestration on that infrastructure; otherwise, use Dagster's local or other executor options.

Install

dagster-k8s on PyPI

pip install dagster-k8s

uv add dagster-k8s

poetry add dagster-k8s

Installing dagster-k8s

Before you install

Active maintenance with a release 7 days old. Low install friction: pure Python wheel with three runtime dependencies (dagster, google-auth, kubernetes). Widely used in the top 5000 PyPI packages tier.

License in practice

Apache-2.0 permissive license allows commercial and private use with minimal restrictions, typical for infrastructure tools in this ecosystem.

Quickstart

pip install dagster-k8s

import dagster_k8s
from dagster import asset, define_asset_job

@asset
def my_asset():
    return data

my_job = define_asset_job("k8s_job")

Requires a running Kubernetes cluster and kubeconfig configured; google-auth and kubernetes client libraries must be able to authenticate to the cluster.

Verify before relying

• Specific Kubernetes versions or API compatibility requirements beyond Python 3.10–3.14.
• Whether dagster-k8s handles multi-tenant or namespace isolation out of the box.
• Performance characteristics and resource overhead for typical workloads on k8s.
• Exact executor configuration options and resource specification capabilities.

Package facts

Evidence: dagster_k8s-0.29.17-py3-none-any.whl

Tags

More Distributed Computing packages