{"categories":[{"label":"Distributed Computing","url":"https://skillfed.io/packages/category/system-distributed-computing/2"}],"enrichment":{"capability":"Integrates Databricks with Dagster's data orchestration framework, enabling you to define and run data pipelines that interact with Databricks clusters and SQL warehouses.","skillfed_tags":["databricks-integration","data-orchestration","etl"],"use_cases":["Build and schedule SQL transformations on Databricks Delta Lake tables, with Dagster tracking lineage and data quality.","Orchestrate multi-step data pipelines that read from Databricks, transform in Python, and write results back to Databricks.","Run PySpark jobs on Databricks clusters via Dagster, with centralized monitoring and error handling across your data platform.","Define machine learning training pipelines that pull features from Databricks and log models back to a Databricks workspace."],"what_it_does":"dagster-databricks is a Dagster integration library that bridges Dagster's declarative data orchestration framework with Databricks infrastructure. It provides Databricks-specific ops and resource components that let you build data pipelines as Python functions, with Dagster handling scheduling, lineage tracking, and observability while your code runs against Databricks clusters and SQL warehouses.\n\nThe package sits within Dagster's broader ecosystem\u2014you declare data assets using Dagster's programming model, and dagster-databricks handles the Databricks-specific plumbing: authentication, cluster interaction, and job submission. It's designed for teams already using Databricks who want Dagster's asset-centric orchestration, testing, and monitoring capabilities layered on top.","worth_installing":"Yes, if you are already using Dagster and Databricks and want a native integration between them. The package is actively maintained, has no known vulnerabilities, and low install friction. Install only if you need Databricks-specific orchestration components; the base dagster library alone may suffice for simpler use cases."},"id":"dagster-databricks","links":{"html":"https://skillfed.io/packages/dagster-databricks","md":"https://skillfed.io/packages/dagster-databricks.md","pypi":"https://pypi.org/project/dagster-databricks/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2026-08-14","license_spdx":"Apache-2.0","license_treatment":"permissive","name":"dagster-databricks","python_support":"supports_current","summary":"Package for Databricks-specific Dagster framework op and resource components."},"popularity":{"monthly_downloads":576517,"position":5928,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"0.29.18"}