dbt-databricks

What it is and what it does

dbt-databricks is a plugin that connects the dbt data transformation framework to Databricks, allowing data analysts and engineers to build, test, and deploy SQL and Python transformations on the Databricks Lakehouse platform. It abstracts away the complexity of Databricks-specific SQL dialects and connection management, letting you write dbt models that compile to optimized Databricks SQL and Python code.

The adapter is purpose-built for Databricks rather than generic Spark, which means it takes advantage of Databricks-native features: Delta tables as the default format, Unity Catalog's three-level namespace (catalog/schema/relation) for governance, and the Photon vectorized execution engine for performance. It depends on dbt-core, dbt-spark, dbt-adapters, and Databricks' own Python SDKs to handle authentication, query execution, and metadata management.

Use it for:

• Build and version-control dbt projects that transform data in Databricks using Delta tables and incremental materializations.
• Organize multi-team data pipelines across catalogs and schemas using Unity Catalog's governance model.
• Mix SQL and Python models in a single dbt project, routing Python models to specific compute via configuration.
• Integrate dbt-databricks into CI/CD workflows or Databricks Jobs for automated data transformation runs.
• Load external data (e.g., from S3) into Delta tables using dbt macros like databricks_copy_into.
• Run data transformation workloads on Databricks with native Photon execution acceleration.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

dbt-databricks is a dbt adapter that enables data transformation workflows on Databricks using SQL and Python, with native support for Delta tables and Unity Catalog.

Yes. dbt-databricks is actively maintained, has no known vulnerabilities, and is the recommended adapter for dbt projects on Databricks. It has low install friction, permissive licensing, and integrates deeply with Databricks' native features. Install it if you are building dbt workflows on Databricks.

Install

dbt-databricks on PyPI

pip install dbt-databricks

uv add dbt-databricks

poetry add dbt-databricks

Installing dbt-databricks

Before you install

Low install friction with a pure-Python wheel. The adapter is actively maintained with a recent release and depends on well-established dbt ecosystem packages plus Databricks' own SDKs.

License in practice

Apache-2.0 is permissive; you can use this adapter in commercial projects, modify it, and distribute it freely as long as you include the license notice.

Quickstart

pip install dbt-databricks

# In your dbt profile (profiles.yml):
your_profile_name:
  target: dev
  outputs:
    dev:
      type: databricks
      schema: [database/schema name]
      host: [your.databrickshost.com]
      http_path: [/sql/your/http/path]
      token: [dapiXXXXXXXXXXXXXXXXXXXXXXX]

Requires Python 3.10 or above; requires a Databricks workspace with SQL or runtime 9.1 LTS or later, plus valid credentials.

Verify before relying

• Performance characteristics when using Photon execution engine compared to standard Spark.
• Limitations when mixing SQL and Python models in the same dbt project on Databricks.
• Support scope for Databricks runtime releases beyond 9.1 LTS.

Package facts

Evidence: dbt_databricks-1.12.4-py3-none-any.whl

Tags

More Database packages