dbt-fabricspark

What it is and what it does

dbt-fabricspark is a dbt adapter that bridges dbt's transformation framework to Apache Spark running in Microsoft Fabric. It connects via Livy endpoints and handles both schema-enabled and non-schema Lakehouses, automatically detecting the configuration and using appropriate naming conventions (three-part for schema-enabled, two-part otherwise). The adapter manages Livy sessions with reuse and robust retry logic, supports multiple materialization types (table, view, incremental with several strategies, seed, snapshot), and includes Fabric Environment support via environmentId configuration.

The package handles Azure authentication through multiple modes (CLI for local development, Service Principal for CI/CD, Fabric Notebook for in-notebook workflows), implements credential masking and thread-safe token refresh for security, and provides resilience through HTTP 5xx retry with exponential backoff and configurable polling timeouts. It is actively maintained, production-stable, and requires Python 3.10 or later.

Use it for:

• Transform raw data in Fabric Lakehouses using dbt SQL models as part of an ELT pipeline
• Organize and cleanse data with incremental models using merge or append strategies on Fabric Spark
• Manage schema-based data organization within a single lakehouse using auto-detected three-part naming
• Develop dbt projects locally with CLI authentication, then deploy to CI/CD with Service Principal credentials
• Build snapshots and seeds on Fabric Spark to track slowly-changing dimensions and load reference data

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

A dbt adapter that connects dbt to Apache Spark in Microsoft Fabric via Livy endpoints, enabling SQL-based data transformation workflows on Fabric Lakehouses with or without schema support.

Yes. This adapter is production-stable, actively maintained (last commit 13 days ago), has no known vulnerabilities, and low install friction. Install it if you use dbt with Microsoft Fabric Spark and need to transform data in Lakehouses. If you do not have a Fabric workspace or are not using dbt, it is not applicable.

Install

dbt-fabricspark on PyPI

pip install dbt-fabricspark

uv add dbt-fabricspark

poetry add dbt-fabricspark

Installing dbt-fabricspark

Before you install

Low friction: pure Python wheel with no compiled dependencies. Active maintenance—last commit 13 days ago, production-stable status. Requires dbt-core and Azure authentication libraries (azure-core, azure-identity), all standard packages.

License in practice

MIT license (permissive). You can use, modify, and distribute this adapter freely in commercial and open-source projects with minimal restrictions.

Quickstart

pip install dbt-fabricspark

# In profiles.yml:
fabric-spark-test:
  target: fabricspark-dev
  outputs:
    fabricspark-dev:
      type: fabricspark
      method: livy
      endpoint: https://api.fabric.microsoft.com/v1
      workspaceid: <workspace-id>
      lakehouseid: <lakehouse-id>
      lakehouse: my_lakehouse
      schema: my_lakehouse
      authentication: CLI
      threads: 1

# Then: dbt run

Requires Microsoft Fabric workspace and lakehouse; Azure CLI authentication (CLI mode) needs optional azure-cli extra. Service Principal (SPN) authentication does not require it.

Verify before relying

• Performance characteristics and typical query latency on Fabric Spark clusters
• Supported incremental materialization strategies beyond those listed (append, merge, insert_overwrite, microbatch, delete+insert)
• Compatibility with dbt Cloud vs. dbt Core-only workflows
• Session reuse behavior and overhead reduction in production CI/CD pipelines

Package facts

Evidence: dbt_fabricspark-1.13.0-py3-none-any.whl

Keywords: Fabric lakehouse, adapter, adapters, database, dbt, dbt Cloud, dbt Core, dbt Labs, dbt-core, elt, fabric, fabricspark, fabricspark adapter, fabricspark dbt, fabricspark dbt adapter, fabricspark dbt adapters, fabricspark dbt cloud, fabricspark dbt core, fabricspark dbt labs, lakehouse, lakehouse dbt, lakehouse dbt adapter, lakehouse dbt adapters, lakehouse dbt cloud, lakehouse dbt core, lakehouse dbt labs, microsoft, spark

Tags

More Database packages