dbt-duckdb

What it is and what it does

dbt-duckdb is an adapter that integrates dbt's data transformation framework with DuckDB, an embedded OLAP database designed for analytics. It lets you write dbt models in SQL or Python and execute them against DuckDB, which can read and write data directly from CSV, JSON, and Parquet files without requiring a separate load step. This enables a lightweight, self-contained data stack suitable for local development, CI/CD testing, or building a data lakehouse with minimal infrastructure.

The adapter supports both in-memory and persistent database modes, DuckDB extensions (like httpfs for cloud storage), the DuckDB Secrets Manager for credential handling, and fsspec-based filesystems for S3, GCS, and Azure Blob Storage. It also integrates with MotherDuck for cloud-hosted DuckDB instances and supports persisting dbt documentation as database comments. The package is actively maintained, supports Python 3.10–3.13, and has no known vulnerabilities.

Use it for:

• Test dbt data pipelines locally or in CI jobs using an in-memory DuckDB database without external dependencies.
• Build a lightweight data lakehouse that transforms CSV, Parquet, and JSON files directly without loading them into a database first.
• Run analytics transformations against cloud storage (S3, GCS, Azure) using DuckDB extensions and fsspec integration.
• Develop and prototype data pipelines on a single machine before deploying to a larger data warehouse.
• Persist dbt model and column documentation as DuckDB comments for exploration and automated tooling.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

dbt-duckdb connects dbt (a SQL/Python transformation framework) to DuckDB, an embedded OLAP database that can read and write CSV, JSON, and Parquet files directly without loading them first.

Yes. dbt-duckdb is actively maintained, has no known vulnerabilities, low install friction, and a permissive license. It is a solid choice if you want to run dbt transformations against an embedded or cloud-hosted DuckDB instance, especially for local development, testing, or lightweight analytics workloads that read/write external files directly.

Install

dbt-duckdb on PyPI

pip install dbt-duckdb

uv add dbt-duckdb

poetry add dbt-duckdb

Installing dbt-duckdb

Before you install

Low friction: pure Python wheel with four runtime dependencies (dbt-core, dbt-adapters, dbt-common, duckdb). Actively maintained with a release 7 days ago and 1334 repository stars.

License in practice

Apache-2 permissive license: you can use, modify, and distribute dbt-duckdb freely in commercial and private projects, with minimal restrictions.

Quickstart

pip install dbt-duckdb

# In profiles.yml:
default:
  outputs:
    dev:
      type: duckdb
      path: /tmp/dbt.duckdb
  target: dev

# Then run dbt commands as usual:
dbt run

Requires dbt-core >= 1.8.x and duckdb >= 1.0.0; Python >= 3.10.

Verify before relying

• Whether the package supports all DuckDB extensions listed in the description (httpfs, parquet, h3, uc_catalog, etc.) in practice.
• Performance characteristics and scalability limits for typical data lakehouse workloads.
• Compatibility with MotherDuck and DuckLake features beyond what the description states.

Package facts

Evidence: dbt_duckdb-1.11.0-py3-none-any.whl

Keywords: setup, distutils

Tags

More Database packages