--- id: dbt-glue version: "1.10.19" license: unclear license_treatment: permissive maintenance: active --- # dbt-glue — dbt adapter for AWS Glue License: permissive · Maintenance: active · Downloads: 361.6K/mo ## What it is and what it does dbt-glue bridges dbt's transformation framework with AWS Glue's managed Spark engine, letting data teams write dbt models that execute as Spark jobs on Glue. It implements the dbt adapter protocol to translate dbt's SQL and Jinja templates into Spark-compatible operations, handling table creation, partitioning, and metadata management through Glue's catalog and Lake Formation permissions. The adapter integrates with Glue Interactive Sessions, meaning your dbt runs spin up Spark clusters on demand rather than requiring persistent infrastructure. It depends on dbt-core, dbt-spark, boto3, and pyarrow, and requires AWS IAM roles and S3 bucket access for both metadata and data storage. Configuration happens through standard dbt profiles, specifying your Glue role, region, worker count, and S3 location for output data. Use it for: - Transform raw data in S3 using dbt models, with results catalogued in Glue and queryable via Athena or other tools. - Build ELT pipelines that cleanse and denormalize data in place using Spark, coordinated through dbt's DAG and testing framework. - Manage data lake transformations with Lake Formation permissions, using dbt to orchestrate schema and table updates. - Run scheduled dbt jobs on AWS Glue without managing Spark clusters, leveraging Glue's serverless session provisioning. - Integrate dbt workflows into AWS data platforms that already use Glue for ETL, unifying transformation logic. ## Worth the install? AI-flagged interpretation of the facts above — verify before relying. dbt-glue is a dbt adapter that enables running dbt transformations against AWS Glue's Spark engine using the Glue Interactive Sessions API. Yes, if you are already running AWS Glue and want to adopt dbt for transformation orchestration and testing. The adapter is actively maintained, has low install friction, and permissive licensing. However, verify that your use case aligns with Spark's capabilities and Glue's cost model, and confirm production-readiness expectations given the Beta status. ## Install pip install dbt-glue uv add dbt-glue poetry add dbt-glue ## Installing dbt-glue Before you install: Low install friction with a pure Python wheel. Active maintenance with recent commits and a stable release cadence; marked as Beta status. Requires dbt-core, dbt-spark, and AWS SDK dependencies (boto3), all of which are well-maintained. License in practice: Permissive license treatment means you can use this adapter in commercial and proprietary projects without significant legal constraints. Quickstart: pip install dbt-glue # In profiles.yml: type: glue role_arn: arn:aws:iam::ACCOUNT:role/GlueRole region: us-east-1 schema: dbt_demo location: s3://bucket/path Requires Python 3.9 or later; AWS credentials and IAM permissions for Glue, Lake Formation, and S3 access must be configured; boto3 and AWS CLI setup recommended. Verify before relying: - Whether the adapter supports all dbt features or has known limitations in Spark/Glue compatibility. - Performance characteristics and cost implications of running dbt models through Glue Interactive Sessions. - Maturity level beyond 'Beta' status and production-readiness guarantees. ## Package facts - License: not declared (permissive) - Python support: supports_current - Install friction: low - Maintenance: active - Downloads: 361.6K/month (top 15,000 on PyPI) - Known vulnerabilities: none known ## Tags dbt adapter aws glue, spark data transformation dbt, aws glue interactive sessions, dbt glue integration, data pipeline aws glue spark, dbt spark adapter, glue etl dbt, aws-glue, dbt-adapter, spark-etl [View on SkillFed](https://skillfed.io/packages/dbt-glue) · [View on PyPI](https://pypi.org/project/dbt-glue/)