brickflows
Deploy scalable workflows to databricks using python
Decision gist · record as of 2026-08-14
Yes, if you are building data pipelines on Databricks and want to manage workflows as Python code. The package is actively maintained, has low install friction, uses permissive licensing, and integrates cleanly with standard Python tooling. It is most valuable for teams already using Python for data work and seeking a declarative, version-controllable alternative to UI-based job configuration.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires Databricks CLI installed and configured with workspace credentials (~/.databrickscfg file).
- Low install friction with a pure-Python wheel and 14 well-established runtime dependencies.
- Actively maintained with a release 7 days ago and recent commits; the project has 228 stars and is in the top 5000 PyPI packages by downloads.
License · maintenance · safety
Apache-2.0 (permissive) — Licensed under Apache-2.0 (permissive), allowing commercial use, modification, and distribution with minimal restrictions—suitable for most production environments.
last release 2026-08-07 (7 days) · last repo commit 2026-08-12 · 228 stars
0 known vulnerabilities (OSV.dev, 2026-08-14) · 813,998 downloads/mo, #4,995 on PyPI
Alternatives
Verify before relying
pip install brickflows
from brickflows import Workflow, Cluster
cluster = Cluster(name="job_cluster", node_type_id="m6gd.xlarge", spark_version="13.3.x-scala2.12")
wf = Workflow("my_workflow", default_cluster=cluster)
@wf.task
def my_task():
print("Hello")- Whether the framework supports Databricks asset bundles or only legacy jobs API.
- Performance characteristics and overhead when orchestrating large numbers of tasks.
- Compatibility with Databricks Unity Catalog features beyond the common_task_parameters example.
What it is and what it does
Brickflow lets you define workflows as Python code using decorators and task dependencies, then deploy them via CLI. Instead of manually configuring jobs in a UI, you write Python functions with @wf.task decorators, specify their dependencies, and the framework translates that into workflow definitions. It handles cluster configuration, notebook execution, Spark SQL, and bash commands as tasks within a single workflow.
The package integrates with click for CLI, Jinja2 for templating, Pydantic for validation, and scheduling libraries (croniter, pendulum, cron-descriptor) to support cron-based triggers and complex task orchestration. It's aimed at teams building data pipelines who want infrastructure-as-code for their workflows, with a focus on local development and CI/CD deployment patterns.
Use it for
- Define multi-task ETL pipelines as Python code with task dependencies, then deploy in one command.
- Parameterize workflows using common_task_parameters to inject catalog, schema, and environment variables across tasks.
- Orchestrate mixed workloads: Python tasks, notebook execution, Spark SQL, and bash commands in one definition.
- Version control workflow definitions alongside code and deploy via CI/CD pipelines.
- Schedule recurring workflows using cron expressions with automatic descriptor generation and timezone handling.
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes, if you are building data pipelines on Databricks and want to manage workflows as Python code.
The package is actively maintained, has low install friction, uses permissive licensing, and integrates cleanly with standard Python tooling. It is most valuable for teams already using Python for data work and seeking a declarative, version-controllable alternative to UI-based job configuration.
Install
brickflows on PyPI
Before you install
Low install friction with a pure-Python wheel and 14 well-established runtime dependencies. Actively maintained with a release 7 days ago and recent commits; the project has 228 stars and is in the top 5000 PyPI packages by downloads.
Requires Databricks CLI installed and configured with workspace credentials (~/.databrickscfg file).
License in practice
Licensed under Apache-2.0 (permissive), allowing commercial use, modification, and distribution with minimal restrictions—suitable for most production environments.
Quickstart
pip install brickflows
from brickflows import Workflow, Cluster
cluster = Cluster(name="job_cluster", node_type_id="m6gd.xlarge", spark_version="13.3.x-scala2.12")
wf = Workflow("my_workflow", default_cluster=cluster)
@wf.task
def my_task():
print("Hello")Verify before relying
- Whether the framework supports Databricks asset bundles or only legacy jobs API.
- Performance characteristics and overhead when orchestrating large numbers of tasks.
- Compatibility with Databricks Unity Catalog features beyond the common_task_parameters example.
Package facts
| License | Apache-2.0 permissive |
| Python support | Capped below the current Python release <3.13,>=3.9 |
| Install friction | Low. Pure-Python wheel |
| Runtime dependencies | 14 packagesJinja2clickcron-descriptorcroniterdatabricks-sdknetworkxpendulumpluggypydanticpython-dateutilpython-decouplepytzpyyamlrequests |
| Maintenance | Actively maintained 7 days since the last release |
| Last repo commit | |
| First released | |
| Downloads | 813,998 / month, #4,995 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
| Classifiers | License :: OSI Approved :: Apache Software LicenseProgramming Language :: Python :: 3Programming Language :: Python :: 3.10Programming Language :: Python :: 3.11Programming Language :: Python :: 3.12Programming Language :: Python :: 3.9 |
Evidence: brickflows-2.0.0-py3-none-any.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “databricks workflow orchestration”
- brickflowsA Python framework and CLI tool for building and deploying workflows…
- dataengineDataengine is a Python framework for orchestrating data pipelines…
- dagster-databricksIntegrates Databricks with Dagster's data orchestration framework,…
Give your agent the search over MCP, or paste the wish link into any chat.
More Build Tools packages
Provides reusable utilities for Python packaging interoperability, including version handling, specifiers, markers, requirements, tags, and metadata parsing according to standards like PEP 440 and PEP 425.
Wraps any iterable to display a real-time progress bar in the terminal or Jupyter notebook, showing iteration count, elapsed time, and estimated time remaining.
pip is the standard installer for Python packages, enabling you to download and install packages from the Python Package Index and other indexes into your Python environment.
Hatchling is a standards-compliant Python build backend that handles packaging, metadata, and distribution of Python projects when configured in a project's pyproject.toml file.
Generates Python gRPC service stubs and message classes from Protocol Buffer definitions, enabling developers to build gRPC clients and servers.
pre-commit is a framework for installing and running git hooks written in any language before commits are made, automating code quality and validation checks across multi-language projects.
Install it if your team needs consistent, automated validation at commit time.
See also databricks-connect · databricks-cli · databricks-bundles · koheesio · dbx · databricks-sdk · dbt-databricks · dlt-meta · databricksapi · nutter