---
id: mad-prefect
version: "2.3.0"
license: unclear
license_treatment: unclear
maintenance: active
---

# mad-prefect

License: unclear · Maintenance: active · Downloads: 104.1K/mo

## What it is and what it does

mad_prefect wraps Prefect workflow functions with a `@asset` decorator that automatically handles data persistence, caching, and querying. When you decorate a function, it becomes a data asset that can materialize its output to disk (in JSON, Parquet, or CSV), reuse cached results within a configurable expiration window, and query the stored data using DuckDB without loading it entirely into memory.

The library is built on seven runtime dependencies: prefect (the orchestration framework), duckdb (for SQL querying), pandas and pyarrow (for data handling), fsspec and sshfs (for filesystem abstraction), and jsonlines (for line-delimited JSON support). It targets Python 3.11 and 3.12, and is actively maintained. Use it to build modular ETL pipelines where each step is independently executable and queryable, or to cache intermediate results in machine learning workflows.

Use it for:

- Build ETL pipelines where each transformation step is a queryable data asset that can be executed and cached independently.
- Cache intermediate machine learning datasets and query them for model training without recomputing or loading full data into memory.
- Enable data analysts to query preprocessed assets via SQL without understanding the underlying retrieval logic.
- Maintain historical snapshots of data assets for auditing and rollback in incremental data processing workflows.
- Reduce redundant computations in multi-step workflows by reusing cached artifacts within configurable expiration windows.

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

Adds a data asset pattern to Prefect workflows, enabling functions to persist, cache, and query data using DuckDB with minimal boilerplate.

Yes, with conditions. The package solves a real problem—adding lightweight data asset management to Prefect—and has low install friction with active maintenance. However, the unclear license status is a blocker for any project with strict licensing requirements; resolve that first. Once licensed, it's a solid choice for Prefect users building data pipelines that benefit from caching, querying, and artifact management.

## Install

pip install mad-prefect
uv add mad-prefect
poetry add mad-prefect

## Installing mad-prefect

Before you install: Low install friction; wheel-only distribution. Depends on seven runtime packages including prefect, duckdb, and fsspec. Actively maintained with a release within the last 126 days.

License in practice: License status is unclear—no SPDX identifier or raw license text provided. Verify the actual license terms before adopting in proprietary or copyleft-sensitive projects.

Quickstart:

pip install mad-prefect

from mad_prefect.data_assets import asset
from datetime import timedelta

@asset(path="/data/results.json", cache_expiration=timedelta(hours=1))
def generate_data():
    return [{"id": 1, "value": "a"}]

result = await generate_data()
query_result = await generate_data.query("WHERE id > 0")

Requires Python 3.11 or 3.12 (capped below 3.13); async/await syntax requires an async context.

Verify before relying:

- Whether the unclear license is a documentation gap or a genuine licensing issue requiring clarification before use.
- Performance characteristics and memory footprint when querying large datasets through DuckDB.
- Compatibility with Prefect versions beyond what runtime dependency pinning may specify.

## Package facts

- License: not declared (unclear)
- Python support: capped_below_current
- Install friction: low
- Maintenance: active
- Downloads: 104.1K/month (top 15,000 on PyPI)
- Known vulnerabilities: none known

## Tags

prefect data assets, workflow data caching, duckdb query integration, prefect pipeline data management, etl asset pattern, data persistence decorator, workflow artifact querying, prefect-integration, data-pipeline, caching

[View on SkillFed](https://skillfed.io/packages/mad-prefect) · [View on PyPI](https://pypi.org/project/mad-prefect/)