fugue

What it is and what it does

Fugue is an abstraction layer that lets you write Python and pandas code once and run it on multiple distributed computing backends without rewriting the core logic. It has two main interfaces: the Fugue API (functions like transform(), load(), save()) that work across all backends, and FugueSQL, an enhanced SQL dialect that can invoke Python functions and run on any backend.

The package solves the problem of code portability in data workflows. Instead of writing separate code for different execution engines, you write it once and Fugue handles the distribution. The library depends on triad, adagio, and pandas; additional backends require optional extras. It is actively maintained, supports current Python versions (3.10+), and has no known vulnerabilities.

Use it for:

• Scale an existing pandas transformation by wrapping it in transform() without rewriting the function logic.
• Define an end-to-end data pipeline in FugueSQL that can run locally for testing or on a distributed backend for production.
• Parallelize a custom Python function across partitions by passing it to Fugue's API instead of writing backend-specific code.
• Load, transform, and save data in a workflow that is agnostic to the execution backend.
• Combine SQL queries with Python-defined transformations in a single FugueSQL statement across multiple backends.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Fugue provides a unified Python interface to write code once and execute it on multiple distributed computing backends with minimal rewrites, plus a SQL-based language for defining workflows.

Yes. Fugue is actively maintained, has low install friction, permissive licensing, and solves a real problem—writing portable distributed code. It is well-suited if you want to scale workflows without major rewrites. Install the base package for the core API; add extras only for the backends you need.

Install

fugue on PyPI

pip install fugue

uv add fugue

poetry add fugue

Installing fugue

Before you install

Low friction installation with a pure-Python wheel. Maintenance is active with recent commits and a release 175 days ago. Core functionality works without extras; optional extras unlock support for additional backends.

License in practice

Apache-2.0 is permissive; you can use, modify, and distribute Fugue freely in commercial and private projects with minimal restrictions.

Quickstart

pip install fugue

import pandas as pd
from fugue import transform

def my_func(df: pd.DataFrame) -> pd.DataFrame:
    return df

df = pd.DataFrame({"a": [1, 2]})
result = transform(df, my_func, schema="*")

Requires Python 3.10 or later. FugueSQL requires the sql extra. Backend support (Spark, Dask, Ray, DuckDB, Polars) requires corresponding optional extras.

Verify before relying

• Whether the CPP antlr parser extra is pre-built for your Python version and platform, or requires a C++ compiler.
• Performance characteristics of the pure Python SQL parser versus the optional CPP parser in your workload.
• Which specific backends (Spark, Dask, Ray, DuckDB, Polars) you need and whether pre-built binaries exist for your environment.

Package facts

Evidence: fugue-0.9.7-py3-none-any.whl

Keywords: distributed, spark, dask, ray, duckdb, sql, dsl, domain specific language

Tags

More Python Modules packages