substrait

What it is and what it does

Substrait is a Python binding for the Substrait specification, a cross-language standard for representing data compute operations. It lets you build query plans programmatically in Python and serialize them for execution by Substrait-compatible engines like DuckDB or DataFusion. The package offers two main APIs: a high-level DataFrame interface (similar to Polars or PySpark) for ergonomic plan authoring, and lower-level proto and builder layers for fine-grained control. Plans can be constructed from scratch, parsed from binary or JSON representations, or consumed from other Substrait producers like Ibis.

The package is experimental and actively developed. It depends on protobuf for serialization and substrait-protobuf for the core message definitions. It is not an execution engine itself—it is purely a plan builder and consumer. The DataFrame API is the recommended approach for hand-authored plans, while the proto layer provides direct access to the underlying Substrait message types for advanced use cases.

Use it for:

• Build query plans in Python and pass them to DuckDB or DataFusion for execution.
• Parse and inspect Substrait plans received from other producers like Ibis.
• Serialize plans to binary or JSON for transmission or storage across language boundaries.
• Construct complex data operations programmatically using the fluent DataFrame API.
• Integrate Substrait plan generation into Python data pipelines or query optimization tools.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Provides a Python interface to construct, manipulate, and serialize Substrait Plans—cross-language specifications for data compute operations—for consumption by engines like DuckDB or DataFusion.

Yes, if you need to generate or consume Substrait plans in Python. Low install friction, active maintenance, no known vulnerabilities, and permissive licensing make it safe to adopt. The DataFrame API is approachable for typical use cases. Not suitable if you need plan execution—use it only as a builder for engines that consume Substrait.

Install

substrait on PyPI

pip install substrait

uv add substrait

poetry add substrait

Installing substrait

Before you install

Low install friction; pure Python wheel with three runtime dependencies (protobuf, substrait-protobuf, substrait-extensions). Package is actively maintained with a release 16 days old.

License in practice

Apache-2.0 permissive license allows commercial and private use with minimal restrictions; suitable for most projects.

Quickstart

pip install substrait

import substrait.dataframe as sub

plan = (
    sub.read_named_table("people", {"id": sub.i64, "age": sub.i64})
    .filter(sub.col("age") > 25)
    .to_plan()
)

serialized = plan.SerializeToString()

Requires Python 3.10 or later (and less than 3.15). For function overload resolution, install the extensions extra: pip install "substrait[extensions]".

Verify before relying

• Whether the DataFrame API fully covers all Substrait specification features or if some plans require the lower-level proto/builders API.
• Performance characteristics when building or serializing large or deeply nested plans.
• Compatibility guarantees with specific versions of DuckDB, DataFusion, or other Substrait consumers.

Package facts

Evidence: substrait-0.30.0-py3-none-any.whl

Tags

More Database packages