prefect-ray

What it is and what it does

prefect-ray is an integration layer that connects Prefect's workflow orchestration engine to Ray, a distributed computing framework. It allows Prefect workflows to execute tasks in parallel across Ray clusters, enabling horizontal scaling of compute-intensive workloads. The package depends on both prefect and ray as runtime dependencies, meaning you need both libraries installed and configured to use it effectively.

Typically used when you have Prefect workflows that need to run tasks in parallel on multiple machines or cores, or when you want to leverage Ray's distributed computing capabilities within a Prefect-orchestrated pipeline. The integration handles the bridge between Prefect's task model and Ray's execution model.

Use it for:

• Run CPU-intensive Prefect tasks in parallel across a Ray cluster to reduce total workflow execution time.
• Scale machine learning training or batch inference jobs orchestrated by Prefect to multiple nodes via Ray.
• Execute data processing pipelines with Prefect while distributing compute across Ray workers.
• Coordinate multi-stage workflows in Prefect where individual stages benefit from Ray's parallel execution.
• Integrate Ray's fault tolerance with Prefect's workflow scheduling and monitoring capabilities.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Integrates Ray distributed computing with Prefect workflow orchestration to enable parallel task execution across Ray clusters.

Yes, if you are already using Prefect and need distributed task execution. The package is actively maintained, has low install friction, carries no licensing restrictions, and supports Python 3.10, 3.11, and 3.12. Install it when you have Prefect workflows that would benefit from Ray's parallel execution and you have Ray infrastructure available.

Install

prefect-ray on PyPI

pip install prefect-ray

uv add prefect-ray

poetry add prefect-ray

Installing prefect-ray

Before you install

Low friction installation with a pure Python wheel. Actively maintained with recent releases. Requires Python 3.10 or later but excludes Python 3.13.

License in practice

Apache License 2.0 is permissive and poses no restrictions on commercial or private use, modification, or redistribution.

Quickstart

pip install prefect-ray

from prefect import flow, task
from prefect_ray import RayTaskRunner

@flow
def my_workflow():
    pass

my_workflow(task_runner=RayTaskRunner())

Requires Ray cluster to be running and accessible; Python 3.10 or later (not 3.13).

Verify before relying

• Whether Ray cluster setup and configuration is documented within the package or requires external Ray knowledge.
• What specific Prefect workflow patterns or task types benefit most from Ray execution versus other executors.
• How the integration handles Ray cluster lifecycle management and fault recovery.

Package facts

Evidence: prefect_ray-0.5.0-py3-none-any.whl

Keywords: prefect

Tags

More Libraries packages