jobflow

What it is and what it does

Jobflow is a workflow orchestration library that lets you build complex computational pipelines by decorating Python functions as jobs and composing them into flows. Jobs are atomic units of work whose inputs and outputs can be serialized to JSON; flows automatically determine job execution order based on data dependencies. The library supports dynamic workflows that can modify themselves during execution, nested flows for natural composition of complex logic, and output storage across multiple database backends through its Maggma integration.

You define workflows using a clean Python API where job outputs are referenced as futures before execution, allowing automatic dependency resolution. Jobflow handles both local execution and distributed execution through jobflow-remote or FireWorks, making it suitable for both development and production high-throughput computing environments. Its main dependencies are PyYAML, maggma, monty, networkx, pydantic, pydantic-settings, and pydash.

Use it for:

• Build multi-step scientific simulations where later computations depend on outputs from earlier ones.
• Orchestrate high-throughput materials or chemistry calculations across multiple machines via a queue system.
• Define self-modifying workflows that spawn new jobs based on intermediate results.
• Store and retrieve job outputs from MongoDB, S3, or other backends managed by Maggma.
• Develop and test workflow logic locally before deploying to a distributed computing cluster.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Jobflow is a Python library for defining and executing computational workflows as directed acyclic graphs of jobs, with support for local execution and integration with workflow managers like jobflow-remote and FireWorks.

Yes. Jobflow is actively maintained, has no known vulnerabilities, supports current Python versions (3.10–3.13), and offers a clean API for workflow composition with strong support for both local and distributed execution. It is well-suited for scientific computing and high-throughput applications where job dependencies and output management are central concerns.

Install

jobflow on PyPI

pip install jobflow

uv add jobflow

poetry add jobflow

Installing jobflow

Before you install

Low friction install with seven runtime dependencies. Actively maintained as of 2026-02-05 with recent commits (2026-08-10), supporting Python 3.10 through 3.13. No known vulnerabilities.

License in practice

Released under a modified BSD license, which is permissive and allows commercial and private use with minimal restrictions.

Quickstart

pip install jobflow

from jobflow import job, Flow

@job
def add(a, b):
    return a + b

add_first = add(1, 5)
add_second = add(add_first.output, 5)
flow = Flow([add_first, add_second])

Requires Python 3.10 or later.

Verify before relying

• Whether jobflow-remote or FireWorks integration requires additional configuration beyond the base install.
• Performance characteristics and scalability limits for large workflow graphs.
• Whether MongoDB, S3, or GridFS backends require separate setup or credentials.

Package facts

Evidence: jobflow-0.3.1-py3-none-any.whl

Keywords: high-throughput, workflow

Tags

More Scientific/Engineering packages

Further reading