taskflow

What it is and what it does

TaskFlow is a structured state management library designed to orchestrate jobs, tasks, and flows in a declarative, highly available manner. It provides a framework for defining workflows as directed acyclic graphs of tasks, with built-in support for error recovery, rollback, and persistence. The library is part of the OpenStack ecosystem but is general-purpose and can be used in any Python project that needs reliable task execution.

The package ships with core execution capabilities and a pluggable architecture for optional backends: persistence layers (e.g., sqlalchemy), execution engines (e.g., worker-based via kombu), and job boards (e.g., kazoo-based). Most of these are opt-in extras; the base installation provides in-memory execution and local task scheduling. It depends on networkx for graph representation, stevedore for plugin management, and oslo libraries for serialization and utilities.

Use it for:

• Build reliable ETL pipelines with automatic retry and rollback on failure.
• Orchestrate multi-step deployment or provisioning workflows in cloud infrastructure.
• Define complex job dependencies and execute them across distributed worker pools.
• Implement long-running background job queues with persistence and recovery.
• Model state transitions and task workflows in microservice architectures.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

TaskFlow is a library for defining and executing workflows as structured state machines, with support for task composition, error handling, persistence, and distributed execution across multiple workers.

Yes, if you need declarative workflow orchestration with built-in fault tolerance and persistence. TaskFlow is production-stable (since 2013), actively maintained, has low install friction, and carries a permissive license. Its 13 runtime dependencies are manageable for most environments. Install it if you're building task-driven systems; skip it if you only need simple job queuing or one-off script execution.

Install

taskflow on PyPI

pip install taskflow

uv add taskflow

poetry add taskflow

Installing taskflow

Before you install

Low friction installation with a pure-Python wheel. Actively maintained (last release 88 days ago). Requires 13 runtime dependencies including networkx, stevedore, and oslo libraries; optional extras like workers, sqlalchemy, and kazoo backends are available for specialized use cases.

License in practice

Apache-2.0 permissive license allows commercial use, modification, and distribution with minimal restrictions; suitable for most proprietary and open-source projects.

Quickstart

pip install taskflow

from taskflow import task, flow

class MyTask(task.Task):
    def execute(self):
        return "result"

my_flow = flow.Flow('my-flow').add(MyTask())
my_flow.run()

Requires Python 3.10 or later. Optional persistence and worker backends (sqlalchemy, kombu, kazoo) must be installed separately via extras if needed.

Verify before relying

• Whether the 13 runtime dependencies are all strictly necessary for basic workflow execution or if many are only needed for optional backends.
• Performance characteristics and scalability limits for large task graphs or high-frequency execution.
• Compatibility with async/await patterns or whether it is callback/synchronous-only.

Package facts

Evidence: taskflow-6.3.0-py3-none-any.whl

Tags

More Libraries packages

Further reading