taskcluster-taskgraph

What it is and what it does

Taskgraph is a Python library that generates and submits task graphs to Taskcluster, a generic task execution platform. It bridges declarative YAML-based task definitions with programmatic Python transforms, allowing CI systems to scale from simple configurations to complex, large-scale pipelines. The library powers Firefox's CI, which runs over 30,000 tasks.

The workflow is straightforward: a decision task invokes taskgraph, which reads YAML task definitions, applies Python-based transforms to alter or clone tasks, optionally removes unnecessary tasks through optimization, and submits the resulting graph to Taskcluster via its API. This combination of declarative configuration and programmatic control is what enables taskgraph to handle CI systems of any complexity.

Use it for:

• Scaling CI from a single .taskcluster.yml file to a multi-thousand-task pipeline with programmatic task generation.
• Applying conditional logic and transforms to task definitions based on repository state, branch, or commit metadata.
• Optimizing task graphs by removing redundant or skipped tasks before submission to reduce CI costs.
• Managing complex Firefox-like CI systems with tens of thousands of interdependent tasks.
• Automating task cloning and parameterization across multiple platforms, architectures, or test suites.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Taskgraph generates and submits task graphs to Taskcluster CI by combining declarative YAML task definitions with Python-based transforms and optimization logic.

Yes, if you are using Taskcluster and need to manage CI beyond a single configuration file. The library is production-stable, actively maintained, has no known vulnerabilities, and is proven at scale in Firefox's CI. The copyleft license (MPL 2.0) is a consideration only if you plan to distribute modified versions. Install friction is low.

Install

taskcluster-taskgraph on PyPI

pip install taskcluster-taskgraph

uv add taskcluster-taskgraph

poetry add taskcluster-taskgraph

Installing taskcluster-taskgraph

Before you install

Low install friction with a pure-Python wheel. Actively maintained with a recent release (16 days old) and an active repository. Requires Python 3.9 or later.

License in practice

Licensed under Mozilla Public License 2.0 (MPL 2.0), a copyleft license. You must disclose source code modifications if you distribute the package, though internal use has no restrictions.

Quickstart

pip install taskcluster-taskgraph

from taskgraph import generate

task_graph = generate.generate_graph()

Requires Python 3.9 or later. Taskgraph is designed to run as a decision task within Taskcluster; standalone use requires a Taskcluster instance and appropriate API credentials.

Verify before relying

• Whether the package can be used outside a Taskcluster environment for local task graph generation and testing.
• Performance characteristics and scalability limits for task graphs beyond Firefox's 30,000+ task scale.
• Detailed API stability guarantees across minor version releases.

Package facts

Evidence: taskcluster_taskgraph-24.2.3-py3-none-any.whl

Tags

More Software Development packages