torchrunx

What it is and what it does

torchrunx is a functional launcher for PyTorch distributed training that replaces CLI-based tools. Instead of shell commands, you write a single Python function and pass it to a Launcher object, which handles environment setup, rank assignment, and result collection across devices. It depends on torch, numpy, cloudpickle for serialization, and fabric for SSH communication.

The package is designed for multi-GPU and multi-machine setups, requiring Linux and SSH with a shared filesystem for multi-machine scenarios. It integrates with standard PyTorch distributed patterns (DistributedDataParallel, environment variables like RANK and LOCAL_RANK) and supports frameworks like Transformers, DeepSpeed, PyTorch Lightning, and Accelerate. Results from each rank are collected and returned to the caller, enabling complex workflows within a single script.

Use it for:

• Launch distributed training across multiple GPUs on a single machine without writing shell scripts
• Coordinate multi-machine training with automatic SSH setup and result aggregation
• Integrate distributed training into larger Python workflows that need to inspect or process per-rank outputs
• Fine-tune large language models using Transformers or DeepSpeed with functional control
• Run single-GPU training with automatic environment variable setup for compatibility

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

torchrunx distributes PyTorch training functions across multiple GPUs or machines using a functional API, eliminating the need for CLI-based launchers.

Yes, with conditions. Install if you need functional, in-script control over distributed PyTorch training and are comfortable with GPLv3 copyleft terms. The low install friction and active maintenance are positive signals, but the young codebase and modest adoption mean it lacks the battle-testing of established alternatives. Best for teams already using GPLv3 or open-source projects; avoid if your codebase must remain proprietary.

Install

torchrunx on PyPI

pip install torchrunx

uv add torchrunx

poetry add torchrunx

Installing torchrunx

Before you install

Low install friction with a pure-Python wheel. Actively maintained with a recent commit on 2026-08-04 and only 10 days since the latest release, though the project is young with a modest star count of 60.

License in practice

Licensed under GNU General Public License Version 3, a copyleft license. Any derivative work or distribution must also be released under GPLv3 and include source code; proprietary use or closed-source distribution is not permitted.

Quickstart

pip install torchrunx

import torchrunx
import torch

def train_fn(output_dir: str) -> str:
    rank = int(__import__('os').environ['RANK'])
    if rank == 0:
        return output_dir
    return None

launcher = torchrunx.Launcher(hostnames=["localhost"], workers_per_host="gpu")
results = launcher.run(train_fn, output_dir="outputs")
checkpoint = results.rank(0)

Requires Linux; multi-machine use requires SSH access and a shared filesystem between hosts.

Verify before relying

• Whether the package works on Windows or macOS, or is truly Linux-only as stated
• Performance overhead compared to native torchrun or accelerate launch
• Maturity and stability guarantees given the first release in 2024-07-13

Package facts

Evidence: torchrunx-0.4.0-py3-none-any.whl

Tags

More Distributed Computing packages