b2luigi

What it is and what it does

b2luigi is a wrapper around the luigi workflow orchestration library that makes it easier to schedule and run batch tasks on local machines or distributed queue systems. It inherits luigi's powerful dependency management system—which automatically resolves task dependencies and parallelizes execution—but adds a more user-friendly interface and built-in support for common queue systems like LSF and HTCondor. The package is maintained by the Belle II Collaboration and is used in production for scientific computing workflows.

You use b2luigi by defining Task subclasses with input, output, and run methods, then letting the scheduler handle dependency resolution and execution. It works with Python 3.8 through 3.11 and depends on nine runtime packages including luigi, GitPython, jinja2, and webdavclient3. The copyleft GPLv3+ license means any derivative work must also be open source.

Use it for:

• Schedule scientific data processing pipelines across a compute cluster with automatic dependency tracking.
• Manage complex multi-stage batch jobs on LSF or HTCondor queue systems with a Python-native interface.
• Build reproducible workflows where task outputs feed into downstream tasks without manual orchestration.
• Integrate version control (via GitPython) and templating (via jinja2) into batch task definitions.
• Monitor and retry failed tasks across distributed systems with built-in tenacity-based resilience.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

b2luigi wraps luigi to simplify scheduling and running batch tasks locally or on queue systems like LSF and HTCondor, adding a user-friendly interface on top of luigi's dependency management.

Yes, if you need to schedule batch tasks on LSF or HTCondor and want a Python-native workflow tool with luigi's dependency management. The active maintenance, low install friction, and production use in Belle II make it reliable. The GPLv3+ license is a blocker only for proprietary software; open-source and internal projects are fine.

Install

b2luigi on PyPI

pip install b2luigi

uv add b2luigi

poetry add b2luigi

Installing b2luigi

Before you install

Low friction: pure Python wheel with nine runtime dependencies (luigi, parse, GitPython, colorama, cachetools, jinja2, setuptools, tenacity, webdavclient3). Active maintenance with a recent commit on 2026-08-13 and a release on 2026-04-17.

License in practice

GPLv3+ copyleft license: any code that links or distributes b2luigi must be released under a compatible open-source license. Acceptable for internal tools and open-source projects, but not for proprietary software.

Quickstart

pip install b2luigi

import luigi
from b2luigi import Task

class MyTask(Task):
    def output(self):
        return luigi.LocalTarget('output.txt')
    def run(self):
        with self.output().open('w') as f:
            f.write('hello')

if __name__ == '__main__':
    luigi.build([MyTask()], local_scheduler=True)

Requires Python 3.8 or later. Queue system integration (LSF, HTCondor) requires those systems to be installed and configured separately.

Verify before relying

• Whether webdavclient3 is used for remote file staging or is an optional dependency.
• Specific queue systems supported beyond LSF and HTCondor mentioned in the description.
• Whether the package works with recent versions of luigi (no version constraint specified in fact sheet).

Package facts

Evidence: b2luigi-1.2.9-py3-none-any.whl

Keywords: luigi, workflow management, belle2

Tags

More Build Tools packages