hydra-submitit-launcher

What it is and what it does

Hydra-submitit-launcher is a plugin that bridges Hydra (a configuration management framework) with Submitit (a Slurm job submission tool). It allows you to define your application configuration in Hydra and then submit jobs to a Slurm cluster without writing cluster-specific code directly in your Python functions. The launcher handles job submission, monitoring, and result retrieval through Hydra's standard launcher interface.

The package is designed for researchers and engineers running parameter sweeps, hyperparameter searches, or batch computations on shared clusters. Instead of manually crafting Slurm submission scripts or calling Submitit directly, you configure your job parameters in Hydra's YAML files and let the launcher handle cluster interaction. It abstracts away Slurm details while preserving direct control over individual job logs and preemption handling.

Use it for:

• Run Hydra-configured machine learning experiments across multiple Slurm jobs with different hyperparameters.
• Submit batch data processing tasks from a Hydra application to a cluster without modifying core application code.
• Integrate parameter sweeps defined in Hydra config files directly to Slurm for large-scale research workflows.
• Switch between local testing and cluster execution by changing Hydra launcher configuration without code changes.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Integrates Hydra configuration framework with Submitit to submit Python functions as Slurm cluster jobs directly from configuration-driven applications.

Yes, if you are already using Hydra for configuration management and have access to a Slurm cluster. The integration is straightforward and the permissive license carries no restrictions. However, note that the package has not been updated since May 2022; verify compatibility with your versions of hydra-core and submitit before committing to production use, and monitor the repository for security or compatibility issues.

Install

hydra-submitit-launcher on PyPI

pip install hydra-submitit-launcher

uv add hydra-submitit-launcher

poetry add hydra-submitit-launcher

Installing hydra-submitit-launcher

Before you install

Low friction install with only two runtime dependencies (hydra-core and submitit). Package is aging—last release was 2022-05-17 with no updates in over 1550 days—but the repository remains active and unarchived.

License in practice

Released under MIT License (permissive), allowing commercial and private use with minimal restrictions.

Quickstart

pip install hydra-submitit-launcher

from hydra import compose, initialize
from hydra.core.config_store import ConfigStore

# Configure Hydra to use submitit launcher
cs = ConfigStore.instance()
cs.store(name="config", node={"hydra": {"launcher": {"_target_": "hydra_submitit_launcher.submitit_launcher.SlurmLauncher"}}})

Requires a Slurm cluster environment; local execution without Slurm will not work as intended.

Verify before relying

• Whether the package works with Python versions beyond 3.10 (classifiers list 3.7–3.10 but no explicit upper bound stated)
• Current compatibility with recent versions of hydra-core and submitit given the aging maintenance status

Package facts

Evidence: hydra_submitit_launcher-1.2.0-py3-none-any.whl

Tags

More Distributed Computing packages