prodigyopt

What it is and what it does

Prodigy is an optimizer for training neural networks in PyTorch that automatically estimates a suitable learning rate during training, removing the need to manually set this hyperparameter. It is based on research published in the paper 'Prodigy: An Expeditiously Adaptive Parameter-Free Learner' and implements an Adam-like algorithm with adaptive learning rate estimation. The optimizer supports weight decay (decoupled or standard L2 regularization), memory-efficient slicing via the `slice_p` parameter, and optional bias correction and warmup safeguards.

Typical usage involves instantiating the optimizer with a network's parameters and a default learning rate of 1.0, then optionally pairing it with a learning rate scheduler like cosine annealing. The package is designed to work out of the box for most training tasks, though the documentation provides tuning guidance for specialized use cases such as diffusion models, where specific settings like `safeguard_warmup=True` and `weight_decay=0.01` are recommended.

Use it for:

• Training standard neural networks (e.g., ResNets) when you want to avoid manual learning rate tuning.
• Fine-tuning models where adaptive learning rate estimation can reduce hyperparameter search overhead.
• Training diffusion models with the recommended settings for `safeguard_warmup`, `use_bias_correction`, and `weight_decay`.
• Memory-constrained training scenarios where `slice_p` can be set to values like 11 to reduce memory consumption.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Prodigy is a PyTorch optimizer that adapts the learning rate automatically during training, eliminating the need to manually tune this critical hyperparameter.

Yes, if you use PyTorch and want to reduce learning rate tuning effort. The optimizer has no external dependencies, is permissively licensed, and receives regular maintenance. However, verify that PyTorch is available in your environment, as it is not declared as a package dependency. The dormant maintenance status means bug fixes or feature additions are unlikely, but the core algorithm is stable and well-documented.

Install

prodigyopt on PyPI

pip install prodigyopt

uv add prodigyopt

poetry add prodigyopt

Installing prodigyopt

Before you install

Low install friction with no runtime dependencies. Maintenance is dormant—the last commit was 2025-01-16, but the repository remains active and the package receives regular downloads.

License in practice

MIT license permits free use, modification, and distribution with minimal restrictions, making it straightforward to integrate into commercial or open-source projects.

Quickstart

pip install prodigyopt

from prodigyopt import Prodigy

opt = Prodigy(net.parameters(), lr=1., weight_decay=0)

Requires PyTorch to be installed separately; the package itself does not declare it as a dependency.

Verify before relying

• Whether PyTorch is an undeclared peer dependency or if the package works without it installed.
• Performance comparison with other adaptive optimizers (Adam, AdamW) on standard benchmarks beyond the paper's experiments.

Package facts

Evidence: prodigyopt-1.1.2-py3-none-any.whl

Tags

More Artificial Intelligence packages

Further reading