torch-optimizer

What it is and what it does

torch-optimizer is a collection of alternative optimization algorithms for PyTorch training, including AdaBound, RAdam, Lamb, DiffGrad, NovoGrad, MADGRAD, and many others. Each optimizer implements a different adaptive or momentum-based gradient descent variant, all exposing the same interface as PyTorch's standard optim module so they can be used as drop-in replacements.

You import the package and instantiate an optimizer by name, passing your model parameters and a learning rate, then call step() during training just as you would with standard optimizers. The package bundles research implementations of algorithms from academic papers, letting you experiment with different optimization strategies without implementing them yourself.

Use it for:

• Experimenting with alternative optimizers like RAdam or Lamb to improve convergence on your specific model and dataset
• Replacing standard Adam with AdaBound to get faster convergence with automatic learning rate scheduling
• Using DiffGrad or NovoGrad when you want gradient-based adaptive methods beyond PyTorch's built-in optimizers
• Prototyping with Ranger for potentially better generalization in deep learning
• Comparing multiple optimizers systematically by swapping them in and out with the same interface

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Provides a collection of alternative optimization algorithms for PyTorch that are compatible with the standard optim module interface.

Yes, if you want to experiment with alternative optimizers and your PyTorch version is not significantly newer than late 2021. The package is dormant but functional, has no known vulnerabilities, and installs cleanly. Be aware that it may not be actively maintained for the latest PyTorch releases, so test compatibility with your environment first.

Install

torch-optimizer on PyPI

pip install torch-optimizer

uv add torch-optimizer

poetry add torch-optimizer

Installing torch-optimizer

Before you install

Low install friction with a pure Python wheel. Maintenance is dormant—last release was October 2021 and last commit March 2024—but the package remains functional and the repository is not archived.

License in practice

Licensed under Apache 2 (permissive), so you can use it freely in commercial and private projects without copyleft obligations.

Quickstart

pip install torch-optimizer

import torch_optimizer as optim

optimizer = optim.DiffGrad(model.parameters(), lr=0.001)
optimizer.step()

Verify before relying

• Whether all listed optimizers remain well-maintained or have known issues in recent PyTorch versions
• Compatibility with PyTorch versions released after the package's last update in October 2021

Package facts

Evidence: torch_optimizer-0.3.0-py3-none-any.whl

Keywords: torch-optimizer, pytorch, accsgd, adabound, adamod, diffgrad, lamb, lookahead, madgrad, novograd, pid, qhadam, qhm, radam, sgdw, yogi, ranger

Tags

More Artificial Intelligence packages

Further reading