---
id: torch-optimizer
version: "0.3.0"
license: Apache 2
license_treatment: permissive
maintenance: dormant
---

# torch-optimizer — pytorch-optimizer

License: permissive · Maintenance: dormant · Downloads: 166.5K/mo

## What it is and what it does

torch-optimizer is a collection of alternative optimization algorithms for PyTorch training, including AdaBound, RAdam, Lamb, DiffGrad, NovoGrad, MADGRAD, and many others. Each optimizer implements a different adaptive or momentum-based gradient descent variant, all exposing the same interface as PyTorch's standard optim module so they can be used as drop-in replacements.

You import the package and instantiate an optimizer by name, passing your model parameters and a learning rate, then call step() during training just as you would with standard optimizers. The package bundles research implementations of algorithms from academic papers, letting you experiment with different optimization strategies without implementing them yourself.

Use it for:

- Experimenting with alternative optimizers like RAdam or Lamb to improve convergence on your specific model and dataset
- Replacing standard Adam with AdaBound to get faster convergence with automatic learning rate scheduling
- Using DiffGrad or NovoGrad when you want gradient-based adaptive methods beyond PyTorch's built-in optimizers
- Prototyping with Ranger for potentially better generalization in deep learning
- Comparing multiple optimizers systematically by swapping them in and out with the same interface

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

Provides a collection of alternative optimization algorithms for PyTorch that are compatible with the standard optim module interface.

Yes, if you want to experiment with alternative optimizers and your PyTorch version is not significantly newer than late 2021. The package is dormant but functional, has no known vulnerabilities, and installs cleanly. Be aware that it may not be actively maintained for the latest PyTorch releases, so test compatibility with your environment first.

## Install

pip install torch-optimizer
uv add torch-optimizer
poetry add torch-optimizer

## Installing torch-optimizer

Before you install: Low install friction with a pure Python wheel. Maintenance is dormant—last release was October 2021 and last commit March 2024—but the package remains functional and the repository is not archived.

License in practice: Licensed under Apache 2 (permissive), so you can use it freely in commercial and private projects without copyleft obligations.

Quickstart:

pip install torch-optimizer

import torch_optimizer as optim

optimizer = optim.DiffGrad(model.parameters(), lr=0.001)
optimizer.step()

Verify before relying:

- Whether all listed optimizers remain well-maintained or have known issues in recent PyTorch versions
- Compatibility with PyTorch versions released after the package's last update in October 2021

## Package facts

- License: Apache 2 (permissive)
- Python support: supports_current
- Install friction: low
- Maintenance: dormant
- Downloads: 166.5K/month (top 15,000 on PyPI)
- Known vulnerabilities: none known

## Tags

pytorch optimizer alternatives, advanced gradient descent methods, radam lamb diffgrad pytorch, custom pytorch optimizers, adaptive learning rate algorithms, pytorch training optimization, novograd adabound madgrad, pytorch-training, optimization-algorithms

[View on SkillFed](https://skillfed.io/packages/torch-optimizer) · [View on PyPI](https://pypi.org/project/torch-optimizer/)