pytorch_optimizer

What it is and what it does

pytorch-optimizer bundles a large collection of optimizer implementations, learning rate schedulers, and loss functions for PyTorch, all accessible through a unified API. Instead of implementing or hunting down individual optimizer variants, you can import them by name or class and use them like standard PyTorch optimizers. The package includes both classic methods and recent research variants, with practical features like foreach support, Lookahead integration, and Gradient Centralization built in.

It's designed for practitioners who want to experiment with modern training techniques without rewriting boilerplate code. The package supports optional ecosystem integrations and has been tested and actively maintained. You can discover available components programmatically, load optimizers by name, or use a factory function to configure them with additional features in one call.

Use it for:

• Quickly experiment with different optimizer variants during model development without implementing each one from scratch
• Build training pipelines that support multiple optimizer choices through a consistent loader API
• Apply practical training enhancements like Lookahead or Gradient Centralization to any optimizer in the collection
• Access recent research optimizers (AdamP, DualAdam, etc.) alongside classic methods in a single package
• Integrate with optional ecosystem tools like bitsandbytes for quantized training when needed

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Provides a collection of modern optimizers, learning rate schedulers, and loss functions for PyTorch training with a consistent API.

Yes. The package has low install friction, active maintenance, a permissive license, no known vulnerabilities, and solves a real friction point in PyTorch training—accessing a broad range of optimizers and schedulers through a consistent API. Install it if you want to experiment with modern training methods or need access to research optimizer variants without implementing them yourself.

Install

pytorch-optimizer on PyPI

pip install pytorch-optimizer

uv add pytorch-optimizer

poetry add pytorch-optimizer

Installing pytorch_optimizer

Before you install

Low install friction with only numpy and torch as runtime dependencies. Actively maintained with a recent release.

License in practice

Apache-2.0 permissive license allows use in commercial and private projects with minimal restrictions.

Quickstart

pip install pytorch-optimizer

from pytorch_optimizer import AdamP

model = YourModel()
optimizer = AdamP(model.parameters(), lr=1e-3)

Requires Python >=3.8 and PyTorch >=1.10. Optional integrations (bitsandbytes, q-galore-torch, torchao) must be installed separately if needed.

Verify before relying

• Whether the 100+ optimizers, 10+ schedulers, and 10+ loss functions cover the specific training methods your project requires
• Performance characteristics and convergence behavior compared to using PyTorch's built-in optimizers directly
• Compatibility with your specific PyTorch version and hardware setup (CPU vs GPU)

Package facts

Evidence: pytorch_optimizer-3.10.1-py3-none-any.whl

Keywords: A2Grad, ADOPT, APOLLO, ASGD, AccSGD, AdEMAMix, AdaBelief, AdaBound, AdaDelta, AdaFactor, AdaGC, AdaHessian, AdaLomo, AdaMax, AdaMod, AdaMuon, AdaNorm, AdaPNM, AdaSmooth, Adai, Adalite, AdamG, AdamMini, AdamP, AdamS, Adan, AggMo, Aida, AliG, Alice, Amos, Apollo, AvaGrad, BCE, BCEFocal, BCOS, Bi-Tempered, CAME, Conda, DAdaptAdaGrad, DAdaptAdam, DAdaptAdan, DAdaptLion, DAdaptSGD, DeMo, Dice, DiffGrad, DualAdam, EXAdam, EmoFact, EmoLynx, EmoNavi, EmoNeco, EmoZeal, FAdam, FOCUS, FTRL, Fira, FlashAdamW, Focal, FocalCosine, FocalTversky, FriendlySAM, Fromage, GCSAM, GSAM, GaLore, Grams, Gravity, GrokFast, Jaccard, Kate, LARS, LDAM, LOMO, LaProp, Lamb, Lion, LoRARite, LookSAM, Lookahead, LovaszHinge, MADGRAD, MARS, MSVAG, Muno, Nero, NovoGrad, OrthoGrad, PAdam, PCGrad, PID, PNM, PSGD, Prodigy, QGaLore, QHAdam, QHM, RACS, RAdam, Ranger, Ranger21, RotoGrad, SAM, SCION, SGDP, SGDW, SM3, SOAP, SPAM, SPlus, SRMM, SWATS, ScalableShampoo, ScheduleFreeAdamW, ScheduleFreeRAdam, ScheduleFreeSGD, Shampoo, SignSGD, Simplified-AdEMAMix, SoftF1, SopihaH, StableAdamW, StableSPAM, TAM, TRAC, Tiger, Tversky, VSGD, WSAM, WSD, Yogi, bSAM, bitsandbytes, deep-learning, lr scheduler, optimizer, pytorch

Tags

More Software Development packages

Further reading