---
id: lion-pytorch
version: "0.2.5"
license: MIT License Copyright (c) 2023 Phil Wang Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in… (full text in the JSON record)
license_treatment: permissive
maintenance: active
---

# lion-pytorch — Lion Optimizer - Pytorch

License: permissive · Maintenance: active · Downloads: 159.3K/mo

## What it is and what it does

Lion is a PyTorch optimizer implementing an evolved sign momentum algorithm that the description positions as a potential successor to AdamW. It depends only on torch and integrates directly into PyTorch's optimizer interface. The optimizer requires careful hyperparameter tuning: learning rates should typically be 3–10x smaller than AdamW equivalents, weight decay values 3–10x larger, and default β1 and β2 values differ from AdamW (0.9 and 0.99 versus 0.9 and 0.999). The description documents mixed empirical results—positive outcomes reported for language modeling and text-to-image training when tuned correctly, but negative results outside the paper's tested domains (reinforcement learning, feedforward networks, hybrid architectures). The author recommends Lion primarily for high batch sizes (64 or above) and notes sensitivity to batch size, data volume, and augmentation.

Use it for:

- Training large language models where a 3x smaller learning rate than AdamW yields better convergence
- Text-to-image model training when hyperparameters are carefully tuned to the specific architecture
- Vision transformer training with cosine decay learning rate schedules
- Scenarios with high batch sizes (64+) where Lion's sign momentum approach may outperform adaptive methods

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

Lion is a PyTorch optimizer that implements an evolved sign momentum algorithm discovered by Google Brain as an alternative to AdamW for training neural networks.

Yes, with conditions. Lion is permissively licensed, has low install friction, and is actively maintained. Install it if you are training models in domains the paper evaluated (language modeling, vision transformers, text-to-image) and are willing to invest in hyperparameter tuning. The description's own updates acknowledge it performs worse than Adam without careful learning rate adjustment and shows negative results outside tested architectures. Not recommended for reinforcement learning, standard feedforward networks, or exploratory work where you cannot afford tuning overhead.

## Install

pip install lion-pytorch
uv add lion-pytorch
poetry add lion-pytorch

## Installing lion-pytorch

Before you install: Low install friction with a single runtime dependency on torch. Actively maintained with a recent release 36 days ago.

License in practice: MIT License permits unrestricted use, modification, and distribution in both open-source and commercial projects with minimal restrictions.

Quickstart:

pip install lion-pytorch

import torch
from torch import nn
from lion_pytorch import Lion

model = nn.Linear(10, 1)
opt = Lion(model.parameters(), lr=1e-4, weight_decay=1e-2)

loss = model(torch.randn(10))
loss.backward()
opt.step()
opt.zero_grad()

Requires Python 3.9 or later. Learning rate and weight decay require careful tuning (typically 3–10x smaller/larger than AdamW); default hyperparameters may not work without adjustment.

Verify before relying:

- Whether Lion's performance gains over AdamW hold across a broad range of model architectures and training regimes beyond those tested in the original paper
- Optimal learning rate schedules and hyperparameter tuning strategies for specific problem domains
- Stability and convergence behavior with batch sizes below 64, given the author's recommendation for high batch sizes

## Package facts

- License: MIT License Copyright (c) 2023 Phil Wang Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal in… (full text in the JSON record) (permissive)
- Python support: supports_current
- Install friction: low
- Maintenance: active
- Downloads: 159.3K/month (top 15,000 on PyPI)
- Known vulnerabilities: none known

## Tags

pytorch optimizer alternative to adam, lion optimizer pytorch, evolved sign momentum optimizer, deep learning optimizer, neural network training optimizer, optimizer, pytorch, training

[View on SkillFed](https://skillfed.io/packages/lion-pytorch) · [View on PyPI](https://pypi.org/project/lion-pytorch/)