---
id: schedulefree
version: "1.4.1"
license: unclear
license_treatment: permissive
maintenance: active
---

# schedulefree — Schedule Free Learning in PyTorch

License: permissive · Maintenance: active · Downloads: 177.7K/mo

## What it is and what it does

Schedule-free learning replaces traditional learning rate schedules with a method that combines interpolation and averaging of optimizer iterates, eliminating the need to specify training duration in advance. The package provides PyTorch implementations of SGDScheduleFree, AdamWScheduleFree, and RAdamScheduleFree, each with a standard form and a reference form that trades memory for numerical stability. The core insight is that the optimizer maintains three sequences: one for gradient evaluation, one for parameter updates, and one for test/validation evaluation, with the averaging mechanism allowing the method to achieve fast early convergence without the instability of standard momentum-based approaches.

The method requires explicit calls to optimizer.train() and optimizer.eval() to switch between the gradient evaluation and validation sequences, and learning rates typically need to be larger than classical schedules. It works without a learning rate schedule but remains compatible with one, and supports warmup through a warmup_steps parameter. The approach is based on a research preprint and has active maintenance with community contributions.

Use it for:

- Training deep neural networks without manually tuning learning rate decay schedules or specifying total training steps upfront.
- Experimenting with different training durations without retuning the optimizer configuration for each run length.
- Wrapping existing PyTorch optimizers via ScheduleFreeWrapper to add schedule-free behavior to custom optimization setups.
- Research projects requiring comparison with Polyak-Ruppert averaging or Nesterov acceleration methods.
- Image classification tasks like MNIST where provided examples demonstrate the integration pattern.

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

Provides schedule-free optimizers for PyTorch that eliminate the need to specify stopping time in advance, replacing momentum with interpolation and averaging to achieve faster training without learning rate schedules.

Yes, if you want to eliminate learning rate schedule tuning and are willing to retune learning rate magnitude and the beta interpolation parameter. The low install friction, active maintenance, permissive license, and zero known vulnerabilities make it safe to try. Not worth installing if your workflow is heavily dependent on standard schedules or if you cannot modify optimizer.train()/eval() calls around your training loop.

## Install

pip install schedulefree
uv add schedulefree
poetry add schedulefree

## Installing schedulefree

Before you install: Low friction installation with only torch and typing-extensions as runtime dependencies. Active maintenance with recent commits and steady release cadence since initial release in April 2024.

License in practice: Permissive Apache license allows commercial and private use with minimal restrictions, making it suitable for most projects.

Quickstart:

pip install schedulefree

import torch
from schedulefree import AdamWScheduleFree

optimizer = AdamWScheduleFree(model.parameters())
for epoch in range(num_epochs):
    optimizer.train()
    # training loop
    optimizer.eval()
    # validation loop

Requires calling optimizer.train() and optimizer.eval() at the same points as model.train() and model.eval(); BatchNorm models need additional handling to update running statistics at the correct sequence point.

Verify before relying:

- Whether the experimental ScheduleFreeWrapper is production-ready or remains research-stage only.
- Performance gains relative to standard schedules on specific model architectures or domains.
- Compatibility with distributed training frameworks and gradient accumulation patterns.
- Specific learning rate multiplier guidance for different model types beyond the 10x-50x range for SGD and 1x-10x for AdamW.

## Package facts

- License: not declared (permissive)
- Python support: supports_current
- Install friction: low
- Maintenance: active
- Downloads: 177.7K/month (top 15,000 on PyPI)
- Known vulnerabilities: none known

## Tags

pytorch optimizer without schedule, schedule-free learning, sgd adamw optimizer pytorch, training without learning rate decay, adaptive optimizer pytorch, gradient descent without schedule, pytorch optimization algorithm, pytorch-optimizer, schedule-free-learning, gradient-descent

[View on SkillFed](https://skillfed.io/packages/schedulefree) · [View on PyPI](https://pypi.org/project/schedulefree/)