torch-ema

What it is and what it does

torch-ema is a small PyTorch utility that maintains exponential moving averages of model parameters during training. It wraps your model's parameters and tracks a smoothed version of their values using a configurable decay rate, typically 0.995 or higher. The main use is to evaluate your model using these averaged weights—often producing better generalization—without disrupting the original training loop or optimizer state.

The library provides a context manager (`average_parameters()`) that temporarily swaps your model's weights with their EMA counterparts for validation, then restores the originals. It also supports manual store/copy/restore operations, device movement (GPU/CPU), and serialization via `state_dict()` for resuming training. Its only runtime dependency is torch, making installation straightforward.

Use it for:

• Evaluate a model on validation data using EMA-smoothed weights to measure generalization improvement without retraining
• Apply EMA during training to reduce overfitting by using averaged parameters at inference time while keeping original weights for continued optimization
• Transfer EMA weights to a second model for deployment or ensemble methods using the `copy_to()` method
• Checkpoint and resume training with EMA state preserved via `state_dict()` and `load_state_dict()`
• Experiment with different decay rates to tune the balance between tracking recent updates and long-term stability

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Computes exponential moving averages of PyTorch model parameters during training, allowing you to evaluate models using smoothed weights without modifying the original training state.

Yes, with conditions. Install if you need EMA for PyTorch model training and can verify the license from the GitHub repository. The package is lightweight, has no security vulnerabilities, and low install friction. However, maintenance is dormant (last release 2021-11-17); test compatibility with your PyTorch version before production use, and monitor the repository for any breaking changes in newer PyTorch releases.

Install

torch-ema on PyPI

pip install torch-ema

uv add torch-ema

poetry add torch-ema

Installing torch-ema

Before you install

Low friction: pure Python wheel with only torch as a runtime dependency. Maintenance is dormant—last release was 2021-11-17 and no updates since, though the repository remains active with recent commits and modest community engagement (445 stars).

License in practice

License treatment is unclear; no SPDX identifier or raw license text is available in the metadata. Verify the actual license from the GitHub repository before use in proprietary or restricted contexts.

Quickstart

pip install torch-ema

import torch
from torch_ema import ExponentialMovingAverage

model = torch.nn.Linear(10, 2)
optimizer = torch.optim.Adam(model.parameters())
ema = ExponentialMovingAverage(model.parameters(), decay=0.995)

# After each training step:
optimizer.step()
ema.update()

# Evaluate with EMA weights:
with ema.average_parameters():
    output = model(x_val)

Verify before relying

• Actual license terms and restrictions (metadata shows 'unclear' treatment with no SPDX or raw license text)
• Python version compatibility (requires_python is unspecified in metadata)
• Whether dormant status (last release 2021-11-17, no updates for years) affects compatibility with recent PyTorch versions

Package facts

Evidence: torch_ema-0.3-py3-none-any.whl

Keywords: pytorch, parameters, deep-learning

Tags

More Artificial Intelligence packages