torcheval

What it is and what it does

TorchEval is a metrics library that abstracts away the complexity of computing model evaluation metrics across different training scenarios. It offers both functional interfaces for immediate computation and class-based interfaces that defer computation, the latter being especially useful for accumulating data across multiple batches or synchronizing metrics in distributed settings.

The library is designed to work on CPU, single GPU, and multi-process or multi-GPU configurations. Its main value lies in simplifying metric synchronization and computation in distributed training, where manual metric aggregation across ranks is error-prone. However, it is currently in pre-alpha (Development Status :: 2 - Pre-Alpha), meaning the API may change and backward compatibility is not guaranteed.

Use it for:

• Compute metrics during training loops with deferred computation across batches using class-based interfaces
• Synchronize and aggregate metrics across multiple GPUs or processes in distributed training without manual rank communication
• Evaluate model performance on CPU or GPU with a consistent interface that handles device placement automatically
• Build custom metrics by extending the class-based interface or using functional metric primitives
• Checkpoint and restore metric state during long training runs using the class-based metric API

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

TorchEval provides a collection of PyTorch model metrics with both functional and class-based interfaces, supporting single-process, multi-GPU, and distributed training scenarios.

Yes, with conditions. Install if you are training models in distributed settings and want to avoid manual metric synchronization logic, or if you prefer a unified metric interface across single and multi-GPU scenarios. Avoid if you require stable APIs and backward compatibility guarantees, as the package is pre-alpha and may introduce breaking changes. The low install friction and active maintenance are favorable, but verify that the available metrics match your evaluation needs.

Install

torcheval on PyPI

pip install torcheval

uv add torcheval

poetry add torcheval

Installing torcheval

Before you install

Low install friction with a single runtime dependency. The package is actively maintained with recent commits, though it remains in pre-alpha status with an API that may change and is not guaranteed to be backward compatible.

License in practice

BSD-3 licensed under a permissive license, allowing commercial and private use with minimal restrictions beyond attribution and liability disclaimers.

Quickstart

pip install torcheval

from torcheval.metrics import MulticlassAccuracy

metric = MulticlassAccuracy()
metric.update(outputs, target)
result = metric.compute()

Requires Python >= 3.7 and PyTorch >= 1.11

Verify before relying

• Whether the pre-alpha status and lack of stable release significantly impacts production readiness beyond API instability
• Performance characteristics and overhead of metric computation compared to manual implementations
• Completeness of the metric collection relative to common evaluation needs
• Whether typing-extensions dependency introduces any compatibility constraints

Package facts

Evidence: torcheval-0.0.7-py3-none-any.whl

Keywords: pytorch, evaluation, metrics

Tags

More Artificial Intelligence packages