---
id: fev
version: "0.9.0"
license: Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for… (full text in the JSON record)
license_treatment: permissive
maintenance: active
---

# fev — fev: Forecast evaluation library

License: permissive · Maintenance: active · Downloads: 120.1K/mo

## What it is and what it does

fev is a benchmarking framework designed to standardize time series forecasting evaluation. It sits between standalone datasets (which offer no reproducibility guarantees) and monolithic end-to-end systems (which bundle models, data, and tasks tightly). The library provides Task objects that wrap datasets from Hugging Face Hub, define evaluation windows and horizons, and compute metrics consistently across different forecasting models.

You define a forecasting task once—specifying dataset, horizon, covariates, and metrics—then iterate over rolling evaluation windows to collect predictions from your model. fev handles the metric computation and produces an evaluation summary that uniquely identifies the task and captures results. Multiple summaries can be aggregated into leaderboards for model comparison. The library is built on top of datasets and scipy, keeping dependencies minimal while supporting point and probabilistic forecasting.

Use it for:

- Benchmark your own forecasting model against standardized tasks to compare performance with other models.
- Create reproducible forecasting benchmarks that other researchers can run identically, ensuring comparable results.
- Aggregate results from multiple forecasting models into a leaderboard to rank them by skill score and win rate.
- Evaluate forecasting models on datasets stored on Hugging Face Hub without downloading data locally.
- Define custom forecasting tasks with specific horizons, covariates, and metrics for domain-specific evaluation.

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

fev is a lightweight benchmarking library for time series forecasting models that provides standardized evaluation workflows, reproducible task definitions, and metric computation without heavy dependencies.

Yes. fev is actively maintained, has low install friction, carries permissive licensing, and fills a genuine gap in forecasting evaluation tooling. It is well-suited for researchers and practitioners who need reproducible benchmarking without the overhead of monolithic systems. The recent release cadence and active repository signal ongoing support.

## Install

pip install fev
uv add fev
poetry add fev

## Installing fev

Before you install: Low install friction: pure Python wheel with only four runtime dependencies (datasets, numpy, pydantic, scipy). Active maintenance with recent releases; last commit 2026-08-14.

License in practice: Apache License 2.0 permits commercial and derivative use with attribution. No restrictions on bundling or modification.

Quickstart:

pip install fev

import fev

task = fev.Task(
    dataset_path="autogluon/chronos_datasets",
    dataset_config="m4_hourly",
    horizon=24,
)

for window in task.iter_windows():
    past_data, future_data = window.get_input_data()

task.evaluation_summary(predictions, model_name="my_model")

Requires Python >=3.10. Datasets are loaded from Hugging Face Hub; internet access needed for default usage.

Verify before relying:

- Whether adapters for popular forecasting libraries are production-ready or experimental.
- Performance characteristics when evaluating large-scale datasets or many models in sequence.
- Stability of the leaderboard infrastructure and submission process for contributed results.

## Package facts

- License: Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for… (full text in the JSON record) (permissive)
- Python support: supports_current
- Install friction: low
- Maintenance: active
- Downloads: 120.1K/month (top 15,000 on PyPI)
- Known vulnerabilities: none known

## Tags

time series forecasting benchmark, forecast evaluation framework, forecasting model comparison, reproducible forecasting tasks, time series evaluation metrics, forecasting leaderboard, benchmark time series models, time-series, benchmarking, forecasting

[View on SkillFed](https://skillfed.io/packages/fev) · [View on PyPI](https://pypi.org/project/fev/)