---
id: pytrec-eval
version: "0.5"
license: unclear
license_treatment: permissive
maintenance: dormant
---

# pytrec-eval — Provides Python bindings for popular Information Retrieval measures implemented within trec_eval.

License: permissive · Maintenance: dormant · Downloads: 96.2K/mo

## What it is and what it does

pytrec_eval is a Python wrapper around TREC's trec_eval tool, a standard reference implementation for computing Information Retrieval evaluation metrics. It lets you compute measures like MAP (Mean Average Precision) and NDCG (Normalized Discounted Cumulative Gain) by passing relevance judgments and ranked results as Python dictionaries, then get back computed scores for each query.

The package is designed to eliminate the need for custom metric implementations in Python research code. It wraps the compiled trec_eval engine via C++ bindings, so it inherits the correctness and performance of the reference implementation. Installation requires a C++ compiler and Python development headers; the package has been dormant since 2020 but carries no known security issues.

Use it for:

- Evaluate information retrieval systems in Python research pipelines without reimplementing standard metrics.
- Compute ranking quality metrics (MAP, NDCG, etc.) for search or recommendation system experiments.
- Perform statistical significance testing between two ranked result sets using standard IR measures.
- Validate IR models by scoring ranked results against ground-truth relevance judgments in batch.

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

Provides Python bindings to compute standard Information Retrieval evaluation measures (MAP, NDCG, and others) against relevance judgments, wrapping the TREC trec_eval tool.

Yes, if you need standard IR evaluation metrics in Python and can meet the C++ build requirements. The package is dormant but stable, carries no known vulnerabilities, and wraps a trusted reference implementation. Install only if you're comfortable with a package that hasn't been updated since 2020 and verify compatibility with your Python version before relying on it in production.

## Install

pip install pytrec-eval
uv add pytrec-eval
poetry add pytrec-eval

## Installing pytrec-eval

Before you install: Installation requires a C++ compiler and Python development headers; the package is dormant (last release 2020-09-07, last commit 2023-10-10) but carries no known vulnerabilities and has accumulated 350 repository stars.

License in practice: Licensed under the MIT license (permissive), allowing commercial and private use; note that the underlying trec_eval tool is licensed separately.

Quickstart:

pip install pytrec_eval

import pytrec_eval

qrel = {'q1': {'d1': 0, 'd2': 1}}
run = {'q1': {'d1': 1.0, 'd2': 0.0}}
evaluator = pytrec_eval.RelevanceEvaluator(qrel, {'map', 'ndcg'})
results = evaluator.evaluate(run)

Requires a C++ compiler and Python development headers; developed for Python 3.5 and requires Python >= 3.

Verify before relying:

- Whether numpy and scipy are runtime dependencies or only build-time requirements (description lists them but fact sheet shows n_runtime: 0).
- Current compatibility with modern Python versions beyond the 3.5 development baseline.
- Whether the package is actively maintained or if dormancy signals end-of-life.

## Package facts

- License: not declared (permissive)
- Python support: supports_current
- Install friction: high
- Maintenance: dormant
- Downloads: 96.2K/month (top 15,000 on PyPI)
- Known vulnerabilities: none known

## Tags

information retrieval evaluation, trec eval python, ranking metrics computation, ir evaluation measures, ndcg map calculation, relevance judgment scoring, information-retrieval, evaluation-metrics, research-tool

[View on SkillFed](https://skillfed.io/packages/pytrec-eval) · [View on PyPI](https://pypi.org/project/pytrec-eval/)