---
id: pytrec-eval-terrier
version: "0.5.10"
license: unclear
license_treatment: permissive
maintenance: active
---

# pytrec-eval-terrier — Provides Python bindings for popular Information Retrieval measures implemented within trec_eval.

License: permissive · Maintenance: active · Downloads: 1.2M/mo

## What it is and what it does

pytrec_eval_terrier is a Python wrapper around TREC's trec_eval evaluation tool, designed to eliminate custom implementations of Information Retrieval metrics in Python. It exposes a simple API for computing standard IR measures—such as MAP (Mean Average Precision) and NDCG (Normalized Discounted Cumulative Gain)—on ranked search results against relevance judgments. The package accepts query-document relevance labels and ranked runs as nested dictionaries, then returns computed metrics per query.

This fork, maintained by the University of Glasgow, provides pre-built wheels for modern Python versions (3.10–3.14) and multiple platforms (Linux, macOS, Windows), reducing installation friction compared to building from source. It depends on numpy and scipy for numerical operations and wraps C++ code from the original trec_eval project, making it both fast and reliable for research and production IR evaluation workflows.

Use it for:

- Evaluate search engine or ranking model performance using standard TREC metrics without writing custom metric code.
- Compute statistical significance between two ranked runs to compare IR system improvements.
- Benchmark information retrieval systems in research papers or academic projects.
- Integrate IR evaluation into automated testing or continuous evaluation pipelines.
- Validate ranking quality during development of search applications or recommendation systems.

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

Provides Python bindings to TREC's trec_eval tool for computing standard Information Retrieval evaluation measures like MAP and NDCG on ranked search results.

Yes. The package is actively maintained, has no known vulnerabilities, installs cleanly on modern Python versions via pre-built wheels, and solves a real problem—standardized IR evaluation—with a proven, well-cited implementation. Use it if you need to evaluate ranked search results against relevance judgments; the MIT license and permissive treatment pose no barrier.

## Install

pip install pytrec-eval-terrier
uv add pytrec-eval-terrier
poetry add pytrec-eval-terrier

## Installing pytrec-eval-terrier

Before you install: Medium install friction due to compiled C++ components; pre-built wheels available for Python 3.10–3.14 on Linux, macOS, and Windows. Actively maintained with recent releases. Depends on numpy and scipy.

License in practice: Licensed under the MIT license (permissive), allowing broad reuse. Note that the underlying trec_eval tool is licensed separately; check its terms if you modify or redistribute.

Quickstart:

pip install pytrec_eval_terrier

import pytrec_eval

qrel = {'q1': {'d1': 0, 'd2': 1}}
run = {'q1': {'d1': 1.0, 'd2': 0.0}}
evaluator = pytrec_eval.RelevanceEvaluator(qrel, {'map', 'ndcg'})
print(evaluator.evaluate(run))

Requires Python 3 or later; numpy and scipy must be installed.

Verify before relying:

- Whether the fork maintains full API compatibility with the original pytrec_eval project.
- Performance characteristics when evaluating large-scale ranking runs.
- Availability of documentation beyond the GitHub README and examples.

## Package facts

- License: not declared (permissive)
- Python support: supports_current
- Install friction: medium
- Maintenance: active
- Downloads: 1.2M/month (top 5,000 on PyPI)
- Known vulnerabilities: none known

## Tags

information retrieval evaluation, trec eval python, ranking metrics MAP NDCG, search result evaluation, ir evaluation measures, relevance assessment, ranking quality metrics, information-retrieval, evaluation-metrics, trec

[View on SkillFed](https://skillfed.io/packages/pytrec-eval-terrier) · [View on PyPI](https://pypi.org/project/pytrec-eval-terrier/)