---
id: ir-measures
version: "0.4.3"
license: unclear
license_treatment: permissive
maintenance: active
---

# ir-measures — Provides a common interface to many IR measure tools

License: permissive · Maintenance: active · Downloads: 162.5K/mo

## What it is and what it does

ir_measures is a Python library that standardizes how you compute information retrieval evaluation metrics across different underlying tools. Instead of learning multiple APIs or command-line syntaxes, you import measure objects (AP, nDCG, P@10, etc.) and call calc_aggregate() or iter_calc() with your qrels (relevance judgments) and run (ranked results). It accepts input in multiple formats—dicts, pandas DataFrames, namedtuples, or TREC-formatted files—and outputs per-query or aggregated metric values.

The package wraps pytrec-eval-terrier as its computation engine and is maintained by the Terrier Team at Glasgow. It also exposes a command-line interface for batch evaluation and integrates with PyTerrier for experiment workflows. Active maintenance, no known vulnerabilities, and permissive licensing make it a stable choice for IR research and evaluation pipelines.

Use it for:

- Evaluate a search or ranking system against standard TREC benchmarks using nDCG, AP, and precision metrics.
- Batch-compute metrics from TREC-formatted qrels and run files via command line without writing Python.
- Integrate metric computation into PyTerrier experiments to compare retrieval algorithms.
- Load qrels from ir_datasets and compute per-query and aggregate metrics programmatically.
- Prototype custom ranking evaluation with flexible input formats (dict, DataFrame, or iterables).

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

Provides a unified Python interface to compute standard information retrieval evaluation metrics (AP, nDCG, P@k, RR, etc.) from qrels and ranked runs, with both programmatic and command-line APIs.

Yes. ir_measures is actively maintained, has no security vulnerabilities, installs with low friction, and provides a clean, unified interface to a core task in IR research and evaluation. Use it if you need to evaluate ranked results against relevance judgments.

## Install

pip install ir-measures
uv add ir-measures
poetry add ir-measures

## Installing ir-measures

Before you install: Low friction install with a single runtime dependency (pytrec-eval-terrier). The package is actively maintained with a recent release and no known vulnerabilities.

License in practice: Licensed under Apache 2.0 (permissive), allowing use in commercial and private projects with minimal restrictions.

Quickstart:

pip install ir-measures

import ir_measures
from ir_measures import AP, nDCG, P

qrels = {'Q0': {'D0': 0, 'D1': 1}}
run = {'Q0': {'D0': 1.2, 'D1': 1.0}}

results = ir_measures.calc_aggregate([AP, nDCG, P@10], qrels, run)
print(results)

Requires Python 3.9 or later.

Verify before relying:

- Whether pytrec-eval-terrier requires system libraries or compilation at install time.
- Performance characteristics when evaluating large-scale runs (thousands of queries).
- Support for custom or user-defined metrics beyond the built-in set.

## Package facts

- License: not declared (permissive)
- Python support: supports_current
- Install friction: low
- Maintenance: active
- Downloads: 162.5K/month (top 15,000 on PyPI)
- Known vulnerabilities: none known

## Tags

IR evaluation metrics, information retrieval assessment, nDCG AP precision calculation, TREC qrels evaluation, ranking quality measurement, search result evaluation, retrieval performance metrics, information-retrieval, evaluation-metrics, trec-benchmark

[View on SkillFed](https://skillfed.io/packages/ir-measures) · [View on PyPI](https://pypi.org/project/ir-measures/)