ranx

What it is and what it does

ranx is a Python library for evaluating and comparing rankings in information retrieval and recommender systems. It implements standard metrics like NDCG, MAP, MRR, precision, recall, and others, using Numba to accelerate vector operations and automatic parallelization. The library also provides statistical tests (paired t-test, Fisher's randomization test, Tukey's HSD) to determine whether differences between ranked results are significant, and can export results as LaTeX tables for scientific publications.

Beyond basic metrics, ranx includes fusion algorithms (CombMNZ, RRF, BayesFuse, and others) to combine multiple ranking runs, normalization strategies to standardize scores across runs, and automatic fusion optimization. It integrates with ir-datasets to load standard IR benchmarks (MSMARCO, etc.) and ranxhub to download and share pre-computed runs. The package is designed specifically for ranking evaluation—not classifier evaluation—and has been presented at ECIR 2022, CIKM 2022, and SIGIR 2023.

Use it for:

• Compute NDCG, MAP, and other metrics to evaluate a search engine or recommender system against ground-truth relevance judgments.
• Run statistical tests to determine whether one ranking algorithm significantly outperforms another.
• Combine multiple ranking runs using fusion algorithms and automatically optimize fusion weights.
• Load standard IR evaluation datasets and pre-computed runs from ranxhub to benchmark new models.
• Generate publication-ready LaTeX tables comparing multiple ranking systems and their metric scores.
• Normalize and compare ranking scores across different systems or datasets using built-in normalization strategies.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

ranx computes ranking evaluation metrics (precision, recall, NDCG, MAP, MRR, and others) for information retrieval and recommender systems, with Numba-accelerated computation and built-in statistical testing.

Yes, if you are evaluating information retrieval or recommender system rankings. ranx is a mature, well-cited library with no known vulnerabilities, permissive licensing, and low install friction. The 372-day gap since the last release suggests aging maintenance, but the repository is not archived and the package remains functional for its core use case. Install it if you need fast, standard-compliant ranking metrics and statistical testing; avoid it if you need active feature development or cutting-edge research implementations.

Install

ranx on PyPI

pip install ranx

uv add ranx

poetry add ranx

Installing ranx

Before you install

Low install friction with a pure-Python wheel distribution. Maintenance is aging—last release was 372 days ago—but the repository remains active and the package has accrued 692 stars.

License in practice

Licensed under permissive terms (MIT), so you can use it freely in commercial and open-source projects without copyleft obligations.

Quickstart

pip install ranx

from ranx import Qrels, Run, evaluate

qrels = Qrels.from_ir_datasets("msmarco-document/dev")
run = Run.from_ranxhub("run-id")
results = evaluate(qrels, run, metrics=["ndcg@10", "map"])

Requires Python 3.8 or later. Numba JIT compilation may add startup latency on first use.

Verify before relying

• Whether the 13 runtime dependencies (numpy, numba, pandas, scipy, etc.) are all required for basic metric computation or only for optional features like fusion and statistical tests.
• Performance characteristics and memory overhead when evaluating very large ranking datasets.
• Whether the package is actively maintained or in maintenance-only mode given the 372-day gap since the last release.

Package facts

Evidence: ranx-0.3.21-py3-none-any.whl

Keywords: trec_eval, information retrieval, recommender systems, evaluation, ranking, fusion, metasearch, numba

Tags

More General packages