ir-datasets
provides a common interface to many IR ad-hoc ranking benchmarks, training datasets, etc.
Decision gist · record as of 2026-08-14
Yes. ir_datasets is actively maintained, has no known vulnerabilities, and solves a real problem for IR researchers and practitioners—centralizing access to fragmented benchmark datasets. The low install friction and permissive MIT license make adoption straightforward. Install it if you work with information retrieval datasets or need to prototype ranking models against standard benchmarks.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires Python 3.8 or later.
- First use of a dataset will download and cache source files, which may take time and disk space depending on dataset size.
- Low install friction with six common runtime dependencies.
License · maintenance · safety
permissive license (permissive) — Licensed under MIT (permissive), allowing free use, modification, and distribution with minimal restrictions.
last release 2026-07-18 (27 days) · last repo commit 2026-07-18 · 8 stars
0 known vulnerabilities (OSV.dev, 2026-08-14) · 752,161 downloads/mo, #5,153 on PyPI
Alternatives
Verify before relying
pip install ir_datasets
import ir_datasets
dataset = ir_datasets.load('msmarco-passage/train')
for doc in dataset.docs_iter():
print(doc.doc_id, doc.text)- Whether the package handles automatic retries or resumable downloads for large dataset transfers
- Memory requirements and performance characteristics for the largest datasets mentioned (e.g., ClueWeb with ~1B documents)
What it is and what it does
ir_datasets is a Python package that centralizes access to information retrieval benchmarks and training datasets used in academic research and industry. It abstracts away the complexity of locating, downloading, and parsing datasets from multiple sources—handling format differences, encoding issues, and file extraction automatically. The package provides both a Python API and command-line interface, allowing you to iterate over documents, queries, and relevance judgments in a consistent format.
The package handles datasets ranging from small benchmarks to massive collections like ClueWeb (approximately 1 billion documents) by using iterators that avoid loading entire datasets into memory. It also provides fast random-access lookups through document stores, supports slicing operations for parallel processing, and automatically fixes known data quality issues in popular datasets like MS-MARCO. When datasets are not publicly available, it provides instructions on how to obtain them.
Use it for
- Build and evaluate information retrieval models using standard benchmarks like TREC, MS-MARCO, or BEIR without manually downloading and parsing each dataset.
- Process large document collections in parallel by using efficient iter slicing to partition data across multiple devices or workers.
- Quickly prototype ranking algorithms by accessing queries, documents, and relevance judgments through a unified iterator interface.
- Access multilingual IR datasets (e.g., HC4, CLIRMatrix) for cross-lingual retrieval research without managing separate download and format conversion steps.
- Build dataset-agnostic evaluation pipelines that work across multiple benchmarks by relying on the common interface and automatic data normalization.
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes.
ir_datasets is actively maintained, has no known vulnerabilities, and solves a real problem for IR researchers and practitioners—centralizing access to fragmented benchmark datasets. The low install friction and permissive MIT license make adoption straightforward. Install it if you work with information retrieval datasets or need to prototype ranking models against standard benchmarks.
Install
ir-datasets on PyPI
Before you install
Low install friction with six common runtime dependencies. Actively maintained with a release within the last month and no known vulnerabilities.
Requires Python 3.8 or later. First use of a dataset will download and cache source files, which may take time and disk space depending on dataset size.
License in practice
Licensed under MIT (permissive), allowing free use, modification, and distribution with minimal restrictions.
Quickstart
pip install ir_datasets
import ir_datasets
dataset = ir_datasets.load('msmarco-passage/train')
for doc in dataset.docs_iter():
print(doc.doc_id, doc.text)Verify before relying
- Whether the package handles automatic retries or resumable downloads for large dataset transfers
- Memory requirements and performance characteristics for the largest datasets mentioned (e.g., ClueWeb with ~1B documents)
Package facts
| License | permissive license permissive |
| Python support | Supports the current Python release >=3.8 |
| Install friction | Low. Pure-Python wheel |
| Runtime dependencies | 6 packageslxmlnumpypyyamlrequeststqdmlz4 |
| Maintenance | Actively maintained 27 days since the last release |
| Last repo commit | |
| First released | |
| Downloads | 752,161 / month, #5,153 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
| Classifiers | License :: OSI Approved :: MIT LicenseOperating System :: OS IndependentProgramming Language :: PythonTopic :: Text ProcessingTopic :: Text Processing :: Indexing |
Evidence: ir_datasets-0.6.3-py3-none-any.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “ir dataset loading”
- ir-datasetsProvides a unified Python interface to download, parse, and iterate…
- trec-car-toolsProvides Python and Java bindings to read TREC Complex Answer…
- mlcroissantValidates and loads datasets described in the MLCommons Croissant…
Give your agent the search over MCP, or paste the wish link into any chat.
More Text Processing packages
A drop-in replacement for Python's standard `re` module that adds advanced regex features like nested sets, fuzzy matching, lookaround in conditionals, and full Unicode case-folding while maintaining backward compatibility.
pyparsing provides a library for building text parsers directly in Python code using composable grammar classes, handling quoted strings, whitespace variation, and embedded comments without regex or lex/yacc.
Install it if you need to parse text or define grammars programmatically.
fonttools manipulates font files in multiple formats (TrueType, OpenType, AFM, Type 1, Mac-specific) and includes TTX, a tool to convert fonts to and from XML text format.
Install it if you need to read, write, or manipulate fonts programmatically or via the TTX command-line tool.
Docutils converts plaintext documentation in reStructuredText format into multiple output formats including HTML, XML, and LaTeX using a modular processing system.
RapidFuzz provides fast fuzzy string matching using Levenshtein Distance and related metrics, implemented mostly in C++ with Python bindings for rapid similarity scoring and approximate string matching.
Install it if you need fuzzy string matching; it's a solid replacement for FuzzyWuzzy with better licensing and performance.
tinycss2 parses CSS strings into token and block objects, and generates CSS strings from those objects, following the CSS Syntax Level 3 specification without enforcing specific properties or values.
Install it if your project requires CSS tokenization or syntax manipulation.
See also ir-measures · trec-car-tools · pytrec-eval · datasets · pytrec-eval-terrier · tfds-nightly · tensorflow-datasets · cinemagoer · arckit · rdata