datasets

HuggingFace community-driven open-source library of datasets

datasets Permissive license Apache 2.0 Active ★ 21,836 v5.0.1 released 2026-07-28

Install

datasets on PyPI

pip

pip install datasets

uv

uv add datasets

poetry

poetry add datasets

Package facts

License	Apache 2.0 (permissive)
Python support	supports the current Python release (>=3.10.0)
Install friction	low — pure-Python wheel
Runtime dependencies	14 — filelock, numpy, pyarrow, dill, pandas, requests, httpx, tqdm, xxhash, multiprocess, fsspec, huggingface-hub, packaging, pyyaml
Maintenance	actively maintained — 16 days since the last release
Last repo commit	2026-08-12
First released	2015-08-18
Popularity	one of the top 1,000 most-downloaded packages on PyPI (30-day window, as of 2026-08-13)
Known vulnerabilities	none known (OSV.dev, checked 2026-08-13)

Evidence: datasets-5.0.1-py3-none-any.whl

Keywords: datasets, machine, learning, datasets

Development Status :: 5 - Production/StableIntended Audience :: DevelopersIntended Audience :: EducationIntended Audience :: Science/ResearchLicense :: OSI Approved :: Apache Software LicenseOperating System :: OS IndependentProgramming Language :: Python :: 3Programming Language :: Python :: 3.10Programming Language :: Python :: 3.11Programming Language :: Python :: 3.12Programming Language :: Python :: 3.13Programming Language :: Python :: 3.14Topic :: Scientific/Engineering :: Artificial Intelligence

About datasets

from the package's own PyPI description — quoted content, verbatim

<a href="https://github.com/huggingface/datasets/actions/workflows/ci.yml?query=branch%3Amain"><img alt="Build" src="https://github.com/huggingface/datasets/actions/workflows/ci.yml/badge.svg?branch=main"></a> <a href="https://github.com/huggingface/datasets/blob/main/LICENSE"><img alt="GitHub" src="https://img.shields.io/github/license/huggingface/datasets.svg?color=blue"></a> <a href="https://huggingface.co/docs/datasets/index.html"><img alt="Documentation"...

Read as markdown · JSON record · Source repository · Homepage

AI interpretation — verify before relying

AI-generated interpretation of the package facts above; every digit, version, license, or vulnerability id it cites is grounded in the facts already shown on this page

Datasets provides one-line loading and efficient preprocessing of thousands of public datasets from the Hugging Face Hub, with native support for multiple formats (CSV, JSON, Parquet, Arrow, etc.) and multi-modal data (text, audio, image, video, PDF, medical imaging).

Low friction: pure Python wheel with 14 well-established runtime dependencies including numpy, pyarrow, pandas, and requests. Actively maintained with last commit 2026-08-12 and strong community backing.

Apache 2.0 permissive license allows commercial and private use with minimal restrictions; you must include a copy of the license and state significant changes, but derivative works and redistribution are permitted.

Usage

pip install datasets

from datasets import load_dataset

squad = load_dataset('rajpurkar/squad')
processed = squad.map(lambda x: {'length': len(x['context'])})

Requires Python >=3.10.0; datasets are fetched from Hugging Face Hub (network access needed for first load).

Verdict: Production-stable, actively maintained library with zero known vulnerabilities and permissive Apache 2.0 licensing. Low install friction and broad format/modality support make it the standard choice for dataset loading in ML workflows. Active development and recent releases confirm ongoing support.

Needs verification

Whether streaming mode achieves claimed performance improvements with Xet backend in typical use cases
Performance characteristics and memory overhead when processing very large datasets with multiprocessing
Actual compatibility with Python 3.13 and 3.14 beyond classifier declarations

load public datasets machine learninghugging face dataset librarydownload preprocess datasets pythonmulti-modal data loadingstreaming dataset without downloaddataset preprocessing pipelinearrow-backed dataset caching