torchtext
Text utilities, models, transforms, and datasets for PyTorch.
Decision gist · record as of 2026-08-14
Yes, but with caution. torchtext remains useful for standard NLP tasks and dataset loading, with no known security vulnerabilities and permissive licensing. However, the abandoned maintenance status (paused since September 2023, archived repository) means no new features or dataset support will be added. Install it if you need its specific datasets or models for existing workflows, but do not expect updates for modern NLP techniques or emerging benchmarks. For new projects, consider actively maintained alternatives.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires torch to be installed first; optional dependencies (spacy, sacremoses, sentencepiece) needed for specific tokenizers.
- Medium install friction due to compiled dependencies (torch, numpy).
- Wheels are pre-built for Python 3.8–3.12 on macOS ARM64, Linux x86_64, and Windows.
License · maintenance · safety
BSD (permissive) — BSD license is permissive and poses no significant restrictions on use or redistribution in most contexts.
last release 2024-04-24 (842 days) · last repo commit 2025-09-10 · 3,554 stars · archived
0 known vulnerabilities (OSV.dev, 2026-08-14) · 279,941 downloads/mo, #8,116 on PyPI
Alternatives
Verify before relying
pip install torchtext
import torchtext
from torchtext.datasets import AG_NEWS
train_dataset = AG_NEWS(split='train')- Whether pre-trained models (RoBERTa, T5, Flan-T5) are still downloadable and functional given the abandoned status.
- Current state of dataset availability and whether all listed datasets (WikiText2, SQuAD, IMDB, etc.) remain accessible.
- Compatibility with recent PyTorch versions beyond those explicitly documented in the version table.
What it is and what it does
torchtext is a PyTorch utility library for text processing and NLP workflows. It bundles raw text dataset iterators for common benchmarks (WikiText, SQuAD, AG_NEWS, etc.), basic NLP building blocks (tokenization, vocabulary), text transformations, and pre-trained models (RoBERTa, T5, XLM-R). The library depends on torch, numpy, requests, and tqdm.
As of September 2023, active development has been paused and the repository is archived. The project will receive maintenance releases but no new features. This means the library is stable for existing use cases but will not evolve to support emerging NLP techniques or datasets.
Use it for
- Loading and preprocessing standard NLP benchmarks like AG_NEWS or SST-2 for text classification experiments.
- Building machine translation pipelines using Multi30k or IWSLT datasets with transformer models.
- Language modeling tasks with WikiText2 or WikiText103 datasets for pretraining or fine-tuning.
- Applying scriptable tokenizers (SentencePiece, GPT-2 BPE, BERT) to raw text in production workflows.
- Accessing pre-trained RoBERTa or T5 models for transfer learning on downstream NLP tasks.
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes, but with caution.
torchtext remains useful for standard NLP tasks and dataset loading, with no known security vulnerabilities and permissive licensing. However, the abandoned maintenance status (paused since September 2023, archived repository) means no new features or dataset support will be added. Install it if you need its specific datasets or models for existing workflows, but do not expect updates for modern NLP techniques or emerging benchmarks. For new projects, consider actively maintained alternatives.
Install
torchtext on PyPI
Before you install
Medium install friction due to compiled dependencies (torch, numpy). Wheels are pre-built for Python 3.8–3.12 on macOS ARM64, Linux x86_64, and Windows. However, the project is archived and in abandoned maintenance status as of September 2023, with no active feature development planned.
Requires torch to be installed first; optional dependencies (spacy, sacremoses, sentencepiece) needed for specific tokenizers.
License in practice
BSD license is permissive and poses no significant restrictions on use or redistribution in most contexts.
Quickstart
pip install torchtext
import torchtext
from torchtext.datasets import AG_NEWS
train_dataset = AG_NEWS(split='train')Verify before relying
- Whether pre-trained models (RoBERTa, T5, Flan-T5) are still downloadable and functional given the abandoned status.
- Current state of dataset availability and whether all listed datasets (WikiText2, SQuAD, IMDB, etc.) remain accessible.
- Compatibility with recent PyTorch versions beyond those explicitly documented in the version table.
Package facts
| License | BSD permissive |
| Python support | Supports the current Python release >=3.8 |
| Install friction | Medium. Platform-specific wheel |
| Runtime dependencies | 4 packagestqdmrequeststorchnumpy |
| Maintenance | Abandoned 842 days since the last release |
| Last repo commit | repository archived |
| First released | |
| Downloads | 279,941 / month, #8,116 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
| Classifiers | Programming Language :: Python :: 3.10Programming Language :: Python :: 3.8Programming Language :: Python :: 3.9 |
Evidence: torchtext-0.18.0-cp310-cp310-macosx_11_0_arm64.whl; torchtext-0.18.0-cp310-cp310-manylinux1_x86_64.whl; torchtext-0.18.0-cp310-cp310-win_amd64.whl; torchtext-0.18.0-cp311-cp311-macosx_11_0_arm64.whl; torchtext-0.18.0-cp311-cp311-manylinux1_x86_64.whl; torchtext-0.18.0-cp311-cp311-win_amd64.whl; torchtext-0.18.0-cp312-cp312-macosx_11_0_arm64.whl; torchtext-0.18.0-cp312-cp312-manylinux1_x86_64.whl; torchtext-0.18.0-cp312-cp312-win_amd64.whl; torchtext-0.18.0-cp38-cp38-macosx_11_0_arm64.whl; torchtext-0.18.0-cp38-cp38-manylinux1_x86_64.whl; torchtext-0.18.0-cp38-cp38-win_amd64.whl; torchtext-0.18.0-cp39-cp39-macosx_11_0_arm64.whl; torchtext-0.18.0-cp39-cp39-manylinux1_x86_64.whl; torchtext-0.18.0-cp39-cp39-win_amd64.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “pytorch nlp datasets”
- torchtexttorchtext provides text datasets, preprocessing transforms, and…
- flairFlair is a PyTorch-based NLP framework that applies pre-trained…
- keras-nlpKeras-NLP provides pretrained models and utilities for natural…
Give your agent the search over MCP, or paste the wish link into any chat.
More Linguistic packages
Detects and normalizes text encoding from unknown or ambiguous sources, supporting all IANA character sets that Python's core library provides codecs for, with the ability to register custom codecs.
tiktoken is a fast BPE tokenizer that converts text into token sequences compatible with OpenAI models, supporting multiple encoding schemes including o200k_base and model-specific encodings.
Install it if you work with OpenAI APIs or need to understand token boundaries in GPT-family models.
Detects character encoding and language in byte sequences with high accuracy, supporting 99 encodings and returning confidence scores, language tags, and MIME types.
Install it if you need to detect character encoding or language in byte data; the rewrite makes it substantially faster and more accurate than its predecessors.
Converts Unicode text to ASCII by transliterating non-ASCII characters into their closest ASCII equivalents, with no runtime dependencies.
However, if transliteration quality or ongoing maintenance matters, consider unidecode instead despite its GPL-only license.
Lark is a parsing library that builds abstract syntax trees from context-free grammars, supporting multiple parsing algorithms (Earley, LALR(1), CYK) with automatic line and column tracking.
Python bindings to the tree-sitter parsing library, enabling incremental parsing and syntax tree analysis for source code.
See also pytorch-pretrained-bert · pytorch-tokenizers · torchvision · torchgeo · spark-nlp · blingfire · flair · torchaudio · webdataset · torchio