torchrl
A modular, primitive-first, python-first PyTorch library for Reinforcement Learning
Decision gist · record as of 2026-08-14
Yes, with conditions. TorchRL is actively maintained, has no known vulnerabilities, and is well-suited for research and production RL systems that need to scale from prototypes to distributed training. However, verify the license status before committing to proprietary use, and ensure your target platform has compatible wheels (Python 3.10–3.14, macOS/Linux/Windows). Medium install friction due to torch and tensordict dependencies is typical for PyTorch-based ML libraries.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires torch and tensordict as runtime dependencies; torch installation may require system-level build tools or pre-built wheels for your platform.
- Medium install friction due to compiled dependencies (torch, tensordict).
- Wheels are available for Python 3.10–3.14 across macOS (ARM64), Linux (x86_64, aarch64), and Windows.
License · maintenance · safety
(unclear) — License treatment is unclear—no SPDX identifier or raw license string is present in the metadata. Verify the actual license before adopting in proprietary or restricted-distribution projects.
last release 2026-07-14 (31 days) · last repo commit 2026-08-14 · 3,519 stars
0 known vulnerabilities (OSV.dev, 2026-08-14) · 1,313,205 downloads/mo, #4,070 on PyPI
Alternatives
Verify before relying
pip install torchrl torch tensordict
from torchrl.envs import PendulumEnv, TransformedEnv
from tensordict.nn import TensorDictModule
from torch import nn
env = TransformedEnv(PendulumEnv())
policy = TensorDictModule(
nn.Sequential(nn.LazyLinear(64), nn.Tanh(), nn.Linear(64, 1)),
in_keys=["observation"],
out_keys=["action"],
)
rollout = env.rollout(max_steps=32, policy=policy)- Whether the unclear license permits commercial or proprietary use without restrictions.
- Performance characteristics and scalability limits for distributed multi-agent training at scale.
- Compatibility with specific environment libraries (Gymnasium, DM Control, Isaac Lab) beyond the documented wrappers.
What it is and what it does
TorchRL is a modular reinforcement learning library built on PyTorch and TensorDict, designed to keep research code close to the PyTorch programming model while scaling from local prototypes to distributed, multi-agent, and model-based workflows. It provides reusable components—environments, policies, collectors, replay buffers, transforms, and loss functions—that communicate through a common TensorDict data model, eliminating the need to rewrite training loops when switching between single-process, vectorized, multiprocess, or distributed execution.
The library emphasizes composability and explicit structure: data carries names, batch dimensions, and device information throughout the training loop, and each component (environment, policy, replay buffer, loss) can be swapped independently. It includes native PyTorch environments, wrappers for popular libraries (Gymnasium, DM Control, Brax, PettingZoo, VMAS, OpenSpiel, Isaac Lab), vectorized containers for local and multiprocess execution, and a rich set of transforms for observation normalization, action scaling, reward shaping, and state reconstruction.
Use it for
- Prototyping single-agent RL algorithms locally, then scaling to distributed training without changing the data model or core training code.
- Building multi-agent RL systems with explicit agent grouping, value normalization, and algorithms like MAPPO and IPPO.
- Training recurrent policies with optimized GRU/LSTM reset handling and scan-based forward passes.
- Integrating custom MuJoCo environments or wrappers for third-party simulators (Gymnasium, Isaac Lab, Brax) with standardized observation and action specs.
- Managing large replay buffers with prioritized sampling, async writes, and optional CUDA-accelerated kernels.
- Implementing offline RL or model-based workflows where the same TensorDict interface handles both real and synthetic trajectories.
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes, with conditions.
TorchRL is actively maintained, has no known vulnerabilities, and is well-suited for research and production RL systems that need to scale from prototypes to distributed training. However, verify the license status before committing to proprietary use, and ensure your target platform has compatible wheels (Python 3.10–3.14, macOS/Linux/Windows). Medium install friction due to torch and tensordict dependencies is typical for PyTorch-based ML libraries.
Install
torchrl on PyPI
Before you install
Medium install friction due to compiled dependencies (torch, tensordict). Wheels are available for Python 3.10–3.14 across macOS (ARM64), Linux (x86_64, aarch64), and Windows. Active maintenance with a release 31 days ago and 3519 repository stars.
Requires torch and tensordict as runtime dependencies; torch installation may require system-level build tools or pre-built wheels for your platform.
License in practice
License treatment is unclear—no SPDX identifier or raw license string is present in the metadata. Verify the actual license before adopting in proprietary or restricted-distribution projects.
Quickstart
pip install torchrl torch tensordict
from torchrl.envs import PendulumEnv, TransformedEnv
from tensordict.nn import TensorDictModule
from torch import nn
env = TransformedEnv(PendulumEnv())
policy = TensorDictModule(
nn.Sequential(nn.LazyLinear(64), nn.Tanh(), nn.Linear(64, 1)),
in_keys=["observation"],
out_keys=["action"],
)
rollout = env.rollout(max_steps=32, policy=policy)Verify before relying
- Whether the unclear license permits commercial or proprietary use without restrictions.
- Performance characteristics and scalability limits for distributed multi-agent training at scale.
- Compatibility with specific environment libraries (Gymnasium, DM Control, Isaac Lab) beyond the documented wrappers.
Package facts
| License | Not declared unclear |
| Python support | Supports the current Python release >=3.10 |
| Install friction | Medium. Platform-specific wheel |
| Runtime dependencies | 7 packagestorchpyvershoptorchnumpypackagingcloudpickletensordict |
| Maintenance | Actively maintained 31 days since the last release |
| Last repo commit | |
| First released | |
| Downloads | 1,313,205 / month, #4,070 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
| Classifiers | Development Status :: 4 - BetaIntended Audience :: DevelopersIntended Audience :: Science/ResearchOperating System :: OS IndependentProgramming Language :: Python :: 3.10Programming Language :: Python :: 3.11Programming Language :: Python :: 3.12Programming Language :: Python :: 3.13Programming Language :: Python :: 3.14Topic :: Scientific/Engineering :: Artificial Intelligence |
Evidence: torchrl-0.13.3-cp310-cp310-macosx_11_0_arm64.whl; torchrl-0.13.3-cp310-cp310-manylinux_2_28_aarch64.whl; torchrl-0.13.3-cp310-cp310-manylinux_2_28_x86_64.whl; torchrl-0.13.3-cp310-cp310-win_amd64.whl; torchrl-0.13.3-cp311-cp311-macosx_11_0_arm64.whl; torchrl-0.13.3-cp311-cp311-manylinux_2_28_aarch64.whl; torchrl-0.13.3-cp311-cp311-manylinux_2_28_x86_64.whl; torchrl-0.13.3-cp311-cp311-win_amd64.whl; torchrl-0.13.3-cp312-cp312-macosx_11_0_arm64.whl; torchrl-0.13.3-cp312-cp312-manylinux_2_28_aarch64.whl; torchrl-0.13.3-cp312-cp312-manylinux_2_28_x86_64.whl; torchrl-0.13.3-cp312-cp312-win_amd64.whl; torchrl-0.13.3-cp313-cp313-macosx_12_0_arm64.whl; torchrl-0.13.3-cp313-cp313-manylinux_2_28_aarch64.whl; torchrl-0.13.3-cp313-cp313-manylinux_2_28_x86_64.whl; torchrl-0.13.3-cp313-cp313-win_amd64.whl; torchrl-0.13.3-cp314-cp314-macosx_12_0_arm64.whl; torchrl-0.13.3-cp314-cp314-manylinux_2_28_aarch64.whl; torchrl-0.13.3-cp314-cp314-manylinux_2_28_x86_64.whl; torchrl-0.13.3-cp314-cp314-win_amd64.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “policy gradient implementation”
- torchrlTorchRL is a PyTorch-native toolkit for building reinforcement…
- dopamine-rlDopamine is a research framework for prototyping reinforcement…
- tianshouTianshou is a PyTorch-based reinforcement learning library that…
Give your agent the search over MCP, or paste the wish link into any chat.
More Artificial Intelligence packages
LiteLLM provides a unified Python interface to call 100+ LLM providers (OpenAI, Anthropic, Gemini, Bedrock, Azure, and others) using OpenAI-compatible API format, available as both a Python SDK and a self-hosted AI Gateway proxy server.
Install it if you need to work with multiple LLM providers or want to centralize LLM routing in your organization.
Client library and CLI tool for downloading, uploading, and managing models, datasets, and repositories on the Hugging Face Hub platform.
Install it if you work with Hugging Face Hub models or datasets.
LangChain provides a framework for building agents and LLM-powered applications by composing language models, tools, and memory through a unified API that abstracts over multiple model providers.
hf-xet provides chunk-based deduplication and efficient file transfer for the Hugging Face Hub, enabling faster uploads and downloads of large files with local disk caching.
Tokenizers converts raw text into token sequences for NLP models, with support for training custom vocabularies and using pre-built tokenizers (BPE, WordPiece) optimized for speed via Rust.
Transformers provides a unified framework for loading, fine-tuning, and running state-of-the-art pretrained models across text, vision, audio, video, and multimodal tasks using PyTorch, JAX, or TensorFlow.
Install it if you need to run or train any transformer-based model for NLP, vision, audio, or multimodal tasks.
See also skrl · rsl-rl-lib · verl · tianshou · lerobot · torchx · torch · torchtext · torchgeo · tensordict-nightly