tianshou
A Library for Deep Reinforcement Learning
Decision gist · record as of 2026-08-14
Yes, if you are building or researching deep reinforcement learning agents. Tianshou offers a mature, well-maintained framework with broad algorithm coverage, clean APIs, and strong performance. Version 2 is a breaking redesign that improves usability and type safety; migration from v1 requires code changes. The 15-dependency stack (torch, numpy, gymnasium, etc.) is standard for ML work. No known vulnerabilities. Suitable for both academic research and production RL applications.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires Python >= 3.11.
- PyTorch and Gymnasium must be installed; torch typically requires a compatible CUDA toolkit or CPU-only build.
- Low install friction with a pure-Python wheel.
License · maintenance · safety
MIT (permissive) — MIT license (permissive) allows commercial and private use with minimal restrictions; you must include a copy of the license and copyright notice in distributions.
last release 2026-04-02 (134 days) · last repo commit 2026-04-03 · 10,926 stars
0 known vulnerabilities (OSV.dev, 2026-08-14) · 114,112 downloads/mo, #12,310 on PyPI
Alternatives
Verify before relying
pip install tianshou
import torch
from tianshou.policy import DQNPolicy
from tianshou.trainer import OffpolicyTrainer
# Define policy, collector, and trainer; call trainer.run()- Whether version 2's breaking changes from version 1 affect existing codebases that depend on Tianshou
- Performance characteristics and scalability limits for multi-agent RL and model-based algorithms marked as experimental
- Specific hardware requirements or GPU memory recommendations for typical training workloads
What it is and what it does
Tianshou is a modular reinforcement learning framework built on PyTorch and Gymnasium that provides both researcher-friendly low-level procedural APIs and practitioner-friendly high-level training interfaces. It implements a broad range of RL algorithms spanning on-policy methods (PG, A2C, PPO, TRPO), off-policy methods (DQN variants, DDPG, TD3, SAC), offline RL (BCQ, CQL), and imitation learning (GAIL, behavioral cloning). Version 2 introduces a cleaner architecture with explicit separation between Algorithm and Policy abstractions, reorganized class hierarchies, and improved parameter naming.
The library emphasizes modularity and performance: it supports vectorized environments (synchronous, asynchronous, and EnvPool-based), recurrent policies for POMDPs, arbitrary state/action representations, multi-GPU training, and TensorBoard/W&B logging. Core components like n-step returns, prioritized experience replay, and the Generalized Advantage Estimator are optimized with Numba JIT compilation. Experimental support for multi-agent RL and model-based methods is included. With 10926 GitHub stars and active maintenance, it is a mature choice for both research and production RL applications.
Use it for
- Train DQN or Rainbow agents on Atari-style discrete action environments with prioritized experience replay and n-step returns
- Implement continuous control policies (DDPG, TD3, SAC) for robotics or MuJoCo benchmarks with vectorized parallel sampling
- Build offline RL agents (BCQ, CQL) from logged interaction data without online environment interaction
- Develop multi-agent RL systems using Tianshou's experimental MARL support with PettingZoo environments
- Prototype imitation learning pipelines (GAIL, behavioral cloning) from expert demonstrations
- Customize training loops and algorithm implementations using the low-level procedural API for research
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes, if you are building or researching deep reinforcement learning agents.
Tianshou offers a mature, well-maintained framework with broad algorithm coverage, clean APIs, and strong performance. Version 2 is a breaking redesign that improves usability and type safety; migration from v1 requires code changes. The 15-dependency stack (torch, numpy, gymnasium, etc.) is standard for ML work. No known vulnerabilities. Suitable for both academic research and production RL applications.
Install
tianshou on PyPI
Before you install
Low install friction with a pure-Python wheel. Active maintenance with recent releases; last commit 2026-04-03. Requires Python 3.11 or later and brings in 15 runtime dependencies including torch, numpy, and gymnasium—a substantial but standard ML stack.
Requires Python >= 3.11. PyTorch and Gymnasium must be installed; torch typically requires a compatible CUDA toolkit or CPU-only build.
License in practice
MIT license (permissive) allows commercial and private use with minimal restrictions; you must include a copy of the license and copyright notice in distributions.
Quickstart
pip install tianshou
import torch
from tianshou.policy import DQNPolicy
from tianshou.trainer import OffpolicyTrainer
# Define policy, collector, and trainer; call trainer.run()Verify before relying
- Whether version 2's breaking changes from version 1 affect existing codebases that depend on Tianshou
- Performance characteristics and scalability limits for multi-agent RL and model-based algorithms marked as experimental
- Specific hardware requirements or GPU memory recommendations for typical training workloads
Package facts
| License | MIT permissive |
| Python support | Supports the current Python release <4.0,>=3.11 |
| Install friction | Low. Pure-Python wheel |
| Runtime dependencies | 15 packagesdeepdiffgymnasiumh5pymatplotlibnumbanumpyoverridespackagingpandaspettingzoosensai-utilstensorboardtorchtqdmvirtualenv |
| Maintenance | Actively maintained 134 days since the last release |
| Last repo commit | |
| First released | |
| Downloads | 114,112 / month, #12,310 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
| Classifiers | Development Status :: 4 - BetaIntended Audience :: Science/ResearchLicense :: OSI Approved :: MIT LicenseProgramming Language :: Python :: 3Programming Language :: Python :: 3.11Programming Language :: Python :: 3.12Programming Language :: Python :: 3.13Programming Language :: Python :: 3.14Topic :: Scientific/Engineering :: Artificial IntelligenceTopic :: Software Development :: Libraries :: Python Modules |
Evidence: tianshou-2.0.1-py3-none-any.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “deep Q-learning DQN implementation”
- tianshouTianshou is a PyTorch-based reinforcement learning library that…
- dopamine-rlDopamine is a research framework for prototyping reinforcement…
- sb3-contribProvides experimental reinforcement learning algorithms and tools…
Give your agent the search over MCP, or paste the wish link into any chat.
More Python Modules packages
Converts domain names between Unicode and ASCII-compatible encoding (Punycode) according to IDNA 2008 and Unicode Technical Standard 46, with security validation and broader script coverage than the standard library.
Install it if you work with internationalized domain names, need to validate domains, or use HTTP clients that depend on it transitively.
Setuptools is a Python build backend and package management tool that handles building, distributing, and installing Python packages, including support for C/C++ extension modules.
PyYAML parses and emits YAML 1.1 data format, enabling serialization and deserialization of configuration files and Python objects to and from human-readable YAML text.
Pydantic validates Python data structures against type hints, coercing and checking input at runtime to ensure it matches a declared schema.
Provides reusable metadata objects for use with PEP-593 `typing.Annotated` to express common constraints like bounds, collection sizes, and predicates on types.
Install it if you use or build libraries that need to express type constraints in a standardized, inspectable way—or if you want to annotate your own types with…
Provides runtime tools to inspect and introspect Python type annotations, enabling programmatic examination of type hints at execution time.
See also dopamine-rl · rsl-rl-lib · stable-baselines3 · gymnasium · pettingzoo · sb3-contrib · skrl · torchrl · gem-llm · open-spiel