tianshou

What it is and what it does

Tianshou is a modular reinforcement learning framework built on PyTorch and Gymnasium that provides both researcher-friendly low-level procedural APIs and practitioner-friendly high-level training interfaces. It implements a broad range of RL algorithms spanning on-policy methods (PG, A2C, PPO, TRPO), off-policy methods (DQN variants, DDPG, TD3, SAC), offline RL (BCQ, CQL), and imitation learning (GAIL, behavioral cloning). Version 2 introduces a cleaner architecture with explicit separation between Algorithm and Policy abstractions, reorganized class hierarchies, and improved parameter naming.

The library emphasizes modularity and performance: it supports vectorized environments (synchronous, asynchronous, and EnvPool-based), recurrent policies for POMDPs, arbitrary state/action representations, multi-GPU training, and TensorBoard/W&B logging. Core components like n-step returns, prioritized experience replay, and the Generalized Advantage Estimator are optimized with Numba JIT compilation. Experimental support for multi-agent RL and model-based methods is included. With 10926 GitHub stars and active maintenance, it is a mature choice for both research and production RL applications.

Use it for:

• Train DQN or Rainbow agents on Atari-style discrete action environments with prioritized experience replay and n-step returns
• Implement continuous control policies (DDPG, TD3, SAC) for robotics or MuJoCo benchmarks with vectorized parallel sampling
• Build offline RL agents (BCQ, CQL) from logged interaction data without online environment interaction
• Develop multi-agent RL systems using Tianshou's experimental MARL support with PettingZoo environments
• Prototype imitation learning pipelines (GAIL, behavioral cloning) from expert demonstrations
• Customize training loops and algorithm implementations using the low-level procedural API for research

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Tianshou is a PyTorch-based reinforcement learning library that implements a wide range of RL algorithms (DQN, PPO, SAC, and others) with both high-level and low-level APIs for training agents on Gymnasium environments.

Yes, if you are building or researching deep reinforcement learning agents. Tianshou offers a mature, well-maintained framework with broad algorithm coverage, clean APIs, and strong performance. Version 2 is a breaking redesign that improves usability and type safety; migration from v1 requires code changes. The 15-dependency stack (torch, numpy, gymnasium, etc.) is standard for ML work. No known vulnerabilities. Suitable for both academic research and production RL applications.

Install

tianshou on PyPI

pip install tianshou

uv add tianshou

poetry add tianshou

Installing tianshou

Before you install

Low install friction with a pure-Python wheel. Active maintenance with recent releases; last commit 2026-04-03. Requires Python 3.11 or later and brings in 15 runtime dependencies including torch, numpy, and gymnasium—a substantial but standard ML stack.

License in practice

MIT license (permissive) allows commercial and private use with minimal restrictions; you must include a copy of the license and copyright notice in distributions.

Quickstart

pip install tianshou

import torch
from tianshou.policy import DQNPolicy
from tianshou.trainer import OffpolicyTrainer

# Define policy, collector, and trainer; call trainer.run()

Requires Python >= 3.11. PyTorch and Gymnasium must be installed; torch typically requires a compatible CUDA toolkit or CPU-only build.

Verify before relying

• Whether version 2's breaking changes from version 1 affect existing codebases that depend on Tianshou
• Performance characteristics and scalability limits for multi-agent RL and model-based algorithms marked as experimental
• Specific hardware requirements or GPU memory recommendations for typical training workloads

Package facts

Evidence: tianshou-2.0.1-py3-none-any.whl

Tags

More Python Modules packages

Further reading