---
id: tianshou
version: "2.0.1"
license: MIT
license_treatment: permissive
maintenance: active
---

# tianshou — A Library for Deep Reinforcement Learning

License: permissive · Maintenance: active · Downloads: 114.1K/mo

## What it is and what it does

Tianshou is a modular reinforcement learning framework built on PyTorch and Gymnasium that provides both researcher-friendly low-level procedural APIs and practitioner-friendly high-level training interfaces. It implements a broad range of RL algorithms spanning on-policy methods (PG, A2C, PPO, TRPO), off-policy methods (DQN variants, DDPG, TD3, SAC), offline RL (BCQ, CQL), and imitation learning (GAIL, behavioral cloning). Version 2 introduces a cleaner architecture with explicit separation between Algorithm and Policy abstractions, reorganized class hierarchies, and improved parameter naming.

The library emphasizes modularity and performance: it supports vectorized environments (synchronous, asynchronous, and EnvPool-based), recurrent policies for POMDPs, arbitrary state/action representations, multi-GPU training, and TensorBoard/W&B logging. Core components like n-step returns, prioritized experience replay, and the Generalized Advantage Estimator are optimized with Numba JIT compilation. Experimental support for multi-agent RL and model-based methods is included. With 10926 GitHub stars and active maintenance, it is a mature choice for both research and production RL applications.

Use it for:

- Train DQN or Rainbow agents on Atari-style discrete action environments with prioritized experience replay and n-step returns
- Implement continuous control policies (DDPG, TD3, SAC) for robotics or MuJoCo benchmarks with vectorized parallel sampling
- Build offline RL agents (BCQ, CQL) from logged interaction data without online environment interaction
- Develop multi-agent RL systems using Tianshou's experimental MARL support with PettingZoo environments
- Prototype imitation learning pipelines (GAIL, behavioral cloning) from expert demonstrations
- Customize training loops and algorithm implementations using the low-level procedural API for research

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

Tianshou is a PyTorch-based reinforcement learning library that implements a wide range of RL algorithms (DQN, PPO, SAC, and others) with both high-level and low-level APIs for training agents on Gymnasium environments.

Yes, if you are building or researching deep reinforcement learning agents. Tianshou offers a mature, well-maintained framework with broad algorithm coverage, clean APIs, and strong performance. Version 2 is a breaking redesign that improves usability and type safety; migration from v1 requires code changes. The 15-dependency stack (torch, numpy, gymnasium, etc.) is standard for ML work. No known vulnerabilities. Suitable for both academic research and production RL applications.

## Install

pip install tianshou
uv add tianshou
poetry add tianshou

## Installing tianshou

Before you install: Low install friction with a pure-Python wheel. Active maintenance with recent releases; last commit 2026-04-03. Requires Python 3.11 or later and brings in 15 runtime dependencies including torch, numpy, and gymnasium—a substantial but standard ML stack.

License in practice: MIT license (permissive) allows commercial and private use with minimal restrictions; you must include a copy of the license and copyright notice in distributions.

Quickstart:

pip install tianshou

import torch
from tianshou.policy import DQNPolicy
from tianshou.trainer import OffpolicyTrainer

# Define policy, collector, and trainer; call trainer.run()

Requires Python >= 3.11. PyTorch and Gymnasium must be installed; torch typically requires a compatible CUDA toolkit or CPU-only build.

Verify before relying:

- Whether version 2's breaking changes from version 1 affect existing codebases that depend on Tianshou
- Performance characteristics and scalability limits for multi-agent RL and model-based algorithms marked as experimental
- Specific hardware requirements or GPU memory recommendations for typical training workloads

## Package facts

- License: MIT (permissive)
- Python support: supports_current
- Install friction: low
- Maintenance: active
- Downloads: 114.1K/month (top 15,000 on PyPI)
- Known vulnerabilities: none known

## Tags

reinforcement learning library pytorch, deep Q-learning DQN implementation, policy gradient PPO SAC algorithms, RL agent training framework, gymnasium environment training, multi-agent reinforcement learning, offline RL algorithms, vectorized environment support, reinforcement-learning, pytorch, multi-agent

[View on SkillFed](https://skillfed.io/packages/tianshou) · [View on PyPI](https://pypi.org/project/tianshou/)