tensorrt

What it is and what it does

TensorRT is NVIDIA's inference optimization library that compiles trained deep learning models into optimized GPU-resident engines. It reduces model size, latency, and memory consumption through layer fusion, quantization, and kernel auto-tuning, making it the standard tool for deploying neural networks on NVIDIA GPUs in production.

Version 11.2.1.2 is a major release that removes legacy APIs (weakly-typed networks, implicit quantization, IPluginV2) in favor of modern alternatives (Strongly Typed Networks, Explicit Quantization, IPluginV3). Installation requires the TensorRT GA build plus CUDA toolkit and system dependencies; the package is actively maintained and widely used (197616 monthly downloads), but carries high install friction due to compiled dependencies and GPU-specific requirements.

Use it for:

• Compile ONNX or PyTorch models into optimized TensorRT engines for low-latency inference on NVIDIA GPUs.
• Deploy large language models or vision transformers with reduced memory footprint using quantization and kernel optimization.
• Benchmark and profile model inference performance across different batch sizes and precision levels on target hardware.
• Integrate optimized inference engines into production services for real-time inference workloads.
• Migrate from TensorRT 10.X to 11.X by refactoring weakly-typed networks to Strongly Typed Networks and updating custom plugins to IPluginV3.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

TensorRT compiles and optimizes deep learning models for deployment on NVIDIA GPUs, reducing latency and memory footprint through layer fusion, quantization, and kernel auto-tuning.

Yes, if you deploy deep learning models on NVIDIA GPUs and need production-grade inference optimization. TensorRT is the standard tool for this task, actively maintained, and widely adopted (top 15000 packages). However, installation requires CUDA toolkit, system-level build tools, and the TensorRT GA build v11.2.1.2; proprietary licensing requires license review for commercial use; and version 11.X breaks backward compatibility with 10.X APIs. Install only if you have GPU infrastructure and can meet compilation prerequisites.

Install

tensorrt on PyPI

pip install tensorrt

uv add tensorrt

poetry add tensorrt

Installing tensorrt

Before you install

High install friction: requires tensorrt_cu13 runtime dependency and TensorRT GA build v11.2.1.2. Actively maintained (last commit 2026-08-04, released 2026-07-30) with strong community adoption (13250 stars), but installation demands CUDA toolkit and system-level build prerequisites.

License in practice

Licensed as Proprietary with unclear license treatment. Users should review NVIDIA's licensing terms before deploying in production or commercial contexts, as the exact scope of permitted use is not standardized under an open-source identifier.

Quickstart

pip install tensorrt

import tensorrt as trt

logger = trt.Logger(trt.Logger.WARNING)
builder = trt.Builder(logger)

Requires CUDA toolkit (recommended versions 13.3.0 or 12.9.0), cuDNN (optional, 8.9), and TensorRT GA build v11.2.1.2 to be downloaded and extracted separately; Python >= 3.10, <= 3.14.x required for building from source.

Verify before relying

• Whether the prebuilt pip package includes all required runtime components or if separate CUDA/CUDNN installation is mandatory.
• Whether tensorrt_cu13 dependency is automatically resolved or requires manual CUDA 13.x environment setup.
• Exact Python version support within the >=3.8 range, given removal of Python 3.9 and older bindings in version 11.X.

Package facts

Evidence: tensorrt-11.2.1.2.tar.gz

Keywords: nvidia, tensorrt, deeplearning, inference

Tags

More Artificial Intelligence packages

Further reading