tensorrt-cu12

What it is and what it does

tensorrt-cu12 is the Python interface to NVIDIA's TensorRT 11.2.1.2 inference engine, optimized for CUDA 12 GPUs. It compiles and executes deep learning models with minimal latency and memory overhead, supporting import paths from ONNX, PyTorch, and HuggingFace. Version 11 introduced breaking changes: weakly-typed networks and implicit quantization have been removed in favor of strongly-typed networks and explicit quantization; IPluginV2 has been replaced by IPluginV3; and Python 3.9 and older are no longer supported.

The package depends on tensorrt_cu12_libs and tensorrt_cu12_bindings, which provide compiled libraries and low-level bindings. Installation requires pre-existing CUDA and TensorRT GA build infrastructure. It is intended for developers deploying inference workloads on NVIDIA GPUs where model throughput and latency are critical, not for training or development on CPU-only systems.

Use it for:

• Compile trained models (ONNX, PyTorch) into optimized TensorRT engines for low-latency serving on NVIDIA GPUs.
• Deploy LLMs, vision models, and encoder-NLP architectures with explicit quantization and graph optimization for production inference.
• Benchmark and profile inference performance across different batch sizes and precision modes.
• Integrate custom layers via IPluginV3 to extend TensorRT's operator coverage for specialized model architectures.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Provides Python bindings for NVIDIA TensorRT 11, enabling high-performance deep learning inference on CUDA 12 GPUs through optimized model compilation and execution.

Yes, if you have CUDA GPU infrastructure and need production-grade inference acceleration. High install friction and proprietary licensing require upfront commitment to NVIDIA's ecosystem. Active maintenance and no known vulnerabilities are positive signals. Not suitable for development without dedicated GPU hardware or for environments where proprietary licensing is prohibited.

Install

tensorrt-cu12 on PyPI

pip install tensorrt-cu12

uv add tensorrt-cu12

poetry add tensorrt-cu12

Installing tensorrt-cu12

Before you install

High install friction: requires CUDA and system-level TensorRT libraries (tensorrt_cu12_libs, tensorrt_cu12_bindings). Package is actively maintained with recent releases, but installation demands pre-existing GPU infrastructure and native dependencies.

License in practice

Licensed as Proprietary with unclear treatment. Terms are not publicly specified; review NVIDIA's licensing documentation before deploying in commercial or restricted environments.

Quickstart

pip install tensorrt-cu12

import tensorrt as trt

logger = trt.Logger(trt.Logger.WARNING)
builder = trt.Builder(logger)

Requires TensorRT GA build v11.2.1.2 pre-installed and Python >=3.10 (3.9 and older no longer supported in 11.X).

Verify before relying

• Whether tensorrt-cu12 can be installed standalone or requires manual TensorRT GA build download and extraction.
• Exact CUDA versions (12.9 or 13.3) supported by this package variant.
• Exact compatibility matrix between Python versions 3.10–3.14 and this package version.
• Whether cuDNN 8.9 is required or optional for this package.

Package facts

Evidence: tensorrt_cu12-11.2.1.2.tar.gz

Keywords: nvidia, tensorrt, deeplearning, inference

Tags

More Artificial Intelligence packages

Further reading