onnxruntime-gpu
ONNX Runtime is a runtime accelerator for Machine Learning models
Decision gist · record as of 2026-08-14
Yes, if you have GPU hardware available and need to run ONNX models in production or research. The package is actively maintained, permissively licensed, and widely used (top 5000 PyPI). Install friction is moderate due to GPU driver/CUDA setup requirements, but the package itself is stable and well-supported. Not suitable if you lack GPU hardware or need CPU-only inference.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires NVIDIA GPU with compatible CUDA/cuDNN libraries installed on the system; GPU drivers and CUDA toolkit must be configured separately.
- Medium install friction due to GPU-specific wheels and binary dependencies.
- The package is actively maintained with a recent release (20 days old) and strong repository signals (21377 stars, active development).
License · maintenance · safety
MIT License (permissive) — MIT License permits commercial and private use with minimal restrictions. You may use, modify, and distribute this package freely provided you include the license notice.
last release 2026-07-25 (20 days) · last repo commit 2026-08-14 · 21,377 stars
0 known vulnerabilities (OSV.dev, 2026-08-14) · 3,493,668 downloads/mo, #2,600 on PyPI
Alternatives
Verify before relying
pip install onnxruntime-gpu
import onnxruntime as rt
sess = rt.InferenceSession('model.onnx')
results = sess.run(None, {'input_name': input_data})- Specific CUDA version requirements and GPU compatibility matrix not provided in fact sheet.
- Performance benchmarks or speedup claims relative to CPU inference not available.
- Whether quantization, optimization, or model conversion tools are included in this package.
What it is and what it does
ONNX Runtime GPU is a performance-focused inference engine that executes Open Neural Network Exchange (ONNX) models on NVIDIA GPUs. It takes pre-trained models in ONNX format and runs them efficiently on GPU hardware, accelerating predictions for machine learning workloads. The package depends on flatbuffers, numpy, packaging, and protobuf for model serialization, numerical operations, and dependency management.
This is the GPU-specific variant of ONNX Runtime, designed for scenarios where inference speed matters and GPU hardware is available. It's commonly used in production systems, real-time inference pipelines, and research environments where model throughput or latency is critical. The package is maintained by Microsoft, actively developed, and supports recent Python versions (3.11–3.14) across Linux and Windows platforms.
Use it for
- Accelerate inference for computer vision models (object detection, image classification) in production systems.
- Run real-time NLP model inference (transformers, language models) with GPU speedup for low-latency predictions.
- Deploy quantized or optimized ONNX models on GPU for high-throughput batch inference in data pipelines.
- Execute recommendation system models on GPU for fast scoring in online serving environments.
- Benchmark or evaluate model performance on GPU hardware during model development and optimization.
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes, if you have GPU hardware available and need to run ONNX models in production or research.
The package is actively maintained, permissively licensed, and widely used (top 5000 PyPI). Install friction is moderate due to GPU driver/CUDA setup requirements, but the package itself is stable and well-supported. Not suitable if you lack GPU hardware or need CPU-only inference.
Install
onnxruntime-gpu on PyPI
Before you install
Medium install friction due to GPU-specific wheels and binary dependencies. The package is actively maintained with a recent release (20 days old) and strong repository signals (21377 stars, active development). Supports Python 3.11 through 3.14 with platform-specific wheels for Linux and Windows.
Requires NVIDIA GPU with compatible CUDA/cuDNN libraries installed on the system; GPU drivers and CUDA toolkit must be configured separately.
License in practice
MIT License permits commercial and private use with minimal restrictions. You may use, modify, and distribute this package freely provided you include the license notice.
Quickstart
pip install onnxruntime-gpu
import onnxruntime as rt
sess = rt.InferenceSession('model.onnx')
results = sess.run(None, {'input_name': input_data})Verify before relying
- Specific CUDA version requirements and GPU compatibility matrix not provided in fact sheet.
- Performance benchmarks or speedup claims relative to CPU inference not available.
- Whether quantization, optimization, or model conversion tools are included in this package.
Package facts
| License | MIT License permissive |
| Python support | Supports the current Python release >=3.11 |
| Install friction | Medium. Platform-specific wheel |
| Runtime dependencies | 4 packagesflatbuffersnumpypackagingprotobuf |
| Maintenance | Actively maintained 20 days since the last release |
| Last repo commit | |
| First released | |
| Downloads | 3,493,668 / month, #2,600 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
| Classifiers | Development Status :: 5 - Production/StableIntended Audience :: DevelopersLicense :: OSI Approved :: MIT LicenseOperating System :: MacOSOperating System :: Microsoft :: WindowsOperating System :: POSIX :: LinuxProgramming Language :: PythonProgramming Language :: Python :: 3 :: OnlyProgramming Language :: Python :: 3.11Programming Language :: Python :: 3.12Programming Language :: Python :: 3.13Programming Language :: Python :: 3.14Topic :: Scientific/EngineeringTopic :: Scientific/Engineering :: Artificial IntelligenceTopic :: Scientific/Engineering :: MathematicsTopic :: Software DevelopmentTopic :: Software Development :: LibrariesTopic :: Software Development :: Libraries :: Python Modules |
Evidence: onnxruntime_gpu-1.28.0-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl; onnxruntime_gpu-1.28.0-cp311-cp311-win_amd64.whl; onnxruntime_gpu-1.28.0-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl; onnxruntime_gpu-1.28.0-cp312-cp312-win_amd64.whl; onnxruntime_gpu-1.28.0-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl; onnxruntime_gpu-1.28.0-cp313-cp313t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl; onnxruntime_gpu-1.28.0-cp313-cp313-win_amd64.whl; onnxruntime_gpu-1.28.0-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl; onnxruntime_gpu-1.28.0-cp314-cp314t-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl; onnxruntime_gpu-1.28.0-cp314-cp314-win_amd64.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “onnx model inference gpu”
- onnxruntime-gpuExecutes ONNX machine learning models on GPU hardware, providing…
- sit4onnxRuns inference tests on ONNX models using TensorRT, CUDA, OpenVINO,…
- optimum-onnxExports Hugging Face transformer models to ONNX format and runs them…
Give your agent the search over MCP, or paste the wish link into any chat.
More Software Development packages
Provides backported and experimental type hints for Python 3.9+, allowing use of newer typing features on older Python versions and enabling early experimentation with type system PEPs before they enter the standard library.
NumPy provides an N-dimensional array object and a comprehensive suite of mathematical, linear algebra, Fourier transform, and random number functions for scientific computing in Python.
FastAPI is a Python web framework for building REST APIs using type hints, with automatic request validation, serialization, and interactive API documentation.
Provides a way to document function parameters, class attributes, return types, and variables inline using Python's `Annotated` type hint syntax instead of traditional docstrings.
Typer builds command-line applications from Python functions using type hints, automatically generating help text, argument parsing, and shell completion.
Install it if you are building CLIs in Python.
Distlib provides low-level packaging utilities for building, distributing, and managing Python software—including metadata handling, version specifiers, wheel support, script installation, and dependency resolution.
See also onnxruntime · onnxruntime-openvino · onnxruntime_extensions · onnx · nvidia-cudnn-cu11 · sit4onnx · nvidia-cudnn-cu12 · scikit-learn-intelex · nvidia-cudnn-cu13 · nvidia-cuda-runtime