Packages
Provides a portable intermediate representation (TileIR) for CUDA kernels that abstracts away language-specific details, enabling kernel compilation and optimization across different contexts.
Provides cuDNN runtime libraries for GPU-accelerated deep neural network operations on NVIDIA CUDA 11 hardware.
Provides cuDNN runtime libraries for GPU-accelerated deep neural network operations, requiring an NVIDIA GPU and CUDA 12 environment.
Provides cuDNN runtime libraries for GPU-accelerated deep neural network primitives, requiring CUDA 13 and nvidia-cublas as dependencies.
Install only if you have CUDA 13 and nvidia-cublas available; otherwise, installation will not resolve the underlying GPU dependencies.
Provides Python and C++ APIs to NVIDIA's cuDNN library, exposing high-performance GPU kernels for scaled dot-product attention, grouped matrix multiplication for mixture-of-experts training, and fused operations optimized for Hopper and Blackwell GPUs.
Install it if you are training or deploying deep learning models on NVIDIA Hopper or Blackwell GPUs and need high-performance attention, grouped GEMM, or quantized…
Provides NVIDIA CUFFT native runtime libraries for GPU-accelerated Fast Fourier Transform computations on CUDA-enabled hardware.
Provides NVIDIA CUFFT native runtime libraries for CUDA 11, enabling GPU-accelerated Fast Fourier Transform computations on compatible systems.
Install only if required as a transitive dependency of an active package, or if you are locked to CUDA 11 and cannot upgrade.
Provides NVIDIA CUFFT native runtime libraries for CUDA 12, enabling GPU-accelerated Fast Fourier Transform computations in Python applications.
However, verify NVIDIA's proprietary license terms for your use case, and ensure your system has compatible NVIDIA hardware and drivers installed before proceeding.
Provides Python bindings for NVIDIA's cuFile GPUDirect storage access libraries, enabling direct GPU-to-storage I/O without CPU involvement.
However, verify the unclear license terms before use in production, and confirm that your system has the required cuFile runtime libraries installed.
Provides Python bindings for NVIDIA's cuFile GPUDirect storage access libraries, enabling direct GPU-to-storage I/O without CPU involvement for CUDA 12 environments.
Provides NVIDIA CURAND native runtime libraries for GPU-accelerated random number generation in Python on Linux, Windows, and aarch64 platforms.
Provides NVIDIA CURAND native runtime libraries for CUDA 11, enabling GPU-accelerated random number generation in Python applications on x86_64 and ARM64 Linux, Windows, and legacy x86 platforms.
No—install only if you are locked into CUDA 11 and have no path to upgrade.
Provides NVIDIA CURAND native runtime libraries for CUDA 12, enabling GPU-accelerated random number generation on supported hardware.
However, verify that your application actually requires it (it is typically pulled in transitively by higher-level libraries) and that you have an NVIDIA GPU and CUDA…
Provides CUDA solver native runtime libraries for GPU-accelerated linear algebra operations on NVIDIA hardware.
Provides NVIDIA CUDA solver native runtime libraries for GPU-accelerated linear algebra and matrix operations on CUDA 11 hardware.
Provides CUDA solver native runtime libraries for GPU-accelerated linear algebra operations on NVIDIA hardware.
Provides NVIDIA CUSPARSE native runtime libraries for GPU-accelerated sparse matrix operations in CUDA applications.
Install only if you have NVIDIA GPU hardware and the CUDA toolkit already set up; this is a runtime library, not a standalone tool.
Provides CUSPARSE native runtime libraries for CUDA 11, enabling sparse matrix operations on NVIDIA GPUs.
Provides NVIDIA CUSPARSE native runtime libraries for GPU-accelerated sparse matrix operations on CUDA 12 systems.
Provides NVIDIA's cuSPARSELt CUDA library for high-performance sparse matrix-matrix multiplication on NVIDIA GPUs with structured sparsity patterns.
However, the proprietary license requires verification against your project's licensing terms, and the aging maintenance status warrants checking NVIDIA's official…
Provides NVIDIA's CUDA library for high-performance sparse matrix-matrix multiplication on GPUs with structured sparsity, supporting mixed-precision computation across multiple data types.
However, verify NVIDIA's proprietary license terms for your use case, confirm your GPU architecture is supported (SM 8.0+), and ensure CUDA 13 is installed and…
Provides a Python native interface for writing high-performance CUDA kernels targeting NVIDIA Tensor Cores on Ampere, Hopper, and Blackwell architectures using CuTe DSL concepts without requiring C++ expertise.
However, verify NVIDIA's proprietary license terms for your use case, and be aware that the API may change before the summer 2026 beta graduation.
Provides Python interfaces for writing high-performance CUDA kernels using CUTLASS DSL concepts without C++ expertise, targeting NVIDIA Tensor Cores on Ampere, Hopper, and Blackwell architectures.
However, verify the unclear license terms before production use, and be aware the package is in public beta—expect potential API changes before summer 2026 graduation.
Provides a Python interface for writing high-performance CUDA kernels using CuTe DSL abstractions, targeting NVIDIA Tensor Cores on Ampere, Hopper, and Blackwell architectures without requiring deep C++ expertise.
However, the public beta status and unclear license terms warrant caution for production use—verify licensing and test stability for your workload before committing.
Provides a Python DSL for writing high-performance CUDA kernels using CUTLASS and CuTe concepts, targeting NVIDIA Tensor Cores on Ampere, Hopper, and Blackwell architectures.
However, verify the unclear license terms for your use case, and note that the package is in beta—expect potential API changes before summer 2026.
Provides a Python DSL for writing high-performance CUDA kernels using CUTLASS concepts like layouts, tensors, and hardware atoms without requiring deep C++ expertise.
However, the unclear license status and beta maturity (target graduation summer 2026) warrant verification of licensing terms and stability requirements before…
Provides GPU-accelerated lossless data compression and decompression using NVIDIA CUDA 12, enabling high-performance compression operations in applications through a generic compression interface.
However, verify that your platform (Linux aarch64/x86_64 or Windows amd64) is supported and that the proprietary NVIDIA SDK license terms are acceptable for your use…
Evaluates language models against standardized benchmarks (MMLU, IFEval, MATH, etc.) by sending requests to model API endpoints and collecting results.
Device-side API extensions for performing mathematical calculations inside CUDA kernels, enabling fused numerical operations to reduce latency and improve application performance.
Python wrapper around NVIDIA's Management Library that exposes GPU monitoring and management functions through a Pythonic interface, converting C library calls to Python methods and error codes to exceptions.
Install it if you need to query or monitor NVIDIA GPUs from Python; the only blocker is requiring NVIDIA drivers and NVML on the target system.
Provides Python bindings to NVIDIA's Management Library (NVML) for querying and monitoring GPU devices, including memory usage, driver version, and device information.
Applies state-of-the-art model optimization techniques—quantization, pruning, Neural Architecture Search, distillation, speculative decoding, and sparsity—to accelerate PyTorch, Hugging Face, and ONNX models for deployment.
Integrates enterprise agents with data sources and tools across any agentic framework, treating agents and workflows as composable function calls.
Not recommended for first-time NeMo Agent Toolkit users; the documentation suggests installing from the source repository first to learn the toolkit.
Provides ATIF schema models for the NVIDIA NeMo Agent Toolkit, enabling structured representation and validation of agent evaluation and interaction data.
NVIDIA NeMo Agent Toolkit is a framework-agnostic library for building, composing, and deploying enterprise agents with integrated tools, data sources, and observability.
Provides evaluation components and CLI commands for the NVIDIA NeMo Agent Toolkit, enabling assessment of agent performance.
Install only if you need evaluation capabilities; it is a specialized subpackage.
Integrates LangChain and LangGraph with NVIDIA's NeMo Agent Toolkit to build agentic AI applications using multiple language model providers and retrieval-augmented generation.
However, verify whether this subpackage can be used independently or requires the parent toolkit, and whether the 20 bundled dependencies align with your actual needs.
Integrates MCP (Model Context Protocol) servers with NVIDIA NeMo Agent Toolkit, allowing agent workflows to discover and call external MCP tools as functions.
Adds OpenTelemetry observability instrumentation to the NVIDIA NeMo Agent Toolkit, enabling trace collection and export for agent applications.
Install only if your workflow involves the NeMo Agent Toolkit; it is not a standalone observability library.
Provides NVIDIA's collective communication library (NCCL) runtime for GPU-accelerated all-reduce, all-gather, reduce, broadcast, and reduce-scatter operations optimized for CUDA 11.