---
id: nvidia-modelopt
version: "0.45.0"
license: Apache-2.0
license_treatment: permissive
maintenance: active
---

# nvidia-modelopt — Nvidia Model Optimizer: A unified library of SOTA model optimization techniques like quantization, pruning, Neural Architecture Search (NAS), distillation, speculative decoding, etc. It compresses deep learning models for downstream deployment frameworks like TensorRT-LLM, TensorRT, vLLM, etc. to optimize inference speed.

License: permissive · Maintenance: active · Downloads: 496.4K/mo

## What it is and what it does

NVIDIA Model Optimizer is a library for compressing and accelerating deep learning models through a suite of optimization techniques. It accepts PyTorch, Hugging Face, or ONNX models as input, applies techniques like quantization (FP8, NVFP4), pruning, distillation, and Neural Architecture Search to reduce model size and latency, then exports optimized checkpoints ready for deployment in inference frameworks like TensorRT-LLM, vLLM, and SGLang.

The package is designed for teams optimizing large language models and vision models for production inference. It integrates with NVIDIA Megatron-LM and Hugging Face Accelerate for training-time optimization, and provides Python APIs to compose multiple techniques together. Dependencies include torch, numpy, omegaconf, pydantic, and other ML-stack standards.

Use it for:

- Quantize large language models to lower precision (FP8, NVFP4) for faster inference without retraining.
- Prune and distill LLMs to reduce model size while maintaining accuracy for deployment on resource-constrained hardware.
- Apply post-training quantization to Hugging Face transformer models for immediate export to TensorRT or vLLM.
- Combine multiple optimization techniques (pruning + distillation + quantization) in a single workflow for maximum throughput gains.
- Export optimized diffusion models for faster image generation on NVIDIA GPUs.

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

Applies state-of-the-art model optimization techniques—quantization, pruning, Neural Architecture Search, distillation, speculative decoding, and sparsity—to accelerate PyTorch, Hugging Face, and ONNX models for deployment.

Yes, if you deploy PyTorch or Hugging Face models on NVIDIA hardware and need to reduce inference latency or memory footprint. The library is actively maintained, permissively licensed, and integrates cleanly with the NVIDIA inference ecosystem. Install friction is low and security record is clean. Not applicable for CPU-only or non-NVIDIA deployments.

## Install

pip install nvidia-modelopt
uv add nvidia-modelopt
poetry add nvidia-modelopt

## Installing nvidia-modelopt

Before you install: Low friction install; pure Python wheel. Active maintenance with recent commits and 3442 repository stars. Depends on 15 runtime packages including torch, numpy, and omegaconf—standard for ML workflows.

License in practice: Apache-2.0 permissive license allows commercial use, modification, and distribution with minimal restrictions.

Quickstart:

pip install nvidia-modelopt

from nvidia_modelopt import quantization
import torch

model = torch.nn.Linear(10, 5)
quantized = quantization.quantize(model)

Requires PyTorch and NVIDIA GPU support; quantization and optimization workflows typically need CUDA-capable hardware for practical speedup.

Verify before relying:

- Whether the package requires NVIDIA-specific GPU hardware or works on CPU for development/testing.
- Performance gains and memory reduction percentages claimed in the description (e.g., 2.6× throughput) for typical model sizes.
- Compatibility with specific downstream frameworks (TensorRT-LLM, vLLM, SGLang) beyond the documented integration points.

## Package facts

- License: Apache-2.0 (permissive)
- Python support: supports_current
- Install friction: low
- Maintenance: active
- Downloads: 496.4K/month (top 15,000 on PyPI)
- Known vulnerabilities: none known

## Tags

model quantization and pruning, llm inference optimization, neural architecture search, model compression techniques, tensorrt model optimization, distillation and sparsity, post-training quantization, model-compression, quantization, inference-optimization

[View on SkillFed](https://skillfed.io/packages/nvidia-modelopt) · [View on PyPI](https://pypi.org/project/nvidia-modelopt/)