optimum-onnx

What it is and what it does

optimum-onnx bridges Hugging Face transformer models and the ONNX ecosystem, enabling you to export PyTorch checkpoints to ONNX format and run them via ONNX Runtime. It provides command-line tooling for export with optional graph optimization and quantization, plus Python classes that wrap ONNX Runtime to maintain a familiar Hugging Face API surface for inference.

The package targets developers who want faster or more portable inference than standard PyTorch, particularly for deployment scenarios where ONNX Runtime's performance characteristics or cross-platform support matter. It depends on transformers, optimum, and onnx, and supports Python 3.9 through 3.13. The project is actively maintained but marked Pre-Alpha, reflecting its early maturity.

Use it for:

• Export a transformer to ONNX and quantize it for faster CPU or GPU inference
• Deploy a model to environments where ONNX Runtime is preferred over PyTorch for size or performance
• Run inference on exported ONNX models using the ORTModelForXXX classes with minimal code changes
• Optimize and benchmark models in ONNX format for production serving

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Exports Hugging Face transformer models to ONNX format and runs them via ONNX Runtime for optimized inference.

Yes, if you need to export Hugging Face models to ONNX for deployment or performance optimization. The low install friction, active maintenance, permissive license, and zero known vulnerabilities make it safe to try. Pre-Alpha status means the API may shift, but the project is backed by Hugging Face and already handles real models.

Install

optimum-onnx on PyPI

pip install optimum-onnx

uv add optimum-onnx

poetry add optimum-onnx

Installing optimum-onnx

Before you install

Low friction install with a pure Python wheel. Active maintenance with recent commits and no known vulnerabilities. Early stage (Pre-Alpha) but backed by Hugging Face infrastructure.

License in practice

Apache-2.0 permissive license allows commercial and private use with minimal restrictions.

Quickstart

pip install "optimum-onnx[onnxruntime]"

from optimum.onnxruntime import ORTModelForCausalLM
from transformers import AutoTokenizer

model = ORTModelForCausalLM.from_pretrained("onnx-community/Llama-3.2-1B", subfolder="onnx")
tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.2-1B")

Requires Python 3.9 or later. GPU inference requires CUDA and cuDNN; avoid installing both onnxruntime and onnxruntime-gpu simultaneously.

Verify before relying

• Performance gains (latency, throughput, memory) compared to PyTorch inference on typical models
• Supported model architectures beyond the examples shown
• Quantization options available and their impact on model accuracy
• Compatibility with Diffusers, Timm, and Sentence Transformers models as mentioned

Package facts

Evidence: optimum_onnx-0.1.0-py3-none-any.whl

Keywords: transformers, quantization, inference, onnx, onnxruntime

Tags

More Artificial Intelligence packages