---
id: sit4onnx
version: "1.0.10"
license: MIT License
license_treatment: permissive
maintenance: aging
---

# sit4onnx — Tools for simple inference testing using TensorRT, CUDA and OpenVINO CPU/GPU and CPU providers. Simple Inference Test for ONNX.

License: permissive · Maintenance: aging · Downloads: 82.8K/mo

## What it is and what it does

sit4onnx is a command-line and Python API tool for testing ONNX model inference across different hardware accelerators and execution providers. It accepts an ONNX model file and runs it repeatedly to measure average inference latency, with options to specify batch sizes, fixed input shapes, and custom test data via numpy arrays or .npy files. The tool supports TensorRT, CUDA, OpenVINO (CPU and GPU), and CPU-only execution, and can output performance profiling results and inference outputs to files.

The package is designed for model validation and benchmarking workflows where you need to verify that a model runs correctly on a target platform and measure its throughput or latency. It handles models with dynamic input shapes by allowing you to specify fixed dimensions, and it returns the final inference result as a numpy array when called from Python code.

Use it for:

- Benchmark an ONNX model's inference speed on TensorRT or CUDA before deploying to production.
- Validate that a model with dynamic input shapes runs correctly by specifying fixed dimensions and test data.
- Compare inference latency across different execution providers (CPU vs. GPU vs. TensorRT) on the same model.
- Profile a model's performance and export timing results to JSON for analysis and reporting.
- Test a model end-to-end with custom numpy array inputs to verify output correctness before integration.

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

Runs inference tests on ONNX models using TensorRT, CUDA, OpenVINO, or CPU execution providers, measuring performance and validating model outputs.

Yes, if you need to test and benchmark ONNX models across different execution providers. The low install friction and permissive license make it a practical choice. However, note that the package is aging (last commit 341 days ago) and you will need to separately install the execution provider libraries (TensorRT, OpenVINO, etc.) that you intend to use.

## Install

pip install sit4onnx
uv add sit4onnx
poetry add sit4onnx

## Installing sit4onnx

Before you install: Low install friction with no runtime dependencies. Maintenance shows aging status—last commit was 341 days ago—but the repository remains active and unarchived.

License in practice: MIT License permits commercial and private use with minimal restrictions, making it suitable for most projects.

Quickstart:

pip install sit4onnx

from sit4onnx import inference

results = inference(
    input_onnx_file_path="model.onnx",
    batch_size=10,
    test_loop_count=10,
    onnx_execution_provider="tensorrt"
)

Requires the target execution provider library (TensorRT, CUDA, OpenVINO, or onnxruntime CPU) to be installed separately; sit4onnx does not bundle these dependencies.

Verify before relying:

- Whether onnxruntime or execution provider libraries (TensorRT, OpenVINO) are automatically installed or must be provided separately.
- Supported versions of TensorRT, CUDA, and OpenVINO beyond the requirement that Python be >=3.6.
- Whether profiling output format and structure are documented beyond the mention of .json files.

## Package facts

- License: MIT License (permissive)
- Python support: supports_current
- Install friction: low
- Maintenance: aging
- Downloads: 82.8K/month (top 15,000 on PyPI)
- Known vulnerabilities: none known

## Tags

onnx model inference testing, tensorrt cuda openvino benchmark, onnx execution provider test, neural network inference validation, model performance profiling onnx, batch inference testing tool, onnx model performance measurement, onnx-inference, model-benchmarking, gpu-acceleration

[View on SkillFed](https://skillfed.io/packages/sit4onnx) · [View on PyPI](https://pypi.org/project/sit4onnx/)