{"categories":[{"label":"Artificial Intelligence","url":"https://skillfed.io/packages/category/scientific-engineering-artificial-intelligence/9"}],"enrichment":{"capability":"Runs small and large language models and multi-modal models on-device and in the cloud using ONNX Runtime as the execution backend.","skillfed_tags":["model-inference","edge-ai","onnx"],"use_cases":["Deploy a quantized LLM locally on a laptop or edge device for inference without cloud API calls","Run multi-modal models in a Python application using pre-converted ONNX weights","Integrate on-device SLM inference into a web service or backend for low-latency responses","Experiment with different model architectures by swapping ONNX model files without code changes","Build a chatbot or text-generation pipeline that runs entirely on-premises for privacy"],"what_it_does":"ONNX Runtime GenAI is a Python library that wraps ONNX Runtime to simplify inference of language models and multi-modal models. It abstracts away the complexity of loading ONNX model files and managing inference sessions, letting you focus on running models rather than configuring the runtime. The library depends on numpy and onnxruntime, and is distributed as pre-built wheels across Windows, Linux, and macOS on both x86 and ARM architectures.\n\nThe package is intended for scenarios where you want to deploy or experiment with LLMs and SLMs locally or in cloud environments without relying on external APIs. It bridges the gap between raw ONNX Runtime (a low-level inference engine) and high-level model frameworks, offering a middle ground for developers who have ONNX-converted models and want a straightforward Python interface to run them.","worth_installing":"Yes, if you have ONNX-converted models and want a straightforward Python interface to run them on-device or in the cloud. The library is actively maintained, has no known vulnerabilities, and offers broad platform coverage. Install only if you already have or plan to convert models to ONNX format; it is not a model hub or download tool."},"id":"onnxruntime-genai","links":{"html":"https://skillfed.io/packages/onnxruntime-genai","md":"https://skillfed.io/packages/onnxruntime-genai.md","pypi":"https://pypi.org/project/onnxruntime-genai/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2026-08-06","license_spdx":null,"license_treatment":"permissive","name":"onnxruntime-genai","python_support":"unspecified","summary":"ONNX Runtime GenAI"},"popularity":{"monthly_downloads":131825,"position":11573,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"0.15.2"}