inference-models

What it is and what it does

inference-models is a unified inference library for running computer vision models trained on Roboflow or built locally. It abstracts away backend selection (PyTorch, ONNX, TensorRT, Hugging Face) and model loading, letting you call a single API regardless of the underlying architecture—whether you're running object detection, segmentation, OCR, or vision-language tasks. The library includes pretrained models (RFDetr, SAM, Florence, DocTR, EasyOCR, YOLO, and others) and supports custom models from the Roboflow platform via API key, as well as local model implementations.

The package is designed for production use and integrates with supervision for annotation and post-processing. It requires Python 3.10–3.12 and pulls in 44 runtime dependencies including numpy, torch, torchvision, transformers, and diffusers, making it best suited for environments with adequate compute and storage. The library reached its first stable release at version 0.19.0 and is actively maintained, though the API may still evolve.

Use it for:

• Load and run a pretrained Roboflow model (e.g., RFDetr) on images without writing backend-specific code.
• Deploy a custom model trained on Roboflow platform in production by loading it with an API key.
• Run local custom model implementations from a directory for production deployment of non-standard architectures.
• Chain inference with supervision to visualize predictions (bounding boxes, masks, etc.) on images.
• Switch between inference backends (PyTorch to TensorRT, for example) without changing application code.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Loads and runs computer vision models from Roboflow or local sources using PyTorch, ONNX, TensorRT, or Hugging Face backends, with a unified API across different model architectures.

Yes, if you are building computer vision applications on Roboflow models or need a unified inference API across multiple backends and model types. The heavy dependency footprint and Python version cap (3.10–3.12) make it unsuitable for lightweight or resource-constrained environments. License treatment is unclear—verify model-specific restrictions before production deployment. No known vulnerabilities as of the query date.

Install

inference-models on PyPI

pip install inference-models

uv add inference-models

poetry add inference-models

Installing inference-models

Before you install

Low install friction with a pure-Python wheel. Active maintenance as of release date. Requires Python 3.10–3.12 and brings in 44 runtime dependencies including numpy, torch, torchvision, transformers, and diffusers; this is a heavy stack suitable for systems with compute capacity.

License in practice

License treatment is unclear in the metadata; the description states Apache 2.0 for the package itself but notes individual models may have different licenses. Verify the actual license terms and model-specific restrictions before production use.

Quickstart

pip install inference-models

from inference_models import AutoModel

model = AutoModel.from_pretrained("rfdetr-base")
predictions = model(image)

Requires Python 3.10–3.12. Heavy dependencies (torch, torchvision, transformers) demand significant disk and memory; GPU optional but recommended for speed.

Verify before relying

• Whether the Apache 2.0 license applies uniformly to all bundled model weights and architectures, or if model-specific restrictions apply at runtime.
• Performance characteristics and latency benchmarks across the supported backends (PyTorch, ONNX, TensorRT, Hugging Face).
• Whether the 44 runtime dependencies can be selectively installed via extras, as the description mentions a 'composable extras system' but the fact sheet does not detail which extras are available.

Package facts

Evidence: inference_models-0.35.2-py3-none-any.whl

Tags

More Artificial Intelligence packages