azureml-inference-server-http

What it is and what it does

Azure Machine Learning Inference HTTP Server is a Flask-based HTTP server designed to host and serve machine learning models deployed through Azure ML. It wraps user-defined scoring scripts (Python functions decorated with input/output schemas) and exposes them as HTTP endpoints, handling request routing, response formatting, and error management. The server includes built-in observability through OpenTelemetry and Azure Monitor integration, allowing operators to collect metrics, traces, and logs for production monitoring.

The package abstracts away much of the HTTP server boilerplate, letting data scientists focus on the scoring logic while the server handles Flask configuration, CORS, health checks, and request/response validation. It supports both gunicorn (Linux/macOS) and waitress (Windows) as WSGI servers and includes features like configurable health check ports and JSON-based server configuration through environment variables.

Use it for:

• Deploy a trained scikit-learn or TensorFlow model as a production HTTP endpoint in Azure ML with automatic request validation and response formatting.
• Monitor inference latency and error rates in production by leveraging built-in OpenTelemetry metrics and Azure Monitor export.
• Host a batch of related scoring functions (e.g., preprocessing, model inference, postprocessing) as separate endpoints on a single server instance.
• Migrate existing Flask-based scoring scripts to Azure ML without rewriting the core logic, using the server's Flask compatibility layer.
• Configure separate health check and inference ports to isolate monitoring traffic from production scoring requests.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Provides an HTTP server for serving Azure Machine Learning models with built-in support for Flask-based scoring scripts, OpenTelemetry observability, and Azure Monitor integration.

Yes, if you are deploying models through Azure ML and need a standardized, Microsoft-supported HTTP server for inference. The package is actively maintained with recent dependency upgrades (Pydantic 2.12 in version 1.5.1) and has no known security vulnerabilities. The aging maintenance status reflects a stable, mature codebase rather than abandonment. However, review the unclear proprietary license terms before use, and ensure your scoring scripts are compatible with Flask 2.x and Pydantic 2.0+.

Install

azureml-inference-server-http on PyPI

pip install azureml-inference-server-http

uv add azureml-inference-server-http

poetry add azureml-inference-server-http

Installing azureml-inference-server-http

Before you install

Low install friction with a pure-Python wheel. Maintenance status is aging (239 days since last release), though the package receives periodic updates for dependency upgrades and security patches.

License in practice

License treatment is unclear; the raw license URL points to a Microsoft proprietary license at https://aka.ms/azureml-sdk-license with no SPDX identifier. Review Microsoft's terms before use in proprietary or open-source contexts.

Quickstart

pip install azureml-inference-server-http

from azureml_inference_server_http.server import Server
from inference_schema.schema_decorators import input_schema, output_schema

# Define your score function with decorators
@input_schema('data', ...)
@output_schema(...)
def score(data):
    return prediction

# Server loads and runs the scoring script

Requires Python 3.9 or later; Python 3.8 support was dropped in version 1.4.0. Pydantic 2.0+ is required as of version 1.1.0, which is a breaking change from Pydantic 1.x.

Verify before relying

• Whether the package works with custom Flask 2.x request handling beyond the documented compatibility layer removal
• Performance characteristics and throughput limits under production load
• Whether OpenTelemetry integration requires additional Azure Monitor configuration beyond the exporter dependency

Package facts

Evidence: azureml_inference_server_http-1.5.1-py3-none-any.whl

Tags

More Artificial Intelligence packages