model-hosting-container-standards

What it is and what it does

This package standardizes how ML models are deployed to Amazon SageMaker by providing a framework-agnostic Python toolkit built on FastAPI. It handles the boilerplate of setting up `/ping` and `/invocations` endpoints that SageMaker expects, with support for framework integration (vLLM, TensorRT-LLM) via decorators and customer customization via environment variables or custom handler functions. The package also supports LoRA adapter injection, automatic dependency installation from model artifacts, and comprehensive logging and error handling.

Typical use: framework developers register default handlers using `@register_ping_handler` and `@register_invocation_handler` decorators; customers then override behavior by placing a `model.py` script in their model artifact folder with `@custom_ping_handler` or `@custom_invocation_handler` decorators, or by setting environment variables. Handler resolution follows a priority chain (environment variables → customer decorators → function discovery → framework decorators), so customization is straightforward without forking the framework.

Use it for:

• Deploy vLLM or TensorRT-LLM models to SageMaker endpoints with standardized health-check and invocation routes.
• Add LoRA adapter support to model inference by injecting adapter IDs from request headers into model calls.
• Customize model behavior per deployment (e.g., different preprocessing logic) by writing a simple `model.py` without modifying framework code.
• Automatically install and manage model-specific dependencies from a `requirements.txt` in model artifacts before server startup.
• Integrate multiple ML frameworks under a single SageMaker hosting standard without writing repetitive endpoint boilerplate.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Provides standardized FastAPI-based handlers and decorators for deploying ML models (vLLM, TensorRT-LLM) to Amazon SageMaker with unified `/ping` and `/invocations` endpoints.

Yes, if you are deploying ML models to Amazon SageMaker and want to avoid writing boilerplate FastAPI handlers. The package is actively maintained, has low install friction, carries a permissive license, and provides real value through decorator-based handler registration and priority-based customization. No known vulnerabilities. Install it as a dependency in your SageMaker container image or framework integration layer.

Install

model-hosting-container-standards on PyPI

pip install model-hosting-container-standards

uv add model-hosting-container-standards

poetry add model-hosting-container-standards

Installing model-hosting-container-standards

Before you install

Low friction: pure Python wheel with no compiled dependencies. Active maintenance (released 60 days ago), supports current Python versions (3.10–3.14). Seven runtime dependencies (fastapi, httpx, jmespath, pydantic, setuptools, starlette, supervisor) are all widely used and stable.

License in practice

Apache-2.0 (permissive): you may use, modify, and distribute this package freely in commercial and open-source projects, provided you include a copy of the license and state any material changes.

Quickstart

# Install
pip install model-hosting-container-standards

# In your FastAPI/vLLM server code
import model_hosting_container_standards.sagemaker as sagemaker_standards
from fastapi import Request, Response
import json

@sagemaker_standards.register_ping_handler
async def ping(raw_request: Request) -> Response:
    return Response(content='{"status": "healthy"}', media_type="application/json")

@sagemaker_standards.register_invocation_handler
async def invocations(raw_request: Request) -> Response:
    body = json.loads(await raw_request.body())
    return Response(content=json.dumps({"predictions": ["result"]}), media_type="application/json")

Requires Python >= 3.10 and FastAPI >= 0.117.1. Intended for use within a SageMaker container or FastAPI application; not a standalone server.

Verify before relying

• Whether the package is actively maintained beyond the single release 60 days ago (repo URL not provided in metadata).
• Whether all seven runtime dependencies are actually required at runtime or if some are optional/development-only.

Package facts

Evidence: model_hosting_container_standards-0.1.16-py3-none-any.whl

Tags

More Artificial Intelligence packages