azure-ai-inference

What it is and what it does

Azure AI Inference is a Python client library that abstracts authentication and API calls to multiple AI model deployment services under a single interface. It supports GitHub Models (free-tier), Azure AI Foundry Serverless and Managed Compute endpoints, and Azure OpenAI Service, allowing developers to write inference code once and switch between providers or compare model performance without rewriting client logic.

The library provides ChatCompletionsClient and EmbeddingsClient classes for synchronous and asynchronous operations. It handles credential management via API keys, GitHub tokens, or Entra ID, and manages REST API versioning. Runtime dependencies are minimal (isodate, azure-core, typing-extensions), and the package is in beta but actively maintained with recent releases.

Use it for:

• Compare inference quality across different AI models using identical Python code and a single client.
• Build applications on GitHub Models for prototyping, then migrate to Azure OpenAI or Foundry endpoints without rewriting client code.
• Implement chat completions or text embeddings against Azure AI Foundry deployments with automatic credential handling.
• Evaluate multiple model providers in A/B testing scenarios by swapping endpoints and credentials.
• Generate text embeddings from various Azure-hosted models for vector search or semantic similarity tasks.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Unified client library for chat completions, text embeddings, and image embeddings across GitHub Models, Azure AI Foundry deployments, and Azure OpenAI Service.

Yes, with conditions. Install if you are building on Azure AI services and want a unified client across GitHub Models, Foundry, or Azure OpenAI—it eliminates provider-specific boilerplate. The library is actively maintained, has no known vulnerabilities, and uses a permissive MIT license. However, it is still in beta (1.0.0b9), so expect potential API changes before a stable release.

Install

azure-ai-inference on PyPI

pip install azure-ai-inference

uv add azure-ai-inference

poetry add azure-ai-inference

Installing azure-ai-inference

Before you install

Low install friction with three lightweight runtime dependencies. Active maintenance with recent releases; last commit 2026-08-14. Beta status (Development Status :: 4) means the API surface may shift before a stable release.

License in practice

MIT License permits commercial and private use with minimal restrictions, making it suitable for production applications.

Quickstart

pip install azure-ai-inference

from azure.ai.inference import ChatCompletionsClient
from azure.core.credentials import AzureKeyCredential

client = ChatCompletionsClient(
    endpoint="https://models.inference.ai.azure.com",
    credential=AzureKeyCredential(github_token),
    model="mistral-large"
)

Requires Python 3.8 or later. Endpoint URL and credentials (API key, GitHub token, or Entra ID) must be provided; for Entra ID, azure-identity package must be installed separately.

Verify before relying

• Whether the REST API version 2024-05-01-preview is stable or subject to breaking changes before 1.0.0 GA.
• Performance characteristics and rate limits when comparing across GitHub Models, Serverless, and Azure OpenAI endpoints.
• Whether async support requires aiohttp as an optional dependency or if it is bundled.

Package facts

Evidence: azure_ai_inference-1.0.0b9-py3-none-any.whl

Keywords: azure, azure sdk

Tags

More Artificial Intelligence packages