---
id: xinference-client
version: "3.1.0"
license: Apache License 2.0
license_treatment: permissive
maintenance: active
---

# xinference-client — Client for Xinference

License: permissive · Maintenance: active · Downloads: 204.3K/mo

## What it is and what it does

xinference-client is a Python HTTP client for the Xinference model serving platform, allowing developers to programmatically launch, retrieve, and query language models running on a Xinference server. It wraps REST API calls into a Python-friendly interface, handling model lifecycle management (launch, get) and inference requests (chat completions with configurable generation parameters).

The package is built on standard HTTP libraries (requests, aiohttp) and integrates with pydantic for data validation. It's designed for scenarios where models are hosted on a separate Xinference server and accessed remotely, rather than for local model execution. The client abstracts away HTTP details and returns structured responses (chat completion objects with token counts, finish reasons, and assistant messages).

Use it for:

- Launch and query language models on a remote Xinference server from a Python application without direct model management.
- Build chatbot or conversational AI applications that delegate inference to a centralized Xinference deployment.
- Integrate model inference into data pipelines or batch processing workflows using a simple client API.
- Prototype or test different language models by swapping model names without changing client code.
- Monitor and manage model lifecycle (launch, retrieve state) programmatically across multiple applications.

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

A Python client library for connecting to and managing Xinference model servers over HTTP, enabling programmatic access to launch, retrieve, and interact with language models.

Yes, if you are running a Xinference server and need a Python client to access it. The package is actively maintained, has low install friction, carries no known vulnerabilities, and uses a permissive license. It is not useful standalone—it requires a running Xinference server to connect to.

## Install

pip install xinference-client
uv add xinference-client
poetry add xinference-client

## Installing xinference-client

Before you install: Low friction installation with a pure Python wheel and four common dependencies (requests, aiohttp, typing-extensions, pydantic). Actively maintained with recent releases; last commit 2026-07-31.

License in practice: Apache License 2.0 is permissive, allowing commercial and private use with minimal restrictions—suitable for most projects without licensing concerns.

Quickstart:

pip install xinference-client

from xinference_client import RESTfulClient as Client

client = Client("http://localhost:9997")
model_uid = client.launch_model(model_name="chatglm2")
model = client.get_model(model_uid)
model.chat("What is the largest animal?", chat_history=[], generate_config={"max_tokens": 1024})

Requires a running Xinference server accessible at the specified URL (e.g., http://localhost:9997).

Verify before relying:

- Whether the package supports async/await patterns beyond aiohttp dependency presence.
- Supported Xinference server versions and compatibility guarantees.
- Whether model launch/retrieval operations have timeout or retry mechanisms.

## Package facts

- License: Apache License 2.0 (permissive)
- Python support: unspecified
- Install friction: low
- Maintenance: active
- Downloads: 204.3K/month (top 15,000 on PyPI)
- Known vulnerabilities: none known

## Tags

xinference client library, restful model server client, language model http client, xinference api wrapper, model management client, chat completion client, remote model inference, model-serving, rest-client, language-models

[View on SkillFed](https://skillfed.io/packages/xinference-client) · [View on PyPI](https://pypi.org/project/xinference-client/)