xinference-client

What it is and what it does

xinference-client is a Python HTTP client for the Xinference model serving platform, allowing developers to programmatically launch, retrieve, and query language models running on a Xinference server. It wraps REST API calls into a Python-friendly interface, handling model lifecycle management (launch, get) and inference requests (chat completions with configurable generation parameters).

The package is built on standard HTTP libraries (requests, aiohttp) and integrates with pydantic for data validation. It's designed for scenarios where models are hosted on a separate Xinference server and accessed remotely, rather than for local model execution. The client abstracts away HTTP details and returns structured responses (chat completion objects with token counts, finish reasons, and assistant messages).

Use it for:

• Launch and query language models on a remote Xinference server from a Python application without direct model management.
• Build chatbot or conversational AI applications that delegate inference to a centralized Xinference deployment.
• Integrate model inference into data pipelines or batch processing workflows using a simple client API.
• Prototype or test different language models by swapping model names without changing client code.
• Monitor and manage model lifecycle (launch, retrieve state) programmatically across multiple applications.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

A Python client library for connecting to and managing Xinference model servers over HTTP, enabling programmatic access to launch, retrieve, and interact with language models.

Yes, if you are running a Xinference server and need a Python client to access it. The package is actively maintained, has low install friction, carries no known vulnerabilities, and uses a permissive license. It is not useful standalone—it requires a running Xinference server to connect to.

Install

xinference-client on PyPI

pip install xinference-client

uv add xinference-client

poetry add xinference-client

Installing xinference-client

Before you install

Low friction installation with a pure Python wheel and four common dependencies (requests, aiohttp, typing-extensions, pydantic). Actively maintained with recent releases; last commit 2026-07-31.

License in practice

Apache License 2.0 is permissive, allowing commercial and private use with minimal restrictions—suitable for most projects without licensing concerns.

Quickstart

pip install xinference-client

from xinference_client import RESTfulClient as Client

client = Client("http://localhost:9997")
model_uid = client.launch_model(model_name="chatglm2")
model = client.get_model(model_uid)
model.chat("What is the largest animal?", chat_history=[], generate_config={"max_tokens": 1024})

Requires a running Xinference server accessible at the specified URL (e.g., http://localhost:9997).

Verify before relying

• Whether the package supports async/await patterns beyond aiohttp dependency presence.
• Supported Xinference server versions and compatibility guarantees.
• Whether model launch/retrieval operations have timeout or retry mechanisms.

Package facts

Evidence: xinference_client-3.1.0-py3-none-any.whl

Tags

More Libraries packages