baseten-performance-client

What it is and what it does

baseten-performance-client is a Python wrapper around a Rust-based HTTP client designed to send high-volume concurrent requests to Baseten's embedding, reranking, and classification APIs. It releases Python's GIL during network I/O, enabling true parallelism for batch operations. The library supports both synchronous and asynchronous usage patterns, with configurable batching, concurrency limits, request timeouts, and connection pooling via HTTP/1.1 or HTTP/2.

The package is built on pyo3, reqwest, and tokio, and works not only with Baseten endpoints but with any HTTP API accepting JSON POST requests. It offers a RequestProcessingPreference object to tune batch size, concurrent request limits, character limits per request, hedge delays, and total operation timeouts. The library includes response objects for embeddings, reranking, and classification, with optional NumPy integration for embeddings.

Use it for:

• Batch embedding generation for large document corpora using Baseten, OpenAI, or Mixedbread APIs with minimal latency.
• Concurrent reranking of search results or document lists with configurable batch sizes and timeouts.
• Generic high-throughput POST request batching to any JSON API endpoint outside Baseten.
• Async/await integration in event-driven applications needing non-blocking embedding or classification calls.
• Performance-critical pipelines where GIL release and connection pooling reduce end-to-end latency.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

A high-performance Python HTTP client for concurrent requests to Baseten embeddings, reranking, and classification endpoints, built with Rust and Tokio to release the GIL during I/O.

Yes, if you need high-throughput concurrent HTTP requests to embeddings or classification APIs and can tolerate medium install friction. The Rust backend and active maintenance (recent release, 1188 stars) suggest reliability. Verify the license claim before use in proprietary code, and confirm the >1200 rps benchmark applies to your workload. The single runtime dependency (requests) is a plus.

Install

baseten-performance-client on PyPI

pip install baseten-performance-client

uv add baseten-performance-client

poetry add baseten-performance-client

Installing baseten-performance-client

Before you install

Medium install friction due to compiled wheels across many platforms (cp313t, cp38-abi3, musllinux variants). Active maintenance with recent releases and 1188 GitHub stars. Single runtime dependency (requests) keeps the footprint small.

License in practice

License treatment is unclear—the description states MIT licensing, but the fact sheet records no SPDX identifier or license_raw value. Verify the actual license before relying on this package in proprietary or GPL-licensed projects.

Quickstart

pip install baseten_performance_client

import os
from baseten_performance_client import PerformanceClient

api_key = os.environ.get("BASETEN_API_KEY")
client = PerformanceClient(base_url="https://model-xxx.api.baseten.co/environments/production/sync", api_key=api_key)
response = client.embed(input=["Hello world"], model="my_model")

Requires Python >=3.8; API key and valid Baseten endpoint URL needed for actual use.

Verify before relying

• Whether the MIT license claim in the description is reflected in official metadata and whether it applies to the Rust components.
• Whether the >1200 rps benchmark holds for your specific workload and infrastructure.
• Compatibility with PyPy beyond the classifier claim—whether all features work or only a subset.

Package facts

Evidence: baseten_performance_client-0.1.13-cp313-cp313t-macosx_10_12_x86_64.whl; baseten_performance_client-0.1.13-cp313-cp313t-macosx_11_0_arm64.whl; baseten_performance_client-0.1.13-cp313-cp313t-manylinux_2_17_i686.manylinux2014_i686.whl; baseten_performance_client-0.1.13-cp313-cp313t-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl; baseten_performance_client-0.1.13-cp313-cp313t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl; baseten_performance_client-0.1.13-cp313-cp313t-musllinux_1_2_aarch64.whl; baseten_performance_client-0.1.13-cp313-cp313t-musllinux_1_2_armv7l.whl; baseten_performance_client-0.1.13-cp313-cp313t-musllinux_1_2_i686.whl; baseten_performance_client-0.1.13-cp313-cp313t-musllinux_1_2_x86_64.whl; baseten_performance_client-0.1.13-cp38-abi3-macosx_10_12_x86_64.whl; baseten_performance_client-0.1.13-cp38-abi3-macosx_11_0_arm64.whl; baseten_performance_client-0.1.13-cp38-abi3-manylinux_2_17_i686.manylinux2014_i686.whl; baseten_performance_client-0.1.13-cp38-abi3-manylinux_2_17_ppc64le.manylinux2014_ppc64le.whl; baseten_performance_client-0.1.13-cp38-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.whl; baseten_performance_client-0.1.13-cp38-abi3-manylinux_2_28_aarch64.whl; baseten_performance_client-0.1.13-cp38-abi3-manylinux_2_28_armv7l.whl; baseten_performance_client-0.1.13-cp38-abi3-musllinux_1_2_aarch64.whl; baseten_performance_client-0.1.13-cp38-abi3-musllinux_1_2_armv7l.whl; baseten_performance_client-0.1.13-cp38-abi3-musllinux_1_2_i686.whl; baseten_performance_client-0.1.13-cp38-abi3-musllinux_1_2_x86_64.whl

Tags

More WWW/HTTP packages