nixl-cu13

What it is and what it does

NIXL is a Python library that wraps NVIDIA's Inference Xfer Library, a C++ communication acceleration layer designed for distributed AI inference. It abstracts over different memory types (CPU, GPU) and storage backends (file, block, object store) through a plugin system, allowing inference frameworks like NVIDIA Dynamo to optimize data movement without reimplementing transport logic.

The package ships as a prebuilt wheel for Linux (Python 3.10–3.14, aarch64 and x86_64) with both CUDA 12 and CUDA 13 backends included; the correct backend is selected automatically at runtime based on the CUDA version reported by PyTorch. It depends on torch and numpy. Development is active, and the library is permissively licensed under MIT and Apache-2.0.

Use it for:

• Accelerate tensor transfers between GPUs in multi-GPU inference deployments running NVIDIA Dynamo
• Abstract GPU memory and storage operations to simplify distributed inference framework implementation
• Benchmark and profile point-to-point communication performance in inference workloads using NIXLBench
• Integrate custom storage backends (file, block, object store) into inference pipelines via NIXL's plugin architecture
• Enable metadata coordination across distributed inference nodes using ETCD integration

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

NIXL provides Python bindings to accelerate point-to-point communications in AI inference frameworks, abstracting CPU and GPU memory alongside file, block, and object storage through a modular plugin architecture.

Yes, if you are building or deploying distributed AI inference on Linux with PyTorch and need to optimize inter-GPU or GPU-storage communication. The permissive dual license, active maintenance, and prebuilt wheels for modern Python versions lower friction. No if you are on macOS, Windows, or not using inference frameworks that benefit from explicit communication acceleration.

Install

nixl-cu13 on PyPI

pip install nixl-cu13

uv add nixl-cu13

poetry add nixl-cu13

Installing nixl-cu13

Before you install

Medium install friction: prebuilt wheels target Linux only (manylinux_2_28, aarch64 and x86_64) for Python 3.10–3.14. Requires torch and numpy at runtime. Package is actively maintained with recent releases.

License in practice

Dual-licensed under MIT and Apache-2.0 (permissive). Both licenses allow commercial and private use with minimal restrictions, making the package safe for most deployment scenarios.

Quickstart

pip install nixl-cu13

import nixl
# Backend selected automatically based on CUDA version from PyTorch
# See docs/python_api.md for detailed usage examples

Linux-only; macOS and Windows not supported. Requires CUDA 13 runtime and PyTorch with matching CUDA version for automatic backend selection.

Verify before relying

• Specific performance gains or benchmarks for typical inference workloads compared to direct PyTorch communication
• Compatibility matrix with specific PyTorch versions and CUDA 13 minor releases
• Whether the package works with PyTorch built against CUDA 12 despite being named cu13

Package facts

Evidence: nixl_cu13-1.4.0-cp310-cp310-manylinux_2_28_aarch64.whl; nixl_cu13-1.4.0-cp310-cp310-manylinux_2_28_x86_64.whl; nixl_cu13-1.4.0-cp311-cp311-manylinux_2_28_aarch64.whl; nixl_cu13-1.4.0-cp311-cp311-manylinux_2_28_x86_64.whl; nixl_cu13-1.4.0-cp312-cp312-manylinux_2_28_aarch64.whl; nixl_cu13-1.4.0-cp312-cp312-manylinux_2_28_x86_64.whl; nixl_cu13-1.4.0-cp313-cp313-manylinux_2_28_aarch64.whl; nixl_cu13-1.4.0-cp313-cp313-manylinux_2_28_x86_64.whl; nixl_cu13-1.4.0-cp314-cp314-manylinux_2_28_aarch64.whl; nixl_cu13-1.4.0-cp314-cp314-manylinux_2_28_x86_64.whl

Tags

More Artificial Intelligence packages

Further reading