---
id: nixl-cu12
version: "1.4.0"
license: MIT AND Apache-2.0
license_treatment: permissive
maintenance: active
---

# nixl-cu12 — NIXL Python API

License: permissive · Maintenance: active · Downloads: 419.1K/mo

## What it is and what it does

NIXL is a Python library that wraps NVIDIA's Inference Xfer Library, a C++ communication acceleration layer designed for AI inference. It provides an abstraction over memory types (CPU, GPU) and storage backends (file, block, object store) through pluggable transport modules, with the goal of reducing data movement overhead in distributed inference scenarios.

The library is intended for integration into inference frameworks like NVIDIA Dynamo rather than direct application use. It ships with pre-built wheels that automatically detect and use the correct CUDA backend (12 or 13) at runtime. Installation is straightforward on Linux via pip, but the package is tightly coupled to PyTorch and CUDA, making it suitable only for GPU-accelerated inference workloads on supported platforms.

Use it for:

- Accelerating tensor transfers between GPU and CPU memory in multi-GPU inference deployments.
- Reducing latency in distributed inference by optimizing point-to-point communication patterns.
- Abstracting storage I/O (file, object store, block device) for inference data staging and caching.
- Integrating into inference frameworks that need vendor-neutral transport abstraction over UCX and other backends.

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

NIXL provides Python bindings to accelerate point-to-point communications in AI inference frameworks, abstracting over CPU/GPU memory and various storage backends through a modular plugin architecture.

Yes, if you are building or integrating an AI inference framework on Linux with GPU support and need optimized data movement abstractions. No, if you are running inference directly in PyTorch without framework-level integration—the library is designed for framework authors, not end-user applications. Requires Linux, Python 3.10+, torch, and numpy; no known vulnerabilities as of the latest release.

## Install

pip install nixl-cu12
uv add nixl-cu12
poetry add nixl-cu12

## Installing nixl-cu12

Before you install: Medium install friction: requires Python 3.10+, torch, and numpy as runtime dependencies. Wheels are pre-built for recent Python versions (3.10–3.14) on x86_64 and aarch64 Linux only. Active maintenance as of the latest release.

License in practice: Dual-licensed under MIT and Apache-2.0 (permissive). Both are business-friendly open-source licenses with minimal restrictions on use or modification.

Quickstart:

pip install nixl-cu12

import nixl
# NIXL automatically selects the correct CUDA backend at runtime based on PyTorch's CUDA version

Linux only (Ubuntu 22.04/24.04 or Fedora tested); macOS and Windows not supported. Requires CUDA 12 runtime and PyTorch with matching CUDA version.

Verify before relying:

- Specific performance gains or latency improvements over direct tensor transfers in real inference workloads.
- Compatibility matrix with specific PyTorch versions and inference frameworks beyond the general CUDA 12 support.
- Whether the plugin architecture is accessible to end users or primarily for framework integrators.

## Package facts

- License: MIT AND Apache-2.0 (permissive)
- Python support: supports_current
- Install friction: medium
- Maintenance: active
- Downloads: 419.1K/month (top 15,000 on PyPI)
- Known vulnerabilities: none known

## Tags

GPU communication acceleration, inference data transfer optimization, AI framework memory abstraction, distributed inference networking, PyTorch GPU data movement, inference framework transport layer, accelerated tensor communication, gpu-acceleration, distributed-inference, cuda-12

[View on SkillFed](https://skillfed.io/packages/nixl-cu12) · [View on PyPI](https://pypi.org/project/nixl-cu12/)