nixl-cu12
NIXL Python API
Decision gist · record as of 2026-08-14
Yes, if you are building or integrating an AI inference framework on Linux with GPU support and need optimized data movement abstractions. No, if you are running inference directly in PyTorch without framework-level integration—the library is designed for framework authors, not end-user applications. Requires Linux, Python 3.10+, torch, and numpy; no known vulnerabilities as of the latest release.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Linux only (Ubuntu 22.04/24.04 or Fedora tested); macOS and Windows not supported.
- Requires CUDA 12 runtime and PyTorch with matching CUDA version.
- Medium install friction: requires Python 3.10+, torch, and numpy as runtime dependencies.
License · maintenance · safety
MIT AND Apache-2.0 (permissive) — Dual-licensed under MIT and Apache-2.0 (permissive). Both are business-friendly open-source licenses with minimal restrictions on use or modification.
last release 2026-08-14 (0 days)
0 known vulnerabilities (OSV.dev, 2026-08-14) · 419,112 downloads/mo, #6,802 on PyPI
Alternatives
Verify before relying
pip install nixl-cu12
import nixl
# NIXL automatically selects the correct CUDA backend at runtime based on PyTorch's CUDA version- Specific performance gains or latency improvements over direct tensor transfers in real inference workloads.
- Compatibility matrix with specific PyTorch versions and inference frameworks beyond the general CUDA 12 support.
- Whether the plugin architecture is accessible to end users or primarily for framework integrators.
What it is and what it does
NIXL is a Python library that wraps NVIDIA's Inference Xfer Library, a C++ communication acceleration layer designed for AI inference. It provides an abstraction over memory types (CPU, GPU) and storage backends (file, block, object store) through pluggable transport modules, with the goal of reducing data movement overhead in distributed inference scenarios.
The library is intended for integration into inference frameworks like NVIDIA Dynamo rather than direct application use. It ships with pre-built wheels that automatically detect and use the correct CUDA backend (12 or 13) at runtime. Installation is straightforward on Linux via pip, but the package is tightly coupled to PyTorch and CUDA, making it suitable only for GPU-accelerated inference workloads on supported platforms.
Use it for
- Accelerating tensor transfers between GPU and CPU memory in multi-GPU inference deployments.
- Reducing latency in distributed inference by optimizing point-to-point communication patterns.
- Abstracting storage I/O (file, object store, block device) for inference data staging and caching.
- Integrating into inference frameworks that need vendor-neutral transport abstraction over UCX and other backends.
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes, if you are building or integrating an AI inference framework on Linux with GPU support and need optimized data movement abstractions.
No, if you are running inference directly in PyTorch without framework-level integration—the library is designed for framework authors, not end-user applications. Requires Linux, Python 3.10+, torch, and numpy; no known vulnerabilities as of the latest release.
Install
nixl-cu12 on PyPI
Before you install
Medium install friction: requires Python 3.10+, torch, and numpy as runtime dependencies. Wheels are pre-built for recent Python versions (3.10–3.14) on x86_64 and aarch64 Linux only. Active maintenance as of the latest release.
Linux only (Ubuntu 22.04/24.04 or Fedora tested); macOS and Windows not supported. Requires CUDA 12 runtime and PyTorch with matching CUDA version.
License in practice
Dual-licensed under MIT and Apache-2.0 (permissive). Both are business-friendly open-source licenses with minimal restrictions on use or modification.
Quickstart
pip install nixl-cu12
import nixl
# NIXL automatically selects the correct CUDA backend at runtime based on PyTorch's CUDA versionVerify before relying
- Specific performance gains or latency improvements over direct tensor transfers in real inference workloads.
- Compatibility matrix with specific PyTorch versions and inference frameworks beyond the general CUDA 12 support.
- Whether the plugin architecture is accessible to end users or primarily for framework integrators.
Package facts
| License | MIT AND Apache-2.0 permissive |
| Python support | Supports the current Python release >=3.10 |
| Install friction | Medium. Platform-specific wheel |
| Runtime dependencies | 2 packagestorchnumpy |
| Maintenance | Actively maintained 0 days since the last release |
| First released | |
| Downloads | 419,112 / month, #6,802 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
Evidence: nixl_cu12-1.4.0-cp310-cp310-manylinux_2_28_aarch64.whl; nixl_cu12-1.4.0-cp310-cp310-manylinux_2_28_x86_64.whl; nixl_cu12-1.4.0-cp311-cp311-manylinux_2_28_aarch64.whl; nixl_cu12-1.4.0-cp311-cp311-manylinux_2_28_x86_64.whl; nixl_cu12-1.4.0-cp312-cp312-manylinux_2_28_aarch64.whl; nixl_cu12-1.4.0-cp312-cp312-manylinux_2_28_x86_64.whl; nixl_cu12-1.4.0-cp313-cp313-manylinux_2_28_aarch64.whl; nixl_cu12-1.4.0-cp313-cp313-manylinux_2_28_x86_64.whl; nixl_cu12-1.4.0-cp314-cp314-manylinux_2_28_aarch64.whl; nixl_cu12-1.4.0-cp314-cp314-manylinux_2_28_x86_64.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “inference data transfer optimization”
- nixl-cu12NIXL provides Python bindings to accelerate point-to-point…
- nixl-cu13NIXL provides Python bindings to accelerate point-to-point…
- mooncake-transfer-engine-cuda13Mooncake Transfer Engine provides high-performance, zero-copy…
Give your agent the search over MCP, or paste the wish link into any chat.
More Artificial Intelligence packages
LiteLLM provides a unified Python interface to call 100+ LLM providers (OpenAI, Anthropic, Gemini, Bedrock, Azure, and others) using OpenAI-compatible API format, available as both a Python SDK and a self-hosted AI Gateway proxy server.
Install it if you need to work with multiple LLM providers or want to centralize LLM routing in your organization.
Client library and CLI tool for downloading, uploading, and managing models, datasets, and repositories on the Hugging Face Hub platform.
Install it if you work with Hugging Face Hub models or datasets.
LangChain provides a framework for building agents and LLM-powered applications by composing language models, tools, and memory through a unified API that abstracts over multiple model providers.
hf-xet provides chunk-based deduplication and efficient file transfer for the Hugging Face Hub, enabling faster uploads and downloads of large files with local disk caching.
Tokenizers converts raw text into token sequences for NLP models, with support for training custom vocabularies and using pre-built tokenizers (BPE, WordPiece) optimized for speed via Rust.
Transformers provides a unified framework for loading, fine-tuning, and running state-of-the-art pretrained models across text, vision, audio, video, and multimodal tasks using PyTorch, JAX, or TensorFlow.
Install it if you need to run or train any transformer-based model for NLP, vision, audio, or multimodal tasks.
See also nixl-cu13 · nixl · libucxx-cu12 · ucxx-cu12 · sgl-kernel · sglang-kernel · nccl4py · tensorrt-cu12-libs · tensorrt-cu12-bindings · spin