transformer-engine-cu13

What it is and what it does

Transformer Engine is a library that speeds up Transformer model training and inference on NVIDIA GPUs by using lower-precision arithmetic (FP8, MXFP8, NVFP4) instead of standard floating-point formats, reducing memory use and computation time. It provides modules for PyTorch and JAX/Flax that integrate FP8 support directly into Transformer layers, handling the scaling factors and precision conversions automatically. The library includes optimized C++ kernels and fused operations for common Transformer patterns, and works with advanced features like mixture-of-experts, tensor parallelism, and sequence parallelism.

The package targets researchers and engineers training or deploying large language models and multimodal Transformers on Hopper, Ada, Ampere, and Blackwell GPU architectures. It depends on packaging, pydantic, and importlib-metadata, and requires CUDA 12.1+ (12.8+ for Blackwell), cuDNN 9.3+, and Python 3.10+. Installation uses platform-specific wheels for x86_64 and aarch64 Linux.

Use it for:

• Train large language models with FP8 precision to reduce memory footprint and training time on supported GPUs.
• Integrate low-precision inference into production pipelines to serve models faster with lower latency.
• Build mixture-of-experts models using optimized kernels for efficient sparse computation.
• Experiment with NVFP4 format on Blackwell GPUs to achieve efficiency gains while maintaining precision.
• Combine FP8 training with tensor or sequence parallelism for distributed training of large models.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Accelerates Transformer model training and inference on NVIDIA GPUs using low-precision formats including FP8, MXFP8, and NVFP4, with optimized kernels and mixed-precision support.

Yes, if you are training or deploying Transformer models on supported NVIDIA GPUs (Hopper, Ada, Ampere, or Blackwell) and want to reduce memory and compute cost. Verify the unclear license terms and ensure your CUDA/cuDNN versions meet the minimum requirements (12.1+ CUDA, 9.3+ cuDNN). No known vulnerabilities. Active maintenance and recent release are positive signals.

Install

transformer-engine-cu13 on PyPI

pip install transformer-engine-cu13

uv add transformer-engine-cu13

poetry add transformer-engine-cu13

Installing transformer-engine-cu13

Before you install

Medium install friction due to platform-specific wheels (aarch64 and x86_64 manylinux) and CUDA 12.1+ requirement. Active maintenance with recent release (3 days old). Requires Python 3.10+.

License in practice

License treatment is unclear—no SPDX identifier or raw license text provided. Verify licensing terms before production use, particularly if integrating into proprietary systems.

Quickstart

pip install transformer-engine-cu13

import transformer_engine.pytorch as te
from transformer_engine.common import recipe

model = te.Linear(768, 3072, bias=True)
fp8_recipe = recipe.DelayedScaling(margin=0, fp8_format=recipe.Format.E4M3)

with te.autocast(enabled=True, recipe=fp8_recipe):
    out = model(inp)

Requires CUDA 12.1+ (12.8+ for Blackwell), cuDNN 9.3+, and an NVIDIA GPU (Hopper, Ada, Ampere, or Blackwell). Python 3.10+ required.

Verify before relying

• Whether the package includes pre-built binaries for all target architectures or requires compilation
• Specific NVIDIA GPU models and CUDA versions supported beyond the general hardware list
• Whether cuDNN 9.3+ is a hard runtime requirement or only needed at build time
• Performance benchmarks or throughput gains claimed in the documentation

Package facts

Evidence: transformer_engine_cu13-2.18.0-py3-none-manylinux_2_28_aarch64.whl; transformer_engine_cu13-2.18.0-py3-none-manylinux_2_28_x86_64.whl

Tags

More Artificial Intelligence packages

Further reading