---
id: xformers
version: "0.0.35"
license: unclear
license_treatment: permissive
maintenance: active
---

# xformers — XFormers: A collection of composable Transformer building blocks.

License: permissive · Maintenance: active · Downloads: 2.2M/mo

## What it is and what it does

XFormers is a library of modular, reusable components for building Transformer neural networks. Rather than implementing entire architectures as monolithic models, it provides composable building blocks—attention mechanisms, layer variants, and other Transformer primitives—that can be combined to reproduce state-of-the-art model designs. The library targets researchers and practitioners who want flexibility in assembling custom Transformer architectures without reinventing core components.

It depends on torch for computation and numpy for numerical operations. Installation requires Python 3.9 or later and involves platform-specific compiled wheels, which adds moderate setup friction but ensures optimized performance on supported systems (Linux x86_64 and Windows x86_64).

Use it for:

- Build custom Transformer architectures by combining xformers blocks instead of implementing layers from scratch.
- Experiment with different attention mechanisms and layer configurations for research or model optimization.
- Integrate efficient attention implementations into existing PyTorch models to reduce memory usage or improve speed.
- Reproduce published Transformer variants by assembling their documented building-block combinations.
- Prototype novel Transformer designs by swapping and reconfiguring composable components.

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

XFormers provides reusable, composable building blocks for constructing Transformer neural network architectures, enabling flexible assembly of state-of-the-art model variants without monolithic implementations.

Yes, if you are building or experimenting with Transformer architectures in PyTorch. XFormers is actively maintained, has no known vulnerabilities, uses a permissive BSD license, and ranks in the top 5000 PyPI packages by download volume. The medium install friction (platform-specific wheels) is a minor trade-off for the modularity and performance benefits it provides. Not necessary if you only use pre-built, monolithic Transformer models.

## Install

pip install xformers
uv add xformers
poetry add xformers

## Installing xformers

Before you install: Medium install friction due to compiled wheel dependencies (xformers ships platform-specific wheels for Python 3.9+ on Linux and Windows). Active maintenance with a recent release as of 2026-02-20. Requires torch and numpy as runtime dependencies.

License in practice: BSD License (permissive) means you can use, modify, and distribute xformers with minimal restrictions, provided you include the license notice.

Quickstart:

pip install xformers torch numpy

import xformers
from xformers.ops import memory_efficient_attention
# Use composable transformer blocks in your model

Requires Python 3.9 or later; torch and numpy must be installed. Platform-specific wheels available for Linux (manylinux_2_28_x86_64) and Windows (win_amd64).

Verify before relying:

- Whether xformers provides performance improvements over standard PyTorch transformer implementations in typical use cases.
- Compatibility with other transformer frameworks or whether it is PyTorch-specific.
- Availability of pre-built wheels for ARM64 or other architectures beyond x86_64.

## Package facts

- License: not declared (permissive)
- Python support: supports_current
- Install friction: medium
- Maintenance: active
- Downloads: 2.2M/month (top 5,000 on PyPI)
- Known vulnerabilities: none known

## Tags

transformer building blocks, composable transformer components, transformer architecture library, modular transformer layers, transformer model construction, attention mechanism implementations, transformer blocks pytorch, transformer-components, pytorch-extension, neural-architecture

[View on SkillFed](https://skillfed.io/packages/xformers) · [View on PyPI](https://pypi.org/project/xformers/)