sageattention
Accurate and efficient 8-bit plug-and-play attention.
Decision gist · record as of 2026-08-14
Yes, if you run transformer inference on RTX4090 or RTX3090 GPUs and can verify that the supported head dimensions and attention patterns match your model. The low install friction, permissive license, and active maintenance make it a reasonable experiment. No, if you use other GPU architectures or require head dimensions outside 64, 96, 128—the stated optimization scope is narrow and performance gains are not guaranteed elsewhere.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires Python>=3.9, torch>=2.3.0, and triton>=2.3.0.
- Performance is optimized for RTX4090 and RTX3090 GPUs; other architectures may not see significant speedup.
- Head dimension must be one of 64, 96, or 128.
License · maintenance · safety
BSD 3-Clause License (permissive) — BSD 3-Clause License is permissive and poses no significant restrictions on use, modification, or distribution in commercial or private projects.
last release 2024-11-20 (632 days) · last repo commit 2026-01-17 · 3,640 stars
0 known vulnerabilities (OSV.dev, 2026-08-14) · 148,488 downloads/mo, #11,031 on PyPI
Alternatives
Verify before relying
pip install sageattention
from sageattention import sageattn
attn_output = sageattn(q, k, v, tensor_layout="HND", is_causal=False, smooth_k=True)- Whether performance gains materialize on GPU architectures other than RTX4090 and RTX3090, given the stated optimization focus
- Actual accuracy impact of disabling smooth_k in production workloads with irregular q, k, v distributions
- Compatibility with transformer models that do not use F.scaled_dot_product_attention directly
What it is and what it does
SageAttention is a specialized attention kernel library that replaces the standard transformer attention computation with quantized variants using 8-bit (or 4-bit in SageAttention2) precision. It integrates smoothing techniques, per-block quantization for queries and keys, and FP16 accumulators to maintain accuracy while reducing memory bandwidth and computation cost during inference.
The package is designed as a drop-in replacement for scaled_dot_product_attention, allowing users to accelerate existing models with minimal code changes. It currently supports head dimensions of 64, 96, and 128, variable sequence lengths between queries and key-values, and group-query attention patterns. Performance is optimized for RTX4090 and RTX3090 GPUs; benefits on other architectures are not guaranteed.
Use it for
- Accelerate video generation models like CogVideoX by replacing their attention layers with quantized kernels
- Speed up large language model inference on supported GPUs by patching scaled_dot_product_attention globally
- Reduce memory bandwidth requirements in transformer-based inference pipelines without retraining
- Deploy transformer models with lower latency on resource-constrained inference servers using RTX-series GPUs
- Benchmark quantized attention accuracy on models with variable sequence lengths per batch
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes, if you run transformer inference on RTX4090 or RTX3090 GPUs and can verify that the supported head dimensions and attention patterns match your model.
The low install friction, permissive license, and active maintenance make it a reasonable experiment. No, if you use other GPU architectures or require head dimensions outside 64, 96, 128—the stated optimization scope is narrow and performance gains are not guaranteed elsewhere.
Install
sageattention on PyPI
Before you install
Low install friction with a pure-Python wheel distribution. The package is in beta status and has been actively maintained since its first release on 2024-10-05, with the latest release on 2024-11-20. Requires Python 3.9+ and torch 2.3.0+, with triton 2.3.0+ as a build dependency.
Requires Python>=3.9, torch>=2.3.0, and triton>=2.3.0. Performance is optimized for RTX4090 and RTX3090 GPUs; other architectures may not see significant speedup. Head dimension must be one of 64, 96, or 128.
License in practice
BSD 3-Clause License is permissive and poses no significant restrictions on use, modification, or distribution in commercial or private projects.
Quickstart
pip install sageattention
from sageattention import sageattn
attn_output = sageattn(q, k, v, tensor_layout="HND", is_causal=False, smooth_k=True)Verify before relying
- Whether performance gains materialize on GPU architectures other than RTX4090 and RTX3090, given the stated optimization focus
- Actual accuracy impact of disabling smooth_k in production workloads with irregular q, k, v distributions
- Compatibility with transformer models that do not use F.scaled_dot_product_attention directly
Package facts
| License | BSD 3-Clause License permissive |
| Python support | Supports the current Python release >=3.9 |
| Install friction | Low. Pure-Python wheel |
| Runtime dependencies | None |
| Maintenance | Aging 632 days since the last release |
| Last repo commit | |
| First released | |
| Downloads | 148,488 / month, #11,031 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
| Classifiers | Development Status :: 4 - BetaIntended Audience :: DevelopersLicense :: OSI Approved :: BSD LicenseOperating System :: OS IndependentProgramming Language :: Python :: 3Programming Language :: Python :: 3.10Programming Language :: Python :: 3.11Programming Language :: Python :: 3.9Topic :: Software Development :: Libraries :: Python Modules |
Evidence: sageattention-1.0.6-py3-none-any.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “quantized attention kernels”
- sageattentionSageAttention provides quantized attention kernels for transformer…
- flashinfer-pythonFlashInfer provides optimized GPU kernels for LLM inference…
- tokenspeed-mlaProvides optimized MLA (Multi-head Latent Attention) kernels for…
Give your agent the search over MCP, or paste the wish link into any chat.
More Python Modules packages
Converts domain names between Unicode and ASCII-compatible encoding (Punycode) according to IDNA 2008 and Unicode Technical Standard 46, with security validation and broader script coverage than the standard library.
Install it if you work with internationalized domain names, need to validate domains, or use HTTP clients that depend on it transitively.
Setuptools is a Python build backend and package management tool that handles building, distributing, and installing Python packages, including support for C/C++ extension modules.
PyYAML parses and emits YAML 1.1 data format, enabling serialization and deserialization of configuration files and Python objects to and from human-readable YAML text.
Pydantic validates Python data structures against type hints, coercing and checking input at runtime to ensure it matches a declared schema.
Provides reusable metadata objects for use with PEP-593 `typing.Annotated` to express common constraints like bounds, collection sizes, and predicates on types.
Install it if you use or build libraries that need to express type constraints in a standardized, inspectable way—or if you want to annotate your own types with…
Provides runtime tools to inspect and introspect Python type annotations, enabling programmatic examination of type hints at execution time.
See also flashinfer-python · fla-core · humming-kernels · flash-attn · cpm-kernels · flashinfer-cubin · local-attention · qwix