flashinfer-cubin

What it is and what it does

flashinfer-cubin is a companion package that bundles pre-compiled GPU kernel binaries for FlashInfer, a library delivering optimized kernels for LLM inference workloads. Instead of compiling kernels on first use, this package provides ready-to-use cubins for attention (including paged KV-cache, decode, prefill, and sparse patterns), matrix multiplication (BF16, FP8, FP4), and mixture-of-experts operations. It targets NVIDIA GPUs from Turing (SM 7.5) through Blackwell (SM 12.1), supporting multiple compute capabilities in a single installation.

The package is designed to accelerate LLM serving by eliminating JIT compilation overhead and enabling offline deployment. It integrates with production inference frameworks and supports low-precision compute (FP8, FP4 quantization). The library is actively maintained and adopted by major inference systems.

Use it for:

• Pre-compile inference kernels for production LLM serving to eliminate startup latency and enable offline deployment
• Accelerate attention operations in batched inference with optimized paged KV-cache and decode kernels
• Deploy quantized models (FP8/FP4) with fused GEMM and MoE kernels for cost-effective inference at scale
• Support mixed-precision inference with BF16 and low-precision compute across diverse GPU architectures
• Enable CUDAGraph and torch.compile compatible serving for low-latency production workloads

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Provides pre-compiled GPU kernel binaries (cubins) for FlashInfer's high-performance attention, GEMM, and MoE operations across diverse NVIDIA GPU architectures.

Yes. Install this if you are deploying FlashInfer-based inference in production or development and want to avoid JIT compilation overhead. The permissive Apache-2.0 license, active maintenance, low install friction, and zero known vulnerabilities make it a safe choice. Verify that your target GPU architecture is included in the 0.6.13 release before committing.

Install

flashinfer-cubin on PyPI

pip install flashinfer-cubin

uv add flashinfer-cubin

poetry add flashinfer-cubin

Installing flashinfer-cubin

Before you install

Low friction installation with a pure Python wheel. Active maintenance with recent releases; last commit 2026-08-14 and 6159 repository stars indicate ongoing development.

License in practice

Apache-2.0 permissive license allows commercial and private use with minimal restrictions; suitable for production deployments.

Quickstart

pip install flashinfer-cubin

import flashinfer

# Pre-compiled kernels are now available for use
flashinfer.show-config

Requires CUDA 12.6, 12.8, 13.0, or 13.1 and an NVIDIA GPU with compute capability SM 7.5 or later.

Verify before relying

• Whether all GPU architectures (SM 7.5 through SM 12.1) are included in this 0.6.13 wheel or if selective compilation is needed
• Runtime memory or disk footprint of the pre-compiled cubin binaries
• Compatibility with CUDA versions other than the documented 12.6, 12.8, 13.0, 13.1
• Whether flashinfer-cubin can be used standalone or requires flashinfer-python as a runtime dependency

Package facts

Evidence: flashinfer_cubin-0.6.13-py3-none-any.whl

Tags

More Python Modules packages

Further reading