{"categories":[{"label":"Artificial Intelligence","url":"https://skillfed.io/packages/category/scientific-engineering-artificial-intelligence/2"}],"enrichment":{"capability":"FlashInfer provides optimized GPU kernels for LLM inference operations\u2014attention, GEMM, and mixture-of-experts\u2014with support for multiple GPU architectures and low-precision quantization.","skillfed_tags":["gpu-acceleration","llm-inference","cuda-kernels"],"use_cases":["Accelerate attention computation in LLM inference servers (e.g., vLLM, SGLang, TensorRT-LLM).","Optimize prefill and decode phases separately for mixed-batch serving scenarios.","Deploy quantized (FP8/FP4) inference for memory-constrained or cost-sensitive GPU clusters.","Implement custom attention patterns (sparse, cascade, MLA) without writing CUDA code.","Reduce latency in speculative decoding and multi-node inference with AllReduce and NVSHMEM support."],"what_it_does":"FlashInfer is a kernel library and generator for high-performance LLM inference on NVIDIA GPUs. It provides unified APIs for attention (including paged/ragged KV-cache, decode, prefill, MLA, cascade, and sparse patterns), matrix multiplication (BF16, FP8, FP4), mixture-of-experts routing, and sampling operations. The library automatically selects the best backend\u2014FlashAttention-2/3, cuDNN, CUTLASS, or TensorRT-LLM\u2014for your hardware and workload.\n\nIt is designed for production serving with support for CUDAGraph and torch.compile, low-precision compute (FP8, FP4 quantization), and modern GPU architectures from Turing (SM 7.5) through Blackwell (SM 12.1). The package ships as a pure-Python wheel that compiles or downloads kernels on first use, with optional pre-compiled packages available for faster initialization and offline deployment.","worth_installing":"Yes. FlashInfer is worth installing if you are building or deploying LLM inference on NVIDIA GPUs. It is actively maintained, has no known vulnerabilities, carries a permissive license, and is already adopted by major projects (vLLM, SGLang, TensorRT-LLM). Install friction is low. The main constraint is GPU hardware: you must have an NVIDIA GPU with compute capability SM 7.5 or later and CUDA 12.6\u201313.1."},"id":"flashinfer-python","links":{"html":"https://skillfed.io/packages/flashinfer-python","md":"https://skillfed.io/packages/flashinfer-python.md","pypi":"https://pypi.org/project/flashinfer-python/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2026-08-11","license_spdx":"Apache-2.0","license_treatment":"permissive","name":"flashinfer-python","python_support":"supports_current","summary":"FlashInfer: Kernel Library for LLM Serving"},"popularity":{"monthly_downloads":5180758,"position":2147,"tier":"top_5000"},"security":{"n_vulnerabilities":0},"version":"0.6.17"}