---
id: sglang-kernel
version: "0.4.6.post1"
license: Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for… (full text in the JSON record)
license_treatment: permissive
maintenance: active
---

# sglang-kernel — Kernel Library for SGLang

License: permissive · Maintenance: active · Downloads: 643.3K/mo

## What it is and what it does

sglang-kernel is a library of hand-optimized CUDA kernels designed to accelerate inference for large language models and vision-language models. It provides custom compute primitives that integrate with PyTorch, allowing LLM inference engines to execute critical operations more efficiently on NVIDIA GPUs.

The package is distributed as pre-compiled wheels for x86_64 and aarch64 architectures, with no runtime Python dependencies. It requires torch == 2.13.0 and Python ≥3.10, and is intended to be used as a low-level acceleration layer within larger inference frameworks rather than as a standalone application. The source code lives in the sglang project repository.

Use it for:

- Accelerate batch matrix multiplication and attention operations in LLM inference engines
- Optimize vision-language model inference by providing specialized CUDA kernels for image and token processing
- Reduce latency and increase throughput in production LLM serving systems that depend on custom kernel performance
- Integrate with PyTorch-based inference frameworks that need fine-grained control over GPU compute primitives

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

sglang-kernel provides optimized CUDA compute primitives for LLM inference engines, enabling efficient inference for large language models and vision-language models through custom kernel operations.

Yes, if you are building or deploying an LLM inference engine that benefits from custom CUDA kernels and you have torch == 2.13.0 and Python ≥3.10 available. The active maintenance, permissive license, and high repository engagement suggest solid backing. Install friction is moderate due to compiled wheels, but no runtime dependencies simplify integration. Not suitable as a standalone package; it is a dependency for inference frameworks.

## Install

pip install sglang-kernel
uv add sglang-kernel
poetry add sglang-kernel

## Installing sglang-kernel

Before you install: Medium install friction due to compiled wheel distribution (aarch64 and x86_64 variants). Active maintenance with recent release 8 days ago and 31807 repository stars. Requires torch == 2.13.0 and Python ≥3.10.

License in practice: Apache License 2.0 (permissive) allows commercial use, modification, and distribution with minimal restrictions. You must include license and copyright notices and document changes.

Quickstart:

pip install sglang-kernel
# Import and use within inference framework that depends on sglang-kernel

Requires torch == 2.13.0 and Python ≥3.10. NVIDIA CUDA GPU required. Known segmentation fault with CUDA 12.6; update ptxas to 12.8.

Verify before relying:

- Whether pre-built wheels cover all target architectures and CUDA versions needed for your deployment
- Performance gains relative to standard PyTorch operations for your specific model and batch sizes
- Exact kernel operations exposed and their integration patterns within inference frameworks

## Package facts

- License: Apache License Version 2.0, January 2004 http://www.apache.org/licenses/ TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION 1. Definitions. "License" shall mean the terms and conditions for… (full text in the JSON record) (permissive)
- Python support: supports_current
- Install friction: medium
- Maintenance: active
- Downloads: 643.3K/month (top 15,000 on PyPI)
- Known vulnerabilities: none known

## Tags

llm inference kernels, cuda optimization primitives, language model acceleration, gpu kernel library, inference engine optimization, torch cuda kernels, vision language model kernels, cuda-kernels, llm-inference, gpu-acceleration

[View on SkillFed](https://skillfed.io/packages/sglang-kernel) · [View on PyPI](https://pypi.org/project/sglang-kernel/)