---
id: sgl-deep-gemm
version: "0.1.5.post2"
license: Apache-2.0
license_treatment: permissive
maintenance: active
---

# sgl-deep-gemm — SGLang fork of DeepGemm

License: permissive · Maintenance: active · Downloads: 692.8K/mo

## What it is and what it does

sgl-deep-gemm is an SGLang-optimized fork of DeepGemm that provides GPU-accelerated matrix multiplication kernels for neural network inference and training workloads. It wraps custom GEMM implementations via apache-tvm-ffi to enable ABI-compatible wheels that work across different Python versions without recompilation, simplifying deployment compared to building from source.

The package is designed for use within SGLang's inference framework and targets NVIDIA GPUs on Linux (x86_64 and aarch64). Installation is straightforward via pip, but it requires Python 3.10 or later and a CUDA-capable GPU. The underlying kernels are optimized for specific tensor operations common in large language model inference.

Use it for:

- Accelerate matrix multiplication operations in SGLang-based LLM inference pipelines.
- Deploy optimized GEMM kernels across multiple Python versions without rebuilding wheels.
- Improve throughput for batch matrix operations on NVIDIA GPUs in production systems.
- Integrate custom GPU kernels into neural network frameworks via TVM's FFI layer.
- Support both x86_64 and aarch64 GPU deployments with a single wheel distribution.

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

sgl-deep-gemm provides optimized GEMM (matrix multiplication) kernels for GPU acceleration, packaged as a PyPI-installable wheel with ABI compatibility across Python versions via apache-tvm-ffi.

Yes, if you are using SGLang for LLM inference on NVIDIA GPUs and need optimized matrix multiplication. The active maintenance, permissive license, and ABI-compatible wheels make it low-friction to adopt. No, if you are not in the SGLang ecosystem or lack a compatible GPU and Linux platform (x86_64 or aarch64).

## Install

pip install sgl-deep-gemm
uv add sgl-deep-gemm
poetry add sgl-deep-gemm

## Installing sgl-deep-gemm

Before you install: Medium install friction due to platform-specific wheels (manylinux2014 x86_64 and aarch64 only) and a compiled dependency on apache-tvm-ffi. Active maintenance with a recent release (9 days old) and steady repository activity.

License in practice: Apache-2.0 permissive license allows commercial and private use with minimal restrictions, making it suitable for most deployment scenarios.

Quickstart:

pip install sgl-deep-gemm

import sgl_deep_gemm
# Use optimized GEMM kernels via apache-tvm-ffi bindings

Requires Python >=3.10 and NVIDIA CUDA-capable GPU; wheels available only for manylinux2014 x86_64 and aarch64 architectures.

Verify before relying:

- What specific GEMM operations and tensor shapes the kernels optimize for.
- Performance benchmarks compared to standard CUDA GEMM implementations.
- Whether apache-tvm-ffi is automatically installed or requires separate setup.
- Compatibility with specific CUDA and cuDNN versions.

## Package facts

- License: Apache-2.0 (permissive)
- Python support: supports_current
- Install friction: medium
- Maintenance: active
- Downloads: 692.8K/month (top 15,000 on PyPI)
- Known vulnerabilities: none known

## Tags

gpu matrix multiplication kernels, deep gemm optimization, cuda gemm library, tvm-based gemm, sglang gpu acceleration, optimized matrix operations, neural network gemm kernels, gpu-acceleration, matrix-operations, llm-inference

[View on SkillFed](https://skillfed.io/packages/sgl-deep-gemm) · [View on PyPI](https://pypi.org/project/sgl-deep-gemm/)