xgrammar

What it is and what it does

This package is a structured generation engine that constrains LLM output to follow user-defined grammars—JSON schemas, regex patterns, or arbitrary context-free grammars—ensuring 100% structural correctness. It integrates with major LLM inference frameworks and achieves near-zero overhead in JSON generation by carefully optimizing the constrained decoding process.

The package is designed for portability across platforms (Linux, macOS, Windows) and hardware (CPU, NVIDIA GPU, AMD GPU, Apple Silicon, TPU). It depends on torch, transformers, apache-tvm-ffi, pydantic, triton, numpy, and typing-extensions, making it suitable for teams already working with modern ML inference stacks. Python 3.8 and above are supported.

Use it for:

• Generate JSON from LLMs with guaranteed schema compliance for API integrations and data pipelines
• Enforce regex patterns in model outputs for structured logs, IDs, or formatted text
• Build systems where tool calls and responses must follow strict context-free grammars
• Reduce latency in production LLM services by eliminating post-generation validation and retry loops
• Integrate structured generation into existing inference deployments without rewriting core code

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Constrains LLM text generation to follow specified grammars (JSON, regex, or context-free rules), ensuring structurally correct output with minimal performance overhead.

Yes, if you need guaranteed structural correctness in LLM outputs. The permissive Apache 2.0 license, active maintenance, and zero known vulnerabilities support production use. Install friction is moderate due to heavy ML dependencies, but wheels are available for standard platforms. Not necessary if you already validate outputs post-generation or don't require strict grammar enforcement.

Install

xgrammar on PyPI

pip install xgrammar

uv add xgrammar

poetry add xgrammar

Installing xgrammar

Before you install

Medium friction: requires seven runtime dependencies including torch, transformers, and apache-tvm-ffi. Wheels are available for current Python versions on Linux, macOS, and Windows. The project is actively maintained with recent commits and no known vulnerabilities.

License in practice

Apache 2.0 is permissive, allowing commercial and private use with minimal restrictions—suitable for most production deployments.

Quickstart

pip install xgrammar
import xgrammar as xgr
# Configure grammar and use with inference engine

Requires torch, transformers, and apache-tvm-ffi; for Apple Silicon MPS support, install with pip install "xgrammar[metal]"

Verify before relying

• Specific performance overhead numbers for non-JSON grammars beyond the 'near-zero' claim for JSON
• Whether the package works standalone or requires integration with a specific inference engine
• Supported grammar complexity limits or known edge cases in context-free grammar handling
• Concrete example of standalone usage without external inference framework integration

Package facts

Evidence: xgrammar-0.2.5-cp310-cp310-macosx_10_14_x86_64.whl; xgrammar-0.2.5-cp310-cp310-macosx_11_0_arm64.whl; xgrammar-0.2.5-cp310-cp310-manylinux_2_26_aarch64.manylinux_2_28_aarch64.whl; xgrammar-0.2.5-cp310-cp310-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl; xgrammar-0.2.5-cp310-cp310-win_amd64.whl

Keywords: machine learning, inference

Tags

More Artificial Intelligence packages

Further reading