---
id: xgrammar
version: "0.2.5"
license: Apache 2.0
license_treatment: permissive
maintenance: active
---

# xgrammar — Efficient, Flexible and Portable Structured Generation

License: permissive · Maintenance: active · Downloads: 7.5M/mo

## What it is and what it does

This package is a structured generation engine that constrains LLM output to follow user-defined grammars—JSON schemas, regex patterns, or arbitrary context-free grammars—ensuring 100% structural correctness. It integrates with major LLM inference frameworks and achieves near-zero overhead in JSON generation by carefully optimizing the constrained decoding process.

The package is designed for portability across platforms (Linux, macOS, Windows) and hardware (CPU, NVIDIA GPU, AMD GPU, Apple Silicon, TPU). It depends on torch, transformers, apache-tvm-ffi, pydantic, triton, numpy, and typing-extensions, making it suitable for teams already working with modern ML inference stacks. Python 3.8 and above are supported.

Use it for:

- Generate JSON from LLMs with guaranteed schema compliance for API integrations and data pipelines
- Enforce regex patterns in model outputs for structured logs, IDs, or formatted text
- Build systems where tool calls and responses must follow strict context-free grammars
- Reduce latency in production LLM services by eliminating post-generation validation and retry loops
- Integrate structured generation into existing inference deployments without rewriting core code

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

Constrains LLM text generation to follow specified grammars (JSON, regex, or context-free rules), ensuring structurally correct output with minimal performance overhead.

Yes, if you need guaranteed structural correctness in LLM outputs. The permissive Apache 2.0 license, active maintenance, and zero known vulnerabilities support production use. Install friction is moderate due to heavy ML dependencies, but wheels are available for standard platforms. Not necessary if you already validate outputs post-generation or don't require strict grammar enforcement.

## Install

pip install xgrammar
uv add xgrammar
poetry add xgrammar

## Installing xgrammar

Before you install: Medium friction: requires seven runtime dependencies including torch, transformers, and apache-tvm-ffi. Wheels are available for current Python versions on Linux, macOS, and Windows. The project is actively maintained with recent commits and no known vulnerabilities.

License in practice: Apache 2.0 is permissive, allowing commercial and private use with minimal restrictions—suitable for most production deployments.

Quickstart:

pip install xgrammar
import xgrammar as xgr
# Configure grammar and use with inference engine

Requires torch, transformers, and apache-tvm-ffi; for Apple Silicon MPS support, install with pip install "xgrammar[metal]"

Verify before relying:

- Specific performance overhead numbers for non-JSON grammars beyond the 'near-zero' claim for JSON
- Whether the package works standalone or requires integration with a specific inference engine
- Supported grammar complexity limits or known edge cases in context-free grammar handling
- Concrete example of standalone usage without external inference framework integration

## Package facts

- License: Apache 2.0 (permissive)
- Python support: supports_current
- Install friction: medium
- Maintenance: active
- Downloads: 7.5M/month (top 5,000 on PyPI)
- Known vulnerabilities: none known

## Tags

constrained decoding for llms, json schema validation generation, structured output from language models, grammar-based text generation, llm output constraint engine, constrained-decoding, llm-inference, structured-generation

[View on SkillFed](https://skillfed.io/packages/xgrammar) · [View on PyPI](https://pypi.org/project/xgrammar/)