llguidance

What it is and what it does

llguidance is a Rust library with Python bindings that implements constrained decoding for large language models. It computes token masks—sets of valid next tokens—that ensure LLM output conforms to a specified grammar, JSON schema, or regular expression. The library supports multiple grammar formats including JSON schemas, regular expressions, and context-free grammars in Lark-like syntax, and can be used directly from Python, Rust, C, or C++.

The library is designed for performance: mask computation takes approximately 50μs per token for a 128k-token vocabulary with negligible startup cost. It has been integrated into major LLM inference frameworks including vLLM, llama.cpp, SGLang, and Chromium, and powers OpenAI's Structured Output feature. It uses Earley's algorithm for parsing combined with regex derivatives and trie-based token traversal to avoid the startup overhead and memory costs of pre-computed automata approaches.

Use it for:

• Enforce JSON schema compliance in LLM outputs for API integrations or data pipelines requiring structured responses.
• Constrain LLM generation to valid regular expressions or domain-specific grammars in real-time inference servers.
• Integrate structured output constraints into vLLM, llama.cpp, or SGLang deployments without significant latency overhead.
• Use within the Guidance Python library to build multi-turn LLM workflows with guaranteed output format compliance.
• Embed in Chromium-based browsers to enforce JSON Schema for the Prompt API's structured output feature.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Enforces structured output from large language models by computing token masks that constrain decoding to valid context-free grammars, JSON schemas, or regular expressions with minimal per-token overhead.

Yes. The package is actively maintained, has no known vulnerabilities, uses a permissive MIT license, and solves a critical problem in LLM inference—ensuring structured output without sacrificing speed. It is production-ready, widely integrated into major frameworks, and backed by peer-reviewed research. Install if you need deterministic structured output from LLMs in any inference context.

Install

llguidance on PyPI

pip install llguidance

uv add llguidance

poetry add llguidance

Installing llguidance

Before you install

Medium install friction due to compiled wheels across multiple platforms and architectures. Package is actively maintained with a recent release and no known vulnerabilities. Requires Python 3.10 or later.

License in practice

MIT license permits unrestricted use, modification, and distribution for commercial and private projects with minimal attribution requirements.

Quickstart

pip install llguidance

import llguidance
# Use with Guidance library or directly via Rust/C bindings
# See integration examples in vLLM, llama.cpp, or SGLang

Requires Python 3.10 or later; compiled wheels available for common platforms but installation may require compilation on unsupported architectures.

Verify before relying

• Exact performance characteristics (50μs per token claim) for specific tokenizer sizes and grammar complexity in real-world deployments.
• Compatibility matrix and tested versions for each integrated project (vLLM, llama.cpp, SGLang, etc.).
• Whether the Python package is the primary interface or primarily a binding for Rust library use.

Package facts

Evidence: llguidance-1.8.0-cp314-cp314t-macosx_10_12_x86_64.whl; llguidance-1.8.0-cp314-cp314t-macosx_11_0_arm64.whl; llguidance-1.8.0-cp314-cp314t-manylinux_2_17_aarch64.manylinux2014_aarch64.whl; llguidance-1.8.0-cp314-cp314t-manylinux_2_17_i686.manylinux2014_i686.whl; llguidance-1.8.0-cp314-cp314t-manylinux_2_17_x86_64.manylinux2014_x86_64.whl; llguidance-1.8.0-cp314-cp314t-win32.whl; llguidance-1.8.0-cp314-cp314t-win_amd64.whl; llguidance-1.8.0-cp314-cp314t-win_arm64.whl; llguidance-1.8.0-cp39-abi3-macosx_10_12_x86_64.whl; llguidance-1.8.0-cp39-abi3-macosx_11_0_arm64.whl; llguidance-1.8.0-cp39-abi3-manylinux_2_31_aarch64.whl; llguidance-1.8.0-cp39-abi3-manylinux_2_31_x86_64.whl; llguidance-1.8.0-cp39-abi3-manylinux_2_34_i686.whl; llguidance-1.8.0-cp39-abi3-manylinux_2_39_riscv64.whl; llguidance-1.8.0-cp39-abi3-win32.whl; llguidance-1.8.0-cp39-abi3-win_amd64.whl; llguidance-1.8.0-cp39-abi3-win_arm64.whl

Tags

More Artificial Intelligence packages

Further reading