pygmars

What it is and what it does

Pygmars is a lightweight lexing and parsing library that builds on simplified, remixed code from NLTK's regex-based tagging and chunking. It transforms sequences of text into labeled Token objects (assigning labels, tracking position and line number), then applies regular-expression-based grammar rules to recognize token sequences and build a parse tree. Each rule has a left-hand side label (non-terminal) and a right-hand side pattern over token labels.

The library is designed for cases where NLTK is overkill—it has no dependencies, a small footprint, and integrates with Pygments lexers to enable lightweight parsing of many programming languages. It was originally built to parse copyright statements in ScanCode Toolkit and is now used for extracting metadata from package manifests and other lightweight language parsing tasks where a full NLP toolkit is unnecessary.

Use it for:

• Parse copyright and author statements from source files by building grammars on top of existing Pygments lexers.
• Extract metadata (such as dependencies) from package manifest files using lightweight regex-based grammar rules.
• Build simple domain-specific language parsers without the overhead of a full NLP framework.
• Tokenize and label text in programming languages supported by Pygments, then apply lightweight grammar-based analysis.
• Integrate lexing and parsing into tools that need lightweight language understanding without external dependencies.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Pygmars builds lightweight lexers and parsers using regular expressions, transforming text into labeled tokens and parse trees. It integrates with Pygments lexers to enable grammar-based parsing of many programming languages.

Yes, if you need lightweight regex-based lexing and parsing without dependencies. The library is stable and permissively licensed, but maintenance is aging (394 days since last release). Install it for copyright detection, manifest parsing, or simple DSL parsing; avoid it if you need active upstream development or support for complex recursive grammars.

Install

pygmars on PyPI

pip install pygmars

uv add pygmars

poetry add pygmars

Installing pygmars

Before you install

Low install friction with no runtime dependencies. Maintenance status is aging—last release was 394 days ago, though the repository remains active and the package is marked Production/Stable.

License in practice

Licensed under Apache-2.0 (permissive), allowing commercial and private use with minimal restrictions. Derived from NLTK; copyright held by nexB Inc. and the NLTK Project.

Quickstart

pip install pygmars

from pygmars.lex import Lexer
from pygmars.parse import Parser, Grammar

lexer = Lexer()
tokens = lexer.lex("your text here")
parser = Parser(Grammar(rules))
tree = parser.parse(tokens)

Requires Python 3.9 or later.

Verify before relying

• Whether the package's aging maintenance status (394 days since last release) affects stability or security for new use cases.
• How well the library handles complex or deeply nested grammar rules in practice.
• Whether Pygments integration covers all 130+ supported languages or a subset.

Package facts

Evidence: pygmars-1.0.0-py3-none-any.whl

Keywords: utilities

Tags

More Software Development packages