match

What it is and what it does

Match is a utility for recovering the character offsets of tokenized words and phrases within their original, untokenized source text. Given a messy, whitespace-irregular, or Unicode-heavy source string and a sequence of tokens (or a single token), it returns all matching spans as (start, end, substring) tuples. This solves the alignment problem that arises when a tokenizer normalizes or splits text in ways that obscure the original character positions—essential for tasks like annotation, error correction, or mapping NLP pipeline outputs back to source documents.

The package handles common real-world messiness: irregular spacing, punctuation normalization (e.g., parentheses converted to -LRB- and -RRB-), and Unicode characters. It depends on nltk and regex for tokenization and pattern matching. The project is no longer actively maintained, with the last release in 2022-10-03 and no commits since 2023-04-11.

Use it for:

• Recover source text spans for tokens output by an NLP tokenizer to annotate or highlight original text.
• Map named entity recognition or part-of-speech tags back to character offsets in messy source documents.
• Align corrected or normalized token sequences with their original positions for error analysis.
• Extract substring context around matched token sequences for debugging tokenization issues.
• Support document annotation workflows where token-level labels must be converted to character offsets.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Maps tokenized words and phrases back to their character offsets in the original, untokenized source text, handling whitespace normalization and Unicode characters.

Yes, if you need token-to-offset alignment and can tolerate an abandoned package. The core functionality is stable and no known vulnerabilities exist. However, consider maintenance risk: no updates since 2022-10-03, and any bugs in Unicode or edge-case handling will not be fixed. Suitable for one-off scripts or internal tools; risky for production systems requiring ongoing support.

Install

match on PyPI

pip install match

uv add match

poetry add match

Installing match

Before you install

High install friction: the package is abandoned (last release 2022-10-03, last commit 2023-04-11) and depends on nltk and regex, which add non-trivial setup overhead. No maintenance activity for an extended period.

License in practice

Licensed under Apache License 2.0 (permissive), allowing commercial and private use with minimal restrictions.

Quickstart

pip install match

import match
result = match.match(original_text, ['token', 'sequence'])
# Returns: [(start_offset, end_offset, 'matched substring'), ...]

Requires nltk and regex as runtime dependencies; Python >= 3.8 required.

Verify before relying

• Whether nltk and regex versions have known incompatibilities or security issues affecting match's usability.
• Whether the abandoned status means critical bugs in Unicode or edge-case tokenization remain unfixed.
• Performance characteristics on very large texts or deeply nested tokenization structures.

Package facts

Evidence: match-0.3.2.tar.gz

Keywords: tokenization

Tags

More Text Processing packages