Packages
Computes ROUGE metrics (ROUGE-N, ROUGE-L, ROUGE-W, ROUGE-S, ROUGE-SU) for automatic evaluation of text summarization quality, with both a pure Python implementation and a Perl script wrapper.
Provides curated sets of English words from multiple word lists (GCIDE and web2) that can be filtered by case and alphanumeric characters.
However, if you need active development, Unicode normalization, or additional word lists, consider whether a maintained alternative or a custom word source would…
Python wrapper for MeCab-ko, a morphological analyzer that tokenizes and tags Korean text into parts of speech and lemmas.
KoNLPy provides Korean natural language processing tools including tokenization, part-of-speech tagging, and linguistic feature extraction by wrapping multiple Korean NLP engines.
Wyoming is a peer-to-peer TCP protocol for voice assistants that exchanges JSONL-formatted messages and PCM audio, enabling communication between voice services like speech recognition, text-to-speech, wake word detection, and intent handling.
Install only if you have a concrete use case requiring Wyoming protocol support; it is not a general-purpose library.
Provides a tree-sitter parser grammar for HCL and Terraform, enabling incremental parsing and syntax analysis of HCL configuration files.
Splits Russian text into sentences and tokens using rule-based heuristics, returning character-offset substrings for precise text reconstruction.
Habachen converts between full-width and half-width characters, and between hiragana and katakana in Japanese text, with a focus on speed and low memory usage.
Converts strings between different text case formats (snake_case, kebab-case, camelCase, PascalCase, etc.) with support for non-ASCII characters and custom word boundaries.
Install it if you need reliable case conversion; the low friction and permissive license make it a safe addition to any project.
Converts text between Cyrillic and Latin scripts bi-directionally, with language-specific support for Serbian, Russian, Ukrainian, Bulgarian, Belarusian, Macedonian, Montenegrin, Tajik, Mongolian, and Greek.
Install it if you need to convert between Cyrillic and Latin scripts for any of the ten supported languages.
BPEmb provides pre-trained subword embeddings in 275 languages based on Byte-Pair Encoding, enabling subword segmentation and embedding lookup for neural NLP models.
However, the 682-day gap since the last release means no bug fixes or updates are forthcoming; verify that the remote model repository remains accessible before…
Pyleri is a left-right parser generator that lets you define a grammar in Python and parse strings against it, with the ability to export that grammar to JavaScript, C, Go, and Java.
Splits unsegmented English text into individual words using a corpus-based statistical approach, handling cases where spaces have been removed or are ambiguous.
However, the 2960-day gap since the last release means no updates for modern Python versions, security patches, or corpus improvements.
Provides a tree-sitter parser grammar for Haskell, enabling incremental parsing and syntax tree generation for Haskell source code.
However, install only if you already use tree-sitter; this package is a grammar plugin, not a standalone parser.
Tokenizes Icelandic text into words, punctuation, numbers, dates, and other token types, with optional sentence segmentation and deep annotation of token properties.
Install it if you need to tokenize or segment Icelandic text.
Conjugates Spanish verbs by tense, mood, and pronoun through a simple class-based API.
However, verify that the tenses and moods you need are implemented—the package does not support all Spanish moods (e.g., no conditional perfect listed in the docs),…
Urduhack provides NLP preprocessing, tokenization, part-of-speech tagging, and named entity recognition for Urdu text, built on TensorFlow and designed to work with Urdu language datasets.
However, verify that its TensorFlow dependencies and pre-trained models work in your environment—no active maintenance means you may need to patch compatibility…
PyTextRank implements graph-based TextRank and related algorithms as a spaCy pipeline extension to extract key phrases and perform extractive summarization on text documents.
Provides Ukrainian morphological dictionaries for pymorphy3, enabling Ukrainian language text analysis and lemmatization.
However, proceed with caution: the package is abandoned (last update 2022-12-01), so dictionary accuracy and compatibility with current Python versions are not…
Looks up demographic information about first and last names across 105 countries, including gender likelihood, popularity rank, and fuzzy/autocomplete search.
Measures Korean speech-to-text accuracy using character error rate (CER), word error rate (WER), character correct rate (CRR), and entity preservation metrics with reproducible evaluation reports.
Cutlet converts Japanese text to romaji (Latin characters) using multiple romanization systems including Modified Hepburn, Kunrei-shiki, and Nihon-shiki, with support for proper noun capitalization and URL slug generation.
Identifies the language of input text using a neural network model, returning BCP-47-style language codes for many languages and scripts.
Converts spelling to IPA phonetic transcription and measures language similarity across independent axes: phonology, reading, spelling, script, genealogy, time, and geography.
Epitran converts written text in various languages and scripts into International Phonetic Alphabet (IPA) notation, supporting multiple language-script pairs with optional preprocessing and postprocessing.
The main gotcha is that English G2P and Chinese/Cantonese support require external resources (Flite or dictionary files), so verify those dependencies match your use…
Provides alternative pinyin (romanized Chinese) data sources to override the default pronunciation mappings in pypinyin, allowing selection of specific pronunciation datasets.
However, dormant maintenance (579 days since last release) means you should verify compatibility with your pypinyin version before relying on it in production, and…
rhoknp is a Python binding for Japanese language processing tools (Juman++, KNP, KWJA) that performs morphological analysis, syntactic parsing, and document-level language analysis on Japanese text.
Install it if you are working with Japanese NLP and require access to Juman++, KNP, or KWJA; note that the underlying analysis engines may need separate installation…
textacy extends spaCy's NLP capabilities with pre- and post-processing tools: text cleaning, entity and n-gram extraction, string similarity metrics, topic modeling, and readability statistics.
However, dormant maintenance (last release 1230 days ago) means you should expect no active support or updates; use it for stable, well-defined tasks rather than as a…
g2pkk converts Korean text to phonetic representation (grapheme-to-phoneme) with cross-platform support, using platform-specific morpheme analyzers to work reliably on Windows and Unix-like systems.
Gruut tokenizes, cleans, and converts text to IPA phonemes for multiple languages, with support for SSML markup and automatic expansion of numbers, dates, and currency.
Not recommended for new projects requiring active support or ongoing language additions.
Provides the small-edition Sudachi dictionary resource for SudachiPy, a Japanese morphological analyzer. The dictionary is downloaded during installation and made available to SudachiPy for tokenization and parsing tasks.
Provides a Korean dictionary for MeCab tokenization, enabling morphological analysis of Korean text when used with mecab-python3 or fugashi.
Adds word-level timestamps and confidence scores to OpenAI's Whisper speech recognition model using Dynamic Time Warping on cross-attention weights, supporting multilingual ASR with optional voice activity detection.
Indic NLP Library provides text processing and natural language processing tools for Indian languages, including tokenization, normalization, transliteration, script conversion, and sentence splitting.
However, verify that external resources are available and compatible with your setup, and plan for no upstream bug fixes or updates.
Calculates lexical diversity metrics (TTR, MTLD, HDD, and others) on tokenized text to measure vocabulary richness and variation.
However, verify compatibility with your Python version before relying on it in production, and consider whether a more actively maintained alternative better fits…
Parses JavaScript objects embedded in HTML or text into Python dictionaries, handling JavaScript syntax that standard JSON parsers reject.
Gruut IPA parses, analyzes, and converts International Phonetic Alphabet (IPA) pronunciations, splitting them into phones or phonemes and translating between IPA, espeak, and SAMPA formats across multiple languages.
However, install friction is high due to source-only distribution, and you should expect no future maintenance or bug fixes.
Provides English language data files for tokenization and IPA phoneme generation, enabling text processing and phonetic analysis for English.
Converts text between different Indic script encodings (Devanagari, Tamil, Telugu, Kannada, Bengali, and others) and transliteration schemes (Harvard-Kyoto, IAST, ITRANS, SLP1, WX).
Install it if you need to work with Indic scripts or transliteration schemes.
Identifies the language of text using a neural network model, returning language codes with confidence scores and reliability indicators.