nlptutti

What it is and what it does

nlptutti is a Python package for evaluating Korean speech-to-text output against reference transcripts. It computes character error rate (CER), word error rate (WER), character correct rate (CRR), and entity/keyword preservation metrics by calculating substitutions, deletions, and insertions from Levenshtein minimum edit distance. The package supports both plain strings and structured formats (JSON, SRT, TSV) and offers two calculation modes: a normalized formula (default, for backward compatibility) and a standard formula (for model comparisons and published benchmarks).

The package handles Korean-specific concerns including punctuation removal, Unicode normalization options for combining characters, and word-boundary handling across different tokenization policies. It can evaluate single sentence pairs, entire corpora with micro/macro aggregation, named entities and keywords against provided dictionaries, and error patterns. Results include detailed edit operation counts and optional provenance tracking (SHA-256, package version, actual calculation parameters) for reproducibility.

Use it for:

• Compare accuracy of cloud STT services (Azure Speech, Amazon Transcribe, Google Cloud Speech-to-Text) using a consistent Korean-aware metric
• Evaluate open-source speech recognition models (Whisper, FunASR) against reference transcripts with standard CER/WER formulas
• Track preservation of named entities and product names in STT output using entity-specific CER and F1 scores
• Reproduce historical evaluation results by maintaining the default normalized calculation mode across versions
• Analyze error patterns (substitutions, deletions, insertions) to diagnose STT failure modes in Korean text

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Measures Korean speech-to-text accuracy using character error rate (CER), word error rate (WER), character correct rate (CRR), and entity preservation metrics with reproducible evaluation reports.

Yes. Active, recently released package with low install friction, permissive MIT license, no known vulnerabilities, and a focused scope solving a real problem for Korean STT evaluation. The single dependency (jiwer) is lightweight. Best for teams evaluating Korean speech recognition systems or comparing STT services; less relevant if you work exclusively with non-Korean languages.

Install

nlptutti on PyPI

pip install nlptutti

uv add nlptutti

poetry add nlptutti

Installing nlptutti

Before you install

Low friction: pure Python wheel with a single runtime dependency (jiwer). Active maintenance with recent release (2 days old) and 73 repository stars. Supports Python 3.8 through 3.14.

License in practice

MIT license permits unrestricted use, modification, and distribution with minimal restrictions—suitable for both open-source and commercial projects.

Quickstart

pip install nlptutti

import nlptutti as metrics

reference = "오늘 날씨가 맑습니다"
hypothesis = "오늘 날씨는 맑습니다"

cer = metrics.get_cer(reference, hypothesis, rate_mode="standard")
print(round(cer["cer"], 4), cer["substitutions"])

Verify before relying

• Whether jiwer's Levenshtein distance implementation handles Korean combining characters correctly without explicit NFC normalization
• Performance characteristics with large corpora (corpus size thresholds for practical use)

Package facts

Evidence: nlptutti-0.0.0.19-py3-none-any.whl

Keywords: STT, ASR, Korean, NLP, CER, WER, CRR, Korean text normalization, Unicode normalization, speech recognition, transcript evaluation, JSON, SRT, TSV, FunASR

Tags

More Artificial Intelligence packages