---
id: nlptutti
version: "0.0.0.19"
license: MIT
license_treatment: permissive
maintenance: active
---

# nlptutti — Korean STT CER, WER, and CRR metrics with explicit normalization and reproducible reports

License: permissive · Maintenance: active · Downloads: 112.7K/mo

## What it is and what it does

nlptutti is a Python package for evaluating Korean speech-to-text output against reference transcripts. It computes character error rate (CER), word error rate (WER), character correct rate (CRR), and entity/keyword preservation metrics by calculating substitutions, deletions, and insertions from Levenshtein minimum edit distance. The package supports both plain strings and structured formats (JSON, SRT, TSV) and offers two calculation modes: a normalized formula (default, for backward compatibility) and a standard formula (for model comparisons and published benchmarks).

The package handles Korean-specific concerns including punctuation removal, Unicode normalization options for combining characters, and word-boundary handling across different tokenization policies. It can evaluate single sentence pairs, entire corpora with micro/macro aggregation, named entities and keywords against provided dictionaries, and error patterns. Results include detailed edit operation counts and optional provenance tracking (SHA-256, package version, actual calculation parameters) for reproducibility.

Use it for:

- Compare accuracy of cloud STT services (Azure Speech, Amazon Transcribe, Google Cloud Speech-to-Text) using a consistent Korean-aware metric
- Evaluate open-source speech recognition models (Whisper, FunASR) against reference transcripts with standard CER/WER formulas
- Track preservation of named entities and product names in STT output using entity-specific CER and F1 scores
- Reproduce historical evaluation results by maintaining the default normalized calculation mode across versions
- Analyze error patterns (substitutions, deletions, insertions) to diagnose STT failure modes in Korean text

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

Measures Korean speech-to-text accuracy using character error rate (CER), word error rate (WER), character correct rate (CRR), and entity preservation metrics with reproducible evaluation reports.

Yes. Active, recently released package with low install friction, permissive MIT license, no known vulnerabilities, and a focused scope solving a real problem for Korean STT evaluation. The single dependency (jiwer) is lightweight. Best for teams evaluating Korean speech recognition systems or comparing STT services; less relevant if you work exclusively with non-Korean languages.

## Install

pip install nlptutti
uv add nlptutti
poetry add nlptutti

## Installing nlptutti

Before you install: Low friction: pure Python wheel with a single runtime dependency (jiwer). Active maintenance with recent release (2 days old) and 73 repository stars. Supports Python 3.8 through 3.14.

License in practice: MIT license permits unrestricted use, modification, and distribution with minimal restrictions—suitable for both open-source and commercial projects.

Quickstart:

pip install nlptutti

import nlptutti as metrics

reference = "오늘 날씨가 맑습니다"
hypothesis = "오늘 날씨는 맑습니다"

cer = metrics.get_cer(reference, hypothesis, rate_mode="standard")
print(round(cer["cer"], 4), cer["substitutions"])

Verify before relying:

- Whether jiwer's Levenshtein distance implementation handles Korean combining characters correctly without explicit NFC normalization
- Performance characteristics with large corpora (corpus size thresholds for practical use)

## Package facts

- License: MIT (permissive)
- Python support: supports_current
- Install friction: low
- Maintenance: active
- Downloads: 112.7K/month (top 15,000 on PyPI)
- Known vulnerabilities: none known

## Tags

korean stt evaluation metrics, cer wer measurement, speech recognition error rate, korean transcription quality, asr performance metrics, korean nlp evaluation, transcript accuracy testing, korean-nlp, speech-recognition, evaluation-metrics

[View on SkillFed](https://skillfed.io/packages/nlptutti) · [View on PyPI](https://pypi.org/project/nlptutti/)