whisper-normalizer

What it is and what it does

whisper_normalizer is a Python implementation of OpenAI's Whisper text normalization algorithm, designed to standardize text for automatic speech recognition (ASR) systems. It reduces unintentional penalties in metrics like WER and CER by converting text in different styles into a standardized form, separating actual transcription errors from formatting or punctuation differences.

The package provides two main normalizers: EnglishTextNormalizer for English text and BasicTextNormalizer for general use. It extends the original Whisper approach with specialized support for Indic languages (including Malayalam) derived from the indic-nlp-library, addressing issues that arise when applying Whisper's default normalization to low-resource and script-heavy languages. The package depends on more_itertools, regex, indic-numtowords, and text2num.

Use it for:

• Normalize transcription output from speech-to-text models before evaluating accuracy metrics
• Preprocess text in Indic languages (Malayalam, Tamil, etc.) while preserving combining marks and diacritics
• Standardize English text for ASR evaluation pipelines compatible with Whisper, MMS, or Conformer models
• Apply consistent text normalization across multilingual speech recognition workflows

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Standardizes and normalizes text using the algorithm from OpenAI's Whisper model, with specialized support for Indic languages and other scripts that use combining marks.

Yes. The package is actively maintained, has no known vulnerabilities, carries a permissive MIT license, and solves a specific problem (Whisper-compatible text normalization) with low install friction. It is particularly valuable if you work with Indic languages or need to evaluate ASR systems using Whisper's normalization standard.

Install

whisper-normalizer on PyPI

pip install whisper-normalizer

uv add whisper-normalizer

poetry add whisper-normalizer

Installing whisper-normalizer

Before you install

Low friction: pure Python wheel with four runtime dependencies (more_itertools, regex, indic-numtowords, text2num). Repository is active with recent commits and no archived status.

License in practice

MIT license permits commercial and private use with minimal restrictions—suitable for most projects.

Quickstart

pip install whisper_normalizer

from whisper_normalizer.english import EnglishTextNormalizer
normalizer = EnglishTextNormalizer()
result = normalizer("I'm a little teapot, short and stout.")

Verify before relying

• Whether BasicTextNormalizer's preserve_marks=True parameter is available in version 0.1.15
• Performance characteristics when normalizing large text volumes
• Whether all Indic languages mentioned in the description are fully supported in this version

Package facts

Evidence: whisper_normalizer-0.1.15-py3-none-any.whl

Keywords: nbdev, jupyter, notebook, openai, whisper, normalizer

Tags

More Python Modules packages