whisper-timestamped

What it is and what it does

whisper-timestamped extends OpenAI's Whisper speech recognition model to predict word-level timestamps and confidence scores for each word in a transcription. Rather than Whisper's native segment-level timestamps (typically 1-second accuracy), this package uses Dynamic Time Warping applied to the model's cross-attention weights to align words precisely with their spoken timing. It processes long audio files with minimal additional memory overhead and works across Whisper's supported languages.

The package integrates optional Voice Activity Detection (VAD) to reduce hallucinations on silence, provides language detection confidence when the language is unspecified, and aims to do word alignment without extra inference steps when possible. It is designed as a drop-in extension to openai-whisper, maintaining API compatibility while adding these timing and confidence features for applications requiring precise word-level synchronization.

Use it for:

• Generate subtitle files with accurate word-level timing for video synchronization and accessibility.
• Extract precise timestamps for each word to enable interactive transcript playback or speaker diarization.
• Detect and filter out hallucinated speech on silence using VAD before transcription to improve accuracy.
• Build searchable transcripts where users can click a word to jump to its exact position in audio.
• Analyze speech patterns by correlating word timing with confidence scores to identify disfluencies or hesitations.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Adds word-level timestamps and confidence scores to OpenAI's Whisper speech recognition model using Dynamic Time Warping on cross-attention weights, supporting multilingual ASR with optional voice activity detection.

Yes, if you need word-level timestamps for Whisper transcriptions and accept GPLv3 copyleft constraints. The package has low install friction, active maintenance, and no known vulnerabilities. Suitable for open-source projects, research, and accessibility tools; less suitable for proprietary closed-source products due to license. Aging status (339 days since release) is acceptable given recent repository activity.

Install

whisper-timestamped on PyPI

pip install whisper-timestamped

uv add whisper-timestamped

poetry add whisper-timestamped

Installing whisper-timestamped

Before you install

Low friction installation via pip; depends on Cython, dtw-python, and openai-whisper. Package is aging (339 days since last release) but repository remains active with 2837 stars and recent commits; maintenance signal is moderate.

License in practice

GPLv3 copyleft license means any derivative work or distribution must also be open-source under GPLv3; suitable for open-source projects but requires careful consideration in proprietary or closed-source contexts.

Quickstart

pip install whisper-timestamped

import whisper_timestamped as whisper
result = whisper.transcribe(model, "audio.mp3")

Requires ffmpeg installed on system; Python >=3.7 (3.9+ recommended); openai-whisper must be installed as a runtime dependency.

Verify before relying

• Whether word-timestamp accuracy remains within stated bounds across all supported languages and audio conditions.
• Performance impact of DTW alignment on inference time for typical audio lengths.
• Compatibility guarantees with future versions of openai-whisper beyond the stated 'any version' claim.

Package facts

Evidence: whisper_timestamped-1.15.9-py3-none-any.whl

Tags

More Linguistic packages

Further reading