pyannote-metrics

What it is and what it does

pyannote.metrics is a toolkit for evaluating speaker diarization systems—the task of determining who spoke when in an audio recording. It computes standard metrics (like diarization error rate), provides diagnostic breakdowns of errors, and supports reproducible evaluation workflows. The package wraps numpy, pandas, scikit-learn, and scipy to handle metric computation and analysis, and integrates with pyannote-core and pyannote-database for data representation.

It is designed for researchers and engineers building or benchmarking speaker diarization pipelines. Rather than a diarization system itself, it is the measurement and analysis layer: you provide reference (ground truth) and hypothesis (predicted) speaker segmentations, and the toolkit computes how well they match, where errors occur, and why. The package has been actively maintained since 2014 and is widely used in the speech processing community.

Use it for:

• Compute diarization error rate (DER) and other standard metrics to benchmark a speaker diarization model against a reference annotation.
• Analyze where a diarization system fails—false alarms, missed speech, speaker confusion—to guide model improvements.
• Generate reproducible evaluation reports for research papers or system documentation.
• Compare multiple diarization hypotheses against the same reference to rank or select the best performer.
• Integrate metric computation into a diarization pipeline's validation or test harness.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Evaluates and analyzes speaker diarization systems by computing metrics, diagnostics, and error analysis for reproducible assessment of who-spoke-when predictions.

Yes, if you are building or evaluating speaker diarization systems. The package is actively maintained, has low install friction, and is a standard tool in the diarization research community. However, verify the license terms first, as the metadata does not declare one explicitly. Not relevant for applications outside speaker diarization.

Install

pyannote-metrics on PyPI

pip install pyannote-metrics

uv add pyannote-metrics

poetry add pyannote-metrics

Installing pyannote-metrics

Before you install

Low friction installation with a pure-Python wheel and six standard scientific dependencies (numpy, pandas, scikit-learn, scipy) plus two pyannote libraries. Actively maintained as of 100 days ago.

License in practice

License status is unclear; no SPDX identifier or raw license text is available in the package metadata. Verify the actual license terms before use in proprietary or restricted contexts.

Quickstart

pip install pyannote.metrics

from pyannote.metrics import DiarizationErrorRate
metric = DiarizationErrorRate()
der = metric(reference, hypothesis)

Requires Python >= 3.10.

Verify before relying

• What license governs pyannote.metrics (unclear in metadata)?
• Does the package support evaluation of speaker overlap or only sequential speaker turns?
• What is the computational cost for large audio files or many speakers?

Package facts

Evidence: pyannote_metrics-4.1-py3-none-any.whl

Tags

More Artificial Intelligence packages