---
id: pyannote-metrics
version: "4.1"
license: unclear
license_treatment: unclear
maintenance: active
---

# pyannote-metrics — A toolkit for reproducible evaluation, diagnostic, and error analysis of speaker diarization systems

License: unclear · Maintenance: active · Downloads: 2.9M/mo

## What it is and what it does

pyannote.metrics is a toolkit for evaluating speaker diarization systems—the task of determining who spoke when in an audio recording. It computes standard metrics (like diarization error rate), provides diagnostic breakdowns of errors, and supports reproducible evaluation workflows. The package wraps numpy, pandas, scikit-learn, and scipy to handle metric computation and analysis, and integrates with pyannote-core and pyannote-database for data representation.

It is designed for researchers and engineers building or benchmarking speaker diarization pipelines. Rather than a diarization system itself, it is the measurement and analysis layer: you provide reference (ground truth) and hypothesis (predicted) speaker segmentations, and the toolkit computes how well they match, where errors occur, and why. The package has been actively maintained since 2014 and is widely used in the speech processing community.

Use it for:

- Compute diarization error rate (DER) and other standard metrics to benchmark a speaker diarization model against a reference annotation.
- Analyze where a diarization system fails—false alarms, missed speech, speaker confusion—to guide model improvements.
- Generate reproducible evaluation reports for research papers or system documentation.
- Compare multiple diarization hypotheses against the same reference to rank or select the best performer.
- Integrate metric computation into a diarization pipeline's validation or test harness.

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

Evaluates and analyzes speaker diarization systems by computing metrics, diagnostics, and error analysis for reproducible assessment of who-spoke-when predictions.

Yes, if you are building or evaluating speaker diarization systems. The package is actively maintained, has low install friction, and is a standard tool in the diarization research community. However, verify the license terms first, as the metadata does not declare one explicitly. Not relevant for applications outside speaker diarization.

## Install

pip install pyannote-metrics
uv add pyannote-metrics
poetry add pyannote-metrics

## Installing pyannote-metrics

Before you install: Low friction installation with a pure-Python wheel and six standard scientific dependencies (numpy, pandas, scikit-learn, scipy) plus two pyannote libraries. Actively maintained as of 100 days ago.

License in practice: License status is unclear; no SPDX identifier or raw license text is available in the package metadata. Verify the actual license terms before use in proprietary or restricted contexts.

Quickstart:

pip install pyannote.metrics

from pyannote.metrics import DiarizationErrorRate
metric = DiarizationErrorRate()
der = metric(reference, hypothesis)

Requires Python >= 3.10.

Verify before relying:

- What license governs pyannote.metrics (unclear in metadata)?
- Does the package support evaluation of speaker overlap or only sequential speaker turns?
- What is the computational cost for large audio files or many speakers?

## Package facts

- License: not declared (unclear)
- Python support: supports_current
- Install friction: low
- Maintenance: active
- Downloads: 2.9M/month (top 5,000 on PyPI)
- Known vulnerabilities: none known

## Tags

speaker diarization evaluation metrics, diarization error analysis, speaker segmentation assessment, reproducible diarization evaluation, speaker turn detection metrics, diarization diagnostic tools, speaker overlap detection metrics, speaker-diarization, evaluation-metrics, audio-processing

[View on SkillFed](https://skillfed.io/packages/pyannote-metrics) · [View on PyPI](https://pypi.org/project/pyannote-metrics/)