---
id: sacrebleu
version: "2.6.0"
license: Apache-2.0
license_treatment: permissive
maintenance: active
---

# sacrebleu — Hassle-free computation of shareable, comparable, and reproducible BLEU, chrF, and TER scores

License: permissive · Maintenance: active · Downloads: 4.1M/mo

## What it is and what it does

SacreBLEU wraps the original BLEU reference implementation with added conveniences for machine translation research. It computes BLEU, chrF, chrF++, and TER metrics on detokenized text, handling tokenization automatically according to WMT standards. The package automatically downloads and manages common test sets (like wmt14, wmt17, wmt21) so you can score against them by name rather than hunting down local files. It outputs results with a version signature that documents exactly how the score was computed, making cross-paper comparisons straightforward.

The tool works both as a command-line utility and as a Python library. It supports multiple tokenizers including language-specific ones for Japanese and Chinese, performs statistical significance testing via paired bootstrap resampling and approximate randomization, and defaults to JSON output (with a text format option for backward compatibility). Dependencies include portalocker, regex, tabulate, numpy, colorama, and lxml for various scoring and output formatting tasks.

Use it for:

- Score machine translation system outputs against standard WMT benchmarks without manually downloading or preprocessing test sets
- Compare BLEU scores across different papers and systems using the reproducible version signature to ensure methodological consistency
- Evaluate translation quality using multiple metrics (BLEU, chrF, TER) in a single command with consistent tokenization
- Perform statistical significance testing on paired translation outputs to determine if improvements are meaningful
- Integrate MT evaluation into research pipelines via the Python API for automated scoring of experimental outputs

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

SacreBLEU computes BLEU, chrF, and TER scores for machine translation evaluation with automatic test set management and reproducible, comparable results across systems.

Yes. SacreBLEU is actively maintained, has no security vulnerabilities, installs with low friction, and is the standard tool for reproducible machine translation evaluation in the research community. The Apache-2.0 license poses no restrictions. Install it if you work with machine translation evaluation, benchmark scoring, or need comparable BLEU/chrF/TER metrics across systems.

## Install

pip install sacrebleu
uv add sacrebleu
poetry add sacrebleu

## Installing sacrebleu

Before you install: Low friction installation with six runtime dependencies already packaged as a wheel. Active maintenance with last commit on 2026-07-17 and a release on 2026-01-12; no security vulnerabilities reported.

License in practice: Apache-2.0 permissive license allows use in commercial and proprietary projects without copyleft obligations.

Quickstart:

pip install sacrebleu

from sacrebleu import BLEU
bleu = BLEU()
score = bleu.corpus_score(hypotheses, [references])
print(score.score)

Requires Python 3.9 or later. Japanese and Korean tokenizer support requires optional dependencies installed via sacrebleu[ja] or sacrebleu[ko].

Verify before relying:

- Whether the package's Python API (corpus_score, etc.) is documented or stable across versions
- Performance characteristics when scoring large translation corpora
- Whether all WMT test sets mentioned in the description are currently available for download

## Package facts

- License: Apache-2.0 (permissive)
- Python support: supports_current
- Install friction: low
- Maintenance: active
- Downloads: 4.1M/month (top 5,000 on PyPI)
- Known vulnerabilities: none known

## Tags

BLEU score computation, machine translation evaluation, MT metric scoring, translation quality assessment, chrF TER metrics, WMT test sets, reproducible translation metrics, machine-translation, evaluation-metrics, nlp-research

[View on SkillFed](https://skillfed.io/packages/sacrebleu) · [View on PyPI](https://pypi.org/project/sacrebleu/)