rouge-metric

What it is and what it does

rouge-metric is a Python library for computing ROUGE (Recall-Oriented Understudy for Gisting Evaluation) metrics, the standard automatic evaluation framework for text summarization. It provides two implementations: a pure Python version that computes ROUGE-N, ROUGE-L, ROUGE-W, ROUGE-S, and ROUGE-SU scores without external process invocation, and a wrapper around the official ROUGE-1.5.5.pl Perl script for compatibility with existing workflows. The Python implementation is language-agnostic and treats documents as token sequences, letting you apply your own tokenization (e.g., nltk for English, jieba for Chinese) before scoring.

The package is designed for researchers and practitioners evaluating automatic summarization systems. It supports single and multiple references per hypothesis, batch evaluation over document collections, and both command-line and programmatic APIs. The pure Python path is fast and produces identical results to the Perl reference on single-document scenarios. However, the project is abandoned as of 2020 and receives no maintenance or updates.

Use it for:

• Evaluate generated summaries against reference summaries in a summarization research project or benchmark.
• Batch-score multiple hypothesis-reference pairs from files to compare summarization model outputs.
• Integrate ROUGE scoring into an NLP pipeline for multi-lingual text summarization with custom tokenization.
• Reproduce results from published summarization papers that report ROUGE-1, ROUGE-2, ROUGE-L metrics.
• Compare summarization systems using command-line ROUGE scoring without writing Python code.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Computes ROUGE metrics (ROUGE-N, ROUGE-L, ROUGE-W, ROUGE-S, ROUGE-SU) for automatic evaluation of text summarization quality, with both a pure Python implementation and a Perl script wrapper.

Yes, if you need ROUGE metrics for summarization evaluation and can tolerate an abandoned package. The library is stable, has low install friction, and produces correct results for its intended use. Install it for research or production summarization evaluation, but do not expect bug fixes or updates. Consider forking or switching to an actively maintained alternative if you encounter issues or need new features.

Install

rouge-metric on PyPI

pip install rouge-metric

uv add rouge-metric

poetry add rouge-metric

Installing rouge-metric

Before you install

Low install friction with only the standard library's typing module as a runtime dependency. However, the package is abandoned—last commit was 2021-02-26 and no releases since 2020-10-21—so expect no maintenance or bug fixes going forward.

License in practice

MIT license is permissive and imposes no restrictions on use, modification, or distribution in your own projects.

Quickstart

pip install rouge-metric

from rouge_metric import PyRouge

rouge = PyRouge(rouge_n=(1, 2), rouge_l=True)
hypotheses = ['how are you i am fine']
references = [['how do you do fine thanks']]
scores = rouge.evaluate(hypotheses, references)
print(scores)

The Perl script wrapper requires Perl to be installed; the pure Python implementation has no external dependencies beyond typing.

Verify before relying

• Whether the package's multi-document results differ materially from ROUGE-1.5.5.pl due to the absence of bootstrap resampling in practice.
• Current compatibility with Python versions released after 2020, given the abandoned maintenance status.

Package facts

Evidence: rouge_metric-1.0.1-py3-none-any.whl

Keywords: rouge, summarization, natural language processing, computational linguistics

Tags

More Linguistic packages