rouge-score

What it is and what it does

rouge-score is a pure Python implementation of ROUGE, the automatic evaluation metric for text summarization. It replicates the original Perl ROUGE package's behavior, implementing ROUGE-N (n-gram overlap), ROUGE-L (longest common subsequence at sentence level), and ROUGE-Lsum (summary-level LCS with union computation). The package also supports optional Porter stemming and bootstrap resampling for confidence intervals, with text normalization built in.

The package is designed for researchers and practitioners who need to evaluate generated summaries against reference summaries. It can be used programmatically via RougeScorer or as a command-line tool to batch-score target and prediction files. No external dependencies are required at runtime, though the source distribution has high install friction.

Use it for:

• Evaluate machine-generated summaries in NLP research pipelines against gold-standard references.
• Benchmark summarization models during development and compare results with published papers.
• Batch-score large sets of predictions and targets from command line with CSV output.
• Compute ROUGE metrics with optional stemming to handle morphological variations in summary text.
• Calculate confidence intervals via bootstrap resampling to assess score stability.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Computes ROUGE scores (ROUGE-N, ROUGE-L, ROUGE-Lsum) to evaluate the quality of generated text summaries against reference summaries using n-gram overlap and longest common subsequence metrics.

Yes, with conditions. rouge-score is the standard pure-Python ROUGE implementation and is actively maintained with no known vulnerabilities. Install it if you need ROUGE metrics for summarization evaluation and can tolerate high install friction from source-only distribution. The gap since the last release in 2022 is a minor concern for stability but not a blocker given active repository maintenance. Suitable for research, benchmarking, and production evaluation pipelines.

Install

rouge-score on PyPI

pip install rouge-score

uv add rouge-score

poetry add rouge-score

Installing rouge-score

Before you install

High install friction: the package is distributed as a source tarball with no wheels, requiring compilation or build tools on installation. Maintenance is active with recent commits, but the latest release was in 2022.

License in practice

Licensed under Apache 2.0 (permissive), allowing commercial and private use with minimal restrictions. No notable licensing constraints for typical adoption.

Quickstart

pip install rouge-score

from rouge_score import rouge_scorer

scorer = rouge_scorer.RougeScorer(['rouge1', 'rougeL'], use_stemmer=True)
scores = scorer.score('The quick brown fox jumps over the lazy dog',
                      'The quick brown dog jumps on the log.')

Requires Python >= 3.7; high install friction due to source-only distribution (no wheels).

Verify before relying

• Whether the package's implementation continues to match the original perl ROUGE results given the time since last release.
• Performance characteristics when scoring large batches of summaries or very long texts.
• Whether bootstrap resampling for confidence intervals is documented with examples.

Package facts

Evidence: rouge_score-0.1.2.tar.gz

Tags

More Artificial Intelligence packages