{"categories":[{"label":"Python Modules","url":"https://skillfed.io/packages/category/software-development-libraries-python-modules/17"},{"label":"Education","url":"https://skillfed.io/packages/category/education"}],"enrichment":{"capability":"Rubric is a Python library for evaluating LLM outputs against weighted criteria using structured rubrics, returning normalized or raw scores based on how many criteria are met.","skillfed_tags":["llm-evaluation","structured-grading","async-framework"],"use_cases":["Evaluate chatbot or RAG system responses against domain-specific quality criteria with weighted importance.","Detect and penalize specific errors in LLM outputs (e.g., hallucinations, missing facts) using negative-weight criteria.","Compare multiple LLM models or prompts by scoring their outputs on the same rubric and aggregating results.","Build automated grading pipelines for educational or content-generation workflows with custom scoring logic.","Validate structured outputs from LLMs by checking for required fields or format compliance as pass/fail criteria."],"what_it_does":"Rubric is a Python library that automates evaluation of LLM outputs by scoring them against weighted criteria using an LLM as the judge. You define a rubric as a list of requirements, each with a weight; the library calls an LLM (via a pluggable generate function) to evaluate whether each criterion is met, then aggregates those verdicts into a final score. The score can be normalized to 0\u20131 or returned as a raw weighted sum.\n\nThe library ships with three evaluation strategies: PerCriterionGrader makes one inference call per criterion in parallel; PerCriterionOneShotGrader makes a single call for all criteria; and RubricAsJudgeGrader asks the LLM for a holistic 0\u2013100 score. You can use the built-in Gemini integration for quick testing or plug in any LLM provider (OpenAI, Anthropic, local models) by writing a custom async generate function. All grading is async and uses Pydantic for structured outputs.","worth_installing":"Yes, if you need to evaluate LLM outputs programmatically against weighted criteria. The library is young (Alpha, 205 days old) but actively maintained, has low install friction, and offers a clean abstraction over LLM-based grading with multiple evaluation strategies. Start with the default Gemini integration for prototyping; for production, implement a custom generate_fn with your preferred LLM provider. No known security vulnerabilities."},"id":"rubric","links":{"html":"https://skillfed.io/packages/rubric","md":"https://skillfed.io/packages/rubric.md","pypi":"https://pypi.org/project/rubric/"},"maintenance":{"status":"aging"},"meta":{"latest_release":"2026-01-21","license_spdx":"MIT","license_treatment":"permissive","name":"rubric","python_support":"supports_current","summary":"rubric"},"popularity":{"monthly_downloads":150565,"position":10961,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"2.2.0"}