{"categories":[{"label":"Testing","url":"https://skillfed.io/packages/category/software-development-testing"}],"enrichment":{"capability":"Harbor Rewardkit defines and runs verifiers for task evaluation, supporting programmatic criteria, LLM judges, and agent judges with MCP server integration.","skillfed_tags":["llm-evaluation","task-grading","mcp-agents"],"use_cases":["Score code quality or correctness by having Claude evaluate generated source files against criteria.","Verify task outputs with simple file-based checks (existence, content matching) without LLM overhead.","Build agent judges that navigate web pages or interact with systems via MCP tools to validate rendered output.","Grade benchmark submissions across multiple criteria in a CI/CD pipeline.","Evaluate multi-step task completion by combining programmatic checks with LLM judgment."],"what_it_does":"Harbor Rewardkit is a lightweight grading toolkit for evaluating environment-based tasks. It lets you define verifiers using three approaches: simple programmatic criteria (file checks), LLM judges that score outputs using Claude or other models, and agent judges that can call MCP server tools during evaluation. The package integrates with the Harbor task framework but can be used independently.\n\nYou define criteria in TOML files or Python code, then run the verifier against task outputs. It handles LLM model selection, tool access control via allowed_tools lists, and binary or scored evaluation results. The single runtime dependency is litellm, which abstracts LLM provider APIs.","worth_installing":"Yes, if you need lightweight task evaluation with LLM or agent judges. The package is actively maintained, has low install friction, and offers a clean API for both simple checks and complex agent-based grading. Best suited for Harbor framework users or teams building evaluation pipelines for AI-generated outputs."},"id":"harbor-rewardkit","links":{"html":"https://skillfed.io/packages/harbor-rewardkit","md":"https://skillfed.io/packages/harbor-rewardkit.md","pypi":"https://pypi.org/project/harbor-rewardkit/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2026-06-27","license_spdx":"Apache-2.0","license_treatment":"permissive","name":"harbor-rewardkit","python_support":"supports_current","summary":"Lightweight grading toolkit for environment-based tasks."},"popularity":{"monthly_downloads":7739787,"position":1700,"tier":"top_5000"},"security":{"n_vulnerabilities":0},"version":"0.1.7"}