---
id: nvidia-nat-eval
version: "1.8.0"
license: Apache-2.0
license_treatment: permissive
maintenance: active
---

# nvidia-nat-eval — Subpackage for evaluation in NVIDIA NeMo Agent Toolkit

License: permissive · Maintenance: active · Downloads: 171.4K/mo

## What it is and what it does

nvidia-nat-eval is a subpackage of the NVIDIA NeMo Agent Toolkit that adds evaluation capabilities for testing and assessing agent systems. It provides evaluation-specific components accessible through the nat.plugins.eval namespace, along with command-line tools for running evaluations. The package depends on nvidia-nat-atif as its sole runtime dependency.

This package targets developers building or testing AI agents, particularly those using retrieval-augmented generation patterns. It supports Python 3.11, 3.12, and 3.13, with active maintenance as of August 2026.

Use it for:

- Evaluate agent response quality in RAG-based systems during development
- Run CLI-based evaluation workflows as part of agent testing pipelines
- Assess agent behavior and performance metrics during iteration
- Integrate evaluation checks into validation workflows

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

Provides evaluation components and CLI commands for the NVIDIA NeMo Agent Toolkit, enabling assessment of agent performance.

Yes, if you are building or testing agents within the NVIDIA NeMo Agent Toolkit ecosystem. The package has low install friction, active maintenance, permissive licensing, and no known vulnerabilities. Install only if you need evaluation capabilities; it is a specialized subpackage.

## Install

pip install nvidia-nat-eval
uv add nvidia-nat-eval
poetry add nvidia-nat-eval

## Installing nvidia-nat-eval

Before you install: Low install friction with a single runtime dependency on nvidia-nat-atif. Actively maintained with recent commits and part of a popular toolkit (2576 GitHub stars).

License in practice: Licensed under Apache-2.0 (permissive), allowing free use, modification, and distribution with minimal restrictions.

Quickstart:

pip install nvidia-nat-eval

import nat.plugins.eval

Requires Python 3.11 or later (3.11, 3.12, or 3.13 supported)

Verify before relying:

- What specific evaluation metrics or methodologies the package implements
- Whether evaluation components work standalone or require other toolkit packages
- What CLI commands are available and their typical use patterns
- How the nat.plugins.eval namespace integrates with broader toolkit workflows

## Package facts

- License: Apache-2.0 (permissive)
- Python support: supports_current
- Install friction: low
- Maintenance: active
- Downloads: 171.4K/month (top 15,000 on PyPI)
- Known vulnerabilities: none known

## Tags

agent evaluation framework, nemo toolkit evaluation, ai agent testing tools, rag evaluation, agent performance assessment, agent-evaluation, nemo-toolkit, rag-systems

[View on SkillFed](https://skillfed.io/packages/nvidia-nat-eval) · [View on PyPI](https://pypi.org/project/nvidia-nat-eval/)