---
id: inspect-ai
version: "0.3.258"
license: MIT License
license_treatment: permissive
maintenance: active
---

# inspect-ai — Framework for large language model evaluations

License: permissive · Maintenance: active · Downloads: 12.1M/mo

## What it is and what it does

Inspect is a framework for evaluating large language models, created by the UK AI Security Institute. It provides a structured environment for running evaluations on any model, with built-in support for prompt engineering, tool use, multi-turn dialogue, and model-graded scoring. The framework includes over 200 pre-built evaluations ready to run out of the box.

The package has a substantial dependency graph (40 runtime dependencies including pydantic, fastapi, boto3, numpy, and AWS integration libraries) reflecting its role as a full-featured evaluation platform. It supports both Python development workflows and web-based UI interaction through a TypeScript/React frontend. Active maintenance and recent releases indicate ongoing development.

Use it for:

- Run standardized evaluations against multiple LLM providers to compare model performance on safety, reasoning, and capability benchmarks
- Build custom evaluation pipelines combining prompt engineering, tool use, and model-graded scoring for domain-specific tasks
- Integrate LLM evaluation into CI/CD workflows to validate model behavior before deployment
- Develop and share new evaluation techniques as Python extensions compatible with the Inspect framework

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

Inspect is a framework for evaluating large language models, providing built-in components for prompt engineering, tool use, multi-turn dialogue, and model-graded scoring across any model.

Yes, if you need to evaluate LLMs systematically. The framework is actively maintained, permissively licensed, and backed by a government security institute. The 40 runtime dependencies and Python 3.10+ requirement are substantial but typical for a full-featured evaluation platform. Install if you're building evaluation pipelines or comparing model outputs; skip if you only need lightweight model testing.

## Install

pip install inspect-ai
uv add inspect-ai
poetry add inspect-ai

## Installing inspect-ai

Before you install: Low friction installation with a wheel distribution. Active maintenance—released 2 days ago with a commit history through 2026-08-14. Requires Python 3.10 or later.

License in practice: MIT License permits commercial and private use with minimal restrictions, making it suitable for most projects without licensing concerns.

Quickstart:

pip install inspect-ai

import inspect_ai
# See https://inspect.aisi.org.uk/ for evaluation setup and model configuration

Requires Python 3.10 or later. 40 runtime dependencies including pydantic, fastapi, boto3, and numpy—ensure your environment can resolve them.

Verify before relying:

- Whether the 200+ pre-built evaluations are sufficient for your specific model types and use cases
- Performance characteristics when running evaluations against multiple models concurrently
- Extensibility mechanism details for custom elicitation and scoring techniques beyond built-in components

## Package facts

- License: MIT License (permissive)
- Python support: supports_current
- Install friction: low
- Maintenance: active
- Downloads: 12.1M/month (top 5,000 on PyPI)
- Known vulnerabilities: none known

## Tags

llm evaluation framework, language model testing, ai model benchmarking, prompt engineering tools, model grading system, llm-evaluation, model-testing, ai-security

[View on SkillFed](https://skillfed.io/packages/inspect-ai) · [View on PyPI](https://pypi.org/project/inspect-ai/)