---
id: terminal-bench
version: "0.2.18"
license: unclear
license_treatment: unclear
maintenance: aging
---

# terminal-bench — Terminal-bench is a collection of tasks and evaluation harness for evaluating AI agents' ability to complete complex tasks in terminal environments.

License: unclear · Maintenance: aging · Downloads: 90.9K/mo

## What it is and what it does

Terminal-Bench is a benchmarking framework designed to evaluate how well AI agents can autonomously complete real-world terminal tasks. It consists of a dataset of approximately 100 tasks (each with an English instruction, test script, and reference solution) and an execution harness that connects language models to a sandboxed terminal environment. The harness supports multiple LLM providers via integrations with anthropic, openai, and litellm, and can run tasks concurrently across Docker containers.

The package is aimed at researchers and developers building LLM agents, benchmarking frameworks, or stress-testing system-level reasoning capabilities. It provides a reproducible, practical evaluation suite for terminal-based workflows—tasks range from compiling code to training models to setting up servers. The CLI tool `tb` lets you run evaluations, submit results to a leaderboard, and contribute new tasks or adapters to the community.

Use it for:

- Evaluate an LLM agent's ability to autonomously complete real-world terminal tasks and compare performance across models
- Benchmark a custom AI agent framework against a standardized task suite with reproducible results
- Stress-test system-level reasoning in language models by running end-to-end workflows like server setup or model training
- Contribute new terminal-based tasks to the Terminal-Bench dataset to expand the evaluation suite
- Submit agent evaluation results to the Terminal-Bench leaderboard for public comparison

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

Terminal-Bench provides a benchmark suite and execution harness for evaluating AI agents' ability to complete real-world terminal tasks autonomously, from code compilation to server setup.

Yes, with conditions. Install if you are actively developing or evaluating AI agents for terminal tasks and can meet the Python 3.12+ and Docker requirements. The package has low install friction and no known vulnerabilities, but verify the license terms before use in commercial contexts. The aging maintenance status and unclear license are minor concerns; the framework is still actively used for agent benchmarking and leaderboard submissions.

## Install

pip install terminal-bench
uv add terminal-bench
poetry add terminal-bench

## Installing terminal-bench

Before you install: Low install friction with a pure-Python wheel distribution. The package is aging (322 days since release) but remains actively maintained; however, it requires Python 3.12+ and depends on 20 runtime packages including Docker, which adds system-level prerequisites beyond the Python ecosystem.

License in practice: License treatment is unclear—no SPDX identifier or raw license text is available in the metadata. Before relying on this package in a commercial or copyleft-sensitive context, verify the actual license from the project repository or documentation.

Quickstart:

pip install terminal-bench

from terminal_bench import run

# Run benchmark with an agent and model
run(agent='terminus', model='anthropic/claude-3-7-latest', dataset_name='terminal-bench-core')

Requires Python 3.12+, Docker, and the uv package manager for full functionality. The execution harness also needs a sandboxed terminal environment to run tasks.

Verify before relying:

- Whether the package is suitable for production use or remains experimental despite beta status
- Exact license terms and any restrictions on commercial or derivative use
- Performance characteristics and typical runtime for the ~100 tasks in the current dataset

## Package facts

- License: not declared (unclear)
- Python support: supports_current
- Install friction: low
- Maintenance: aging
- Downloads: 90.9K/month (top 15,000 on PyPI)
- Known vulnerabilities: none known

## Tags

ai agent benchmarking terminal, llm evaluation framework, autonomous task testing, agent performance evaluation, terminal environment sandbox, ai agent stress testing, end-to-end task benchmark, ai-agent-eval, benchmark-suite, terminal-automation

[View on SkillFed](https://skillfed.io/packages/terminal-bench) · [View on PyPI](https://pypi.org/project/terminal-bench/)