nbval

What it is and what it does

nbval is a pytest plugin that integrates Jupyter notebooks into your test suite. Instead of manually running notebooks to check outputs, nbval treats each code cell as a test case, executing it and comparing the actual output against what was stored in the .ipynb file. This ensures that reference notebooks, documentation, and examples stay in sync with their expected behavior and don't break silently over time.

The plugin works by launching an IPython kernel, sending each cell's code to it via jupyter-client, and collecting the output messages. It supports output sanitization via regex rules, integration with pytest-cov for coverage reporting, and parallel execution via pytest-xdist (with the `--dist loadscope` flag). You can run all notebooks in a directory with `pytest --nbval`, or target specific files, and use `--nbval-lax` to only check cells marked with `#NBVAL_CHECK_OUTPUT`.

Use it for:

• Validate that tutorial and documentation notebooks execute without errors and produce expected outputs.
• Catch breaking changes in dependencies by running notebooks as part of your CI/CD pipeline.
• Ensure reproducibility of computational results stored in reference notebooks.
• Generate code coverage reports from notebook execution using pytest-cov integration.
• Run notebooks in parallel across multiple workers while keeping each notebook's cells on the same kernel.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

A pytest plugin that treats Jupyter notebooks as test files, executing each cell and validating that outputs match the stored results in the .ipynb file.

Yes, if you maintain Jupyter notebooks as part of your documentation or reference material and want to ensure they stay executable and correct. The low install friction and permissive license make it a practical choice. However, note that the project is aging (last release March 2024)—verify compatibility with your current pytest and Jupyter versions before relying on it in new projects.

Install

nbval on PyPI

pip install nbval

uv add nbval

poetry add nbval

Installing nbval

Before you install

Low install friction with a pure-Python wheel. Maintenance is aging—last release was in March 2024—but the repository remains active and unarchived with modest community engagement (453 stars).

License in practice

BSD license (permissive) means you can use, modify, and distribute nbval freely in commercial and private projects with minimal restrictions.

Quickstart

pip install nbval
pytest --nbval my_notebook.ipynb

Requires pytest, jupyter-client, and ipykernel to be installed; notebooks must be in .ipynb format.

Verify before relying

• Whether the aging maintenance status (893 days since last release) affects compatibility with recent Jupyter/pytest versions.
• Performance characteristics when running large notebooks or many notebooks in parallel.

Package facts

Evidence: nbval-0.11.0-py2.py3-none-any.whl

Tags

More Testing packages