nbmake

What it is and what it does

nbmake is a pytest plugin that treats Jupyter notebooks as executable test suites. It runs notebook cells sequentially using nbclient and the ipykernel, capturing output and failures just like pytest captures test results. You can configure cell timeouts, allow specific cells to raise exceptions, skip cells, mock variables mid-execution, and run post-cell assertions—all via notebook metadata. The plugin integrates with pytest's standard discovery and reporting, and supports parallel execution via pytest-xdist for faster CI runs.

The primary use case is maintaining documentation notebooks that must stay runnable and up-to-date. Research and ML engineers use it to ensure teaching materials and package examples don't break as dependencies evolve. It can optionally write executed notebooks back to disk, speeding up builds for documentation tools like jupyter-book and nbsphinx that consume the output. The tool assumes notebooks are primarily developed interactively but consumed as rendered docs, not as live notebooks.

Use it for:

• Validate that all code examples in a package's documentation notebooks execute without errors in CI.
• Test ML training notebooks by mocking long-running cells and asserting outputs at checkpoints.
• Ensure teaching material notebooks run end-to-end with current package versions before release.
• Parallelize execution of a large repository of slow notebooks to reduce CI time.
• Find missing imports across a directory of partially-working notebooks without running them to completion.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

A pytest plugin that executes Jupyter notebooks as tests, with support for parallel execution, cell-level error handling, timeouts, and optional output writeback for documentation builds.

Yes, if you maintain Jupyter notebooks as part of documentation, teaching materials, or package examples and want to automate their execution in CI. The low install friction and tight pytest integration make it straightforward to add. The aging maintenance status (599 days since last release) is a minor concern but not a blocker—the project is active and has no known vulnerabilities. Not necessary if notebooks are only consumed as rendered HTML and never need to be re-executed.

Install

nbmake on PyPI

pip install nbmake

uv add nbmake

poetry add nbmake

Installing nbmake

Before you install

Low friction install with a small, focused dependency tree (ipykernel, nbclient, nbformat, pygments, pytest). Last release was 599 days ago; the project is in aging maintenance but remains active with no archived status.

License in practice

Apache-2.0 permissive license allows use in commercial and private projects with minimal restrictions.

Quickstart

pip install pytest nbmake
pytest --nbmake **/*.ipynb

Requires Python >=3.8.0 and an active Jupyter kernel (default 'python3' or custom kernel via --nbmake-kernel flag).

Verify before relying

• Whether parallel execution with pytest-xdist is stable for large notebook suites in production CI environments.
• Performance characteristics when mocking or skipping cells in notebooks with hundreds of cells.
• Compatibility with recent versions of nbsphinx and jupyter-book beyond the documentation examples.

Package facts

Evidence: nbmake-1.5.5-py3-none-any.whl

Tags

More Testing packages