ncnn

What it is and what it does

ncnn is a C++-based neural network inference engine with Python bindings that specializes in running pre-trained deep learning models efficiently on mobile, embedded, and desktop hardware. It provides CPU and Vulkan GPU backends and is designed to minimize latency and memory footprint for deployment scenarios where model training is not needed—only inference. The typical workflow involves converting a PyTorch or ONNX model to ncnn format using the pnnx tool, then loading and executing the converted model files (.param and .bin) through the Python API.

The package depends on numpy for array handling, opencv-python for image operations, and utility libraries (tqdm, requests, portalocker) for model downloading and progress tracking. It is actively maintained by Tencent and widely used in production applications. Pre-built wheels are available for many platforms and Python versions, though installation involves compiled binaries that may require platform-specific dependencies.

Use it for:

• Deploy trained PyTorch or ONNX models to mobile phones or edge devices for real-time inference without model training.
• Run computer vision models (image classification, object detection) on resource-constrained hardware with minimal latency.
• Accelerate inference on desktop or server CPUs using optimized C++ execution with Python control.
• Build inference pipelines that load and execute multiple pre-converted models sequentially or in parallel.
• Integrate deep learning inference into applications targeting iOS, Android, or embedded Linux platforms.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

ncnn is a neural network inference framework that loads and runs pre-trained deep learning models on CPU and Vulkan GPU backends, with Python bindings for model execution.

Yes, if you have pre-trained models to deploy and need efficient inference on mobile or embedded hardware. The permissive BSD-3 license and active maintenance are favorable. Install friction is moderate due to compiled wheels, but pre-built binaries for common platforms reduce friction. Not suitable if you need to train models or require a high-level training API—ncnn is inference-only. Verify that your target platform and model architecture are supported before committing.

Install

ncnn on PyPI

pip install ncnn

uv add ncnn

poetry add ncnn

Installing ncnn

Before you install

Medium install friction due to compiled wheels across many platforms (macOS, Linux, Windows, various architectures). Active maintenance with recent releases; last commit 2026-08-13. Depends on numpy, opencv-python, tqdm, requests, and portalocker.

License in practice

BSD-3 permissive license allows commercial and private use with minimal restrictions; you must include the license text in distributions.

Quickstart

pip install ncnn
import ncnn
import numpy as np

net = ncnn.Net()
net.load_param("model.ncnn.param")
net.load_model("model.ncnn.bin")

x = np.zeros((3, 224, 224), np.float32)
mat = ncnn.Mat(x)
ex = net.create_extractor()
ex.input("in0", mat)
ret, out = ex.extract("out0")

Requires pre-converted ncnn model files (.param and .bin); use pnnx to convert PyTorch or ONNX models first.

Verify before relying

• Whether Vulkan GPU acceleration is available on all supported platforms or requires additional system libraries.
• Performance benchmarks compared to other inference frameworks on target hardware.
• Supported model architectures and any limitations in layer coverage.

Package facts

Evidence: ncnn-1.0.20260526-cp310-cp310-macosx_11_0_arm64.whl; ncnn-1.0.20260526-cp310-cp310-macosx_11_0_x86_64.whl; ncnn-1.0.20260526-cp310-cp310-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl; ncnn-1.0.20260526-cp310-cp310-manylinux_2_24_i686.manylinux_2_28_i686.whl; ncnn-1.0.20260526-cp310-cp310-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl; ncnn-1.0.20260526-cp310-cp310-manylinux_2_31_armv7l.whl; ncnn-1.0.20260526-cp310-cp310-manylinux_2_39_riscv64.whl; ncnn-1.0.20260526-cp310-cp310-musllinux_1_2_aarch64.whl; ncnn-1.0.20260526-cp310-cp310-musllinux_1_2_armv7l.whl; ncnn-1.0.20260526-cp310-cp310-musllinux_1_2_i686.whl; ncnn-1.0.20260526-cp310-cp310-musllinux_1_2_riscv64.whl; ncnn-1.0.20260526-cp310-cp310-musllinux_1_2_x86_64.whl; ncnn-1.0.20260526-cp310-cp310-win32.whl; ncnn-1.0.20260526-cp310-cp310-win_amd64.whl; ncnn-1.0.20260526-cp310-cp310-win_arm64.whl; ncnn-1.0.20260526-cp311-cp311-macosx_11_0_arm64.whl; ncnn-1.0.20260526-cp311-cp311-macosx_11_0_x86_64.whl; ncnn-1.0.20260526-cp311-cp311-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl; ncnn-1.0.20260526-cp311-cp311-manylinux_2_24_i686.manylinux_2_28_i686.whl; ncnn-1.0.20260526-cp311-cp311-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl

Tags

More Artificial Intelligence packages

Further reading