nncf

What it is and what it does

NNCF is a framework for compressing neural networks to reduce inference latency and model size. It implements post-training algorithms (quantization, weight compression, activation sparsity) and training-time methods (quantization-aware training, pruning, weight-only quantization with LoRA). The framework works with PyTorch, TorchFX, ONNX, and OpenVINO models, offering a unified Python API for model graph transformation and compression.

The package is designed for practitioners who need to optimize models for deployment on resource-constrained devices or edge hardware. It integrates into third-party ecosystems including HuggingFace Optimum Intel, Ultralytics, ExecuTorch, and torch.compile, making it accessible within existing training and export pipelines. Runtime dependencies include networkx, numpy, scikit-learn, scipy, and visualization tools like pydot and rich.

Use it for:

• Quantize large language models (e.g., Llama) to INT4 weights for efficient OpenVINO deployment via HuggingFace Optimum.
• Apply post-training quantization to ONNX models without retraining to reduce inference latency.
• Fine-tune PyTorch models with quantization-aware training to maintain accuracy while reducing model size.
• Prune and sparsify neural networks during training to accelerate inference on edge devices.
• Export PyTorch models to OpenVINO format with INT4-quantized weights for production deployment.
• Integrate compression into custom training pipelines via the unified NNCF API and Git patch examples.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

NNCF provides post-training and training-time compression algorithms for neural networks, optimizing inference in OpenVINO, PyTorch, TorchFX, and ONNX with minimal accuracy loss.

Yes. NNCF is actively maintained, has no known vulnerabilities, installs with low friction, and is widely adopted in production ecosystems (HuggingFace, Ultralytics, ExecuTorch, torch.compile). Choose it if you need to compress models for inference optimization across PyTorch, ONNX, or OpenVINO. Requires Python 3.10+; verify backend-specific system requirements for your target framework.

Install

nncf on PyPI

pip install nncf

uv add nncf

poetry add nncf

Installing nncf

Before you install

Low friction install as a pure Python wheel. Active maintenance with a recent release (9 days old) and steady repository activity. Requires Python 3.10 or later and pulls in 12 runtime dependencies including numpy, scikit-learn, and scipy.

License in practice

Apache-2.0 permissive license allows commercial and private use with minimal restrictions; you must include a copy of the license and state significant changes.

Quickstart

pip install nncf

import nncf
from nncf import compress_model
# Compress a model loaded from PyTorch, ONNX, or OpenVINO

Requires Python 3.10 or later; backend-specific system requirements depend on whether you use PyTorch, ONNX, or OpenVINO models.

Verify before relying

• Specific accuracy drop ranges for different compression algorithms and model types.
• Performance benchmarks (inference speedup, model size reduction) for representative models.
• GPU memory requirements for training-time compression with distributed training.
• Compatibility matrix details between NNCF version 3.3.0 and specific PyTorch/ONNX/OpenVINO versions.

Package facts

Evidence: nncf-3.3.0-py3-none-any.whl

Keywords: bert, classification, compression, hawq, mixed-precision-training, mmdetection, nas, nlp, object-detection, pruning, quantization, quantization-aware-training, semantic-segmentation, sparsity, transformers

Tags

More Artificial Intelligence packages

Further reading