diffq

What it is and what it does

DiffQ is a PyTorch quantization framework that reduces model size by automatically determining optimal bit-widths for individual weights or weight groups during training. It uses pseudo quantization noise injection to make the quantization process differentiable, allowing the bit allocation itself to be optimized as part of the training loop alongside model weights.

The package integrates with standard PyTorch training pipelines: you attach a DiffQuantizer to your model before creating the optimizer, then add a model-size penalty term to your loss function. During training, the quantizer learns which weights need more bits and which can use fewer, trading off compression against accuracy. At inference, it automatically switches to true quantized weights, and you can export the compressed model to disk or TorchScript format.

Use it for

• Compress large transformer or CNN models for deployment on memory-constrained devices while maintaining accuracy
• Automatically determine per-layer or per-group bit allocations during training without manual hyperparameter tuning
• Export quantized models to TorchScript for optimized inference with reduced memory footprint
• Research differentiable quantization methods and compare compression-accuracy tradeoffs across architectures
• Reduce model size for distributed training or inference in bandwidth-limited environments

Worth the install?

AI-flagged interpretation of the facts on this page. Verify before relying on it.

Install

diffq on PyPI

Before you install

Medium install friction due to compiled dependencies (Cython, numpy, torch). The package is abandoned as of 2023-05-05 with no recent maintenance, so expect no bug fixes or compatibility updates.

Requires Python 3.7 or later and PyTorch 1.7.1 ideally. Compiled dependencies (Cython, numpy, torch) must be installed; prebuilt wheels are available for common platforms but may not exist for all architectures.

License in practice

Licensed under CC-BY-NC 4.0 (noncommercial), which restricts use to non-commercial purposes only. Commercial deployment or integration into proprietary products is not permitted without explicit permission.

Quickstart

pip install diffq

import torch
from diffq import DiffQuantizer

model = MyModel()
optim = torch.optim.Adam(model.parameters())
quantizer = DiffQuantizer(model)
quantizer.setup_optimizer(optim)

for batch in loader:
    loss = criterion(model(x), y) + 1e-3 * quantizer.model_size()
    optim.zero_grad()
    loss.backward()
    optim.step()

torch.save(quantizer.get_quantized_state(), "model.th")

Package facts

Evidence: diffq-0.2.4-cp310-cp310-macosx_10_9_universal2.whl; diffq-0.2.4-cp310-cp310-macosx_10_9_x86_64.whl; diffq-0.2.4-cp310-cp310-macosx_11_0_arm64.whl; diffq-0.2.4-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl; diffq-0.2.4-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl; diffq-0.2.4-cp310-cp310-win32.whl; diffq-0.2.4-cp310-cp310-win_amd64.whl; diffq-0.2.4-cp37-cp37m-macosx_10_9_x86_64.whl; diffq-0.2.4-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl; diffq-0.2.4-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl; diffq-0.2.4-cp37-cp37m-win32.whl; diffq-0.2.4-cp37-cp37m-win_amd64.whl; diffq-0.2.4-cp38-cp38-macosx_10_9_universal2.whl; diffq-0.2.4-cp38-cp38-macosx_10_9_x86_64.whl; diffq-0.2.4-cp38-cp38-macosx_11_0_arm64.whl; diffq-0.2.4-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl; diffq-0.2.4-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl; diffq-0.2.4-cp38-cp38-win32.whl; diffq-0.2.4-cp38-cp38-win_amd64.whl; diffq-0.2.4-cp39-cp39-macosx_10_9_universal2.whl

Tags

More Artificial Intelligence packages

See also optimum-quanto · ai-edge-quantizer · vector-quantize-pytorch · qonnx · aqtp · nvidia-modelopt · model-compression-toolkit · torchao · auto-gptq · qwix