diffq

What it is and what it does

DiffQ is a PyTorch quantization framework that reduces model size by automatically determining optimal bit-widths for individual weights or weight groups during training. It uses pseudo quantization noise injection to make the quantization process differentiable, allowing the bit allocation itself to be optimized as part of the training loop alongside model weights.

The package integrates with standard PyTorch training pipelines: you attach a DiffQuantizer to your model before creating the optimizer, then add a model-size penalty term to your loss function. During training, the quantizer learns which weights need more bits and which can use fewer, trading off compression against accuracy. At inference, it automatically switches to true quantized weights, and you can export the compressed model to disk or TorchScript format.

Use it for:

• Compress large transformer or CNN models for deployment on memory-constrained devices while maintaining accuracy
• Automatically determine per-layer or per-group bit allocations during training without manual hyperparameter tuning
• Export quantized models to TorchScript for optimized inference with reduced memory footprint
• Research differentiable quantization methods and compare compression-accuracy tradeoffs across architectures
• Reduce model size for distributed training or inference in bandwidth-limited environments

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

DiffQ performs differentiable quantization of PyTorch models using pseudo quantization noise, automatically tuning the number of bits per weight to balance model size and accuracy.

No. The package is abandoned (last commit 2023-05-05) and licensed under CC-BY-NC 4.0, restricting use to non-commercial purposes only. Unless you are conducting non-commercial research and can tolerate no future updates, use an actively maintained quantization library instead.

Install

diffq on PyPI

pip install diffq

uv add diffq

poetry add diffq

Installing diffq

Before you install

Medium install friction due to compiled dependencies (Cython, numpy, torch). The package is abandoned as of 2023-05-05 with no recent maintenance, so expect no bug fixes or compatibility updates.

License in practice

Licensed under CC-BY-NC 4.0 (noncommercial), which restricts use to non-commercial purposes only. Commercial deployment or integration into proprietary products is not permitted without explicit permission.

Quickstart

pip install diffq

import torch
from diffq import DiffQuantizer

model = MyModel()
optim = torch.optim.Adam(model.parameters())
quantizer = DiffQuantizer(model)
quantizer.setup_optimizer(optim)

for batch in loader:
    loss = criterion(model(x), y) + 1e-3 * quantizer.model_size()
    optim.zero_grad()
    loss.backward()
    optim.step()

torch.save(quantizer.get_quantized_state(), "model.th")

Requires Python 3.7 or later and PyTorch 1.7.1 ideally. Compiled dependencies (Cython, numpy, torch) must be installed; prebuilt wheels are available for common platforms but may not exist for all architectures.

Verify before relying

• Whether the package remains compatible with PyTorch versions released after 2023-05-05
• Whether TorchScript export (noted as experimental) is production-ready
• Whether int8 in-memory support mentioned as 'coming up' was ever implemented

Package facts

Evidence: diffq-0.2.4-cp310-cp310-macosx_10_9_universal2.whl; diffq-0.2.4-cp310-cp310-macosx_10_9_x86_64.whl; diffq-0.2.4-cp310-cp310-macosx_11_0_arm64.whl; diffq-0.2.4-cp310-cp310-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl; diffq-0.2.4-cp310-cp310-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl; diffq-0.2.4-cp310-cp310-win32.whl; diffq-0.2.4-cp310-cp310-win_amd64.whl; diffq-0.2.4-cp37-cp37m-macosx_10_9_x86_64.whl; diffq-0.2.4-cp37-cp37m-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl; diffq-0.2.4-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl; diffq-0.2.4-cp37-cp37m-win32.whl; diffq-0.2.4-cp37-cp37m-win_amd64.whl; diffq-0.2.4-cp38-cp38-macosx_10_9_universal2.whl; diffq-0.2.4-cp38-cp38-macosx_10_9_x86_64.whl; diffq-0.2.4-cp38-cp38-macosx_11_0_arm64.whl; diffq-0.2.4-cp38-cp38-manylinux_2_5_i686.manylinux1_i686.manylinux_2_12_i686.manylinux2010_i686.whl; diffq-0.2.4-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl; diffq-0.2.4-cp38-cp38-win32.whl; diffq-0.2.4-cp38-cp38-win_amd64.whl; diffq-0.2.4-cp39-cp39-macosx_10_9_universal2.whl

Tags

More Artificial Intelligence packages