compressed-tensors

What it is and what it does

compressed-tensors extends the safetensors format to create a single, standardized way to store compressed neural network models. Rather than each quantization technique (GPTQ, AWQ, SmoothQuant, INT8, FP8, etc.) having its own checkpoint format, this library provides a unified representation that can handle weight-only quantization, activation quantization, KV cache quantization, and both unstructured and semi-structured sparsity patterns.

You use it to save quantized models to disk and load them back, eliminating the friction of supporting multiple compression formats. The library integrates with Hugging Face models and PyTorch, so you can apply post-training quantization, calibrate on data, compress weights, and save the result in a single consistent format that downstream inference engines can understand.

Use it for:

• Save a quantized LLM checkpoint after post-training quantization (PTQ) in a format that multiple inference engines can load
• Experiment with different quantization schemes (W4A16, W8A8, etc.) on the same model without managing separate storage formats
• Build a model deployment pipeline that accepts compressed-tensors checkpoints and applies them uniformly across different quantization methods
• Store both weight and activation quantization metadata alongside compressed weights for reproducible model optimization
• Handle semi-structured sparsity patterns (e.g., 2:4 sparsity) alongside quantization in a single unified checkpoint

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Provides a unified format for storing and loading compressed neural network tensors, supporting multiple quantization and sparsity schemes like GPTQ, AWQ, SmoothQuant, INT8, and FP8.

Yes. The package is actively maintained, has no known vulnerabilities, installs with low friction, and solves a real problem in the LLM deployment pipeline—unifying the fragmented landscape of quantization formats. If you work with quantized models or need to support multiple compression schemes, it's a practical choice.

Install

compressed-tensors on PyPI

pip install compressed-tensors

uv add compressed-tensors

poetry add compressed-tensors

Installing compressed-tensors

Before you install

Low friction installation with a pure Python wheel. Active maintenance with a recent release and steady commit activity. Depends on torch, transformers, pydantic, loguru, and psutil—all standard packages in the ML ecosystem.

License in practice

Licensed under Apache 2.0 (permissive), allowing commercial use, modification, and distribution with minimal restrictions.

Quickstart

pip install compressed-tensors

from compressed_tensors.quantization import QuantizationConfig, apply_quantization_config
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("model_name", device_map="cuda:0")
config = QuantizationConfig.parse_file("config.json")
apply_quantization_config(model, config)

Requires torch and transformers to be installed; GPU recommended for typical use cases.

Verify before relying

• Whether the package requires specific versions of torch or transformers, or works across a range of versions
• Whether there are system-level dependencies needed beyond the listed Python packages

Package facts

Evidence: compressed_tensors-0.18.0-py3-none-any.whl

Tags

More Artificial Intelligence packages

Further reading