---
id: compressed-tensors
version: "0.18.0"
license: Apache 2.0
license_treatment: permissive
maintenance: active
---

# compressed-tensors — Library for utilization of compressed safetensors of neural network models

License: permissive · Maintenance: active · Downloads: 7.3M/mo

## What it is and what it does

compressed-tensors extends the safetensors format to create a single, standardized way to store compressed neural network models. Rather than each quantization technique (GPTQ, AWQ, SmoothQuant, INT8, FP8, etc.) having its own checkpoint format, this library provides a unified representation that can handle weight-only quantization, activation quantization, KV cache quantization, and both unstructured and semi-structured sparsity patterns.

You use it to save quantized models to disk and load them back, eliminating the friction of supporting multiple compression formats. The library integrates with Hugging Face models and PyTorch, so you can apply post-training quantization, calibrate on data, compress weights, and save the result in a single consistent format that downstream inference engines can understand.

Use it for:

- Save a quantized LLM checkpoint after post-training quantization (PTQ) in a format that multiple inference engines can load
- Experiment with different quantization schemes (W4A16, W8A8, etc.) on the same model without managing separate storage formats
- Build a model deployment pipeline that accepts compressed-tensors checkpoints and applies them uniformly across different quantization methods
- Store both weight and activation quantization metadata alongside compressed weights for reproducible model optimization
- Handle semi-structured sparsity patterns (e.g., 2:4 sparsity) alongside quantization in a single unified checkpoint

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

Provides a unified format for storing and loading compressed neural network tensors, supporting multiple quantization and sparsity schemes like GPTQ, AWQ, SmoothQuant, INT8, and FP8.

Yes. The package is actively maintained, has no known vulnerabilities, installs with low friction, and solves a real problem in the LLM deployment pipeline—unifying the fragmented landscape of quantization formats. If you work with quantized models or need to support multiple compression schemes, it's a practical choice.

## Install

pip install compressed-tensors
uv add compressed-tensors
poetry add compressed-tensors

## Installing compressed-tensors

Before you install: Low friction installation with a pure Python wheel. Active maintenance with a recent release and steady commit activity. Depends on torch, transformers, pydantic, loguru, and psutil—all standard packages in the ML ecosystem.

License in practice: Licensed under Apache 2.0 (permissive), allowing commercial use, modification, and distribution with minimal restrictions.

Quickstart:

pip install compressed-tensors

from compressed_tensors.quantization import QuantizationConfig, apply_quantization_config
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained("model_name", device_map="cuda:0")
config = QuantizationConfig.parse_file("config.json")
apply_quantization_config(model, config)

Requires torch and transformers to be installed; GPU recommended for typical use cases.

Verify before relying:

- Whether the package requires specific versions of torch or transformers, or works across a range of versions
- Whether there are system-level dependencies needed beyond the listed Python packages

## Package facts

- License: Apache 2.0 (permissive)
- Python support: unspecified
- Install friction: low
- Maintenance: active
- Downloads: 7.3M/month (top 5,000 on PyPI)
- Known vulnerabilities: none known

## Tags

quantized model storage format, compressed tensor serialization, llm quantization checkpoint, model compression format, safetensors quantization, neural network weight compression, sparsity and quantization format, model-compression, quantization, llm-deployment

[View on SkillFed](https://skillfed.io/packages/compressed-tensors) · [View on PyPI](https://pypi.org/project/compressed-tensors/)