{"categories":[{"label":"Software Development","url":"https://skillfed.io/packages/category/software-development/13"},{"label":"Scientific/Engineering","url":"https://skillfed.io/packages/category/scientific-engineering/5"},{"label":"Python Modules","url":"https://skillfed.io/packages/category/software-development-libraries-python-modules/15"},{"label":"Artificial Intelligence","url":"https://skillfed.io/packages/category/scientific-engineering-artificial-intelligence/7"},{"label":"Mathematics","url":"https://skillfed.io/packages/category/scientific-engineering-mathematics/2"}],"enrichment":{"capability":"llmcompressor optimizes large language models for efficient deployment by applying quantization, pruning, and other compression techniques compatible with vLLM inference.","skillfed_tags":["model-optimization","quantization","inference-acceleration"],"use_cases":["Reduce model size and memory footprint for single-GPU deployment of large models like Llama or Qwen variants.","Apply post-training quantization (PTQ) to existing checkpoints without retraining, using calibration data.","Compress Mixture-of-Experts models by pruning less-relevant experts while maintaining accuracy.","Prepare quantized models for vLLM inference with guaranteed format compatibility.","Experiment with different quantization schemes (W4A8, W8A16, NVFP4, etc.) on custom models."],"what_it_does":"llmcompressor is a PyTorch-based library for compressing large language models through quantization, pruning, and related techniques. It integrates with Hugging Face models and outputs compressed checkpoints in the compressed-tensors format, which vLLM can load directly for inference. The library supports multiple quantization precisions (int8, fp8, NVFP4, MXFP4, etc.) and algorithms (GPTQ, AWQ, SmoothQuant, AutoRound, REAP), with built-in support for weight-only, weight-activation, KV cache, and attention quantization.\n\nTypical usage involves loading a Hugging Face model, applying a compression recipe (via YAML configuration or Python API), and saving the result for deployment. The library handles distributed training (DDP) and disk offloading to compress very large models on limited hardware. It depends on torch, transformers, datasets, accelerate, and several specialized packages like auto-round and compressed-tensors.","worth_installing":"Yes. Active maintenance, no known vulnerabilities, permissive license, and low install friction make it a solid choice for anyone deploying LLMs with vLLM. The substantial dependency footprint (torch, transformers, etc.) is expected for this use case. Start with the step-by-step compression guide in the documentation to select an appropriate quantization scheme for your model and hardware."},"id":"llmcompressor","links":{"html":"https://skillfed.io/packages/llmcompressor","md":"https://skillfed.io/packages/llmcompressor.md","pypi":"https://pypi.org/project/llmcompressor/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2026-08-11","license_spdx":null,"license_treatment":"permissive","name":"llmcompressor","python_support":"supports_current","summary":"A library for compressing large language models utilizing the latest techniques and research in the field for both training aware and post training techniques. The library is designed to be flexible and easy to use on top of PyTorch and HuggingFace Transformers, allowing for quick experimentation."},"popularity":{"monthly_downloads":221408,"position":9280,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"0.13.0"}