{"categories":[{"label":"Artificial Intelligence","url":"https://skillfed.io/packages/category/scientific-engineering-artificial-intelligence/6"}],"enrichment":{"capability":"AutoRound quantizes large language models and vision-language models to 2\u20134 bits with minimal accuracy loss, using sign-gradient descent and supporting multiple export formats and inference backends.","skillfed_tags":["model-compression","quantization","llm-inference"],"use_cases":["Compress a 7B LLM to 2\u20133 bits for edge deployment or cost-effective cloud inference","Export a quantized model to GGUF format for use with llama.cpp or other C++ inference engines","Quantize a vision-language model for efficient multimodal inference on resource-constrained hardware","Generate a mixed-precision quantization scheme automatically to balance accuracy and model size","Integrate quantized models into vLLM or SGLang for fast batch inference with reduced memory footprint"],"what_it_does":"AutoRound is a quantization toolkit that compresses large language models and vision-language models to ultra-low bit widths (2\u20134 bits) while preserving accuracy. It uses sign-gradient descent to find optimal quantization parameters with minimal tuning overhead. The package integrates with popular inference frameworks (vLLM, SGLang, Transformers) and supports multiple export formats (AutoRound, AutoAWQ, AutoGPTQ, GGUF), making quantized models portable across different deployment environments.\n\nThe toolkit offers several quantization recipes\u2014from fast round-to-nearest (RTN) baseline to more accurate iterative methods\u2014and includes utilities for multi-GPU quantization, mixed-precision schemes, and multiple calibration datasets. It targets both researchers optimizing model accuracy at low bits and practitioners seeking to reduce model size and inference latency for deployment.","worth_installing":"Yes. AutoRound is actively maintained, has no known vulnerabilities, installs with low friction, and is permissively licensed. It is well-suited for anyone needing to compress LLMs or VLMs for inference\u2014whether for research, edge deployment, or cost reduction. Start with the CLI recipes (auto-round-best, auto-round-rtn) if you want quick results, or use the Python API for fine-grained control."},"id":"auto-round","links":{"html":"https://skillfed.io/packages/auto-round","md":"https://skillfed.io/packages/auto-round.md","pypi":"https://pypi.org/project/auto-round/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2026-07-13","license_spdx":null,"license_treatment":"permissive","name":"auto-round","python_support":"supports_current","summary":"Repository of AutoRound: Advanced Weight-Only Quantization Algorithm for LLMs"},"popularity":{"monthly_downloads":300133,"position":7847,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"0.14.2"}