{"categories":[{"label":"Artificial Intelligence","url":"https://skillfed.io/packages/category/scientific-engineering-artificial-intelligence/2"}],"enrichment":{"capability":"TorchAO applies quantization and sparsity techniques to PyTorch models for faster training and inference with reduced memory usage, working natively with torch.compile() and FSDP2.","skillfed_tags":["model-optimization","quantization","llm-inference"],"use_cases":["Reduce inference latency and memory footprint of large language models by quantizing weights to int4 or float8 for deployment on resource-constrained hardware.","Accelerate multi-GPU pre-training of large models by applying float8 training with FSDP2 to reduce communication overhead and memory pressure.","Fine-tune quantized models using quantization-aware training (QAT) to recover accuracy degradation from aggressive quantization.","Apply semi-structured 2:4 sparsity patterns to transformer models to achieve training and inference speedups with minimal accuracy loss.","Integrate quantization into existing HuggingFace Transformers workflows via the TorchAoConfig API without rewriting model loading code."],"what_it_does":"TorchAO is a quantization and sparsity library that reduces model size and accelerates both training and inference by converting weights and activations to lower-precision formats (int4, float8, etc.) and applying structured sparsity patterns. It integrates directly into compilation and distributed training pipelines, allowing you to apply these optimizations to large language models and other neural networks with minimal code changes.\n\nThe library supports multiple quantization strategies\u2014weight-only quantization for inference, dynamic quantization for both training and serving, and quantization-aware training (QAT) to recover accuracy lost during quantization. It is designed to work out-of-the-box with HuggingFace Transformers models and has been validated on large-scale training runs, achieving reported speedups with acceptable accuracy trade-offs.","worth_installing":"Yes, if you are optimizing models for inference speed or training efficiency and can tolerate the license ambiguity. The package is actively maintained, has no external runtime dependencies, and is widely adopted in production serving. Verify the license terms in the repository before use in proprietary contexts; otherwise, install friction is minimal and performance gains are substantial."},"id":"torchao","links":{"html":"https://skillfed.io/packages/torchao","md":"https://skillfed.io/packages/torchao.md","pypi":"https://pypi.org/project/torchao/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2026-08-03","license_spdx":null,"license_treatment":"unclear","name":"torchao","python_support":"unspecified","summary":"Package for applying ao techniques to GPU models"},"popularity":{"monthly_downloads":3707418,"position":2522,"tier":"top_5000"},"security":{"n_vulnerabilities":0},"version":"0.18.0"}