{"categories":[{"label":"WWW/HTTP","url":"https://skillfed.io/packages/category/internet-www-http/2"}],"enrichment":{"capability":"FastWARC parses WARC (Web ARChive) files\u2014compressed or uncompressed\u2014at high speed using Rust-backed bindings, supporting WARC/1.0 and WARC/1.1 with Gzip, Zstd, and LZ4 compression.","skillfed_tags":["web-archiving","rust-bindings","data-processing"],"use_cases":["Parse Common Crawl WARC dumps or other web archive collections for research or data extraction.","Build web crawl pipelines that need to read compressed WARC files without decompressing to disk first.","Extract HTTP responses, metadata, or payloads from archived web data at scale.","Integrate WARC parsing into information retrieval or web analytics workflows.","Process historical web snapshots stored in WARC format for archival or compliance tasks."],"what_it_does":"FastWARC is a Python library that reads WARC archive files\u2014the standard format for storing web crawl data\u2014using a Rust implementation for speed. It handles both compressed (Gzip, Zstd, LZ4) and uncompressed WARC/1.0 and WARC/1.1 streams. The library is part of the ChatNoir Resiliparse toolkit for web data processing and is designed for workloads that need to ingest and parse large volumes of web archive data efficiently.\n\nYou install it via pip and import it to iterate over records in WARC files. It depends on brotli, click, tqdm, and typing_extensions. The package requires Python 3.10 or later and is actively maintained with recent releases and no known security issues.","worth_installing":"Yes. FastWARC is actively maintained, has no known vulnerabilities, is licensed permissively, and offers pre-built wheels for modern Python versions on all major platforms. Install it if you work with WARC files or web archives; the Rust backend and broad compression support make it a solid choice for this specific task."},"id":"fastwarc","links":{"html":"https://skillfed.io/packages/fastwarc","md":"https://skillfed.io/packages/fastwarc.md","pypi":"https://pypi.org/project/fastwarc/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2026-07-20","license_spdx":"Apache-2.0","license_treatment":"permissive","name":"FastWARC","python_support":"supports_current","summary":"The world's fastest WARC parsing library written in Rust with bindings for Python."},"popularity":{"monthly_downloads":1402572,"position":3945,"tier":"top_5000"},"security":{"n_vulnerabilities":0},"version":"1.0.9"}