{"categories":[{"label":"WWW/HTTP","url":"https://skillfed.io/packages/category/internet-www-http/5"}],"enrichment":{"capability":"Read and iterate over WARC (Web ARChive) and WET (WARC Extracted Text) files, extracting record metadata and content from web crawl archives.","skillfed_tags":["web-archiving","data-extraction"],"use_cases":["Extract and process records from web crawl archives stored in WARC format for data analysis or research.","Parse WET files to access extracted text content from archived web pages.","Build data pipelines that read WARC archives and filter or transform records based on metadata.","Access specific records within large WARC files using seek functionality for efficient random access.","Integrate web archive data into workflows that require crawled web content."],"what_it_does":"warc3-wet is a Python library for reading and parsing WARC and WET archive files\u2014the standard formats used to store web crawls. It provides a simple interface to open these files and iterate over records, extracting metadata headers like WARC-Target-URI and Content-Length. The library is a Python 3 port of an older warc package; this fork adds support for WET files and seeking within archives.\n\nThe package has no external runtime dependencies, making it lightweight to install. It is classified as Beta-status software. The repository remains accessible but has not seen a release in over two years, suggesting the package is in a stable, maintenance-only state.","worth_installing":"Yes, if you need to work with WARC or WET files. The package is stable, has no dependencies, and handles a specific file format well. The dormant maintenance status is not a blocker for read-only use, but verify GPLv2 compatibility with your project first. No known security vulnerabilities."},"id":"warc3-wet","links":{"html":"https://skillfed.io/packages/warc3-wet","md":"https://skillfed.io/packages/warc3-wet.md","pypi":"https://pypi.org/project/warc3-wet/"},"maintenance":{"status":"dormant"},"meta":{"latest_release":"2024-07-17","license_spdx":null,"license_treatment":"copyleft","name":"warc3-wet","python_support":"unspecified","summary":"Python library to work with ARC and WARC files"},"popularity":{"monthly_downloads":219855,"position":9319,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"0.2.5"}