{"categories":[{"label":"WWW/HTTP","url":"https://skillfed.io/packages/category/internet-www-http/6"}],"enrichment":{"capability":"Reads and iterates over WARC and WET archive files, extracting records with their metadata and content for web crawl analysis.","skillfed_tags":["web-archive","abandoned"],"use_cases":["Parse ClueWeb09 web crawl archives to extract URLs and content metadata for research or analysis.","Iterate over WARC files to build indexes or extract specific records for information retrieval tasks.","Read WET files to access extracted text content from archived web pages.","Migrate legacy Python 2 WARC processing pipelines to Python 3 using this ported library."],"what_it_does":"warc3-wet-clueweb09 is a Python library for reading WARC (Web ARChive) and WET (WARC Extracted Text) files, which are standard formats for storing web crawls. It provides an interface to iterate over archive records and access their metadata fields and content. The package is a Python 3 port of an older library, with modifications to handle parsing issues specific to ClueWeb09 archives.\n\nThe library has no runtime dependencies and is designed for straightforward use: open an archive file, iterate over records, and extract fields like WARC-Target-URI and Content-Length. However, the package is abandoned (last updated 2020-12-07 with no commits since 2021-12-06), so it receives no maintenance, bug fixes, or updates for compatibility with newer Python versions.","worth_installing":"No, unless you have a specific, narrow need to read ClueWeb09 archives and cannot use an actively maintained alternative. The package is abandoned, Python version support is unspecified, and GPLv2 copyleft licensing restricts use in proprietary projects. High install friction and no maintenance path make it a poor choice for new projects."},"id":"warc3-wet-clueweb09","links":{"html":"https://skillfed.io/packages/warc3-wet-clueweb09","md":"https://skillfed.io/packages/warc3-wet-clueweb09.md","pypi":"https://pypi.org/project/warc3-wet-clueweb09/"},"maintenance":{"status":"abandoned"},"meta":{"latest_release":"2020-12-07","license_spdx":null,"license_treatment":"copyleft","name":"warc3-wet-clueweb09","python_support":"unspecified","summary":"Python library to work with ARC and WARC files, with fixes for ClueWeb09"},"popularity":{"monthly_downloads":142309,"position":11213,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"0.2.5"}