Resiliparse

What it is and what it does

Resiliparse is a collection of parsing and analysis tools for web archive data, built with Rust and C++ for performance and wrapped with Python bindings. It is part of the ChatNoir web analytics toolkit and depends on fastwarc for WARC file handling. The package is designed to handle large-scale web archive processing efficiently, with pre-built binaries available for supported platforms.

The library targets developers and researchers working with web crawl data, Common Crawl archives, and similar large-scale web datasets. Installation is straightforward via pip for supported platforms; building from source requires vcpkg and C++ tooling but is documented. The project is actively maintained and carries no known security vulnerabilities.

Use it for:

• Parse and extract content from WARC (Web ARChive) files at scale for research or analytics.
• Process Common Crawl data or other large web archives for information extraction pipelines.
• Build web analytics and data mining workflows that require efficient HTML and metadata parsing.
• Integrate web archive processing into data science or NLP projects that need robust, fast parsing.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Resiliparse provides fast, robust parsing and analysis tools for web archive data, written in Rust and C++ with Python bindings, and includes support for processing web content through its fastwarc dependency.

Yes, if you work with web archives or WARC files. Resiliparse is actively maintained, carries no known vulnerabilities, and offers pre-built wheels for supported platforms. Medium install friction is acceptable for the performance and robustness it provides. The permissive Apache-2.0 license poses no restrictions. Not relevant if you don't need web archive processing.

Install

resiliparse on PyPI

pip install resiliparse

uv add resiliparse

poetry add resiliparse

Installing Resiliparse

Before you install

Medium install friction due to compiled Rust and C++ components; pre-built wheels available for modern Python versions on macOS, Linux, and Windows. Active maintenance with recent releases; last commit 2026-07-20.

License in practice

Apache-2.0 permissive license allows commercial use, modification, and distribution with minimal restrictions; suitable for most projects.

Quickstart

pip install resiliparse

import resiliparse
# See https://resiliparse.chatnoir.eu for detailed usage examples

Requires Python >=3.8; building from source requires vcpkg and C++ build tools.

Verify before relying

• Specific parsing capabilities and API surface beyond 'web archive data' are not detailed in the fact sheet.
• Performance characteristics compared to alternative web parsing libraries are not documented.
• Integration patterns with fastwarc and typical workflows are not shown in the excerpt.
• Which specific Python versions have pre-built wheels available beyond the friction evidence shown.

Package facts

Evidence: resiliparse-1.0.9-cp310-cp310-macosx_12_0_arm64.whl; resiliparse-1.0.9-cp310-cp310-macosx_12_0_x86_64.whl; resiliparse-1.0.9-cp310-cp310-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl; resiliparse-1.0.9-cp310-cp310-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl; resiliparse-1.0.9-cp310-cp310-win_amd64.whl; resiliparse-1.0.9-cp311-cp311-macosx_12_0_arm64.whl; resiliparse-1.0.9-cp311-cp311-macosx_12_0_x86_64.whl; resiliparse-1.0.9-cp311-cp311-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl; resiliparse-1.0.9-cp311-cp311-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl; resiliparse-1.0.9-cp311-cp311-win_amd64.whl; resiliparse-1.0.9-cp312-cp312-macosx_12_0_arm64.whl; resiliparse-1.0.9-cp312-cp312-macosx_12_0_x86_64.whl; resiliparse-1.0.9-cp312-cp312-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl; resiliparse-1.0.9-cp312-cp312-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl; resiliparse-1.0.9-cp312-cp312-win_amd64.whl; resiliparse-1.0.9-cp313-cp313-macosx_12_0_arm64.whl; resiliparse-1.0.9-cp313-cp313-macosx_12_0_x86_64.whl; resiliparse-1.0.9-cp313-cp313-manylinux_2_24_aarch64.manylinux_2_28_aarch64.whl; resiliparse-1.0.9-cp313-cp313-manylinux_2_24_x86_64.manylinux_2_28_x86_64.whl; resiliparse-1.0.9-cp313-cp313-win_amd64.whl

Tags

More WWW/HTTP packages