boilerpy3

Python port of Boilerpipe, for HTML boilerplate removal and text extraction

boilerpy3 v1.0.7 207.8K downloads/30d#9,541 on PyPI★ 96

Permissive license Apache 2.0 DORMANT released 2023-11-01

What it is and what it does

BoilerPy3 is a Python port of the Boilerpipe library, a text extraction tool that identifies and extracts the main content from HTML pages while discarding boilerplate elements like navigation, advertisements, and sidebars. It provides multiple extractors tuned for different scenarios—ArticleExtractor for news articles, DefaultExtractor for generic pages, and others for specialized use cases. The library works by analyzing HTML structure and text density to classify blocks as content or boilerplate.

The package has no runtime dependencies and supports Python 3.6 and later. It offers three main interfaces: extracting plain text, extracting marked HTML chunks, or retrieving a document object with additional metadata like title. The documentation notes that while URL-fetching methods are provided, using an external HTTP library like Requests is recommended for production use.

Use it for:

Extract article text from news websites or blog posts for content aggregation or archival systems.
Clean HTML before feeding it to NLP pipelines or text analysis tools that need main content only.
Build web scrapers that need to isolate article body from page chrome and advertisements.
Preprocess HTML for machine learning training data where boilerplate noise would degrade model quality.
Implement fallback content extraction when more specialized algorithms fail (via raise_on_failure parameter).

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Extracts main article text and content from HTML, filtering out boilerplate like navigation, ads, and sidebars using heuristics tuned for news articles and similar documents.

Yes, if you need straightforward HTML boilerplate removal and your use case tolerates a dormant library. The package is stable, dependency-free, and widely used, making it a low-risk choice for text extraction. However, do not install if you require active maintenance, support for modern web frameworks, or performance guarantees on contemporary HTML patterns—consider alternatives if those are critical.

Install

boilerpy3 on PyPI

pip

pip install boilerpy3

uv

uv add boilerpy3

poetry

poetry add boilerpy3

Installing boilerpy3

Before you install

Low install friction with no runtime dependencies. Dormant maintenance status (last commit 2024-08-20, no release since 2023-11-01) means bug fixes and updates are unlikely, though the package remains functional for its core use case.

License in practice

Apache 2.0 permissive license allows free use, modification, and distribution with minimal restrictions—suitable for most commercial and open-source projects.

Quickstart

pip install boilerpy3

from boilerpy3 import extractors

extractor = extractors.ArticleExtractor()
content = extractor.get_content('<html>...</html>')

Verify before relying

How well the Boilerpipe 1.2-equivalent algorithm performs on modern web layouts compared to more recent extraction libraries.
Whether the package handles edge cases in malformed or unusual HTML gracefully when raise_on_failure is False.
Current download volume and active user base given the dormant maintenance status.

Package facts

License	Apache 2.0 (permissive)
Python support	supports the current Python release (>=3.6)
Install friction	low — pure-Python wheel
Runtime dependencies	none
Maintenance	dormant — 1,017 days since the last release
Last repo commit	2024-08-20
First released	2019-08-09
Downloads	207,848/month — #9,541 on PyPI (30-day window, as of 2026-08-14)
Known vulnerabilities	none known (OSV.dev, checked 2026-08-14)

Evidence: boilerpy3-1.0.7-py3-none-any.whl

Keywords: boilerpipe, boilerpy, html text extraction, text extraction, full text extraction

Development Status :: 4 - BetaLicense :: OSI Approved :: Apache Software LicenseProgramming Language :: Python :: 3Programming Language :: Python :: 3.10Programming Language :: Python :: 3.6Programming Language :: Python :: 3.7Programming Language :: Python :: 3.8Programming Language :: Python :: 3.9Topic :: Utilities