{"categories":[{"label":"Utilities","url":"https://skillfed.io/packages/category/utilities/8"}],"enrichment":{"capability":"Extracts main article text and content from HTML, filtering out boilerplate like navigation, ads, and sidebars using heuristics tuned for news articles and similar documents.","skillfed_tags":["content-extraction","html-parsing","web-scraping"],"use_cases":["Extract article text from news websites or blog posts for content aggregation or archival systems.","Clean HTML before feeding it to NLP pipelines or text analysis tools that need main content only.","Build web scrapers that need to isolate article body from page chrome and advertisements.","Preprocess HTML for machine learning training data where boilerplate noise would degrade model quality.","Implement fallback content extraction when more specialized algorithms fail (via raise_on_failure parameter)."],"what_it_does":"BoilerPy3 is a Python port of the Boilerpipe library, a text extraction tool that identifies and extracts the main content from HTML pages while discarding boilerplate elements like navigation, advertisements, and sidebars. It provides multiple extractors tuned for different scenarios\u2014ArticleExtractor for news articles, DefaultExtractor for generic pages, and others for specialized use cases. The library works by analyzing HTML structure and text density to classify blocks as content or boilerplate.\n\nThe package has no runtime dependencies and supports Python 3.6 and later. It offers three main interfaces: extracting plain text, extracting marked HTML chunks, or retrieving a document object with additional metadata like title. The documentation notes that while URL-fetching methods are provided, using an external HTTP library like Requests is recommended for production use.","worth_installing":"Yes, if you need straightforward HTML boilerplate removal and your use case tolerates a dormant library. The package is stable, dependency-free, and widely used, making it a low-risk choice for text extraction. However, do not install if you require active maintenance, support for modern web frameworks, or performance guarantees on contemporary HTML patterns\u2014consider alternatives if those are critical."},"id":"boilerpy3","links":{"html":"https://skillfed.io/packages/boilerpy3","md":"https://skillfed.io/packages/boilerpy3.md","pypi":"https://pypi.org/project/boilerpy3/"},"maintenance":{"status":"dormant"},"meta":{"latest_release":"2023-11-01","license_spdx":null,"license_treatment":"permissive","name":"boilerpy3","python_support":"supports_current","summary":"Python port of Boilerpipe, for HTML boilerplate removal and text extraction"},"popularity":{"monthly_downloads":207848,"position":9541,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"1.0.7"}