newspaper4k
Simplified python article discovery & extraction.
Decision gist · record as of 2026-08-14
Yes. The package is actively maintained, has low install friction, carries no known vulnerabilities, and uses a permissive MIT license. It is well-suited for article scraping and curation tasks where you need structured extraction from HTML. The main gotcha is that bulk scraping can be slow and may trigger IP blocks on news sites; start with small batches and respect robots.txt.AI-flagged interpretation of the facts on this page — verify before relying
Before you install
- Requires Python 3.10 or later.
- Article extraction quality depends on correct language detection; if the wrong language is specified, text extraction may fail.
- Low friction installation with 11 runtime dependencies.
License · maintenance · safety
MIT (permissive) — MIT license permits commercial and private use with minimal restrictions; suitable for most projects.
last release 2026-07-19 (26 days)
0 known vulnerabilities (OSV.dev, 2026-08-14) · 355,703 downloads/mo, #7,288 on PyPI
Alternatives
Verify before relying
pip install newspaper4k
from newspaper4k import article
obj = article('https://example.com/article')
print(obj.title)
print(obj.authors)
print(obj.text)- Whether multithreaded download performance scales well with large numbers of articles
- Robustness of language auto-detection across non-Latin scripts
- Rate-limiting or IP-blocking mitigation strategies when bulk-scraping news sites
What it is and what it does
Newspaper4k is a continuation of the unmaintained newspaper3k library, designed to extract structured article data from web pages and news websites. It downloads and parses HTML to recover article text, authors, publish dates, top images, and embedded media, then applies NLP techniques to extract keywords and generate summaries. The library supports multiple languages through automatic meta-tag detection or explicit configuration, and can build a Source object to discover and download all articles from a news website's front page and RSS feeds using multithreaded downloads.
The package maintains backward compatibility with newspaper3k's API, so existing code continues to work. It includes a command-line interface for one-off article extraction and special integration with Google News via an optional extra. Core dependencies are standard web-scraping and parsing libraries: beautifulsoup4, lxml, requests, pillow for image handling, feedparser for RSS, and python-dateutil for timestamp parsing.
Use it for
- Extract article text and metadata from a single URL for content aggregation or archival
- Build a news source crawler to discover and download all articles from a website's homepage and category pages
- Automatically summarize and extract keywords from articles for content curation dashboards
- Parse multilingual news sites by auto-detecting language from meta tags or specifying it explicitly
- Integrate Google News as a source to fetch recent articles by country and topic
Worth the install?
AI-flagged interpretation of the facts on this page. Verify before relying on it.
Yes.
The package is actively maintained, has low install friction, carries no known vulnerabilities, and uses a permissive MIT license. It is well-suited for article scraping and curation tasks where you need structured extraction from HTML. The main gotcha is that bulk scraping can be slow and may trigger IP blocks on news sites; start with small batches and respect robots.txt.
Install
newspaper4k on PyPI
Before you install
Low friction installation with 11 runtime dependencies. Actively maintained with recent release 26 days ago; requires Python 3.10 or later.
Requires Python 3.10 or later. Article extraction quality depends on correct language detection; if the wrong language is specified, text extraction may fail.
License in practice
MIT license permits commercial and private use with minimal restrictions; suitable for most projects.
Quickstart
pip install newspaper4k
from newspaper4k import article
obj = article('https://example.com/article')
print(obj.title)
print(obj.authors)
print(obj.text)Verify before relying
- Whether multithreaded download performance scales well with large numbers of articles
- Robustness of language auto-detection across non-Latin scripts
- Rate-limiting or IP-blocking mitigation strategies when bulk-scraping news sites
Package facts
| License | MIT permissive |
| Python support | Supports the current Python release >=3.10 |
| Install friction | Low. Pure-Python wheel |
| Runtime dependencies | 11 packagesbeautifulsoup4brotlifeedparserlxmlpillowpython-dateutilpyyamlrequeststldextracttyping-extensionsw3lib |
| Maintenance | Actively maintained 26 days since the last release |
| First released | |
| Downloads | 355,703 / month, #7,288 on PyPI 30-day window, as of 2026-08-14 |
| Known vulnerabilities | None known OSV.dev, checked 2026-08-14 |
| Classifiers | Development Status :: 4 - BetaIntended Audience :: DevelopersIntended Audience :: EducationIntended Audience :: Information TechnologyIntended Audience :: Science/ResearchLicense :: OSI Approved :: MIT LicenseNatural Language :: EnglishProgramming Language :: Python :: 3Topic :: Software Development :: Libraries :: Python ModulesTopic :: Text ProcessingTopic :: Text Processing :: Markup :: HTML |
Evidence: newspaper4k-0.9.6-py3-none-any.whl
Tags
Let your AI agent find packages like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 14,416 PyPI packages by what they can do, searchable in plain language.
wish › “web scraper for articles”
- newspaper4kExtracts article text, metadata, and images from web pages and news…
- goose3Extracts article text, metadata, images, and embedded videos from web…
- gnewsGNews searches Google News RSS feeds and returns article metadata as…
Give your agent the search over MCP, or paste the wish link into any chat.
More Python Modules packages
Converts domain names between Unicode and ASCII-compatible encoding (Punycode) according to IDNA 2008 and Unicode Technical Standard 46, with security validation and broader script coverage than the standard library.
Install it if you work with internationalized domain names, need to validate domains, or use HTTP clients that depend on it transitively.
Setuptools is a Python build backend and package management tool that handles building, distributing, and installing Python packages, including support for C/C++ extension modules.
PyYAML parses and emits YAML 1.1 data format, enabling serialization and deserialization of configuration files and Python objects to and from human-readable YAML text.
Pydantic validates Python data structures against type hints, coercing and checking input at runtime to ensure it matches a declared schema.
Provides reusable metadata objects for use with PEP-593 `typing.Annotated` to express common constraints like bounds, collection sizes, and predicates on types.
Install it if you use or build libraries that need to express type constraints in a standardized, inspectable way—or if you want to annotate your own types with…
Provides runtime tools to inspect and introspect Python type annotations, enabling programmatic examination of type hints at execution time.
See also gnews · goose3 · newspaper3k · readable-content · boilerpy3 · date-guesser · trafilatura · Scrapy · breadability · Crawl4AI