scrapling

What it is and what it does

Scrapling is a web scraping framework designed to handle both simple single-page requests and large-scale concurrent crawls. It combines HTTP fetchers (with stealth and anti-bot capabilities), a Scrapy-like spider API for structured crawling, and adaptive parsing that learns from website changes and automatically relocates elements when page layouts shift. The framework includes built-in support for proxy rotation, session management, pause/resume checkpointing, and export to JSON, CSV, and XML.

The package targets modern web scraping challenges: it claims to bypass Cloudflare Turnstile out of the box, supports headless browser automation via Playwright, and provides both synchronous and asynchronous APIs. Its spider framework handles concurrent requests with per-domain throttling, robots.txt compliance, and automatic retry logic for blocked requests. Runtime dependencies are minimal (lxml, cssselect, orjson, tld, w3lib, typing_extensions), keeping the base installation lightweight.

Use it for:

• Scrape product listings from e-commerce sites that use Cloudflare protection or dynamic content loading.
• Build a full-scale web crawler with concurrent requests, automatic proxy rotation, and pause/resume capability.
• Extract data from Shopify stores using the built-in ShopifySpider template to iterate products via JSON API.
• Iterate over XML feeds, RSS, or CSV sources with ready-made feed spider templates.
• Monitor website changes by using adaptive parsing to relocate elements after design updates without rewriting selectors.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Scrapling is a web scraping and crawling framework that handles single requests to full-scale crawls, with built-in anti-bot bypass, adaptive element relocation, proxy rotation, and concurrent spider support.

Yes. Scrapling is actively maintained, has low install friction, carries a permissive BSD license, and addresses real modern web scraping pain points (anti-bot bypass, adaptive parsing, concurrent crawling). The recent release, high star count, and zero known vulnerabilities indicate a mature, well-tested library. Install it if you need to scrape protected or dynamic websites at scale; skip it if you only need simple static HTML parsing.

Install

scrapling on PyPI

pip install scrapling

uv add scrapling

poetry add scrapling

Installing scrapling

Before you install

Low install friction with a pure-Python wheel and six lightweight runtime dependencies. The package is actively maintained with a recent release (4 days old) and substantial community engagement (73959 stars), suggesting stable, well-tested code.

License in practice

BSD 3-Clause License permits commercial and private use with minimal restrictions—you may use, modify, and distribute the code provided you retain the license notice and disclaimer.

Quickstart

pip install scrapling

from scrapling.fetchers import StealthyFetcher
p = StealthyFetcher.fetch('https://example.com', headless=True)
products = p.css('.product', adaptive=True)

Requires Python 3.10 or later. Browser-based fetchers (StealthyFetcher, DynamicFetcher) require a local Chromium/Chrome installation or a remote CDP endpoint.

Verify before relying

• Whether adaptive element relocation works reliably across different website redesigns in production.
• Performance characteristics and memory usage when running concurrent crawls at scale.
• Specific anti-bot systems reliably bypassed beyond Cloudflare Turnstile (e.g., Akamai, DataDome).

Package facts

Evidence: scrapling-0.4.14-py3-none-any.whl

Keywords: web-scraping, scraping, automation, browser-automation, data-extraction, html-parsing, undetectable, playwright, selenium-alternative, web-crawler, browser, crawling, headless, scraper, chrome

Tags

More Libraries packages