{"categories":[{"label":"WWW/HTTP","url":"https://skillfed.io/packages/category/internet-www-http/6"}],"enrichment":{"capability":"A Scrapy download handler that replaces HTTP/HTTPS requests with curl-cffi to impersonate browser TLS signatures and JA3 fingerprints, allowing scraping of sites that detect and block non-browser clients.","skillfed_tags":["web-scraping","anti-bot-bypass","tls-fingerprint"],"use_cases":["Scrape sites that reject requests with non-browser TLS signatures by impersonating Chrome, Firefox, or Safari.","Rotate browser fingerprints across requests to avoid detection patterns based on consistent client profiles.","Test web applications' anti-bot defenses by simulating requests from different browser versions and platforms.","Extract data from sites protected by basic JA3 fingerprint checks without rewriting scraper logic outside Scrapy.","Combine browser impersonation with Scrapy's middleware and pipeline ecosystem for complex scraping workflows."],"what_it_does":"scrapy-impersonate is a Scrapy download handler that integrates curl-cffi to perform HTTP requests with spoofed browser fingerprints. Instead of Scrapy's default HTTP client, it routes requests through curl-cffi, which can mimic the TLS signatures and JA3 fingerprints of real browsers\u2014Chrome, Firefox, Safari, and Edge across multiple versions and platforms. This allows web scrapers to bypass basic fingerprint-based bot detection that rejects requests from non-browser clients.\n\nThe package works by replacing Scrapy's standard download handlers and providing a middleware that can randomly rotate between supported browser profiles on a per-request basis. You configure it via Scrapy settings, then tag individual requests with an `impersonate` meta key to specify which browser to emulate. The asyncio-based Twisted reactor is required for proper async execution. It's designed for developers who need to scrape sites with fingerprint-based anti-bot protections but want to stay within the Scrapy framework rather than switching to a different HTTP client entirely.","worth_installing":"Yes, if you need Scrapy-based scraping against sites with fingerprint-based bot detection. The package is actively maintained, has low install friction, carries a permissive MIT license, and integrates cleanly into Scrapy's architecture. No security vulnerabilities are known. The main gotcha is the requirement for Python 3.10+ and explicit asyncio reactor configuration\u2014verify that your Scrapy version and environment support this before committing."},"id":"scrapy-impersonate","links":{"html":"https://skillfed.io/packages/scrapy-impersonate","md":"https://skillfed.io/packages/scrapy-impersonate.md","pypi":"https://pypi.org/project/scrapy-impersonate/"},"maintenance":{"status":"active"},"meta":{"latest_release":"2026-05-18","license_spdx":null,"license_treatment":"permissive","name":"scrapy-impersonate","python_support":"supports_current","summary":"Scrapy download handler that can impersonate browser fingerprints"},"popularity":{"monthly_downloads":132327,"position":11556,"tier":"top_15000"},"security":{"n_vulnerabilities":0},"version":"1.7.0"}