---
id: scrapy-impersonate
version: "1.7.0"
license: MIT
license_treatment: permissive
maintenance: active
---

# scrapy-impersonate — Scrapy download handler that can impersonate browser fingerprints

License: permissive · Maintenance: active · Downloads: 132.3K/mo

## What it is and what it does

scrapy-impersonate is a Scrapy download handler that integrates curl-cffi to perform HTTP requests with spoofed browser fingerprints. Instead of Scrapy's default HTTP client, it routes requests through curl-cffi, which can mimic the TLS signatures and JA3 fingerprints of real browsers—Chrome, Firefox, Safari, and Edge across multiple versions and platforms. This allows web scrapers to bypass basic fingerprint-based bot detection that rejects requests from non-browser clients.

The package works by replacing Scrapy's standard download handlers and providing a middleware that can randomly rotate between supported browser profiles on a per-request basis. You configure it via Scrapy settings, then tag individual requests with an `impersonate` meta key to specify which browser to emulate. The asyncio-based Twisted reactor is required for proper async execution. It's designed for developers who need to scrape sites with fingerprint-based anti-bot protections but want to stay within the Scrapy framework rather than switching to a different HTTP client entirely.

Use it for:

- Scrape sites that reject requests with non-browser TLS signatures by impersonating Chrome, Firefox, or Safari.
- Rotate browser fingerprints across requests to avoid detection patterns based on consistent client profiles.
- Test web applications' anti-bot defenses by simulating requests from different browser versions and platforms.
- Extract data from sites protected by basic JA3 fingerprint checks without rewriting scraper logic outside Scrapy.
- Combine browser impersonation with Scrapy's middleware and pipeline ecosystem for complex scraping workflows.

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

A Scrapy download handler that replaces HTTP/HTTPS requests with curl-cffi to impersonate browser TLS signatures and JA3 fingerprints, allowing scraping of sites that detect and block non-browser clients.

Yes, if you need Scrapy-based scraping against sites with fingerprint-based bot detection. The package is actively maintained, has low install friction, carries a permissive MIT license, and integrates cleanly into Scrapy's architecture. No security vulnerabilities are known. The main gotcha is the requirement for Python 3.10+ and explicit asyncio reactor configuration—verify that your Scrapy version and environment support this before committing.

## Install

pip install scrapy-impersonate
uv add scrapy-impersonate
poetry add scrapy-impersonate

## Installing scrapy-impersonate

Before you install: Low friction: pure Python wheel with only two runtime dependencies (curl-cffi and scrapy). Actively maintained with a recent release (88 days ago) and steady repository activity.

License in practice: MIT license is permissive; you can use, modify, and distribute this package freely in commercial or private projects with minimal restrictions.

Quickstart:

pip install scrapy-impersonate

# In settings.py or custom_settings:
DOWNLOAD_HANDLERS = {
    "http": "scrapy_impersonate.ImpersonateDownloadHandler",
    "https": "scrapy_impersonate.ImpersonateDownloadHandler",
}
TWISTED_REACTOR = "twisted.internet.asyncioreactor.AsyncioSelectorReactor"
USER_AGENT = ""

# In spider:
yield scrapy.Request(
    url,
    meta={"impersonate": "chrome133a"},
)

Requires Python 3.10 or later and the asyncio-based Twisted reactor to be explicitly configured in Scrapy settings.

Verify before relying:

- Whether curl-cffi's browser impersonation reliably defeats modern anti-bot systems beyond basic TLS/JA3 detection.
- Performance overhead compared to standard Scrapy HTTP requests.
- Compatibility with Scrapy middleware and request/response pipelines when using the custom download handler.

## Package facts

- License: MIT (permissive)
- Python support: supports_current
- Install friction: low
- Maintenance: active
- Downloads: 132.3K/month (top 15,000 on PyPI)
- Known vulnerabilities: none known

## Tags

scrapy browser impersonation, TLS fingerprint spoofing, JA3 fingerprint scraping, bypass anti-bot detection, curl-cffi scrapy integration, browser user-agent impersonation, scrapy download handler, web-scraping, anti-bot-bypass, tls-fingerprint

[View on SkillFed](https://skillfed.io/packages/scrapy-impersonate) · [View on PyPI](https://pypi.org/project/scrapy-impersonate/)