httpxthrottlecache

What it is and what it does

httpxthrottlecache is a convenience wrapper around HTTPX that bundles rate limiting and HTTP response caching into a single manager. You configure it once with a cache mode (disabled, file-based, or S3-backed), a rate limit (requests per second), and optional cache rules (site and path regex patterns mapped to TTLs), then use it for synchronous, asynchronous, or batch requests. The package abstracts away the complexity of assembling HTTPX transports and making opinionated choices about when and how to cache.

It supports four cache modes: disabled (rate limiting only), Hishel-File (file-backed), Hishel-S3 (S3-backed), and FileCache (simpler file storage using Last-Modified revalidation). Rate limiting is powered by pyrate-limiter, a leaky-bucket implementation that works across threads and can be extended to multiprocess or distributed scenarios. The package was born from real-world needs—specifically, working around strict rate limits and unhelpful caching headers on certain APIs.

Use it for:

• Scrape rate-limited APIs without exceeding request quotas or hammering servers.
• Batch-download many files over HTTP while respecting rate limits and caching responses locally.
• Build async request pipelines that throttle concurrency and reuse cached responses across multiple tasks.
• Cache HTTP responses with custom TTL rules per site and path, overriding server cache headers when needed.
• Share a single rate limiter across multiple threads using pyrate-limiter's in-memory backend.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Wraps HTTPX with built-in rate limiting and caching transports, letting you control request throttling and response caching through a single manager interface.

Yes, if you need rate limiting and caching together for HTTPX. The package is actively maintained, has low install friction, and solves a real problem. However, verify the license before use in proprietary contexts, and be aware it is young with minimal adoption. Start with a small trial to confirm the cache and rate-limit behavior matches your use case.

Install

httpxthrottlecache on PyPI

pip install httpxthrottlecache

uv add httpxthrottlecache

poetry add httpxthrottlecache

Installing httpxthrottlecache

Before you install

Low friction: pure Python wheel with only three runtime dependencies (aiofiles, filelock, pyrate-limiter). Active maintenance—released 2026-08-13 with no known vulnerabilities. Requires you to install one of two extras to select your HTTP client backend.

License in practice

License status is unclear—no SPDX identifier or raw license text is recorded. Verify the license at the GitHub repository before adopting in proprietary or copyleft-sensitive contexts.

Quickstart

pip install 'httpxthrottlecache[httpx2]'

from httpxthrottlecache import HttpxThrottleCache

with HttpxThrottleCache(cache_mode="Hishel-File", cache_dir="_cache", rate_limiter_enabled=True, request_per_sec_limit=10) as manager:
    with manager.http_client() as client:
        response = client.get("https://example.com")
        print(response.status_code)

Requires Python >=3.10. Must install one of two extras to select HTTP client backend.

Verify before relying

• Production readiness given first release 2025-08-10 and low adoption (4 stars).
• Whether Hishel-S3 caching backend is fully documented and tested.
• Whether multiprocess rate limiting via alternative pyrate-limiter backends is production-ready.

Package facts

Evidence: httpxthrottlecache-0.6.1-py3-none-any.whl

Keywords: edgar, httpx, pyrate_limiter, sec

Tags

More WWW/HTTP packages