cached-path

What it is and what it does

cached-path is a file utility that abstracts away the difference between local paths and remote URLs, letting you treat them uniformly. Pass it a local path, HTTP URL, S3 URI, Google Cloud Storage path, or HuggingFace Hub reference—it checks if the file exists locally, downloads and caches it if remote, and returns the local path. It also supports extracting specific files from tar and zip archives on the fly.

The package is built for AI/ML workflows where models and datasets live in different places (local disk, cloud storage, model hubs) and you want a single code path to handle all of them. It handles authentication via custom headers, manages a configurable cache directory (defaulting to ~/.cache/cached_path/), and uses filelock to coordinate concurrent access.

Use it for:

• Download and cache model weights from HuggingFace Hub in a single call without manual path logic.
• Build data pipelines that work with datasets stored on S3, Google Cloud Storage, or local disk using identical code.
• Extract a specific file from a remote tarball or zipfile and cache it locally without manual archive handling.
• Authenticate to private repositories using bearer tokens and cache the downloaded files automatically.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Provides a unified interface to access local and remote files (HTTP, HTTPS, S3, Google Cloud Storage, HuggingFace Hub) with automatic caching and archive extraction.

Yes. The package solves a real problem in ML/data workflows—unifying file access across local and cloud storage—with low install friction, active maintenance, no known vulnerabilities, and permissive licensing. It's a thin, focused utility that does one thing well and integrates cleanly with common cloud providers and model hubs.

Install

cached-path on PyPI

pip install cached-path

uv add cached-path

poetry add cached-path

Installing cached-path

Before you install

Low install friction with a pure Python wheel. Depends on requests, rich, filelock, boto3, google-cloud-storage, huggingface-hub, and packaging. Actively maintained with recent commits and no archived status.

License in practice

Licensed under Apache 2.0 (permissive), allowing commercial and derivative use with minimal restrictions—just retain license notices and disclose modifications.

Quickstart

pip install cached-path

from cached_path import cached_path

# Download and cache a remote file
local_path = cached_path("https://example.com/model.bin")

# Or access a HuggingFace Hub file
local_path = cached_path("hf://epwalsh/bert-xsmall-dummy/pytorch_model.bin")

# Extract a file from a remote archive
local_path = cached_path("model.tar.gz!weights.th", extract_archive=True)

Requires Python 3.9 or later

Verify before relying

• Whether beaker:// URL support requires optional beaker-py installation or is enabled by default
• Performance characteristics when caching large files or handling concurrent access via filelock

Package facts

Evidence: cached_path-1.8.10-py3-none-any.whl

Tags

More Artificial Intelligence packages