s3pathlib

What it is and what it does

s3pathlib wraps boto3 with a pathlib-inspired API so you work with S3 objects and directories using familiar Python path operations. Instead of calling boto3 client methods directly, you construct S3Path objects from bucket and key components (or S3 URIs and ARNs), then call methods like write_text(), read_text(), open(), and delete() as if they were local filesystem paths.

The package handles the complexity of S3's flat object model and pagination limits by providing recursive folder operations (upload_dir, copy_to, delete), object filtering by size/extension/etag/mtime, and file-like I/O for streaming large objects. It maintains a context object that holds your boto3 session, so you configure credentials once and reuse them across all S3Path instances.

Use it for:

• Upload entire local directory trees to S3 with pattern matching (e.g., all .py files) without manual pagination.
• Copy or delete S3 folders recursively in a single call, avoiding manual list-and-delete loops.
• Stream large S3 objects line-by-line or into pandas DataFrames using context-manager syntax.
• Filter S3 objects by size, extension, or modification time before processing, reducing unnecessary data transfer.
• Inspect S3 object metadata (etag, size, human-readable size) and folder statistics (object count, total size) with simple method calls.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

s3pathlib provides a pathlib-like object-oriented interface to AWS S3, letting you construct, query, and manipulate S3 objects and directories using familiar Python path syntax.

Yes, if you work regularly with S3 and prefer a Pythonic path-like interface over raw boto3 calls. The low install friction and permissive license make it a safe add. Caveat: maintenance is aging (last release 367 days ago), so verify it still works with your boto3 version and check for any open issues matching your use case before committing to production.

Install

s3pathlib on PyPI

pip install s3pathlib

uv add s3pathlib

poetry add s3pathlib

Installing s3pathlib

Before you install

Low install friction; pure Python wheel with six runtime dependencies (boto3, boto_session_manager, func_args, iterproxy, pathlib_mate, smart_open). Last release was 367 days ago; repo is not archived but maintenance signal is aging.

License in practice

Apache-2.0 permissive license; you may use, modify, and distribute freely in commercial and private projects with minimal restrictions.

Quickstart

from s3pathlib import S3Path
import boto3
from s3pathlib import context

context.attach_boto_session(boto3.session.Session(region_name="us-east-1"))
p = S3Path("bucket", "folder", "file.txt")
p.write_text("data")
print(p.etag, p.size)

Requires valid AWS credentials configured in your boto3 session (via environment, config file, or explicit profile).

Verify before relying

• Whether the package handles concurrent S3 operations safely or has thread-safety guarantees.
• Performance characteristics when iterating over buckets with millions of objects.
• Compatibility with S3-compatible services (MinIO, LocalStack) beyond AWS S3.

Package facts

Evidence: s3pathlib-2.3.6-py3-none-any.whl

Tags

More Utilities packages