---
id: obstore
version: "0.11.0"
license: unclear
license_treatment: permissive
maintenance: active
---

# obstore

License: permissive · Maintenance: active · Downloads: 10.5M/mo

## What it is and what it does

obstore is a Rust-backed Python library that abstracts away the differences between cloud storage providers—S3, Google Cloud Storage, Azure Storage, and S3-compatible services—behind a single, type-hinted interface. It handles both synchronous and asynchronous operations, streaming data transfer, automatic multipart uploads for large files, and credential refresh, all with minimal Python dependencies (only typing-extensions at runtime).

The package is designed for high-throughput workloads where you need to interact with object storage from Python without the overhead of traditional clients. It exposes file-like APIs, integrates with fsspec, and can return list results as Apache Arrow tables for memory efficiency. Zero-copy data exchange via the buffer protocol minimizes overhead between Rust and Python.

Use it for:

- Bulk download or upload of files from multiple cloud providers using a single API without provider-specific code.
- High-concurrency async workloads that need to fetch many small objects from S3 or compatible storage efficiently.
- Streaming large files from cloud storage to avoid loading entire objects into memory.
- Automated credential refresh in long-running applications that interact with expiring cloud credentials.
- Integration with geospatial or data science pipelines that read from cloud-hosted datasets via fsspec.

## Worth the install?

AI-flagged interpretation of the facts above — verify before relying.

obstore provides a unified Python interface to Amazon S3, Google Cloud Storage, Azure Storage, and S3-compliant APIs, with both sync and async support, streaming uploads and downloads, and automatic multipart handling.

Yes, if you need a unified, high-performance interface to multiple cloud storage backends. The permissive MIT license, active maintenance, lack of heavy Python dependencies, and strong async support make it a solid choice for cloud-native Python applications. Medium install friction (compiled wheels) is manageable on common platforms. No known vulnerabilities.

## Install

pip install obstore
uv add obstore
poetry add obstore

## Installing obstore

Before you install: Medium install friction due to compiled Rust bindings, but wheels are available for many platforms including macOS, Linux, and Windows across multiple architectures. Active maintenance with a recent release.

License in practice: Licensed under MIT (permissive), so you can use it freely in commercial and private projects without restriction.

Quickstart:

pip install obstore

import obstore

# Create S3 client and get object
store = obstore.S3Store()
data = store.get("s3://bucket/key")

Requires Python 3.10 or later. Compiled wheels available for most platforms, but installation may require a build environment on unsupported architectures.

Verify before relying:

- Whether the 9x throughput claim vs fsspec and 2.8x vs aioboto3 holds across your specific workload and concurrency patterns.
- Whether Apache Arrow format support requires additional dependencies or configuration beyond what's listed.

## Package facts

- License: not declared (permissive)
- Python support: supports_current
- Install friction: medium
- Maintenance: active
- Downloads: 10.5M/month (top 5,000 on PyPI)
- Known vulnerabilities: none known

## Tags

s3 python client, cloud storage interface, async object storage, streaming uploads downloads, multipart upload handler, s3 compatible storage, google cloud storage python, cloud-storage, async-io, rust-bindings

[View on SkillFed](https://skillfed.io/packages/obstore) · [View on PyPI](https://pypi.org/project/obstore/)