tensorstore

What it is and what it does

TensorStore is a C++ and Python library for storing and accessing large multi-dimensional arrays across diverse backends—local disks, cloud storage (Google Cloud Storage, S3), HTTP servers, and in-memory storage—with a single API. It abstracts away the complexity of different array formats (zarr, N5) and storage systems, letting you work with arrays as if they were local while actually reading and writing remotely.

The library emphasizes high-throughput access through asynchronous I/O, read caching, and transactional semantics with ACID guarantees. It supports advanced indexing and virtual views, and handles safe concurrent access from multiple processes and machines via optimistic concurrency control. Runtime dependencies are numpy and ml_dtypes.

Use it for:

• Store and retrieve large scientific datasets (genomics, microscopy, climate data) across cloud and local storage with a uniform interface.
• Build machine learning pipelines that read training data from remote zarr arrays with transparent caching and concurrent access.
• Implement distributed array computations where multiple workers read/write to the same array with transactional consistency.
• Migrate array workloads between storage backends (e.g., local to cloud) without changing application code.
• Access N5 or zarr datasets from remote HTTP servers with asynchronous I/O for high-latency networks.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

TensorStore reads and writes large multi-dimensional arrays across multiple storage backends (local filesystem, cloud storage, S3, HTTP) with a uniform API supporting zarr and N5 formats.

Yes. TensorStore is production-stable (Development Status 5), actively maintained with a recent release, has no known vulnerabilities, and solves a real problem for anyone working with large arrays across multiple storage systems. The permissive Apache-2.0 license and broad platform support (Python 3.11+, macOS, Linux, Windows) make it a low-risk addition. Install it if you need a uniform API for multi-backend array storage; skip it if your arrays fit in local memory and you're not crossing storage boundaries.

Install

tensorstore on PyPI

pip install tensorstore

uv add tensorstore

poetry add tensorstore

Installing tensorstore

Before you install

Medium install friction due to compiled wheels for multiple Python versions and platforms. Active maintenance with a recent release (9 days ago) and steady repository activity. Requires Python 3.11 or later.

License in practice

Apache-2.0 permissive license allows commercial and private use with minimal restrictions; you must include a copy of the license and state significant changes.

Quickstart

pip install tensorstore

import tensorstore as ts
import numpy as np

# Open or create an array
array = ts.open({"driver": "zarr", "kvstore": "file:///path/to/data"}).result()

Requires Python 3.11 or later; compiled wheels available for macOS (x86_64, arm64), Linux (x86_64, aarch64), and Windows (amd64).

Verify before relying

• Whether the asynchronous API requires explicit async/await patterns or integrates transparently with standard Python code.
• Performance characteristics and memory overhead for typical workloads compared to direct numpy or zarr access.
• Specific ACID guarantee semantics and transaction isolation levels under concurrent access.

Package facts

Evidence: tensorstore-0.1.85-cp311-cp311-macosx_10_14_x86_64.whl; tensorstore-0.1.85-cp311-cp311-macosx_11_0_arm64.whl; tensorstore-0.1.85-cp311-cp311-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl; tensorstore-0.1.85-cp311-cp311-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl; tensorstore-0.1.85-cp311-cp311-win_amd64.whl; tensorstore-0.1.85-cp312-cp312-macosx_10_14_x86_64.whl; tensorstore-0.1.85-cp312-cp312-macosx_11_0_arm64.whl; tensorstore-0.1.85-cp312-cp312-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl; tensorstore-0.1.85-cp312-cp312-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl; tensorstore-0.1.85-cp312-cp312-win_amd64.whl; tensorstore-0.1.85-cp313-cp313-macosx_10_14_x86_64.whl; tensorstore-0.1.85-cp313-cp313-macosx_11_0_arm64.whl; tensorstore-0.1.85-cp313-cp313-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl; tensorstore-0.1.85-cp313-cp313-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl; tensorstore-0.1.85-cp313-cp313-win_amd64.whl; tensorstore-0.1.85-cp314-cp314-macosx_10_15_x86_64.whl; tensorstore-0.1.85-cp314-cp314-macosx_11_0_arm64.whl; tensorstore-0.1.85-cp314-cp314-manylinux_2_27_aarch64.manylinux_2_28_aarch64.whl; tensorstore-0.1.85-cp314-cp314-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl; tensorstore-0.1.85-cp314-cp314t-macosx_10_15_x86_64.whl

Tags

More Libraries packages