--- id: tensorstore version: "0.1.85" license: Apache-2.0 license_treatment: permissive maintenance: active --- # tensorstore — Read and write large, multi-dimensional arrays License: permissive · Maintenance: active · Downloads: 3.6M/mo ## What it is and what it does TensorStore is a C++ and Python library for storing and accessing large multi-dimensional arrays across diverse backends—local disks, cloud storage (Google Cloud Storage, S3), HTTP servers, and in-memory storage—with a single API. It abstracts away the complexity of different array formats (zarr, N5) and storage systems, letting you work with arrays as if they were local while actually reading and writing remotely. The library emphasizes high-throughput access through asynchronous I/O, read caching, and transactional semantics with ACID guarantees. It supports advanced indexing and virtual views, and handles safe concurrent access from multiple processes and machines via optimistic concurrency control. Runtime dependencies are numpy and ml_dtypes. Use it for: - Store and retrieve large scientific datasets (genomics, microscopy, climate data) across cloud and local storage with a uniform interface. - Build machine learning pipelines that read training data from remote zarr arrays with transparent caching and concurrent access. - Implement distributed array computations where multiple workers read/write to the same array with transactional consistency. - Migrate array workloads between storage backends (e.g., local to cloud) without changing application code. - Access N5 or zarr datasets from remote HTTP servers with asynchronous I/O for high-latency networks. ## Worth the install? AI-flagged interpretation of the facts above — verify before relying. TensorStore reads and writes large multi-dimensional arrays across multiple storage backends (local filesystem, cloud storage, S3, HTTP) with a uniform API supporting zarr and N5 formats. Yes. TensorStore is production-stable (Development Status 5), actively maintained with a recent release, has no known vulnerabilities, and solves a real problem for anyone working with large arrays across multiple storage systems. The permissive Apache-2.0 license and broad platform support (Python 3.11+, macOS, Linux, Windows) make it a low-risk addition. Install it if you need a uniform API for multi-backend array storage; skip it if your arrays fit in local memory and you're not crossing storage boundaries. ## Install pip install tensorstore uv add tensorstore poetry add tensorstore ## Installing tensorstore Before you install: Medium install friction due to compiled wheels for multiple Python versions and platforms. Active maintenance with a recent release (9 days ago) and steady repository activity. Requires Python 3.11 or later. License in practice: Apache-2.0 permissive license allows commercial and private use with minimal restrictions; you must include a copy of the license and state significant changes. Quickstart: pip install tensorstore import tensorstore as ts import numpy as np # Open or create an array array = ts.open({"driver": "zarr", "kvstore": "file:///path/to/data"}).result() Requires Python 3.11 or later; compiled wheels available for macOS (x86_64, arm64), Linux (x86_64, aarch64), and Windows (amd64). Verify before relying: - Whether the asynchronous API requires explicit async/await patterns or integrates transparently with standard Python code. - Performance characteristics and memory overhead for typical workloads compared to direct numpy or zarr access. - Specific ACID guarantee semantics and transaction isolation levels under concurrent access. ## Package facts - License: Apache-2.0 (permissive) - Python support: supports_current - Install friction: medium - Maintenance: active - Downloads: 3.6M/month (top 5,000 on PyPI) - Known vulnerabilities: none known ## Tags multi-dimensional array storage, zarr and n5 format support, cloud storage for large arrays, distributed array access, high-performance tensor storage, remote array caching, asynchronous array i/o, array-storage, cloud-native, distributed-computing [View on SkillFed](https://skillfed.io/packages/tensorstore) · [View on PyPI](https://pypi.org/project/tensorstore/)