tensorizer

What it is and what it does

tensorizer is a PyTorch serialization library designed to decouple large models from container images and enable fast streaming loads from remote storage. It writes torch.nn.Module instances to a binary format that can be read back via HTTP/HTTPS, S3, Redis, or local filesystem, with streaming deserialization that avoids downloading entire models to disk first.

The package is built for serverless and containerized inference workflows where model load time directly impacts latency. By storing serialized models separately from container images, you can update models without rebuilding containers and stream weights directly into GPU memory. It supports local filesystem access for development, S3 for cloud object storage, and HTTP/HTTPS endpoints for any compatible server. Redis support is available but marked preliminary and not recommended for production model deployment.

Use it for:

• Reduce KNative serverless function cold-start latency by streaming large models from S3 instead of embedding them in container images.
• Deploy and update multi-gigabyte language models without rebuilding or redeploying container images.
• Load models directly into GPU memory from HTTP/HTTPS endpoints at network wire-speed without intermediate disk storage.
• Share model state between inference pods via Redis for per-request data loading.
• Serialize and deserialize models locally for development and testing with the same fast streaming path used in production.

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Serializes and deserializes PyTorch modules and tensors to/from HTTP, HTTPS, S3, Redis, and local filesystem endpoints with streaming support for fast model loading.

Yes, if you deploy large PyTorch models in containerized or serverless environments and need to minimize load latency. The permissive MIT license, active maintenance, and zero known vulnerabilities make it safe to adopt. Install friction is low. Not necessary for single-machine development or models small enough to embed in container images.

Install

tensorizer on PyPI

pip install tensorizer

uv add tensorizer

poetry add tensorizer

Installing tensorizer

Before you install

Low friction install with a pure-Python wheel. Maintenance is active with recent commits and stable production status. Eight runtime dependencies include torch, numpy, and cloud/storage libraries (boto3, redis, hiredis); libnacl requires a system libsodium library.

License in practice

MIT License (permissive) allows commercial and private use with minimal restrictions; attribution required but no copyleft obligations.

Quickstart

pip install tensorizer

from tensorizer import TensorDeserializer
import torch

deserializer = TensorDeserializer("s3://bucket/model.tensors", device="cuda")
deserializer.load_into_module(model)
deserializer.close()

libnacl requires system libsodium library; boto3 credentials needed for S3 access; torch and CUDA/CPU device setup required for deserialization.

Verify before relying

• Whether the ~5GB/s wire-speed claim applies to typical network conditions outside 40GbE lab environments.
• Redis support maturity and whether preliminary status affects production reliability.
• Memory overhead during serialization/deserialization of very large models.

Package facts

Evidence: tensorizer-2.12.1-py3-none-any.whl

Keywords: tensorizer, machine learning, serialization, tensor, pytorch

Tags

More Internet packages