litert-torch

What it is and what it does

LiteRT Torch bridges PyTorch and on-device deployment by converting trained PyTorch models into LiteRT's .tflite format. The library integrates with torch.export() to capture models, then serializes them for execution on mobile and IoT hardware. It provides two main workflows: a PyTorch Converter (Beta) for general model conversion, and a Generative API (Alpha) specialized for transformer-based models and large language models with quantization support.

The package targets developers building edge ML applications where models must run entirely on-device without cloud connectivity. It handles the conversion pipeline and offers initial support for CPU, GPU, and NPU execution. With 19 runtime dependencies including torch, transformers, jax, and tf-nightly, it brings a substantial environment; deployment of the converted .tflite files themselves requires the separate LiteRT runtime.

Use it for:

• Convert a trained vision model to .tflite for inference on Android or iOS devices
• Quantize and optimize a transformer-based LLM for on-device deployment using the Generative API
• Export a PyTorch model for IoT edge devices that cannot reach cloud inference services
• Package a converted model and tokenizer into a .litertlm container for production mobile deployment
• Benchmark model performance on target hardware using LiteRT's compiled model API after conversion

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Converts PyTorch models to .tflite format for on-device deployment on Android, iOS, and IoT devices via LiteRT, with CPU coverage and initial GPU and NPU support.

Yes, if you are converting PyTorch models for on-device deployment and targeting Android, iOS, or IoT. The library is actively maintained, has low install friction, and offers both general and LLM-specific conversion paths. Conditions: requires Python 3.10 through 3.13 (3.11 recommended), Linux, and a large dependency footprint; the Generative API is Alpha, so production LLM workflows should expect iteration.

Install

litert-torch on PyPI

pip install litert-torch

uv add litert-torch

poetry add litert-torch

Installing litert-torch

Before you install

Low friction; pure Python wheel. Active maintenance with release 10 days old and recent commits. However, 19 runtime dependencies including torch, transformers, jax, and tf-nightly create a substantial environment footprint.

License in practice

Permissive license allows commercial and private use with minimal restrictions.

Quickstart

pip install litert-torch

import torch
import litert_torch

model = torch.nn.Linear(10, 2)
with torch.no_grad():
    sample_inputs = (torch.randn(1, 10),)
    edge_model = litert_torch.convert(model.eval(), sample_inputs)
edge_model.export("model.tflite")

Python >=3.10 and <3.14 required; Python 3.11 highly recommended. Linux only. Requires PyTorch >=2.4.0 and tf-nightly.

Verify before relying

• Specific coverage of Core ATen operators and which PyTorch operations are unsupported in conversion
• Performance benchmarks or latency comparisons for converted models on target devices
• Quantization support details beyond the mention of Generative API quantization
• Whether GPU/NPU support is production-ready or experimental

Package facts

Evidence: litert_torch-0.9.3-py3-none-any.whl

Keywords: On-Device ML, AI, Google, TFLite, LiteRT, PyTorch, LLMs, GenAI

Tags

More Software Development packages