truss

What it is and what it does

Truss is a deployment-focused CLI that bridges the gap between local model development and production serving. It wraps your model code, weights, and dependencies into a containerized service that behaves consistently across development and production environments. The tool abstracts away Docker, Kubernetes, and infrastructure configuration—you write a YAML config specifying the model source, hardware (GPU type), and serving engine, then push to deploy.

The package integrates with Baseten's inference stack and supports multiple serving frameworks including vLLM, SGLang, TensorRT-LLM, transformers, diffusers, PyTorch, and TensorFlow. It handles GPU allocation, secrets management, caching, and autoscaling when deployed to Baseten or compatible infrastructure. The live-reload feature lets you iterate on config changes without full rebuilds, and deployed models expose OpenAI-compatible REST APIs by default.

Use it for:

• Deploy a Hugging Face transformer model to a production API endpoint with GPU acceleration and automatic scaling
• Package a custom PyTorch or TensorFlow model with preprocessing logic and serve it via an OpenAI-compatible interface
• Iterate rapidly on model serving config (quantization, sequence length, hardware) with live reload during development
• Manage model dependencies and weights in a single reproducible package that works identically in dev and production
• Optimize inference performance using TensorRT-LLM compilation for large language models without manual container setup

Worth the install?

AI-flagged interpretation of the facts on this page — verify before relying

Truss is a CLI tool for packaging ML models with their serving logic and deploying them to production, handling containerization, dependency management, and GPU configuration automatically.

Yes. Truss is actively maintained, has no known vulnerabilities, and low install friction. It's worth installing if you need to deploy ML models to production and want to skip Docker/Kubernetes boilerplate. Best suited for teams using Baseten or those comfortable with containerized deployments; verify first whether your target infrastructure is supported.

Install

truss on PyPI

pip install truss

uv add truss

poetry add truss

Installing truss

Before you install

Low install friction with a pure-Python wheel. Active maintenance (last commit 2026-08-13) and 1188 repository stars indicate steady development. Supports Python 3.9 through 3.14.

License in practice

MIT license permits commercial and private use with minimal restrictions, making it suitable for production deployments without licensing concerns.

Quickstart

pip install truss
truss init my-model
cd my-model
# Edit config.yaml with model and hardware specs
truss push  # Deploy to Baseten or your infrastructure

Requires a Baseten account and API key for deployment, or compatible infrastructure. Models are deployed via containerization, not run locally by default.

Verify before relying

• Whether Truss can deploy models to infrastructure other than Baseten without additional configuration
• Performance overhead of the Truss serving layer compared to direct framework deployment
• Supported model frameworks beyond those mentioned (vLLM, SGLang, TensorRT-LLM, transformers, diffusers, PyTorch, TensorFlow)

Package facts

Evidence: truss-0.18.25-py3-none-any.whl

Keywords: AI, MLOps, Machine Learning, Model Deployment, Model Serving

Tags

More Artificial Intelligence packages