--- id: truss version: "0.18.25" license: MIT license_treatment: permissive maintenance: active --- # truss — A seamless bridge from model development to model delivery License: permissive · Maintenance: active · Downloads: 1.3M/mo ## What it is and what it does Truss is a deployment-focused CLI that bridges the gap between local model development and production serving. It wraps your model code, weights, and dependencies into a containerized service that behaves consistently across development and production environments. The tool abstracts away Docker, Kubernetes, and infrastructure configuration—you write a YAML config specifying the model source, hardware (GPU type), and serving engine, then push to deploy. The package integrates with Baseten's inference stack and supports multiple serving frameworks including vLLM, SGLang, TensorRT-LLM, transformers, diffusers, PyTorch, and TensorFlow. It handles GPU allocation, secrets management, caching, and autoscaling when deployed to Baseten or compatible infrastructure. The live-reload feature lets you iterate on config changes without full rebuilds, and deployed models expose OpenAI-compatible REST APIs by default. Use it for: - Deploy a Hugging Face transformer model to a production API endpoint with GPU acceleration and automatic scaling - Package a custom PyTorch or TensorFlow model with preprocessing logic and serve it via an OpenAI-compatible interface - Iterate rapidly on model serving config (quantization, sequence length, hardware) with live reload during development - Manage model dependencies and weights in a single reproducible package that works identically in dev and production - Optimize inference performance using TensorRT-LLM compilation for large language models without manual container setup ## Worth the install? AI-flagged interpretation of the facts above — verify before relying. Truss is a CLI tool for packaging ML models with their serving logic and deploying them to production, handling containerization, dependency management, and GPU configuration automatically. Yes. Truss is actively maintained, has no known vulnerabilities, and low install friction. It's worth installing if you need to deploy ML models to production and want to skip Docker/Kubernetes boilerplate. Best suited for teams using Baseten or those comfortable with containerized deployments; verify first whether your target infrastructure is supported. ## Install pip install truss uv add truss poetry add truss ## Installing truss Before you install: Low install friction with a pure-Python wheel. Active maintenance (last commit 2026-08-13) and 1188 repository stars indicate steady development. Supports Python 3.9 through 3.14. License in practice: MIT license permits commercial and private use with minimal restrictions, making it suitable for production deployments without licensing concerns. Quickstart: pip install truss truss init my-model cd my-model # Edit config.yaml with model and hardware specs truss push # Deploy to Baseten or your infrastructure Requires a Baseten account and API key for deployment, or compatible infrastructure. Models are deployed via containerization, not run locally by default. Verify before relying: - Whether Truss can deploy models to infrastructure other than Baseten without additional configuration - Performance overhead of the Truss serving layer compared to direct framework deployment - Supported model frameworks beyond those mentioned (vLLM, SGLang, TensorRT-LLM, transformers, diffusers, PyTorch, TensorFlow) ## Package facts - License: MIT (permissive) - Python support: supports_current - Install friction: low - Maintenance: active - Downloads: 1.3M/month (top 5,000 on PyPI) - Known vulnerabilities: none known ## Tags ML model deployment CLI, model serving containerization, AI model production packaging, machine learning model deployment, GPU-enabled model serving, model to API endpoint, MLOps deployment tool, model-deployment, mlops, containerization [View on SkillFed](https://skillfed.io/packages/truss) · [View on PyPI](https://pypi.org/project/truss/)