pytorch-deployment
Move trained PyTorch models from notebooks to production environments by mastering serialization formats like TorchScript (via tracing or scripting) and ONNX export. Learn quantization and pruning strategies to shrink model size and accelerate inference across C++, mobile, and edge devices.
PyTorch Deployment helps you serialize and optimize models for production using TorchScript, ONNX, and quantization.
AI-generated summary based on this skill's SKILL.md
Install
tondevrel/scientific-agent-skills/pytorch-deployment
git clone https://github.com/tondevrel/scientific-agent-skills
cp -r scientific-agent-skills/skills/pytorch-deployment ~/.claude/skills/pytorch-deploymentnpx skillfed install tondevrel/scientific-agent-skills/pytorch-deploymentFrequently asked questions
AI-generated answers based on this skill's SKILL.md and metadata
How do you deploy PyTorch models to production?
pytorch-deployment teaches you to move trained models from notebooks into production by mastering serialization formats like TorchScript and ONNX export. The skill covers exporting models for C++ inference via libtorch, deploying to mobile and edge devices, and optimizing through quantization and pruning to reduce size and accelerate inference speed.
What's the difference between TorchScript tracing vs scripting?
pytorch-deployment explains that TorchScript tracing records tensor operations by running your model once with sample inputs, capturing the execution graph—ideal for models with fixed control flow. Scripting, by contrast, parses Python code directly into TorchScript, preserving dynamic control flow and loops, making it better for models with conditional logic.
How do you export a PyTorch model to ONNX format?
pytorch-deployment covers ONNX export as a key serialization strategy for cross-platform inference. You use torch.onnx.export() to convert your model into the ONNX interchange format, enabling deployment in non-Python environments and runtime engines like ONNX Runtime, TensorRT, and others without requiring PyTorch.
What quantization techniques reduce PyTorch model size?
pytorch-deployment teaches quantization strategies—particularly INT8 quantization—to shrink model size and boost inference speed. These techniques reduce floating-point weights and activations to lower precision, cutting memory footprint and latency on edge and mobile devices while maintaining acceptable accuracy.
How can you run PyTorch inference without Python?
pytorch-deployment shows how to deploy neural networks in non-Python environments using libtorch (C++ API) or ONNX Runtime. Export your model via TorchScript or ONNX, then load and run inference in C++, mobile apps, or edge platforms like Jetson, eliminating the Python runtime dependency.
What's the best approach for PyTorch mobile deployment?
pytorch-deployment covers mobile optimization by combining serialization (TorchScript export), quantization to reduce model size, and the PyTorch Mobile interpreter for efficient on-device inference. These techniques enable fast, lightweight model execution on smartphones and embedded devices with limited compute and memory.
SKILL.md
rendered from the published skill — quoted content, verbatim
PyTorch - Deployment & Production Engineering
Deploying a model in a high-performance environment often means removing the Python dependency. This guide covers how to serialize models into formats that can be loaded in C++, optimized for edge devices, or executed in high-throughput inference engines like TensorRT.
When to Use
- Moving a model from a Jupyter Notebook to a production web server (FastAPI/Go/Rust).
- Embedding a neural network into a C++ application (LibTorch).
- Running inference on mobile devices (iOS/Android) or edge hardware (NVIDIA Jetson).
- Accelerating inference speed using specialized hardware backends (OpenVINO, TensorRT).
- Ensuring model reproducibility across different versions of PyTorch.
Core Principles
1. Scripting vs. Tracing
- Tracing: PyTorch runs the model once with "example data" and records all operations. Fast, but ignores Python control flow
(truncated - see the full file via the links below)
Read as markdown · JSON record · Browse the source repository
File tree — 1 file
skills/pytorch-deployment/SKILL.md