agentsop-llm-engine-selection
This skill guides you through selecting an LLM serving engine by analyzing hardware topology, workload profile, and operational constraints rather than benchmark headlines. It maps seven engines—vLLM, SGLang, TensorRT-LLM, TGI, llama.cpp, Ollama, and MLX—to their strengths across GPU clusters, edge devices, and single-user scenarios, then walks you through a five-step decision workflow to eliminate incompatible options and benchmark your top candidates.
agentsop-llm-engine-selection helps you pick the best LLM inference engine by matching hardware, workload, and constraints to engine strengths.
AI-generated summary based on this skill's SKILL.md
Decision gist · record as of 2026-06-30
agentsop-llm-engine-selection helps you pick the best LLM inference engine by matching hardware, workload, and constraints to engine strengths. This skill guides you through selecting an LLM serving engine by analyzing hardware topology, workload profile, and operational constraints rather than benchmark headlines. It maps seven engines—vLLM, SGLang, TensorRT-LLM, TGI, llama.cpp, Ollama, and MLX—to their strengths across GPU clusters, edge devices, and single-user scenarios, then walks you through a five-step decision workflow to eliminate incompatible options and benchmark your top candidates.
Use it when
- agentsop-llm-engine-selection compares these engines across performance tradeoffs and benchmarking methodology.
- agentsop-llm-engine-selection evaluates llama.cpp, Ollama, and MLX for resource-constrained environments.
Verify before relying
Read SKILL.md below before installing (5 files). Open directory: indexed for reading, not audited.
Install
agentsope/SkillAlchemy/agentsop-llm-engine-selection · repository language: Python
Open directory. Skills are indexed for reading, not audited. Review a skill's body before installing it.
Frequently asked questions
AI-generated answers based on this skill's SKILL.md and metadata
Which LLM inference engine should I use for my specific hardware and workload?
agentsop-llm-engine-selection guides you through a five-step decision workflow that analyzes your hardware topology, workload profile, and operational constraints. The skill maps seven engines—vLLM, SGLang, TensorRT-LLM, TGI, llama.cpp, Ollama, and MLX—to their strengths across GPU clusters, edge devices, and single-user scenarios, helping you eliminate incompatible options and identify your best candidates before benchmarking.
How do vLLM, SGLang, and TensorRT-LLM compare for production LLM serving?
agentsop-llm-engine-selection compares these engines across performance tradeoffs and benchmarking methodology. vLLM excels in multi-user GPU cluster deployments with high throughput; SGLang adds structured output and agent-friendly features; TensorRT-LLM offers maximum performance on NVIDIA hardware but requires more optimization expertise. The skill helps you weigh these differences against your specific constraints—latency, throughput, hardware topology, and operational overhead.
What inference engine should I choose for CPU-only or edge deployment?
agentsop-llm-engine-selection evaluates llama.cpp, Ollama, and MLX for resource-constrained environments. llama.cpp and Ollama suit CPU-only servers with minimal dependencies; MLX is optimized for Apple Silicon. The skill's decision workflow helps you match engine capabilities—quantization support, model format compatibility, and inference speed—to your edge hardware and latency requirements.
Should I migrate from TGI to vLLM or another serving runtime?
agentsop-llm-engine-selection provides a migration evaluation framework by comparing runtime performance, feature gaps, and operational switching costs. The skill assesses whether your current bottleneck is throughput, latency, structured output support, or cost, then determines if migration gains justify redeployment effort. It guides you through benchmarking your top candidates on your actual workload before committing to a switch.
How do I design a multi-tier LLM serving deployment across heterogeneous hardware?
agentsop-llm-engine-selection supports heterogeneous deployment design by mapping each engine to its optimal tier—vLLM or TensorRT-LLM for GPU clusters, MLX for Apple Silicon, llama.cpp or Ollama for CPU fallback. The skill helps you route requests intelligently, balance load across tiers, and select engines that share compatible model formats to minimize conversion overhead and operational complexity.
What is the right LLM inference benchmarking methodology for engine comparison?
agentsop-llm-engine-selection explains performance tradeoff evaluation across inference engines, emphasizing that benchmark headlines often hide hardware-specific tuning and workload assumptions. The skill guides you to benchmark your top candidates on your actual hardware, model size, batch profile, and latency requirements rather than relying on published results, ensuring fair comparison and production-realistic performance estimates.
SKILL.md
Rendered from the published skill. Quoted content, verbatim.
LLM Engine Selection SOP
> State-of-the-art warning. This skill is dated May 2026. The inference-engine landscape moves in 3–6 month cycles (TGI exited maintenance into deprecation in late 2025; SGLang's
(truncated - see the full file via the links below)
File tree — 5 files
skills/agentsop-llm-engine-selection/README.md
skills/agentsop-llm-engine-selection/SKILL.md
skills/agentsop-llm-engine-selection/intermediate/operation_candidates.json
skills/agentsop-llm-engine-selection/references/R1-source-evidence.md
skills/agentsop-llm-engine-selection/references/R2-decision-flowchart.md
Let your AI agent find skills like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 56,283 agent skills by what they can do, searchable in plain language.
wish › “Choose the right LLM inference engine for a specific hardware/workload/constraint combination”
Give your agent the search over MCP, or paste the wish link into any chat. No install? Search from any chat →
Related skills
A structured decision guide for deploying and tuning vLLM in production environments. Covers the PagedAttention memory model, continuous batching, quantization tradeoffs, tensor/pipeline parallelism choices, and step-by-step operational workflows for diagnosing throughput, latency, and out-of-memory issues. Includes comparisons to alternative inference engines and guidance on when vLLM is the right fit.
This skill guides you through deploying language models with production-grade inference engines tailored to your hardware and use case. Choose between vLLM for maximum throughput on GPUs, llama.cpp for CPU and edge devices, or Ollama for quick local experimentation. Learn quantization strategies, memory optimization, and platform-specific tuning to balance speed and resource constraints.
This skill delivers operational patterns for tuning LLM inference performance, cost, and reliability in production. It covers latency budgeting, continuous batching, KV-cache optimization, speculative decoding, quantization strategies, and parallelism choices—with decision trees and checklists to guide infrastructure and serving stack tuning.
GPU Server Management guides you through provisioning and configuring NVIDIA GPU infrastructure for LLM inference and model training. It covers driver and CUDA toolkit installation, Docker GPU integration, multi-GPU topology setup, and production monitoring with DCGM and Prometheus metrics.
TensorRT-LLM accelerates large language model inference on NVIDIA GPUs through advanced optimization techniques including quantization, in-flight batching, and multi-GPU parallelism. Achieve production-grade throughput and latency for real-time applications with support for 100+ models.
Ollama lets you deploy and serve open-weight language models on your machine with automatic GPU detection. It exposes an OpenAI-compatible API endpoint, integrates with PenguinHarness for model registration, and supports flexible context window configuration for agent workloads.
More skills model-serving (MIT) · serving-llms-vllm (MIT)