agentsop-llm-engine-selection
This skill guides you through selecting an LLM serving engine by analyzing hardware topology, workload profile, and operational constraints rather than benchmark headlines. It maps seven engines—vLLM, SGLang, TensorRT-LLM, TGI, llama.cpp, Ollama, and MLX—to their strengths across GPU clusters, edge devices, and single-user scenarios, then walks you through a five-step decision workflow to eliminate incompatible options and benchmark your top candidates.
agentsop-llm-engine-selection helps you pick the best LLM inference engine by matching hardware, workload, and constraints to engine strengths.
AI-generated summary based on this skill's SKILL.md
Install
agentsope/SkillAlchemy/agentsop-llm-engine-selection · repository language: Python
git clone https://github.com/agentsope/SkillAlchemy
cp -r SkillAlchemy/skills/agentsop-llm-engine-selection ~/.claude/skills/agentsop-llm-engine-selectionnpx skillfed install agentsope/SkillAlchemy/agentsop-llm-engine-selectionFrequently asked questions
AI-generated answers based on this skill's SKILL.md and metadata
Which LLM inference engine should I use for my specific hardware and workload?
agentsop-llm-engine-selection guides you through a five-step decision workflow that analyzes your hardware topology, workload profile, and operational constraints. The skill maps seven engines—vLLM, SGLang, TensorRT-LLM, TGI, llama.cpp, Ollama, and MLX—to their strengths across GPU clusters, edge devices, and single-user scenarios, helping you eliminate incompatible options and identify your best candidates before benchmarking.
How do vLLM, SGLang, and TensorRT-LLM compare for production LLM serving?
agentsop-llm-engine-selection compares these engines across performance tradeoffs and benchmarking methodology. vLLM excels in multi-user GPU cluster deployments with high throughput; SGLang adds structured output and agent-friendly features; TensorRT-LLM offers maximum performance on NVIDIA hardware but requires more optimization expertise. The skill helps you weigh these differences against your specific constraints—latency, throughput, hardware topology, and operational overhead.
What inference engine should I choose for CPU-only or edge deployment?
agentsop-llm-engine-selection evaluates llama.cpp, Ollama, and MLX for resource-constrained environments. llama.cpp and Ollama suit CPU-only servers with minimal dependencies; MLX is optimized for Apple Silicon. The skill's decision workflow helps you match engine capabilities—quantization support, model format compatibility, and inference speed—to your edge hardware and latency requirements.
Should I migrate from TGI to vLLM or another serving runtime?
agentsop-llm-engine-selection provides a migration evaluation framework by comparing runtime performance, feature gaps, and operational switching costs. The skill assesses whether your current bottleneck is throughput, latency, structured output support, or cost, then determines if migration gains justify redeployment effort. It guides you through benchmarking your top candidates on your actual workload before committing to a switch.
How do I design a multi-tier LLM serving deployment across heterogeneous hardware?
agentsop-llm-engine-selection supports heterogeneous deployment design by mapping each engine to its optimal tier—vLLM or TensorRT-LLM for GPU clusters, MLX for Apple Silicon, llama.cpp or Ollama for CPU fallback. The skill helps you route requests intelligently, balance load across tiers, and select engines that share compatible model formats to minimize conversion overhead and operational complexity.
What is the right LLM inference benchmarking methodology for engine comparison?
agentsop-llm-engine-selection explains performance tradeoff evaluation across inference engines, emphasizing that benchmark headlines often hide hardware-specific tuning and workload assumptions. The skill guides you to benchmark your top candidates on your actual hardware, model size, batch profile, and latency requirements rather than relying on published results, ensuring fair comparison and production-realistic performance estimates.
SKILL.md
rendered from the published skill — quoted content, verbatim
LLM Engine Selection SOP
> State-of-the-art warning. This skill is dated May 2026. The inference-engine landscape moves in 3–6 month cycles (TGI exited maintenance into deprecation in late 2025; SGLang's
(truncated - see the full file via the links below)
Read as markdown · JSON record · Browse the source repository
File tree — 5 files
skills/agentsop-llm-engine-selection/README.md
skills/agentsop-llm-engine-selection/SKILL.md
skills/agentsop-llm-engine-selection/intermediate/operation_candidates.json
skills/agentsop-llm-engine-selection/references/R1-source-evidence.md
skills/agentsop-llm-engine-selection/references/R2-decision-flowchart.md