{"enrichment":{"faq":[{"a":"model-serving-kubernetes enables production ML inference on Kubernetes using KServe or NVIDIA Triton. You define InferenceService resources that handle model deployment, versioning, and traffic routing. KServe abstracts Kubernetes complexity with built-in model server integration, while Triton offers lower-level control for high-throughput batch inference. Both support GPU allocation and automatic scaling based on request metrics.","q":"How do I deploy machine learning models on Kubernetes?"},{"a":"Yes. model-serving-kubernetes supports canary deployments through traffic splitting\u2014route a percentage of requests to new model versions while keeping most traffic on stable versions. This enables safe rollouts and A/B testing of model updates. You configure traffic percentages in the InferenceService spec, allowing gradual validation before full promotion.","q":"Can model-serving-kubernetes implement canary deployments for ML models?"},{"a":"model-serving-kubernetes autoscales inference workloads based on GPU and request metrics. It integrates with Kubernetes' Horizontal Pod Autoscaler to spawn inference pods when demand rises and shrink when idle. You set target metrics like request latency or GPU utilization, enabling cost-efficient scaling of expensive GPU resources.","q":"What autoscaling options does model-serving-kubernetes provide for GPU workloads?"},{"a":"model-serving-kubernetes manages multiple model versions through traffic splitting and canary patterns. Deploy different versions as separate predictors within a single InferenceService, then allocate traffic percentages to each. This supports gradual rollouts, instant rollbacks, and A/B testing without redeploying infrastructure.","q":"How does model-serving-kubernetes handle multiple model versions in production?"},{"a":"Yes. model-serving-kubernetes configures NVIDIA Triton for high-throughput batch inference with dynamic batching, model repositories on S3, and GPU optimization. Triton maximizes throughput by batching requests and supports multiple frameworks (PyTorch, TensorFlow). It's ideal when you need fine-grained control over inference performance.","q":"Is NVIDIA Triton supported for high-throughput batch inference?"},{"a":"model-serving-kubernetes supports both: KServe provides a higher-level abstraction with simpler setup, automatic model server selection, and built-in canary/traffic splitting. Triton offers lower-level control, dynamic batching, and multi-framework support for maximum throughput. Choose KServe for ease, Triton for performance tuning.","q":"What's the difference between KServe and Triton in model-serving-kubernetes?"}],"shadow_tags":["inference-orchestration","gpu-workload-scheduling","model-versioning-strategy","progressive-deployment","batch-optimization","containerized-ml-ops","cloud-model-registry","request-rate-scaling","multi-framework-support","production-ml-infrastructure"],"summary_rewrite":"Run production ML inference on Kubernetes using KServe or NVIDIA Triton, with built-in support for canary traffic splitting, request-based autoscaling, and GPU resource allocation. The skill covers model versioning, A/B testing patterns, and dynamic batching for throughput optimization."},"files":[{"bytes":8481,"path":"devops/orchestration/model-serving-kubernetes/SKILL.md","sha256":"9d2019080321a0af0be7ae82d46ce3bbcf6fa3509b66185130fd91fb4ec3a422","url":"https://skillfed.io/files/BagelHole/DevOps-Security-Agent-Skills/model-serving-kubernetes/a305c617/SKILL.md"}],"id":"BagelHole/DevOps-Security-Agent-Skills/model-serving-kubernetes","links":{"html":"https://skillfed.io/BagelHole/DevOps-Security-Agent-Skills/model-serving-kubernetes","md":"https://skillfed.io/BagelHole/DevOps-Security-Agent-Skills/model-serving-kubernetes.md","repo":"https://github.com/BagelHole/DevOps-Security-Agent-Skills"},"meta":{"agents_supported":[],"first_seen":"2026-07-28","forks":4,"language":"Shell","last_updated":"2026-05-22","license":"MIT","name":"model-serving-kubernetes","publisher":"BagelHole","stars":44},"relations":{"similar":[{"id":"JosiahSiegel/claude-plugin-marketplace/ml-cloud-deployment"},{"id":"BagelHole/DevOps-Security-Agent-Skills/llmops-platform-engineering"},{"id":"BagelHole/DevOps-Security-Agent-Skills/llm-inference-scaling"},{"id":"BagelHole/DevOps-Security-Agent-Skills/gpu-kubernetes-operations"},{"id":"google/skills/gke-inference"},{"id":"JosiahSiegel/claude-plugin-marketplace/ml-inference-optimization"},{"id":"BagelHole/DevOps-Security-Agent-Skills/multi-tenant-llm-hosting"},{"id":"JosiahSiegel/claude-plugin-marketplace/aks-automatic-2025"},{"id":"BagelHole/DevOps-Security-Agent-Skills/gcp-gke"},{"id":"scitix/siclaw/volcano-node-resources"}]},"slug":{"owner":"BagelHole","repo":"DevOps-Security-Agent-Skills","skill":"model-serving-kubernetes"},"version":"a305c617"}