{"enrichment":{"faq":[{"a":"multi-tenant-llm-hosting isolates tenants through Kubernetes namespaces, network policies, and RBAC rules that prevent cross-tenant access. GPU quotas are enforced per namespace, and request routing uses per-tenant API keys to ensure each customer's workload stays logically and physically separated. vLLM model instances can be deployed in dedicated namespace pods with resource limits, blocking noisy-neighbor interference.","q":"How do you isolate tenants on shared GPU infrastructure?"},{"a":"multi-tenant-llm-hosting integrates billing and usage tracking by tagging every inference request with a tenant identifier, then aggregating token counts, GPU time, and API calls per customer. Cost attribution flows through API gateway logs and Kubernetes metrics into a billing system, allowing you to recover infrastructure costs and charge customers based on actual consumption rather than flat fees.","q":"What does multi-tenant-llm-hosting enable for cost attribution?"},{"a":"multi-tenant-llm-hosting applies per-tenant quotas via Kubernetes ResourceQuotas and LimitRanges in each namespace, plus API gateway rate-limiting rules keyed to tenant API keys. Request routers check quotas before forwarding to vLLM backends, rejecting excess requests with 429 responses. This prevents any single tenant from exhausting shared GPU or memory resources.","q":"How can you implement per-tenant quotas and rate limiting?"},{"a":"multi-tenant-llm-hosting uses namespace isolation, network policies blocking inter-namespace traffic, and RBAC to restrict service account permissions. API keys authenticate tenant identity before request routing, and model cache/KV storage is partitioned per tenant. Kubernetes audit logs track all access, enabling compliance verification and incident response.","q":"What security measures prevent cross-tenant data leakage?"},{"a":"multi-tenant-llm-hosting prevents noisy-neighbor problems by enforcing per-tenant GPU and memory quotas, prioritizing requests with QoS classes, and isolating model instances in separate pods. Fair resource sharing is maintained through Kubernetes scheduling constraints and vLLM queue management, ensuring one tenant's spike doesn't starve others.","q":"How does multi-tenant-llm-hosting prevent noisy-neighbor issues?"},{"a":"multi-tenant-llm-hosting follows a layered architecture: API gateway (tenant authentication and rate limiting), request router (quota checks and tenant tagging), vLLM inference pods (namespace-isolated), and billing aggregator (usage tracking). Each tenant gets a Kubernetes namespace with ResourceQuotas, and metrics flow to a central billing system for cost recovery and reporting.","q":"What is the typical architecture for multi-tenant LLM platform design?"}],"shadow_tags":["tenant-isolation","cost-attribution","resource-quotas","api-gateway-routing","workload-prioritization","billing-integration","noisy-neighbor-mitigation","multi-model-inference","data-partitioning"],"summary_rewrite":"Build shared LLM inference platforms that safely serve multiple teams or customers without compromising security or performance. This skill covers tenant identity enforcement, per-tenant API keys, namespace isolation, quota configuration, request routing, and cost attribution\u2014enabling you to run efficient, governed multi-tenant deployments on Kubernetes with vLLM and API gateways."},"files":[{"bytes":17212,"path":"infrastructure/local-ai/multi-tenant-llm-hosting/SKILL.md","sha256":"638ceede90fd2016540aa592f1fd1062c7f0fb7bb7929f4ded0834a4713ca7e9","url":"https://skillfed.io/files/BagelHole/DevOps-Security-Agent-Skills/multi-tenant-llm-hosting/efb9ad4a/SKILL.md"}],"id":"BagelHole/DevOps-Security-Agent-Skills/multi-tenant-llm-hosting","links":{"html":"https://skillfed.io/BagelHole/DevOps-Security-Agent-Skills/multi-tenant-llm-hosting","md":"https://skillfed.io/BagelHole/DevOps-Security-Agent-Skills/multi-tenant-llm-hosting.md","repo":"https://github.com/BagelHole/DevOps-Security-Agent-Skills"},"meta":{"agents_supported":[],"first_seen":"2026-07-28","forks":4,"language":"Shell","last_updated":"2026-05-22","license":"MIT","name":"multi-tenant-llm-hosting","publisher":"BagelHole","stars":44},"relations":{"similar":[{"id":"BagelHole/DevOps-Security-Agent-Skills/llm-inference-scaling"},{"id":"BagelHole/DevOps-Security-Agent-Skills/ai-security-hardening"},{"id":"mblode/agent-skills/multi-tenant-architecture"},{"id":"agentsope/SkillAlchemy/agentsop-multi-tenant-rag"},{"id":"julianobarbosa/claude-code-skills/keyvault-csi-driver"},{"id":"BagelHole/DevOps-Security-Agent-Skills/model-serving-kubernetes"},{"id":"BagelHole/DevOps-Security-Agent-Skills/llmops-platform-engineering"},{"id":"dadbodgeoff/drift/multi-tenancy"},{"id":"BagelHole/DevOps-Security-Agent-Skills/gpu-kubernetes-operations"},{"id":"JosiahSiegel/claude-plugin-marketplace/aks-automatic-2025"}]},"slug":{"owner":"BagelHole","repo":"DevOps-Security-Agent-Skills","skill":"multi-tenant-llm-hosting"},"version":"efb9ad4a"}