gpu-server-management
GPU Server Management guides you through provisioning and configuring NVIDIA GPU infrastructure for LLM inference and model training. It covers driver and CUDA toolkit installation, Docker GPU integration, multi-GPU topology setup, and production monitoring with DCGM and Prometheus metrics.
GPU Server Management helps you provision and configure NVIDIA GPU servers for AI inference and training workloads.
AI-generated summary based on this skill's SKILL.md
Decision gist · record as of 2026-05-22
GPU Server Management helps you provision and configure NVIDIA GPU servers for AI inference and training workloads. GPU Server Management guides you through provisioning and configuring NVIDIA GPU infrastructure for LLM inference and model training. It covers driver and CUDA toolkit installation, Docker GPU integration, multi-GPU topology setup, and production monitoring with DCGM and Prometheus metrics.
Use it when
- GPU Server Management provides end-to-end LLM server provisioning.
- GPU Server Management explains MIG setup for workload isolation.
Verify before relying
Read SKILL.md below before installing (1 file). Open directory: indexed for reading, not audited.
Install
BagelHole/DevOps-Security-Agent-Skills/gpu-server-management · repository language: Shell
Open directory. Skills are indexed for reading, not audited. Review a skill's body before installing it.
Frequently asked questions
AI-generated answers based on this skill's SKILL.md and metadata
How do I install NVIDIA GPU drivers on Ubuntu?
GPU Server Management covers driver installation across distributions. On Ubuntu, download drivers from NVIDIA's website or use package managers (apt), then disable Nouveau, install build tools, and run the installer. GPU Server Management also guides RHEL/CentOS setups. After installation, verify with `nvidia-smi` and enable persistence mode for production workloads.
What's the best way to set up a GPU server for LLM?
GPU Server Management provides end-to-end LLM server provisioning. Start by installing NVIDIA drivers and CUDA toolkit, then configure Docker with the NVIDIA Container Toolkit for containerized inference. Set up multi-GPU topology with NVLink if available, enable MIG partitioning for multi-tenant isolation, and deploy monitoring with DCGM exporter and Prometheus for real-time metrics tracking.
How do I configure MIG partitioning on A100/H100 GPUs?
GPU Server Management explains MIG setup for workload isolation. Enable MIG mode via nvidia-smi, partition GPUs into instances (1g, 2g, 3g profiles), and assign compute instances to applications. This allows multiple workloads to run safely in parallel. GPU Server Management covers instance configuration, memory allocation, and verification steps for production deployments.
How can I monitor GPU temperature and utilization in real-time?
GPU Server Management recommends using nvidia-smi for quick checks and DCGM exporter with Prometheus for production monitoring. nvidia-smi displays temperature, power draw, and utilization instantly. For persistent metrics, GPU Server Management guides deploying DCGM exporter to expose GPU health data to Prometheus, enabling dashboards and alerting on thermal throttling or performance degradation.
What should I do about GPU XID errors and thermal throttling?
GPU Server Management addresses common GPU failures and performance issues. XID errors indicate hardware or driver problems—check logs, update drivers, and verify cooling. For thermal throttling, GPU Server Management recommends adjusting power limits with nvidia-smi, improving airflow, and monitoring DCGM metrics. Persistent issues may require hardware replacement or workload redistribution across servers.
How do I optimize GPU memory for inference servers?
GPU Server Management covers memory optimization techniques for inference. Disable ECC if not required to free VRAM, use mixed precision (FP16/INT8), enable persistence mode to reduce allocation overhead, and configure memory pooling in inference frameworks. GPU Server Management also guides multi-GPU load balancing and MIG partitioning to maximize throughput while minimizing latency on production inference clusters.
SKILL.md
Rendered from the published skill. Quoted content, verbatim.
GPU Server Management
Provision, configure, and monitor NVIDIA GPU servers for AI inference and training workloads.
When to Use This Skill
Use this skill when: - Setting up a new GPU server for LLM inference or model training - Installing or upgrading NVIDIA drivers and CUDA toolkit - Configuring Docker with NVIDIA Container Toolkit for GPU workloads - Partitioning A100/H100 GPUs with MIG for multi-tenant workloads - Troubleshooting GPU errors, driver issues, or thermal throttling
Prerequisites
- Ubuntu 22.04 LTS (recommended) or RHEL 8/9
- NVIDIA GPU (A10G, A100, H100, RTX 4090, or L40S recommended)
- Root or sudo access
- Internet access for package downloads
Driver Installation (Ubuntu)
```bash
Remove old drivers
sudo apt purge -y 'nvidia' 'cuda' 'libcuda*' sudo apt autoremove -y
Add NVIDIA package repository
distribution=$(.
(truncated - see the full file via the links below)
File tree — 1 file
infrastructure/servers/gpu-server-management/SKILL.md
Let your AI agent find skills like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 56,283 agent skills by what they can do, searchable in plain language.
wish › “Set up NVIDIA GPU server infrastructure for AI workloads”
Give your agent the search over MCP, or paste the wish link into any chat. No install? Search from any chat →
Related skills
Deploy and operate production-grade GPU clusters in Kubernetes with built-in support for NVIDIA device plugins, MIG partitioning, and time-slicing. Monitor GPU health via DCGM metrics and Prometheus, configure autoscaling policies, and troubleshoot scheduling and driver issues across your AI infrastructure.
This skill guides you through selecting an LLM serving engine by analyzing hardware topology, workload profile, and operational constraints rather than benchmark headlines. It maps seven engines—vLLM, SGLang, TensorRT-LLM, TGI, llama.cpp, Ollama, and MLX—to their strengths across GPU clusters, edge devices, and single-user scenarios, then walks you through a five-step decision workflow to eliminate incompatible options and benchmark your top candidates.
This skill enables dynamic scaling of LLM inference workloads across Kubernetes clusters using KEDA and Prometheus metrics tied to GPU utilization and request queues. It covers vLLM deployment, queue-based job scaling with Redis, spot instance strategies, and cluster autoscaler configuration to handle traffic spikes while optimizing costs.
A structured decision guide for deploying and tuning vLLM in production environments. Covers the PagedAttention memory model, continuous batching, quantization tradeoffs, tensor/pipeline parallelism choices, and step-by-step operational workflows for diagnosing throughput, latency, and out-of-memory issues. Includes comparisons to alternative inference engines and guidance on when vLLM is the right fit.
This skill walks you through deploying and operating Redis across single-instance, Sentinel, and cluster modes. Learn core commands for strings, hashes, lists, and sorted sets; configure persistence with RDB snapshots and AOF; implement caching, rate limiting, pub/sub, and distributed locking patterns; and run Redis in Docker with monitoring.
Discover GGUF-compatible models on Hugging Face Hub and run them locally using llama.cpp across CPU, Mac Metal, CUDA, and ROCm hardware. The skill guides you through searching the Hub, selecting the right quantization, and launching models with llama-cli or llama-server, with fallback paths for custom file naming and conversion from Transformers weights when needed.
More skills arm-templates (MIT) · cis-benchmarks (MIT) · vllm-server (MIT) · model-serving-kubernetes (MIT)