gpu-server-management

GPU Server Management guides you through provisioning and configuring NVIDIA GPU infrastructure for LLM inference and model training. It covers driver and CUDA toolkit installation, Docker GPU integration, multi-GPU topology setup, and production monitoring with DCGM and Prometheus metrics.

GPU Server Management helps you provision and configure NVIDIA GPU servers for AI inference and training workloads.

AI-generated summary based on this skill's SKILL.md

★ 44 ⁮ 4 MIT updated 2026-05-22 by BagelHole

Install

BagelHole/DevOps-Security-Agent-Skills/gpu-server-management · repository language: Shell

Manual (git clone) CLI (skillfed)

git clone https://github.com/BagelHole/DevOps-Security-Agent-Skills
cp -r DevOps-Security-Agent-Skills/infrastructure/servers/gpu-server-management ~/.claude/skills/gpu-server-management

npx skillfed install BagelHole/DevOps-Security-Agent-Skills/gpu-server-management

Frequently asked questions

AI-generated answers based on this skill's SKILL.md and metadata

How do I install NVIDIA GPU drivers on Ubuntu?

GPU Server Management covers driver installation across distributions. On Ubuntu, download drivers from NVIDIA's website or use package managers (apt), then disable Nouveau, install build tools, and run the installer. GPU Server Management also guides RHEL/CentOS setups. After installation, verify with `nvidia-smi` and enable persistence mode for production workloads.

What's the best way to set up a GPU server for LLM?

GPU Server Management provides end-to-end LLM server provisioning. Start by installing NVIDIA drivers and CUDA toolkit, then configure Docker with the NVIDIA Container Toolkit for containerized inference. Set up multi-GPU topology with NVLink if available, enable MIG partitioning for multi-tenant isolation, and deploy monitoring with DCGM exporter and Prometheus for real-time metrics tracking.

How do I configure MIG partitioning on A100/H100 GPUs?

GPU Server Management explains MIG setup for workload isolation. Enable MIG mode via nvidia-smi, partition GPUs into instances (1g, 2g, 3g profiles), and assign compute instances to applications. This allows multiple workloads to run safely in parallel. GPU Server Management covers instance configuration, memory allocation, and verification steps for production deployments.

How can I monitor GPU temperature and utilization in real-time?

GPU Server Management recommends using nvidia-smi for quick checks and DCGM exporter with Prometheus for production monitoring. nvidia-smi displays temperature, power draw, and utilization instantly. For persistent metrics, GPU Server Management guides deploying DCGM exporter to expose GPU health data to Prometheus, enabling dashboards and alerting on thermal throttling or performance degradation.

What should I do about GPU XID errors and thermal throttling?

GPU Server Management addresses common GPU failures and performance issues. XID errors indicate hardware or driver problems—check logs, update drivers, and verify cooling. For thermal throttling, GPU Server Management recommends adjusting power limits with nvidia-smi, improving airflow, and monitoring DCGM metrics. Persistent issues may require hardware replacement or workload redistribution across servers.

How do I optimize GPU memory for inference servers?

GPU Server Management covers memory optimization techniques for inference. Disable ECC if not required to free VRAM, use mixed precision (FP16/INT8), enable persistence mode to reduce allocation overhead, and configure memory pooling in inference frameworks. GPU Server Management also guides multi-GPU load balancing and MIG partitioning to maximize throughput while minimizing latency on production inference clusters.

SKILL.md

rendered from the published skill — quoted content, verbatim