llm-fine-tuning
Configure end-to-end LLM fine-tuning from single-GPU LoRA to multi-node distributed training. This skill covers QLoRA quantization for large models on consumer hardware, Axolotl framework setup, DeepSpeed ZeRO optimization, DPO alignment, and adapter merging for production deployment.
LLM Fine-Tuning Infrastructure sets up LoRA and QLoRA training on GPUs with distributed frameworks.
AI-generated summary based on this skill's SKILL.md
Decision gist · record as of 2026-05-22
LLM Fine-Tuning Infrastructure sets up LoRA and QLoRA training on GPUs with distributed frameworks. Configure end-to-end LLM fine-tuning from single-GPU LoRA to multi-node distributed training. This skill covers QLoRA quantization for large models on consumer hardware, Axolotl framework setup, DeepSpeed ZeRO optimization, DPO alignment, and adapter merging for production deployment.
Use it when
- llm-fine-tuning supports QLoRA.
- llm-fine-tuning enables distributed training through DeepSpeed ZeRO optimization and FSDP (Fully Sharded Data Parallel).
Verify before relying
Read SKILL.md below before installing (1 file). Open directory: indexed for reading, not audited.
Similar skills
Install
BagelHole/DevOps-Security-Agent-Skills/llm-fine-tuning · repository language: Shell
Open directory. Skills are indexed for reading, not audited. Review a skill's body before installing it.
Frequently asked questions
AI-generated answers based on this skill's SKILL.md and metadata
How do I fine-tune Llama 3 on my own data with llm-fine-tuning?
llm-fine-tuning provides end-to-end infrastructure for fine-tuning Llama 3 and other open-source LLMs on custom datasets. Start by setting up LoRA or QLoRA adapters depending on your GPU memory constraints. For consumer hardware, QLoRA enables 4-bit quantization to fit large models on a single GPU. Use Axolotl framework configuration to define your dataset, model, and training hyperparameters, then launch training. After completion, merge adapters back into the base model for deployment.
What is QLoRA fine-tuning and how does it work on a single GPU?
llm-fine-tuning supports QLoRA, which combines 4-bit quantization with LoRA (Low-Rank Adaptation) to fine-tune large models like 70B parameter models on a single consumer GPU. QLoRA quantizes the base model to 4-bit precision while keeping trainable low-rank adapter weights in full precision. This dramatically reduces memory usage compared to full fine-tuning, making it practical for resource-constrained environments while maintaining competitive performance.
How do I configure distributed multi-GPU training with DeepSpeed or FSDP?
llm-fine-tuning enables distributed training through DeepSpeed ZeRO optimization and FSDP (Fully Sharded Data Parallel). Configure your Axolotl training config to specify DeepSpeed ZeRO stages (1, 2, or 3) or FSDP settings. ZeRO-3 shards model parameters, gradients, and optimizer states across GPUs for maximum memory efficiency on large models. FSDP offers similar functionality with PyTorch native support. Both approaches scale training across multiple GPUs or nodes for faster convergence on large datasets.
What alignment methods does llm-fine-tuning support for model training?
llm-fine-tuning implements RLHF (Reinforcement Learning from Human Feedback), DPO (Direct Preference Optimization), and instruction-tuning pipelines for model alignment. DPO provides a simpler alternative to RLHF by directly optimizing for preferred outputs without a separate reward model. Instruction-tuning formats your data as instruction-response pairs to improve task-following behavior. Configure your alignment approach in Axolotl with appropriate loss functions and data formatting.
How do I export and merge LoRA adapters for production serving with vLLM?
llm-fine-tuning provides utilities to merge trained LoRA adapters back into the base model for production deployment. After training completes, use the merge function to combine adapter weights with the original model. Export the merged model in standard formats compatible with vLLM inference engine. This unified model can then be deployed for low-latency serving without requiring separate adapter loading during inference.
What dataset preparation and efficiency optimizations are included?
llm-fine-tuning supports sample packing to combine multiple short sequences into single training examples, reducing padding overhead and improving GPU utilization. Flash Attention integration accelerates attention computation during training. Prepare datasets in standard formats (JSONL, Parquet, or HuggingFace datasets) and configure preprocessing in Axolotl. These optimizations together reduce training time and memory consumption while maintaining model quality.
SKILL.md
Rendered from the published skill. Quoted content, verbatim.
LLM Fine-Tuning Infrastructure
Train and fine-tune open-source LLMs efficiently — from LoRA on a single GPU to distributed full fine-tuning across multi-node clusters.
When to Use This Skill
Use this skill when: - Fine-tuning an LLM on domain-specific data (legal, medical, code, support) - Running QLoRA to fine-tune 70B models on consumer GPUs - Setting up distributed training with DeepSpeed or FSDP - Exporting fine-tuned adapters for production serving - Implementing RLHF, DPO, or instruction tuning pipelines
Prerequisites
- NVIDIA GPU(s) with 24GB+ VRAM (RTX 4090 / A100 / H100)
- CUDA 12.1+ and
nvidia-smiworking - Python 3.10+ with
pip - Hugging Face account and
HF_TOKENfor gated models - 500GB+ disk for model weights and training data
Quick Start: QLoRA Fine-Tuning
```bash pip install transformers datasets trl peft bitsandbytes
(truncated - see the full file via the links below)
File tree — 1 file
infrastructure/local-ai/llm-fine-tuning/SKILL.md
Let your AI agent find skills like this
Example. Real query, live index.
You found this page by searching. An agent finds it by wishing: SkillFed indexes 56,283 agent skills by what they can do, searchable in plain language.
wish › “Set up infrastructure to fine-tune open-source LLMs with LoRA/QLoRA on available GPUs”
Give your agent the search over MCP, or paste the wish link into any chat. No install? Search from any chat →
Related skills
Unsloth accelerates LLM fine-tuning on consumer and datacenter GPUs through optimized LoRA and QLoRA training, cutting both speed and memory requirements dramatically. It handles supervised fine-tuning, reinforcement learning with GRPO, vision model adaptation, and TTS training across 300+ model architectures, with direct export to GGUF for deployment on Ollama and llama.cpp.
This skill teaches post-training techniques for aligning language models to human preferences. It covers supervised fine-tuning, direct preference optimization (DPO), and online reinforcement learning methods like RLOO and GRPO, with complete workflows and practical examples.
LLMOps Platform Engineering teaches you to architect internal LLM platforms that balance rapid experimentation with production safety. You'll implement model promotion pipelines with automated quality and safety gates, canary validation, and rollback capabilities, plus set up A/B testing infrastructure and observability across Kubernetes and cloud inference.
vllm-server guides you through deploying and configuring vLLM—a high-performance open-source LLM serving engine—for production workloads. Set up continuous batching, multi-GPU tensor parallelism, model quantization, and OpenAI-compatible API endpoints to serve models like Llama and Mistral at scale. Includes Docker deployment, performance tuning, monitoring with Prometheus metrics, and troubleshooting for common VRAM and throughput issues.
This skill teaches you to apply reinforcement learning techniques for aligning language models with human preferences. It covers supervised fine-tuning for instruction following, direct preference optimization for preference alignment, PPO and GRPO for reward-based training, and reward model development—all integrated with HuggingFace Transformers.
Fine-tuning with TRL provides post-training methods to align language models with human preferences through multiple approaches. Train models on instruction data with SFT, optimize for preference alignment via DPO without a separate reward model, or run full RLHF pipelines combining supervised fine-tuning, reward modeling, and PPO optimization. GRPO offers memory-efficient online reinforcement learning for resource-constrained setups.
More skills llm-inference-scaling (MIT) · huggingface-llm-trainer (Apache-2.0)