llm-fine-tuning
Configure end-to-end LLM fine-tuning from single-GPU LoRA to multi-node distributed training. This skill covers QLoRA quantization for large models on consumer hardware, Axolotl framework setup, DeepSpeed ZeRO optimization, DPO alignment, and adapter merging for production deployment.
LLM Fine-Tuning Infrastructure sets up LoRA and QLoRA training on GPUs with distributed frameworks.
AI-generated summary based on this skill's SKILL.md
Install
BagelHole/DevOps-Security-Agent-Skills/llm-fine-tuning · repository language: Shell
git clone https://github.com/BagelHole/DevOps-Security-Agent-Skills
cp -r DevOps-Security-Agent-Skills/infrastructure/local-ai/llm-fine-tuning ~/.claude/skills/llm-fine-tuningnpx skillfed install BagelHole/DevOps-Security-Agent-Skills/llm-fine-tuningFrequently asked questions
AI-generated answers based on this skill's SKILL.md and metadata
How do I fine-tune Llama 3 on my own data with llm-fine-tuning?
llm-fine-tuning provides end-to-end infrastructure for fine-tuning Llama 3 and other open-source LLMs on custom datasets. Start by setting up LoRA or QLoRA adapters depending on your GPU memory constraints. For consumer hardware, QLoRA enables 4-bit quantization to fit large models on a single GPU. Use Axolotl framework configuration to define your dataset, model, and training hyperparameters, then launch training. After completion, merge adapters back into the base model for deployment.
What is QLoRA fine-tuning and how does it work on a single GPU?
llm-fine-tuning supports QLoRA, which combines 4-bit quantization with LoRA (Low-Rank Adaptation) to fine-tune large models like 70B parameter models on a single consumer GPU. QLoRA quantizes the base model to 4-bit precision while keeping trainable low-rank adapter weights in full precision. This dramatically reduces memory usage compared to full fine-tuning, making it practical for resource-constrained environments while maintaining competitive performance.
How do I configure distributed multi-GPU training with DeepSpeed or FSDP?
llm-fine-tuning enables distributed training through DeepSpeed ZeRO optimization and FSDP (Fully Sharded Data Parallel). Configure your Axolotl training config to specify DeepSpeed ZeRO stages (1, 2, or 3) or FSDP settings. ZeRO-3 shards model parameters, gradients, and optimizer states across GPUs for maximum memory efficiency on large models. FSDP offers similar functionality with PyTorch native support. Both approaches scale training across multiple GPUs or nodes for faster convergence on large datasets.
What alignment methods does llm-fine-tuning support for model training?
llm-fine-tuning implements RLHF (Reinforcement Learning from Human Feedback), DPO (Direct Preference Optimization), and instruction-tuning pipelines for model alignment. DPO provides a simpler alternative to RLHF by directly optimizing for preferred outputs without a separate reward model. Instruction-tuning formats your data as instruction-response pairs to improve task-following behavior. Configure your alignment approach in Axolotl with appropriate loss functions and data formatting.
How do I export and merge LoRA adapters for production serving with vLLM?
llm-fine-tuning provides utilities to merge trained LoRA adapters back into the base model for production deployment. After training completes, use the merge function to combine adapter weights with the original model. Export the merged model in standard formats compatible with vLLM inference engine. This unified model can then be deployed for low-latency serving without requiring separate adapter loading during inference.
What dataset preparation and efficiency optimizations are included?
llm-fine-tuning supports sample packing to combine multiple short sequences into single training examples, reducing padding overhead and improving GPU utilization. Flash Attention integration accelerates attention computation during training. Prepare datasets in standard formats (JSONL, Parquet, or HuggingFace datasets) and configure preprocessing in Axolotl. These optimizations together reduce training time and memory consumption while maintaining model quality.
SKILL.md
rendered from the published skill — quoted content, verbatim
LLM Fine-Tuning Infrastructure
Train and fine-tune open-source LLMs efficiently — from LoRA on a single GPU to distributed full fine-tuning across multi-node clusters.
When to Use This Skill
Use this skill when: - Fine-tuning an LLM on domain-specific data (legal, medical, code, support) - Running QLoRA to fine-tune 70B models on consumer GPUs - Setting up distributed training with DeepSpeed or FSDP - Exporting fine-tuned adapters for production serving - Implementing RLHF, DPO, or instruction tuning pipelines
Prerequisites
- NVIDIA GPU(s) with 24GB+ VRAM (RTX 4090 / A100 / H100)
- CUDA 12.1+ and
nvidia-smiworking - Python 3.10+ with
pip - Hugging Face account and
HF_TOKENfor gated models - 500GB+ disk for model weights and training data
Quick Start: QLoRA Fine-Tuning
```bash pip install transformers datasets trl peft bitsandbytes
(truncated - see the full file via the links below)
Read as markdown · JSON record · Browse the source repository
File tree — 1 file
infrastructure/local-ai/llm-fine-tuning/SKILL.md