llamafactory
LlamaFactory streamlines LLM fine-tuning through YAML-driven configs, supporting LoRA, QLoRA, and full-parameter training methods like SFT and DPO. Register datasets, train your model, merge adapters, and export for standalone deployment or API serving. The skill guides you through GPU requirements, base model selection, and inference setup.
LlamaFactory fine-tunes open-weight LLMs via YAML configs, supporting LoRA, QLoRA, and full-parameter methods.
AI-generated summary based on this skill's SKILL.md
Install
Prism-Shadow/penguin-harness/llamafactory · repository language: TypeScript
git clone https://github.com/Prism-Shadow/penguin-harness
cp -r penguin-harness/packages/skills/skills/llamafactory ~/.claude/skills/llamafactorynpx skillfed install Prism-Shadow/penguin-harness/llamafactoryFrequently asked questions
AI-generated answers based on this skill's SKILL.md and metadata
How do I fine-tune a model with LlamaFactory?
LlamaFactory streamlines LLM fine-tuning through YAML-driven configs. Define your base model, dataset, training method (SFT, DPO, or LoRA), and hardware specs in a YAML file, then run the CLI. LlamaFactory handles dataset registration, parameter initialization, and training loops. You can choose full-parameter tuning or memory-efficient LoRA/QLoRA adapters depending on your GPU resources.
What is the LlamaFactory LoRA adapter setup process?
LlamaFactory's LoRA adapter setup involves configuring rank, alpha, and target modules in your YAML training config. Register your dataset in the framework's format (Alpaca, ShareGPT, or custom), specify the base model, set adapter output directory, and launch training. LlamaFactory applies low-rank decomposition to model weights, keeping memory footprint minimal while maintaining adaptation quality.
How do I merge LoRA adapters and export the model?
LlamaFactory provides merge and export utilities to combine trained LoRA adapters with base model weights into a standalone model. Use the export command specifying adapter paths and output format. The merged model can then be deployed via vLLM, Ollama, or other inference servers without requiring adapter files, simplifying production serving.
Can I use LlamaFactory's web UI for no-code training?
Yes, LlamaFactory includes a web UI for no-code model training and parameter tuning. The interface guides you through dataset upload, base model selection, training method choice, and hyperparameter adjustment without writing YAML manually. This makes LlamaFactory accessible to users without CLI experience while maintaining full training flexibility.
What GPU memory does LlamaFactory require for fine-tuning?
LlamaFactory's GPU requirements depend on your training method. Full-parameter training demands substantial VRAM; LoRA and QLoRA drastically reduce memory needs, enabling fine-tuning on consumer GPUs. The framework provides memory optimization guides and supports multi-GPU setups. Your actual requirements vary by model size, batch size, and sequence length configured in YAML.
How do I deploy a fine-tuned model after training?
After training, LlamaFactory supports deployment via inference APIs or interactive chat interfaces. Export your merged model, then serve it using vLLM, Ollama, or LlamaFactory's built-in inference server. Configure API endpoints, batch settings, and quantization as needed. The exported model runs standalone without training infrastructure, ready for production workloads.
SKILL.md
rendered from the published skill — quoted content, verbatim
LlamaFactory Fine-Tuning
LlamaFactory fine-tunes open-weight LLMs (LoRA/QLoRA and full-parameter; SFT, DPO and more) through the llamafactory-cli command driven by YAML configs.
Before you start
If the user's message only invokes this skill (e.g. "use llamafactory skill") without a concrete request, ask the user what they want to fine-tune. Do not run any command until the goal is clear.
Confirm before training:
- GPU memory (
nvidia-smi) — it bounds the model size and method; LoRA needs far less than full fine-tuning. - The base model: a Hugging Face id or a local path.
- The dataset: where it lives and which format it is in.
- The goal: SFT with LoRA is the usual starting point.
Install
```bash git clone --depth 1 https://github.com/hiyouga/LlamaFactory.git cd LlamaFactory pip install -e . pip install -r requirements/metrics.txt # optional: evaluation
(truncated - see the full file via the links below)
Read as markdown · JSON record · Browse the source repository
File tree — 2 files
packages/skills/skills/llamafactory/SKILL.md
packages/skills/skills/llamafactory/icon.svg