# trl-training

> Claude/Codex skill by [@huggingface](https://omgskills.com/library/huggingface/index.md) · 19k stars · Updated 2026-09-11

Post-train LLMs with TRL (Transformers Reinforcement Learning) — SFT, DPO, GRPO, KTO, and reward-model training. Use when writing or debugging training code with the TRL Python API or the trl CLI.

## Install

```sh
git clone https://github.com/huggingface/trl /tmp/trl && ln -s /tmp/trl/skills/trl-training ~/.claude/skills/trl-training
```

## From README

TRL Each method pairs a Trainer class with a Config dataclass. Configs extend transformers.TrainingArguments, so all of its arguments work in any trainer config. | Trainer | Dataset type | |---|---| | SFTTrainer | language modeling or prompt-completion | | DPOTrainer | preference (chosen/rejected pairs) | | GRPOTrainer | prompt-only + reward function(s) | | DistillationTrainer | prompt-only + a teacher model (on-policy distillation) | | KTOTrainer | unpaired preference (per-sample bool label) | | RewardTrainer | preference (chosen/rejected pairs); trains a scalar reward model, not a policy | Many more trainers (OnlineDPO, ORPO, CPO, GKD, …) live in trl.experimental with unstable APIs: https://huggingface.co/docs/trl/experimentaloverview Pass model as a string and route loading kwargs through modelinitkwargs (e.g. {"dtype": "bfloat16", "attnimplementation": "kernels-community/flash-attn2"}) instead of calling frompretrained yourself.

## Links

- HTML page: https://omgskills.com/skills/huggingface/trl/trl-training/
- GitHub: https://github.com/huggingface/trl
- Author: [@huggingface](https://omgskills.com/library/huggingface/index.md)

## More skills by this author

- [add-or-fix-type-checking](https://omgskills.com/skills/huggingface/transformers/add-or-fix-type-checking/index.md) — Fixes broken typing checks detected by ty, make typing, or make check-repo.
   - [@huggingface](https://omgskills.com/library/huggingface/index.md) · 165k stars
- [train-sentence-transformers](https://omgskills.com/skills/huggingface/sentence-transformers/train-sentence-transformers/index.md) — Train or fine-tune sentence-transformers models across \`SentenceTransformer\` (bi-encoder, dense or static embedding...
   - [@huggingface](https://omgskills.com/library/huggingface/index.md) · 19k stars
- [sync-models](https://omgskills.com/skills/huggingface/chat-ui/add-model-descriptions/index.md) — Sync chat-ui's model config with the HuggingFace router — add descriptions for new models, flag reasoning-capable ones,...
   - [@huggingface](https://omgskills.com/library/huggingface/index.md) · 11k stars