trl-training
Post-train LLMs with TRL (Transformers Reinforcement Learning) — SFT, DPO, GRPO, KTO, and reward-model training. Use when writing or debugging training code with the TRL Python API or the trl CLI.
Install
git clone https://github.com/huggingface/trl /tmp/trl && ln -s /tmp/trl/skills/trl-training ~/.claude/skills/trl-trainingFrom README
TRL Each method pairs a Trainer class with a Config dataclass. Configs extend transformers.TrainingArguments, so all of its arguments work in any trainer config. | Trainer | Dataset type | |---|---| | SFTTrainer | language modeling or prompt-completion | | DPOTrainer | preference (chosen/rejected pairs) | | GRPOTrainer | prompt-only + reward function(s) | | DistillationTrainer | prompt-only + a teacher model (on-policy distillation) | | KTOTrainer | unpaired preference (per-sample bool label) | | RewardTrainer | preference (chosen/rejected pairs); trains a scalar reward model, not a policy | Many more trainers (OnlineDPO, ORPO, CPO, GKD, …) live in trl.experimental with unstable APIs: https://huggingface.co/docs/trl/experimentaloverview Pass model as a string and route loading kwargs through modelinitkwargs (e.g. {"dtype": "bfloat16", "attnimplementation": "kernels-community/flash-attn2"}) instead of calling frompretrained yourself.
More skills
add-or-fix-type-checking
Fixes broken typing checks detected by ty, make typing, or make check-repo.
train-sentence-transformers
Train or fine-tune sentence-transformers models across `SentenceTransformer` (bi-encoder, dense or static embedding...
sync-models
Sync chat-ui's model config with the HuggingFace router — add descriptions for new models, flag reasoning-capable ones,...