trl-training
Post-train LLMs with TRL (Transformers Reinforcement Learning) — SFT, DPO, GRPO, KTO, and reward-model training. Use when writing or debugging training code with the TRL Python API or the trl CLI.
Install
With your agent
Install OMGSkills skill "huggingface/trl:trl-training" into this agent's normal skills directory: call the OMGSkills MCP get_install_plan tool for your agent and run the script it returns (it installs only the pinned commit and verifies SKILL.md), and stop if MCP or the pin is unavailable. Do not overwrite or change existing skills without asking me.
Manual command
git clone https://github.com/huggingface/trl /tmp/trl && ln -s /tmp/trl/skills/trl-training ~/.claude/skills/trl-trainingFrom README
TRL Each method pairs a Trainer class with a Config dataclass. Configs extend transformers.TrainingArguments, so all of its arguments work in any trainer config. | Trainer | Dataset type | |---|---| | SFTTrainer | language modeling or prompt-completion | | DPOTrainer | preference (chosen/rejected pairs) | | GRPOTrainer | prompt-only + reward function(s) | | DistillationTrainer | prompt-only + a teacher model (on-policy distillation) | | KTOTrainer | unpaired preference (per-sample bool label) | | RewardTrainer | preference (chosen/rejected pairs); trains a scalar reward model, not a policy | Many more trainers (OnlineDPO, ORPO, CPO, GKD, …) live in trl.experimental with unstable APIs: https://huggingface.co/docs/trl/experimentaloverview Pass model as a string and route loading kwargs through modelinitkwargs (e.g. {"dtype": "bfloat16", "attnimplementation": "kernels-community/flash-attn2"}) instead of calling frompretrained yourself.
More skills
add-or-fix-type-checking
Fixes broken typing checks detected by ty, make typing, or make check-repo.
train-sentence-transformers
Train or fine-tune sentence-transformers models across `SentenceTransformer` (bi-encoder, dense or static embedding...
hf-cli
Hugging Face Hub CLI (`hf`) for downloading, uploading, and managing models, datasets, spaces, buckets, repos, papers,...