TRL

TRL is a library from Hugging Face for post-training foundation models using techniques such as Supervised Fine-Tuning, Direct Preference Optimization, and Group Relative Policy Optimization. It is built on the Transformers ecosystem and supports many model architectures and modalities. The library scales from a single GPU to multi-node clusters and integrates with PEFT and quantization for memory-efficient training.