QLoRA
Efficient fine-tuning method using 4-bit quantized base model with LoRA adapters.
About
QLoRA from the University of Washington is an efficient fine-tuning method that backpropagates gradients through a frozen 4-bit quantized model into LoRA adapters, letting a 65B parameter model be fine-tuned on a single 48 GB GPU while preserving 16-bit fine-tuning quality. It uses 4-bit NormalFloat quantization via bitsandbytes and integrates with Hugging Face PEFT and Transformers. Released under the MIT license.
Reviews (0)
Leave a Review
No reviews yet. Be the first to review!
Details
- Category
- Model Training & Fine-Tuning
- Price
- Free
- Platform
- Local/Desktop
- Difficulty
- Intermediate (3/5)
- License
- MIT
- Minimum VRAM
- 8 GB
- Added
- Apr 3, 2026
Related Tools
Efficient LLM quantization preserving important weight channels.
Video model fine-tuning toolkit by Hugging Face Diffusers team.
Low-code framework for building custom AI models by Predibase.
Library for training LLMs with reinforcement learning (RLHF, DPO, PPO).
All-in-one framework for fine-tuning 100+ LLMs with web UI.
No-code tool by Hugging Face for training ML models automatically.