Article Source

• Everything you need to know about Fine-tuning and Merging LLMs

Everything you need to know about Fine-tuning and Merging LLMs

• Recorded live in San Francisco at the AI Engineer World’s Fair.

Abstract

Fine-tuning LLMs is a fundamental technique for companies to customize models for their specific needs. In this talk, we will cover when fine-tuning is appropriate, popular libraries for efficient fine-tuning, and key techniques. We will explore both supervised fine-tuning (LoRA, QLoRA) and preference alignment (PPO, DPO, KTO) methods.