When, Why and How to Fine-Tune LLMs for Enterprise Applications

Опубликовано: 15 Июль 2024
на канале: Snorkel AI

354

Many enterprise applications call for customizing large language models (LLMs), but not all of them. Knowing when to focus on adapting your LLM and when to focus on other parts of your LLM applications poses challenges.

Snorkel AI researcher Tom Walshe walks through why you should align LLMs, when to do it in your development roadmap, and several key techniques for aligning your model for your domain and tasks.

This presentation is excerpted from a longer webinar. See the rest of it here: • How to Fine-Tune LLMs to Perform Specializ...

See more videos on retrieval augmented generation (RAG) applications here: • RAG: Building enterprise ready retrieval-a...

Timestamps:

00:00 Introduction
00:21 Fine-Tuning Overview
00:48 When and Why to Fine-Tune
01:10 Importance of Fine-Tuning for Enterprise Use Cases
01:53 Advantages of Fine-Tuning
02:50 When to Fine-Tune an LLM
04:01 Error Modes in LLMs
05:01 Techniques for Fine-Tuning
05:18 Data Considerations for Fine-Tuning
06:41 Supervised Fine-Tuning
08:28 Reinforcement Learning from Human Feedback
10:40 Direct Preference Optimization
11:45 Recent Optimization Techniques
12:50 Lessons Learned from Alpaca Reval
13:57 Data Requirements for Fine-Tuning
16:40 Lessons Learned in Data Development

#largelanguagemodels #enterpriseai #alignment