Top Ten Fine Tuning Tips

Trelis Research
Trelis Research
2 هزار بار بازدید - ماه قبل - ➡️ Get Life-time Access to
➡️ Get Life-time Access to the Complete Scripts (and future improvements): https://Trelis.com/ADVANCED-fine-tuning
➡️ One-click fine-tuning and LLM templates: https://github.com/TrelisResearch/one...
➡️ Newsletter: https://blog.Trelis.com
➡️ Resources/Support/Discord: https://Trelis.com/About

VIDEO RESOURCES:
- Slides: https://docs.google.com/presentation/...

TIMESTAMPS:
0:00 Top Ten Fine-tuning Tips
0:12 Tip 1: Start with a Small Model
0:49 Tip 2: Use LoRA or QLoRA
1:39 Tip 3: Create 10 manual questions
2:27 Tip 4: Create datasets manually
3:05 Tip 5: Start training with just 100 rows
3:38 Tip 6: Always create a validation data split
4:27 Tip 7: Start by only training on one GPU
5:44 Tip 8: Use weights and biases for logging
6:31 Scale up rows, tuning type, then model size
7:18 Tip 9: Consider unsupervised fine-tuning if you've lots of data
7:49 Tip 10: Use preference fine-tuning (ORPO)
9:21 Recap of the ten tips
10:54 Ten tips applied to multi-modal fine-tuning
11:31 Playlists to watch
12:59 Trelis repo overview
13:17 ADVANCED Fine-tuning repo (Trelis.com/ADVANCED-fine-tuning)
17:08 Training on completions only
18:10 ADVANCED fine-tuning repo CONTINUED
21:56 ADVANCED vision (Trelis.com/ADVANCED-vision)
22:35 ADVANCED inference (trelis.com/enterprise-server-api-and-inference-guide/)
24:21 ADVANCED transcription (trelis.com/ADVANCED-transcription)
24:39 Support + Resources (Trelis.com/About)
ماه قبل در تاریخ 1403/03/20 منتشر شده است.
2,072 بـار بازدید شده
... بیشتر