Fine-tuning Whisper to learn my Chinese dialect (Teochew)

In this video, we train a speech recognition model for the Teochew language, also known as Chaozhou Dialect (潮州话). Teochew, spoken by 10 million people in Southern China, is part of the Min Nan language family and is distantly related to Mandarin and Cantonese. We set up a data pipeline and fine-tune OpenAI's Whisper to understand Teochew, using transfer learning from Mandarin and Cantonese. Check out how we inspect the training using TensorBoard, evaluate model outputs with Streamlit and Gradio, and learn about the linguistics of Teochew.

The model is open source and available: https://huggingface.co/efficient-nlp/...

0:00 - Intro
0:35 - Basics of Teochew language
4:37 - Data pipeline
9:19 - Whisper model architecture
10:53 - Multitask training format
12:24 - Fine-tuning Whisper
15:52 - Tensorboard visualization
17:48 - Data inspection tool
19:21 - Evaluation and results
22:23 - Comparison with other languages
23:43 - Easy and hard cases
24:58 - Demo sentence 1
26:25 - Demo sentence 2