Fine-Tune Your Own Tiny-Llama on Custom Dataset

What to learn how to customize tiny llama on your own dataset? Here is how to do it.

🦾 Discord: Discord: discord
☕ Buy me a Coffee: https://ko-fi.com/promptengineering
|🔴 Patreon: Patreon: PromptEngineering
💼Consulting: https://calendly.com/engineerprompt/c...
📧 Business Contact: [email protected]
Become Member: http://tinyurl.com/y5h28s6h

💻 Pre-configured localGPT VM: https://bit.ly/localGPT (use Code: PromptEngineering for 50% off).  


LINKS:
TinyLlama Github: https://github.com/jzhang38/TinyLlama
TinyLlama Chat-Demo: http://tinyurl.com/e48uftef
TinyLlama Paper: https://arxiv.org/pdf/2401.02385.pdf
Google Colab: http://tinyurl.com/4eny9cvc
Blogpost: http://tinyurl.com/krskjumf
Learn about LoRA: Mixtral: Fine-tune Mixtral 8x7B (MoE) on Custo...
The Dataset: https://huggingface.co/datasets/burke...

TIMESTAMPS:
[00:00] Introduction
[01:07] Training dataset
[02:29] Setting Up the Code
[04:19] Formatting the Dataset
[06:49] Setting Up the Model and LoRAs
[09:58] Training the Model
[10:37] Merging the Model and LoRa Adapters
[11:31] Inference and Testing
[14:02] Conclusion and Future Prospects



All Interesting Videos:
Everything LangChain: LangChain

Everything LLM: Large Language Models

Everything Midjourney: MidJourney Tutorials

AI Image Generation: AI Image Generation Tutorials

• #science_technology
• #prompt_engineering
• #prompt_engineer
• #tiny_llama
• #tinyllama
• #how_to_fine_tune_llama
• #llm