Offline Hugging Face Model Inferencing without LM Studio, Llama.cpp, Ollama or Colab
484 بار بازدید -
2 ماه پیش
-
Trying to run a local
Trying to run a local Hugging Face model on an old 2012 MacBook Pro? In this video I will show you how to run an offline/local #HuggingFace base model. I will do this without using #LMStudio, #Ollama, #Llama.cpp, #JupiterNotebooks or #GoogleColab. Of course, the models will be no bigger than 7b parameters. This will be a foundation so that over time, we can add features like conversational loops, tts, stt, streaming or image generation.
But how do you do this if you have a computer that is incompatible with LM Studio or Llama.cpp?
Or what if you just want to explore using the Hugging Face base models without using the quantized models?
I have not found anything online that shows you how to do that. It may be there but I got tired of searching.
The closest that I've found is on each model's Hugging Face page, but the authors tend to leave out explanations or tutorials on how to do this.
I also see videos where people use Colab or Jupiter notebooks but don’t explain how to move the code from the notebook to a python environment and then code it in a python script.
I'll share what I discovered regarding offline model inferencing without LM Studio, Llama.cpp or Notebooks. This will get your local inference started. You can create the next iteration on your own...or wait until the next video on this. Learn with me as I learn.
00:00 Introduction
01:40 Create environment
03:42 Find base correct model
06:17 Get code
07:00 Run Code & dependencies
07:41 Data saving tip
12:58 Version 1.01
Please consider following the GitHub Page and like the repo for this video:
https://github.com/VideotronicMaker/O...
But how do you do this if you have a computer that is incompatible with LM Studio or Llama.cpp?
Or what if you just want to explore using the Hugging Face base models without using the quantized models?
I have not found anything online that shows you how to do that. It may be there but I got tired of searching.
The closest that I've found is on each model's Hugging Face page, but the authors tend to leave out explanations or tutorials on how to do this.
I also see videos where people use Colab or Jupiter notebooks but don’t explain how to move the code from the notebook to a python environment and then code it in a python script.
I'll share what I discovered regarding offline model inferencing without LM Studio, Llama.cpp or Notebooks. This will get your local inference started. You can create the next iteration on your own...or wait until the next video on this. Learn with me as I learn.
00:00 Introduction
01:40 Create environment
03:42 Find base correct model
06:17 Get code
07:00 Run Code & dependencies
07:41 Data saving tip
12:58 Version 1.01
Please consider following the GitHub Page and like the repo for this video:
https://github.com/VideotronicMaker/O...
2 ماه پیش
در تاریخ 1403/04/23 منتشر شده
است.
484
بـار بازدید شده