GPT3 Tutorial: How to Download And Use GPT3(GPT Neo)

In this video, I go over how to download and run the open-source implementation of GPT3, called GPT Neo.  This model is 2.7 billion parameters, which is the same size as GPT3 Ada.  The results are very good and are a large improvement over GPT-2.  I am excited to play around with this model more and for the future of even larger NLP models.

Notebook https://github.com/mallorbc/GPTNeo_no...
GPT Neo github https://github.com/EleutherAI/gpt-neo (use the first release tag)
GPT Neo HuggingFace docs https://huggingface.co/transformers/m...
A useful article about transformer parameters https://huggingface.co/blog/how-to-ge...

00:00 - GPT3 Background
01:07  - GPT3 Interview
02:06  - GPT Neo Github
03:14  - GPT Neo HuggingFace
03:52  - Setting up Anaconda and Jupyter
05:05  - Starting the Jupyter notebook
06:14  - Installing dependencies in the notebook
06:58  - Importing needed dependencies
07:33  - Selecting what GPT model to use
08:45  - Checking our computer hardware
09:36  - Loading the tokenizer
09:55  - Giving our inputs
10:39  - Generating the tokens with the model
11:42  - Decoding and reading the result
13:22  - Reflections on Transformers
14:26  - Outro questions future work

• #science_technology
• #ai
• #machine_learning
• #engineering
• #gpt
• #gpt3
• #gpt2
• #nlp
• #tutorial
• #software_engineering
• #how_to
• #text_generation