Skip to content

JaredLevi18/Mini-GPT-model

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

4 Commits
 
 
 
 
 
 

Repository files navigation

Mini-GPT-model

Hi, I present a model that applies the transformer technique, following the paper "Attention is all you need". The model you'll see was trained using Google Colab's platform. The model was trained for 50 minutes in a GPU, one of the challenges was to make it run, in a good time, originally the model had 10 million parameters, but due to the big scale of the model, and the time it would take to train it, I had to reduce the model's size to make it have only more that 1 million parameters roughly speaking. The input is a text from Shakespeare, and the output is a text that imitates his sytle of writing, or at least it tries, again the loss can be improved but with the cost of time, witch I currently don't have. If you have time and a good GPU, I hightly suggest that you run the model and share the results with a pull request. Well that's it, have a nice day.

About

Mini GPT model, based on Chat GPT.

Resources

Stars

Watchers

Forks

Packages

No packages published