ToyGPT, inspired by Andrej Karpathy’s GPT from scratch, creates a toy generative pre-trained transformer at its most basic level using a simple bigram language model with attention to help educate on the basics of creating an LLM from scratch.
education machine-learning ai neural-network toy neural-networks educational gpt stem bigram-model aritificial-intelligence generative-pretrained-transformer generative-ai generative-pre-trained-transformer generative-pretrained-transformers toygpt
-
Updated
Nov 28, 2024 - Jupyter Notebook