Welcome to BabyJoey, a streamlined Australian language model designed for efficient performance. This document provides a detailed overview of the project structure to help you get started quickly.
Comprehensive documentation for the project, explaining the purpose, setup instructions, and usage of BabyJoey.
BabyJoey is a lightweight language model inspired by GPT-1, featuring 115 million parameters and designed for tasks that require a balance of efficiency and performance. It's perfect for educational purposes, research, and small-scale applications.
- Clone the Repository:
git clone https://github.com/yourusername/babyjoey.git cd babyjoey
- Install Dependencies:
pip install -r requirements.txt
- Configure the Model:
Modify
config.py
to set your desired hyperparameters and configurations. - Prepare Data:
Ensure your datasets are in place and correctly referenced in
data/dataloader.py
.
- Training the Model:
bash scripts/train.sh
- Evaluating the Model:
bash scripts/evaluate.sh
The entry point of BabyJoey. This script parses arguments, sets up configurations, and invokes the training loop.
Lists the dependencies required to run BabyJoey, which can be installed using pip
.
- model.py: Defines the BabyJoey model architecture, including the 12 decoder blocks.
- dataloader.py: Manages data loading logic, including data preprocessing, batching, and any necessary data augmentation.
- trainer.py: Contains the training loop, validation logic, and evaluation metrics to train and assess BabyJoey.
- utils.py: Provides utility functions used throughout the project, such as logging, saving/loading models, and calculating metrics.
- config.py: Contains configuration settings and hyperparameters for easy management and modification.
- config.yaml: This is where you change the parameters, and it updates the main config file
- train.sh: A shell script to start the training process.
- evaluate.sh: A shell script to initiate the evaluation process.
Stores log files generated during training and evaluation for debugging and monitoring purposes.
Stores model checkpoints saved during training to allow for resuming or fine-tuning.
We hope you enjoy working with BabyJoey! If you have any questions, feel free to reach out to our support team or check the detailed documentation in the README.md
. Happy coding! 🚀