We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
🚀 Feature Description
Faster training without schedules https://github.com/facebookresearch/schedule_free
Solution integrate optimizer option like AdamWScheduleFree see https://github.com/facebookresearch/schedule_free https://arxiv.org/pdf/2405.15682
AdamWScheduleFree
Additional context Potentially faster convergence, for future pretrains and finetunes. I have no data to share, didn't try it myself.
The text was updated successfully, but these errors were encountered:
No branches or pull requests
🚀 Feature Description
Faster training without schedules
https://github.com/facebookresearch/schedule_free
Solution
integrate optimizer option like
AdamWScheduleFreesee https://github.com/facebookresearch/schedule_free
https://arxiv.org/pdf/2405.15682
Additional context
Potentially faster convergence, for future pretrains and finetunes. I have no data to share, didn't try it myself.
The text was updated successfully, but these errors were encountered: