Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

make coding scores based on unit tests #3

Open
rmminusrslash opened this issue Aug 15, 2023 · 0 comments
Open

make coding scores based on unit tests #3

rmminusrslash opened this issue Aug 15, 2023 · 0 comments

Comments

@rmminusrslash
Copy link

Hey,

great initiative to track local llms!

Would you be open to talking about how the scores are created?

  • I created some gpt-4 scores in a project in the past and found them not good enough (they would fluctuate based on input sentences with the same meanings, scores somewhat too arbitrary, different days would give different scores for the same input). At least you should pin the gpt-4 version so you have better control when they roll updates to gpt-4

  • For code one could add unit tests to check the created functions

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant