New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Remove smaller limit for legacy bfloat16 serialization #505

Draft

borzunov wants to merge 1 commit into main from payload-size

Collaborator

borzunov commented Sep 5, 2023 •

edited

Loading

Revert #251 since it's not needed after #311. This may improve fine-tuning efficiency for medium-sized batches.

TODO:

Test it with increasingly larger batches. Watch that we switch from rpc_forward to rpc_forward_stream (can be distinguished using server logs) without errors.


          Remove smaller limit for legacy bfloat16 serialization

b945e38

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet