Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure Model is Correctly Loaded for Augmentation Purposes #77

Open
fabianlim opened this issue Aug 29, 2024 · 0 comments
Open

Ensure Model is Correctly Loaded for Augmentation Purposes #77

fabianlim opened this issue Aug 29, 2024 · 0 comments
Labels
question Further information is requested

Comments

@fabianlim
Copy link
Contributor

There is a need to ensure that the model has been correctly loaded before augmenting the model.

  • if there is no model_loader being called, we do not have visiblity into the flags passed into from_pretrained
  • so we will not know how the model is loaded
  • however, we need to have certain knowledge in some use cases.

Use Case 1: multipack needs to know if a linear or quadratic attention has been set.

  • it is safe now because padding_free in FMS-hf-tuning checks that flash_attn is set
  • however, in the future if multipack is independent of padding free plugin, then we need to do this extra check
@fabianlim fabianlim changed the title Ensure Model is Correct Loaded Depending on Plugin For Augmentation Purposes Ensure Model is Correctly Loaded for Augmentation Purposes Oct 11, 2024
@fabianlim fabianlim added the question Further information is requested label Nov 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

1 participant