You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
There is a need to ensure that the model has been correctly loaded before augmenting the model.
if there is no model_loader being called, we do not have visiblity into the flags passed into from_pretrained
so we will not know how the model is loaded
however, we need to have certain knowledge in some use cases.
Use Case 1: multipack needs to know if a linear or quadratic attention has been set.
it is safe now because padding_free in FMS-hf-tuning checks that flash_attn is set
however, in the future if multipack is independent of padding free plugin, then we need to do this extra check
The text was updated successfully, but these errors were encountered:
fabianlim
changed the title
Ensure Model is Correct Loaded Depending on Plugin For Augmentation Purposes
Ensure Model is Correctly Loaded for Augmentation Purposes
Oct 11, 2024
There is a need to ensure that the model has been correctly loaded before augmenting the model.
from_pretrained
Use Case 1: multipack needs to know if a linear or quadratic attention has been set.
flash_attn
is setmultipack
is independent of padding free plugin, then we need to do this extra checkThe text was updated successfully, but these errors were encountered: