Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Seed for Adapter Initialization? #653

Open
Migabaj opened this issue Feb 23, 2024 · 2 comments · May be fixed by #786
Open

Seed for Adapter Initialization? #653

Migabaj opened this issue Feb 23, 2024 · 2 comments · May be fixed by #786
Assignees
Labels
enhancement New feature or request

Comments

@Migabaj
Copy link

Migabaj commented Feb 23, 2024

I've been trying to train multiple instances of a model with adapters and I would like it so that the adapters initialize with the exact same weights if they have the same seed.

I tried to find whether it's possible, but it seems like it isn't. Is there any way to do that?

@Migabaj Migabaj added the enhancement New feature or request label Feb 23, 2024
@TimoImhof TimoImhof self-assigned this Jan 24, 2025
@TimoImhof
Copy link
Contributor

Hi @Migabaj,

Sorry for getting back to you just now; I know this issue has been open for a while now. If I understand you correctly, you were interested in the possibility of initializing adapters with the same weights for one or multiple models, correct?

I drafted PR #786, which introduces an additional parameter for resetting the seed before every weight initialization to a specified value, which should lead to the outcome you desired.

I don't know if this is still relevant to you, but I thought it might still be an interesting feature to have when conducting research and experiments in general.

@Migabaj
Copy link
Author

Migabaj commented Jan 27, 2025

@TimoImhof thank you! I have finished working on the project that I needed this for, but I am planning on using AdapterHub in the near future, so this is still helpful :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants