Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add dense Hessian Laplace #99

Open
SamDuffield opened this issue Jun 6, 2024 · 1 comment
Open

Add dense Hessian Laplace #99

SamDuffield opened this issue Jun 6, 2024 · 1 comment
Labels
good first issue Good for newcomers new method New algorithm

Comments

@SamDuffield
Copy link
Contributor

Might be worth adding a laplace.dense_hessian method since the Hessian is easy to compute with torch.func.hessian.

The main problem with this is that the Hessian is not guaranteed to be positive semidefinite, however there may be cases where it is i.e. when we are sure we have optimized exactly to the MAP or simple models. We should add an epsilon parameter though that adds a nugget epsilon * torch.eye(dim) to the total Hessian that will ensure positive definiteness for epilson large enough.

I think we should should not add a laplace.diag_hessian method since it is not easy to compute in contrast to the Fisher methods.

@SamDuffield SamDuffield added good first issue Good for newcomers new method New algorithm labels Jun 6, 2024
@jcqcai
Copy link
Contributor

jcqcai commented Oct 7, 2024

Hi, Sam. I just wanted to let you know that I have submitted a PR for this: #112

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
good first issue Good for newcomers new method New algorithm
Projects
None yet
Development

No branches or pull requests

2 participants