Other types of variable fidelity surrogate

[archermarx]

It would be nice to add a few options for variable fidelity surrogate. The current one uses an additive bridge function, but there are other options. Multiplicative bridge functions would be low hanging fruit. There's also linear-regression multi-fidelity surrogates and multi-fidelity Kriging. I think i can tackle this, but I thought I'd make an issue to solicit ideas and feedback.

[ChrisRackauckas]

Can you post some articles on this?

(BTW, you might want to put something in for https://sciml.ai/dev/#sciml_summer_fellowship)

[archermarx]

Yeah sure, here's an article on the Linear Regression Mutli-Fidelity Surrogate (LR-MFS):
https://arxiv.org/ftp/arxiv/papers/1705/1705.02956.pdf

A generalized hybrid bridge function (discusses multiplicative and additive as well)
https://www.sciencedirect.com/science/article/pii/S127096381200017X?via%3Dihub

Multi-fidelity co-kriging:
https://rss.onlinelibrary.wiley.com/doi/10.1111/1467-9868.00294

[archermarx]

w/r/t the fellowship, I don't think I'd have time to commit to this full-time. I'm in my fourth year of PhD and am ramping up to some publications this fall. I'm happy to contribute to the surrogates package, though. My work for an upcoming conference paper involves multi-fidelity surrogate modeling and it'd be nice to contribute rather than roll my own and have it languish.

[archermarx]

I'm currently using a sort of homebrew variable fidelity surrogate using Stheno/AbstractGPs. What's nice is that using those, if both your low fidelity model and (additive) bridge function are Gaussian processes, then your corrected high fidelity surrogate is also Gaussian, so you can still easily predict the standard error as well as posterior mean at every point, allowing you to use normal Bayesian optimization methods (EGO/LCB, etc). We should be able to make something similar work with our Kriging models as well, would need to add a method to std_error_at_point for VariableFidelitySurrogate where both models are Kriging surrogates.

For multiplicative surrogates of the form there are a few tricky points. The product of two Gaussian processes is non-Gaussian, so predicting the standard error becomes a bit more difficult. Additionally, if the low-fidelity surrogate is near zero, we get divide-by-zero issues when constructing the bridge function $\phi = y_{hf} / y_{lf}$.

[ChrisRackauckas]

We should be able to make something similar work with our Kriging models as well, would need to add a method to std_error_at_point for VariableFidelitySurrogate where both models are Kriging surrogates.

That sounds reasonable enough to me.