acerbilab
Fail the test of bads('test') - missing optimization toolbox #6
-
|
The output looks like following: Test with noisy function (noisy sphere). Optimization terminated: reached maximum number of function evaluations OPTIONS.MaxFunEvals. Final value (not-noisy): 1.550 (true value: 0) with 100 fun evals. Test with heteroskedastic, specified noisy function (noisy sphere). Optimization terminated: reached maximum number of function evaluations OPTIONS.MaxFunEvals. Final value (not-noisy): 3.011 (true value: 0) with 200 fun evals. ===========================================================================
|
Beta Was this translation helpful? Give feedback.
Replies: 2 comments 5 replies
-
|
Have you tried multiple times? Does it keep failing? I just ran it now with no issues. |
Beta Was this translation helpful? Give feedback.
-
|
Yes, I tried to run it multiple times and it always fails. I also try both using the zip file and the git clone. However, both of them failed. Here is a detailed illustration of the actions I took. I first clone the repository, then in a Matlab terminal. I type |
Beta Was this translation helpful? Give feedback.
-
|
When you try to run it again, do you get different runs, even though they always fail? |
Beta Was this translation helpful? Give feedback.
-
|
Yes, the numbers of the optimization traces are different and the final value of the noisy test are different either. (The true value is always 0) |
Beta Was this translation helpful? Give feedback.
-
|
I cannot think of a reason why BADS would slightly but consistently fail the noisy runs, nor I have encountered this before. The only thing I can think is that there is some MATLAB function which is called only (or mostly) when performing a stochastic optimization, which for some reason behaves differently (or worse) than expected. Do you have MATLAB's optimization toolbox installed? |
Beta Was this translation helpful? Give feedback.
-
|
Thanks! After installing the optimization toolbox, the problem is solved. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for going through the troubleshooting. In conclusion, the issue was caused by the lack of MATLAB's Optimization Toolbox. |
Beta Was this translation helpful? Give feedback.
Thanks for going through the troubleshooting.
In conclusion, the issue was caused by the lack of MATLAB's Optimization Toolbox.
BADS can run without the optimization toolbox, but it will use some worse methods for internal optimization (namely, to fit the surrogate Gaussian process hyperparameters). As a consequence, the performance of BADS is substantially reduced.