You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
As of now, the hypersweeper does not allow for continuing SMAC runs because it overwrites previous runs by initializing the job index with zero for every optimization run instead of using the current config index from SMAC when submitting a new batch.
Example:
Run SMAC with the HS for 100 trials. After 10 trials, something crashes.
Runhistory: configs [0-9]
Sweep directories: [0-9]
Now, if we want to continue the run, i.e., start a new HS optimization:
Runhistory: configs [0-9] from previous run + configs [10-19]
Sweep directories: [0-9] again -> this is the problem, we use the same sweep directories again for different configs
One solution could be so use the configuration index from SMAC if it is contained in the object returned by smac.ask(). Otherwise the length of the run history could be used to determine the next
The text was updated successfully, but these errors were encountered:
Sounds good, but I don't think the TrialInfo we get from smac.ask() actually contains the config ID. @benjamc any idea how to best get that info? Is it in the configuration?
As of now, the hypersweeper does not allow for continuing SMAC runs because it overwrites previous runs by initializing the job index with zero for every optimization run instead of using the current config index from SMAC when submitting a new batch.
Example:
Run SMAC with the HS for 100 trials. After 10 trials, something crashes.
Runhistory: configs [0-9]
Sweep directories: [0-9]
Now, if we want to continue the run, i.e., start a new HS optimization:
Runhistory: configs [0-9] from previous run + configs [10-19]
Sweep directories: [0-9] again -> this is the problem, we use the same sweep directories again for different configs
One solution could be so use the configuration index from SMAC if it is contained in the object returned by smac.ask(). Otherwise the length of the run history could be used to determine the next
The text was updated successfully, but these errors were encountered: