This code supplements the paper [1] and can be used to replicate the numerical experiments.
Now accepted for poster presentation at CLeaR 2024 (to appear in PMLR).
This code is a fork of Benchpress [2] that allows us to evaluate a variety of structure learning algorithms.
Our analysis of the influence of hyperparameter selection on the performance of the algorithms is the extra code added on top of Benchpress, and which can be found under the hyperparams folder.
- Download this code.
- Go to the official documentation of Benchpress and perform the installation steps.
- Open a bash command and enter this repository:
cd hyperparams-causal-discovery. - To run all the experiments, execute:
bash run.sh. - (warning: the above step may take weeks to complete)
The results are also available in this repository, here.
All the post-processing code of the results can be found in the Python Notebooks (.ipynb files) here. Note running these may require some additional Python packages to be installed.
Executing the above files produces all kinds of figures that can be also found here. Only selected figures are presented in the paper.
- [1] D. Machlanski, S. Samothrakis, and P. Clarke, ‘Robustness of Algorithms for Causal Structure Learning to Hyperparameter Choice’. arXiv, Oct. 27, 2023. doi: 10.48550/arXiv.2310.18212.
- [2] F. L. Rios, G. Moffa, and J. Kuipers, “Benchpress: A Scalable and Versatile Workflow for Benchmarking Structure Learning Algorithms.” arXiv, Jul 8, 2021. doi: 10.48550/arXiv.2107.03863.