To test our MMC code using pre-obtained individual metrics, start here
If you would like to train your own models, and generate your own metrics, start here.
This code is provided for research and development use only. This code is not intended for use in clinical decision-making or for any other clinical use and the performance of the code for clinical use has not been established. This source code requires selection of a reference and test datasets by the user. Microsoft does not warrant the suitability or accuracy of any predictive model generated using this source code. You bear sole responsibility for selection of a training dataset and for evaluation and use of any resulting model
- Setup [Tutorial]
- Train a customized VAE to detect image-based drift [Notebook][Script]
- Train a frontal-only model that is presumed to be deployed for further experiments [Notebook][Script]
- Generate VAE latent representations on PadChest [Notebook][Script]
- Generate model predictions on PadChest [Notebook][Script]
- Generate performance and individual metrics trials:
- Unmodified PadChest data stream [Notebook][Script]
- Performance degradation through hard data mining [Notebook][Script]
- Clinical Workflow Failure through injecting lateral X-Ray images to a frontal-only model [Notebook][Script]
- Clinical Workflow Failure through injecting pediatric Chest X-Ray images to a frontal-only model [Coming Soon]
- Unification of statistical and model performance-based drift metrics [Notebook]
- Explore VAE latent space [Notebook]