Integration of ConvNeXt U-Net and CBAM U-Net into the medical image segmentation framework MIScnn

This github repository documents the code files used in training and generating predictions with CBAM U-Net, ConvNeXt U-Net and a common U-Net by integrating the architectures into the medical image segmentation framework MIScnn on the 2 datasets ISIC2018 and Kvasir-SEG as part of my bachelor thesis "Integration of ConvNeXt U-Net and CBAM U-Net into the medical image segmentation framework MIScnn" at the university of augsburg. We also include two jupyter notebooks evaluating the models performances on one of the datasets respectively. Please refer to the table of contents for an overview of the repositories general structure.

In order to reproduce the results, you can use the code files of the repository to train and generate predictions yourself. To do this, make sure to download the datasets (see Libraries/modules), save them in the expected folder structure (see Expected dataset format) and execute the corresponding scripts. For example, training CBAM U-Net on the Kvasir-SEG dataset, generating predictions on it and evaluating the results would encompass executing 'Bachelor/cbam/kvasir/cbam_train.py', then 'Bachelor/cbam/kvasir/cbam_predict.py' and finally running 'Bachelor/Evaluation_kvasir.ipynb'.

Note: Make sure to change any data paths in the code files to where you have your data stored on your system

---

Table of contents

• Integration of ConvNeXt U-Net and CBAM U-Net into the medical image segmentation framework MIScnn
• General structure
• Evaluation
• Expected dataset format
• Libraries/modules

---

General structure

The repository is seperated into distinct folders, each encompassing the code base for one of the 3 utilized architectures and subfolders for both datasets. A generic example can be found below:

• Bachelor \
  • Images \
  • model_1 \
    • dataset_1 \
      • predictions \
      • model_1_train.py
      • model_1_predict.py
      • model_1.hdf5
      • model_1.log
      • model_1.py
    • dataset_2 \
      • ...
      • ...
  • model_2 \
    • ...
  • model_..\
  • Evaluation_dataset_1.ipynb
  • ...
• Bachelorarbeit.pdf
• README.md

\ denotes a folder
predictions \ contains the predictions generated by the architecture on a specific dataset
model_1_train.py is the pipeline used in the training process
model_1_predict.py is the pipeline used in generating predictions on the images of the dataset
model_1.hdf5 is a file of the saved weights after training model_1 on dataset_1. It can be used to load the weights in another file/module.
model_1.log includes the logging output of the training procedure
model_1.py encompasses the actual model architecture
Evaluation_dataset_1.ipynb contains the jupyter notebook documenting the evaluation process on dataset_1
Images generated in the evaluation noteboks are stored in Images \

Evaluation

Evaluating the models performances is done via two jupyter notebooks, who guide the viewer through the process of visually sampling some of the generated predictions and comparing them against the ground truth and the original image, taking a look at some inaccurate predictions and lastly calculating metric scores evaluating the predictions quality.

Expected dataset format

Due to the structure of the training and prediction pipelines, the datasets are expected to conform to the following format:

1. Seperate your dataset into a training, test and validation subset.
2. Assemble the 3 subsets in the following folder tree:
   • dataset \
     • training \ data \
       • Image_1
         • image.png
         • segmentation.png
       • Image_2
         • image.png
         • ...
       • ...
     • validation \ data
       • Image_24
         • ...
     • test \ data
       • ...

Where image.png is the original image and segmentation.png is the provided ground truth segmentation mask \

Note: Due to the implementation of MIScnns training methods, the training subset also needs to contain the images of the validation subset. In the above example, Image_24 would need to also be contained under training \ data \

Libraries/modules

• MIScnn: https://github.com/frankkramer-lab/MIScnn

• miseval: https://github.com/frankkramer-lab/miseval

---

• ISIC2018: https://challenge.isic-archive.com/landing/2018/
• Download: https://challenge.isic-archive.com/data/#2018

---

• Kvasir-SEG: https://www.researchgate.net/publication/337250576_Kvasir-SEG_A_Segmented_Polyp_Dataset
• Download: https://datasets.simula.no/kvasir-seg/

---