Unable to recover original labels via invert function in segmentation.

[talhaanwarch]

I am trying to read validation data mask using nibabel and inverted mask got via post transform using monai. As both are ground, they should be same. But in my case, they are not same
Here is my transform function, i have used same for training and validation

train_transforms = T.Compose(
            [
        T.LoadImaged(keys=["image", "label"]),
        T.AddChanneld(keys=["image", "label"]),
        T.Spacingd(keys=["image", "label"],pixdim=(1.5, 1.5, 2.0),mode=("bilinear", "nearest")),
        T.ScaleIntensityd(keys=["image"]),
        T.CropForegroundd(keys=["image", "label"], source_key="image"),
        T.RandCropByPosNegLabeld(
            keys=["image", "label"],
            label_key="label",
            spatial_size=(64, 64, 64),
            pos=1,
            neg=1,
            num_samples=8,
            image_key="image",
            image_threshold=0,
        ),
        T.ToTensord(keys=["image", "label"]),
            ]
        )

Here is post transform function

post_transforms = T.Compose([
        T.ToTensord(keys="pred"), 
        T.Activationsd(keys="pred", sigmoid=True),
        T.AsDiscreted(keys="pred", threshold_values=True),
        T.Invertd(
            keys=["pred",'label'],  # invert the `pred` data field, also support multiple fields
            transform=train_transforms,
            orig_keys="image",  # get the previously applied pre_transforms information on the `img` data field,
                              # then invert `pred` based on this information. we can use same info
                              # for multiple fields, also support different orig_keys for different fields
            meta_keys=["pred_meta_dict","label_meta_dict"],  # key field to save inverted meta data, every item maps to `keys`
            orig_meta_keys="image_meta_dict",  # get the meta data from `img_meta_dict` field when inverting,
                                             # for example, may need the `affine` to invert `Spacingd` transform,
                                             # multiple fields can use the same meta data to invert
            meta_key_postfix="meta_dict",  # if `meta_keys=None`, use "{keys}_{meta_key_postfix}" as the meta key,
                                           # if `orig_meta_keys=None`, use "{orig_keys}_{meta_key_postfix}",
                                           # otherwise, no need this arg during inverting
            nearest_interp=True,  # change to use "nearest" mode in interpolation when inverting
            to_tensor=True,  # convert to PyTorch Tensor after inverting
        ),
        #SaveImaged(keys="pred", meta_keys="pred_meta_dict", output_dir="./out", output_postfix="seg", resample=False),
    ])

Here is the conversion process

from monai.data import decollate_batch
dataloader=model.val_dataloader()
model.eval()
post_data=[]
with torch.no_grad():
  for d in dataloader:
    images = d["image"].to(device)
    # define sliding window size and batch size for windows inference
    infer_outputs = sliding_window_inference(inputs=images, roi_size=(32 , 32, 32), sw_batch_size=2, predictor=model)
    infer_outputs = decollate_batch(infer_outputs)
    for (infer_output, infer_output_data) in zip(infer_outputs, decollate_batch(d)):
      infer_output_data["pred"] = infer_output
      post_data.append(post_transforms(infer_output_data))
      break

Some further script

out=post_data[0]
out['image'].shape,out['label'].shape,out['pred'].shape

results in

(torch.Size([1, 64, 64, 64]),
 torch.Size([1, 512, 512, 33]),
 torch.Size([2, 512, 512, 33]))

This is how images are loaded without monai

import nibabel as nib
val_data=[]
for i in range(len(val_files)):
  img = nib.load(val_files[i]['image']).get_fdata()
  lab = nib.load(val_files[i]['label']).get_fdata()
  val_data.append((img,lab))

and the first image has shape

raw_val=val_data[0]
raw_val[0].shape,raw_val[1].shape#raw label and segment

of
((512, 512, 33), (512, 512, 33))

You can see two images are not same

As both are ground truth one is input ground truth and other is transform apply on ground truth and invert transform apply to get it back. so I assume both should be same

[Nic-Ma]

Hi @talhaanwarch ,

Thanks for your interest and experiment with MONAI.
The reason is that you set nearest_interp=True in the Invertd which will use nearest as interpolation method when inverting, so the result is not "that smooth".
You can take a look at this tutorial for reference:
https://github.com/Project-MONAI/tutorials/blob/master/modules/decollate_batch.ipynb

Thanks.

[Nic-Ma]

Let me update the inference example for it.
Project-MONAI/tutorials#275

Thanks.