Empty voxels when bringing back softseg to native space

[sandrinebedard]

Description

For #2, we want to bring back the softseg to the native space due to reasons explained here.

Right now, we use sct_apply_transfowith linear interpolation

contrast-agnostic-softseg-spinalcord/process_data.sh

Line 187 in 88fbfc3

sct_apply_transfo -i ${file_softseg}.nii.gz -d ${file_path}.nii.gz -w ${warping_field_inv}.nii.gz -x linear -o ${file_path}_softseg.nii.gz

However, the resulting softseg sometime have empty voxels at the extremity of the segmentation while in T2w space, the segmentation is complete:

I also openned it in fsleyes to make sure it wasn't just the threshold of 0.5 appplied in the qc report:

(sub-amu01_acq-T1w_MTS)

The softseg in T2w space:

This doesn't happen all the time, I am wondering if this won't be a problem when we have all the manual segmentations since registration is based on the segmentation and slices in the extremity aren't always good.

This is the segmentation with stc_deepseg_sc used here for sub-amu01_acq-T1w_MTS:

[jcohenadad]

Wow, that is a strange bug. I would start investigating what happens if you change -x linear with -x nn or -x spline.

Then, I would open the warping field and check if there are discontinuities at the location where you "loose" voxels. My gut feeling is that the warping field has some issues which we would need to address.

[sandrinebedard]

I would start investigating what happens if you change -x linear with -x nn or -x spline.

Same thing happens with -x nn and -x spline:

With -x spline:

With -x nn:

The problem must be with the warping field, I'll check for discontinuities!

[sandrinebedard]

Hmm, there are no discontinuities on the warping field where the empty voxels are...

Here is the warping field overlaid on the image (warping field is in blue and opacity reduded):

I also applied the transformation to the coverage image (image of 1 that we registered to T2w space) to check if the same thing happens and yes:

[jcohenadad]

@sandrinebedard I'm having a hard time understanding what is the orientation on #11 (comment). Could you please indicate A/P/I/S/R/L in the figure, and also show a view of the spinal cord (maybe overlaid) so we can understand what location in the warping field corresponds to the discontinuity in the segmentation.

[sandrinebedard]

@sandrinebedard I'm having a hard time understanding what is the orientation on #11 (comment). Could you please indicate A/P/I/S/R/L in the figure, and also show a view of the spinal cord (maybe overlaid) so we can understand what location in the warping field corresponds to the discontinuity in the segmentation.

Of course! I just updated #11 (comment)!

[jcohenadad]

ah! i think i know what this is. The Z component of the warping field (which is a 4D file) is "pushing" a voxel from outside (ie: defined as 0) inside the segmentation. A way to overcome this edge effect is simply by padding with mirroring the image when estimating the transformation.

Alternatively (easier) you can get rid of all edge slices.

[sandrinebedard]

Ahh I see!

Alternatively (easier) you can get rid of all edge slices.

Getting rid of the edge slices may be fine for T1w, but for T2star (same thing for MTS data and dwi), removing 2 slices is a bit extreme since we don't have a lot.

I can try padding!

[sandrinebedard]

I tried padding with mirroring the image when estimating the transformation with sct_register_multimodal. Since the transformation is based on the segmentation, registration was worst for some images.
e.g:
sub-barcelona01_acq-T1w_MTS_reg.nii.gz with padding (2 slices at both extremities):

without padding:

I tried padding one slice over the segmentation and one under with the top and bottom slices. For now, it seams to correct the problem, but I will test with more images to make sure we don't have the problem anymore, in the case the edge effect affects more than one slice.

sub-barcelona01_acq-T1w_MTS_reg.nii.gz with padding of one slice at both extremities:

[jcohenadad]

How much padding did you do? I would start with only one slice at both extremities.

[sandrinebedard]

How much padding did you do? I would start with only one slice at both extremities.

I tested out with 2 slices at both extremities and then 1 : one slice at both extremities is what worked the best for regsitration and resolved the edge effect for the images I tested out!

The only thing I need to test out is for this case where 2 slices were affected if padding with 1 slice at both extremities resolves the problem!

[sandrinebedard]

I tested out the image sub-tokyoSkyra_T1w that had 2 slices affected with the edge effect and with padding of 1 slice at each extremity, the problem is resolved 🎉

[joshuacwnewton]

ah! i think i know what this is. The Z component of the warping field (which is a 4D file) is "pushing" a voxel from outside (ie: defined as 0) inside the segmentation. A way to overcome this edge effect is simply by padding with mirroring the image when estimating the transformation.

Have you encountered this in the past, @jcohenadad?

I ask because I'm wondering if this behavior is common enough that we might want to check for and/or document it on SCT's end. (e.g. in sct_apply_transfo, checking the resulting transformed image for empty slices, then warning the user that padding might be helpful to fix the issue.)