Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

heudiconv crashes when converting a derived ADC map from Siemens scanner #633

Open
1 of 2 tasks
neurorepro opened this issue Feb 8, 2023 · 5 comments · May be fixed by #634
Open
1 of 2 tasks

heudiconv crashes when converting a derived ADC map from Siemens scanner #633

neurorepro opened this issue Feb 8, 2023 · 5 comments · May be fixed by #634

Comments

@neurorepro
Copy link
Contributor

neurorepro commented Feb 8, 2023 •
edited
Loading

Summary

heudiconv crashes when converting a derived ADC map from Siemens scanner with the following trace:

INFO: Running heudiconv version 0.11.6 latest 0.11.6
INFO: Need to process 1 study sessions
INFO: PROCESSING STARTS: {'subject': 'mysubj', 'outdir': '/mnt/projects/data/dcm_bids/bug_dwi/bids_mysubj/', 'session': '01'}
INFO: Processing 7 dicoms
INFO: Analyzing 7 dicoms
Traceback (most recent call last):
  File "/opt/conda/shared/envs/ni38/lib/python3.8/runpy.py", line 193, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/opt/conda/shared/envs/ni38/lib/python3.8/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/me/.vscode/extensions/ms-python.python-2022.18.2/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/__main__.py", line 39, in <module>
    cli.main()
  File "/home/me/.vscode/extensions/ms-python.python-2022.18.2/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 430, in main
    run()
  File "/home/me/.vscode/extensions/ms-python.python-2022.18.2/pythonFiles/lib/python/debugpy/adapter/../../debugpy/launcher/../../debugpy/../debugpy/server/cli.py", line 284, in run_file
    runpy.run_path(target, run_name="__main__")
  File "/home/me/.vscode/extensions/ms-python.python-2022.18.2/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 321, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/home/me/.vscode/extensions/ms-python.python-2022.18.2/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 135, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/home/me/.vscode/extensions/ms-python.python-2022.18.2/pythonFiles/lib/python/debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_runpy.py", line 124, in _run_code
    exec(code, run_globals)
  File "/mnt/projects/bids/golestani_data/test_numpy_error/test_heudiconv.py", line 20, in <module>
    runner(args)
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/heudiconv/cli/run.py", line 24, in main
    workflow(**kwargs)
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/heudiconv/main.py", line 379, in workflow
    prep_conversion(sid,
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/heudiconv/convert.py", line 172, in prep_conversion
    seqinfo = group_dicoms_into_seqinfos(
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/heudiconv/dicoms.py", line 202, in group_dicoms_into_seqinfos
    mwinfo = validate_dicom(filename, dcmfilter)
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/heudiconv/dicoms.py", line 106, in validate_dicom
    del mw.series_signature[sig]
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/nibabel/onetime.py", line 142, in __get__
    val = self.getter(obj)
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/nibabel/nicom/dicomwrappers.py", line 642, in series_signature
    signature['image_shape'] = (self.image_shape, eq)
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/nibabel/onetime.py", line 142, in __get__
    val = self.getter(obj)
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/nibabel/nicom/dicomwrappers.py", line 516, in image_shape
    first_frame = self.frames[0]
  File "/opt/conda/shared/envs/ni38/lib/python3.8/site-packages/pydicom/multival.py", line 102, in __getitem__
    return self._list[index]
IndexError: list index out of range

This is generated by frame_indices = np.delete(frame_indices, stackid_dim_idx, axis=1) in the if-block:

if stackid_tag in dim_seq:
    stackid_dim_idx = dim_seq.index(stackid_tag)
    frame_indices = np.delete(frame_indices, stackid_dim_idx, axis=1)
    dim_seq.pop(stackid_dim_idx)

Looks like this is because frame_indices is empty (so no second axis), which should be expected for a derived map though, since earlier there is:

frame_indices = np.array(
    [frame.FrameContentSequence[0].DimensionIndexValues
     for frame in self.frames])

with

self.frames = pydicom.Sequence(
    frame for frame in self.frames if
    frame.MRDiffusionSequence[0].DiffusionDirectionality
    != 'ISOTROPIC'
    )

So for a derived map self.frames will become empty, so frame_indices will be so as well, and so the if-block will bring hell.

The thing i don't understand is why we are running the if-block in this case, and why the condition evaluates to true in my case where stackid_tag=2134102 and dim_seq[0] = (0020, 9056). But i have to admit i didn't look into the __int__ function of the dim_seq items.

Just in case this is useful, here is the metadata for this single problematic dcm file:

{
	"Modality": "MR",
	"MagneticFieldStrength": 3,
	"ImagingFrequency": 123.226,
	"Manufacturer": "Siemens",
	"ManufacturersModelName": "MAGNETOM Prisma",
	"InstitutionName": "Neurocenter",
	"InstitutionAddress": "Brain avenue",
	"DeviceSerialNumber": "42",
	"StationName": "AWP42",
	"BodyPartExamined": "BRAIN",
	"PatientPosition": "HFS",
	"ProcedureStepDescription": "Not trying to crash",
	"SoftwareVersions": "syngo MR XA30",
	"MRAcquisitionType": "2D",
	"SeriesDescription": "dwi_acq-B2850x1D5MMISO_afid-beta_ADC",
	"ProtocolName": "dwi_acq-B2850x1D5MMISO_afid-beta",
	"ScanningSequence": "EP",
	"SequenceVariant": "SK\\SP",
	"ScanOptions": "PFP\\FS",
	"PulseSequenceName": "*epse2d1_150",
	"ImageType": ["DERIVED", "PRIMARY", "DIFFUSION", "ADC"],
	"ImageTypeText": ["DERIVED", "PRIMARY", "DIFFUSION", "ADC", "ND"],
	"NonlinearGradientCorrection": false,
	"RawImage": false,
	"SeriesNumber": 51,
	"AcquisitionTime": "17:11:9.695000",
	"SliceThickness": 1.5,
	"SpacingBetweenSlices": 1.5,
	"EchoTime": 0.074,
	"RepetitionTime": 6.7,
	"MTState": false,
	"FlipAngle": 90,
	"PartialFourier": 0.875,
	"BaseResolution": 150,
	"ShimSetting": [
		-1273,
		-11693,
		6264,
		288,
		261,
		-78,
		-194,
		57	],
	"TxRefAmp": 233.778,
	"PhaseResolution": 1,
	"ReceiveCoilName": "HeadNeck_64",
	"ReceiveCoilActiveElements": "HC1-6",
	"CoilString": "HeadNeck_64",
	"PulseSequenceDetails": "%SiemensSeq%\\ep2d_diff",
	"RefLinesPE": 45,
	"CoilCombinationMethod": "Adaptive Combine",
	"MultibandAccelerationFactor": 2,
	"PercentPhaseFOV": 100,
	"PercentSampling": 100,
	"EchoTrainLength": 44,
	"PartialFourierDirection": "PHASE",
	"AcquisitionMatrixPE": 150,
	"ReconMatrixPE": 150,
	"BandwidthPerPixelPhaseEncode": 26.316,
	"ParallelReductionFactorInPlane": 3,
	"ParallelAcquisitionTechnique": "SMS",
	"EffectiveEchoSpacing": 0.000253331,
	"DerivedVendorReportedEchoSpacing": 0.000759994,
	"TotalReadoutTime": 0.0377464,
	"PixelBandwidth": 1515,
	"DwellTime": 2.2e-06,
	"PhaseEncodingDirection": "j-",
	"SliceTiming": [
		6.6225,
		0,
		6.7625,
		0.14,
		6.9025,
		0.2825,
		7.045,
		0.4225,
		7.185,
		0.5625,
		7.325,
		0.705,
		7.4675,
		0.845,
		7.6075,
		0.9875,
		7.7475,
		1.1275,
		7.89,
		1.2675,
		8.03,
		1.41,
		8.17,
		1.55,
		8.3125,
		1.69,
		8.4525,
		1.8325,
		8.595,
		1.9725,
		8.735,
		2.1125,
		8.875,
		2.255,
		9.0175,
		2.395,
		9.1575,
		2.535,
		9.2975,
		2.6775,
		9.44,
		2.8175,
		9.58,
		2.9575,
		9.72,
		3.1,
		9.8625,
		3.24,
		10.0025,
		3.3825,
		10.1425,
		3.5225,
		10.285,
		3.6625,
		10.425,
		3.805,
		10.565,
		3.945,
		10.7075,
		4.085,
		10.8475,
		4.2275,
		10.99,
		4.3675,
		11.13,
		4.5075,
		11.27,
		4.65,
		11.4125,
		4.79,
		11.5525,
		4.93,
		11.6925,
		5.0725,
		11.835,
		5.2125,
		11.975,
		5.3525,
		12.115,
		5.495,
		12.2575,
		5.635,
		12.3975,
		5.7775,
		12.5375,
		5.9175,
		12.68,
		6.0575,
		12.82,
		6.2,
		12.96,
		6.34,
		13.1025,
		6.48	],
	"ImageOrientationPatientDICOM": [
		1,
		0,
		0,
		0,
		1,
		0	],
	"InPlanePhaseEncodingDirectionDICOM": "COL",
	"ConversionSoftware": "dcm2niix",
	"ConversionSoftwareVersion": "v1.0.20220720"
}

Platform details:

Choose one:

  • Local environment
  • Container
  • Heudiconv version:

Running heudiconv version 0.11.6 latest 0.11.6
yarikoptic added a commit to dbic/heudiconv that referenced this issue Feb 8, 2023
@yarikoptic
Also tollerate IndexError while trying to delete a field in mw
24f4d88 
Closes nipy#633

Somewhat of a blind fix without looking to deep why we get IndexError
instead of KeyError
@yarikoptic yarikoptic linked a pull request Feb 8, 2023 that will close this issue
Open
@neurorepro
Copy link
Contributor Author

neurorepro commented Feb 8, 2023 •
edited
Loading

@yarikoptic I got a new error with #634 . This definitely counts as progress:

INFO: Running heudiconv version 0.11.6.post31+g24f4d88 latest 0.11.6
INFO: Need to process 1 study sessions
INFO: PROCESSING STARTS: {'subject': 'mysubj', 'outdir': '/mnt/projects/data/dcm_bids/bids_dwi_min_newheudi/', 'session': '01'}
INFO: Processing 7 dicoms
INFO: Analyzing 7 dicoms
/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/nibabel/nicom/dicomwrappers.py:525: UserWarning: Derived images found and removed
  warnings.warn('Derived images found and removed')
Traceback (most recent call last):
  File "/home/michael/.conda/envs/heudiconv310_dwi/bin/heudiconv", line 8, in <module>
    sys.exit(main())
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/heudiconv/cli/run.py", line 24, in main
    workflow(**kwargs)
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/heudiconv/main.py", line 379, in workflow
    prep_conversion(sid,
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/heudiconv/convert.py", line 172, in prep_conversion
    seqinfo = group_dicoms_into_seqinfos(
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/heudiconv/dicoms.py", line 232, in group_dicoms_into_seqinfos
    if mw.is_same_series(mwgroup[idx]):
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/nibabel/nicom/dicomwrappers.py", line 357, in is_same_series
    my_sig = self.series_signature
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/nibabel/onetime.py", line 142, in __get__
    val = self.getter(obj)
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/nibabel/nicom/dicomwrappers.py", line 625, in series_signature
    signature['image_shape'] = (self.image_shape, eq)
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/nibabel/onetime.py", line 142, in __get__
    val = self.getter(obj)
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/nibabel/nicom/dicomwrappers.py", line 505, in image_shape
    first_frame = self.frames[0]
  File "/home/michael/.conda/envs/heudiconv310_dwi/lib/python3.10/site-packages/pydicom/multival.py", line 104, in __getitem__
    return self._list[index]
IndexError: list index out of range

@neurorepro
Copy link
Contributor Author

@yarikoptic looking at the conversion of other metrics with version 0.11.6 of heudiconv (not the PR), i also got the error for TRACEW, but not for FA and ColFA (which stands for something like colored FA).

Serendipitously i noticed that the third dimension of ColFA is not properly retrieved by heudiconv which shows dim3 to be 1 instead of 94. This may be due to the image datatype (RGB24) and Siemens colored FA volumes are probably not on top of heudiconv backlog.

@yarikoptic
Copy link
Member

given that we get through nibabel/nicom/dicomwrappers.py", line 357, in is_same_series -- the issue might even be in nibabel , and worth checking/filing there.

@yarikoptic
Copy link
Member

also happens upon the same del mw.series_signature[sig] for

@yarikoptic yarikoptic mentioned this issue Aug 9, 2023
Closed
@yarikoptic
Copy link
Member

@neurorepro could you try with current fresh nibabel? current heudiconv image has it: #670 (comment)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Also tolerate IndexError while trying to delete a field in mw dbic/heudiconv
2 participants
@yarikoptic @neurorepro