Custom BIDS converter - CAPS outputs were not found for some image(s): FileNotFoundError: No such file or directory #338

jmcawood · 2021-04-30T09:37:26Z

jmcawood
Apr 30, 2021

Hi I am new to clinica and have been slowly working through a bunch of issues however I can not solve this one. Can some one help please?

clinica run t1-volume BIDS/ test/ test-group
The t1-volume pipeline is divided into 4 parts:
t1-volume-tissue-segmentation pipeline: Tissue segmentation, bias correction and spatial normalization to MNI space
t1-volume-create-dartel pipeline: Inter-subject registration with the creation of a new DARTEL template
t1-volume-dartel2mni pipeline: DARTEL template to MNI
t1-volume-parcellation pipeline: Atlas statistics

Part 1/4: Running t1-volume-segmentation pipeline
The pipeline will be run on the following 2 image(s):
sub-S24022 | ses-01,
sub-S45030 | ses-01,
The pipeline will last approximately 10 minutes per image.

[Warning] You did not specify the number of threads to run in parallel (--n_procs argument).
Computation time can be shorten as you have 48 CPUs on this computer. We recommend using 47 threads.

How many threads do you want to use? If you do not answer within 15 sec, default value of 47 will be taken. Use --n_procs argument if you want to disable this message next time.
[00:07:15] Running pipeline for sub-S45030 | ses-01
[00:07:15] Running pipeline for sub-S24022 | ses-01

[00:12:47] Pipeline finished with errors.

CAPS outputs were not found for some image(s):
Implementation on which image(s) failed will appear soon.

Documentation can be found here: https://aramislab.paris.inria.fr/clinica/docs/public/latest/
If you need support, do not hesitate to ask: https://groups.google.com/forum/#!forum/clinica-user
Alternatively, you can also open an issue on GitHub: https://github.com/aramis-lab/clinica/issues

Here is the error file traceback

nipypecli crash crash-20210430-001245-user-3-T1wToMni.a0-c9d67331-0f44-454b-b822-fe20293f8872.pklz

File: /home/user/Documents/ADNI Data/RAW DATA/Uncompressed Data/AD/crash-20210430-001245-user-3-T1wToMni.a0-c9d67331-0f44-454b-b822-fe20293f8872.pklz
Node: t1-volume-tissue-segmentation.3-T1wToMni
Working directory: /tmp/tmp7jnpv21k/t1-volume-tissue-segmentation/da6acef93050a86f660ea26f80c21d4beb50efc7/3-T1wToMni

Node inputs:

deformation_field =
fwhm =
in_files =
interpolation =
mask = 0
matlab_cmd =
mfile = True
paths =
use_mcr =
use_v8struct = True

Traceback:
Traceback (most recent call last):
File "/home/user/miniconda3/envs/clinicaEnv/lib/python3.7/site-packages/nipype/interfaces/base/core.py", line 537, in aggregate_outputs
setattr(outputs, key, val)
File "/home/user/miniconda3/envs/clinicaEnv/lib/python3.7/site-packages/nipype/interfaces/base/traits_extension.py", line 426, in validate
value = super(MultiObject, self).validate(objekt, name, newvalue)
File "/home/user/miniconda3/envs/clinicaEnv/lib/python3.7/site-packages/traits/trait_types.py", line 2515, in validate
return TraitListObject(self, object, name, value)
File "/home/user/miniconda3/envs/clinicaEnv/lib/python3.7/site-packages/traits/trait_list_object.py", line 585, in init
notifiers=[self.notifier],
File "/home/user/miniconda3/envs/clinicaEnv/lib/python3.7/site-packages/traits/trait_list_object.py", line 213, in init
super().init(self.item_validator(item) for item in iterable)
File "/home/user/miniconda3/envs/clinicaEnv/lib/python3.7/site-packages/traits/trait_list_object.py", line 213, in
super().init(self.item_validator(item) for item in iterable)
File "/home/user/miniconda3/envs/clinicaEnv/lib/python3.7/site-packages/traits/trait_list_object.py", line 865, in _item_validator
return trait_validator(object, self.name, value)
File "/home/user/miniconda3/envs/clinicaEnv/lib/python3.7/site-packages/nipype/interfaces/base/traits_extension.py", line 330, in validate
value = super(File, self).validate(objekt, name, value, return_pathlike=True)
File "/home/user/miniconda3/envs/clinicaEnv/lib/python3.7/site-packages/nipype/interfaces/base/traits_extension.py", line 135, in validate
self.error(objekt, name, str(value))
File "/home/user/miniconda3/envs/clinicaEnv/lib/python3.7/site-packages/traits/base_trait_handler.py", line 75, in error
object, name, self.full_info(object, name, value), value
traits.trait_errors.TraitError: Each element of the 'out_files' trait of an ApplySegmentationDeformationOutput instance must be a pathlike object or string representing an existing file, but a value of '/tmp/tmp7jnpv21k/t1-volume-tissue-segmentation/da6acef93050a86f660ea26f80c21d4beb50efc7/3-T1wToMni/wsub-S24022_ses-01_T1w.nii' <class 'str'> was specified.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/home/user/miniconda3/envs/clinicaEnv/lib/python3.7/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
result["result"] = node.run(updatehash=updatehash)
File "/home/user/miniconda3/envs/clinicaEnv/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 516, in run
result = self._run_interface(execute=True)
File "/home/user/miniconda3/envs/clinicaEnv/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 635, in _run_interface
return self._run_command(execute)
File "/home/user/miniconda3/envs/clinicaEnv/lib/python3.7/site-packages/nipype/pipeline/engine/nodes.py", line 741, in _run_command
result = self._interface.run(cwd=outdir)
File "/home/user/miniconda3/envs/clinicaEnv/lib/python3.7/site-packages/nipype/interfaces/base/core.py", line 436, in run
outputs = self.aggregate_outputs(runtime)
File "/home/user/miniconda3/envs/clinicaEnv/lib/python3.7/site-packages/nipype/interfaces/base/core.py", line 544, in aggregate_outputs
raise FileNotFoundError(msg)

FileNotFoundError: No such file or directory '['/tmp/tmp7jnpv21k/t1-volume-tissue-segmentation/da6acef93050a86f660ea26f80c21d4beb50efc7/3-T1wToMni/wsub-S24022_ses-01_T1w.nii']' for output 'out_files' of a ApplySegmentationDeformation interface

The file that could not be found "wsub-S24022_ses-01_T1w.nii" was created and saved in my /Documents/MATLAB/ working directory, I'm not sure if this is the issue.

omar-rifai · 2021-04-30T11:17:25Z

omar-rifai
Apr 30, 2021

Hi @jmcawood,

I have had similar issues while using symbolic paths in the past. It it the case for you?

Regards,

0 replies

jmcawood · 2021-04-30T12:28:30Z

jmcawood
Apr 30, 2021
Author

@omar-rifai I'm not sure, each time I run the following command regardless of how I reference the BIDS and CAPS directories that error is displayed.

Clinica pipline used

clinica run t1-volume BIDS_DIRECTORY CAPS_DIRECTORY GROUP_LABEL

The file wsub-S24022_ses-01_T1w.nii that could not be found was saved in my MATLAB working directory beside the folder t1-volume-tissue-segmentation.

However the error says it should be located within the following directory

/tmp/tmp7jnpv21k/t1-volume-tissue-segmentation/da6acef93050a86f660ea26f80c21d4beb50efc7/3-T1wToMni/

Not sure if this helps.

Regards,

0 replies

omar-rifai · 2021-04-30T15:30:17Z

omar-rifai
Apr 30, 2021

Can you please try running the command again referencing the working directory explicitely?

0 replies

jmcawood · 2021-05-01T14:13:58Z

jmcawood
May 1, 2021
Author

@omar-rifai after running the command again this is the error output

FileNotFoundError: No such file or directory '['/Documents/ADNI Data/RAW DATA/Uncompressed Data/AD/working-directory/t1-volume-tissue-segmentation/89344040bbb9ce7290abf78ba402f02e563bf574/3-T1wToMni/wsub-S12824_ses-01_T1w.nii']' for output 'out_files' of a ApplySegmentationDeformation interface

The working directory contained the following folders
t1-volume-tissue-segmentation
89344040bbb9ce7290abf78ba402f02e563bf574
c037b55054a941b654e09abc66ad8516532f2519

and files

graph1.json
index.html
d3.js
graph.json

0 replies

jmcawood · 2021-05-02T12:43:39Z

jmcawood
May 2, 2021
Author

I'm not sure if the conversion from dicom to bids effects this.
But I was unable to used the built in adni-to-bids, I could not download dcm2nii.

So I used dcm2niix with the MRIcroGL GUI.

0 replies

omar-rifai · 2021-05-03T07:20:11Z

omar-rifai
May 3, 2021

@jmcawood, Yes indeed, that might be the issue. Can you please try the suggestions in this post to download dcm2nii ?

0 replies

jmcawood · 2021-05-03T07:44:36Z

jmcawood
May 3, 2021
Author

@omar-rifai I will try that.

This was in the clinica.log file

[MultiProc] Running 2 tasks, and 0 jobs ready. Free memory (GB): 906.48/906.88, Free processors: 45/47.
Currently running:
* t1-volume-tissue-segmentation.3-T1wToMni
* t1-volume-tissue-segmentation.3-T1wToMni
210503-17:41:03,486 nipype.workflow WARNING:
Storing result file without outputs
210503-17:41:03,487 nipype.workflow WARNING:
[Node] Error on "t1-volume-tissue-segmentation.3-T1wToMni" (/tmp/tmpo2vvry0o/t1-volume-tissue-segmentation/c0503aa754f1e10030bd8e7e16da1b42a5b2576b/3-T1wToMni)
210503-17:41:03,839 nipype.workflow ERROR:
Node 3-T1wToMni.a1 failed to run on host .

0 replies

jmcawood · 2021-05-03T08:53:33Z

jmcawood
May 3, 2021
Author

I was unable to download and use the file as suggested in issue #191

0 replies

omar-rifai · 2021-05-03T08:55:14Z

omar-rifai
May 3, 2021

@jmcawood, did you try both suggested methods? What problem did you encouter?

Thanks!

0 replies

jmcawood · 2021-05-03T08:58:05Z

jmcawood
May 3, 2021
Author

The version suggested was not available, so I went with an older version 11-2010. I extracted this and then moved the dcm2nii file to another folder and then added it to the path with the following code

Dcm2nii

export PATH="$HOME/Documents/dcm2nii/dcm2nii:$PATH"

But I still get this error

[Error] Clinica could not find dcm2nii tool from MRIcron in your PATH environment: this can be downloaded from https://www.nitrc.org/frs/?group_id=152 (choose the 2016 version).

0 replies

omar-rifai · 2021-05-03T09:06:13Z

omar-rifai
May 3, 2021

I just tried downloading the 2016 version from this link and it seems available. Also you can try typing dcm2nii in your terminal to see if it was correctly added to your PATH environment prior to running clinica.

0 replies

jmcawood · 2021-05-03T09:11:27Z

jmcawood
May 3, 2021
Author

How do I use the source code to get dcm2nii

0 replies

omar-rifai · 2021-05-03T09:30:02Z

omar-rifai
May 3, 2021

Inside the dcm2nii directory, you can find the dcm2nii.app tool. You can reference this in your $PATH and then (after restarting your terminal), check that it is correctly installed by typing dcm2nii in the terminal.

0 replies

jmcawood · 2021-05-03T09:52:14Z

jmcawood
May 3, 2021
Author

I get this now when I type dcm2nii into my terminal

bash: MRIcron-1.0.20160502/dcm2nii/dcm2nii.app/Contents/MacOS/dcm2nii: cannot execute binary file

So I ran the clinica adni-to-bids and it executed however it didn't output any .nii or .json files

0 replies

jmcawood · 2021-05-03T13:09:03Z

jmcawood
May 3, 2021
Author

The converter just keeps displaying this for each ID

No fMRI image path found for subject 168_S_6908 in visit bl with image ID 1416085

Paths of fMRI images found. Exporting images into BIDS ...

[FMRI] No path specified for 011_S_0021 in session m144 1/2124

0 replies

omar-rifai · 2021-05-03T14:35:42Z

omar-rifai
May 3, 2021

With regards to the first issue, it may be an incompatibility issue with your operating system. In case you are running a 32 bits system (or you are not running mac os) you might need to download another version of the software. In which case, you'll need to follow @mdiazmel recommendation in issue #191 to find the version that suits your system.

For the second issue, is it not a problem as t1-volume only requires t1w MRI images.

0 replies

jmcawood · 2021-05-03T14:40:58Z

jmcawood
May 3, 2021
Author

I am running a linux os.

0 replies

jmcawood · 2021-05-04T12:20:14Z

jmcawood
May 4, 2021
Author

@omar-rifai I went back to using my own dcm to BIDS converter and got the initial issue FileNotFoundError.

I then copied the missing files which were located in my MATLAB folder, into the correct folders within the clinica t1-volume working directory. I then ran the pipeline again and everything worked fine. The pipeline went through its 4 stages with zero errors and the stats .tsv files were created.

My question is, why is the pipeline saving the wsub-_ses-01_T1w.nii files in my MATLAB working directory instead of the correct one eg

working-directory/t1-volume-tissue-segmentation/89344040bbb9ce7290abf78ba402f02e563bf574/3-T1wToMni/

Does anyone know why it might doing this. Is there some configuration that specifies this? Is it a bug? or have I missed something simple?

Kind Regards
jmcawood

0 replies

omar-rifai · 2021-05-04T13:11:47Z

omar-rifai
May 4, 2021

Hi @jmcawood,

Thanks for the feedback. I'm sorry you couldn't get the adni-to-bids tool working. Were you unable to find a version of dcm2nii which worked on your system?

I suspect the issue might come from the resulting BIDS directory that might not be identical with the one generated by our tool. I'll ping @14thibea, which might have another insight on this.

Regards,

0 replies

jmcawood · 2021-05-04T13:28:45Z

jmcawood
May 4, 2021
Author

Yeah I was struggling to get a version that worked.

My data was from ADNI but you don't have to use the adni-to-bids converter do you? Clinica just requires a valid bids structure doesnt it?

My converter still uses dcm2niix however I do the naming myself and have the bare minimum in my dataset_description.json

Thank you for your help so far

0 replies

omar-rifai · 2021-05-04T15:26:06Z

omar-rifai
May 4, 2021

No, of course, the only requirement is that the output is BIDs compliant and that all the data needed for the pipelines are present.

0 replies

omar-rifai · 2021-05-19T06:56:58Z

omar-rifai
May 19, 2021

Hi again,
After discussing the issue with @14thibea, it may indeed be the case that your custom BIDs converter uses conventions not handled by the t1-volume pipeline.
That is, the t1-volume pipeline currently only handles a subset of cases of the BIDS specification. This has been reported as an issue and will be fixed in a future release.

Hope this helps.

Regards,

0 replies

14thibea · 2021-05-19T07:06:16Z

14thibea
May 19, 2021
Collaborator

Hi @jmcawood,

Could you please give us an example of name in your BIDS so we can understand what the problem is?

0 replies

jmcawood · 2021-05-19T09:27:41Z

jmcawood
May 19, 2021
Author

Hi @14thibea and @omar-rifai, thankyou for the reply

The output of my bids converter looks like this

{
"Name": "Patients with AD",
"BIDSVersion": "1.0.2"
}

Here is the tree view of by BIDS folder with only one patient

BIDS
└──dataset_description.json
└──participants.tsv
└───sub-S12365
└───ses-M00
└───anat
└───sub-S12365_ses-M00_T1w.json
└───sub-S12365_ses-M00_T1w.nii

participants.tsv only has the participant_id and session_id.
My dataset_description.json contains only this

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Custom BIDS converter - CAPS outputs were not found for some image(s): FileNotFoundError: No such file or directory #338

{{title}}

Replies: 24 comments

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Select a reply

Custom BIDS converter - CAPS outputs were not found for some image(s): FileNotFoundError: No such file or directory #338

Replies: 24 comments

jmcawood Apr 30, 2021 Author

jmcawood May 1, 2021 Author

jmcawood May 2, 2021 Author

jmcawood May 3, 2021 Author

jmcawood May 3, 2021 Author

jmcawood May 3, 2021 Author

Dcm2nii

jmcawood May 3, 2021 Author

jmcawood May 3, 2021 Author

jmcawood May 3, 2021 Author

The converter just keeps displaying this for each ID

jmcawood May 3, 2021 Author

jmcawood May 4, 2021 Author

jmcawood May 4, 2021 Author

14thibea May 19, 2021 Collaborator

jmcawood May 19, 2021 Author

jmcawood
Apr 30, 2021
Author

jmcawood
May 1, 2021
Author

jmcawood
May 2, 2021
Author

jmcawood
May 3, 2021
Author

jmcawood
May 3, 2021
Author

jmcawood
May 3, 2021
Author

jmcawood
May 3, 2021
Author

jmcawood
May 3, 2021
Author

jmcawood
May 3, 2021
Author

jmcawood
May 3, 2021
Author

jmcawood
May 4, 2021
Author

jmcawood
May 4, 2021
Author

14thibea
May 19, 2021
Collaborator

jmcawood
May 19, 2021
Author