dwi-preprocessing-using-t1 fail with RuntimeError: Mismatch in number of expected outputs
Closed this issue · 8 comments
Describe the bug
dwi-preprocessing pipeline fails with the below errors. The images have earlier been processed with the adni to bids converter.
To Reproduce
clinica run dwi-preprocessing-using-t1 -tsv ./adni/data/test_dwi.tsv ./adni/bids ./adni/caps-dwi-test -wd ./tmp
Expected behavior
I expected the pipeline to complete successfully.
Stdout/stderr
2024-04-19 15:57:24,618:INFO:The pipeline will be run on the following 2 image(s):
2024-04-19 15:57:24,618:INFO: sub-ADNI002S0413 | ses-M132,
2024-04-19 15:57:24,618:INFO: sub-ADNI002S1155 | ses-M126,
2024-04-19 15:57:24,618:INFO:List available in /home/espen/forskningsdata/tmp/dwi-preprocessing-using-t1/participants.tsv
2024-04-19 15:57:24,618:INFO:Computational time will depend of the number of volumes in your DWI dataset and the use of CUDA.
2024-04-19 15:58:50,786:INFO:sub-ADNI002S0413_ses-M132 has completed
Traceback (most recent call last):
File "/home/espen/forskningsdata/clinica/.venv/bin/clinica", line 8, in <module>
sys.exit(main())
^^^^^^
File "/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/clinica/cmdline.py", line 92, in main
cli()
File "/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/clinica/pipelines/dwi/preprocessing/t1/cli.py", line 69, in cli
pipeline.run(plugin="MultiProc", plugin_args={"n_procs": n_procs})
File "/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/clinica/pipelines/engine.py", line 715, in run
raise e
File "/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/clinica/pipelines/engine.py", line 696, in run
exec_graph = Workflow.run(self, plugin, plugin_args, update_hash)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/nipype/pipeline/engine/workflows.py", line 638, in run
runner.run(execgraph, updatehash=updatehash, config=self.config)
File "/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/nipype/pipeline/plugins/base.py", line 178, in run
self._clean_queue(jobid, graph, result=result)
File "/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/nipype/pipeline/plugins/base.py", line 256, in _clean_queue
raise RuntimeError("".join(result["traceback"]))
RuntimeError: Traceback (most recent call last):
File "/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
result["result"] = node.run(updatehash=updatehash)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
result = self._run_interface(execute=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
return self._run_command(execute)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node rename_into_caps.
Traceback:
Traceback (most recent call last):
File "/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/nipype/interfaces/base/core.py", line 397, in run
runtime = self._run_interface(runtime)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/nipype/interfaces/utility/wrappers.py", line 147, in _run_interface
raise RuntimeError("Mismatch in number of expected outputs")
RuntimeError: Mismatch in number of expected outputs
Desktop (please complete the following information):
- OS: Fedora 39 (same error on Debian-based container)
- 8.0.0 (same error with 7.7.7)
- FSL 6.0.7
- Python 3.11.8
Other logs
pypeline.log:
240419-15:58:52,757 nipype.utils DEBUG:
Loading pkl: /home/espen/forskningsdata/tmp/dwi-preprocessing-using-t1/17193e348755931216a0e71560893ee6046866b7/rename_into_caps/result_rename_into_caps.pklz
240419-15:58:52,757 nipype.workflow ERROR:
Node rename_into_caps.a0 failed to run on host fedora-mainframe.
240419-15:58:52,757 nipype.workflow ERROR:
Saving crash info to /home/espen/forskningsdata/crash-20240419-155852-espen-rename_into_caps.a0-12ec6482-1012-4897-8d50-9b813a4ff908.pklz
Traceback (most recent call last):
File "/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
result["result"] = node.run(updatehash=updatehash)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
result = self._run_interface(execute=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
return self._run_command(execute)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node rename_into_caps.
Traceback:
Traceback (most recent call last):
File "/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/nipype/interfaces/base/core.py", line 397, in run
runtime = self._run_interface(runtime)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/nipype/interfaces/utility/wrappers.py", line 147, in _run_interface
raise RuntimeError("Mismatch in number of expected outputs")
RuntimeError: Mismatch in number of expected outputs
❯ nipypecli show crash-20240419-143826-espen-rename_into_caps.a1-176ae1d4-7341-4c36-831c-72abd677330b.pklz
{'node': dwi-preprocessing-using-t1.rename_into_caps.a1,
'traceback': ['Traceback (most recent call last):\n',
' File '
'"/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/nipype/pipeline/plugins/multiproc.py", '
'line 67, in run_node\n'
' result["result"] = node.run(updatehash=updatehash)\n'
' ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n',
' File '
'"/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/nipype/pipeline/engine/nodes.py", '
'line 527, in run\n'
' result = self._run_interface(execute=True)\n'
' ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n',
' File '
'"/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/nipype/pipeline/engine/nodes.py", '
'line 645, in _run_interface\n'
' return self._run_command(execute)\n'
' ^^^^^^^^^^^^^^^^^^^^^^^^^^\n',
' File '
'"/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/nipype/pipeline/engine/nodes.py", '
'line 771, in _run_command\n'
' raise NodeExecutionError(msg)\n',
'nipype.pipeline.engine.nodes.NodeExecutionError: Exception '
'raised while executing Node rename_into_caps.\n'
'\n'
'Traceback:\n'
'\tTraceback (most recent call last):\n'
'\t File '
'"/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/nipype/interfaces/base/core.py", '
'line 397, in run\n'
'\t runtime = self._run_interface(runtime)\n'
'\t ^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n'
'\t File '
'"/home/espen/forskningsdata/clinica/.venv/lib64/python3.11/site-packages/nipype/interfaces/utility/wrappers.py", '
'line 147, in _run_interface\n'
'\t raise RuntimeError("Mismatch in number of expected '
'outputs")\n'
'\tRuntimeError: Mismatch in number of expected outputs\n'
'\n']}
Please advice if there is anything I can do to debug this issue further. Any help much appreciated! Thanks!
Hi @espentrydal
Thanks for reporting. I'm not sure what's causing this bug atm.
Could you provide the content of the pipeline's input BIDS folder (which should be the output folder of the ADNI to BIDS converter if I followed correctly) ? (the output of tree would do just fine as I'm mostly interested in the names of the files/folders).
I'm also wondering whether you converted your dataset to BIDS with the same version of Clinica (0.8.0) or whether this was the result of a previous conversion with an older version of clinica ?
Thanks for getting back to me. I've uploaded the output of tree of the bids-folder:
bids-folder.txt
The conversion was made with an earlier version, 0.7.x (not sure of the exact version).
The -tsv file listing the inputs:
❯ cat test_dwi.tsv
participant_id session_id
sub-ADNI002S0413 ses-M132
sub-ADNI002S1155 ses-M126
Thanks for the information @espentrydal ! 👍
I downloaded sub-ADNI002S0413 again and did the full analysis you're trying to do (adni-to-bids converter and dwi-preproc-using-t1 on session M132).
I first encountered an issue in the converter as it seems like one clinical data CSV file has now a different name than before (DXSUM_PDXCONV_ADNIALL seems to be named DXSUM_PDXCONV now... Did you noticed that too ?).
I'll open a dedicated issue tomorrow.
After this, I was able to reproduce the error, which occurs at the end of the DWI preprocessing pipeline.
I believe this is a bug I introduced while refactoring the pipeline a few months ago. 😓
I'm not sure why this wasn't caught by our non regression test suite (as this pipeline takes a lot of time and ressources to complete, it tends to fail a lot in our CI, so I wouldn't be surprised if it never finished successfully since I made the modifications...).
I'll work on fixing these two issues tomorrow and will make a patch release afterwards.
Thanks a lot @NicolasGensollen !
The data I downloaded uses the old name, so that must be new. But I had to remove a date-suffix that had been added to all clinical data csv-files to make the conversion work.
@espentrydal I also believe this is new.
The date-suffix issue should have been fixed in clinica 0.7.7.
@espentrydal I just made a patch release of Clinica (version 0.8.1) which should fix both issues (the one with the converter and the one with the DWI pipeline).
I'll close this but feel free to re-open if you're still facing the issue with 0.8.1.
Great, thanks!