aramis-lab/clinica

Clinica Pipeline Compatibility Issue with Ants Package Versions

Closed this issue · 11 comments

souravraha commented

Describe the bug
The Clinica pipelines crash when using certain versions of the ants package, posing challenges in determining compatibility.

To Reproduce
Steps to reproduce the behavior:

  1. Install the development version of Clinica.
  2. Install either the latest version of ants (from source or via conda-forge) or version 2.3.1 (via conda-aramislab) as specified in the documentation.
  3. Execute a command such as clinica run pet-linear --save_pet_in_t1w_space -wd tmp bids CAPS 18FAV45 cerebellumPons2 -tsv subject_list_18FAV45.tsv.
  4. Observe the pipeline crashing or encountering errors.

Expected behavior
The Clinica pipeline should execute smoothly without encountering errors or crashes, irrespective of the version of the ants package being utilized.

Additional context
Issue #728 reports a similar error, but lacks specific solutions beyond the trial and error method with ants versions. This issue significantly impacts the usability and reliability of Clinica for users dependent on affected pipelines. Identifying the precise version of the ants package causing the issue would greatly aid users in resolving this compatibility problem.

NicolasGensollen commented

Hi @souravraha

Thanks for reporting this issue.
I am a bit surprised by the 2.3.1 version recommended by the documentation. We recommend using ANTs >= 2.5.0 as indicated on the "third-party" installation page of the documentation: https://aramislab.paris.inria.fr/clinica/docs/public/dev/Third-party/

After looking a bit, I think I see the issue. It seems like the link in the repo's README is not pointing to the latest version of the documentation but to an older version of the page I linked above.
Could you confirm that this is how you got there ?

In any case, I'm going to open a PR to fix this asap.

NicolasGensollen commented

@souravraha Actually there's no need for a PR. The links are OK but the corresponding documentation hosted on our server was out of sync for some reason. I made the update directly on the server such that the links of the README should now point to the correct version of the documentation (you might have to refresh or clear your browser's cache).

souravraha commented

Hello @NicolasGensollen,

Thank you for getting in touch! Upon conducting a search for "clinica documentation" on Google, I located the primary link directing to https://aramislab.paris.inria.fr/clinica/docs/public/latest/.

Encountering the error while utilizing the latest version of ANTs (2.5.1) was unexpected, particularly considering its flawless execution in the t1-linear pipeline I previously executed. That's when I consulted the documentation and experimented with version 2.3.1. However, the issue persists unresolved.

NicolasGensollen commented

@souravraha OK, all links with "latest" were pointing to an outdated version of the docs. This has just been fixed by manually updating the docs on the server. Using ANTs 2.5.1 is the correct move and should definitely work.

Could you provide more information relative to the error you're getting with ANTS 2.5.1 ? (OS name and version, Python version, Clinica version, command used, and the error traceback)

souravraha commented

@NicolasGensollen

Operating System: Ubuntu 20.04
Clinica Version: 0.7.7 (Development)
Python Version: 3.10 (Required by Clinica Development)

The command is mentioned in OP. Furthermore, the situation parallels that of issue #728, wherein T1 processing functions correctly while PET processing encounters errors, with an identical error traceback.

souravraha commented

@NicolasGensollen

I revisited the latest version of the ants package (compiled manually from source) and encountered some tracebacks that could aid in debugging:

For the command:

clinica run t1-linear bids CAPS -wd /DATA/user/tmp/ -tsv subject_list_t1.tsv

I received the following ouput:

...
2024-02-05 20:03:19,279:WARNING:sub-ADNI941S7074 | ses-M000
2024-02-05 20:03:19,279:WARNING:sub-ADNI941S7085 | ses-M000
2024-02-05 20:03:19,279:WARNING:Image(s) will be ignored by Clinica.
2024-02-05 20:03:19,314:INFO:The pipeline will be run on the following 2 image(s):
2024-02-05 20:03:19,314:INFO:   sub-ADNI098S6593 | ses-M012,
2024-02-05 20:03:19,314:INFO:   sub-ADNI114S2392 | ses-M120,
2024-02-05 20:03:19,314:INFO:The pipeline will last approximately 6 minutes per image.

Followed by:

Traceback (most recent call last):
  File "/DATA/user/miniconda3/envs/clinica/bin/clinica", line 6, in <module>
    sys.exit(main())
  File "/DATA1/user/clinica/clinica/cmdline.py", line 92, in main
    cli()
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/DATA1/user/clinica/clinica/pipelines/t1_linear/t1_linear_cli.py", line 65, in cli
    pipeline.run(plugin="MultiProc", plugin_args={"n_procs": n_procs})
  File "/DATA1/user/clinica/clinica/pipelines/engine.py", line 715, in run
    raise e
  File "/DATA1/user/clinica/clinica/pipelines/engine.py", line 696, in run
    exec_graph = Workflow.run(self, plugin, plugin_args, update_hash)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/nipype/pipeline/engine/workflows.py", line 638, in run
    runner.run(execgraph, updatehash=updatehash, config=self.config)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/nipype/pipeline/plugins/base.py", line 166, in run
    self._clean_queue(jobid, graph, result=result)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/nipype/pipeline/plugins/base.py", line 244, in _clean_queue
    raise RuntimeError("".join(result["traceback"]))
RuntimeError: Traceback (most recent call last):
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 524, in run
    result = self._run_interface(execute=True)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 642, in _run_interface
    return self._run_command(execute)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 750, in _run_command
    raise NodeExecutionError(
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node n4biascorrection.

Traceback (most recent call last):
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 454, in aggregate_outputs
    setattr(outputs, key, val)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/nipype/interfaces/base/traits_extension.py", line 330, in validate
    value = super(File, self).validate(objekt, name, value, return_pathlike=True)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/nipype/interfaces/base/traits_extension.py", line 135, in validate
    self.error(objekt, name, str(value))
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/traits/base_trait_handler.py", line 74, in error
    raise TraitError(
traits.trait_errors.TraitError: The 'output_image' trait of a N4BiasFieldCorrectionOutputSpec instance must be a pathlike object or string representing an
existing file, but a value of '/DATA/user/tmp/t1-linear/64732e3da010fb717cd736c4b3912d1d82b2a4b1/n4biascorrection/sub-ADNI114S2392_ses-M120_T1w_correc
ted.nii.gz' <class 'str'> was specified.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 401, in run
    outputs = self.aggregate_outputs(runtime)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 461, in aggregate_outputs
    raise FileNotFoundError(msg)
FileNotFoundError: No such file or directory '/DATA/user/tmp/t1-linear/64732e3da010fb717cd736c4b3912d1d82b2a4b1/n4biascorrection/sub-ADNI114S2392_ses-
M120_T1w_corrected.nii.gz' for output 'output_image' of a N4BiasFieldCorrection interface

For the command:

clinica run pet-linear --save_pet_in_t1w_space -wd /DATA/user/tmp/ bids CAPS 18FAV45 cerebellumPons2 -tsv subject_list_18FAV45.tsv

I encountered the following ouput:

2024-02-05 20:19:21,076:INFO:The pipeline will be run on the following 2947 image(s):
2024-02-05 20:19:21,077:INFO:   sub-ADNI002S0295 | ses-M060,
...
2024-02-05 20:19:21,078:INFO:The pipeline will last approximately 3 minutes per image.

followed by:

Traceback (most recent call last):
  File "/DATA/user/miniconda3/envs/clinica/bin/clinica", line 6, in <module>
    sys.exit(main())
  File "/DATA1/user/clinica/clinica/cmdline.py", line 92, in main
    cli()
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/DATA1/user/clinica/clinica/pipelines/pet_linear/pet_linear_cli.py", line 87, in cli
    pipeline.run(plugin="MultiProc", plugin_args={"n_procs": n_procs})
  File "/DATA1/user/clinica/clinica/pipelines/engine.py", line 715, in run
    raise e
  File "/DATA1/user/clinica/clinica/pipelines/engine.py", line 696, in run
    exec_graph = Workflow.run(self, plugin, plugin_args, update_hash)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/nipype/pipeline/engine/workflows.py", line 638, in run
    runner.run(execgraph, updatehash=updatehash, config=self.config)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/nipype/pipeline/plugins/base.py", line 166, in run
    self._clean_queue(jobid, graph, result=result)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/nipype/pipeline/plugins/base.py", line 244, in _clean_queue
    raise RuntimeError("".join(result["traceback"]))
RuntimeError: Traceback (most recent call last):
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/nipype/pipeline/plugins/multiproc.py", line 67, in run_node
    result["result"] = node.run(updatehash=updatehash)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 524, in run
    result = self._run_interface(execute=True)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 642, in _run_interface
    return self._run_command(execute)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/nipype/pipeline/engine/nodes.py", line 750, in _run_command
    raise NodeExecutionError(
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node antsRegistrationT1W2MNI.

Traceback (most recent call last):
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 454, in aggregate_outputs
    setattr(outputs, key, val)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/traits/trait_types.py", line 2695, in validate
    return TraitListObject(self, object, name, value)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/traits/trait_list_object.py", line 584, in __init__
    super().__init__(
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/traits/trait_list_object.py", line 213, in __init__
    super().__init__(self.item_validator(item) for item in iterable)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/traits/trait_list_object.py", line 213, in <genexpr>
    super().__init__(self.item_validator(item) for item in iterable)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/traits/trait_list_object.py", line 867, in _item_validator
    return trait_validator(object, self.name, value)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/nipype/interfaces/base/traits_extension.py", line 330, in validate
    value = super(File, self).validate(objekt, name, value, return_pathlike=True)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/nipype/interfaces/base/traits_extension.py", line 135, in validate
    self.error(objekt, name, str(value))
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/traits/base_trait_handler.py", line 74, in error
    raise TraitError(
traits.trait_errors.TraitError: Each element of the 'reverse_forward_transforms' trait of a RegistrationOutputSpec instance must be a pathlike object or st
ring representing an existing file, but a value of '/DATA/user/tmp/pet-linear/ca771a7355b4a9b78ecac7d044154c9314c19753/antsRegistrationT1W2MNI/transfo
rm0Warp.nii.gz' <class 'str'> was specified.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 401, in run
    outputs = self.aggregate_outputs(runtime)
  File "/DATA/user/miniconda3/envs/clinica/lib/python3.10/site-packages/nipype/interfaces/base/core.py", line 461, in aggregate_outputs
    raise FileNotFoundError(msg)
FileNotFoundError: No such file or directory '['/DATA/user/tmp/pet-linear/ca771a7355b4a9b78ecac7d044154c9314c19753/antsRegistrationT1W2MNI/transform0W
arp.nii.gz']' for output 'reverse_forward_transforms' of a Registration interface

I encountered similar issues yesterday when attempting to run the pet-linear pipeline following the t1-linear pipeline. Now I'm not sure whether to attribute these errors to the images rather than to the ants package or its installation.

NicolasGensollen commented

@souravraha let's look at t1-linear first.

If you look in the folder /DATA/user/tmp/t1-linear/64732e3da010fb717cd736c4b3912d1d82b2a4b1/n4biascorrection you should be able to find the command which was run by the Nipype interface (probably in a command.txt file and it should start with "N4BiasFieldCorrection ...").

Could you try running it directly in your terminal to see the error you get ?

  • 👍1
ghisvail commented
souravraha commented

Thank you for the guidance, @NicolasGensollen! I managed to locate the command in the command.txt file as suggested and tried running it directly in my terminal. However, I encountered an unexpected issue unrelated to the command itself.

It turns out the errors I've been experiencing with all versions of ANTs mentioned in the original post were caused by a rather peculiar culprit – a space in the name of my Dropbox folder! It's quite unbelievable how such a seemingly innocuous detail could lead to such headaches.

After relocating my BIDS and CAPS directories to a more appropriately named location, I still encountered an error related to missing shared libraries when using the conda-installed ANTs version (2.5.1). Specifically, I received the following message:

N4BiasFieldCorrection: error while loading shared libraries: libITKIOBruker-5.3.so.1: cannot open shared object file: No such file or directory

However, I'm pleased to report that upon manually compiling ANTs from source, even this error vanished into thin air. As a result, I've successfully executed the T1 pipeline and am currently in the process of running the PET pipeline.

Once again, thank you very much for your assistance! It's greatly appreciated.

ghisvail commented
souravraha commented

@ghisvail,
Certainly. Here's a refined breakdown of the steps I undertook:

  1. Initially, I manually compiled the latest version of ANTs to execute the T1 pipeline successfully.
  2. To ensure backup, I transferred my BIDS and CAPS folders to Dropbox. After this I started encountering issues (OP).
  3. Despite attempts to rectify the issue by switching between different versions such as conda-forge and aramislab, including versions like aramislab 2.3.1, the problem persisted.
  4. I shared the output related to the latest conda-forge version of ANTs on GitHub for further examination.
  5. With guidance from @NicolasGensollen, I pinpointed the root cause of the error, which was the space in the folder name.
  6. Despite relocating the folders, the pipelines continued to malfunction even with the latest conda-forge version of ANTs.
  7. Eventually, after all of this, I successfully executed the pipelines by reverting to the manual compilation method.

I trust this refined breakdown sheds light on the installation process and the hurdles faced during execution.