How to specify RemoteData as output?

[bilke]

I want to run multiple consecutive shell jobs on a specific remote computer. The output data files of one job should be the input data files of the next.

I see that specifying a RemoteData as input is possible by specifying it in the nodes-dict. But how to specify the RemoteData as an output? I would like to use RemoteData because the data may be large and I would like to avoid transferring it to the local repository.

[sphuber]

The working directory of a job is automatically attached as a RemoteData node using the remote_folder link label. So you should be able to do the following:

results, node = launch_shell_job(
   ...
)

results, node = launch_shell_job(
    ...,
    nodes={
        'remote_previous_job': node.outputs.remote_folder,
    }
)

Note that there currently is a bug in the code that prevents this code from functioning properly as the submission script of the remote folder input are overwriting that of the new jobs script. See #58

I have fixed the problem in https://github.com/sphuber/aiida-shell/tree/feature/058/remote-folder-symlink but it requires a change in aiida-core that is not yet released. So for the time being you would also have to check out the main branch of aiida-core and install that.

It should be released soon in v2.6, which hopefully drops in a few weeks

[sphuber]

By the way, questions about usage (both aiida-core and aiida-shell) might be better suited on our Discourse forum: https://aiida.discourse.group/
There are a lot more people active there and this way the questions will also reach more users and can hopefully be useful to them as well.