Parallel Execution Throws numerous RuntimeErrors

[M-Colley]

Hello there, I have installed the newest version of PhySO today and used conda as described here.

Without changing anything at the config or for using parallelization, the program runs into numerous:

anaconda3\envs\physo\Lib\multiprocessing\spawn.py", line 138, in _check_not_importing_main
raise RuntimeError('''
RuntimeError:
An attempt has been made to start a new process before the
current process has finished its bootstrapping phase.

    This probably means that you are not using fork to start your
    child processes and you have forgotten to use the proper idiom
    in the main module:

        if __name__ == '__main__':
            freeze_support()
            ...

    The "freeze_support()" line can be omitted if the program
    is not going to be frozen to produce an executable.

I have now disabled this via

physo.physym.reward.USE_PARALLEL_EXE = False
physo.physym.reward.USE_PARALLEL_OPTI_CONST = False

I am on a Windows 11 host system with Python 3.11. Any idea why this could happen?

[WassimTenachi]

Hi @M-Colley,

Windows and MACs require this guard before running in parallel mode :
if __name__ == '__main__':

Also these systems can not run in parallel mode from a notebook.

There is a functional example that uses this and that should work perfectly on any system here: https://github.com/WassimTenachi/PhySO/blob/main/demo/demo_mechanical_energy/demo_mechanical_energy.py

This is due to Windows and MACs systems "spawning" new processes, this can be checked via:

import torch.multiprocessing as mp
mp_start_method = mp.get_start_method()

This is kind of unfortunate but I have played with it quite some time and I am pretty sure there is no way around it...

Does this answer your question ?

Wassim

[M-Colley]

Thanks for the clarification!