Parallel runs try to write to the same file
chrisgorgo opened this issue · 1 comments
chrisgorgo commented
When running multiple C-PAC instances (one for each participant) on the same host they try to concurrently write to the same log file
Traceback (most recent call last):
File "/code/run.py", line 289, in <module>
plugin='MultiProc', plugin_args=plugin_args)
File "/usr/local/bin/miniconda/lib/python2.7/site-packages/CPAC/pipeline/cpac_runner.py", line 465, in run
create_group_log_template(sub_scan_map, c.logDirectory)
File "/usr/local/bin/miniconda/lib/python2.7/site-packages/CPAC/utils/utils.py", line 1866, in create_group_log_template
os.makedirs(reportdir)
File "/usr/local/bin/miniconda/lib/python2.7/os.py", line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 17] File exists: '/output/data/log/reports'
Possible solution would be to use UUID or participant label to write the log to a unique file for each run.
ccraddock commented
we changed the logfile name to fix this. let us know if the problem still happens