How to parallelize independent runs?

[dsgranadosv]

Hi! I am using CONAN for making contact maps for multiple independent trajectories generated by clustering analysis. I have tried to build a bash script for analyzing all the systems at the same time using GNU Parallel with no luck. My problem relies in the folders that Conan writes for every run (frames, matrices, aggregate, etc) which are created in the folder of the bash script instead of being created in the the folder where the CONAN input file is located and executed

Do you know how to avoid this behaviour? Do you recommend any particular approach for running in parallel several conan inputs?

Thanks in advance!

[csabadaday]

Hi Diego, I would just rewrite the bash script such that CONAN is run from the proper working directory each time. This is what I do when I run it on several trajectories. In a way, it makes more sense this way, since usually I only have 1 input file and 10 trajectories, not vice versa. So something like (assuming you choose Protein-H): for i in {1..10} do cd $i echo 2 | conan.py ../conan.inp cd ..; done Parallel runs will not really help, since the main bottleneck of conan is file I/O (in particular, reading in the big dmf.xpm file) and that is, as far as I understand, done in serial. Another thing you can do is run gmx mdmat only as a first step, and then unite the dmf.xpm files (concatenate). If you want to make the time information unique, you can use the option "PATCH_TIME yes". Once you are happy with your conan input, just stop the session and copy the gmx command (every gmx command repeats the command on the screen). This will make the run faster, assuming you want to average your quantities across independent runs, or want to cluster all frames, or .... Let me know if these are good solutions to your problem. Best, ?Csaba Dr. Csaba Daday Postdoc Molecular Biomechanics HITS gGmbH Schloss-Wolfsbrunnenweg 35 69118 Heidelberg Germany phone +49 6221 533 311 fax +49 6221 533 298 email: csaba.daday@h-its.org http://www.h-its.org<http://www.h-its.org/>

…

_________________________________________________ Amtsgericht Mannheim / HRB 337446 Managing Director: Dr. Gesa Schönberger Scientific Director: Prof. Dr. Michael Strube

________________________________ From: Diego Granados <notifications@github.com> Sent: Monday, May 7, 2018 4:06 AM To: HITS-MBM/conan Cc: Subscribed Subject: [HITS-MBM/conan] How to parallelize independent runs? (#1) Hi! I am using conan for make contact maps for multiple independent trajectories generated by clustering analysis. I have tried to build a bash script for analyzing all the systems at the same time using GNU Parallel with no luck. My problem relies in the folders that Conan writes for every run (frames, matrices, aggregate, etc) which are created in the folder of the bash script instead of being created in the the folder where the conan input file is located. Do you know how to avoid this behaviour? Do you recommend any particular approach for running in parallel several conan inputs? Thanks in advance! - You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub<#1>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AclzuoB3KYlp7kn-JnJ7CwDSJAL2KQWsks5tv6wdgaJpZM4T0TfN>.

[csabadaday]

oops forgot to close this one. TL;DR: I/O doesn't allow much parallelism anyway so I recommend running conan 'in series.' In most cases, unless you're plotting many frames (in which case parallel processing again wouldn't help), it's fast enough.