How to do multithreading for PyRate?

[BiodeB]

Dear Expert,

I'm new in PyRate and after the first 10 million run with the mG model the ESS values are very small. I think at least 100 million runs with multiple replicates may give some satisfactory outome. However, the first 10 million runs took more than 65 hrs. Therefore, I wonder to know that is it possible to do multithreading of PyRate.

Thanks a bunch,
Debajyoti

[dsilvestro]

Dear Debajyoti, Thank you fo your message. I am a bit surprised that 10M iterations took so long. Is PyRate's C++ library working on your machine? If it is you should see the message “Module FastPyRateC was loaded.” When you launch an analysis. If instead you get “Module FastPyRateC was not found.” It means that the library might need to be compiled specifically for your operating system. You can find more info here: https://github.com/dsilvestro/PyRate/tree/master/pyrate_lib/fastPyRateC <https://github.com/dsilvestro/PyRate/tree/master/pyrate_lib/fastPyRateC> Let me know if that was the issue! Best, Daniele

…

On 22 Feb 2021, at 13:13, Debajyoti Kabiraj ***@***.***> wrote: Dear Expert, I'm new in PyRate and after the first 10 million run with the mG model the ESS values are very small. I think at least 100 million runs with multiple replicates may give some satisfactory outome. However, the first 10 million runs took more than 65 hrs. Therefore, I wonder to know that is it possible to do multithreading of PyRate. Thanks a bunch, Debajyoti — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#26>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB2NORJY53MMPWUC7HIJPC3TAJC6ZANCNFSM4YAM34LQ>.

[BiodeB]

Dear Sir,

Thank you very much for your kind response, however I did according to your suggestion and invoked the program using following command

python PyRate.py dip_pbdb_dataSSF_PyRate.py -A 2 -mG -j 1 -n 50000000 -s 5000

And it is showing that

Module FastPyRateC was loaded. However, previously it was also showing similar message.
More essentially, it seems that -n and -s parameters are not working and by default the program sample the parameters every 1,000 iterations and it is displaying message...

PyRate.py:2373: DeprecationWarning: This function is deprecated. Please call randint(0, 1 + 1) instead
  ind=np.random.random_integers(0,len(M))
PyRate.py:2369: DeprecationWarning: This function is deprecated. Please call randint(0, 1 + 1) instead
  ind=np.random.random_integers(0,len(L))
PyRate.py:2369: DeprecationWarning: This function is deprecated. Please call randint(0, 2 + 1) instead
  ind=np.random.random_integers(0,len(L))
PyRate.py:2369: DeprecationWarning: This function is deprecated. Please call randint(0, 3 + 1) instead
  ind=np.random.random_integers(0,len(L))
PyRate.py:2369: DeprecationWarning: This function is deprecated. Please call randint(0, 4 + 1) instead
  ind=np.random.random_integers(0,len(L))
PyRate.py:2369: DeprecationWarning: This function is deprecated. Please call randint(0, 5 + 1) instead
  ind=np.random.random_integers(0,len(L))
PyRate.py:2373: DeprecationWarning: This function is deprecated. Please call randint(0, 2 + 1) instead
  ind=np.random.random_integers(0,len(M))
PyRate.py:2373: DeprecationWarning: This function is deprecated. Please call randint(0, 3 + 1) instead
  ind=np.random.random_integers(0,len(M))

Another thing I want know that approximately how much time require for 10 million iteration as my analysis included 2023 fossil record.

Thanks,
Debajyoti

[dsilvestro]

Thank you for pointing out this warning, I just pushed an update that should fix it.

Running 10M iterations on a standard dataset should definitely take less than 65 hours! Please note that the -A 4 algorithm (implementing RJMCMC) should be preferred over the -A 2.

If you send us the data by email we can take a look and see if there is any reason any particular issue with the input data.

Cheers,
Daniele