Need for future.batchtools package
jl5000 opened this issue · 1 comments
I'm creating a drake project using parallel processing (locally, not using a cluster) and the user manual has left me a bit uncertain as to which packages I actually require. The future
package is directed to be installed, so that is clear. However, I'm not sure if "be sure to install future.batchtools
" is a necessity or a suggestion. If it is required, it's also not clear whether it needs to be loaded with library()
or just needs to be installed.
future.batchtools
is only required for make(parallelism = "future")
on clusters, and no library()
call is required for either future
or future.batchtools
. In your case, you do not need future.batchtools
. However, the future.callr
package has more reliable backend infrastructure than the default backends.
# install.packages("future")
# install.packages("future.callr")
future::plan(future.callr::callr, workers = 2)
make(plan, parallelism = "future", jobs = 2)
Even better, clustermq
has a multicore backend, and for drake
I think it will be your fastest option.
# install.packages("clustermq")
options(clustermq.scheduler = "multicore")
make(plan, parallelism = "clustermq", jobs = 2)