local_slurm_array on Windows
Opened this issue · 0 comments
chjackson commented
I'm finding the rslurm
package very useful for making the workflow smoother for using HPC with R - thanks for developing it.
I've been developing submission scripts locally to make sure they are working before sending them to the server, but I found that local_slurm_array
didn't work on Windows. I think this was because it relied on bash
shell syntax to set the SLURM_ARRAY_TASK_ID
environment variable, that was not understood by the Windows command line.
I think this can be made system-independent by using the R functions Sys.setenv
and Sys.unsetenv
. An adaptation of this function that works for me is
local_slurm_array_portable <- function (slr_job, rscript_path = NULL)
{
if (!inherits(slr_job, "slurm_job")) {
stop("argument of local_slurm_array should be a slurm_job")
}
if (is.null(rscript_path)) {
rscript_path <- file.path(R.home("bin"), "Rscript")
}
slurm_run_path <- paste0("_rslurm_", slr_job$jobname)
wd <- getwd()
setwd(slurm_run_path)
for (i in 1:slr_job$nodes - 1) {
Sys.setenv(SLURM_ARRAY_TASK_ID=i)
system(paste0(rscript_path, " --vanilla slurm_run.R"))
}
Sys.unsetenv("SLURM_ARRAY_TASK_ID")
setwd(wd)
return(slr_job)
}