rmbl-sdp/rSDP

Extraction functions fail when parallelizing via foreach()

Closed this issue · 1 comments

Right now it's not possible to run the extraction function (sdp_extract_data()) inside an expression parallelized via foreach().

location_df <- data.frame(SiteName=c("Roaring Judy","Gothic","Galena Lake"),
                          Lat=c(38.716995,38.958446,39.021644),
                          Lon=c(-106.853186,-106.988934,-107.072569))
location_sv <- terra::vect(location_df,geom=c("Lon","Lat"),crs="EPSG:4327")
days <- seq(as.Date("2007-10-01"),as.Date("2007-10-31"),by="day")

cl <- parallel::makeCluster(2L)
doParallel::registerDoParallel(cl)

extr_list4 <- foreach::foreach(i=1:length(days),.packages=c("terra","devtools","rSDP")) %dopar% {
  #devtools::load_all() ## During package development.
  tmax4 <- rSDP::sdp_get_raster("R4D004",date_start=days[i],date_end=days[i],verbose=FALSE)
  extr_dat <- rSDP::sdp_extract_data(tmax4,locations_proj,verbose=FALSE)[,3]
  (extr_dat)
}
parallel::stopCluster(cl)

This gives an error:

: NULL value as symbol address.

I think it's related to this issue affecting functions in terra more generally.

Update: the problem with the code above is passing the SpatVector objects to and from the parallel workers. It's possible to work around this by converting SpatVector objects to sf before sending them to the remote workers and for export back to the home process. I've updated the examples in the README and vignettes to reflect this.