Extraction functions fail when parallelizing via foreach()
Closed this issue · 1 comments
ikb-rmbl commented
Right now it's not possible to run the extraction function (sdp_extract_data()
) inside an expression parallelized via foreach()
.
location_df <- data.frame(SiteName=c("Roaring Judy","Gothic","Galena Lake"),
Lat=c(38.716995,38.958446,39.021644),
Lon=c(-106.853186,-106.988934,-107.072569))
location_sv <- terra::vect(location_df,geom=c("Lon","Lat"),crs="EPSG:4327")
days <- seq(as.Date("2007-10-01"),as.Date("2007-10-31"),by="day")
cl <- parallel::makeCluster(2L)
doParallel::registerDoParallel(cl)
extr_list4 <- foreach::foreach(i=1:length(days),.packages=c("terra","devtools","rSDP")) %dopar% {
#devtools::load_all() ## During package development.
tmax4 <- rSDP::sdp_get_raster("R4D004",date_start=days[i],date_end=days[i],verbose=FALSE)
extr_dat <- rSDP::sdp_extract_data(tmax4,locations_proj,verbose=FALSE)[,3]
(extr_dat)
}
parallel::stopCluster(cl)
This gives an error:
: NULL value as symbol address.
I think it's related to this issue affecting functions in terra
more generally.
ikb-rmbl commented
Update: the problem with the code above is passing the SpatVector
objects to and from the parallel workers. It's possible to work around this by converting SpatVector
objects to sf
before sending them to the remote workers and for export back to the home process. I've updated the examples in the README and vignettes to reflect this.