Error setup_r5(): Invalid number of points in LineString
Opened this issue · 2 comments
First of all, thank you so much for developing this very useful package!
I am working on calculating a travel time matrix (by car) between each "iris" (essentially neighborhoods) in France. Since the full .pbf file for France was too large for r5r to process (4.5 GB), I first filtered the file following guidelines from conveyal . Then, I implemented a solution where I select a 100 km buffer around each French department, calculate the bounding box, and crop the .pbf file accordingly.
For cropping the .pbf file, I used Osmosis as recommended in FAQ 4 of r5r. However, for several departments I encountered errors when setting up r5. Specifically, I was receiving this error: Error in setup_r5(...) : java.lang.NullPointerException.
According to these issues (1, 2), it seemed to me that this error may occur when ways and relations are excluded during cropping. Therefore, I modified the Osmosis cropping code by including clipIncompleteEntities=yes (as suggested here).
This approach has allowed me to compute the travel time matrix for most departments successfully. However, I am still encountering an issue with two departments (departments code==33 and 60). When setting up r5r using the pbf files related to these departments, I receive the following error:
r5r_core <- setup_r5("D:\\France\\network_60\\bo", verbose = FALSE)
No raster .tif files found. Using elevation = 'NONE'.
Using cached R5 version from C:/Users/Asus/AppData/Local/R/win-library/4.2/r5r/jar/r5-v6.9-all.jar
2024-11-11 17:10:06,587 [main] ERROR c.c.r.s.StreetLayer - Continuing to load but ignoring generalized costs due to exception: java.lang.RuntimeException: All ways are expected to have generalized cost tags. Missing: slope_1:forward
Error in setup_r5(sprintf("D:\\France\\network_%s\\bo", dep_code), verbose = FALSE) :
java.lang.IllegalArgumentException: Invalid number of points in LineString (found 1 - must be 0 or >= 2)This is the result form running the r5r::r5r_sitrep() function:
r5r::r5r_sitrep()
$r5r_package_version
[1] ‘1.0.1’
$r5_jar_version
[1] "6.9"
$java_version
[1] "11.0.20"
$set_memory
[1] "-Xmx12G"
$session_info
R version 4.2.2 (2022-10-31 ucrt)
Platform: x86_64-w64-mingw32/x64 (64-bit)
Running under: Windows 10 x64 (build 26100)
Matrix products: default
locale:
[1] LC_COLLATE=Italian_Italy.utf8 LC_CTYPE=Italian_Italy.utf8 LC_MONETARY=Italian_Italy.utf8
[4] LC_NUMERIC=C LC_TIME=Italian_Italy.utf8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] mapview_2.11.0 accessibility_1.2.0 fs_1.5.2 arrow_13.0.0 forcats_0.5.2
[6] stringr_1.5.0 dplyr_1.1.4 purrr_1.0.1 readr_2.1.3 tidyr_1.3.1
[11] tibble_3.2.1 ggplot2_3.4.0 tidyverse_1.3.2 sf_1.0-9 r5r_1.0.1
[16] rJava_1.0-6
loaded via a namespace (and not attached):
[1] httr_1.4.4 bit64_4.0.5 jsonlite_1.8.4 modelr_0.1.10
[5] Rdpack_2.4 assertthat_0.2.1 stats4_4.2.2 sp_1.5-1
[9] googlesheets4_1.0.1 cellranger_1.1.0 yaml_2.3.6 pillar_1.9.0
[13] backports_1.4.1 lattice_0.20-45 glue_1.6.2 digest_0.6.30
[17] checkmate_2.1.0 rbibutils_2.2.11 rvest_1.0.3 leaflet.providers_1.9.0
[21] colorspace_2.0-3 htmltools_0.5.5 pkgconfig_2.0.3 broom_1.0.1
[25] raster_3.6-11 haven_2.5.1 s2_1.1.1 webshot_0.5.4
[29] scales_1.3.0 satellite_1.0.4 terra_1.6-47 tzdb_0.4.0
[33] timechange_0.1.1 proxy_0.4-27 googledrive_2.0.0 generics_0.1.3
[37] ellipsis_0.3.2 withr_2.5.2 cli_3.4.1 magrittr_2.0.3
[41] crayon_1.5.2 readxl_1.4.1 fansi_1.0.3 xml2_1.3.3
[45] class_7.3-20 tools_4.2.2 data.table_1.14.6 hms_1.1.2
[49] gargle_1.2.1 lifecycle_1.0.3 munsell_0.5.0 reprex_2.0.2
[53] compiler_4.2.2 e1071_1.7-12 rlang_1.1.2 classInt_0.4-8
[57] units_0.8-0 grid_4.2.2 rstudioapi_0.14 htmlwidgets_1.5.4
[61] crosstalk_1.2.0 leafem_0.2.0 base64enc_0.1-3 wk_0.7.0
[65] codetools_0.2-18 gtable_0.3.1 DBI_1.1.3 R6_2.5.1
[69] lubridate_1.9.0 fastmap_1.1.0 bit_4.0.5 utf8_1.2.2
[73] KernSmooth_2.23-20 stringi_1.7.8 Rcpp_1.0.9 vctrs_0.6.5
[77] png_0.1-8 leaflet_2.1.1 dbplyr_2.2.1 tidyselect_1.2.0
Do you have any idea on waht might be causing this issue? I am including my code, and the cropped .pbf file for one of the two departments causing the error.
Thank you in advance for your help!
Here is the full r code:
options(java.parameters = '-Xmx12G')
options(scipen=999)
library(sf)
library(r5r)
library(tidyverse)
library(arrow)
library(fs)
library(accessibility)
library(mapview)
departement_fr <- st_read(here::here("coordinates\\ADMIN-EXPRESS-COG_1-1__SHP__FRA_2018-04-03\\ADMIN-EXPRESS-COG\\1_DONNEES_LIVRAISON_2018-03-28\\ADE-COG_1-1_SHP_LAMB93_FR\\DEPARTEMENT.shp"))
load("data\\iris_adm.Rdata")
load("data\\school_france_ttm.Rdata")
fr_iris <- st_transform(fr_iris, 4326)
fr_iris_tmp <- fr_iris %>%
select(code_iris)
centroids_fr <- as.data.frame(st_coordinates(st_centroid(fr_iris$geometry))) %>%
rename(lon = 1, lat = 2)
centroids_fr <- cbind(centroids_fr, fr_iris_tmp)
# List all unique department codes
unique_departments <- unique(departement_fr$INSEE_DEP)
# Initialize timing for the full process
total_start_time <- Sys.time()
# Initialize a vector to store department codes with errors
error_departments <- c()
for (dep_code in unique_departments ) {
cat(paste0("Processing department: ", dep_code, "\n"))
# Select current department
department_x <- departement_fr %>%
st_transform(4326) %>%
filter(INSEE_DEP == dep_code)
# Create a 100 km buffer around the selected department
buffer_100km <- st_buffer(department_x, dist = 100000) # 100 km
# Find all neighborhoods that intersect with this 100 km buffer
neighborhoods_in_buffer <- fr_iris %>%
st_filter(buffer_100km, .predicate = st_intersects)
# Compute bounding box for area
area_bbox <- st_bbox(neighborhoods_in_buffer)
area_bbox_sf <- st_as_sfc(area_bbox)
# Define paths for temporary files created by Osmosis
folder_path <- sprintf("D:\\France\\network_%s\\", dep_code)
dir.create(folder_path, recursive = TRUE)
smaller_pbf <- sprintf("D:\\France\\network_%s\\smaller_%s.pbf", dep_code,dep_code)
network_dat <- "D:\\France\\network\\network.dat"
network_settings <- "D:\\France\\network\\network_settings.json"
mapdb <- paste0(smaller_pbf, ".mapdb")
mapdb_p <- paste0(smaller_pbf, ".mapdb.p")
large_pbf_path <- "D:\\France\\ttm\\filtered.osm.pbf"
# Run Osmosis command to create a smaller .pbf file
osmosis_cmd <- sprintf("%s --read-pbf %s --bounding-box left=%s bottom=%s right=%s top=%s clipIncompleteEntities=yes --write-pbf %s",
"osmosis",
large_pbf_path,
as.numeric(area_bbox$xmin), as.numeric(area_bbox$ymin),
as.numeric(area_bbox$xmax), as.numeric(area_bbox$ymax),
smaller_pbf)
# Execute the Osmosis command
shell(osmosis_cmd, translate = TRUE)
gc()
# Try to set up r5r core with error handling
r5r_core <- tryCatch({
setup_r5(sprintf("D:\\France\\network_%s\\", dep_code), verbose = FALSE)
}, error = function(e) {
cat(paste0("Error in setting up r5r for department ", dep_code, ": ", e$message, "\n"))
# Append the department code to error_departments vector
error_departments <<- c(error_departments, dep_code)
return(NULL) # Return NULL if there's an error
})
# Skip to the next iteration if setup_r5 failed
if (is.null(r5r_core)) {
files_to_delete <- c(folder_path, smaller_pbf, network_dat, network_settings, mapdb, mapdb_p)
for (file_path in files_to_delete) {
if (file.exists(file_path)) {
file.remove(file_path)
cat(paste0("Deleted temporary file: ", file_path, "\n"))
}
}
next
}
stop_r5(r5r_core)
gc()
files_to_delete <- c(folder_path, smaller_pbf, network_dat, network_settings, mapdb, mapdb_p)
for (file_path in files_to_delete) {
if (file.exists(file_path)) {
file.remove(file_path)
cat(paste0("Deleted temporary file: ", file_path, "\n"))
}
}
gc()
}
total_end_time <- Sys.time()
cat(paste0("Total processing time: ", total_end_time - total_start_time, " seconds.\n"))
Hi @rafapereirabr , I’m sorry to bother you directly. I was wondering if you think this issue is not strictly related to r5r, and if I should seek help elsewhere. Many thanks again!
Hi @gmariani95 . I'm sorry for the slow response and thanks for such detailed explanation. We have never encountered this error message before. Basde on this part of the message "Invalid number of points in LineString (found 1 - must be 0 or >= 2)", it seems to me that there was a problem with one or more road segments when they got cropped.
I guess a simple solution would be to use a slightly larger or smaller buffer for these areas. Could you maybe try a 105 km buffer for these two areas and see if it works ?