Errors with vrt and warnings on unavailable resources with get_worldclim_precipitation and calc_precipitation_wc

[fBedecarrats]

Hello,
I'm runnning into some errors when running get_worldclim_precipitation and calc_precipitation_wc.
The following code:

library(tidyverse)
library(sf)
library(mapme.biodiversity) # This is CRAN version
library(geodata)

mada <- gadm("Madagascar", level = 0, resolution = 2, path = ".")
mada_dt <- mada %>%
  st_as_sf() %>%
  st_cast("POLYGON")

mada_dt <- mada_dt %>%
  get_resources(get_worldclim_precipitation(years = 2000:2023)) %>%
  calc_indicators(calc_precipitation_wc(engine = "extract", stats = "mean"))

produces the following errors:

Some target years are not available for prec.
Found a column named 'assetid'. Overwritting its values with a unique identifier.
Error in (function (.x, .y, .f, ..., .progress = FALSE)  : 
  ℹ In index: 1.
Caused by error:
ℹ In index: 1.
Caused by error in `purrr::map()`:
ℹ In index: 1.
Caused by error:
! [vrt] cannot create vrt. Error #0
In addition: Warning messages:
1: In CPL_gdaltranslate(source, destination, options, oo, config_options,  :
  GDAL Message 1: HTTP response code on https://biogeo.ucdavis.edu/data/worldclim/v2.1/hist/wc2.1_2.5m_prec_2000-2009.zip: 0
2: In CPL_gdaltranslate(source, destination, options, oo, config_options,  :
  GDAL Message 1: HTTP response code on https://biogeo.ucdavis.edu/data/worldclim/v2.1/hist/wc2.1_2.5m_prec_2010-2018.zip: 0
3: In r@ptr$make_vrt(x, options, opt) :
  GDAL Error 4: /tmp/RtmpHHOenM/file94d2b521153/worldclim_precipitation/wc2.1_2.5m_prec_2000-01.tif: No such file or directory
4: In r@ptr$make_vrt(x, options, opt) :
  GDAL Message 1: Can't open /tmp/RtmpHHOenM/file94d2b521153/worldclim_precipitation/wc2.1_2.5m_prec_2000-01.tif. Skipping it

[goergen95]

Hmm, the source seems to be unavailable. To me, this looks like a good opportunity to re-work the WorldClim resources/indicators to support any of the data sets listed here. Not sure yet when I will be able to do this.

Since you seem to be experimenting with the latest changes to the package I have some comments:

• there is no longer a need to cast to 'POLYGON', as 'MULTIPOLYGON' is now also supported
• you can use mapme_options() outdir argument to persist data on your disk, if you wish. By default, it points to the R session's temporal directory, so the package will attempt to re-download in new sessions. You can also set it to NULL which will not download any files, but leverage GDAL's virtual filesystem to read the data directly from the source data set. Performance will depend on the connection to the server, and we do temporal downloads for sources which do not support this approach (e.g. WorldPop).

[fBedecarrats]

Well received, thanks!

[duboisl-afd]

Hi,
I have the same error message when running get_nelson_et_al and calc_traveltime. Here is the error message I get when running this code :

 library(sf)
 library(mapme.biodiversity)
 outdir <- file.path(tempdir(), "mapme-data")
dir.create(outdir, showWarnings = FALSE)
 
 mapme_options(
     outdir = outdir,
    verbose = FALSE
 )
 
 aoi <- system.file("extdata", "sierra_de_neiba_478140_2.gpkg",
                    package = "mapme.biodiversity"
 ) %>%
   read_sf() %>%
   get_resources(get_nelson_et_al()) 

Error in sf::gdal_utils(util = util, source = source, destination = destination,  : 
  gdal_utils translate: an error occured
Warning messages:
1: In CPL_gdaltranslate(source, destination, options, oo, config_options,  :
  GDAL Error 1: TIFFFillStrip:Read error at scanline 9930; got 0 bytes, expected 19850
2: In CPL_gdaltranslate(source, destination, options, oo, config_options,  :
  GDAL Error 1: TIFFReadEncodedStrip() failed.
3: In CPL_gdaltranslate(source, destination, options, oo, config_options,  :
  GDAL Error 1: /vsicurl/https://figshare.com/ndownloader/files/14189831, band 1: IReadBlock failed at X offset 0, Y offset 9931: TIFFReadEncodedStrip() failed.

[goergen95]

Hi @fBedecarrats, just merged #317 with which we now support different resolution for the whole historical timeseries from 1960-2021. FYI, please also find PR #318, which is going to add precipitation data from CHELSA which is reported to have lower biases in areas of high altitude and low gauge station density.

[fBedecarrats]

This is great! Thanks @goergen95 !

[fBedecarrats]

Just to report after re-implementing this: it works now! Many thanks @goergen95 !
A note : I think that the documentation on the corresponding resource and indicator would need more details. I'll propose something (not now).

[goergen95]

Glad to hear that it works! Looking forward to your contribution. 👍

On a related note, I am looking forward to harmonize the different precipitation indicators. I am dissatisfied with the current solution and am planning to calculate area-weighted average precipitation via exactextractr. Would love to hear your suggestions, but I'd guess a dedicated issue or a discussion would be the right place.