Error in normalizePath at bq_table_download

[jj-9]

Hello all,

hopefully this will not be a dumb error on my side.

I'm facing a problem during the bq_table_download of a bq_dataset_query.

It seems to occur because of "withr::defer(file.remove(path_first_chunk))" because the tmp file is not found.
(line 137 of bq-download.R)

I'm having the error with :
R version 4.2.2 Patched (2022-11-10 r83330)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 20.04.5 LTS

When running under windows it works like a charm.

The resulting error is :

Complete
Billed: 0 B
Downloading first chunk of data.
Erreur dans normalizePath(path, mustWork = TRUE) : 
  path[1]="/tmp/RtmpeJDeHu/bq-download-207a84fe61ce0.json": Aucun fichier ou dossier de ce type
Message d'avis :
Dans file.remove(path_first_chunk) :
  impossible d'effacer le fichier '/tmp/RtmpeJDeHu/bq-download-207a84fe61ce0.json', à cause de 'Aucun fichier ou dossier de ce type'

# here, a reprex

library(bigrquery)

bq_auth(path = "~/mySA_auth_file.json")# replace this with your auth

billing <- bq_test_project() # replace this with your project ID
sql <- "SELECT year, month, day, weight_pounds FROM `publicdata.samples.natality`"

ds <- bq_dataset("publicdata", "samples")
tb <-
  bq_dataset_query(
    ds,
    query = sql,
    billing = billing
  )

rslt <- bq_table_download(tb)

```r sessionInfo()

#R version 4.2.2 Patched (2022-11-10 r83330)
#Platform: x86_64-pc-linux-gnu (64-bit)
#Running under: Ubuntu 20.04.5 LTS

#Matrix products: default
#BLAS: /usr/lib/x86_64-linux-gnu/blas/libblas.so.3.9.0
#LAPACK: /usr/lib/x86_64-linux-gnu/lapack/liblapack.so.3.9.0

#locale:

[1] LC_CTYPE=fr_FR.UTF-8 LC_NUMERIC=C LC_TIME=fr_FR.UTF-8 LC_COLLATE=fr_FR.UTF-8

[5] LC_MONETARY=fr_FR.UTF-8 LC_MESSAGES=fr_FR.UTF-8 LC_PAPER=fr_FR.UTF-8 LC_NAME=C

[9] LC_ADDRESS=C LC_TELEPHONE=C LC_MEASUREMENT=fr_FR.UTF-8 LC_IDENTIFICATION=C

#attached base packages:
#[1] stats graphics grDevices utils datasets methods base

#other attached packages:
#[1] httr_1.4.4 gargle_1.3.0 bigrquery_1.4.1

#loaded via a namespace (and not attached):

[1] Rcpp_1.0.10 rstudioapi_0.14 magrittr_2.0.3 hms_1.1.2 progress_1.2.2 bit_4.0.5 R6_2.5.1

[8] rlang_1.0.6 fansi_1.0.4 tools_4.2.2 utf8_1.2.3 cli_3.6.0 DBI_1.1.3 withr_2.5.0

#[15] ellipsis_0.3.2 askpass_1.1 openssl_2.0.5 bit64_4.0.5 assertthat_0.2.1 tibble_3.1.8 lifecycle_1.0.3
#[22] crayon_1.5.2 brio_1.1.3 vctrs_0.5.2 fs_1.6.1 curl_5.0.0 glue_1.6.2 compiler_4.2.2
#[29] pillar_1.8.1 prettyunits_1.1.1 jsonlite_1.8.4 pkgconfig_2.0.3

</details>

[jj-9]

Hello again,

I found that the problem is not where I thought, the multi_run actually gets back a 401 error from google which I don't understand because I was authenticated as I've been able to run the query job :

{
  "error": {
    "code": 401,
    "message": "Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.",
    "errors": [
      {
        "message": "Login Required.",
        "domain": "global",
        "reason": "required",
        "location": "Authorization",
        "locationType": "header"
      }
    ],
    "status": "UNAUTHENTICATED",
    "details": [
      {
        "@type": "type.googleapis.com/google.rpc.ErrorInfo",
        "reason": "CREDENTIALS_MISSING",
        "domain": "googleapis.com",
        "metadata": {
          "method": "google.cloud.bigquery.v2.TableDataService.List",
          "service": "bigquery.googleapis.com"
        }
      }
    ]
  }
}

The bq_download_chunk_handle function does valorize the :
headers <- signed$headers
And there's actually a bearer in the handle headers

Best regards,
JJ

[jj-9]

Hello,

after some debug I found that the lines of the bq_download_chunk_handle function :

  h <- curl::new_handle(url = url)
  curl::handle_setopt(h, useragent = bq_ua())
  curl::handle_setheaders(h, .list = headers)

Don't apply the existing values to the handle, only the URL is set.
(ie header is raw(0))

So ... may be some kind of issue from curl (!)

regards
JJ

[hadley]

Unfortunately I can't reproduce your error, so I can't fix it, but I can make the spurious errors about file deletions go away.