RProtoBuf vignette example: Unserializing httr::POST() request to OpenCPU results in `character(0)`
philipp-baumann opened this issue · 9 comments
Dear RProtoBuf development team,
First of all, thanks for the excellent R interface and the good documentation that you provide to Protocol Buffers. Thanks also to Jeroen Ooms for the awesome OpenCPU platform. To me it looks like the below issue I observed might also be related to httr or OpenCPU, but I post it here because it is included in the RProtoBuf vignette. Hope it's OK to open an issue here.
I was testing the following the example "8.2. HTTP GET: Retrieving an R object":
# see `vignette("RProtoBuf-paper")`
# requires httr >= 0.2.99
library("httr")
library("RProtoBuf")
args <- list(n = 42, mean = 100)
payload <- serialize_pb(args, NULL)
req <- httr::POST(
url = "https://public.opencpu.org/ocpu/library/stats/R/rnorm/pb",
body = payload,
httr::add_headers(
"Content-Type" = "application/x-protobuf")
)
# This should be the output of stats::rnorm(n = 42, mean = 100)
output <- unserialize_pb(req$content)Instead of getting an atomic vector of type "double", I get:
> print(output)
character(0)
The Content-Type entry in the request seems to be correctly updated:
> req$request$headers
Accept "application/json, text/xml, application/xml, */*" Content-Type "application/x-protobuf"
When I use
protolite::unserialize_pb(req$content)I get the following error message: "Error in cpp_unserialize_pb(msg) : Failed to parse protobuf message".
Here is the structure of the response list:
`> str(req)
List of 10
$ url : chr "https://public.opencpu.org/ocpu/library/stats/R/rnorm/pb"
$ status_code: int 400
$ headers :List of 18
..$ date : chr "Wed, 18 Apr 2018 07:18:39 GMT"
..$ content-type : chr "text/plain; charset=utf-8"
..$ content-length : chr "80"
..$ connection : chr "keep-alive"
..$ access-control-allow-origin : chr ""
..$ access-control-expose-headers : chr "Location, X-ocpu-session, Content-Type, Cache-Control"
..$ access-control-allow-headers : chr "Origin, Content-Type, Accept, Accept-Encoding, Cache-Control, Authorization"
..$ access-control-allow-credentials: chr "true"
..$ x-ocpu-r : chr "R version 3.4.4 (2018-03-15)"
..$ x-ocpu-locale : chr "en_US.UTF-8"
..$ x-ocpu-time : chr "2018-04-18 07:18:39 UTC"
..$ x-ocpu-version : chr "2.0.7"
..$ x-ocpu-server : chr "rApache"
..$ vary : chr "Accept-Encoding"
..$ content-encoding : chr "gzip"
..$ expect-ct : chr "max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct\""
..$ server : chr "cloudflare"
..$ cf-ray : chr "40d55bcf0ef29c35-AMS"
..- attr(, "class")= chr [1:2] "insensitive" "list"
$ all_headers:List of 1
..$ :List of 3
.. ..$ status : int 400
.. ..$ version: chr "HTTP/1.1"
.. ..$ headers:List of 18
.. .. ..$ date : chr "Wed, 18 Apr 2018 07:18:39 GMT"
.. .. ..$ content-type : chr "text/plain; charset=utf-8"
.. .. ..$ content-length : chr "80"
.. .. ..$ connection : chr "keep-alive"
.. .. ..$ access-control-allow-origin : chr ""
.. .. ..$ access-control-expose-headers : chr "Location, X-ocpu-session, Content-Type, Cache-Control"
.. .. ..$ access-control-allow-headers : chr "Origin, Content-Type, Accept, Accept-Encoding, Cache-Control, Authorization"
.. .. ..$ access-control-allow-credentials: chr "true"
.. .. ..$ x-ocpu-r : chr "R version 3.4.4 (2018-03-15)"
.. .. ..$ x-ocpu-locale : chr "en_US.UTF-8"
.. .. ..$ x-ocpu-time : chr "2018-04-18 07:18:39 UTC"
.. .. ..$ x-ocpu-version : chr "2.0.7"
.. .. ..$ x-ocpu-server : chr "rApache"
.. .. ..$ vary : chr "Accept-Encoding"
.. .. ..$ content-encoding : chr "gzip"
.. .. ..$ expect-ct : chr "max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct\""
.. .. ..$ server : chr "cloudflare"
.. .. ..$ cf-ray : chr "40d55bcf0ef29c35-AMS"
.. .. ..- attr(, "class")= chr [1:2] "insensitive" "list"
$ cookies :'data.frame': 1 obs. of 7 variables:
..$ domain : chr "#HttpOnly_.opencpu.org"
..$ flag : logi TRUE
..$ path : chr "/"
..$ secure : logi FALSE
..$ expiration: POSIXct[1:1], format: "2019-04-18 08:53:30"
..$ name : chr "__cfduid"
..$ value : chr "db99b77dc04437119a50898ed6ba335791524034410"
$ content : raw [1:60] 50 4f 53 54 ...
$ date : POSIXct[1:1], format: "2018-04-18 07:18:39"
$ times : Named num [1:6] 0 0.00509 0.02521 0.25854 0.64512 ...
..- attr(, "names")= chr [1:6] "redirect" "namelookup" "connect" "pretransfer" ...
$ request :List of 7
..$ method : chr "POST"
..$ url : chr "https://public.opencpu.org/ocpu/library/stats/R/rnorm/pb"
..$ headers : Named chr [1:2] "application/json, text/xml, application/xml, /" "application/x-protobuf"
.. ..- attr(, "names")= chr [1:2] "Accept" "Content-Type"
..$ fields : NULL
..$ options :List of 4
.. ..$ useragent : chr "libcurl/7.43.0 r-curl/3.0 httr/1.3.1"
.. ..$ post : logi TRUE
.. ..$ postfieldsize: int 58
.. ..$ postfields : raw [1:58] 08 05 42 0c ...
..$ auth_token: NULL
..$ output : list()
.. ..- attr(, "class")= chr [1:2] "write_memory" "write_function"
..- attr(, "class")= chr "request"
$ handle :Class 'curl_handle'
- attr(*, "class")= chr "response"`
Here is my SessionInfo() output:
`> sessionInfo()
R version 3.4.2 (2017-09-28)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: OS X El Capitan 10.11.6
Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] RProtoBuf_0.4.11 httr_1.3.1
loaded via a namespace (and not attached):
[1] compiler_3.4.2 protolite_1.7 R6_2.2.2 tools_3.4.2 RCurl_1.95-4.10
[6] curl_3.0 yaml_2.1.15 Rcpp_0.12.16 knitr_1.17 jsonlite_1.5
[11] bitops_1.0-6 `
Do you have any idea why deserialization of the OpenCPU request to https://demo.ocpu.io/stats/R/rnorm/pb does not work as expected in my setup (related to httr or OpenCPU server API)?
Thanks a lot for your support!
Best,
Philipp
@jeroen Can you take a look? I can confirm Philipp's problem from here:
R> library("httr")
R> library("RProtoBuf")
R> args <- list(n=42, mean=100)
R> payload <- serialize_pb(args, NULL)
R> payload
[1] 08 05 42 0c 08 02 12 08 00 00 00 00 00 00 45 40 42 0c 08 02 12 08 00 00 00 00 00 00 59 40 5a 05 6e 61 6d 65 73 62 13 08 00 2a 05 0a 01 6e 10 00 2a 08 0a 04 6d 65 61 6e 10 00
R> req <- POST (
+ url = "https://demo.ocpu.io/stats/R/rnorm/pb",
+ body = payload,
+ add_headers (
+ "Content-Type" = "application/x-protobuf"
+ )
+ )
R> output <- unserialize_pb(req$content)
R> print(output)
character(0)
R> Is that a change on the OpenCPU side?
Thanks. It looks like I changed the content type to remove the x- prefix from the content type. It should be either "application/protobuf" or "application/rprotobuf". Try this:
library("httr")
library("RProtoBuf")
args <- list(n = 42, mean = 100)
payload <- serialize_pb(args, NULL)
req <- httr::POST(
url = "https://cloud.opencpu.org/ocpu/library/stats/R/rnorm/pb",
body = payload,
httr::add_headers(
"Content-Type" = "application/protobuf")
)
httr::stop_for_status(req)
output <- unserialize_pb(req$content)I have no recollection of making this change, but apologies anyway :)
Oh and while we're at it, the demo server changed to https://cloud.opencpu.org/.
I'll update the vignette real quick.
There are some more ocpu.org examples in the vignette which seem to have an issue. Can you poke?
Great, thanks a lot for the quick response and the update/fixes! :) Looking forward to continue exploring possibilities of R server application development with OpenCPU and RProtoBuf.
The only other opencpu example seems to work for me?
There are more of the old URLs as well:
edd@rob:~/git/rprotobuf/vignettes(master)$ ag ocpu
RProtoBuf-paper.Rnw
1246: GET https://demo.ocpu.io/MASS/data/Animals/pb
1269:req <- GET('https://demo.ocpu.io/MASS/data/Animals/pb')
1293:req = urllib2.Request('https://demo.ocpu.io/MASS/data/Animals/pb')
1332: url = "https://cloud.opencpu.org/ocpu/library/stats/R/rnorm/pb",
edd@rob:~/git/rprotobuf/vignettes(master)$ If it works, fine. No need to wake sleeping dogs.
Yes that all seems fine. Thanks.