Better test environment for API requests
Closed this issue · 3 comments
Following up on the discussion in #72 . Running the full test suite leads to the package build time running in excess of 20 mins, mostly due to latency and/or connection issues with the FRED server. Using a more elegant web testing framework such as nealrichardson/httptest would handle test calls more efficiently.
Working on mocking API requests with nealrichardson/httptest. Running into error in basic setup so posting here for reference. Captured requests with httptest::capture_request() stored in tests/testthat but then wrapping requests with httptest::with_mock_api() throws an error
> getwd()
[1] "/Users/sboysel/repos/fredr/tests/testthat"
>
> devtools::load_all(here::here())
Loading fredr
> library(httptest)
> options(httptest.verbose=TRUE)
>
> skip_if_no_key <- function() {
+ testthat::skip_if(
+ condition = identical(Sys.getenv("FRED_API_KEY"), ""),
+ message = "FRED API key is not set."
+ )
+ }
>
> # mock paths
> httptest::.mockPaths()
[1] "."
>
> # capture
> httptest::start_capturing()
> readRenviron(here::here(".Renviron"))
> fredr::fredr_set_key(Sys.getenv("FRED_API_KEY"))
> fredr_category(category_id = 10L)
Writing /Users/sboysel/repos/fredr/tests/testthat/api.stlouisfed.org/fred/category-af4199.json
# A tibble: 1 x 3
id name parent_id
<int> <chr> <int>
1 10 Population, Employment, & Labor Markets 0
> httptest::stop_capturing()
>
> # test with mock api
> httptest::with_mock_api({
+ test_that("fredr_category()", {
+ skip_if_no_key()
+ expect_silent(ctg <- fredr_category(category_id = 10L))
+ })
+ })
Tracing function "form_file" as seen from package "httr"
Error: Test failed: 'fredr_category()'
* `ctg <- fredr_category(category_id = 10L)` produced messages.
Untracing function "form_file" in package "curl"
> So httptest::.mockPaths() are default (relative to tests/testthat). Can't see if I'm missing something fundamental from nealrichardson/httptest docs.
Edit: From the error message, perhaps with_mock_api() is generating messages causing expect_silent() to raise an error.
Another complication to testing: switching from httr::GET to httr::RETRY in fredr::fredr_request() introduced some lag in the testing suite given some tests expect failure for bad endpoints. By default, fredr::fredr_request() will run the request up to 10 times for any failure. I will expose a retry_times argument in fredr::fredr_request(), passed directly to the times argument of internal httr::RETRY. The default will be 10 but we can reduce the number of tries in expect_error() tests for which immediate failure is desired (e.g. bad endpoints, malformed requests).