bcgov/shinyrems

test if rems duckdb works with shinyrems

Closed this issue ยท 4 comments

New test branch with rems called rems duckdb created to deal with vroom size error when downloading historic data. rems duckdb branch successfully downloads historic data as per readme; then shinyrems can be launched. This is the only way to launch app currently. sqlite way of downloading historic data using current shinyrems shinyrems::run_ems_app("all") results in vroom size error.
`Error: The size of the connection buffer (131072) was not large enough
to fit a complete line:

  • Increase it by setting Sys.setenv("VROOM_CONNECTION_SIZE")

I have gone through the different branches and functions testing it several times. Here is a general outline of what was done to test the new duckdb branch to confirm shinyrems still launches correctly.

  • launched the app with the current master branch of rems and ran shinyrems::run_ems_app("all")
    • app launched and worked as expected
  • cleared the data cache using remove_data_cache("all")
  • launched the app by running shinyrems::run_ems_app("historic")
    • app launched and worked as expected
  • cleared the data cache using remove_data_cache("all")
  • launched the app by running shinyrems::run_ems_app("2yr")
    • app launched and worked as expected
  • cleared the data cache using remove_data_cache("all")



  • built the rems package locally using the duckdb branch and ran shinyrems::run_ems_app("all")
    • app launched and worked as expected
  • cleared the data cache using remove_data_cache("all")
  • launched the app by running shinyrems::run_ems_app("historic")
    • app launched and worked as expected
  • cleared the data cache using remove_data_cache("all")
  • launched the app by running shinyrems::run_ems_app("2yr")
    • app launched and worked as expected
  • cleared the data cache using remove_data_cache("all")

I went through these steps a couple times as I was unable to recreate the vroom size error in any of my runs. The odd time I did get an error in downloading the data but these errors appeared to be due to download issues from the internet flickering in and out, since the download takes a while a small break in connectivity would cause the download to error out.

In summary it appears the duckdb branch of rems works to launch the shinyrems app.

This is great, thanks! Please let me know then if I should merge that branch to master @HeatherGranger @KarHarker @aylapear. I'd love to wrap that up.

@ateucher @KarHarker @aylapear yes it looks like we're good to merge that branch to master. Thanks for following up!

Hi all - I have merged the duckdb to master now bcgov/rems#63

Thanks Andy and Ayla!