Fetch 10 minute or hourly data
adamhsparks opened this issue · 5 comments
@PaulMelloy and I are working on another project that requires some wind data and Paul cobbled together, quite nicely, some code that used bomrang's sweep_for_stations() and get_met_data() from stationaRy to get hourly data for our research. This made me think, we can use stationaRy as a Depends Imports and bomrang can fetch sub-daily historical data.
After Jim Hester's rstudio::conf(2020L) talk I was wondering if Github Actions could be leveraged to collect this sort of data on your behalf, adding to a database every so often.
Oooh. Nice idea! We've been fetching the forecast data and saving it, that would be a good test case for this idea. Right now we just needed it for a single event, so no big deal.
However, I was thinking that we'd add this functionality to bomrang for others as well, so I opened this issue.
As I'm tinkering with this. I'm wondering. Do we simply adapt get_historical_weather() or come up with a new function?
get_historical_weather() seems reasonable, since it is historical values, however the BOM values and the NOAA values returned by the query differ, so depending on a param input you would get different columns in your dataframe depending on what you requested.
Any thoughts from anyone?
Very cool!
The only thing I'd mention by way of caution is that NOAA's data very likely has different standards to BOM's for a range of things, from the very simply (eg. missing data values and units) through to subtler things like reporting frequency and varying precision of station locations by country (I can think of at least one country that feeds into ISD with a single decimal point on station location, so their stations look like they're all on a grid 😆). Reporting frequency can be a headache in ISD.
If you link to this data, it'll be obviously different since it's subdaily, but we might want to consider documenting (or linking to documentation of) some of the differences that users might want to be wary of.