Need a better way to set freq for requests for recent data
Closed this issue · 2 comments
- HydroFunctions version: 0.1.8dev
- Python version: 3.7
Description
If you make a request without specifying a period or a date, you will get the most recent data. Sometimes, the most recent data is very old. Sometimes, one parameter is very old, and another is very recent.
Hydrofunctions has no way of figuring out the frequency of a table that only has one row, so it assumes that the frequency is 15 minutes.
It would be better to assume the frequency is 1 day for 'dv', and 15 minutes for 'iv'.
Better still, it shouldn't assign a frequency to these types of request. If it does, and then attempts to interpolate between two different dates, you can get some crazy results.
What I Did
problem = hf.NWIS('01541200')
problem.df().shape
>>>(2017441, 4)
This crazy site collected temperature for the last time in 1961, and it also has stream discharge for right now. So hydrofunctions set the frequency to 15 minutes and created a dataframe with a row for every 15 minutes from 1961 to 2019.
Sigh.
In the end, I decided to set the frequency to 0 minutes for these types of request. I might revisit this later, since it makes the NWIS.repr() return <0 * Minutes> as the frequency, when maybe might be better. I don't know. We'll see.
I added some markers in places where it is possible to recognize that the request is only for the most recent version possible. But the json also has some things in it that identifies this type of request.