DOI-USGS/dataretrieval-python

Different date format

Closed this issue · 4 comments

jpouliot10 commented

There is an issue when we try to pull data from multiple sites when time series have two differents date format.

The function read_json() try to keep the default date format. In the first case the type of the date is an object and in the second case it is a dateformat64. There is a mismatch when we try to merge two dataframe with different object type.

2020-01-01T00:00:00.000-06:00 (type is Object)
2020-01-01 00:00:00.000-06:00 (type is datetime64)

I fixed it with the "keep_default_dates=False".

        record_df = pd.read_json(record_json,
                                 orient='records',
                                 dtype={'value': 'float64',
                                        'qualifiers': 'unicode'},
                                 keep_default_dates=False)

Thanks

thodson-usgs commented

Can you provide an example? I need to determine whether this change effects any other code.

jpouliot10 commented
jpouliot10 commented

Hi,
Try the station : 011277905 for the date 2010-07-20 to 2010-07-20
Error message: --- Timezones don't match. 'pytz.FixedOffset(-300)' != 'pytz.FixedOffset(-240)'

Another exemple is :
Station : 02405500 for the date 2010-03-12 to 2010-03-21
Error message : You are trying to merge on object and datetime64[ns, pytz.FixedOffset(-300)] columns. If you wish to proceed you should use pd.concat

elbeejay commented

This might be related to #60, and as such the issue might be resolved by #62