kujaku11/mt_metadata

MTime does not accept channel_summary timestamps

Closed this issue · 3 comments

Given an mth5 channel_summary dataframe, an example start column element is:

Timestamp('1980-01-01 00:00:00+0000', tz='UTC')

When I try to pass the start,end when asking for a run_ts, I get

ValueError: Time zone must be UTC

Even though the pandas Timestamp says it is UTC.

I checked the MTime() tests in

mt_metadata/tests/test_mttime.py

and couldn't see a test that uses pandas Timestamps. However, if I just pass MTime a normal pandas timestamp:

MTime(pd.Timestamp(1980,1,1,0,0,0,0.0000))
it works fine.

So it is something specficaly about the form of the timestamp that is inside the channel_summary

channel_summary_df uses this call to make the start column

pd.to_datetime(df.start.str.decode("utf-8"))

Works: MTime(pd.to_datetime("1980-01-01 00:00:00"))
Does not work: MTime(pd.to_datetime("1980-01-01 00:00:00+00:00"))

Here is a snippet showing how I worked around it for now.
This is in aurora/pipelines/process_mth5.py

def fix_time(tstmp):
    """
    One-off temporary workaround for mt_metadata issue #86

    Parameters
    ----------
    tstmp: pd.Timestamp
        Timestamp with a format that is resulting in ValueError: Time zone must be UTC

    Returns
    -------
    out: datetime.datetime
        The pandas timestamp as a datetime.datetime object
    """
    import datetime
    year = tstmp.year
    month = tstmp.month
    day = tstmp.day
    hour = tstmp.hour
    minute = tstmp.minute
    second = tstmp.second
    out = datetime.datetime(year, month,day,hour,minute,second)
    return out

@kkappler This should work now, closing for now