baztian/jaydebeapi

Timestamp milliseconds wrongly interpreted if milliseconds start with zeros

mirafranc opened this issue · 1 comments

Timestamp milliseconds wrongly interpreted if milliseconds start with zeros:

This issue occurred during Athena database testing (only 3 digits are supported for milliseconds). The timestamp type is recognised as 6 digits (but 3 digits set in the DB):
('endtime', DBAPITypeObject('TIMESTAMP'), 23, 23, 23, 6, 1)

  • Millisecond values starting 0 are wrongly interpreted to 3 digit format:
    2020-01-05 11:02:14.080 -> 2020-01-05 11:02:14.800
  • Millisecond values starting numeric value are interpreted to 6 digit format:
    2020-01-07 03:25:20.743 -> 2020-01-07 03:25:20.743000

can be easily fixed by following code update:

def _to_datetime(rs, col):
    java_val = rs.getTimestamp(col)
    if not java_val:
        return
    d = datetime.datetime.strptime(str(java_val)[:19], "%Y-%m-%d %H:%M:%S")
    d = d.replace(microsecond=int(str(java_val.getNanos()).zfill(9)[:6]))
    return str(d)

I tested the code using Athena, Oracle, Sap - milliseconds are correctly evaluated for timestamps