Timestamp milliseconds wrongly interpreted if milliseconds start with zeros
mirafranc opened this issue · 1 comments
mirafranc commented
Timestamp milliseconds wrongly interpreted if milliseconds start with zeros:
This issue occurred during Athena database testing (only 3 digits are supported for milliseconds). The timestamp type is recognised as 6 digits (but 3 digits set in the DB):
('endtime', DBAPITypeObject('TIMESTAMP'), 23, 23, 23, 6, 1)
- Millisecond values starting 0 are wrongly interpreted to 3 digit format:
2020-01-05 11:02:14.080 -> 2020-01-05 11:02:14.800 - Millisecond values starting numeric value are interpreted to 6 digit format:
2020-01-07 03:25:20.743 -> 2020-01-07 03:25:20.743000
mirafranc commented
can be easily fixed by following code update:
def _to_datetime(rs, col):
java_val = rs.getTimestamp(col)
if not java_val:
return
d = datetime.datetime.strptime(str(java_val)[:19], "%Y-%m-%d %H:%M:%S")
d = d.replace(microsecond=int(str(java_val.getNanos()).zfill(9)[:6]))
return str(d)
I tested the code using Athena, Oracle, Sap - milliseconds are correctly evaluated for timestamps