Athena read_sql_query with pyarrow backend trims time in timestamp
Aleksei-Poliakov opened this issue · 0 comments
Aleksei-Poliakov commented
Describe the bug
Running this query:
wr.athena.read_sql_query("SELECT TIMESTAMP '2024-06-24 9:30:51'", dtype_backend='pyarrow')
yields 2024-06-24 instead of 2024-06-24 09:30:51. It seems like timestamp from Athena is mapped to date64[pyarrow] instead of timestamp[ns][pyarrow]
How to Reproduce
wr.athena.read_sql_query("SELECT TIMESTAMP '2024-06-24 9:30:51'", dtype_backend='pyarrow')
Expected behavior
The result should be similar to running with numpy backend:
wr.athena.read_sql_query("SELECT TIMESTAMP '2024-06-24 9:30:51'")
which correctly gives back 2024-06-24 09:30:51
Your project
No response
Screenshots
No response
OS
Linux
Python version
3.12
AWS SDK for pandas version
3.8.0
Additional context
No response