singer-io/tap-postgres

Minimum date range not supported

Opened this issue · 1 comments

Source database has a date field with default values of "0001-01-01"

The tap fails when it hits one of these, with "Year 0 is out of range"

Happens on incremental.py, line 97

After some investigation, Python's minimum date is the epoch (1970). Postgres supports something like 4700 BC.

When the value is read from the database, psycopg2 attempts to map the postgres data type to the corresponding python data type. The value is below the minimum value supported by Python, so the conversion fails.

The psycopg2 best practice in this case is to write a custom type caster to identify the situation and resolve it. In this case, the resolution could be to just convert to a python string.

See https://www.psycopg.org/docs/advanced.html#type-casting-from-sql-to-python