bug: a flaw in is_boolean_type causes _conform_primitive_property to coerce non-boolean fields to boolean
Opened this issue · 0 comments
Singer SDK Version
0.38.0
Is this a regression?
- Yes
Python Version
3.11
Bug scope
Taps (catalog, state, etc.)
Operating System
Ubuntu 22.04.4 LTS
Description
Multiple taps (tap-postgres, tap-universal-file) use monkeypatched logic for _conform_primitive_property
or is_boolean_type
. This is because the default logic for these functions causes non-boolean types to be coerced to either True
or False
.
For example, a field with a value of "abc"
and a jsonschema type of ["boolean", "string"]
would cause is_boolean_type()
to return True
. Then as a result, _conform_primitive_property()
would coerce "abc"
to boolean, resulting in that field's value being True
.
Here's the patched is_boolean_type
that I implemented for tap-universal-file: https://github.com/MeltanoLabs/tap-universal-file/blob/4850e7d933de1b2c4a8f623c5cb7fa089eb1f1a0/tap_universal_file/client.py#L21-L50
Related discussion of _conform_primitive_property
, but with regard to dates/datetimes instead of booleans: #1831
Code
No response