Improve the behavior of the standard load methods in targets
Opened this issue · 0 comments
pnadolny13 commented
Based on a discussion in target-postgres MeltanoLabs/target-postgres#266.
The current Singer convention in targets is to upsert records. If key properties are set then the target will update any records that exist instead of inserting.
Thoughts are it relates to the standard load_method
configs:
- Should the default actually be
upsert
? Or do we want to remove the default all together to make this config become more of an override to force the target to act a certain way vs a required value? - We should implement logic so that the target throws an error if someone selects
upsert
but theres no key properties available - If the default behavior of the SDK was always to upsert then we should change the logic to first check the load_method, if its set to
append-only
then dont upsert even if key properties are available. - Do existing targets get this functionality out of the box properly or is there anything they need to implement? If so, we should document it.