[Bug] JSON parsing error in Redshift
Closed this issue · 1 comments
fivetran-reneeli commented
Is there an existing issue for this?
- I have searched the existing issues
Describe the issue
Error downstream in fivetran_platform_connector_status and fivetran_platform_audit_table due to the message_data the log table getting truncated due to lengthy json value. The incomplete json is causing the model runs to fail.
In order to support these json objects in Redshift the SUPER type is necessary but it is complicated.
As a workaround, we should update in the package to filter message_data first for only necessary keys prior to any parsing (event_subtype='status')
Relevant error log or model output
No response
Expected behavior
json to be parsed correctly and package to run
dbt Project configurations
na
Package versions
na
What database are you using dbt with?
redshift
dbt Version
na
Additional Context
No response
Are you willing to open a PR to help address this issue?
- Yes.
- Yes, but I will need assistance and will schedule time during our office hours for guidance
- No.
fivetran-catfritz commented
This has been addressed in release v1.5.0, which is now live on the dbt hub. Closing the issue!