meltano/sdk

feat: Add rollback support for overwrite load method

pnadolny13 opened this issue · 1 comments

Originally driven by #1783 which was closed by #1893 but doesnt have safe backup/rollback support for the overwrite method.

The current status of this is described in #1893 (comment) and a draft PR is in #1941.

The overwrite load method currently drops the table before attempting to load so if the sync fails then the user is left with an empty table (or no table if recreate fails). This issue is to create a workflow to:

  • alter existing table to back it up instead of dropping it
  • attempt to recreate the table and load the new data
  • if success, delete the back up table
  • if failed, delete the new table and rename the backup table to the original name

This has been marked as stale because it is unassigned, and has not had recent activity. It will be closed after 21 days if no further activity occurs. If this should never go stale, please add the evergreen label, or request that it be added.