Add clear keys parameter for finalize and sort and finalize staged data methods controlling what happens if an exception is thrown
Closed this issue · 0 comments
Is your feature request related to a problem? Please describe.
There are two strategies on failure. Either delete all staged data or orphan it and leave it to the user to delete it. Both have pros and cons.
If there is an exception thrown because of the staged data the user won't be able to proceed until the keys are deleted. Thus it makes sense to remove the keys. Otherwise the user must have try/catch clauses and perform custom deletes. On the other hand it would be annoying if a user was staging data for a long period of time and the data disappears due to a failure.
In any case append data keys must be deleted if the operation was successful.
Note: #1799 is making changes to how both finalize methods behave on failure as it clears all staged data if an exception is thrown.
Describe the solution you'd like
Add a function parameter to both python calls so that the user can decide on the strategy
Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
Additional context
Add any other context or screenshots about the feature request here.
Duplicated by #1849