sensedeep/dynamodb-onetable

BatchWrite does not handle Unprocessed Items correctly and ignores them

AlexStansfield opened this issue · 1 comments

Describe the bug

During a BatchWrite operation some writes may fail due to things like Throttling. The response will include the Unprocessed items so that they can be processed again to ensure they are written to the database.

The BatchWrite method in OneTable does have the code to handle this, but looks at the wrong place to get the Unprocessed Items so it never finds any and just continues without reprocessing them.

Note: I've only tested this with the SDK v3. I don't know if the same issues exists on v2.

To Reproduce

Write a large number (e.g 50,000) items into a table using a BatchWrite, because of 25 item max batch you'll need to do multiple requests. In order to ensure you trigger Throttling issue you can run multiple BatchWrites concurrently.

After the write has been complete check how many items were written into the table. If you experienced throttling then the number of items will be less than the number you intended.

Expected behavior

All items to be written to database.

I have a PR from April that highlights the minor changes required to fix it: #470

Thanks Alex, merged.