mapbox/dynamodb-replicator

Even more logging

Closed this issue · 1 comments

Debugging failed lambda invocations is a difficult thing. Some insights on what we need to log better:

Replication function

  • We will only run one replication per key, even if that key is affected more than once. We need a list of the unique keys, and a cross-checkable list of the keys that were affected
  • Keep a running count of number of records that have been successfully replicated for quick and easy comparison

Backup function

  • We run each change, even if that means running more than once per unique key. This means we need a list of each change-key combo, cross-checkable against the changes that have been implemented
  • I'm wondering if we should try and use a setTimeout to actually print a list of changes that failed to be implemented in 58s or something.
  • consider, for each change/key combination in an invocation, logging an md5sum of it. Its difficult to search cloudwatch logs for JSON objects, which is what you'd like to to in order to confirm that a change/key was retried.

Closed by #43