[Broker] Batched uploads to s3
Opened this issue · 0 comments
AaronVasquez commented
https://github.com/aws/aws-sdk-go/blob/master/service/s3/s3manager/upload.go#L404
Instead of storing a JSON containing a batch, we can upload each chunk as a separate object in s3. We should mirror the logic of storing in badger as much as possible to make swapping between the different stores easy.
We should also mirror how we store chunks on the tangle so that download logic is similar.