Does migration re-upload everything?
jkawamoto opened this issue · 2 comments
It looks like GenericChunkUpdater.UpdateChunk uploads all shards regardless of how many shards need to be migrated. For example, even if only one host is replaced from the contract set and one shard needs to be migrated, we need to recalculate shards and upload them to all hosts. This re-upload also looks like it doesn't remove the previous sectors. That means migration produces garbage. Is that correct?
UpdateChunk calls UploadChunk, and implementations of UploadChunk should skip reuploading to hosts that are already present (search for skip in strategies.go). If you're using an OverdriveChunkUploader, you might generate garbage due to overdrive, but it definitely shouldn't be reuploading every shard. Let me know if you observe otherwise.
I found it, thanks. I was wondering that because sometimes all host return contract is locked by another party error. Maybe that is because DownloadChunk locks contracts rather than UploadChunk.