ncw/swift

ObjectCopy and ObjectMove do not understand large objects

Opened this issue · 1 comments

Connection.ObjectCopy (and, therefore, Connection.ObjectMove) issues only a single COPY request to copy an object. However, this leads to large objects being concatenated into a single object. The copy operation is therefore needlessly expensive, or may even fail if the object becomes too large because of the concatenation.

ObjectCopy and ObjectMove should (either automatically or upon explicit choice of the caller) detect large objects and copy only the manifest instead of the entire object.

This issue is especially severe because, at least for SLOs, the caller cannot work around this problem because of #101.

ncw commented

I'd say that if you asked for a copy, then the segments should be copied too, otherwise you've just duplicated the manifest which doesn't preserve the data in the event of a delete, say.

Likewise the move should probably move the segments too.

Having ObjectCopy and ObjectMove do this automatically seems desirable. I'm a bit concerned about the copy or move only half completeing though...

I'm not sure of the right solution here.