Multiple "successful" jobs do not end up with all files copied.
Opened this issue · 3 comments
I tried to copy a directory structure while it was being deleted (didn't actually know that was happening).
This resulted in 100+ errors of stale NFS file handles and file/dir not found. These were presented in the Motuz error box of the copy dialog.
At the bottom of the errors, rclone added "ERROR : Attempt 2/3 succeeded" and Motuz shows the job as "Successful" and green.
There were many files missing from the destination.
This worries me greatly - how many users glossed over errors, saw green, and deleted their source data to avoid charges?
3441
Just clarifying, based on some offline discussion....
Of course we cannot and should not guarantee that all files will copy in a scenario like this (where the source directory is being deleted during the copy).
The issue is motuz indicated green/success when not all files were copied.
This is still a tricky issue to know how to fix. We could, before the copy, make a list of all source files, and at the end of the copy, ensure that all those files exist in the destination. But that list could already be incomplete (relative to the pre-deletion source directory) by the time it is created, if a delete is in progress, and if it's a quick copy, or involves copying files that are not being deleted in that instant, we'd still see green (possibly), but all it really tells us is that some subset of files were copied, not everything we expected to copy (assuming we did not know about the delete).
We can and should improve the chances that we will detect this situation. I am not sure it is possible to do so 100% of the time.