peak/s5cmd

cp --show-progress command stops at 50% as completed when copying files (single or multiple)

Opened this issue · 1 comments

josephroosen commented

Running s5cmd v2.2.2 with the build from the releases for 64bit linux on Ubuntu 22.04 latest updates. I am sending large folders of files to Digital Ocean Spaces and getting some strange behavior from the cp --show-progress feature. I am noticing sometimes there is no feedback at all and the bar stays blank or shows no transfer happening and keeps counting down and other times it is going seemingly normal and then just stops at 50% but all files copied as expected. It seems to be a bug of some sort. This happens with multiple files in the folder being copied or just one file.

example 1

s5cmd -stat cp --show-progress . s3://bucket/path/path2/some.other.path/
50.00%  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━───────────────────────────────────  11.02 GB / 22.04 GB (8.78 MB/s) 20m55s (32/32)

Operation       Total   Error   Success
cp              1       0       1

example 2

s5cmd cp --show-progress . s3://bucket/path/path2/some.other.path/
50.00%  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━────────────────────────────────────  387.71 MB / 775.40 MB (1.13 MB/s) 5m43s (3/3)

example 3

s5cmd cp --show-progress . s3://bucket/path/path2/some.other.path/
50.00%  ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━───────────────────────────────────  164.76 MB / 329.51 MB (740.28 kB/s) 3m43s (1/1)

In the example above the file being copied is in fact 329.51MB so it isn't like it is double counting the total space.

I have tried using it with or without the -stat function in front of it with no difference but as you can tell from example 1 above, it seems to count all 32 files as 1 cp operation. I am not quite sure what to make of it.

Thanks for any help you can provide here.

lm-jkominek commented

Sorry for the necro, but I just noticed this exact same thing, so I came here to see if it's something that occurred to others. It got me so worried that I actually went back to using gsutil to upload important files to my GCP bucket, to be 100% sure that no content was lost...

Hopefully this is just a small book-keeping bug rather than anything serious. I checked and it happens in 2.2.2 (current), 2.2.1 and 2.2.0, when --show-progress was added.

  • 👍1