unexpected EOF when sync/cp files.
Opened this issue · 2 comments
I'm copying 2 dirs within the bucket, and some files I would get an "unexpected EOF" error, this was using the sync command, with only the --endpoint-url set, as we use Linode and not the very very pricey AWS.
Thinking that it might be the files, I tried a few files individually, no error, which confirms, not the file or the endpoints.
So I moved to the cp -n -u -s command, got this today.
Same "unexpected EOF" error on some files, but it spat out this as well.
panic: runtime error: invalid memory address or nil pointer dereference [signal SIGSEGV: segmentation violation code=0x1 addr=0x28 pc=0xa35623]
goroutine 81859 [running]:
github.com/peak/s5cmd/v2/command.Copy.shouldOverride({0xc0000a1400, 0xc0000a15e0, {0xb8d255, 0x2}, {0xc0000dcfc0, 0x68}, 0x0, 0x1, 0x1, 0x1, ...}, ...)
/home/runner/work/s5cmd/s5cmd/command/cp.go:850 +0x1a3
github.com/peak/s5cmd/v2/command.Copy.doDownload({0xc0000a1400, 0xc0000a15e0, {0xb8d255, 0x2}, {0xc0000dcfc0, 0x68}, 0x0, 0x1, 0x1, 0x1, ...}, ...)
/home/runner/work/s5cmd/s5cmd/command/cp.go:613 +0x118
github.com/peak/s5cmd/v2/command.Copy.prepareDownloadTask.func1()
/home/runner/work/s5cmd/s5cmd/command/cp.go:567 +0xf3
github.com/peak/s5cmd/v2/parallel.(*Manager).Run.func1()
/home/runner/work/s5cmd/s5cmd/parallel/parallel.go:57 +0x8a
created by github.com/peak/s5cmd/v2/parallel.(*Manager).Run
/home/runner/work/s5cmd/s5cmd/parallel/parallel.go:53 +0xca
It looks like a mem leak, and for large copies the application fails to release malloc to proceed in the cp process.
The system we are running this on has:
RAM: 8GB
CPU Cores: 2x2
Node: Proxmox VM
s5cmd version
v2.2.2-48f7e59
We are trying to cp 6+TB/25000 objects as an offsite backup location from s3 to local NFS store.
Experiencing the same error. Using different concurrency flags / different size buckets didn't help (bucket size ranging from 5Gi to 60Gi). I couldn't find any evidence of a memory leak.
The command was run through the image, using the same above version with 64 CPU and 128Gi of RAM, on openshift.
Experiencing the same error. Using different concurrency flags / different size buckets didn't help (bucket size ranging from 5Gi to 60Gi). I couldn't find any evidence of a memory leak.
The command was run through the image, using the same above version with 64 CPU and 128Gi of RAM, on openshift.
Updating: my problem was caused due to my object storage provider limiting the amount of connections to a bucket at the same time. When the concurrency rate surpassed it, that's when the error started.