abbat/ydcmd

read operation timed out

Closed this issue · 2 comments

Hi,
I have YdCMD running on more than 10 servers but never had this problem... I just made a new Server last week and setup cron to sync my backup folder to YDisk.
Everytime the Cron command runs, the File gets uploaded but also I receive in the cron Email.

Transfer: /backup/admin.2016-09-15.tar (2.97G) -> disk:/site/backup/adms.2016-09-15.tar
('The read operation timed out',)

Can you detect what can be the problem ?
I ran the command in --verbose --debug mode and here is the output.

`~# sudo /usr/local/bin/ydcmd --verbose --debug --rsync put /backup disk:/site/
--> GET https://cloud-api.yandex.net/v1/disk/resources?path=disk%3A%2Fsite%2Fbackup%2F&limit=0&offset=0
--> Connected to cloud-api.yandex.net:443 (TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256)
--> GET https://cloud-api.yandex.net/v1/disk/resources?path=disk%3A%2Fsite%2Fbackup%2F&limit=100&offset=0
--> Connected to cloud-api.yandex.net:443 (TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256)
Transfer: /backup/adms.2016-09-15.tar (2.97G) -> disk:/site/backup/adms.2016-09-15.tar
--> GET https://cloud-api.yandex.net/v1/disk/resources/upload?path=disk%3A%2Fsite%2Fbackup%2Fadms.2016-09-15.tar&overwrite=true
--> Connected to cloud-api.yandex.net:443 (TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256)
--> PUT https://uploader4j.disk.yandex.net:443/upload-target/xxxx.xxxx
--> File: /backup/adms.2016-09-15.tar
--> Connected to uploader4j.disk.yandex.net:443 (TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256)
^[[A^[[A^[[A^[[A^[[B^[[C^[[C^[[B^[[C^[[B^[[B^[[B^[[B^[[B^[[B^[[B--> Retry 1/3: ('The read operation timed out',)
--> GET https://cloud-api.yandex.net/v1/disk/resources/upload?path=disk%3A%2Fsite%2Fbackup%2Fadms.2016-09-15.tar&overwrite=true
--> Connected to cloud-api.yandex.net:443 (TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256)
--> PUT https://uploader16m.disk.yandex.net:443/upload-target/xxxx.xxxx
--> File: /backup/adms.2016-09-15.tar
--> Connected to uploader16m.disk.yandex.net:443 (TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256)
--> Retry 2/3: ('The read operation timed out',)
--> GET https://cloud-api.yandex.net/v1/disk/resources/upload?path=disk%3A%2Fsite%2Fbackup%2Fadms.2016-09-15.tar&overwrite=true
--> Connected to cloud-api.yandex.net:443 (TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256)
--> PUT https://uploader15g.disk.yandex.net:443/upload-target/xxxx.xxxx
--> File: /backup/adms.2016-09-15.tar
--> Connected to uploader15g.disk.yandex.net:443 (TLSv1.2 ECDHE-RSA-AES128-GCM-SHA256)
--> Retry 3/3: ('The read operation timed out',)
('The read operation timed out',)

`

abbat commented

Would you try --progress option? Is timeout raised on 100% upload or early? Also you can increase --timeout option (default 30 sec) to 60 for example.

P.S. Also I change your issue text to hide possible secret parts of URLs.

Thanks the --timeout=300 --retries=3 works fine now on this server. I noticed that's what I use on my other servers too.

I guess it is necessary to use --timeout option if Syncing large files to keep the connection ongoing. Can you include this tip in your documentation please ? Would be good for everyone.

Thanks