Make FTP connector more resilient by retrying after a timeout?
Closed this issue · 1 comments
niconoe commented
I recently found the following in the log file (and it seems less files were subsequently transferred at the time):
Traceback (most recent call last):
File "/home/ubuntu/data-repository/file_transfer/baltrad_to_s3.py", line 54, in <module>
sys.exit(main())
File "/home/ubuntu/data-repository/file_transfer/baltrad_to_s3.py", line 23, in main
btos.transfer(name_match="_vp_", overwrite=False, limit=None)
File "/home/ubuntu/data-repository/file_transfer/datamover/transporters.py", line 92, in transfer
self.ftp._ftp.retrbinary('RETR ' + filename, f.write)
File "/home/ubuntu/miniconda2/envs/enramtransfer/lib/python3.5/ftplib.py", line 441, in retrbinary
with self.transfercmd(cmd, rest) as conn:
File "/home/ubuntu/miniconda2/envs/enramtransfer/lib/python3.5/ftplib.py", line 398, in transfercmd
return self.ntransfercmd(cmd, rest)[0]
File "/home/ubuntu/miniconda2/envs/enramtransfer/lib/python3.5/ftplib.py", line 360, in ntransfercmd
source_address=self.source_address)
File "/home/ubuntu/miniconda2/envs/enramtransfer/lib/python3.5/socket.py", line 712, in create_connection
raise err
File "/home/ubuntu/miniconda2/envs/enramtransfer/lib/python3.5/socket.py", line 703, in create_connection
sock.connect(sa)
TimeoutError: [Errno 110] Connection timed out
peterdesmet commented
This should be investigated a part of vptstools, closing here.