Slow upload speeds and eventual error "The write operation timed out"
Closed this issue · 2 comments
I am seeing very odd behaviour with ia upload in that most of the time I get very slow upload speed.
My internet speed is 1MB/s upload which I only get around 1 in 4 tries with ia.
Each upload do (~400MB CD-ROM .ISO.7z file) should take a handful of minutes to upload.
Even when I do get 1.0s/MB (as reported in ia) the next upload will most likely be slow.
Uploads show as taking multiple hours.
Eventually, I will get "The write operation timed out".
I've checked and I have no other devices using my internet at the time of upload.
Speed test shows my connection is operating as fast or faster than advertised by my ISP.
It's an inconvenience that I cannot leave the process to run automatically due to these crashes/failures.
How can I debug or resolve this issue?
Setup
- internetarchive/ia 3.0.2
- Linux
Related?
Tail end of crash output:
increment
raise six.reraise(type(error), error, _stacktrace)
File "/volume1/@appstore/py3k/usr/local/lib/python3.8/site-packages/urllib3/packages/six.py", line 769, in reraise
raise value.with_traceback(tb)
File "/volume1/@appstore/py3k/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "/volume1/@appstore/py3k/usr/local/lib/python3.8/site-packages/urllib3/connectionpool.py", line 398, in _make_request
conn.request(method, url, **httplib_request_kw)
File "/volume1/@appstore/py3k/usr/local/lib/python3.8/site-packages/urllib3/connection.py", line 239, in request
super(HTTPConnection, self).request(method, url, body=body, headers=headers)
File "/volume1/@appstore/py3k/usr/local/lib/python3.8/http/client.py", line 1294, in request
self._send_request(method, url, body, headers, encode_chunked)
File "/volume1/@appstore/py3k/usr/local/lib/python3.8/http/client.py", line 1340, in _send_request
self.endheaders(body, encode_chunked=encode_chunked)
File "/volume1/@appstore/py3k/usr/local/lib/python3.8/http/client.py", line 1289, in endheaders
self._send_output(message_body, encode_chunked=encode_chunked)
File "/volume1/@appstore/py3k/usr/local/lib/python3.8/http/client.py", line 1084, in _send_output
self.send(chunk)
File "/volume1/@appstore/py3k/usr/local/lib/python3.8/http/client.py", line 1006, in send
self.sock.sendall(data)
File "/volume1/@appstore/py3k/usr/local/lib/python3.8/ssl.py", line 1204, in sendall
v = self.send(byte_view[count:])
File "/volume1/@appstore/py3k/usr/local/lib/python3.8/ssl.py", line 1173, in send
return self._sslobj.write(data)
urllib3.exceptions.ProtocolError: ('Connection aborted.', timeout('The write operation timed out'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/volume1/@appstore/py3k/usr/local/bin//ia", line 33, in <module>
sys.exit(load_entry_point('internetarchive==3.0.2', 'console_scripts', 'ia')())
File "/volume1/@appstore/py3k/usr/local/lib/python3.8/site-packages/internetarchive/cli/ia.py", line 171, in main
sys.exit(ia_module.main(argv, session))
File "/volume1/@appstore/py3k/usr/local/lib/python3.8/site-packages/internetarchive/cli/ia_upload.py", line 298, in main
r = _upload_files(item, local_file, upload_kwargs_copy, prev_identifier,
File "/volume1/@appstore/py3k/usr/local/lib/python3.8/site-packages/internetarchive/cli/ia_upload.py", line 95, in _upload_files
response = item.upload(files, **upload_kwargs)
File "/volume1/@appstore/py3k/usr/local/lib/python3.8/site-packages/internetarchive/item.py", line 1190, in upload
resp = self.upload_file(body,
File "/volume1/@appstore/py3k/usr/local/lib/python3.8/site-packages/internetarchive/item.py", line 1006, in upload_file
response = self.session.send(prepared_request,
File "/volume1/@appstore/py3k/usr/local/lib/python3.8/site-packages/internetarchive/session.py", line 543, in send
raise e
File "/volume1/@appstore/py3k/usr/local/lib/python3.8/site-packages/internetarchive/session.py", line 540, in send
reraise_modify(e, e.request.url, prepend=False) # type: ignore
File "/volume1/@appstore/py3k/usr/local/lib/python3.8/site-packages/internetarchive/session.py", line 537, in send
r = super().send(request, **kwargs)
File "/volume1/@appstore/py3k/usr/local/lib/python3.8/site-packages/requests/sessions.py", line 701, in send
r = adapter.send(request, **kwargs)
File "/volume1/@appstore/py3k/usr/local/lib/python3.8/site-packages/requests/adapters.py", line 547, in send
raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: (ProtocolError('Connection aborted.', timeout('The write operation timed out')), 'https://s3.us.archive.org/macpeople-1999-07-01/MACPEOPLE-1999-07-01.ISO.7z')
uploading MACPEOPLE-1999-07-01.ISO.7z: 51%|█████ | 205/401 [2:49:38<2:42:11, 49.65s/MiB]
I did a traceroute and saw things were being held up at my ISP servers. I wondered if they were penalising me for using one of their old modems. So I switched to the most recent one they gave me and everything is uploading full speed with no slowing down.
Even with my new router the timeout still happens, but it takes a lot longer to do so.
Will reopen if I can reproduce.