Error when downloading TinyTaskonomy
AvivSham opened this issue · 0 comments
AvivSham commented
Hi All,
I followed the README file and run the following command after succecfully installing aria2 and omnidata-tools.
It seems like for some files the connection to the server is broken.
Here is the trace:
[DL:97MiB][#dcfcb1 0B/7.4GiB(0%)][#a82352 450MiB/821MiB(54%)][#d9c894 335MiB/2.4GiB(13%)][#c72378 265MiB/1.1GiB(21%)]
11/01 17:36:58 [NOTICE] Download complete: compressed//segment_unsup25d__taskonomy__almena.tar
*** Download Progress Summary as of Tue Nov 1 17:38:32 2022 ***
=====================================================================================================================
[#dcfcb1 0B/7.4GiB(0%) CN:1 DL:0B]
FILE: compressed//rgb__taskonomy__alfred.tar
---------------------------------------------------------------------------------------------------------------------
[#a82352 697MiB/821MiB(84%) CN:16 DL:0B]
FILE: compressed//depth_euclidean__taskonomy__almena.tar
---------------------------------------------------------------------------------------------------------------------
[#d9c894 583MiB/2.4GiB(23%) CN:16 DL:0B]
FILE: compressed//edge_texture__taskonomy__almena.tar
---------------------------------------------------------------------------------------------------------------------
[#c72378 502MiB/1.1GiB(41%) CN:16 DL:0B]
FILE: compressed//keypoints2d__taskonomy__almena.tar
---------------------------------------------------------------------------------------------------------------------
[#abb076 856MiB/1.8GiB(46%) CN:16 DL:0B]
FILE: compressed//keypoints3d__taskonomy__almena.tar
---------------------------------------------------------------------------------------------------------------------
[#5042be 1.1GiB/1.8GiB(60%) CN:16 DL:0B]
FILE: compressed//normal__taskonomy__almena.tar
---------------------------------------------------------------------------------------------------------------------
[#4ec56e 456MiB/1.7GiB(25%) CN:16 DL:0B]
FILE: compressed//principal_curvature__taskonomy__almena.tar
---------------------------------------------------------------------------------------------------------------------
[#c90ad6 358MiB/2.8GiB(12%) CN:16 DL:0B]
FILE: compressed//rgb__taskonomy__almena.tar
---------------------------------------------------------------------------------------------------------------------
[#26a78e 770MiB/1.0GiB(71%) CN:16 DL:0B]
FILE: compressed//reshading__taskonomy__almena.tar
---------------------------------------------------------------------------------------------------------------------
[#34d64e 0B/9.4GiB(0%) CN:1 DL:0B]
FILE: compressed//rgb_large__taskonomy__almena.tar
---------------------------------------------------------------------------------------------------------------------
[#e8a3d4 0B/60MiB(0%) CN:1 DL:0B]
FILE: compressed//segment_unsup2d__taskonomy__almena.tar
---------------------------------------------------------------------------------------------------------------------
[#0efe21 0B/55MiB(0%) CN:1 DL:0B]
FILE: compressed//class_object__taskonomy__almota.tar
---------------------------------------------------------------------------------------------------------------------
[#bb4e30 0B/2.7GiB(0%) CN:1 DL:0B]
FILE: compressed//edge_texture__taskonomy__almota.tar
---------------------------------------------------------------------------------------------------------------------
[DL:0B][#dcfcb1 0B/7.4GiB(0%)][#a82352 697MiB/821MiB(84%)][#d9c894 583MiB/2.4GiB(23%)][#c72378 502MiB/1.1GiB(41%)][#a[FAILURE] Uncaught error when processing model https://datasets.epfl.ch/taskonomy/almena_keypoints3d.tar (stacktrace below)
[FAILURE] Uncaught error when processing model https://datasets.epfl.ch/taskonomy/almena_rgb_large.tar (stacktrace below)
[FAILURE] Uncaught error when processing model https://datasets.epfl.ch/taskonomy/almena_edge_texture.tar (stacktrace below)
[FAILURE] Uncaught error when processing model https://datasets.epfl.ch/taskonomy/almena_depth_euclidean.tar (stacktrace below)
[FAILURE] Uncaught error when processing model https://datasets.epfl.ch/taskonomy/almota_edge_texture.tar (stacktrace below)
[FAILURE] Uncaught error when processing model https://datasets.epfl.ch/taskonomy/almena_rgb.tar (stacktrace below)
[FAILURE] Uncaught error when processing model https://datasets.epfl.ch/taskonomy/almena_normal.tar (stacktrace below)
[FAILURE] Uncaught error when processing model https://datasets.epfl.ch/taskonomy/almena_keypoints2d.tar (stacktrace below)
[FAILURE] Uncaught error when processing model https://datasets.epfl.ch/taskonomy/almota_class_object.tar (stacktrace below)
[FAILURE] Uncaught error when processing model https://datasets.epfl.ch/taskonomy/almena_reshading.tar (stacktrace below)
[FAILURE] Uncaught error when processing model https://datasets.epfl.ch/taskonomy/alfred_rgb.tar (stacktrace below)
[FAILURE] Uncaught error when processing model https://datasets.epfl.ch/taskonomy/almena_principal_curvature.tar (stacktrace below)
[FAILURE] Uncaught error when processing model https://datasets.epfl.ch/taskonomy/almena_segment_unsup2d.tar (stacktrace below)
multiprocess.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/opt/conda/lib/python3.8/site-packages/urllib3/connectionpool.py", line 449, in _make_request
six.raise_from(e, None)
File "<string>", line 3, in raise_from
File "/opt/conda/lib/python3.8/site-packages/urllib3/connectionpool.py", line 444, in _make_request
httplib_response = conn.getresponse()
File "/opt/conda/lib/python3.8/http/client.py", line 1344, in getresponse
response.begin()
File "/opt/conda/lib/python3.8/http/client.py", line 307, in begin
version, status, reason = self._read_status()
File "/opt/conda/lib/python3.8/http/client.py", line 268, in _read_status
line = str(self.fp.readline(_MAXLINE + 1), "iso-8859-1")
File "/opt/conda/lib/python3.8/socket.py", line 669, in readinto
return self._sock.recv_into(b)
socket.timeout: timed out
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.8/site-packages/requests/adapters.py", line 439, in send
resp = conn.urlopen(
File "/opt/conda/lib/python3.8/site-packages/urllib3/connectionpool.py", line 787, in urlopen
retries = retries.increment(
File "/opt/conda/lib/python3.8/site-packages/urllib3/util/retry.py", line 550, in increment
raise six.reraise(type(error), error, _stacktrace)
File "/opt/conda/lib/python3.8/site-packages/urllib3/packages/six.py", line 770, in reraise
raise value
File "/opt/conda/lib/python3.8/site-packages/urllib3/connectionpool.py", line 703, in urlopen
httplib_response = self._make_request(
File "/opt/conda/lib/python3.8/site-packages/urllib3/connectionpool.py", line 451, in _make_request
self._raise_timeout(err=e, url=url, timeout_value=read_timeout)
File "/opt/conda/lib/python3.8/site-packages/urllib3/connectionpool.py", line 340, in _raise_timeout
raise ReadTimeoutError(
urllib3.exceptions.ReadTimeoutError: HTTPConnectionPool(host='localhost', port=6800): Read timed out. (read timeout=60.0)
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/opt/conda/lib/python3.8/site-packages/multiprocess/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/opt/conda/lib/python3.8/site-packages/omnidata_tools/dataset/download.py", line 298, in process_model
raise e
File "/opt/conda/lib/python3.8/site-packages/omnidata_tools/dataset/download.py", line 283, in process_model
tar_fpath = download_tar(
File "/opt/conda/lib/python3.8/site-packages/omnidata_tools/dataset/download.py", line 161, in download_tar
success = wait_on(aria2api, res)
File "/opt/conda/lib/python3.8/site-packages/omnidata_tools/dataset/download.py", line 187, in wait_on
while not (a2api.get_downloads([gid])[0].is_complete or a2api.get_downloads([gid])[0].has_failed):
File "/opt/conda/lib/python3.8/site-packages/aria2p/api.py", line 298, in get_downloads
downloads.append(Download(self, self.client.tell_status(gid)))
File "/opt/conda/lib/python3.8/site-packages/aria2p/client.py", line 877, in tell_status
return self.call(self.TELL_STATUS, [gid, keys]) # type: ignore
File "/opt/conda/lib/python3.8/site-packages/aria2p/client.py", line 262, in call
return self.res_or_raise(self.post(payload))
File "/opt/conda/lib/python3.8/site-packages/aria2p/client.py", line 358, in post
return requests.post(self.server, data=payload, timeout=self.timeout).json()
File "/opt/conda/lib/python3.8/site-packages/requests/api.py", line 119, in post
return request('post', url, data=data, json=json, **kwargs)
File "/opt/conda/lib/python3.8/site-packages/requests/api.py", line 61, in request
return session.request(method=method, url=url, **kwargs)
File "/opt/conda/lib/python3.8/site-packages/requests/sessions.py", line 542, in request
resp = self.send(prep, **send_kwargs)
File "/opt/conda/lib/python3.8/site-packages/requests/sessions.py", line 655, in send
r = adapter.send(request, **kwargs)
File "/opt/conda/lib/python3.8/site-packages/requests/adapters.py", line 529, in send
raise ReadTimeout(e, request=request)
requests.exceptions.ReadTimeout: HTTPConnectionPool(host='localhost', port=6800): Read timed out. (read timeout=60.0)
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/opt/conda/bin/omnitools.download", line 8, in <module>
sys.exit(download())
File "/opt/conda/lib/python3.8/site-packages/fastcore/script.py", line 119, in _f
return tfunc(**merge(args, args_from_prog(func, xtra)))
File "/opt/conda/lib/python3.8/site-packages/omnidata_tools/dataset/download.py", line 304, in download
errors = list(tqdm.tqdm(p.imap(process_model, models), total=len(models)))
File "/opt/conda/lib/python3.8/site-packages/tqdm/_tqdm.py", line 1060, in __iter__
for obj in iterable:
File "/opt/conda/lib/python3.8/site-packages/multiprocess/pool.py", line 868, in next
raise value
requests.exceptions.ReadTimeout: None: None
1%|▉ | 137/10600 [05:12<6:37:42, 2.28s/it]
root@a804f8c0dde0:/images#
11/01 17:38:41 [NOTICE] Shutdown sequence commencing... Press Ctrl-C again for emergency shutdown.
[DL:50KiB][#dcfcb1 0B/7.4GiB(0%)][#a82352 697MiB/821MiB(84%)][#d9c894 583MiB/2.4GiB(23%)][#c72378 502MiB/1.1GiB(41%)]
11/01 17:38:44 [NOTICE] Download GID#dcfcb1a904b54c49 not complete: compressed//rgb__taskonomy__alfred.tar
11/01 17:38:44 [NOTICE] Download GID#a82352ebd95c4aec not complete: compressed//depth_euclidean__taskonomy__almena.tar
11/01 17:38:44 [NOTICE] Download GID#d9c89480fd02a3cb not complete: compressed//edge_texture__taskonomy__almena.tar
11/01 17:38:44 [NOTICE] Download GID#c723789b4a0b6657 not complete: compressed//keypoints2d__taskonomy__almena.tar
11/01 17:38:44 [NOTICE] Download GID#abb076fee190d5d8 not complete: compressed//keypoints3d__taskonomy__almena.tar
11/01 17:38:44 [NOTICE] Download GID#5042be4145f66b55 not complete: compressed//normal__taskonomy__almena.tar
11/01 17:38:44 [NOTICE] Download GID#4ec56efd51f8b934 not complete: compressed//principal_curvature__taskonomy__almena.tar
11/01 17:38:44 [NOTICE] Download GID#c90ad6e2c8686ae1 not complete: compressed//rgb__taskonomy__almena.tar
11/01 17:38:44 [NOTICE] Download GID#26a78e95921374b3 not complete: compressed//reshading__taskonomy__almena.tar
Download Results:
gid |stat|avg speed |path/URI
======+====+===========+=======================================================
dcfcb1|INPR| 0B/s|compressed//rgb__taskonomy__alfred.tar
a82352|INPR| 2.7MiB/s|compressed//depth_euclidean__taskonomy__almena.tar
d9c894|INPR| 2.4MiB/s|compressed//edge_texture__taskonomy__almena.tar
c72378|INPR| 2.4MiB/s|compressed//keypoints2d__taskonomy__almena.tar
abb076|INPR| 2.9MiB/s|compressed//keypoints3d__taskonomy__almena.tar
5042be|INPR| 3.6MiB/s|compressed//normal__taskonomy__almena.tar
4ec56e|INPR| 2.5MiB/s|compressed//principal_curvature__taskonomy__almena.tar
c90ad6|INPR| 2.4MiB/s|compressed//rgb__taskonomy__almena.tar
26a78e|INPR| 2.8MiB/s|compressed//reshading__taskonomy__almena.tar
Status Legend:
(INPR):download in-progress.
aria2 will resume download if the transfer is restarted.
If there are any errors, then see the log file. See '-l' option in help/man page for details.
For example #dcfcb1 remains on 0MB while other files been downloaded. I must say I tried to download the dataset on multiple machines and still same error came up.
Please help.
@alexsax
@amir32002