Certificate verify failed when downloading OASIS data
Closed this issue · 0 comments
KumoLiu commented
[2024-05-20T04:03:05.050Z] Running ./3d_registration/learn2reg_oasis_unpaired_brain_mr.ipynb
[2024-05-20T04:03:05.050Z] Checking PEP8 compliance...
[2024-05-20T04:03:05.983Z] Running notebook...
[2024-05-20T04:03:06.240Z] Before:
[2024-05-20T04:03:06.240Z] "max_epochs = 50\n",
[2024-05-20T04:03:06.240Z] "- Train the network longer, i.e., use `max_epochs = 200` in cell [Hyperparameters](#hyperparameters). \n",
[2024-05-20T04:03:06.240Z] After:
[2024-05-20T04:03:06.240Z] "max_epochs = 1\n",
[2024-05-20T04:03:06.240Z] "- Train the network longer, i.e., use `max_epochs = 1` in cell [Hyperparameters](#hyperparameters). \n",
[2024-05-20T04:03:06.240Z] Before:
[2024-05-20T04:03:06.240Z] "val_interval = 1\n",
[2024-05-20T04:03:06.240Z] After:
[2024-05-20T04:03:06.240Z] "val_interval = 1\n",
[2024-05-20T04:03:14.315Z] MONAI version: 1.3.1rc9
[2024-05-20T04:03:14.315Z] Numpy version: 1.24.4
[2024-05-20T04:03:14.315Z] Pytorch version: 2.3.0a0+40ec155e58.nv24.03
[2024-05-20T04:03:14.315Z] MONAI flags: HAS_EXT = False, USE_COMPILED = False, USE_META_DICT = False
[2024-05-20T04:03:14.315Z] MONAI rev id: b16f54ab82e665f8643043ffba5c917fa417121b
[2024-05-20T04:03:14.315Z] MONAI __file__: /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/__init__.py
[2024-05-20T04:03:14.315Z]
[2024-05-20T04:03:14.315Z] Optional dependencies:
[2024-05-20T04:03:14.315Z] Pytorch Ignite version: 0.4.11
[2024-05-20T04:03:14.315Z] ITK version: 5.3.0
[2024-05-20T04:03:14.315Z] Nibabel version: 5.2.1
[2024-05-20T04:03:14.315Z] scikit-image version: 0.22.0
[2024-05-20T04:03:14.315Z] scipy version: 1.12.0
[2024-05-20T04:03:14.315Z] Pillow version: 10.2.0
[2024-05-20T04:03:14.315Z] Tensorboard version: 2.16.2
[2024-05-20T04:03:14.315Z] gdown version: 5.2.0
[2024-05-20T04:03:14.315Z] TorchVision version: 0.18.0a0
[2024-05-20T04:03:14.315Z] tqdm version: 4.66.2
[2024-05-20T04:03:14.315Z] lmdb version: 1.4.1
[2024-05-20T04:03:14.315Z] psutil version: 5.9.4
[2024-05-20T04:03:14.315Z] pandas version: 1.5.3
[2024-05-20T04:03:14.315Z] einops version: 0.7.0
[2024-05-20T04:03:14.315Z] transformers version: 4.40.2
[2024-05-20T04:03:14.315Z] mlflow version: 2.13.0
[2024-05-20T04:03:14.315Z] pynrrd version: 1.0.0
[2024-05-20T04:03:14.315Z] clearml version: 1.16.1
[2024-05-20T04:03:14.315Z]
[2024-05-20T04:03:14.315Z] For details about installing the optional dependencies, please visit:
[2024-05-20T04:03:14.315Z] https://docs.monai.io/en/latest/installation.html#installing-the-recommended-dependencies
[2024-05-20T04:03:14.315Z]
[2024-05-20T04:03:16.826Z] papermill --progress-bar --log-output -k python3
[2024-05-20T04:03:17.081Z] /usr/local/lib/python3.10/dist-packages/papermill/iorw.py:149: UserWarning: the file is not specified with any extension : -
[2024-05-20T04:03:17.081Z] warnings.warn(f"the file is not specified with any extension : {os.path.basename(path)}")
[2024-05-20T04:03:37.507Z]
Executing: 0%| | 0/37 [00:00<?, ?cell/s]
Executing: 3%|▎ | 1/37 [00:01<00:38, 1.06s/cell]
Executing: 8%|▊ | 3/37 [00:10<02:09, 3.80s/cell]
Executing: 14%|█▎ | 5/37 [00:16<01:48, 3.38s/cell]
Executing: 24%|██▍ | 9/37 [00:17<00:44, 1.59s/cell]
Executing: 24%|██▍ | 9/37 [00:20<01:02, 2.22s/cell]
[2024-05-20T04:03:37.507Z] /usr/local/lib/python3.10/dist-packages/papermill/iorw.py:149: UserWarning: the file is not specified with any extension : -
[2024-05-20T04:03:37.507Z] warnings.warn(f"the file is not specified with any extension : {os.path.basename(path)}")
[2024-05-20T04:03:37.507Z] Traceback (most recent call last):
[2024-05-20T04:03:37.507Z] File "/usr/local/bin/papermill", line 8, in <module>
[2024-05-20T04:03:37.507Z] sys.exit(papermill())
[2024-05-20T04:03:37.507Z] File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1157, in __call__
[2024-05-20T04:03:37.507Z] return self.main(*args, **kwargs)
[2024-05-20T04:03:37.507Z] File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1078, in main
[2024-05-20T04:03:37.507Z] rv = self.invoke(ctx)
[2024-05-20T04:03:37.507Z] File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 1434, in invoke
[2024-05-20T04:03:37.507Z] return ctx.invoke(self.callback, **ctx.params)
[2024-05-20T04:03:37.507Z] File "/usr/local/lib/python3.10/dist-packages/click/core.py", line 783, in invoke
[2024-05-20T04:03:37.507Z] return __callback(*args, **kwargs)
[2024-05-20T04:03:37.507Z] File "/usr/local/lib/python3.10/dist-packages/click/decorators.py", line 33, in new_func
[2024-05-20T04:03:37.507Z] return f(get_current_context(), *args, **kwargs)
[2024-05-20T04:03:37.507Z] File "/usr/local/lib/python3.10/dist-packages/papermill/cli.py", line 235, in papermill
[2024-05-20T04:03:37.507Z] execute_notebook(
[2024-05-20T04:03:37.507Z] File "/usr/local/lib/python3.10/dist-packages/papermill/execute.py", line 131, in execute_notebook
[2024-05-20T04:03:37.507Z] raise_for_execution_errors(nb, output_path)
[2024-05-20T04:03:37.507Z] File "/usr/local/lib/python3.10/dist-packages/papermill/execute.py", line 251, in raise_for_execution_errors
[2024-05-20T04:03:37.507Z] raise error
[2024-05-20T04:03:37.507Z] papermill.exceptions.PapermillExecutionError:
[2024-05-20T04:03:37.507Z] ---------------------------------------------------------------------------
[2024-05-20T04:03:37.507Z] Exception encountered at "In [4]":
[2024-05-20T04:03:37.507Z] ---------------------------------------------------------------------------
[2024-05-20T04:03:37.507Z] SSLCertVerificationError Traceback (most recent call last)
[2024-05-20T04:03:37.507Z] File /usr/lib/python3.10/urllib/request.py:1348, in AbstractHTTPHandler.do_open(self, http_class, req, **http_conn_args)
[2024-05-20T04:03:37.507Z] 1347 try:
[2024-05-20T04:03:37.507Z] -> 1348 h.request(req.get_method(), req.selector, req.data, headers,
[2024-05-20T04:03:37.507Z] 1349 encode_chunked=req.has_header('Transfer-encoding'))
[2024-05-20T04:03:37.507Z] 1350 except OSError as err: # timeout error
[2024-05-20T04:03:37.507Z]
[2024-05-20T04:03:37.507Z] File /usr/lib/python3.10/http/client.py:1283, in HTTPConnection.request(self, method, url, body, headers, encode_chunked)
[2024-05-20T04:03:37.507Z] 1282 """Send a complete request to the server."""
[2024-05-20T04:03:37.507Z] -> 1283 self._send_request(method, url, body, headers, encode_chunked)
[2024-05-20T04:03:37.508Z]
[2024-05-20T04:03:37.508Z] File /usr/lib/python3.10/http/client.py:1329, in HTTPConnection._send_request(self, method, url, body, headers, encode_chunked)
[2024-05-20T04:03:37.508Z] 1328 body = _encode(body, 'body')
[2024-05-20T04:03:37.508Z] -> 1329 self.endheaders(body, encode_chunked=encode_chunked)
[2024-05-20T04:03:37.508Z]
[2024-05-20T04:03:37.508Z] File /usr/lib/python3.10/http/client.py:1278, in HTTPConnection.endheaders(self, message_body, encode_chunked)
[2024-05-20T04:03:37.508Z] 1277 raise CannotSendHeader()
[2024-05-20T04:03:37.508Z] -> 1278 self._send_output(message_body, encode_chunked=encode_chunked)
[2024-05-20T04:03:37.508Z]
[2024-05-20T04:03:37.508Z] File /usr/lib/python3.10/http/client.py:1038, in HTTPConnection._send_output(self, message_body, encode_chunked)
[2024-05-20T04:03:37.508Z] 1037 del self._buffer[:]
[2024-05-20T04:03:37.508Z] -> 1038 self.send(msg)
[2024-05-20T04:03:37.508Z] 1040 if message_body is not None:
[2024-05-20T04:03:37.508Z] 1041
[2024-05-20T04:03:37.508Z] 1042 # create a consistent interface to message_body
[2024-05-20T04:03:37.508Z]
[2024-05-20T04:03:37.508Z] File /usr/lib/python3.10/http/client.py:976, in HTTPConnection.send(self, data)
[2024-05-20T04:03:37.508Z] 975 if self.auto_open:
[2024-05-20T04:03:37.508Z] --> 976 self.connect()
[2024-05-20T04:03:37.508Z] 977 else:
[2024-05-20T04:03:37.508Z]
[2024-05-20T04:03:37.508Z] File /usr/lib/python3.10/http/client.py:1455, in HTTPSConnection.connect(self)
[2024-05-20T04:03:37.508Z] 1453 server_hostname = self.host
[2024-05-20T04:03:37.508Z] -> 1455 self.sock = self._context.wrap_socket(self.sock,
[2024-05-20T04:03:37.508Z] 1456 server_hostname=server_hostname)
[2024-05-20T04:03:37.508Z]
[2024-05-20T04:03:37.508Z] File /usr/lib/python3.10/ssl.py:513, in SSLContext.wrap_socket(self, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, session)
[2024-05-20T04:03:37.508Z] 507 def wrap_socket(self, sock, server_side=False,
[2024-05-20T04:03:37.508Z] 508 do_handshake_on_connect=True,
[2024-05-20T04:03:37.508Z] 509 suppress_ragged_eofs=True,
[2024-05-20T04:03:37.508Z] 510 server_hostname=None, session=None):
[2024-05-20T04:03:37.508Z] 511 # SSLSocket class handles server_hostname encoding before it calls
[2024-05-20T04:03:37.508Z] 512 # ctx._wrap_socket()
[2024-05-20T04:03:37.508Z] --> 513 return self.sslsocket_class._create(
[2024-05-20T04:03:37.508Z] 514 sock=sock,
[2024-05-20T04:03:37.508Z] 515 server_side=server_side,
[2024-05-20T04:03:37.508Z] 516 do_handshake_on_connect=do_handshake_on_connect,
[2024-05-20T04:03:37.508Z] 517 suppress_ragged_eofs=suppress_ragged_eofs,
[2024-05-20T04:03:37.508Z] 518 server_hostname=server_hostname,
[2024-05-20T04:03:37.508Z] 519 context=self,
[2024-05-20T04:03:37.508Z] 520 session=session
[2024-05-20T04:03:37.508Z] 521 )
[2024-05-20T04:03:37.508Z]
[2024-05-20T04:03:37.508Z] File /usr/lib/python3.10/ssl.py:1100, in SSLSocket._create(cls, sock, server_side, do_handshake_on_connect, suppress_ragged_eofs, server_hostname, context, session)
[2024-05-20T04:03:37.508Z] 1099 raise ValueError("do_handshake_on_connect should not be specified for non-blocking sockets")
[2024-05-20T04:03:37.508Z] -> 1100 self.do_handshake()
[2024-05-20T04:03:37.508Z] 1101 except (OSError, ValueError):
[2024-05-20T04:03:37.508Z]
[2024-05-20T04:03:37.508Z] File /usr/lib/python3.10/ssl.py:1371, in SSLSocket.do_handshake(self, block)
[2024-05-20T04:03:37.508Z] 1370 self.settimeout(None)
[2024-05-20T04:03:37.508Z] -> 1371 self._sslobj.do_handshake()
[2024-05-20T04:03:37.508Z] 1372 finally:
[2024-05-20T04:03:37.508Z]
[2024-05-20T04:03:37.508Z] SSLCertVerificationError: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)
[2024-05-20T04:03:37.508Z]
[2024-05-20T04:03:37.508Z] During handling of the above exception, another exception occurred:
[2024-05-20T04:03:37.508Z]
[2024-05-20T04:03:37.508Z] URLError Traceback (most recent call last)
[2024-05-20T04:03:37.508Z] Cell In[4], line 7
[2024-05-20T04:03:37.508Z] 5 if not os.path.exists(data_dir):
[2024-05-20T04:03:37.508Z] 6 os.mkdir(data_dir)
[2024-05-20T04:03:37.508Z] ----> 7 download_and_extract(resource, compressed_file, data_dir)
[2024-05-20T04:03:37.508Z]
[2024-05-20T04:03:37.508Z] File /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/apps/utils.py:335, in download_and_extract(url, filepath, output_dir, hash_val, hash_type, file_type, has_base, progress)
[2024-05-20T04:03:37.508Z] 333 with tempfile.TemporaryDirectory() as tmp_dir:
[2024-05-20T04:03:37.508Z] 334 filename = filepath or Path(tmp_dir, _basename(url)).resolve()
[2024-05-20T04:03:37.508Z] --> 335 download_url(url=url, filepath=filename, hash_val=hash_val, hash_type=hash_type, progress=progress)
[2024-05-20T04:03:37.508Z] 336 extractall(filepath=filename, output_dir=output_dir, file_type=file_type, has_base=has_base)
[2024-05-20T04:03:37.508Z]
[2024-05-20T04:03:37.508Z] File /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/apps/utils.py:226, in download_url(url, filepath, hash_val, hash_type, progress, **gdown_kwargs)
[2024-05-20T04:03:37.508Z] 221 raise RuntimeError(
[2024-05-20T04:03:37.508Z] 222 f"Download of file from {download_url}, received from {url} "
[2024-05-20T04:03:37.508Z] 223 + f" to {filepath} failed due to network issue or denied permission."
[2024-05-20T04:03:37.508Z] 224 )
[2024-05-20T04:03:37.508Z] 225 else:
[2024-05-20T04:03:37.508Z] --> 226 _download_with_progress(url, tmp_name, progress=progress)
[2024-05-20T04:03:37.508Z] 227 if not tmp_name.exists():
[2024-05-20T04:03:37.508Z] 228 raise RuntimeError(
[2024-05-20T04:03:37.508Z] 229 f"Download of file from {url} to {filepath} failed due to network issue or denied permission."
[2024-05-20T04:03:37.508Z] 230 )
[2024-05-20T04:03:37.508Z]
[2024-05-20T04:03:37.508Z] File /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/apps/utils.py:119, in _download_with_progress(url, filepath, progress)
[2024-05-20T04:03:37.508Z] 117 except (URLError, HTTPError, ContentTooShortError, OSError) as e:
[2024-05-20T04:03:37.508Z] 118 logger.error(f"Download failed from {url} to {filepath}.")
[2024-05-20T04:03:37.508Z] --> 119 raise e
[2024-05-20T04:03:37.508Z]
[2024-05-20T04:03:37.508Z] File /home/jenkins/agent/workspace/Monai-notebooks/MONAI/monai/apps/utils.py:112, in _download_with_progress(url, filepath, progress)
[2024-05-20T04:03:37.508Z] 109 self.update(b * bsize - self.n) # will also set self.n = b * bsize
[2024-05-20T04:03:37.508Z] 111 with TqdmUpTo(unit="B", unit_scale=True, unit_divisor=1024, miniters=1, desc=_basename(filepath)) as t:
[2024-05-20T04:03:37.508Z] --> 112 urlretrieve(url, filepath, reporthook=t.update_to)
[2024-05-20T04:03:37.508Z] 113 else:
[2024-05-20T04:03:37.508Z] 114 if not has_tqdm and progress:
[2024-05-20T04:03:37.508Z]
[2024-05-20T04:03:37.508Z] File /usr/lib/python3.10/urllib/request.py:241, in urlretrieve(url, filename, reporthook, data)
[2024-05-20T04:03:37.508Z] 224 """
[2024-05-20T04:03:37.508Z] 225 Retrieve a URL into a temporary location on disk.
[2024-05-20T04:03:37.508Z] 226
[2024-05-20T04:03:37.508Z] (...)
[2024-05-20T04:03:37.508Z] 237 data file as well as the resulting HTTPMessage object.
[2024-05-20T04:03:37.508Z] 238 """
[2024-05-20T04:03:37.508Z] 239 url_type, path = _splittype(url)
[2024-05-20T04:03:37.508Z] --> 241 with contextlib.closing(urlopen(url, data)) as fp:
[2024-05-20T04:03:37.508Z] 242 headers = fp.info()
[2024-05-20T04:03:37.508Z] 244 # Just return the local path and the "headers" for file://
[2024-05-20T04:03:37.508Z] 245 # URLs. No sense in performing a copy unless requested.
[2024-05-20T04:03:37.508Z]
[2024-05-20T04:03:37.508Z] File /usr/lib/python3.10/urllib/request.py:216, in urlopen(url, data, timeout, cafile, capath, cadefault, context)
[2024-05-20T04:03:37.508Z] 214 else:
[2024-05-20T04:03:37.508Z] 215 opener = _opener
[2024-05-20T04:03:37.508Z] --> 216 return opener.open(url, data, timeout)
[2024-05-20T04:03:37.508Z]
[2024-05-20T04:03:37.508Z] File /usr/lib/python3.10/urllib/request.py:519, in OpenerDirector.open(self, fullurl, data, timeout)
[2024-05-20T04:03:37.508Z] 516 req = meth(req)
[2024-05-20T04:03:37.508Z] 518 sys.audit('urllib.Request', req.full_url, req.data, req.headers, req.get_method())
[2024-05-20T04:03:37.508Z] --> 519 response = self._open(req, data)
[2024-05-20T04:03:37.508Z] 521 # post-process response
[2024-05-20T04:03:37.508Z] 522 meth_name = protocol+"_response"
[2024-05-20T04:03:37.508Z]
[2024-05-20T04:03:37.508Z] File /usr/lib/python3.10/urllib/request.py:536, in OpenerDirector._open(self, req, data)
[2024-05-20T04:03:37.508Z] 533 return result
[2024-05-20T04:03:37.508Z] 535 protocol = req.type
[2024-05-20T04:03:37.508Z] --> 536 result = self._call_chain(self.handle_open, protocol, protocol +
[2024-05-20T04:03:37.508Z] 537 '_open', req)
[2024-05-20T04:03:37.508Z] 538 if result:
[2024-05-20T04:03:37.508Z] 539 return result
[2024-05-20T04:03:37.508Z]
[2024-05-20T04:03:37.508Z] File /usr/lib/python3.10/urllib/request.py:496, in OpenerDirector._call_chain(self, chain, kind, meth_name, *args)
[2024-05-20T04:03:37.508Z] 494 for handler in handlers:
[2024-05-20T04:03:37.508Z] 495 func = getattr(handler, meth_name)
[2024-05-20T04:03:37.508Z] --> 496 result = func(*args)
[2024-05-20T04:03:37.508Z] 497 if result is not None:
[2024-05-20T04:03:37.508Z] 498 return result
[2024-05-20T04:03:37.508Z]
[2024-05-20T04:03:37.508Z] File /usr/lib/python3.10/urllib/request.py:1391, in HTTPSHandler.https_open(self, req)
[2024-05-20T04:03:37.508Z] 1390 def https_open(self, req):
[2024-05-20T04:03:37.508Z] -> 1391 return self.do_open(http.client.HTTPSConnection, req,
[2024-05-20T04:03:37.508Z] 1392 context=self._context, check_hostname=self._check_hostname)
[2024-05-20T04:03:37.508Z]
[2024-05-20T04:03:37.508Z] File /usr/lib/python3.10/urllib/request.py:1351, in AbstractHTTPHandler.do_open(self, http_class, req, **http_conn_args)
[2024-05-20T04:03:37.508Z] 1348 h.request(req.get_method(), req.selector, req.data, headers,
[2024-05-20T04:03:37.508Z] 1349 encode_chunked=req.has_header('Transfer-encoding'))
[2024-05-20T04:03:37.508Z] 1350 except OSError as err: # timeout error
[2024-05-20T04:03:37.508Z] -> 1351 raise URLError(err)
[2024-05-20T04:03:37.508Z] 1352 r = h.getresponse()
[2024-05-20T04:03:37.508Z] 1353 except:
[2024-05-20T04:03:37.508Z]
[2024-05-20T04:03:37.508Z] URLError: <urlopen error [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate (_ssl.c:1007)>