models--Rostlab--ProstT5_fp16 error with Downloading
CynthiaChibani opened this issue · 3 comments
- phold version: 0.1.4
- Python version: 3.11
- Operating System: Linux HPC
Description
Hey There,I am trying to run phold on the output of pharokka, I get the following error:
2024-06-24 15:01:17.736 | WARNING | phold.features.predict_3Di:get_T5_model:115 - No available GPU was found, but --cpu was not specified
2024-06-24 15:01:17.737 | WARNING | phold.features.predict_3Di:get_T5_model:118 - ProstT5 will be run with CPU only
2024-06-24 15:01:17.737 | INFO | phold.features.predict_3Di:get_T5_model:121 - Using device: cpu
2024-06-24 15:01:17.737 | INFO | phold.features.predict_3Di:get_T5_model:127 - Loading T5 from: ../phold/Rostlab/ProstT5_fp16
2024-06-24 15:01:17.738 | INFO | phold.features.predict_3Di:get_T5_model:128 - If ../phold/Rostlab/ProstT5_fp16 is not found, it will be downloaded
Traceback (most recent call last):
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/urllib3/connection.py", line 196, in _new_conn
sock = connection.create_connection(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/urllib3/util/connection.py", line 85, in create_connection
raise err
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/urllib3/util/connection.py", line 73, in create_connection
sock.connect(sa)
OSError: [Errno 101] Network is unreachable
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/urllib3/connectionpool.py", line 789, in urlopen
response = self._make_request(
^^^^^^^^^^^^^^^^^^^
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/urllib3/connectionpool.py", line 490, in _make_request
raise new_e
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/urllib3/connectionpool.py", line 466, in _make_request
self._validate_conn(conn)
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/urllib3/connectionpool.py", line 1095, in _validate_conn
conn.connect()
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/urllib3/connection.py", line 615, in connect
self.sock = sock = self._new_conn()
^^^^^^^^^^^^^^^^
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/urllib3/connection.py", line 211, in _new_conn
raise NewConnectionError(
urllib3.exceptions.NewConnectionError: <urllib3.connection.HTTPSConnection object at 0x1478ebf2d190>: Failed to establish a new connection: [Errno 101] Network is unreachable
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/requests/adapters.py", line 667, in send
resp = conn.urlopen(
^^^^^^^^^^^^^
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/urllib3/connectionpool.py", line 843, in urlopen
retries = retries.increment(
^^^^^^^^^^^^^^^^^^
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/urllib3/util/retry.py", line 519, in increment
raise MaxRetryError(_pool, url, reason) from reason # type: ignore[arg-type]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
urllib3.exceptions.MaxRetryError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /Rostlab/ProstT5_fp16/resolve/main/model.safetensors (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x1478ebf2d190>: Failed to establish a new connection: [Errno 101] Network is unreachable'))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/zfshome/sunam162/micromamba/envs/pholdENV/bin/phold", line 10, in
sys.exit(main())
^^^^^^
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/phold/init.py", line 1355, in main
main_cli()
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/click/core.py", line 1157, in call
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/phold/init.py", line 281, in run
subcommand_predict(
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/phold/subcommands/predict.py", line 125, in subcommand_predict
prediction_success = get_embeddings(
^^^^^^^^^^^^^^^
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/phold/features/predict_3Di.py", line 359, in get_embeddings
model, vocab = get_T5_model(model_dir, model_name, cpu)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/phold/features/predict_3Di.py", line 129, in get_T5_model
model = T5EncoderModel.from_pretrained(model_name, cache_dir=f"{model_dir}/").to(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/transformers/modeling_utils.py", line 3415, in from_pretrained
if not has_file(pretrained_model_name_or_path, safe_weights_name, **has_file_kwargs):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/transformers/utils/hub.py", line 629, in has_file
r = requests.head(url, headers=headers, allow_redirects=False, proxies=proxies, timeout=10)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/requests/api.py", line 100, in head
return request("head", url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/requests/api.py", line 59, in request
return session.request(method=method, url=url, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/requests/sessions.py", line 589, in request
resp = self.send(prep, **send_kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/requests/sessions.py", line 703, in send
r = adapter.send(request, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/zfshome/sunam162/micromamba/envs/pholdENV/lib/python3.11/site-packages/requests/adapters.py", line 700, in send
raise ConnectionError(e, request=request)
requests.exceptions.ConnectionError: HTTPSConnectionPool(host='huggingface.co', port=443): Max retries exceeded with url: /Rostlab/ProstT5_fp16/resolve/main/model.safetensors (Caused by NewConnectionError('<urllib3.connection.HTTPSConnection object at 0x1478ebf2d190>: Failed to establish a new connection: [Errno 101] Network is unreachable'))
The error seems to be huggingface related, however, I do have the models downloaded and extracted. I additionally tried using the zenodo link and I still get the same error.
- command
for i in $(ls -d NODE_OUT/);do phold run -i ${i%%/}/*.gbk -d ../phold/ -o ${i%%/}_phold; done
in the error:
../phold/Rostlab/ProstT5_fp16
and in the phold database path:
../phold/models--Rostlab--ProstT5_fp16
Could it simply be a path error?
this is probably just a hugging face error :) - please try again in a few hours. The zenodo backup code should be functional soon when I made the next release.
George
please for next release provide documentation about what needs to be downloaded and a way to use it locally.
some HPC cluster does not provide internet access on compute note (see #44).
we are on the same configuration.
regards
Eric
Hi @EricDeveaud and @CynthiaChibani ,
I have implemented a backup-solution for the download that will be available in the next release. The issue is Hugging Face (where ProstT5 lives) goes down from time to time, which I think is what happened here, so I have put a backup on Zenodo.
George