Cannot access gated repo for url https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2

Question

Cannot access gated repo for url https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2

4ayyar opened this issue 6 months ago · 21 comments

getting following error when running 'PGPT_PROFILES=ollama make run ' after fresh install (no cache).

OSError: You are trying to access a gated repo.
Make sure to have access to it at https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2.
401 Client Error. (Request ID: Root=1-6621fd8f-45a5ffb02852831b1f476fbc;d622d97e-b698-4e30-89c4-e31278dd17ca)

Cannot access gated repo for url https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2/resolve/main/config.json.
Repo model mistralai/Mistral-7B-Instruct-v0.2 is gated. You must be authenticated to access it.
make: *** [run] Error 1

Answer 1 · 2024-04-19T06:00:23.000Z

I am also seeing this issue

Answer 2 · 2024-04-19T08:15:19.000Z

this is the full stacktrace:

Traceback (most recent call last):
  File "/home/worker/app/.venv/lib/python3.11/site-packages/injector/__init__.py", line 798, in get
    return self._context[key]
           ~~~~~~~~~~~~~^^^^^
KeyError: <class 'private_gpt.ui.ui.PrivateGptUi'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/worker/app/.venv/lib/python3.11/site-packages/injector/__init__.py", line 798, in get
    return self._context[key]
           ~~~~~~~~~~~~~^^^^^
KeyError: <class 'private_gpt.server.ingest.ingest_service.IngestService'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/worker/app/.venv/lib/python3.11/site-packages/injector/__init__.py", line 798, in get
    return self._context[key]
           ~~~~~~~~~~~~~^^^^^
KeyError: <class 'private_gpt.components.llm.llm_component.LLMComponent'>

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/worker/app/.venv/lib/python3.11/site-packages/huggingface_hub/utils/_errors.py", line 270, in hf_raise_for_status
    response.raise_for_status()
  File "/home/worker/app/.venv/lib/python3.11/site-packages/requests/models.py", line 1021, in raise_for_status
    raise HTTPError(http_error_msg, response=self)
requests.exceptions.HTTPError: 401 Client Error: Unauthorized for url: https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2/resolve/main/config.json```

Answer 3 · 2024-04-19T09:44:33.000Z

You need to follow the steps given by HuggingFace first (here):

pip install huggingface_hub
huggingface-cli login
here you enter the Token that you create in the Huggingface website
After that you should be able to work as usual

Answer 4 · 2024-04-19T10:33:47.000Z

This is still a bug however. In my final settings (from two profiles, lets say), I dont use anything from Huggingface, but I still see this error.

Current workaround if you are using privategpt without using anything from huggingface is to comment out the llm and embedding sections in the default settings.yaml file, but fill them in your settings-<profile_name>.yaml override file.

# llm:
#   mode: llamacpp
#   # Should be matching the selected model
#   max_new_tokens: 512
#   context_window: 3900
#   tokenizer: mistralai/Mistral-7B-Instruct-v0.2
#   temperature: 0.1      # The temperature of the model. Increasing the temperature will make the model answer more creatively. A value of 0.1 would be more factual. (Default: 0.1)

# embedding:
#   # Should be matching the value above in most cases
#   mode: huggingface
#   ingest_mode: simple
#   embed_dim: 384 # 384 is for BAAI/bge-small-en-v1.5

Answer 5 · 2024-04-19T12:19:37.000Z

You need to follow the steps given by HuggingFace first (here):

pip install huggingface_hub

huggingface-cli login
here you enter the Token that you create in the Huggingface website
After that you should be able to work as usual

Thanks this sorted me

Answer 6 · 2024-04-19T15:14:52.000Z

This is still a bug however. In my final settings (from two profiles, lets say), I dont use anything from Huggingface, but I still see this error.

Current workaround if you are using privategpt without using anything from huggingface is to comment out the llm and embedding sections in the default settings.yaml file, but fill them in your settings-<profile_name>.yaml override file.
# llm:
#   mode: llamacpp
#   # Should be matching the selected model
#   max_new_tokens: 512
#   context_window: 3900
#   tokenizer: mistralai/Mistral-7B-Instruct-v0.2
#   temperature: 0.1      # The temperature of the model. Increasing the temperature will make the model answer more creatively. A value of 0.1 would be more factual. (Default: 0.1)

# embedding:
#   # Should be matching the value above in most cases
#   mode: huggingface
#   ingest_mode: simple
#   embed_dim: 384 # 384 is for BAAI/bge-small-en-v1.5

This solution worked.

Answer 7 · 2024-04-21T06:18:28.000Z

This is still a bug however. In my final settings (from two profiles, lets say), I dont use anything from Huggingface, but I still see this error.

Current workaround if you are using privategpt without using anything from huggingface is to comment out the llm and embedding sections in the default settings.yaml file, but fill them in your settings-<profile_name>.yaml override file.
# llm:
#   mode: llamacpp
#   # Should be matching the selected model
#   max_new_tokens: 512
#   context_window: 3900
#   tokenizer: mistralai/Mistral-7B-Instruct-v0.2
#   temperature: 0.1      # The temperature of the model. Increasing the temperature will make the model answer more creatively. A value of 0.1 would be more factual. (Default: 0.1)

# embedding:
#   # Should be matching the value above in most cases
#   mode: huggingface
#   ingest_mode: simple
#   embed_dim: 384 # 384 is for BAAI/bge-small-en-v1.5

I'm still getting:

PermissionError: [Errno 13] Permission denied: 'tiktoken_cache'

How were you able to sort it out (using settings-ollama.yaml file without any changes)

Answer 8 · 2024-04-21T06:28:04.000Z

You probably need to recursive chown the /home/worker/app directory to the worker user. I assume you ran into this running in docker?

Answer 9 · 2024-04-21T06:38:39.000Z

You probably need to recursive chown the /home/worker/app directory to the worker user. I assume you ran into this running in docker?

Correct.

Indeed, after adding:

RUN chown -R worker:worker /home/worker/app

(and applying your solution of course)

to Dockerfile.local it's now working

Thanks a lot

Answer 10 · 2024-04-24T21:31:55.000Z

Create an account on Huggingface.co and then create a Token in its settings -> Access Tokens.
Add token to your ENV as HUGGINGFACE_TOKEN or just hardcode in setup.py if testing.
Add token arg in following places in setup.py

hf_hub_download(
    repo_id=settings().llamacpp.llm_hf_repo_id,
    filename=settings().llamacpp.llm_hf_model_file,
    cache_dir=models_cache_path,
    local_dir=models_path,
    resume_download=resume_download,
    token=settings().huggingface.access_token, # add this
)

AutoTokenizer.from_pretrained(
    pretrained_model_name_or_path=settings().llm.tokenizer,
    cache_dir=models_cache_path,
    token=settings().huggingface.access_token, # add this
)

Navigate to Mistral-7B-Instruct-v0.2 on Huggingface.co and accept its privacy policy.
Run setup

This solved the same issue for me.

Answer 11 · 2024-05-05T04:58:00.000Z

You need to follow the steps given by HuggingFace first (here):

pip install huggingface_hub

huggingface-cli login
here you enter the Token that you create in the Huggingface website
After that you should be able to work as usual

I get valid the token but then this appears

FileNotFoundError: [WinError 2] The system can't find the specified file

Answer 12 · 2024-05-09T00:59:28.000Z

There's another bug in ollama_settings.yaml which can cause PGPT_PROFILES=ollama make run fails.

You should use embedding_api_base instead of api_base for embedding.

The source code of embedding_component.py did require embedding_api_base property.

ollama:
llm_model: mistral
embedding_model: nomic-embed-text
#api_base: http://localhost:11434
embedding_api_base: http://localhost:11434

Answer 13 · 2024-05-09T12:41:11.000Z

Create an account on Huggingface.co and then create a Token in its settings -> Access Tokens.
- Add token to your ENV as HUGGINGFACE_TOKEN or just hardcode in setup.py if testing.
- Add token arg in following places in setup.py

hf_hub_download(
    repo_id=settings().llamacpp.llm_hf_repo_id,
    filename=settings().llamacpp.llm_hf_model_file,
    cache_dir=models_cache_path,
    local_dir=models_path,
    resume_download=resume_download,
    token=settings().huggingface.access_token, # add this
)

AutoTokenizer.from_pretrained(
    pretrained_model_name_or_path=settings().llm.tokenizer,
    cache_dir=models_cache_path,
    token=settings().huggingface.access_token, # add this
)

* Navigate to Mistral-7B-Instruct-v0.2 on Huggingface.co and accept its privacy policy.

* Run setup

This solved the same issue for me.

This solved for me too! thanks a lot

Answer 14 · 2024-05-18T09:19:31.000Z

After

Create an account on Huggingface.co and then create a Token in its settings -> Access Tokens.

Add token to your ENV as HUGGINGFACE_TOKEN or just hardcode in setup.py if testing.

Add token arg in following places in setup.py
hf_hub_download(
    repo_id=settings().llamacpp.llm_hf_repo_id,
    filename=settings().llamacpp.llm_hf_model_file,
    cache_dir=models_cache_path,
    local_dir=models_path,
    resume_download=resume_download,
    token=settings().huggingface.access_token, # add this
)

AutoTokenizer.from_pretrained(
    pretrained_model_name_or_path=settings().llm.tokenizer,
    cache_dir=models_cache_path,
    token=settings().huggingface.access_token, # add this
)
Navigate to Mistral-7B-Instruct-v0.2 on Huggingface.co and accept its privacy policy.

Run setup

This solved the same issue for me.

That fixed it for me. Also if you are getting a We couldn't connect to 'https://huggingface.co' after than, change your token to Write, that should fix it.

Answer 15 · 2024-05-21T20:19:07.000Z

Advanced Troubleshooting Steps

Check Network Issues:
- Ensure no network restrictions block Hugging Face.
- Use a VPN to bypass regional restrictions.

Environment Isolation:

Create a fresh virtual environment to rule out dependency conflicts:

python -m venv newenv
source newenv/bin/activate  # or newenv\Scripts\activate on Windows
pip install huggingface_hub
huggingface-cli login

Token Scope Verification:
- Ensure the token scope includes read and write permissions. Update token permissions on Hugging Face if needed.
Inspect Detailed Logs:
- Enable detailed logging to capture more information about the error:
```
import logging
logging.basicConfig(level=logging.DEBUG)
```

Use Direct API Calls:

Verify access by making direct API calls using requests:

import requests

headers = {"Authorization": f"Bearer {your_token}"}
response = requests.get("https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2/resolve/main/config.json", headers=headers)
print(response.status_code, response.text)

If you need further tailored assistance, please provide specific details about the error logs or configurations you are using. This will help diagnose the issue more accurately.

Answer 16 · 2024-05-23T07:54:12.000Z

Why is the tokenizer needed and why does it access an external site?

Answer 17 · 2024-05-23T07:59:10.000Z

The tokenizer is a great question for you to ask Google or any Large Language Models. As for the internet calls, it is due to model licensing requiring permission to access certain paid or "premium" models. It makes it so you can't go and make money or train your own AI without giving them credit or such. If you are asking about the API method I just posted, that is a workaround to test to see if the issue is specific to the installed python libraries caching something or if it is due to an issue on huggingface's side.

Answer 18 · 2024-05-24T07:24:51.000Z

I think in the case of ollama such a hugging face requirement should not exist. Btw this gated repo issue is happening to me on Ubuntu 22.04 and not on Windows.

Answer 19 · 2024-05-24T09:22:01.000Z

I think in the case of ollama such a hugging face requirement should not exist. Btw this gated repo issue is happening to me on Ubuntu 22.04 and not on Windows.

Got the same on Debian12 but solved

Answer 20 · 2024-06-11T13:58:01.000Z

You need to follow the steps given by HuggingFace first (here):

pip install huggingface_hub

huggingface-cli login
here you enter the Token that you create in the Huggingface website
After that you should be able to work as usual

After doing the above, I still have to browse to the URL: https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.2
and then click on the button to grant me access to the model. Only then it worked for me.

Answer 21 · 2024-07-25T09:57:35.000Z

This worked for me:

accepted its privacy policy
created a Access Token with the 'finegrained' permission under the account settings (Ticked all options under Repositories and Inference.)
ran pip install --upgrade huggingface_hub, and add this two lines into your code.

from huggingface_hub import login
login(token="the access token I just created")