Error 403 - cannot download dataset with PyDrive
laszloturanyi opened this issue · 3 comments
Hi,
I followed the PyDrive method to download the data. However after succesfully authenticating, I get the following Error 403
Authentication successful.
authorized access to google drive API!
Parsing JSON metadata...
Downloading 70001 files...
- done processing 0/70001 filesTraceback (most recent call last):
File "/home/tlaci/anaconda3/envs/pydrive/lib/python3.8/site-packages/pydrive/files.py", line 235, in FetchMetadata
metadata = self.auth.service.files().get(fileId=file_id,
File "/home/tlaci/anaconda3/envs/pydrive/lib/python3.8/site-packages/googleapiclient/_helpers.py", line 134, in positional_wrapper
return wrapped(*args, **kwargs)
File "/home/tlaci/anaconda3/envs/pydrive/lib/python3.8/site-packages/googleapiclient/http.py", line 907, in execute
raise HttpError(resp, content, uri=self.uri)
googleapiclient.errors.HttpError: <HttpError 403 when requesting https://www.googleapis.com/drive/v2/files/1nGp-wUlRHFJ6aJitkZZlHVTICTLKLJfT?alt=json returned "User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=971122666526">
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "download_ffhq_aging.py", line 374, in <module>
run_cmdline(sys.argv)
File "download_ffhq_aging.py", line 369, in run_cmdline
run(**vars(args))
File "download_ffhq_aging.py", line 348, in run
download_files(specs, dst_dir, output_size, drive=drive, **download_kwargs)
File "download_ffhq_aging.py", line 209, in download_files
raise exc_info[1].with_traceback(exc_info[2])
File "download_ffhq_aging.py", line 219, in _download_thread
pydrive_utils.pydrive_download(drive, spec['file_url'], spec['file_path'])
File "/media/tlaci/F85E3F6B5E3F2234/FFHQ-Aging-Dataset/pydrive_utils.py", line 40, in pydrive_download
pydrive_file.GetContentFile(save_path)
File "/home/tlaci/anaconda3/envs/pydrive/lib/python3.8/site-packages/pydrive/files.py", line 210, in GetContentFile
self.FetchContent(mimetype, remove_bom)
File "/home/tlaci/anaconda3/envs/pydrive/lib/python3.8/site-packages/pydrive/files.py", line 42, in _decorated
self.FetchMetadata()
File "/home/tlaci/anaconda3/envs/pydrive/lib/python3.8/site-packages/pydrive/auth.py", line 75, in _decorated
return decoratee(self, *args, **kwargs)
File "/home/tlaci/anaconda3/envs/pydrive/lib/python3.8/site-packages/pydrive/files.py", line 239, in FetchMetadata
raise ApiRequestError(error)
pydrive.files.ApiRequestError: <HttpError 403 when requesting https://www.googleapis.com/drive/v2/files/1nGp-wUlRHFJ6aJitkZZlHVTICTLKLJfT?alt=json returned "User Rate Limit Exceeded. Rate of requests for user exceed configured project quota. You may consider re-evaluating expected per-user traffic to the API and adjust project quota limits accordingly. You may monitor aggregate quota usage and adjust limits in the API Console: https://console.developers.google.com/apis/api/drive.googleapis.com/quotas?project=971122666526">
Opening the link the .json message shows a bit more detailed message.
{
"error": {
"errors": [
{
"domain": "usageLimits",
"reason": "dailyLimitExceededUnreg",
"message": "Daily Limit for Unauthenticated Use Exceeded. Continued use requires signup.",
"extendedHelp": "https://code.google.com/apis/console"
}
],
"code": 403,
"message": "Daily Limit for Unauthenticated Use Exceeded. Continued use requires signup."
}
}
If the authentication happened how can the problem be "Unauthenticated Use Exceeded"?
Can you please verify that the ffhq-dataset folder is in your shared with me folder on the google drive account you used pydrive with?
I can verify that. However, after 24 hours I could downloaded it. I think it is rather related with the limitations at the owner's side.
"Continued use requires signup." This message seems to be misleading.
Thanks for the update!
These are indeed limitations imposed by google, which we have no way to control.
Same thing with the error messages, these are google error messages, not ours, so we can't change them.