sdushantha/gitdir

Getting error -> urllib.error.HTTPError: HTTP Error 403: Forbidden

rangarajanps opened this issue · 5 comments

rangarajanps commented

Hi,

I tried using to download a folder as below and getting 403 Forbidden error. It is a public repository. Please clarify what are the other pre-requisite to run this.

F:\GitRepo\spring-hateaos>gitdir https://github.com/eugenp/tutorials/tree/master/spring-boot-rest
Traceback (most recent call last):
  File "f:\python\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "f:\python\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "F:\Python\Scripts\gitdir.exe\__main__.py", line 5, in <module>
  File "f:\python\lib\site-packages\gitdir\__main__.py", line 3, in <module>
    main()
  File "f:\python\lib\site-packages\gitdir\gitdir.py", line 150, in main
    total_files = download(url, flatten, args.output_dir)
  File "f:\python\lib\site-packages\gitdir\gitdir.py", line 64, in download
    response = urllib.request.urlretrieve(api_url)
  File "f:\python\lib\urllib\request.py", line 248, in urlretrieve
    with contextlib.closing(urlopen(url, data)) as fp:
  File "f:\python\lib\urllib\request.py", line 223, in urlopen
    return opener.open(url, data, timeout)
  File "f:\python\lib\urllib\request.py", line 532, in open
    response = meth(req, response)
  File "f:\python\lib\urllib\request.py", line 642, in http_response
    'http', request, response, code, msg, hdrs)
  File "f:\python\lib\urllib\request.py", line 570, in error
    return self._call_chain(*args)
  File "f:\python\lib\urllib\request.py", line 504, in _call_chain
    result = func(*args)
  File "f:\python\lib\urllib\request.py", line 650, in http_error_default
    raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 403: Forbidden

Thanks,
Ranga

sdushantha commented

@rangarajanps Since the error is 403: Forbidden, were you trying to download from a private repo? gitdir is not able to download content from private repos at the moment.
If that is not the case, could you send the me the url to the repo you were trying to download so that I can test it myself?

rangarajanps commented
  • 👍1
sdushantha commented

@rangarajanps Okay, so it is download many of the files and then after a while, it gives an error.
I am pretty sure this can be solved by adding an useragent. I will fix this as soon as possible.

Note to self: use this method, https://stackoverflow.com/a/24226797/9215267

sdushantha commented

I will work on this during this weekend.

sdushantha commented

@rangarajanps This has been fixed now :)