Emersont1/itchio

TypeError: expected string or bytes-like object

RonaldH opened this issue · 7 comments

Downloading Troika! Numinous Edition - edit
Traceback (most recent call last):
  File "c:\Python310\lib\runpy.py", line 196, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "c:\Python310\lib\runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "c:\Python310\lib\site-packages\itchio\downloader\__main__.py", line 26, in <module>
    lib.download_library()
  File "c:\Python310\lib\site-packages\itchio\library.py", line 32, in download_library
    game.download(self.login)
  File "c:\Python310\lib\site-packages\itchio\game.py", line 62, in download
    itchio.utils.download(url, path, self.name +" - "+file)
  File "c:\Python310\lib\site-packages\itchio\utils.py", line 11, in download
    filename = re.search(r'filename="(.+)"', cd).group(1)
  File "c:\Python310\lib\re.py", line 200, in search
    return _compile(pattern, flags).search(string)
TypeError: expected string or bytes-like object

The game in question is: https://melsonian-arts-council.itch.io/troika-numinous-edition/ purchased from the Ukraine bundle.

The type of cd is none, thus the header doesn't exist, if I understand it correctly.

Please see #9 & #11.

Now #17 is merged in, I can close this

The error still occurs for me:

Downloading Timebomb - timebomb-web
Traceback (most recent call last):
  File "/usr/lib/python3.8/runpy.py", line 194, in _run_module_as_main
    return _run_code(code, main_globals, None,
  File "/usr/lib/python3.8/runpy.py", line 87, in _run_code
    exec(code, run_globals)
  File "/home/alex/itch.io/.venv/lib/python3.8/site-packages/itchio/downloader/__main__.py", line 26, in <module>
    lib.download_library()
  File "/home/alex/itch.io/.venv/lib/python3.8/site-packages/itchio/library.py", line 32, in download_library
    game.download(self.login)
  File "/home/alex/itch.io/.venv/lib/python3.8/site-packages/itchio/game.py", line 62, in download
    itchio.utils.download(url, path, self.name +" - "+file)
  File "/home/alex/itch.io/.venv/lib/python3.8/site-packages/itchio/utils.py", line 11, in download
    filename = re.search(r'filename="(.+)"', cd).group(1)
  File "/usr/lib/python3.8/re.py", line 201, in search
    return _compile(pattern, flags).search(string)
TypeError: expected string or bytes-like object

Before the run I upgraded with: pip install git+https://github.com/emersont1/itchio

Resolved https://github.com/emersont1/itchio to commit 72e1295cace7bdbd79e7ba53e1f2d6231d45cc71

Commit ID is correct.

I created a workaround which works for me:
utils.py - Ignore all downloads that do not have a Content-Disposition set. You are not able to get a filename thus the error message is bad, but it does not break.

import requests
import re
import os

from clint.textui import progress

def download(url, path, desc):
    print(f"Downloading {desc}")
    rsp = requests.get(url, stream=True)
    cd = rsp.headers.get("Content-Disposition")
    if(cd != None):
        filename = re.search(r'filename="(.+)"', cd).group(1)
        total_length = int(rsp.headers.get('content-length'))

        if os.path.exists(f"{path}/{filename}"):
            if os.path.getsize(f"{path}/{filename}") == total_length:
                print(f"{filename} already exists, skipping")
                return f"{path}/{filename}", False
            else:
                print(f"{filename} exists but is incomplete, downloading again")

        with open(f"{path}/{filename}", "wb") as f:
            for chunk in progress.bar(rsp.iter_content(chunk_size=1024), expected_size=(total_length/1024) + 1):
                if chunk:
                    f.write(chunk)
                    f.flush()
        return f"{path}/{filename}", True
    return f"{url}", False

Does this not fix it?

if rsp.headers.get('content-length') is None or rsp.headers.get("Content-Disposition") is None:

I deleted the itchio lib and reinstalled new with itchiodl and now it works! I assume I did the update wrong and it does not work as I thought it would (just rerun the old pip command).

In short: My fault, works now!