m-haisham/novelsave

Webnovel api changed so crawler needs to be updated

m-haisham opened this issue · 1 comments

Error

Traceback (most recent call last):
  File "c:\users\user\appdata\local\programs\python\python38\lib\threading.py", line 932, in _bootstrap_inner
    self.run()
  File "c:\users\user\appdata\local\programs\python\python38\lib\site-packages\novelsave\concurrent\action.py", line 20, in run
    self.target(*ipt[0], **ipt[1])
  File "c:\users\user\appdata\local\programs\python\python38\lib\site-packages\novelsave\novelsave.py", line 247, in task
    ch = self.source.chapter(partial_c.url)
  File "c:\users\user\appdata\local\programs\python\python38\lib\site-packages\novelsave\sources\webnovel.py", line 66, in chapter
    wchapter = self.api.chapter(*UrlTools.from_chapter_url(url))
  File "c:\users\user\appdata\local\programs\python\python38\lib\site-packages\webnovel\api\parsed.py", line 39, in chapter
    response = super().chapter(novel_id, chapter_id)
  File "c:\users\user\appdata\local\programs\python\python38\lib\site-packages\webnovel\api\base.py", line 45, in chapter
    return self.validate(response)
  File "c:\users\user\appdata\local\programs\python\python38\lib\site-packages\webnovel\api\base.py", line 126, in validate
    parsed = json.loads(response.content)
  File "c:\users\user\appdata\local\programs\python\python38\lib\json\__init__.py", line 357, in loads
    return _default_decoder.decode(s)
  File "c:\users\user\appdata\local\programs\python\python38\lib\json\decoder.py", line 337, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "c:\users\user\appdata\local\programs\python\python38\lib\json\decoder.py", line 355, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

The error is due to the request returning a 404 response.

Reasons

Tinkering with the webnovel api showed that it was updated from /apiajax/chapter/getContent to /go/pcm/chapter/getContent.

I have tested with changing the request endpoint, it didnt seem to work. Currently looking for solutions.

webnovelbot was updated to accommodate the api change.

update to the latest version

pip install webnovelbot==0.2.9