Webnovel api changed so crawler needs to be updated
m-haisham opened this issue · 1 comments
m-haisham commented
Error
Traceback (most recent call last):
File "c:\users\user\appdata\local\programs\python\python38\lib\threading.py", line 932, in _bootstrap_inner
self.run()
File "c:\users\user\appdata\local\programs\python\python38\lib\site-packages\novelsave\concurrent\action.py", line 20, in run
self.target(*ipt[0], **ipt[1])
File "c:\users\user\appdata\local\programs\python\python38\lib\site-packages\novelsave\novelsave.py", line 247, in task
ch = self.source.chapter(partial_c.url)
File "c:\users\user\appdata\local\programs\python\python38\lib\site-packages\novelsave\sources\webnovel.py", line 66, in chapter
wchapter = self.api.chapter(*UrlTools.from_chapter_url(url))
File "c:\users\user\appdata\local\programs\python\python38\lib\site-packages\webnovel\api\parsed.py", line 39, in chapter
response = super().chapter(novel_id, chapter_id)
File "c:\users\user\appdata\local\programs\python\python38\lib\site-packages\webnovel\api\base.py", line 45, in chapter
return self.validate(response)
File "c:\users\user\appdata\local\programs\python\python38\lib\site-packages\webnovel\api\base.py", line 126, in validate
parsed = json.loads(response.content)
File "c:\users\user\appdata\local\programs\python\python38\lib\json\__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "c:\users\user\appdata\local\programs\python\python38\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "c:\users\user\appdata\local\programs\python\python38\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
The error is due to the request returning a 404 response.
Reasons
Tinkering with the webnovel api showed that it was updated from /apiajax/chapter/getContent
to /go/pcm/chapter/getContent
.
I have tested with changing the request endpoint, it didnt seem to work. Currently looking for solutions.
m-haisham commented
webnovelbot was updated to accommodate the api change.
update to the latest version
pip install webnovelbot==0.2.9