eight04/ComicCrawler

Support yoedge

eight04 opened this issue · 1 comments

Requests has problems to parse yoedge's header:

DEBUG:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1):
 smp.yoedge.com
send: b'GET /smp-app/1017018/shinmangaplayer/index.html HTTP/1.1\r\nUser-Agent:
Mozilla/5.0 (Windows NT 6.1; WOW64; rv:54.0) Gecko/20100101 Firefox/54.0\r\nAcce
pt-Encoding: gzip, deflate\r\nAccept: text/html,application/xhtml+xml,applicatio
n/xml;q=0.9,*/*;q=0.8\r\nConnection: keep-alive\r\nHost: smp.yoedge.com\r\nAccep
t-Language: zh-TW,en-US;q=0.7,en;q=0.3\r\nDNT: 1\r\nUpgrade-Insecure-Requests: 1
\r\n\r\n'
reply: 'HTTP/1.1 302 Moved Temporarily\r\n'
DEBUG:requests.packages.urllib3.connectionpool:http://smp.yoedge.com:80 "GET /sm
p-app/1017018/shinmangaplayer/index.html HTTP/1.1" 302 None
WARNING:requests.packages.urllib3.connectionpool:Failed to parse headers (url=ht
tp://smp.yoedge.com:80/smp-app/1017018/shinmangaplayer/index.html): [MissingHead
erBodySeparatorDefect()], unparsed data: 'Status 302: \r\nLocation: http://smp.y
oedge.com:80/smp-app/1017018/shinmangaplayer/index.html?__okraw\r\n\r\n'
Traceback (most recent call last):
  File "C:\Users\Owner\AppData\Roaming\Python\Python36\site-packages\requests\pa
ckages\urllib3\connectionpool.py", line 403, in _make_request
    assert_header_parsing(httplib_response.msg)
  File "C:\Users\Owner\AppData\Roaming\Python\Python36\site-packages\requests\pa
ckages\urllib3\util\response.py", line 66, in assert_header_parsing
    raise HeaderParsingError(defects=defects, unparsed_data=unparsed_data)
requests.packages.urllib3.exceptions.HeaderParsingError: [MissingHeaderBodySepar
atorDefect()], unparsed data: 'Status 302: \r\nLocation: http://smp.yoedge.com:8
0/smp-app/1017018/shinmangaplayer/index.html?__okraw\r\n\r\n'
header: Server header: Date header: Content-Type header: Transfer-Encoding heade
r: Connection

It seems that this behavior is builtin to python's stdlib:
https://github.com/kennethreitz/requests/issues/3098

Fixed in 2017.4.18.