解析不了web,python3.11、feapder1.8.5
pigeon-fancier opened this issue · 1 comments
pigeon-fancier commented
需知
升级feapder,保证feapder是最新版,若BUG仍然存在,则详细描述问题
pip install --upgrade feapder
解析不了web
File "D:\application\code-env\python3.11\Lib\site-packages\feapder\core\parser_control.py", line 568, in deal_request
results = parser.parse(request, response)
│ │ │ └ <Response [200]>
│ │ └ <Request https://m.baidu.com/>
│ └ <function SpiderTest.parse at 0x000001ACC6F16DE0>
└ <SpiderTest(Thread-1, started 16256)>
import feapder
class SpiderTest(feapder.AirSpider):
def start_requests(self):
# for i in range(1, 15):
yield feapder.Request("https://m.baidu.com/")
def parse(self, request, response):
response.encoding_errors = 'ignore'
print(response.text)
def parse_detail(self, request, response):
"""
解析详情
"""
# 取url
url = request.url
# 取title
title = request.title
# 解析正文
content = response.xpath(
'string(//div[@class="content"])'
).extract_first() # string 表达式是取某个标签下的文本,包括子标签文本
print("url", url)
print("title", title)
print("content", content)
if __name__ == "__main__":
SpiderTest().start()
Boris-code commented
3.11 用 feapder==1.8.6b6
pip3 install feapder==1.8.6b6
后面会发布正式版