SpiderClub/weibospider
:zap: A distributed crawler for weibo, building with celery and requests.
PythonMIT
Issues
- 6
运行 python3 config/create_all.py 报错
#218 opened by Shallowhave - 1
运行python config/create_all.py 报错
#217 opened by Shallowhave - 0
- 0
- 1
请问这个爬虫爬取关键词的话是只能爬取50页的上限吗?
#214 opened by SinruW - 2
登入帐号时遇到要求扫码登入,是Weibo有改版吗?
#210 opened by leo1357904 - 2
非酋做配置,试错笔记
#212 opened by JeanYoung5 - 0
微博爬虫的合理阈值
#213 opened by chenyinghong - 1
爬取不到数据,启动 work 页面 输出的都是一些爬取失败 和 warning 信息 类似:
#211 opened by gaoshangle - 0
threading.Thread.isAlive has been deprecated and removed in Python 3.9 in favour of is_alive
#208 opened by tirkarthi - 9
云打码平台好像失效了,之前那个超级鹰平台的issues下的temp_verification我按照操作来可是出了奇怪的bug,请问能根据新的打码平台更新一下吗,麻烦了
#207 opened by zjyzh - 2
请问如何添加代理池?
#192 opened by myrainbowandsky - 0
运行worker 就报错了,我的redie 配置和爬虫配置密码都对的:[2020-04-25 19:53:50,902: ERROR/MainProcess] consumer: Cannot connect to redis://:**@localhost:6379/6: Client sent AUTH, but no password is set.
#206 opened by uyplayer - 1
执行login_first.py之后显示ValueError: not enough values to unpack (expected 3, got 0)
#205 opened by keithkang1986 - 2
启动worker时执行到**[2020-04-02 12:36:58,850: INFO/MainProcess] mingle: all alone**就不再继续
#204 opened by keithkang1986 - 6
无法看见转推信息; 评论,点赞,回复数等也是0
#190 opened by myrainbowandsky - 0
- 3
请问可以抓到多久之前的围脖
#188 opened by myrainbowandsky - 1
能否用多线程代替多台机器来爬取?
#191 opened by myrainbowandsky - 0
抓取 user_relation。 user.py 有bug
#202 opened by myrainbowandsky - 0
- 16
mysql数据库里user_relation这样表 一直是空,是哪里有问题?
#199 opened by myrainbowandsky - 3
如何限定时间段,爬取从某年月日到某年月日的微博?
#198 opened by myrainbowandsky - 3
- 0
- 1
请问repost_crawler抓取的数据意义
#195 opened by myrainbowandsky - 8
- 2
微博不显示等级
#193 opened by thekingofcity - 4
根据说明文档执行,这个程序运行不了
#179 opened by GXNU156489 - 3
能不能用mongo而不用mysql?
#184 opened by myrainbowandsky - 2
如何在一台机器上同时搜索多个关键词?
#189 opened by myrainbowandsky - 0
请问如何验证程序在抓取数据了
#187 opened by myrainbowandsky - 0
Redis transport requires redis-py versions 3.2.0 or later. You have 2.10.5
#186 opened by myrainbowandsky - 0
无法用python config/create_all.py 制表
#185 opened by myrainbowandsky - 13
- 2
【关于云打码】目前看来是失效了,近期我会针对这块做个兼容。
#182 opened by OneCodeMonkey - 4
安装依赖lib的时候 celery==4.1.0报错
#183 opened by logie - 2
不设置邮箱可以么?邮箱那部分不知道怎么设置,看不太懂。
#180 opened by mengguiyouziyi - 6
- 2
worker启动后会自动的退出
#176 opened by holoodst - 0
在哪新建weibo数据库?
#175 opened by fcrw - 0
- 1
- 2
全部按正常流程进行的,最后启动worker命令失败,输出日志如下
#170 opened by OneCodeMonkey - 0
为啥我的 celery 启动worker 正常成功了,用flower查看是确显示 offline,而且执行 python user_first.py 等等也没爬到信息?
#169 opened by OneCodeMonkey - 1
注释太少了 有些源码看不懂鸭
#167 opened by 1814931012 - 2
No cookie in cookies pool. Maybe all accounts are banned, or all cookies are expired
#166 opened by xzzlhjdn - 2
Message Error: Couldn't apply scheduled task user_task: MISCONF Redis is configured to save RDB snapshots, but is currently not able to persist on disk. Commands that may modify the data set are disabled. Please check Redis logs for details about the error.
#165 opened by xzzlhjdn - 2
用户主页的新内容不能抓取
#164 opened by zengjian - 2
报错requests.exceptions.ConnectionError: HTTPSConnectionPool(host='passport.weibo.com', port=443): Max retries exceeded with url: /visitor/genvisitor
#163 opened by xzzlhjdn