qinxuye/cola

爬取新浪微博出错

Aya-Yang opened this issue · 3 comments

Aya-Yang commented

在Ubuntu环境下单机爬取新浪微博是正常的，但是在Mac上会出错，错误信息是'CounterClient' object has no attribute 'inc',此外在Ubuntu上当通过python虚拟环境跑weibo的__init.py也会有同样的问题。部分错误代码

AttributeError: 'CounterClient' object has no attribute 'inc' get 3211200050 url: http://weibo.com/3211200050/follow get 1898353550 url: http://weibo.com/1898353550/follow Error when handle bundle: 1898353550, url: http://weibo.com/1898353550/follow 'CounterClient' object has no attribute 'inc' Traceback (most recent call last): File "/usr/local/lib/python2.7/site-packages/cola/job/executor.py", line 519, in _parse_with_process_exception res = self._parse(parser_cls, options, bundle, url) File "/usr/local/lib/python2.7/site-packages/cola/job/executor.py", line 447, in _parse for obj in res: File "/Users/xuewei/cola/app/weibo/parsers.py", line 707, in parse self.counter.inc('processed_%s_list_page' % counter_type, 1) AttributeError: 'CounterClient' object has no attribute 'inc'

qinxuye commented

看错误感觉是你mac和ubuntu上的cola版本过旧了。

先试下更新cola的代码看看。

Aya-Yang commented

确实是这样，可能我直接通过pip install cola安装的是一个旧的版本，用python setup.py install就好了，谢谢

qinxuye commented

我重新发布了版本，可能之前pypi上的比较旧了。
现在用pip install应该也没有问题了。