ydf0509/funboost

python3.6.8和pymongo4.0.2有bug

yan1990y opened this issue · 4 comments

mongodb是4.0版本,无密码,也能看到启动正常,但是一旦发布任务再消费,就会报错。
`2023-02-03 01:01:30 - function_error - "/usr/local/lib/python3.6/site-packages/funboost/utils/decorators.py:92" - __handle_exception - ERROR - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
记录错误日志,调用方法--> [ concrete_realization_of_publish ] 第 0 次错误重试, 错误类型是:<class 'pymongo.errors.AutoReconnect'> 127.0.0.1:27017: connection pool paused
pymongo.errors.AutoReconnect: connection pool paused

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.6/site-packages/funboost/utils/decorators.py", line 76, in __handle_exception
result = func(*args, **keyargs)
File "/usr/local/lib/python3.6/site-packages/funboost/publishers/mongomq_publisher.py", line 24, in concrete_realization_of_publish
self.queue.put(json.loads(msg))
File "/usr/local/lib/python3.6/site-packages/funboost/utils/dependency_packages/mongomq/mongomq.py", line 106, in put
return self.collection.insert_one(job)
File "/usr/local/lib64/python3.6/site-packages/pymongo/collection.py", line 549, in insert_one
session=session,
File "/usr/local/lib64/python3.6/site-packages/pymongo/collection.py", line 494, in _insert_one
self.__database.client._retryable_write(acknowledged, _insert_command, session)
File "/usr/local/lib64/python3.6/site-packages/pymongo/mongo_client.py", line 1386, in _retryable_write
return self._retry_with_session(retryable, func, s, None)
File "/usr/local/lib64/python3.6/site-packages/pymongo/mongo_client.py", line 1274, in _retry_with_session
return self._retry_internal(retryable, func, session, bulk)
File "/usr/local/lib64/python3.6/site-packages/pymongo/mongo_client.py", line 1299, in _retry_internal
with self._get_socket(server, session) as sock_info:
File "/usr/lib64/python3.6/contextlib.py", line 81, in enter
return next(self.gen)
File "/usr/local/lib64/python3.6/site-packages/pymongo/mongo_client.py", line 1136, in _get_socket
with server.get_socket(self.__all_credentials, handler=err_handler) as sock_info:
File "/usr/lib64/python3.6/contextlib.py", line 81, in enter
return next(self.gen)
File "/usr/local/lib64/python3.6/site-packages/pymongo/pool.py", line 1407, in get_socket
sock_info = self._get_socket(all_credentials)
File "/usr/local/lib64/python3.6/site-packages/pymongo/pool.py", line 1472, in _get_socket
self._raise_if_not_ready(emit_event=True)
File "/usr/local/lib64/python3.6/site-packages/pymongo/pool.py", line 1443, in _raise_if_not_ready
_raise_connection_failure(self.address, AutoReconnect("connection pool paused"))
File "/usr/local/lib64/python3.6/site-packages/pymongo/pool.py", line 270, in _raise_connection_failure
raise AutoReconnect(msg) from error
pymongo.errors.AutoReconnect: 127.0.0.1:27017: connection pool paused`
网上查说pymongo版本有bug,但是pip安装高版本会提示与funboost的版本要求不兼容。

经测试,不仅mongodb作为消息队列的中间件无法使用,连开启消费状态、结果web页面的库也写不进去。但是在show dbs的时候能看到库已经建立了

可以安装高版本pymongo,提示无所谓,自己看文档文档10.0

image

主要是你还没有说服务端mongo版本,以及mongo连接有没有写错

其实我第一句就写了~~服务端是4.0的,连接没错,因为已经看到db已经建立了。我先试试高版本

后来经过测试,发现是因为我用uwsgi部署,设置了processes=4,在4线程的情况下就会出现这个问题。如果改成processes=1或者不用uwsgi,直接单线程运行都是可以的。目前没有彻底解决这个问题,暂时的解决方案是单独把需要用到任务队列的任务单独部署,设置线程为1,然后内部调用。