New greenlet version terminates phovea server
Closed this issue · 5 comments
keckelt commented
- Release number or git hash: develop branch
- Environment (local or deployed): local and daily/nightly builds
Steps to reproduce
- Build & run latest ordino/coral product (
docker-compose build --pull --no-cache)
Observed behavior
phovea server terminates with Segmentation fault errors:
api_1 | init config
api_1 | configuration file: /phovea/config.json
api_1 | 09:53:41 INFO phovea_server.launch: enabling development mode
api_1 | 09:53:41 INFO phovea_server._plugin_parser: looking for development neighbors
api_1 | 09:53:41 INFO phovea_server._plugin_parser: discovered 12 plugins: ['cohort', 'phovea_clue', 'phovea_data_mongo', 'phovea_data_redis', 'phovea_security_flask', 'phovea_security_store_generated', 'phovea_server', 'phovea_ui', 'phovea_workspace', 'phovea_workspace', 'tdp_core', 'tdp_publicdb']
api_1 | 09:53:41 INFO phovea_server._plugin_parser: importing module: cohort
api_1 | 09:53:41 INFO phovea_server._plugin_parser: importing module: phovea_clue
api_1 | 09:53:41 INFO phovea_server._plugin_parser: importing module: phovea_data_mongo
api_1 | 09:53:41 INFO phovea_server._plugin_parser: importing module: phovea_data_redis
api_1 | 09:53:41 INFO phovea_server._plugin_parser: importing module: phovea_security_flask
api_1 | 09:53:41 INFO phovea_server._plugin_parser: importing module: phovea_security_store_generated
api_1 | 09:53:41 INFO phovea_server._plugin_parser: importing module: phovea_server
api_1 | 09:53:41 INFO phovea_server._plugin_parser: importing module: tdp_core
api_1 | 09:53:41 INFO phovea_server._plugin_parser: importing module: tdp_publicdb
api_1 | 09:53:41 INFO phovea_server.launch: add command api
api_1 | 09:53:41 INFO phovea_server.launch: loading and initializing the command: api
api_1 | 09:53:41 INFO phovea_server.plugin: importing phovea_server.server
api_1 | /usr/local/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144 from C header, got 152 from PyObject
api_1 | return f(*args, **kwds)
api_1 | /usr/local/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144 from C header, got 152 from PyObject
api_1 | return f(*args, **kwds)
api_1 | /usr/local/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144 from C header, got 152 from PyObject
api_1 | return f(*args, **kwds)
api_1 | /usr/local/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144 from C header, got 152 from PyObject
api_1 | return f(*args, **kwds)
api_1 | /usr/local/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144 from C header, got 152 from PyObject
api_1 | return f(*args, **kwds)
api_1 | /usr/local/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144 from C header, got 152 from PyObject
api_1 | return f(*args, **kwds)
api_1 | /usr/local/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144 from C header, got 152 from PyObject
api_1 | return f(*args, **kwds)
api_1 | /usr/local/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144 from C header, got 152 from PyObject
api_1 | return f(*args, **kwds)
api_1 | /usr/local/lib/python3.7/importlib/_bootstrap.py:219: RuntimeWarning: greenlet.greenlet size changed, may indicate binary incompatibility. Expected 144 from C header, got 152 from PyObject
api_1 | return f(*args, **kwds)
api_1 | 09:53:41 INFO cohort.db: Setting up the db view.
api_1 | 09:53:41 INFO cohort.db: Creating a DBConnector for the cohort database.
api_1 | 09:53:41 INFO phovea_server.plugin: importing tdp_publicdb.sql
api_1 | 09:53:41 INFO phovea_server.plugin: read currently executed command from config: unknown
api_1 | 09:53:41 INFO phovea_server.plugin: creating singleton idmanager phovea_data_redis.assigner
api_1 | 09:53:41 INFO phovea_server.plugin: creating singleton cachemanager phovea_data_redis.cache
api_1 | 09:53:41 INFO phovea_server.plugin: creating singleton security_manager phovea_security_flask.flask_login_impl
api_1 | 09:53:41 INFO phovea_server.plugin: creating singleton mappingmanager phovea_server.mapper
api_1 | 09:53:41 INFO phovea_server.plugin: creating singleton db-migration-manager tdp_core.dbmigration
api_1 | 09:53:41 INFO phovea_server.plugin: creating singleton registry server
api_1 | 09:53:41 INFO phovea_server.plugin: importing tdp_core.dbmigration
api_1 | 09:53:41 INFO tdp_core.dbmigration: Initializing DBMigrationManager
api_1 | 09:53:41 INFO tdp_core.dbmigration: DBMigration found: tdp_publicdb
api_1 | 09:53:41 INFO tdp_core.dbmanager: run greenifier: psycopg2
api_1 | 09:53:41 INFO phovea_server.plugin: importing tdp_core.sql_use_gevent
api_1 | 09:53:41 INFO tdp_core.sql_use_gevent: patching psycopg2 to be green
api_1 | 09:53:42 INFO phovea_server.launch: add command instance to parser
api_1 | 09:53:42 INFO phovea_server.server: init application: <Flask 'phovea_server.mainapp'>
api_1 | 09:53:42 INFO phovea_server.plugin: importing phovea_security_flask.flask_login_impl
api_1 | 09:53:42 INFO phovea_server.plugin: importing phovea_security_store_generated.store
api_1 | 09:53:42 INFO phovea_server.server: add application: db_connector at namespace: /api/cohortdb/db
api_1 | 09:53:42 INFO phovea_server.server: add application: caleydo-clue-screenshot at namespace: /api/clue
api_1 | 09:53:42 INFO phovea_server.server: add application: caleydo-dataset at namespace: /api/dataset
api_1 | 09:53:42 INFO phovea_server.server: add application: caleydo-idtype at namespace: /api/idtype
api_1 | 09:53:42 INFO phovea_server.server: add application: tdp_core at namespace: /api/tdp/proxy
api_1 | 09:53:42 INFO phovea_server.server: add application: db_connector at namespace: /api/tdp/db
api_1 | 09:53:42 INFO phovea_server.server: add application: tdp_storage at namespace: /api/tdp/storage
api_1 | 09:53:42 INFO phovea_server.server: add application: tdp_swagger at namespace: /api/tdp/ui
api_1 | 09:53:42 INFO phovea_server.server: add application: tdp_config at namespace: /api/tdp/config
api_1 | 09:53:42 INFO phovea_server.server: add application: tdp_xlsx2json at namespace: /api/tdp/xlsx
api_1 | 09:53:42 INFO phovea_server.server: add application: db-migration-api at namespace: /api/tdp/db-migration
api_1 | 09:53:42 INFO phovea_server.server: prepare server that will listen on 0.0.0.0:80
api_1 | Segmentation fault
app-cohort_api_1 exited with code 139
From the nightly builds I see that there is a new greenlet version used:
keckelt commented
The greenlet versin 4.17 above was released yesterday: https://pypi.org/project/greenlet/#history
It's a dependency through gevent: https://github.com/python-greenlet/greenlet#who-is-using-greenlet
And gevent also released a new version yesterday, assumingly to stay compatible: https://pypi.org/project/gevent/#history
thinkh commented
I can confirm that this error is caused by the new greenlet version 0.4.17. Other people noticed the issue as well: python-greenlet/greenlet#178
For now just use the old version by adding greenlet==0.4.16 to the requirements.txt.
thinkh commented
And gevent also released a new version yesterday, assumingly to stay compatible: pypi.org/project/gevent/#history
Maybe updating gevent solves the issue?
jamadden commented
gevent 20.9.0 was released at the same time greenlet 0.4.17 was un-yanked. On Python 3.7 and higher, gevent 20.9.0 is required to use (a standard build of) greenlet 0.4.17.
gevent 1.4 was never tested with any release of greenlet beyond 0.4.14, and does not support Python 3.8 or 3.9.