docker-compose failed to start app_wes-worker_1 and app_wes_1

Question

docker-compose failed to start app_wes-worker_1 and app_wes_1

Closed this issue 2 years ago · 2 comments

Describe the bug
cwl-WES does not work after docker-compose up with multiple issues

docker logs -f app_flower_1:

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/site-packages/tornado/gen.py", line 828, in callback
    result_list.append(f.result())
  File "/usr/local/lib/python3.6/concurrent/futures/_base.py", line 398, in result
    return self.__get_result()
  File "/usr/local/lib/python3.6/concurrent/futures/_base.py", line 357, in __get_result
    raise self._exception
  File "/usr/local/lib/python3.6/concurrent/futures/thread.py", line 55, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/usr/local/lib/python3.6/site-packages/celery/app/control.py", line 126, in conf
    return self._request('conf', with_defaults=with_defaults)
  File "/usr/local/lib/python3.6/site-packages/celery/app/control.py", line 81, in _request
    timeout=self.timeout, reply=True,
  File "/usr/local/lib/python3.6/site-packages/celery/app/control.py", line 436, in broadcast
    limit, callback, channel=channel,
  File "/usr/local/lib/python3.6/site-packages/kombu/pidbox.py", line 304, in _broadcast
    chan = channel or self.connection.default_channel
  File "/usr/local/lib/python3.6/site-packages/kombu/connection.py", line 819, in default_channel
    self.connection
  File "/usr/local/lib/python3.6/site-packages/kombu/connection.py", line 802, in connection
    self._connection = self._establish_connection()
  File "/usr/local/lib/python3.6/site-packages/kombu/connection.py", line 757, in _establish_connection
    conn = self.transport.establish_connection()
  File "/usr/local/lib/python3.6/site-packages/kombu/transport/pyamqp.py", line 130, in establish_connection
    conn.connect()
  File "/usr/local/lib/python3.6/site-packages/amqp/connection.py", line 294, in connect
    self.transport.connect()
  File "/usr/local/lib/python3.6/site-packages/amqp/transport.py", line 120, in connect
    self._connect(self.host, self.port, self.connect_timeout)
  File "/usr/local/lib/python3.6/site-packages/amqp/transport.py", line 161, in _connect
    self.sock.connect(sa)
ConnectionRefusedError: [Errno 111] Connection refused

[[E 210329 14:39:53 events:123] Failed to capture events: '[Errno 111] Connection refused', trying again in 4 seconds.
[[E 210329 14:39:57 events:123] Failed to capture events: '[Errno 111] Connection refused', trying again in 8 seconds.
[[I 210329 14:40:05 mixins:224] Connected to amqp://guest:**@rabbitmq:5672//

docker logs -f app_wes-worker_1:

Traceback (most recent call last):
File "/usr/local/bin/celery", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.6/site-packages/celery/main.py", line 16, in main
_main()
File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 322, in main
cmd.execute_from_commandline(argv)
File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 484, in execute_from_commandline
super(CeleryCommand, self).execute_from_commandline(argv)))
File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 275, in execute_from_commandline
return self.handle_argv(self.prog_name, argv[1:])
File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 476, in handle_argv
return self.execute(command, argv)
File "/usr/local/lib/python3.6/site-packages/celery/bin/celery.py", line 408, in execute
).run_from_argv(self.prog_name, argv[1:], command=argv[0])
File "/usr/local/lib/python3.6/site-packages/celery/bin/worker.py", line 223, in run_from_argv
return self(*args, **options)
File "/usr/local/lib/python3.6/site-packages/celery/bin/base.py", line 238, in call
ret = self.run(*args, **kwargs)
File "/usr/local/lib/python3.6/site-packages/celery/bin/worker.py", line 257, in run
**kwargs)
File "/usr/local/lib/python3.6/site-packages/celery/worker/worker.py", line 96, in init
self.app.loader.init_worker()
File "/usr/local/lib/python3.6/site-packages/celery/loaders/base.py", line 114, in init_worker
self.import_default_modules()
File "/usr/local/lib/python3.6/site-packages/celery/loaders/base.py", line 109, in import_default_modules
return [self.import_task_module(m) for m in self.default_modules]
File "/usr/local/lib/python3.6/site-packages/celery/loaders/base.py", line 109, in
return [self.import_task_module(m) for m in self.default_modules]
File "/usr/local/lib/python3.6/site-packages/celery/loaders/base.py", line 95, in import_task_module
return self.import_from_cwd(module)
File "/usr/local/lib/python3.6/site-packages/celery/loaders/base.py", line 104, in import_from_cwd
package=package,
File "/usr/local/lib/python3.6/site-packages/celery/utils/imports.py", line 104, in import_from_cwd
return imp(module, package=package)
File "/usr/local/lib/python3.6/site-packages/celery/loaders/base.py", line 98, in import_module
return importlib.import_module(module, package=package)
File "/usr/local/lib/python3.6/importlib/init.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "", line 994, in _gcd_import
File "", line 971, in _find_and_load
File "", line 955, in _find_and_load_unlocked
File "", line 665, in _load_unlocked
File "", line 678, in exec_module
File "", line 219, in _call_with_frames_removed
File "/app/cwl_wes/tasks/tasks/cancel_run.py", line 11, in
from pymongo import collection as Collection
File "/usr/local/lib/python3.6/site-packages/pymongo/init.py", line 77, in
from pymongo.collection import ReturnDocument
File "/usr/local/lib/python3.6/site-packages/pymongo/collection.py", line 20, in
from bson.code import Code
File "/usr/local/lib/python3.6/site-packages/bson/code.py", line 18, in
from bson.py3compat import abc, string_type, PY3, text_type
ImportError: cannot import name 'abc'

docker-compose ps:

  Name                    Command                 State                                                    Ports                                              
app_flower_1 flower flower --broker=amq ... Up 0.0.0.0:5555->5555/tcp
app_mongodb_1 docker-entrypoint.sh mongod Up 0.0.0.0:27017->27017/tcp
app_rabbitmq_1 docker-entrypoint.sh rabbi ... Up 15671/tcp, 15672/tcp, 15691/tcp, 15692/tcp, 25672/tcp, 4369/tcp, 5671/tcp, 0.0.0.0:5672->5672/tcp
app_wes-worker_1 bash -c cd /app/cwl_wes; c ... Restarting
app_wes_1 bash -c cd /app/cwl_wes; g ... Restarting

Both services app_wes-worker_1 and app_wes_1 are always in Restarting state.

To Reproduce
I tried the latest git version as described in the documentation:

Create dirs:
Clone the repo: git clone https://github.com/elixir-cloud-aai/cwl-WES.git app
Modify the config with local TES settings: vi cwl_wes/config/app_config.yaml
Start the services: docker-compose up -d --build

I also tried the version from the release https://github.com/elixir-cloud-aai/cwl-WES/releases/tag/v0.1.1-dev but with same outcome.

Expected behavior
I expect all services up and running with WES API accessible via http://0.0.0.0:8080/ga4gh/wes/v1/ui

Software
Azure Ubuntu VM as
uname -a
Linux genx 5.4.0-1023-azure #23~18.04.1-Ubuntu SMP Thu Aug 20 14:46:48 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux

docker -v
Docker version 19.03.6, build 369ce74a3c

docker-compose -v
docker-compose version 1.28.6, build 5db8d86f

Additional context
I am working on linking together TRS+RDS->WES->TES use case

Answer 1 · 2021-03-30T09:16:48.000Z

@uniqueg
seems that the issue was related to py-bson/bson#82
in short, you don't need a py-bson if you installing pymongo too
I simply commented out py-bson from the requirements.txt and re-build the container

Answer 2 · 2022-06-19T21:49:58.000Z

Thanks so much