fail crashes client thread
Closed this issue · 0 comments
Describe the bug
A job failed and the client threw a stack trace leading to the thread crashing and the client no longer accepting work
Aug 24 21:32:50 overcloud-controller-0 directord[990448]: DEBUG Found task results for [ 6ad24e42-308e-41ab-895a-da85f58f1a32 ].
Aug 24 21:32:50 overcloud-controller-0 directord[990448]: ERROR Job failed [ 6ad24e42-308e-41ab-895a-da85f58f1a32 ]
Aug 24 21:32:50 overcloud-controller-0 directord[990448]: Process Process-3:
Aug 24 21:32:50 overcloud-controller-0 directord[990448]: Traceback (most recent call last):
Aug 24 21:32:50 overcloud-controller-0 directord[990448]: File "/opt/directord/lib64/python3.8/site-packages/zmq/sugar/socket.py", line 576, in send_multipart
Aug 24 21:32:50 overcloud-controller-0 directord[990448]: memoryview(msg)
Aug 24 21:32:50 overcloud-controller-0 directord[990448]: TypeError: memoryview: a bytes-like object is required, not 'str'
Aug 24 21:32:50 overcloud-controller-0 directord[990448]: During handling of the above exception, another exception occurred:
Aug 24 21:32:50 overcloud-controller-0 directord[990448]: Traceback (most recent call last):
Aug 24 21:32:50 overcloud-controller-0 directord[990448]: File "/usr/lib64/python3.8/multiprocessing/process.py", line 315, in _bootstrap
Aug 24 21:32:50 overcloud-controller-0 directord[990448]: self.run()
Aug 24 21:32:50 overcloud-controller-0 directord[990448]: File "/usr/lib64/python3.8/multiprocessing/process.py", line 108, in run
Aug 24 21:32:50 overcloud-controller-0 directord[990448]: self._target(*self._args, **self._kwargs)
Aug 24 21:32:50 overcloud-controller-0 directord[990448]: File "/opt/directord/lib64/python3.8/site-packages/directord/client.py", line 557, in run_job
Aug 24 21:32:50 overcloud-controller-0 directord[990448]: self.job_q_results()
Aug 24 21:32:50 overcloud-controller-0 directord[990448]: File "/opt/directord/lib64/python3.8/site-packages/directord/client.py", line 430, in job_q_results
Aug 24 21:32:50 overcloud-controller-0 directord[990448]: self._set_job_status(
Aug 24 21:32:50 overcloud-controller-0 directord[990448]: File "/opt/directord/lib64/python3.8/site-packages/directord/utils.py", line 110, in exit
Aug 24 21:32:50 overcloud-controller-0 directord[990448]: self.ctx.driver.socket_send(
Aug 24 21:32:50 overcloud-controller-0 directord[990448]: File "/opt/directord/lib64/python3.8/site-packages/directord/drivers/zmq.py", line 340, in socket_send
Aug 24 21:32:50 overcloud-controller-0 directord[990448]: return socket.send_multipart(message_parts)
Aug 24 21:32:50 overcloud-controller-0 directord[990448]: File "/opt/directord/lib64/python3.8/site-packages/zmq/sugar/socket.py", line 581, in send_multipart
Aug 24 21:32:50 overcloud-controller-0 directord[990448]: raise TypeError(
Aug 24 21:32:50 overcloud-controller-0 directord[990448]: TypeError: Frame 4 ('Job omitted, parent failure') does not support the buffer interface.