Worker can't complete Tasks
Closed this issue · 1 comments
Hello,
my new worker has processed some tasks, but it can't complete any tasks - I'm frustrating.
The log of the docker container shows:
2019-11-24 04:44:06.284 INFO 7 --- [taskScheduler-1] c.i.worker.executor.TaskManagerService : Some tasks are using CPU [tasksUsingCpu:1, maxTasksUsingCpu:3] 2019-11-24 04:44:36.284 INFO 7 --- [taskScheduler-1] c.i.worker.executor.TaskManagerService : Some tasks are using CPU [tasksUsingCpu:1, maxTasksUsingCpu:3] 2019-11-24 04:44:45.569 INFO 7 --- [cTaskExecutor-1] c.i.worker.pubsub.SubscriptionService : PubSub service received taskNotification [chainTaskId:0x5dfd2a5c9e6ef87fd44b3ba7ca0230efb2f9e3f2441b3b4f514c9b170edf1daa, type:PLEASE_ABORT_CONSENSUS_REACHED] 2019-11-24 04:44:45.570 INFO 7 --- [ task-24] c.i.worker.executor.TaskManagerService : Unset task using CPU [tasksUsingCpu:0] 2019-11-24 04:44:45.571 INFO 7 --- [ task-24] com.iexec.worker.utils.FileHelper : Folder has been deleted [path:/tmp/iexec-worker/-ZeraWorker02/0x5dfd2a5c9e6ef87fd44b3ba7ca0230efb2f9e3f2441b3b4f514c9b170edf1daa] 2019-11-24 04:44:45.571 INFO 7 --- [ task-24] com.iexec.worker.result.ResultService : The result of the chainTaskId has been deleted [chainTaskId:0x5dfd2a5c9e6ef87fd44b3ba7ca0230efb2f9e3f2441b3b4f514c9b170edf1daa] 2019-11-24 04:46:58.600 INFO 7 --- [cTaskExecutor-1] c.i.worker.pubsub.SubscriptionService : PubSub service received taskNotification [chainTaskId:0x5dfd2a5c9e6ef87fd44b3ba7ca0230efb2f9e3f2441b3b4f514c9b170edf1daa, type:PLEASE_COMPLETE] 2019-11-24 04:46:58.601 INFO 7 --- [ task-25] c.i.w.executor.TaskNotificationService : update replicate request [chainTaskId:0x5dfd2a5c9e6ef87fd44b3ba7ca0230efb2f9e3f2441b3b4f514c9b170edf1daa, status:COMPLETING, details:null] 2019-11-24 04:46:58.785 ERROR 7 --- [ task-25] com.iexec.worker.feign.BaseFeignClient : Failed to make http call [action:updateReplicateStatus, status:403 FORBIDDEN, attempt:0] 2019-11-24 04:47:00.952 ERROR 7 --- [ task-25] com.iexec.worker.feign.BaseFeignClient : Failed to make http call [action:updateReplicateStatus, status:403 FORBIDDEN, attempt:1] 2019-11-24 04:47:03.108 ERROR 7 --- [ task-25] com.iexec.worker.feign.BaseFeignClient : Failed to make http call [action:updateReplicateStatus, status:403 FORBIDDEN, attempt:2] 2019-11-24 04:47:05.108 INFO 7 --- [ task-25] c.i.w.executor.TaskNotificationService : update replicate response [chainTaskId:0x5dfd2a5c9e6ef87fd44b3ba7ca0230efb2f9e3f2441b3b4f514c9b170edf1daa, status:COMPLETING, next:null] 2019-11-24 04:47:05.108 INFO 7 --- [ task-25] c.i.worker.executor.TaskManagerService : Unset task using CPU [tasksUsingCpu:0] 2019-11-24 04:47:05.108 INFO 7 --- [ task-25] com.iexec.worker.utils.FileHelper : Folder doesn't exist so can't be deleted [path:/tmp/iexec-worker/-ZeraWorker02/0x5dfd2a5c9e6ef87fd44b3ba7ca0230efb2f9e3f2441b3b4f514c9b170edf1daa] 2019-11-24 04:47:05.108 WARN 7 --- [ task-25] com.iexec.worker.result.ResultService : The result of the chainTaskId couldn't be deleted [chainTaskId:0x5dfd2a5c9e6ef87fd44b3ba7ca0230efb2f9e3f2441b3b4f514c9b170edf1daa, deletedInMap:false, deletedTaskFolder:false] 2019-11-24 04:47:05.108 INFO 7 --- [ task-25] c.i.w.executor.TaskNotificationService : update replicate request [chainTaskId:0x5dfd2a5c9e6ef87fd44b3ba7ca0230efb2f9e3f2441b3b4f514c9b170edf1daa, status:COMPLETE_FAILED, details:null] 2019-11-24 04:47:05.397 ERROR 7 --- [ task-25] com.iexec.worker.feign.BaseFeignClient : Failed to make http call [action:updateReplicateStatus, status:403 FORBIDDEN, attempt:0] 2019-11-24 04:47:07.810 ERROR 7 --- [ task-25] com.iexec.worker.feign.BaseFeignClient : Failed to make http call [action:updateReplicateStatus, status:403 FORBIDDEN, attempt:1] 2019-11-24 04:47:09.975 ERROR 7 --- [ task-25] com.iexec.worker.feign.BaseFeignClient : Failed to make http call [action:updateReplicateStatus, status:403 FORBIDDEN, attempt:2] 2019-11-24 04:47:11.975 INFO 7 --- [ task-25] c.i.w.executor.TaskNotificationService : update replicate response [chainTaskId:0x5dfd2a5c9e6ef87fd44b3ba7ca0230efb2f9e3f2441b3b4f514c9b170edf1daa, status:COMPLETE_FAILED, next:null] 2019-11-24 04:47:11.975 INFO 7 --- [ task-25] c.i.worker.pubsub.SubscriptionService : Received notification [notification:TaskNotification(chainTaskId=0x5dfd2a5c9e6ef87fd44b3ba7ca0230efb2f9e3f2441b3b4f514c9b170edf1daa, workersAddress=[], taskNotificationType=PLEASE_COMPLETE, taskNotificationExtra=null)] 2019-11-24 04:47:12.713 INFO 7 --- [ task-25] c.i.worker.pubsub.SubscriptionService : Unsubscribed from topic [chainTaskId:0x5dfd2a5c9e6ef87fd44b3ba7ca0230efb2f9e3f2441b3b4f514c9b170edf1daa] 2019-11-24 04:47:12.714 WARN 7 --- [ task-25] c.i.w.executor.TaskNotificationService : No more actions to do [chainTaskId:0x5dfd2a5c9e6ef87fd44b3ba7ca0230efb2f9e3f2441b3b4f514c9b170edf1daa] 2019-11-24 04:54:46.789 ERROR 7 --- [taskScheduler-1] com.iexec.worker.feign.BaseFeignClient : Failed to make http call [action:ping, status:401 UNAUTHORIZED, attempt:0] 2019-11-24 04:54:49.202 ERROR 7 --- [taskScheduler-1] com.iexec.worker.feign.BaseFeignClient : Failed to make http call [action:ping, status:401 UNAUTHORIZED, attempt:1] 2019-11-24 04:54:51.508 ERROR 7 --- [taskScheduler-1] com.iexec.worker.feign.BaseFeignClient : Failed to make http call [action:ping, status:401 UNAUTHORIZED, attempt:2] 2019-11-24 04:54:53.833 WARN 7 --- [taskScheduler-1] com.iexec.worker.PingService : The worker cannot ping the core! [sessionId:]
The Worker becomes every time type:PLEASE_ABORT_CONSENSUS_REACHED - It deletes the Resultset.
After that it received: type:PLEASE_COMPLETE - It will completing, but can't push the result because it is deleted.
The type:PLEASE_ABORT_CONSENSUS_REACHED is wrong, i have check the task yesterday and the limit for this are in 4days... - In the WebStatusPage it is not synchron with the worker Status ...
All tasks that the Worker received shows COMPUTING.
Check it here: https://public-pool.iex.ec/d/WorkerInfo/worker-info?orgId=2&var-worker=0x7019dfbe066d27d76e99c428227f7dae5aa6db8b
I won't resolve tasks for free - and when a worker aborted - the requester should pay a small money for processing. I think the system is probably unfair.
Please fix the bugs so that the great project has a future.
Thank you @Jonas18175 for reporting. We released the version 4 of the worker where we added some enhancements and fixed some issues. Please update and open another issue if you have problems.
Closing this...