Only 1 active worker from 800
Closed this issue · 1 comments
ludovit-ubrezi commented
Hello, running worker with this configuration
func main() {
concurrency := runtime.NumCPU() * 200
srv := asynq.NewServer(
asynq.RedisClientOpt{Addr: redisHost + ":" + redisPort, Password: redisPassword, DB: 1},
asynq.Config{
Concurrency: concurrency,
Queues: map[string]int{tasks.QueueName: 9},
StrictPriority: true,
},
)
// close publisher connection functionality
defer tasks.Publisher.Channel.Close()
// Use asynq.HandlerFunc adapter for a handler function
if err := srv.Run(asynq.HandlerFunc(tasks.HandleMonitorDeliveryTask)); err != nil {
log.Fatal(err)
}
}
Problem is that even Concurency is set to 800 workers there is always only 1 worker processing queue. This is why the Pending queue is not processing correctly and whole system has delays. (screenshots below)
Code of function tasks.HandleMonitorDeliveryTask is working correctly and fast because when i load all the data, it processes Task with group of 5000 items and it load into RabbitMQ -> 1 million items in e.g. 1 minute.
But when Task has 1 item it slows down due to only 1 worker processing this queue even when there is 800 workers setup for server it only run 1.
Is there something what i missing in configuration? How to setup it?
ludovit-ubrezi commented
bad forum