hibiken/asynqmon

Only 1 active worker from 800

Closed this issue · 1 comments

Hello, running worker with this configuration

func main() {
	concurrency := runtime.NumCPU() * 200

	srv := asynq.NewServer(
		asynq.RedisClientOpt{Addr: redisHost + ":" + redisPort, Password: redisPassword, DB: 1},
		asynq.Config{
			Concurrency:    concurrency,
			Queues:         map[string]int{tasks.QueueName: 9},
			StrictPriority: true,
		},
	)

	// close publisher connection functionality
	defer tasks.Publisher.Channel.Close()

	// Use asynq.HandlerFunc adapter for a handler function
	if err := srv.Run(asynq.HandlerFunc(tasks.HandleMonitorDeliveryTask)); err != nil {
		log.Fatal(err)
	}
}

Problem is that even Concurency is set to 800 workers there is always only 1 worker processing queue. This is why the Pending queue is not processing correctly and whole system has delays. (screenshots below)

Code of function tasks.HandleMonitorDeliveryTask is working correctly and fast because when i load all the data, it processes Task with group of 5000 items and it load into RabbitMQ -> 1 million items in e.g. 1 minute.

But when Task has 1 item it slows down due to only 1 worker processing this queue even when there is 800 workers setup for server it only run 1.

Is there something what i missing in configuration? How to setup it?

Screenshot 2024-02-27 at 12 30 38
Screenshot 2024-02-27 at 12 30 06

bad forum