Config with only subscribers
nico3dfx opened this issue · 11 comments
Hi,
I have a config file with about 3k queues, bindings and subscriptions.
Every time the process starts I have a warmup time of about 15 minutes. The greatest time is for checking the queues and the bindings.
A little time is spent to subscribe the consumer to the queue.
Is it possible to have a config for another process with queues and bindings and a config for a process with only subscriptions to speed up the time for the consumers to listen the queues?
Thanks,
Nico.
Hi Nico,
Having two configs doesn't make much sense to me, they would both run in parallel, so you would risk subscribing to a queue that didn't exist.
Alternatively if you are confident the queue exists and are configured correctly, you can disable checking the queues / exchanges.
Another thing that might help is if there was an async option. Rascal this serially because it makes it far easier to work out where problem occurs, but I have no problem performing the checks asynchronously or even in batches.
If you're happy to have a go, I think it should be a case of changing this line to use async.eachLimit.
The default limit should be 1 for backwards compatibility. I'm not sure how I would specify a different value through config though, as the "queues" config object contains a set of queues, with each key being the queue name. It might have to go at the top level instead, so a little more thought required there.
Rinse and repeat for https://github.com/guidesmiths/rascal/blob/master/lib/amqp/tasks/assertExchanges.js#L6 and https://github.com/guidesmiths/rascal/blob/master/lib/amqp/tasks/applyBindings.js#L12
Hi Cressie,
if can help you, with debug active I notice that several minutes are "lost" when Rascal apply bindings to exchange
2022-09-28T10:37:26.201Z rascal:tasks:applyBindings Binding queue: QUEUE_NAME to exchange: EXCHANGE_NAME with binding key: FIRST.SECOND.other1
2022-09-28T10:37:26.241Z rascal:tasks:applyBindings Binding queue: QUEUE_NAME to exchange: EXCHANGE_NAME with binding key: FIRST.SECOND.other2
2022-09-28T10:37:26.283Z rascal:tasks:applyBindings Binding queue: QUEUE_NAME to exchange: EXCHANGE_NAME with binding key: FIRST.SECOND.other3
2022-09-28T10:37:26.324Z rascal:tasks:applyBindings Binding queue: QUEUE_NAME to exchange: EXCHANGE_NAME with binding key: FIRST.SECOND.other4
(and so on)
I have 6k log rows of "rascal:tasks:applyBindings" with a test configuration file of 450 queues!
Time:
- start: 2022-09-28T10:33:50.768Z
- end: 2022-09-28T10:38:00.839Z
About 4+ minutes to have consumers in every queue.
Unfortunately making the calls asynchronous doesn't help. The reason is that behind the scenes amqplib is waiting for a response before sending the next call.
With a creating 1,000 exchanges with a concurrency of 10 resulted in the following debug (I added a post success message locally)
rascal:tasks:assertExchanges Asserting exchange: a0490c6c-3814-453d-b0bf-253cb28bf693:e1 +0ms
rascal:tasks:assertExchanges Asserting exchange: a0490c6c-3814-453d-b0bf-253cb28bf693:e2 +0ms
rascal:tasks:assertExchanges Asserting exchange: a0490c6c-3814-453d-b0bf-253cb28bf693:e3 +1ms
rascal:tasks:assertExchanges Asserting exchange: a0490c6c-3814-453d-b0bf-253cb28bf693:e4 +0ms
rascal:tasks:assertExchanges Asserting exchange: a0490c6c-3814-453d-b0bf-253cb28bf693:e5 +0ms
rascal:tasks:assertExchanges Asserting exchange: a0490c6c-3814-453d-b0bf-253cb28bf693:e6 +0ms
rascal:tasks:assertExchanges Asserting exchange: a0490c6c-3814-453d-b0bf-253cb28bf693:e7 +0ms
rascal:tasks:assertExchanges Asserting exchange: a0490c6c-3814-453d-b0bf-253cb28bf693:e8 +0ms
rascal:tasks:assertExchanges Asserting exchange: a0490c6c-3814-453d-b0bf-253cb28bf693:e9 +0ms
rascal:tasks:assertExchanges Asserting exchange: a0490c6c-3814-453d-b0bf-253cb28bf693:e10 +0ms
rascal:tasks:assertExchanges Successfully asserted exchange: a0490c6c-3814-453d-b0bf-253cb28bf693:e1 +3ms
rascal:tasks:assertExchanges Asserting exchange: a0490c6c-3814-453d-b0bf-253cb28bf693:e11 +0ms
rascal:tasks:assertExchanges Successfully asserted exchange: a0490c6c-3814-453d-b0bf-253cb28bf693:e2 +3ms
However using wireshark the Exchange.Declare and Exchange.Declare-Ok messages still come in sequence
No. Time Source Destination Protocol Length Info
41 0.298930 ::1 ::1 AMQP 141 Exchange.Declare x=a0490c6c-3814-453d-b0bf-253cb28bf693:e1
43 0.301350 ::1 ::1 AMQP 88 Exchange.Declare-Ok
45 0.302188 ::1 ::1 AMQP 141 Exchange.Declare x=a0490c6c-3814-453d-b0bf-253cb28bf693:e2
47 0.304333 ::1 ::1 AMQP 88 Exchange.Declare-Ok
49 0.305039 ::1 ::1 AMQP 141 Exchange.Declare x=a0490c6c-3814-453d-b0bf-253cb28bf693:e3
51 0.306679 ::1 ::1 AMQP 88 Exchange.Declare-Ok
53 0.307083 ::1 ::1 AMQP 141 Exchange.Declare x=a0490c6c-3814-453d-b0bf-253cb28bf693:e4
55 0.308682 ::1 ::1 AMQP 88 Exchange.Declare-Ok
57 0.309298 ::1 ::1 AMQP 141 Exchange.Declare x=a0490c6c-3814-453d-b0bf-253cb28bf693:e5
59 0.311408 ::1 ::1 AMQP 88 Exchange.Declare-Ok
61 0.311851 ::1 ::1 AMQP 141 Exchange.Declare x=a0490c6c-3814-453d-b0bf-253cb28bf693:e6
63 0.314042 ::1 ::1 AMQP 88 Exchange.Declare-Ok
65 0.314599 ::1 ::1 AMQP 141 Exchange.Declare x=a0490c6c-3814-453d-b0bf-253cb28bf693:e7
67 0.332144 ::1 ::1 AMQP 88 Exchange.Declare-Ok
69 0.332802 ::1 ::1 AMQP 141 Exchange.Declare x=a0490c6c-3814-453d-b0bf-253cb28bf693:e8
71 0.339870 ::1 ::1 AMQP 88 Exchange.Declare-Ok
73 0.340720 ::1 ::1 AMQP 141 Exchange.Declare x=a0490c6c-3814-453d-b0bf-253cb28bf693:e9
75 0.343136 ::1 ::1 AMQP 88 Exchange.Declare-Ok
77 0.343977 ::1 ::1 AMQP 142 Exchange.Declare x=a0490c6c-3814-453d-b0bf-253cb28bf693:e10
79 0.346291 ::1 ::1 AMQP 88 Exchange.Declare-Ok
Therefore the best option I can think of is to set the assert attribute to false and remove the bindings. This will speed up your start-up significantly, but you will need another way of asserting new queues/exchanges and binding them.
The other option is to change amqplib to support the nowait option, but the change looks complicated, and as one of the maintainers, I don't think anyone will get to it quickly. Especially as it's a mostly useless feature - if you are going to the trouble of creating a queue, it is hard to imagine a scenario where you wouldn't want to wait until it had been successfully created.
I've thought of another way that might help. If I use multiple channels, then amqplib will be able to create the queues and exchanges concurrently. I ran a quick test, and with 10 channels the time to create 1,000 exchanges dropped from 1,794ms to 465ms. I'll continue to experiment.
I've just published rascal@16.1.0. You can control the concurrency / number of channels used during initialisation by setting the concurrency attribute of the vhost. e.g.
{
"vhosts": {
"/": {
"concurrency": 10
}
}
}I suspect it will still take some time to create thousands of queues, but should make a significant improvement
Removed some debug from v16.1.1
Tested with the same config (6k log rows of "rascal:tasks:applyBindings" with a test configuration file of 450 queues)
Now the warm up time is about 30 seconds!
Before (About 4+ minutes):
- start: 2022-09-28T10:33:50.768Z
- end: 2022-09-28T10:38:00.839Z
After (About 30 seconds):
- start: 2022-09-29T07:08:56.234Z
- end: 2022-09-29T07:09:25.387Z
Inside the config I have 5 vhosts with param "concurrency": 10
About 11 seconds with concurrency = 30
- start: 2022-09-29T07:25:16.214Z
- end: 2022-09-29T07:25:27.162Z
Great! Thanks for updating. OK to close?
Yes to close.
Thank you!
Thank you too. I think this is a great improvement to Rascal, and wouldn't have crossed my mind if you hadn't taken the time to share it.