DeclareDelayedMessageExchange Holds Main Thread
SilentBlueD666 opened this issue · 4 comments
Hi
When using the option UseDelayedMessageExchange with automaticallyDeclareExchange = true on timeout configuration and startAutomatically = false on the main Rebus options, the method DeclareDelayedMessageExchange in RabbitMqTransport.cs keeps hold of the main thread in a loop, however because it's waiting for connection to RabbitMq to be ready Existing connection found to be CLOSED log message every second, the rest of the application is not able to bootstrap and therefore not start the bus.
After 60 seconds get following exception:
Hosting failed to start
Rebus.Injection.ResolutionException: Could not resolve Rebus.Bus.IBus with decorator depth 0 - registrations: Rebus.Injection.Injectionist+Handler
---> Rebus.Exceptions.RebusApplicationException: Delayed message exchange declaration for '@RebusDelayed' failed
---> RabbitMQ.Client.Exceptions.OperationInterruptedException: The AMQP operation was interrupted: AMQP close-reason, initiated by Peer, code=503, text='COMMAND_INVALID - unknown exchange type 'x-delayed-message'', classId=40, methodId=10
at RabbitMQ.Client.Impl.ModelBase.ModelRpc(MethodBase method, ContentHeaderBase header, Byte[] body)
at RabbitMQ.Client.Framing.Impl.Model._Private_ExchangeDeclare(String exchange, String type, Boolean passive, Boolean durable, Boolean autoDelete, Boolean internal, Boolean nowait, IDictionary`2 arguments)
at RabbitMQ.Client.Impl.ModelBase.ExchangeDeclare(String exchange, String type, Boolean durable, Boolean autoDelete, IDictionary`2 arguments)
at RabbitMQ.Client.Impl.AutorecoveringModel.ExchangeDeclare(String exchange, String type, Boolean durable, Boolean autoDelete, IDictionary`2 arguments)
at Rebus.RabbitMq.RabbitMqTransport.DeclareDelayedMessageExchange(String exchangeName)
--- End of inner exception stack trace ---
at Rebus.RabbitMq.RabbitMqTransport.DeclareDelayedMessageExchange(String exchangeName)
at Rebus.Config.RabbitMqDelayedMessageExchangeExtensions.DelayedMessageExchangeTransportDecorator.Initialize()
at Rebus.Config.RebusConfigurer.<>c__DisplayClass14_0.<Start>b__29(IResolutionContext c)
at Rebus.Injection.Injectionist.Resolver`1.InvokeResolver(IResolutionContext context)
at Rebus.Injection.Injectionist.ResolutionContext.Get[TService]()
--- End of inner exception stack trace ---
at Rebus.Injection.Injectionist.ResolutionContext.Get[TService]()
at Rebus.Injection.Injectionist.Get[TService]()
at Rebus.Config.RebusConfigurer.Start()
at Rebus.Config.DelayedStartupConfigurationExtensions.Create(RebusConfigurer configurer)
at Rebus.ServiceProvider.Internals.RebusInitializer.<GetLazyInitializer>b__9_0()
at Rebus.ServiceProvider.Internals.RebusBackgroundService.ExecuteAsync(CancellationToken stoppingToken)
at Microsoft.Extensions.Hosting.Internal.Host.<StartAsync>b__15_1(IHostedService service, CancellationToken token)
at Microsoft.Extensions.Hosting.Internal.Host.ForeachService[T](IEnumerable`1 services, CancellationToken token, Boolean concurrent, Boolean abortOnFirstException, List`1 exceptions, Func`3 operation)The reason I have no auto start is due to me needing to make sure all the rest of the app is bootstrapped and started before I can start getting messages from the Bus.
Only way to get working is to manually setup the exchange and set automaticallyDeclareExchange = false
The reason I have no auto start is due to me needing to make sure all the rest of the app is bootstrapped and started before I can start getting messages from the Bus.
Do you need to bootstrap anything besides registering it in the container?
Rebus will not start receiving messages before all of the registrations have been made and the background services get started by the hosting environment, so if you register your stuff in the container (possible via the factory method overloads), then everything will be ready at the right time.
If you do need to be able to start Rebus like you're trying to, could you maybe create a reproduction as a test? Thanks in advance 🙂
Hi OK, very sorry after sleeping, I now feel stupid!!! While I had installed the RabbitMQ Delayed Message Plugin in to RabbitMQ, checking enabled plugins... and I must of forgot to press enter on the enable plugin command 🙄 now working.
However to give you more context on the bootstrapping, I am using Microsoft Orleans so the app start-up in the following way:
- Register all services to DI (Orleans, Rebus & ASP.NET)
- Start Orleans Silo
- Connect to Orleans Cluster
- Create Orleans Streams
- Start Rebus (this is done via a Orleans Lifetime cycle manager that executes once Silo/Cluster active and running)
- Start ASP.NET
The Orleans Silo/Cluster must be running before Rebus workers, as messages could already be waiting in a queue while the service was offline, when messages are then processed before Orleans is ready, exceptions are thrown and the messages end up in error queue.
Rebus will not start receiving messages before all of the registrations have been made and the background services get started by the hosting environment
I think why I got different behaviour that was out of the norm was because Orleans doesn't use the .NET Background service abstraction to run/start.
So... if I understand you correctly, you figured it out, right? 🙂
Closing.... please let me know if there's anything else I can do.
Yep good to close 😁