arcus-azure/arcus.messaging

Gracefully stop hosted services when console is stopped

stijnmoreels opened this issue · 0 comments

Is your feature request related to a problem? Please describe.
When a project is stopped by the user locally, the TCP health probe and message pump fail with an rather big (expected) stop exception.

[08:17:28 ERR] BackgroundService failed
System.Net.Sockets.SocketException (995): The I/O operation has been aborted because of either a thread exit or an application request.
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Net.Sockets.Socket>.GetResult(Int16 token)
   at System.Net.Sockets.TcpListener.<AcceptTcpClientAsync>g__WaitAndWrap|31_0(ValueTask`1 task)
   at Arcus.Messaging.Health.Tcp.TcpHealthListener.AcceptConnectionAsync(HealthReport report)
   at Arcus.Messaging.Health.Tcp.TcpHealthListener.ExecuteAsync(CancellationToken stoppingToken)
   at Microsoft.Extensions.Hosting.Internal.Host.TryExecuteBackgroundServiceAsync(BackgroundService backgroundService)
[08:17:28 VRB] Found secret with name 'ARCUS_SERVICEBUS_CONNECTIONSTRING'
[08:17:28 FTL] The HostOptions.BackgroundServiceExceptionBehavior is configured to StopHost. A BackgroundService has thrown an unhandled exception, and the IHost instance is stopping. To avoid this behavior, configure this to Ignore; however the BackgroundService will not be restarted.
System.Net.Sockets.SocketException (995): The I/O operation has been aborted because of either a thread exit or an application request.
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource<System.Net.Sockets.Socket>.GetResult(Int16 token)
   at System.Net.Sockets.TcpListener.<AcceptTcpClientAsync>g__WaitAndWrap|31_0(ValueTask`1 task)
   at Arcus.Messaging.Health.Tcp.TcpHealthListener.AcceptConnectionAsync(HealthReport report)
   at Arcus.Messaging.Health.Tcp.TcpHealthListener.ExecuteAsync(CancellationToken stoppingToken)
   at Microsoft.Extensions.Hosting.Internal.Host.TryExecuteBackgroundServiceAsync(BackgroundService backgroundService)
[0

It will also cause strange exception messages, with non-existing messages:

 Closing message pump '2b39997d-665f-4466-aeca-52c6b22f8441' on entity path
[08:17:29 FTL] Unable to process message from /Subscriptions/Receive-All with client 2b39997d-665f-4466-aeca-52c6b22f8441

Describe the solution you'd like
We could improve user experience by more gracefully stop the TCP health probe and message pump