Method "ForceTerminateTaskOrchestrationAsync" throwing error
Opened this issue · 2 comments
Hi, We were using durable task framework in one of our projects having NET Framework 4.6.2 from past few years and it was working fine.
Now, we decided to upgrade .NET version from framework to .NET 6.0. After the upgrade, we found out that Termination functionality is not working as expected and throwing an error that "Operation is not valid due to the current state of the object."
Below is the stack trace,
` > Monitoring.Controllers.HomeController.TerminateOrchestration(Monitoring.Models.OrchestrationTerminateContext model) Line 376 C#
[Lightweight Function]
Microsoft.AspNetCore.Mvc.Core.dll!Microsoft.AspNetCore.Mvc.Infrastructure.ActionMethodExecutor.SyncActionResultExecutor.Execute(Microsoft.AspNetCore.Mvc.Infrastructure.IActionResultTypeMapper mapper, Microsoft.Extensions.Internal.ObjectMethodExecutor executor, object controller, object[] arguments) Unknown
Microsoft.AspNetCore.Mvc.Core.dll!Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync.__Logged|12_1(Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker invoker) Unknown
Microsoft.AspNetCore.Mvc.Core.dll!Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeActionMethodAsync() Unknown
Microsoft.AspNetCore.Mvc.Core.dll!Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(ref Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.State next, ref Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Scope scope, ref object state, ref bool isCompleted) Unknown
Microsoft.AspNetCore.Mvc.Core.dll!Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync() Unknown
Microsoft.AspNetCore.Mvc.Core.dll!Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(ref Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.State next, ref Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Scope scope, ref object state, ref bool isCompleted) Unknown
Microsoft.AspNetCore.Mvc.Core.dll!Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAsync() Unknown
Microsoft.AspNetCore.Mvc.Core.dll!Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeNextActionFilterAwaitedAsync() Unknown
Microsoft.AspNetCore.Mvc.ViewFeatures.dll!Microsoft.AspNetCore.Mvc.Controller.OnActionExecutionAsync(Microsoft.AspNetCore.Mvc.Filters.ActionExecutingContext context, Microsoft.AspNetCore.Mvc.Filters.ActionExecutionDelegate next) Line 365 C#
Microsoft.AspNetCore.Mvc.Core.dll!Microsoft.AspNetCore.Mvc.Filters.ControllerActionFilter.OnActionExecutionAsync(Microsoft.AspNetCore.Mvc.Filters.ActionExecutingContext context, Microsoft.AspNetCore.Mvc.Filters.ActionExecutionDelegate next) Unknown
Microsoft.AspNetCore.Mvc.Core.dll!Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Next(ref Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.State next, ref Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.Scope scope, ref object state, ref bool isCompleted) Unknown
Microsoft.AspNetCore.Mvc.Core.dll!Microsoft.AspNetCore.Mvc.Infrastructure.ControllerActionInvoker.InvokeInnerFilterAsync() Unknown
Microsoft.AspNetCore.Mvc.Core.dll!Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(ref Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.State next, ref Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Scope scope, ref object state, ref bool isCompleted) Unknown
Microsoft.AspNetCore.Mvc.Core.dll!Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeNextResourceFilter() Unknown
Microsoft.AspNetCore.Mvc.Core.dll!Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Next(ref Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.State next, ref Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.Scope scope, ref object state, ref bool isCompleted) Unknown
Microsoft.AspNetCore.Mvc.Core.dll!Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeFilterPipelineAsync() Unknown
Microsoft.AspNetCore.Mvc.Core.dll!Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeAsync.__Logged|17_1(Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker invoker) Unknown
Microsoft.AspNetCore.Mvc.Core.dll!Microsoft.AspNetCore.Mvc.Infrastructure.ResourceInvoker.InvokeAsync() Unknown
Microsoft.AspNetCore.Mvc.Core.dll!Microsoft.AspNetCore.Mvc.Routing.ControllerRequestDelegateFactory.CreateRequestDelegate.AnonymousMethod__0(Microsoft.AspNetCore.Http.HttpContext context) Unknown
Microsoft.AspNetCore.Routing.dll!Microsoft.AspNetCore.Routing.EndpointMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext httpContext) Unknown
Microsoft.AspNetCore.Authorization.Policy.dll!Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext context) Unknown
Microsoft.AspNetCore.Routing.dll!Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext httpContext) Unknown
Microsoft.AspNetCore.StaticFiles.dll!Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext context) Unknown
Microsoft.AspNetCore.HttpsPolicy.dll!Microsoft.AspNetCore.HttpsPolicy.HttpsRedirectionMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext context) Unknown
Microsoft.AspNetCore.Diagnostics.dll!Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext context) Unknown
Microsoft.AspNetCore.HostFiltering.dll!Microsoft.AspNetCore.HostFiltering.HostFilteringMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext context) Unknown
Microsoft.WebTools.BrowserLink.Net.dll!Microsoft.WebTools.BrowserLink.Net.BrowserLinkMiddleware.InvokeAsync(Microsoft.AspNetCore.Http.HttpContext context) Unknown
Microsoft.WebTools.BrowserLink.Net.dll!Microsoft.WebTools.BrowserLink.Net.VsContentMiddleware.InvokeAsync(Microsoft.AspNetCore.Http.HttpContext context) Unknown
Microsoft.AspNetCore.Watch.BrowserRefresh.dll!Microsoft.AspNetCore.Watch.BrowserRefresh.BrowserRefreshMiddleware.InvokeAsync(Microsoft.AspNetCore.Http.HttpContext context) Unknown
Microsoft.AspNetCore.Http.Abstractions.dll!Microsoft.AspNetCore.Builder.Extensions.MapWhenMiddleware.Invoke(Microsoft.AspNetCore.Http.HttpContext context) Unknown
Microsoft.AspNetCore.Hosting.dll!Microsoft.AspNetCore.Hosting.HostingApplication.ProcessRequestAsync(Microsoft.AspNetCore.Hosting.HostingApplication.Context context) Unknown
Microsoft.AspNetCore.Server.Kestrel.Core.dll!Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequests<Microsoft.AspNetCore.Hosting.HostingApplication.Context>(Microsoft.AspNetCore.Hosting.Server.IHttpApplication<Microsoft.AspNetCore.Hosting.HostingApplication.Context> application) Unknown
Microsoft.AspNetCore.Server.Kestrel.Core.dll!Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http.HttpProtocol.ProcessRequestsAsync<Microsoft.AspNetCore.Hosting.HostingApplication.Context>(Microsoft.AspNetCore.Hosting.Server.IHttpApplication<Microsoft.AspNetCore.Hosting.HostingApplication.Context> application) Unknown
Microsoft.AspNetCore.Server.Kestrel.Core.dll!Microsoft.AspNetCore.Server.Kestrel.Core.Internal.Http2.Http2Stream<Microsoft.AspNetCore.Hosting.HostingApplication.Context>.Execute() Unknown
System.Private.CoreLib.dll!System.Threading.ThreadPoolWorkQueue.Dispatch() Unknown
System.Private.CoreLib.dll!System.Threading.PortableThreadPool.WorkerThread.WorkerThreadStart() Unknown
System.Private.CoreLib.dll!System.Threading.Thread.StartCallback() Unknown
`
The complete code is below,
`public class Test
{
private readonly TaskHubClient _taskHubClient;
private readonly IResourceCreator _resourceCreator;
public Client(string serviceBus, string taskHubName, string azureStorageTableName,
string azureStorageAccount, string jumpStartTableName
, IOrchestrationLookupController orchestrationLookupController)
{
_resourceCreator = new AzureResourceCreator(serviceBus, azureStorageAccount, taskHubName);
_resourceCreator.CreateResources();
_taskHubClient = new TaskHubClient(_resourceCreator.OrchestrationServiceAndClient);
}
public bool TerminateOrchestration(string instanceId, string terminationMessage)
{
if (string.IsNullOrWhiteSpace(instanceId))
{
return false;
}
AsyncContext.Run(() =>
_resourceCreator.OrchestrationServiceAndClient.ForceTerminateTaskOrchestrationAsync(instanceId,
terminationMessage));
return true;
}
}
`
The code from DurableTaskFramework (DurableTask.ServiceBus.ServiceBusOrchestrationService) is as below,
`
public async Task ForceTerminateTaskOrchestrationAsync(string instanceId, string reason)
{
TaskMessage message = new TaskMessage
{
OrchestrationInstance = new OrchestrationInstance
{
InstanceId = instanceId
},
Event = new ExecutionTerminatedEvent(-1, reason)
};
await SendTaskOrchestrationMessageAsync(message);
}
public Task SendTaskOrchestrationMessageAsync(TaskMessage message)
{
// Issue occurs here........
return SendTaskOrchestrationMessageBatchAsync(message);
}
public async Task SendTaskOrchestrationMessageBatchAsync(params TaskMessage[] messages)
{
if (messages.Length != 0)
{
Task<DurableTask.ServiceBus.Common.Abstraction.Message>[] array = new Task<DurableTask.ServiceBus.Common.Abstraction.Message>[messages.Length];
for (int i = 0; i < messages.Length; i++)
{
array[i] = GetBrokeredMessageAsync(messages[i]);
}
DurableTask.ServiceBus.Common.Abstraction.Message[] messageList = await Task.WhenAll(array);
await orchestrationBatchMessageSender.SendAsync(messageList).ConfigureAwait(continueOnCapturedContext: false);
}
}`
It started throwing error in .NET 6.0 but working fine in .Net framework
Your call stack only includes code from your project (Monitoring.Controllers.HomeController). Are you sure this exception is coming from the Durable Task Framework?
Hi @cgillum, thanks for responding. I have updated the complete stack trace for reference also added the DurableTaskFramework code flow where actually exception occurs. The below method inside "DurableTask.ServiceBus.ServiceBusOrchestrationService.cs" is causing exception to occur,
public Task SendTaskOrchestrationMessageAsync(TaskMessage message) { return SendTaskOrchestrationMessageBatchAsync(message); }