Durable Function fails to enter activity after net8 and isolated model upgrades

[nickolai-p]

Description

Upgraded an existing set of durable functions to .Net8 and isolated worker model from .Net6 in-process model following this https://learn.microsoft.com/en-us/azure/azure-functions/durable/durable-functions-dotnet-isolated-overview. The code compiles and the functions run far enough to start a new orchestration but fail to enter any activity function code.

The error appears in GeneratedFunctionExecutor.g.cs ExecuteAsync()
line: var inputBindingResult = await inputBindingFeature.BindFunctionInputAsync(context);
Error message: Error converting value "{"CorrelationId":"724c2b60-461a-4fb0-97a5-5af3a067360d","MyId":"117cd1de-9e94-4466-a8f1-98ea4a72201d","MyBoolParamater":false}" to type 'Microsoft.DurableTask.TaskOrchestrationContext'. Path '', line 1, position 158.

The code in question seems to be trying to convert or deserialize the function input into the TaskOrchestrationContext type. I have tried modifying the input to be a string or a basic class and have tried changing the json seriliazitation provider back and forth between System.Text.Json and Newtonsoft.Json but the failure persists at this same line even though the exact error message might be slightly different.

Expected behavior

• Orchestration starter function is entered
• Orchestration function is entered
• First Activity Function is entered

Actual behavior

• Orchestration starter function is entered
• Orchestration function is entered
• Error generated in auto-generated code file prior to entering Activity Function
• Orchestration Fails and is unable to enter Activity Function

Relevant source code snippets

[Function(ActivityName)]
public async Task MyActivity1(
    [ActivityTrigger] string input)
{
    var inputMessage = JsonSerializer.Deserialize<MyMessage>(input);

    var result = await _myGateway.DoStuff(inputMessage);

    // throw to trigger retry
    if (result.IsFailure)
    {
        throw new Exception(result.ErrorMessage);
    }
}

Known workarounds

I have attempted to follow the guidance from this issue but it has not worked caused the second issue mentioned at the bottom of the linked issue: #2527

App Details

• Microsoft.Azure.Functions.Worker.Extensions.DurableTask 1.1.4:
• Azure Functions runtime version (V4):
• C#:

Screenshots

If deployed to Azure

• Timeframe issue observed: 8/2 - 8/5/24 the issue happens consistently when running locally or deployed to azure
• Function App name: I can't share my function app or storage account name as this is an issue I am experiencing with the company I work at.
• Function name(s):I can't share my function app or storage account name as this is an issue I am experiencing with the company I work at.
• Azure region: East US
• Orchestration instance ID(s): 471813fb-beb3-410e-9939-733621e95f91, 471813fb-beb3-410e-9939-733621e95f91
• Azure storage account name:I can't share my function app or storage account name as this is an issue I am experiencing with the company I work at.

[cldbrr]

I have similar issue:
I am using latest empty durable function from template (no changes) with latest packages. Deployed on corporate Azure Classic Azure Function (windows) net8 isolated

I am getting following error:

{ "message": "Function 'Function1' failed with an unhandled exception.", "details": "Microsoft.Azure.WebJobs.Host.FunctionInvocationException: Exception while executing function: Functions.Function1 ---> DurableTask.Core.Exceptions.OrchestrationFailureException: Orchestration history state was either missing from the input or not a string value. at Microsoft.Azure.WebJobs.Extensions.DurableTask.OutOfProcMiddleware.<>c__DisplayClass10_0.<<CallOrchestratorAsync>b__0>d.MoveNext() in D:\\a\\_work\\1\\s\\src\\WebJobs.Extensions.DurableTask\\OutOfProcMiddleware.cs:line 145 --- End of stack trace from previous location --- at Microsoft.Azure.WebJobs.Host.Executors.TriggeredFunctionExecutor1.<>c__DisplayClass7_0.<<TryExecuteAsync>b__0>d.MoveNext() in D:\\a\\_work\\1\\s\\src\\Microsoft.Azure.WebJobs.Host\\Executors\\TriggeredFunctionExecutor.cs:line 51 --- End of stack trace from previous location --- at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.InvokeWithTimeoutAsync(IFunctionInvoker invoker, ParameterHelper parameterHelper, CancellationTokenSource timeoutTokenSource, CancellationTokenSource functionCancellationTokenSource, Boolean throwOnTimeout, TimeSpan timerInterval, IFunctionInstance instance) in D:\\a\\_work\\1\\s\\src\\Microsoft.Azure.WebJobs.Host\\Executors\\FunctionExecutor.cs:line 581 at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithWatchersAsync(IFunctionInstanceEx instance, ParameterHelper parameterHelper, ILogger logger, CancellationTokenSource functionCancellationTokenSource) in D:\\a\\_work\\1\\s\\src\\Microsoft.Azure.WebJobs.Host\\Executors\\FunctionExecutor.cs:line 527 at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(IFunctionInstanceEx instance, FunctionStartedMessage message, FunctionInstanceLogEntry instanceLogEntry, ParameterHelper parameterHelper, ILogger logger, CancellationToken cancellationToken) in D:\\a\\_work\\1\\s\\src\\Microsoft.Azure.WebJobs.Host\\Executors\\FunctionExecutor.cs:line 306 --- End of inner exception stack trace --- at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(IFunctionInstanceEx instance, FunctionStartedMessage message, FunctionInstanceLogEntry instanceLogEntry, ParameterHelper parameterHelper, ILogger logger, CancellationToken cancellationToken) in D:\\a\\_work\\1\\s\\src\\Microsoft.Azure.WebJobs.Host\\Executors\\FunctionExecutor.cs:line 352 at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.TryExecuteAsync(IFunctionInstance functionInstance, CancellationToken cancellationToken) in D:\\a\\_work\\1\\s\\src\\Microsoft.Azure.WebJobs.Host\\Executors\\FunctionExecutor.cs:line 108" }

[nickolai-p]

@CHR7457C7 have you seen this issue? #2804

[cldbrr]

@nickolai-p fixed according to #2804 and got following error this one more similar:

{
   "message": "Function 'Function1' failed with an unhandled exception.",
   "details": "Microsoft.Azure.WebJobs.Host.FunctionInvocationException: Exception while executing function: Functions.Function1
---> System.InvalidOperationException: Exception binding parameter 'context'
 ---> System.MissingMethodException: Method not found: 'Void Microsoft.DurableTask.Protobuf.ExecutionStartedEvent.set_CorrelationData(System.String)'.
   at Microsoft.Azure.WebJobs.Extensions.DurableTask.ProtobufUtils.ToHistoryEventProto(HistoryEvent e)
  at System.Linq.Enumerable.SelectListIterator`2.MoveNext()
   at Google.Protobuf.Collections.RepeatedField`1.AddRange(IEnumerable`1 values)
   at Microsoft.Azure.WebJobs.Extensions.DurableTask.OrchestrationTriggerAttributeBindingProvider.OrchestrationTriggerBinding.BindAsync(Object value, ValueBindingContext context) in D:\\a\\_work\\1\\s\\src\\WebJobs.Extensions.DurableTask\\Bindings\\OrchestrationTriggerAttributeBindingProvider.cs:line 168
   at Microsoft.Azure.WebJobs.Host.Indexers.FunctionIndexer.TriggerWrapper.BindAsync(Object value, ValueBindingContext context) in D:\\a\\_work\\1\\s\\src\\Microsoft.Azure.WebJobs.Host\\Indexers\\FunctionIndexer.cs:line 551
   at Microsoft.Azure.WebJobs.Host.Triggers.TriggeredFunctionBinding`1.BindCoreAsync(ValueBindingContext context, Object value, IDictionary`2 parameters) in D:\\a\\_work\\1\\s\\src\\Microsoft.Azure.WebJobs.Host\\Triggers\\TriggeredFunctionBinding.cs:line 57
   --- End of inner exception stack trace ---
   at Microsoft.Azure.WebJobs.Host.Executors.DelayedException.Throw() in D:\\a\\_work\\1\\s\\src\\Microsoft.Azure.WebJobs.Host\\Executors\\DelayedException.cs:line 27
   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ParameterHelper.PrepareParametersAsync() in D:\\a\\_work\\1\\s\\src\\Microsoft.Azure.WebJobs.Host\\Executors\\FunctionExecutor.cs:line 922
   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithWatchersAsync(IFunctionInstanceEx instance, ParameterHelper parameterHelper, ILogger logger, CancellationTokenSource functionCancellationTokenSource) in D:\\a\\_work\\1\\s\\src\\Microsoft.Azure.WebJobs.Host\\Executors\\FunctionExecutor.cs:line 492
   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(IFunctionInstanceEx instance, FunctionStartedMessage message, FunctionInstanceLogEntry instanceLogEntry, ParameterHelper parameterHelper, ILogger logger, CancellationToken cancellationToken) in D:\\a\\_work\\1\\s\\src\\Microsoft.Azure.WebJobs.Host\\Executors\\FunctionExecutor.cs:line 306
   --- End of inner exception stack trace ---\r\n   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.ExecuteWithLoggingAsync(IFunctionInstanceEx instance, FunctionStartedMessage message, FunctionInstanceLogEntry instanceLogEntry, ParameterHelper parameterHelper, ILogger logger, CancellationToken cancellationToken) in D:\\a\\_work\\1\\s\\src\\Microsoft.Azure.WebJobs.Host\\Executors\\FunctionExecutor.cs:line 352
   at Microsoft.Azure.WebJobs.Host.Executors.FunctionExecutor.TryExecuteAsync(IFunctionInstance functionInstance, CancellationToken cancellationToken) in D:\\a\\_work\\1\\s\\src\\Microsoft.Azure.WebJobs.Host\\Executors\\FunctionExecutor.cs:line 108"
}

[erichkray]

@cldbrr, did you ever resolve this?