Grpc.Net.Client : "Received message exceeds the maximum configured message size."
sengiv opened this issue · 3 comments
When orchestration output exceeds ~3MB "message exceeds..." exception is thrown.
Is there a way to change the limit?
Current hack to make work:
I'm sure there is a better way to do this.
Running on Azure Functions .NET 7
I think this is a limitation of the gRPC channel between the durable client and the durable extension running in the host.
We might need to fix this by updating the host to increase the max gRPC message size. Ideally, we could use the same configuration that exists for the Functions host gRPC message size.
Java is likely impacted by this issue as well.
@jviau, would you be able to help with this? Hopefully it's a simple fix?
@sengiv, I don't think we want to be increasing message limits. Say we raise it to 6MB, then someone exceeds that, what then? Do we just keep raising it? There needs to be some line drawn on what the max gRPC message limit is, and we might as well stick with the default (4MB). Even then, there is no place for us to universally apply this new limit. Each gRPC server implementation of Durable will need to make its own choice on message limits, so no solution would be 100% universally applied. In the case of Durable functions, yes, we do own that. But not forever actually - right now we have our own gRPC server, but eventually we will be leveraging the existing gRPC server in the functions host. At that point we will no longer own this might not be unable to raise the limits.
If you are going to have orchestration output of such a significant size, I recommend you find some intermediate way to store and retrieve the output. Durable is not designed/intended to be a high-bandwidth output system. I recommend looking into technologies designed for just that. For example, you could use Azure blob storage and have your orchestration write to that blob at the end. Or better yet, append to the blob over the duration of the orchestration (if your design allows that).
Closing as won't fix as per my comment above.