Orleans Message Rejection Exception on Azure: Unable to Connect to Silo Endpoint
SebastianDevelops opened this issue · 0 comments
SebastianDevelops commented
I’m encountering issues with an Orleans application running on Azure. For some odd reason when the silo instances table contains data, I get the following:
Orleans.Runtime.OrleansMessageRejectionException: Exception while sending message: Orleans.Runtime.Messaging.ConnectionFailedException: Unable to connect to S10.0.0.254:20038:83628028, will retry after 981.0283ms
at Orleans.Runtime.Messaging.ConnectionManager.GetConnectionAsync(SiloAddress endpoint) in /_/src/Orleans.Core/Networking/ConnectionManager.cs:line 99
at Orleans.Runtime.Messaging.MessageCenter.<SendMessage>g__SendAsync|29_0(MessageCenter messageCenter, ValueTask`1 connectionTask, Message msg) in /_/src/Orleans.Runtime/Messaging/MessageCenter.cs:line 226
at Orleans.Serialization.Invocation.ResponseCompletionSource`1.GetResult(Int16 token) in /_/src/Orleans.Serialization/Invocation/ResponseCompletionSource.cs:line 230
at System.Threading.Tasks.ValueTask`1.ValueTaskSourceAsTask.<>c.<.cctor>b__4_0(Object state)
Private ports are configured to provide 2 ports and Orleans is configured like this:
var endpointAddress =
IPAddress.Parse(context.Configuration["WEBSITE_PRIVATE_IP"]!);
var strPorts =
context.Configuration["WEBSITE_PRIVATE_PORTS"]!.Split(',');
if (strPorts.Length < 2)
throw new Exception("Insufficient private ports configured.");
var (siloPort, gatewayPort) =
(int.Parse(strPorts[0]), int.Parse(strPorts[1]));
siloBuilder
.ConfigureEndpoints(endpointAddress, siloPort, gatewayPort)
.UseAzureStorageClustering(opt =>
{
opt.TableName = orleansSettings.TableStorageConfig.TableName;
opt.TableServiceClient = new TableServiceClient(settings.TableStorageConfig.ConnectionString);
})
.Configure<ClusterOptions>(options =>
{
options.ClusterId = settings.ClusterConfiguration.ClusterId;
options.ServiceId = settings.ClusterConfiguration.ServiceId;
}).AddAzureBlobGrainStorage(Constants.Orleans.GrainStorageProviderName, (AzureBlobStorageOptions options) =>
{
options.BlobServiceClient = new BlobServiceClient(settings.BlobstorageConfig.ConnectionString);
options.ContainerName = Invariant($"{settings.EnvironmentPrefix}{Constants.Orleans.GrainStorageProviderName}");
})
Any insights on how I can work past this issue? Besides clearing the instances table?