AmazonSQS configured to use IAM roles with service account fails to authenticate

[johnsimons]

Describe the bug

Description

When ServiceControl is using SQS transport configured to use IAM roles with a service account it fails to start with the following exception:

Exception

servicecontrol-audit-8445b8c47c-4775x setup 2024-07-08 15:14:49.8180|1|Error|Program|Unhandled exception was caught.|System.InvalidOperationException: Assembly AWSSDK.SecurityToken could not be found or loaded. This assembly must be available at runtime to use Amazon.Runtime.AssumeRoleAWSCredentials. servicecontrol-audit-8445b8c47c-4775x setup ---> System.IO.FileNotFoundException: Could not load file or assembly 'AWSSDK.SecurityToken, Culture=neutral, PublicKeyToken=null'. The system cannot find the file specified. servicecontrol-audit-8445b8c47c-4775x setup servicecontrol-audit-8445b8c47c-4775x setup File name: 'AWSSDK.SecurityToken, Culture=neutral, PublicKeyToken=null' servicecontrol-audit-8445b8c47c-4775x setup at System.Reflection.RuntimeAssembly.InternalLoad(AssemblyName assemblyName, StackCrawlMark& stackMark, AssemblyLoadContext assemblyLoadContext, RuntimeAssembly requestingAssembly, Boolean throwOnFileNotFound) servicecontrol-audit-8445b8c47c-4775x setup at System.Reflection.Assembly.Load(AssemblyName assemblyRef) servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.Internal.ServiceClientHelpers.GetSDKAssembly(String assemblyName) servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.Internal.ServiceClientHelpers.LoadTypeFromAssembly(String assemblyName, String className) servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.Internal.ServiceClientHelpers.LoadServiceConfigType(String assemblyName, String serviceConfigClassName) servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.Internal.ServiceClientHelpers.CreateServiceConfig(String assemblyName, String serviceConfigClassName) servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.AssumeRoleWithWebIdentityCredentials.CreateClient() servicecontrol-audit-8445b8c47c-4775x setup --- End of inner exception stack trace --- servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.AssumeRoleWithWebIdentityCredentials.CreateClient() servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.AssumeRoleWithWebIdentityCredentials.GenerateNewCredentialsAsync() servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.RefreshingAWSCredentials.GetCredentialsAsync() servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.Internal.CredentialsRetriever.InvokeAsync[T](IExecutionContext executionContext) servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext) servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext) servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext) servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext) servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.Internal.ErrorCallbackHandler.InvokeAsync[T](IExecutionContext executionContext) servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.Internal.MetricsHandler.InvokeAsync[T](IExecutionContext executionContext) servicecontrol-audit-8445b8c47c-4775x setup at NServiceBus.Transport.SQS.QueueCreator.CreateQueueIfNecessary(String address, Boolean createDelayedDeliveryQueue, CancellationToken cancellationToken) in /_/src/NServiceBus.Transport.SQS/QueueCreator.cs:line 37 servicecontrol-audit-8445b8c47c-4775x setup at NServiceBus.SqsTransport.Initialize(HostSettings hostSettings, ReceiveSettings[] receivers, String[] sendingAddresses, CancellationToken cancellationToken) in /_/src/NServiceBus.Transport.SQS/Configure/SqsTransport.cs:line 260 servicecontrol-audit-8445b8c47c-4775x setup at ServiceControl.Transports.TransportCustomization`1.ProvisionQueues(TransportSettings transportSettings, IEnumerable`1 additionalQueues) in /_/src/ServiceControl.Transports/TransportCustomization.cs:line 132 servicecontrol-audit-8445b8c47c-4775x setup at ServiceControl.Audit.Infrastructure.Hosting.Commands.SetupCommand.Execute(HostArguments args, Settings settings) in /_/src/ServiceControl.Audit/Infrastructure/Hosting/Commands/SetupCommand.cs:line 36 servicecontrol-audit-8445b8c47c-4775x setup at ServiceControl.Audit.Infrastructure.Hosting.Commands.CommandRunner.Execute(HostArguments args, Settings settings) in /_/src/ServiceControl.Audit/Infrastructure/Hosting/Commands/CommandRunner.cs:line 15 servicecontrol-audit-8445b8c47c-4775x setup at Program.$(String[] args) in /_/src/ServiceControl.Audit/Program.cs:line 27 servicecontrol-audit-8445b8c47c-4775x setup at Program.(String[] args) servicecontrol-audit-8445b8c47c-4775x setup Unhandled exception. System.InvalidOperationException: Assembly AWSSDK.SecurityToken could not be found or loaded. This assembly must be available at runtime to use Amazon.Runtime.AssumeRoleAWSCredentials. servicecontrol-audit-8445b8c47c-4775x setup ---> System.IO.FileNotFoundException: Could not load file or assembly 'AWSSDK.SecurityToken, Culture=neutral, PublicKeyToken=null'. The system cannot find the file specified. servicecontrol-audit-8445b8c47c-4775x setup servicecontrol-audit-8445b8c47c-4775x setup File name: 'AWSSDK.SecurityToken, Culture=neutral, PublicKeyToken=null' servicecontrol-audit-8445b8c47c-4775x setup at System.Reflection.RuntimeAssembly.InternalLoad(AssemblyName assemblyName, StackCrawlMark& stackMark, AssemblyLoadContext assemblyLoadContext, RuntimeAssembly requestingAssembly, Boolean throwOnFileNotFound) servicecontrol-audit-8445b8c47c-4775x setup at System.Reflection.Assembly.Load(AssemblyName assemblyRef) servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.Internal.ServiceClientHelpers.GetSDKAssembly(String assemblyName) servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.Internal.ServiceClientHelpers.LoadTypeFromAssembly(String assemblyName, String className) servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.Internal.ServiceClientHelpers.LoadServiceConfigType(String assemblyName, String serviceConfigClassName) servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.Internal.ServiceClientHelpers.CreateServiceConfig(String assemblyName, String serviceConfigClassName) servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.AssumeRoleWithWebIdentityCredentials.CreateClient() servicecontrol-audit-8445b8c47c-4775x setup --- End of inner exception stack trace --- servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.AssumeRoleWithWebIdentityCredentials.CreateClient() servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.AssumeRoleWithWebIdentityCredentials.GenerateNewCredentialsAsync() servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.RefreshingAWSCredentials.GetCredentialsAsync() servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.Internal.CredentialsRetriever.InvokeAsync[T](IExecutionContext executionContext) servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext) servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.Internal.RetryHandler.InvokeAsync[T](IExecutionContext executionContext) servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext) servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.Internal.CallbackHandler.InvokeAsync[T](IExecutionContext executionContext) servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.Internal.ErrorCallbackHandler.InvokeAsync[T](IExecutionContext executionContext) servicecontrol-audit-8445b8c47c-4775x setup at Amazon.Runtime.Internal.MetricsHandler.InvokeAsync[T](IExecutionContext executionContext) servicecontrol-audit-8445b8c47c-4775x setup at NServiceBus.Transport.SQS.QueueCreator.CreateQueueIfNecessary(String address, Boolean createDelayedDeliveryQueue, CancellationToken cancellationToken) in /_/src/NServiceBus.Transport.SQS/QueueCreator.cs:line 37 servicecontrol-audit-8445b8c47c-4775x setup at NServiceBus.SqsTransport.Initialize(HostSettings hostSettings, ReceiveSettings[] receivers, String[] sendingAddresses, CancellationToken cancellationToken) in /_/src/NServiceBus.Transport.SQS/Configure/SqsTransport.cs:line 260 servicecontrol-audit-8445b8c47c-4775x setup at ServiceControl.Transports.TransportCustomization`1.ProvisionQueues(TransportSettings transportSettings, IEnumerable`1 additionalQueues) in /_/src/ServiceControl.Transports/TransportCustomization.cs:line 132 servicecontrol-audit-8445b8c47c-4775x setup at ServiceControl.Audit.Infrastructure.Hosting.Commands.SetupCommand.Execute(HostArguments args, Settings settings) in /_/src/ServiceControl.Audit/Infrastructure/Hosting/Commands/SetupCommand.cs:line 36 servicecontrol-audit-8445b8c47c-4775x setup at ServiceControl.Audit.Infrastructure.Hosting.Commands.CommandRunner.Execute(HostArguments args, Settings settings) in /_/src/ServiceControl.Audit/Infrastructure/Hosting/Commands/CommandRunner.cs:line 15 servicecontrol-audit-8445b8c47c-4775x setup at Program.$(String[] args) in /_/src/ServiceControl.Audit/Program.cs:line 27 servicecontrol-audit-8445b8c47c-4775x setup at Program.(String[] args)

Expected behavior

It authenticates and assumes the role.

Actual behavior

It fails to authenticate

Versions

5.0 and above

Steps to reproduce

Configure ServiceControl to run with SQS transport with assume role for the service account

[jpalac]

Will be shipped with release 5.4.1