Test failure: NtlmAuthenticationTest
aspnet-hello opened this issue · 10 comments
This test fails occasionally with the following error:
Assert.Equal() Failure
Expected: OK
Actual: BadGateway
at E2ETests.Validator.VerifyHomePage(HttpResponseMessage response, Boolean useNtlmAuthentication) in C:\BuildAgent\work\33bdfc1cae7b2a38\modules\MusicStore\test\MusicStore.E2ETests\Implementation\Validator.cs:line 94
at E2ETests.Validator.VerifyNtlmHomePage(HttpResponseMessage response) in C:\BuildAgent\work\33bdfc1cae7b2a38\modules\MusicStore\test\MusicStore.E2ETests\Implementation\Validator.cs:line 132
at E2ETests.NtlmAuthenticationTests.NtlmAuthenticationTest(TestVariant variant) in C:\BuildAgent\work\33bdfc1cae7b2a38\modules\MusicStore\test\MusicStore.E2ETests\NtlmAuthentationTest.cs:line 100
--- End of stack trace from previous location where exception was thrown ---
------- Stdout: -------
| [0.006s] TestLifetime Information: Starting test NtlmAuthenticationTest-Server: IISExpress, TFM: net461, Type: Standalone, Arch: x64, ANCM: V1, Host: OutOfProcess at 2018-09-20T19:19:49
| [0.001s] TestLifetime Information: Starting test NtlmAuthentication_Server: IISExpress, TFM: net461, Type: Standalone, Arch: x64, ANCM: V1, Host: OutOfProcess at 2018-09-20T19:19:49
| [0.002s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: Deploying [Variation] :: ServerType=IISExpress, Runtime=Clr, Arch=x64, BaseUrlHint=, Publish=False
| [0.003s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: Executing: C:\BuildAgent\work\33bdfc1cae7b2a38\modules\MusicStore\samples\MusicStore\bin\Release\net461\MusicStore.exe
| [0.003s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: ContentRoot: C:\BuildAgent\work\33bdfc1cae7b2a38\modules\MusicStore\samples\MusicStore
| [0.003s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: Attempting to start IIS Express on port: 63581
| [0.010s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: Executing command : C:\Program Files\IIS Express\iisexpress.exe /site:HttpTestSite /config:C:\BuildAgent\temp\buildTmp\tmp626D.tmp /trace:error /systray:false
| [0.010s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: SET ASPNETCORE_ENVIRONMENT=NtlmAuthentication
| [0.010s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: SET ASPNETCORE_DETAILEDERRORS=true
| [0.010s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: SET Data__DefaultConnection__ConnectionString=Data Source=(localdb)\MSSQLLocalDB;Initial Catalog=MusicStore_Test_202b08db4ce7419b8af5b4a00dfcb79b;Integrated Security=True;MultipleActiveResultSets=True;Connect Timeout=30
| [0.010s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: SET LAUNCHER_PATH=C:\BuildAgent\work\33bdfc1cae7b2a38\modules\MusicStore\samples\MusicStore\bin\Release\net461\MusicStore.exe
| [0.011s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: SET LAUNCHER_ARGS=
| [0.011s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: SET ASPNETCORE_CONTENTROOT=C:\BuildAgent\work\33bdfc1cae7b2a38\modules\MusicStore\samples\MusicStore
| [0.014s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: iisexpress Process 11736 started
| [0.081s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: iisexpress stdout: Starting IIS Express ...
| [0.097s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: iisexpress stdout: Initializing the W3 Server Started CTC = 216687296
| [0.103s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: iisexpress stdout: W3 Server initializing WinSock. CTC = 216687296
| [0.103s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: iisexpress stdout: W3 Server WinSock initialized. CTC = 216687296
| [0.103s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: iisexpress stdout: W3 Server ThreadPool initialized (ipm has signalled). CTC = 216687296
| [0.125s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: iisexpress stdout: Start listenerChannel http:0
| [0.126s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: iisexpress stdout: Successfully registered URL "http://localhost:63581/" for site "HttpTestSite" application "/"
| [0.126s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: iisexpress stdout: Registration completed for site "HttpTestSite"
| [0.126s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: iisexpress stdout: AppPool 'IISExpressAppPool' initialized
| [0.126s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: iisexpress stdout: InitComplete event signalled
| [0.127s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: iisexpress stdout: IIS Express is running.
| [0.127s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: Started iisexpress successfully. Process Id : 11736, Port: 63581
| [0.127s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: Application ready at URL: http://localhost:63581/
| [0.127s] NtlmAuthenticationTest Warning: Retry count 1..
| [0.130s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: iisexpress stdout: IncrementMessages called
| [0.131s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: iisexpress stdout: Request started: "GET" http://localhost:63581/
| [0.132s] Microsoft.AspNetCore.Server.IntegrationTesting.IIS.IISExpressDeployer Information: iisexpress stdout: Request ended: http://localhost:63581/ with HTTP status 401.2
| [0.135s
...
Other tests within that build may have failed with a similar message, but they are not listed here. Check the link above for more info.
This test failed on master.
CC @mkArtakMSFT
This issue was made automatically. If there is a problem contact ryanbrandenburg.
Please use this workflow to address this flaky test issue, including checking applicable checkboxes and filling in the applicable "TODO" entries:
-
Is this actually a flaky test?
- No, this is a regular test failure, fix the test/product (TODO: Link to commit/PR)
- Yes, proceed below...
-
Is this test failure caused by product code flakiness? (Either this product, or another product this test depends on.)
- File a bug against the product (TODO: Link to other bug)
- Is it possible to change the test to avoid the flakiness?
- Yes? Go to the "Change the test!" section.
- No?
- Disable the test (TODO: Link to PR/commit)
- Wait for other bug to be resolved
- Wait for us to get build that has the fix
- Re-enable our test (TODO: Link to PR/commit)
- Close this bug
-
Is it that the test itself is flaky? This includes external transient problems (e.g. remote server problems, file system race condition, etc.)
- Is there is a way to change our test to avoid this flakiness?
- Yes? Change the test!
- Change the test to avoid the flakiness, for example by using a different test strategy, or by adding retries w/ timeouts (TODO: Link to PR/commit)
- Run the test 100 times locally as a sanity check.
- Close this bug
- No?
- Is there any logging or extra information that we could add to make this more diagnosable when it happens again?
- Yes?
- Add the logging (TODO: Link to PR/commit)
- No?
- Delete the test because flaky tests are not useful (TODO: Link to PR/commit)
- Yes?
- Is there any logging or extra information that we could add to make this more diagnosable when it happens again?
- Yes? Change the test!
- Is there is a way to change our test to avoid this flakiness?
This comment was made automatically. If there is a problem contact ryanbrandenburg.
Smoke_Tests failed with about the same error on master.
Smoke_Tests failed with about the same error on master.
Smoke_Tests failed with about the same error on master.
Smoke_Tests failed with about the same error on master.
NtlmAuthenticationTest failed with about the same error on master.
OpenIdConnectTestSuite failed with about the same error on master.
Smoke_Tests failed with about the same error on master.
Smoke_Tests failed with about the same error on master.
Smoke_Tests failed with about the same error on master.
This comment was made automatically. If there is a problem contact ryanbrandenburg.
NtlmAuthenticationTest failed with about the same error on master.
NtlmAuthenticationTest failed with about the same error on master.
OpenIdConnectTestSuite failed with about the same error on master.
OpenIdConnectTestSuite failed with about the same error on master.
OpenIdConnectTestSuite failed with about the same error on master.
NtlmAuthenticationTest failed with about the same error on master.
NtlmAuthenticationTest failed with about the same error on master.
NtlmAuthenticationTest failed with about the same error on master.
Smoke_Tests failed with about the same error on master.
Smoke_Tests failed with about the same error on master.
Smoke_Tests failed with about the same error on master.
OpenIdConnectTestSuite failed with about the same error on master.
NtlmAuthenticationTest failed with about the same error on master.
OpenIdConnectTestSuite failed with about the same error on master.
Smoke_Tests failed with about the same error on master.
This comment was made automatically. If there is a problem contact ryanbrandenburg.
There were 3 failures with about the same error on master:
NtlmAuthenticationTest
OpenIdConnectTestSuite
NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
This comment was made automatically. If there is a problem contact ryanbrandenburg.
There were 3 failures with about the same error on master:
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
There were 3 failures with about the same error on master:
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
There were 3 failures with about the same error on master:
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
This comment was made automatically. If there is a problem contact ryanbrandenburg.
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
This comment was made automatically. If there is a problem contact ryanbrandenburg.
There were 3 failures with about the same error on master:
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
There were 3 failures with about the same error on master:
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
- NtlmAuthenticationTest
This comment was made automatically. If there is a problem contact ryanbrandenburg.
There were 3 failures with about the same error on master:
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
There were 3 failures with about the same error on master:
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
There were 3 failures with about the same error on master:
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
This comment was made automatically. If there is a problem contact ryanbrandenburg.
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
There were 3 failures with about the same error on master:
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
There were 3 failures with about the same error on master:
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
- NtlmAuthenticationTest
There were 3 failures with about the same error on master:
- NtlmAuthenticationTest
- OpenIdConnectTestSuite
- NtlmAuthenticationTest
Fixed by resolution for dotnet/aspnetcore#3544