swift-server/async-http-client

flaky tests in `HTTPClientSOCKSTests` if run in parallel

Closed this issue · 0 comments

dnadoba commented

Observed Failures

testProxySOCKS

Test Suite 'Selected tests' started at 2021-11-25 17:55:58.033
Test Suite 'HTTPClientSOCKSTests' started at 2021-11-25 17:55:58.035
Test Case 'HTTPClientSOCKSTests.testProxySOCKS' started at 2021-11-25 17:55:58.036
<EXPR>:0: error: HTTPClientSOCKSTests.testProxySOCKS : threw error "bind(descriptor:ptr:bytes:): Address already in use (errno: 98)"
Test Case 'HTTPClientSOCKSTests.testProxySOCKS' failed (0.022 seconds)
Test Suite 'HTTPClientSOCKSTests' failed at 2021-11-25 17:55:58.058
	 Executed 1 test, with 1 failure (1 unexpected) in 0.022 (0.022) seconds
Test Suite 'Selected tests' failed at 2021-11-25 17:55:58.058
	 Executed 1 test, with 1 failure (1 unexpected) in 0.022 (0.022) seconds
Build step 'Execute shell' marked build as failure

https://ci.swiftserver.group/job/async-http-client-swift55-prb/371/console

testProxySOCKSMisbehavingServer

Test Suite 'Selected tests' started at 2021-11-25 18:01:29.035
Test Suite 'HTTPClientSOCKSTests' started at 2021-11-25 18:01:29.038
Test Case 'HTTPClientSOCKSTests.testProxySOCKSMisbehavingServer' started at 2021-11-25 18:01:29.038
<EXPR>:0: error: HTTPClientSOCKSTests.testProxySOCKSMisbehavingServer : threw error "bind(descriptor:ptr:bytes:): Address already in use (errno: 98)"
Test Case 'HTTPClientSOCKSTests.testProxySOCKSMisbehavingServer' failed (0.01 seconds)
Test Suite 'HTTPClientSOCKSTests' failed at 2021-11-25 18:01:29.048
	 Executed 1 test, with 1 failure (1 unexpected) in 0.01 (0.01) seconds
Test Suite 'Selected tests' failed at 2021-11-25 18:01:29.048
	 Executed 1 test, with 1 failure (1 unexpected) in 0.01 (0.01) seconds

https://ci.swiftserver.group/job/async-http-client-swift55-prb/372/console
https://ci.swiftserver.group/job/async-http-client-swift53-prb/714/console

testProxySOCKSFailureInvalidServer

Test Suite 'Selected tests' started at 2021-11-25 18:02:00.075
Test Suite 'HTTPClientSOCKSTests' started at 2021-11-25 18:02:00.092
Test Case 'HTTPClientSOCKSTests.testProxySOCKSFailureInvalidServer' started at 2021-11-25 18:02:00.092
/code/Tests/AsyncHTTPClientTests/HTTPClient+SOCKSTests.swift:121: error: HTTPClientSOCKSTests.testProxySOCKSFailureInvalidServer : XCTAssertThrowsError failed: did not throw error - 
Test Case 'HTTPClientSOCKSTests.testProxySOCKSFailureInvalidServer' failed (0.181 seconds)
Test Suite 'HTTPClientSOCKSTests' failed at 2021-11-25 18:02:00.273
	 Executed 1 test, with 1 failure (0 unexpected) in 0.181 (0.181) seconds
Test Suite 'Selected tests' failed at 2021-11-25 18:02:00.273
	 Executed 1 test, with 1 failure (0 unexpected) in 0.181 (0.181) seconds
1
Build step 'Execute shell' marked build as failure

https://ci.swiftserver.group/job/async-http-client-swift52-prb/1013/console

Probable Root Cause

We start a server with a fixed port but should let the operation system chose one for us:

async-http-client/Tests/AsyncHTTPClientTests/SOCKSTestUtils.swift

Line 60 in 3b18e76

self.channel = try bootstrap.bind(host: "localhost", port: 1080).wait()

parallel tests were enabled in #495