crash on ubuntu: syncShutdown() must not be called when on an EventLoop.
shenfu1991 opened this issue · 4 comments
i use Vapor, macOS work fine,but ubuntu20.04 crash
let httpClient = HTTPClient(eventLoopGroupProvider: .createNew)
do {
var request = try HTTPClient.Request(url: URLString, method: .GET)
// request.headers.add(name: "X-MBX-APIKEY", value: API_Key)
// request.body = .string(<#T##string: String##String#>)
httpClient.execute(request: request).whenComplete { result in
switch result {
case .failure(let error):
print(error)
case .success(let response):
if response.status == .ok {
do {
let json = try JSONSerialization.jsonObject(with: response.body!)
print(json)
} catch {
}
} else {
}
}
try? httpClient.syncShutdown()
}
}catch {
}
full log:
AsyncHTTPClient/HTTPClient.swift:166: Fatal error: BUG DETECTED: syncShutdown() must not be called when on an EventLoop.
Calling syncShutdown() on any EventLoop can lead to deadlocks.
Current eventLoop: SelectableEventLoop { selector = Selector { descriptor = 28 }, thread = NIOThread(name = NIO-ELT-1-#0) }
Current stack trace:
0 libswiftCore.so 0x00007f66fec80930 _swift_stdlib_reportFatalErrorInFile + 112
1 libswiftCore.so 0x00007f66fe94b8ec + 1444076
2 libswiftCore.so 0x00007f66fe94b708 + 1443592
3 libswiftCore.so 0x00007f66fe94a1f0 _assertionFailure(_:_:file:line:flags:) + 419
4 rnx_2 0x000055de1108ebf2 + 4336626
5 rnx_2 0x000055de1108e6a3 + 4335267
6 rnx_2 0x000055de10f44ad5 + 2984661
7 rnx_2 0x000055de118ba5bc + 12903868
8 rnx_2 0x000055de118b25d0 + 12871120
9 rnx_2 0x000055de118b38f3 + 12876019
10 rnx_2 0x000055de118b3830 + 12875824
11 rnx_2 0x000055de118b3e42 + 12877378
12 rnx_2 0x000055de118b403f + 12877887
13 rnx_2 0x000055de110e8484 + 4703364
14 rnx_2 0x000055de110ec611 + 4720145
15 rnx_2 0x000055de110ec3dc + 4719580
16 rnx_2 0x000055de110ebddc + 4718044
17 rnx_2 0x000055de110eebb9 + 4729785
18 rnx_2 0x000055de110113c6 + 3822534
19 rnx_2 0x000055de1100e772 + 3811186
20 rnx_2 0x000055de1100c55c + 3802460
21 rnx_2 0x000055de110123cf + 3826639
22 rnx_2 0x000055de1101f3bc + 3879868
23 rnx_2 0x000055de11bb2d31 + 16018737
24 rnx_2 0x000055de11bb6278 + 16032376
25 rnx_2 0x000055de11bb90b0 + 16044208
26 rnx_2 0x000055de11baefd2 + 16003026
27 rnx_2 0x000055de11bb45ec + 16025068
28 rnx_2 0x000055de11b7c927 + 15796519
29 rnx_2 0x000055de11b7d00d + 15798285
30 rnx_2 0x000055de11b827ec + 15820780
31 rnx_2 0x000055de11bebbcf + 16251855
32 rnx_2 0x000055de11bee702 + 16262914
33 rnx_2 0x000055de11bee809 + 16263177
34 libpthread.so.0 0x00007f66fef55609 + 34313
35 libc.so.6 0x00007f66fe4bc0f0 clone + 67
Received signal 4. Backtrace:
0x55de113464af, Backtrace.(printBacktrace in _B82A8C0ED7C904841114FDF244F9E58E)(signal: Swift.Int32) -> () at /root/rnx_2/x-bot/.build/checkouts/>
0x55de113467d9, closure #1 (Swift.Int32) -> () in static Backtrace.Backtrace.install(signals: Swift.Array) -> () at /root/rnx_2/x-bo>
0x55de113467f8, @objc closure #1 (Swift.Int32) -> () in static Backtrace.Backtrace.install(signals: Swift.Array) -> () at /root/rnx_>
0x7f66fef6141f
0x7f66fe94a39f
0x55de1108ebf1, AsyncHTTPClient.HTTPClient.syncShutdown(requiresCleanClose: Swift.Bool) throws -> () at /root/rnx_2/x-bot/.build/checkouts/async->
0x55de1108e6a2, AsyncHTTPClient.HTTPClient.syncShutdown() throws -> () at /root/rnx_2/x-bot/.build/checkouts/async-http-client/Sources/AsyncHTTPC>
0x55de10f44ad4, closure #1 @Sendable (Swift.Result) -> () in App.CoreNetwokingManager.request(m>
0x55de118ba5bb, closure #1 @Sendable () -> NIOCore.CallbackList in NIOCore.EventLoopFuture._publicWhenComplete(@Sendable (Swift.Result
if remove: try? httpClient.syncShutdown()，
Fatal error: Client not shut down before the deinit. Please call client.syncShutdown() when no longer neede.
full log:
AsyncHTTPClient/HTTPClient.swift:141: Fatal error: Client not shut down before the deinit. Please call client.syncShutdown() when no longer neede> Current stack trace: 0 libswiftCore.so 0x00007f9a47720930 _swift_stdlib_reportFatalErrorInFile + 112 1 libswiftCore.so 0x00007f9a473eb8ec + 1444076 2 libswiftCore.so 0x00007f9a473eb708 + 1443592 3 libswiftCore.so 0x00007f9a473ea1f0 _assertionFailure(_:_:file:line:flags:) + 419 4 rnx_2 0x000055c5cd083547 + 4334919 5 rnx_2 0x000055c5cd0ead1e + 4758814 6 rnx_2 0x000055c5cd0eac93 + 4758675 7 rnx_2 0x000055c5cd08335f + 4334431 8 rnx_2 0x000055c5cd083569 + 4334953 9 libswiftCore.so 0x00007f9a476a1c3b + 4287547 10 libswiftCore.so 0x00007f9a476a23db + 4289499 11 rnx_2 0x000055c5cd09a99d + 4430237 12 libswiftCore.so 0x00007f9a476a1c3b + 4287547 13 libswiftCore.so 0x00007f9a476a23db + 4289499 14 rnx_2 0x000055c5cd0b753e + 4547902 15 rnx_2 0x000055c5cd0e1a03 + 4721155 16 rnx_2 0x000055c5cd0e1a55 + 4721237 17 libswiftCore.so 0x00007f9a476a1c3b + 4287547 18 libswiftCore.so 0x00007f9a476a23db + 4289499 19 rnx_2 0x000055c5cd00e798 + 3856280 20 rnx_2 0x000055c5cd012af6 + 3873526 21 rnx_2 0x000055c5cd001478 + 3802232 22 rnx_2 0x000055c5cd0072df + 3826399 23 rnx_2 0x000055c5cd0142cc + 3879628 24 rnx_2 0x000055c5cdba7c71 + 16018545 25 rnx_2 0x000055c5cdbab1b8 + 16032184 26 rnx_2 0x000055c5cdbadff0 + 16044016 27 rnx_2 0x000055c5cdba3f12 + 16002834 28 rnx_2 0x000055c5cdba952c + 16024876 29 rnx_2 0x000055c5cdb71867 + 15796327 30 rnx_2 0x000055c5cdb71f4d + 15798093 31 rnx_2 0x000055c5cdb7772c + 15820588 32 rnx_2 0x000055c5cdbe0b0f + 16251663 33 rnx_2 0x000055c5cdbe3642 + 16262722 34 rnx_2 0x000055c5cdbe3749 + 16262985 35 libpthread.so.0 0x00007f9a479f5609 + 34313 36 libc.so.6 0x00007f9a46f5c0f0 clone + 67 Received signal 4. Backtrace: 0x55c5cd33b3bf, Backtrace.(printBacktrace in _B82A8C0ED7C904841114FDF244F9E58E)(signal: Swift.Int32) -> () at /root/rnx_2/x-bot/.build/checkouts/> 0x55c5cd33b6e9, closure #1 (Swift.Int32) -> () in static Backtrace.Backtrace.install(signals: Swift.Array) -> () at /root/rnx_2/x-bo> 0x55c5cd33b708, @objc closure #1 (Swift.Int32) -> () in static Backtrace.Backtrace.install(signals: Swift.Array) -> () at /root/rnx_> 0x7f9a47a0141f 0x7f9a473ea39f 0x55c5cd083546, closure #1 () -> () in AsyncHTTPClient.HTTPClient.deinit at /root/rnx_2/x-bot/.build/checkouts/async-http-client/Sources/AsyncHTT> 0x55c5cd0ead1d, closure #1 () -> Swift.Bool in implicit closure #1 () -> Swift.Bool in AsyncHTTPClient.debugOnly(() -> ()) -> () at /root/rnx_2/x> 0x55c5cd0eac92, AsyncHTTPClient.debugOnly(() -> ()) -> () at /root/rnx_2/x-bot/.build/checkouts/async-http-client/Sources/AsyncHTTPClient/Utils.s> 0x55c5cd08335e, AsyncHTTPClient.HTTPClient.deinit at /root/rnx_2/x-bot/.build/checkouts/async-http-client/Sources/AsyncHTTPClient/HTTPClient.swif> 0x55c5cd083568, AsyncHTTPClient.HTTPClient.__deallocating_deinit at /root/rnx_2/x-bot/.build/checkouts/async-http-client/Sources/AsyncHTTPClient/>
When an error is thrown, you syncShutdown is never called.
You can use a defer block right after creating your client to make sure it gets shutdown when you exit the scope.
let httpClient = HTTPClient(eventLoopGroupProvider: .createNew)
defer {
try? httpClient.syncShutdown() // <--- call the shutdown here
}
do {
var request = try HTTPClient.Request(url: URLString, method: .GET)
// request.headers.add(name: "X-MBX-APIKEY", value: API_Key)
// request.body = .string(<#T##string: String##String#>)
httpClient.execute(request: request).whenComplete { result in
switch result {
case .failure(let error):
print(error)
case .success(let response):
if response.status == .ok {
do {
let json = try JSONSerialization.jsonObject(with: response.body!)
print(json)
} catch {
// you should do sth here, at least log the error
}
} else {
}
}
}
} catch {
// you should do sth here, at least log the error
}
So you are running into two different issues. Lets first talk about
Fatal error: BUG DETECTED: syncShutdown() must not be called when on an EventLoop.
The closure that is passed to whenComplete will be executed on the EventLoop of the HTTPClient. Shutting down the HTTPClient also involves shutting down the EventLoop which we can't do while we are executing code on it. However, we can start the shutdown asynchronously by calling the "non-sync" version which takes a DispatchQueue and closure.
Lets talk about the second issue:
if remove: try? httpClient.syncShutdown()，
Fatal error: Client not shut down before the deinit. Please call client.syncShutdown() when no longer neede.
As @t089 rightly points out, you are not shutting down the HTTPClient on all code paths. However, if you are doing it as @t089 proposed, you will shutdown the client too early and your request will likely be canceled. This is because the defer is executed as soon as we have submitted the request.
If you continue to use the promise/delegate API, you need to make sure you call shutdown on all code paths and wait until all request have completed. However, you can also switch to the new async/await API to make your life a bit easier as we can just call await client.shutdown() at the end of the scope:
async-http-client/Examples/GetJSON/GetJSON.swift
Lines 36 to 50 in 59bfb96
As @t089 rightly points out, you are not shutting down the HTTPClient on all code paths. However, if you are doing it as @t089 proposed, you will shutdown the client too early and your request will likely be canceled. This is because the defer is executed as soon as we have submitted the request.
Whoops good catch, I was already so into async/await land that I did not see the future callbacks, sorry for the confusion!
@dnadoba Thank you very much, the analysis is very detailed and I have benefited a lot.