swiftlang/swift-foundation

Crash When Streaming Files with Vapor - Related to swift-foundation

Closed this issue · 2 comments

omrd commented

After much discussion with @gwynne, it seems there's an issue with thread safety. Specifically, the problem is that getgrgid(3) is used in a manner that is not thread-safe. Possibley getpwuid(3) as well, for that matter.

The following backtrace shows an issue when accessing file properties

Debugging will be degraded due to missing types. Rebuilding the project will regenerate the needed module files.
* thread #2, name = 'omrd', stop reason = signal SIGSEGV: address not mapped to object (fault address: 0x13)
  * frame #0: 0x0000000004dff208 omrd`__libc_realloc + 168
    frame #1: 0x0000000004e131b8 omrd`getdelim + 244
    frame #2: 0x0000000004e0ef80 omrd`__getgrent_a + 76
    frame #3: 0x0000000004e0eaec omrd`__getgr_a + 132
    frame #4: 0x0000000004e0ea18 omrd`getgrgid + 60
    frame #5: 0x0000000003fd3afc omrd`partial apply for closure #1 in _FileManagerImpl.attributesOfItem(atPath:) [inlined] FoundationEssentials._nameFor(gid=$ss6UInt32VD @ scalar) -> Swift.Optional<Swift.String> at FileManager+Files.swift:50:21 [opt]
    frame #6: 0x0000000003fd3af4 omrd`partial apply for closure #1 in _FileManagerImpl.attributesOfItem(atPath:) at FileManager+Files.swift:195:28 [opt]
    frame #7: 0x0000000003fd37fc omrd`partial apply for closure #1 in _FileManagerImpl.attributesOfItem(atPath:) at FileManager+Files.swift:649:41 [opt]
    frame #8: 0x0000000003fd36ec omrd`partial apply for closure #1 in _FileManagerImpl.attributesOfItem(atPath:) at <compiler-generated>:0 [opt]
    frame #9: 0x000000000410f0a8 omrd`partial apply for closure #1 in String.withFileSystemRepresentation<A>(_:) [inlined] closure #1 ($0=<unavailable>, block=<unavailable>) throws -> τ_0_0 in Swift.String.withFileSystemRepresentation<τ_0_0>((Swift.Optional<Swift.UnsafePointer<Swift.Int8>>) throws -> τ_0_0) throws -> τ_0_0 at String+Internals.swift:152:17 [opt]
    frame #10: 0x000000000410f0a4 omrd`partial apply for closure #1 in String.withFileSystemRepresentation<A>(_:) at <compiler-generated>:0 [opt]
    frame #11: 0x000000000426ed1c omrd`closure #1 in _StringGuts.withCString<τ_0_0>($0=<unavailable>, body=<unavailable>) at StringGuts.swift:212:18 [opt]
    frame #12: 0x00000000043076fc omrd`partial apply for closure #1 in _StringGuts.withCString<A>(_:) at <compiler-generated>:0 [opt]
    frame #13: 0x0000000004383358 omrd`String.withCString<A>(_:) [inlined] reabstraction thunk helper <τ_0_0> from @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.Int8>) -> (@out τ_0_0, @error @owned Swift.Error) to @escaping @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.Int8>) -> (@out τ_0_0, @error @out Swift.Error) at <compiler-generated>:0 [opt]
    frame #14: 0x0000000004383350 omrd`String.withCString<A>(_:) [inlined] partial apply forwarder for reabstraction thunk helper <τ_0_0> from @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.Int8>) -> (@out τ_0_0, @error @owned Swift.Error) to @escaping @callee_guaranteed (@unowned Swift.UnsafeBufferPointer<Swift.Int8>) -> (@out τ_0_0, @error @out Swift.Error) at <compiler-generated>:0 [opt]
    frame #15: 0x0000000004383350 omrd`String.withCString<A>(_:) [inlined] generic partial specialization <serialized, Signature = @escaping @convention(thin) @convention(method) <τ_0_0><τ_1_0, τ_1_1, τ_1_2 where τ_0_0 == Swift.UInt8, τ_1_0 == Swift.Int8, τ_1_1 == Swift.Error, τ_1_2: ~Swift.Copyable> (@guaranteed @callee_guaranteed @substituted <τ_0_0, τ_0_1, τ_0_2 where τ_0_0: ~Swift.Copyable, τ_0_0: ~Swift.Escapable, τ_0_1: ~Swift.Copyable, τ_0_1: ~Swift.Escapable, τ_0_2: ~Swift.Copyable, τ_0_2: ~Swift.Escapable> (@unowned Swift.UnsafeBufferPointer<τ_0_0>) -> (body=<unavailable>) for <Swift.Int8Swift.Errorτ_1_2>, @unowned Swift.UnsafeBufferPointer<Swift.UInt8>) -> (@out τ_1_2, @error @owned Swift.Error)> of Swift.UnsafeBufferPointer< where τ_0_0: ~Swift.Copyable>.withMemoryRebound<τ_0_0, τ_0_1, τ_0_2 where τ_1_1: Swift.Error, τ_1_0: ~Swift.Copyable, τ_1_2: ~Swift.Copyable>(to: τ_1_0.Type, _: (Swift.UnsafeBufferPointer<τ_1_0>) throws(τ_1_1) -> τ_1_2) throws(τ_1_1) -> τ_1_2 at UnsafeBufferPointer.swift:0 [opt]
    frame #16: 0x000000000438334c omrd`String.withCString<A>(_:) [inlined] closure #1 (utf8=<unavailable>, f=<unavailable>) throws -> τ_0_0 in Swift._StringGuts.withFastCChar<τ_0_0>((Swift.UnsafeBufferPointer<Swift.Int8>) throws -> τ_0_0) throws -> τ_0_0 at StringGuts.swift:170:23 [opt]
    frame #17: 0x000000000438334c omrd`String.withCString<A>(_:) [inlined] Swift._StringGuts.withFastUTF8<τ_0_0>(f=<unavailable>) throws -> τ_0_0) throws -> τ_0_0 at StringGuts.swift:152:16 [opt]
    frame #18: 0x0000000004383338 omrd`String.withCString<A>(_:) [inlined] Swift._StringGuts.withFastCChar<τ_0_0>(f=<unavailable>) throws -> τ_0_0) throws -> τ_0_0 at StringGuts.swift:169:21 [opt]
    frame #19: 0x0000000004383338 omrd`String.withCString<A>(_:) [inlined] Swift._StringGuts.withCString<τ_0_0>(body=<unavailable>) throws -> τ_0_0) throws -> τ_0_0 at StringGuts.swift:211:21 [opt]
    frame #20: 0x000000000438330c omrd`String.withCString<τ_0_0>(body=0x000000000410f094 omrd`partial apply forwarder for closure #1 (Swift.UnsafePointer<Swift.Int8>) throws -> A in Swift.String.withFileSystemRepresentation<A>((Swift.Optional<Swift.UnsafePointer<Swift.Int8>>) throws -> A) throws -> A at <compiler-generated>) at LifetimeManager.swift:264:22 [opt]
    frame #21: 0x000000000410f704 omrd`String.withFileSystemRepresentation<τ_0_0>(block=<unavailable>) at String+Internals.swift:151:20 [opt]
    frame #22: 0x0000000003fca2fc omrd`FileManager.withFileSystemRepresentation<τ_0_0>(path=<unavailable>, body=<unavailable>) at SwiftFileManager.swift:337:18 [opt]
    frame #23: 0x0000000003fd0874 omrd`_FileManagerImpl.attributesOfItem(path=Swift.String @ 0x0000b251c6d14bb0) at FileManager+Files.swift:639:25 [opt]
    frame #24: 0x0000000003fc7bfc omrd`merged FoundationEssentials.FileManager.contentsOfDirectory(atPath: Swift.String) throws -> Swift.Array<Swift.String> + 72
    frame #25: 0x0000000003fc9f00 omrd`FoundationEssentials.FileManager.attributesOfItem(atPath: Swift.String) throws -> Swift.Dictionary<FoundationEssentials.FileAttributeKey, Any> + 20
    frame #26: 0x00000000036d7568 omrd`FileIO.streamFile(path=Swift.String @ 0x0000fffff7ff7450, chunkSize=131072, mediaType=<unavailable>, onCompleted=0x00000000036c9cb0 omrd`closure #1 @Sendable (Swift.Result<(), Swift.Error>) -> () in default argument 3 of Vapor.FileIO.streamFile(at: Swift.String, chunkSize: Swift.Int, mediaType: Swift.Optional<Vapor.HTTPMediaType>, onCompleted: (Swift.Result<(), Swift.Error>) -> ()) -> Vapor.Response at FileIO.swift:130) at FileIO.swift:134:55
    frame #27: 0x0000000003d2aa28 omrd`closure #1 in static Server.setupRoutes(request=<could not get Swift runtime type info for type $s5Vapor7RequestCD>, app=<could not get Swift runtime type info for type $s5Vapor11ApplicationCD>) at Routes.swift:26:29
    frame #28: 0x00000000045ab610 omrd`swift::runJobInEstablishedExecutorContext(swift::Job*) [inlined] swift::AsyncTask::runInFullyEstablishedContext(this=0x0000fffff7ddfc60) at Task.h:367:12
    frame #29: 0x00000000045ab608 omrd`swift::runJobInEstablishedExecutorContext(job=0x0000fffff7ddfc60) at Actor.cpp:237:11
    frame #30: 0x00000000045ac6c8 omrd`::swift_job_run(swift::Job *, swift::SerialExecutorRef) [inlined] swift_job_runImpl(job=<unavailable>, executor=SerialExecutorRef @ 0x0000b251c6e187e0) at Actor.cpp:1921:3
    frame #31: 0x00000000045ac64c omrd`swift_job_run(job=0x0000fffff7ddfc60, executor=SerialExecutorRef @ 0x0000b251c6e6c4a0) at CompatibilityOverrideConcurrency.def:114:1
    frame #32: 0x000000000465cd0c omrd`_dispatch_continuation_pop at inline_internal.h:2455:2
    frame #33: 0x000000000465ccd8 omrd`_dispatch_continuation_pop [inlined] _dispatch_continuation_pop_inline(dou=dispatch_object_t @ x20, dic=<unavailable>, flags=<unavailable>, dqu=dispatch_queue_class_t @ x19) at inline_internal.h:2498:3
    frame #34: 0x000000000465cc24 omrd`_dispatch_continuation_pop(dou=dispatch_object_t @ x20, dic=0x0000fffff7ff7998, flags=<unavailable>, dqu=dispatch_queue_class_t @ x19) at queue.c:287:2
    frame #35: 0x000000000465cb44 omrd`_dispatch_async_redirect_invoke(dc=0x0000fffff7d5e480, dic=0x0000fffff7ff7998, flags=131072) at queue.c:798:2
    frame #36: 0x00000000046667e0 omrd`_dispatch_worker_thread [inlined] _dispatch_continuation_pop_inline(dou=dispatch_object_t @ x21, dic=0x0000fffff7ff7998, flags=131072, dqu=dispatch_queue_class_t @ x19) at inline_internal.h:2496:3
    frame #37: 0x00000000046667a4 omrd`_dispatch_worker_thread [inlined] _dispatch_root_queue_drain(dq=0x0000000004fa5c00, pri=<unavailable>, flags=131072) at queue.c:6114:3
    frame #38: 0x0000000004666704 omrd`_dispatch_worker_thread(context=0x0000000004fa5c00) at queue.c:6249:3
    frame #39: 0x0000000004e2213c omrd`start + 144
    frame #40: 0x0000000004e207c0 omrd`__clone + 48
(lldb)

The relevant code in swift-foundation is https://github.com/swiftlang/swift-foundation/blob/main/Sources/FoundationEssentials/FileManager/FileManager%2BFiles.swift#L45-L57).

They have checked to make sure FileManager is used in a thread safe pattern, which they confirmed that it was.

Link to Vapor's file handling.
https://github.com/vapor/vapor/blob/1310c6f5462e23bca08fcf7c26448891a4908294/Sources/Vapor/Utilities/FileIO.swift#L134

parkera commented

Thanks - we're looking into it.