Creating Task crashes the runtime

[jtodaone]

Swift 5.6.1, Swift-Colab Install Script Latest Release (v2.0), Google-Colab.

I tried to test concurrency, but it fails to launch anything. Creating any Task (structured or detached) on top level crashes the runtime.

First I imported Foundation and _Concurrency. Without _Concurrency it just doesn't recognise Task.

import Foundation
import _Concurrency

And then launching one of the following crashes:

Task {
    for i in 1...100{
        print(i)
        try? await Task.sleep(nanoseconds: 1_000_000_000)
    }
}

Task {}

Task.detached {}

I don't know if it will help, but here's the log I got. It seems like it just contains bunch of restart messages.

app.log

[philipturner]

That log doesn't really give any useful information, but this is good to note. I have recently been working on a version 2.1 that fixes many bugs and I/O problems. I think I remember that I couldn't use Task without importing _Concurrency. I'll make sure to investigate this crash before releasing v2.1.

[philipturner]

If you're trying to utilize multiple CPU cores, you may want to consider the following stats. A Colab VM has two CPU cores, each of which has hyperthreading. So that is 4 CPU threads. LLDB and the Stdout handlers require that you leave at least 1 thread free to collect Stdout for a good user experience. So it you're trying to parallelize a problem on the CPU, aim for 3 threads.

[philipturner]

I just released Swift-Colab 2.1, which should allow you to view the print output while a cell is executing. There was a regression in v2.0 that stopped output from appearing until the cell had finished.

The problem isn't about dynamically loading libswift_Concurrency. I tried that (see below), but it has no effect on the behavior. Rather, it's probably already pre-loaded just like other Standard Library binaries.

import Foundation
dlopen(
  "/opt/swift/toolchain/usr/lib/swift/linux/libswift_Concurrency.so",
  RTLD_NOW)

All forms of errors should be handled, including proper runtime crashes. That was a big focus of v2.1. So it's likely a segfault within the your Swift code that was executing inside the interpreter. Nothing can recover from that, I'm afraid.

Interestingly, it doesn't crash when you call print after the task's initializer. I frontloaded the v2.1 release just to demonstrate this example.

import Foundation
import _Concurrency
print("1")
Task {
    for i in 1...100 {
        print("q", i)
        try? await Task.sleep(nanoseconds: 1_000_000_000)
    }
}
print("2")
for i in 0..<100 {
   print("v", i)
   usleep(1_000_000)
}

It works! The output is below. I'm kind of burnt out on investigating Colab bugs; I'd like to get back to working on S4TF (and I'm still too lazy to properly document Swift-Colab even now).

1
2
v 0
q 1
v 1
q 2
v 2
q 3
v 3
q 4

The fix will likely come in v2.2, if there is one. Maybe I could insert an extra print() expression after a code cell, after LLDB has extracted the textual description of whatever was in the actual last code line. I would have to handle stdout specially though, because that adds another newline. Furthermore, I would run the line import _Concurrency at kernel startup. Will adding a blank print() after your concurrency code work for your purposes for now?

// No import Foundation needed for this snippet
import _Concurrency
Task {}
print()

[jtodaone]

Thanks for the quick investigation! It seems like it will make do for the moment. Thanks again for keeping this project alive!

[philipturner]

I found that all you have to do to prevent the crash is something this simple:

_ = 0

And the Task will continue executing in the next block of code, where it was not originally declared. I made an infinitely running task in a previous code block and now it was leaking into stdout because it kept calling print in the background. I think this behavior goes against the design philosophy of Jupyter notebooks, but oh well.

[philipturner]

I identified what I think is the culprit. If you change this:

Task {}

To the following, it doesn't crash.

let x = Task {}

In the second situation, it doesn't fetch the Task object's metadata for printing in output. That means the crash happens when reading a description of the task in LLDB, so the crash happens in C++ code.

[philipturner]

I have fixed the bug. I wasn't checking whether a C-style string pointer was null. The fix will be upstreamed in v2.2.

swift-colab/Sources/JupyterKernel/SwiftKernel/InitSwift.swift

Lines 29 to 36 in f4965f9

	func initSwift() throws {
	try initReplProcess()
	try initKernelCommunicator()
	try initConcurrency()
	sigintHandler = SIGINTHandler()
	sigintHandler.start()
	}

swift-colab/Sources/JupyterKernel/SwiftKernel/InitSwift.swift

Lines 74 to 82 in f4965f9

	fileprivate func initConcurrency() throws {
	// If this is a pre-concurrency Swift version, the import is a no-op.
	let result = execute(code: """
	import _Concurrency
	""")
	if result is ExecutionResultError {
	throw Exception("Error importing _Concurrency: \(result)")
	}
	}

swift-colab/Sources/LLDBProcess/LLDBProcess.cpp

Lines 127 to 131 in f4965f9

	if (errorType == eErrorTypeInvalid && unowned_desc == NULL) {
	// The last line of code created a `Task`. This has a null description, so
	// act as if it's a `SuccessWithoutValue`.
	errorType = eErrorTypeGeneric;
	}

[philipturner]

Concurrency now works on the latest release of Swift-Colab, v2.2. I added a test notebook for concurrency (link).