LISTENING_EVENT_PORT_DISCONNECTED does not happen on macOS

Question

LISTENING_EVENT_PORT_DISCONNECTED does not happen on macOS

EricCraigen opened this issue a year ago · 23 comments

Hi,

I am currently developing a cross-platform Kotlin/Java application that consumes jSerialComm. I discovered that ports would remain open when the device was physically disconnected from the system. I was able to solve this by adding a DataListener to listen for the LISTENING_EVENT_PORT_DISCONNECTED event. I am then able to close the port manually in code which then prevents that device from showing up in future calls to getCommPorts.

This method works perfectly on Windows 10/11 and Linux (Ubuntu 22.04); however, it appears that this event is not supported in macOS. The macOS version I am developing on is Ventura 13.4.1 (22F82) and the system is an M1 Mac Mini.

Is this expected? Or could I be bunging something up?

If this is expected and the LISTENING_EVENT_PORT_DISCONNECTED event is not supported on macOS, is there any possible way to have this become supported?

If fixing this is not an option due to native OS limitations, do you suggest any work around's to be able to detect when a device is physically unplugged from the macOS system so I can manually close the port?

Let me know if you need any further information from me.

Thanks much in advance and I look forward to hearing back soon!

Answer 1 · 2023-06-29T16:16:44.000Z

Hi!

The LISTENING_EVENT_PORT_DISCONNECTED is definitely supported on MacOS (in fact, it uses the exact same native code under-the-hood as the Linux version). I just tested this on my own Mac, and I couldn't reproduce your issue.

It's surprising that it wouldn't be working on Mac for you when it works on the other OS's (I would assume if there was an obvious bug, it wouldn't work anywhere). Is there any way you could post a minimal set of code that exhibits this issue on your computer?

Thanks!

Answer 2 · 2023-06-29T19:10:42.000Z

Thanks for the prompt reply! I am glad to hear that it should work and it's likely some mistake in how I've implemented it. Hopefully, we can figure out how that is.

The minimal code/pseudo code is as follows:

We call SerialPort.getCommPorts() and then create a SerialDevice object that contains the SerialPort that was identified by SerialPort.portDescription to be the device we support. When this SerialDevice object is initialized, we add a SerialPortDataListener that listens for the LISTENING_EVENT_PORT_DISCONNECTED event. When this event is received we then call SerialPort.closePort() so that the port does not get returned from future calls to SerialPort.getCommPorts().

So the logic flow is:

Call SerialPort.getCommPorts()
Identify the port by SerialPort.portDescription that we care about
Open the port and "interrogate" the device to ensure its the device we want, then create the SerialDevice object with the SerialPortDataListener
At some point the device becomes unplugged from the system
The SerialPortDataListener receives the LISTENING_EVENT_PORT_DISCONNECTED event
We then manually close the port and all SerialDevice resources
The device is no longer returned in calls to SerialPort.getCommPorts
We are then able to determine the device is no longer available on the system and remove it from the UI.

The code for the SerialPortDataListener is as follows:

this.serialPort.addDataListener(object : SerialPortDataListener {
    override fun getListeningEvents(): Int =
        SerialPort.LISTENING_EVENT_PORT_DISCONNECTED

    override fun serialEvent(event: SerialPortEvent) {
        if (event.eventType == SerialPort.LISTENING_EVENT_PORT_DISCONNECTED) {
            // device was unplugged while port was still open

            this@SerialDevice.close()
        }
    }
})

this@SerialDevice.close() is where SerialPort.closePort() gets called.

This works perfectly on Linux and Windows, but the event is never received on macOS for me. I am able to step into the serialEvent override code on macOS if I change the getListeningEvents override to:

override fun getListeningEvents(): Int =
        SerialPort.LISTENING_EVENT_PORT_DISCONNECTED or SerialPort.LISTENING_EVENT_DATA_RECEIVED

It behaves how I would expect with the additional listening event, but the LISTENING_EVENT_PORT_DISCONNECTED never gets received.

Please let me know if there is anything else you need from me and I'll get it back as soon as possible.

Cheers!

Answer 3 · 2023-06-29T23:26:22.000Z

If I generated a few native MacOS binaries, would you be able to run/test them? Specifically, I need to see exactly what is happening on the event polling side of things when you disconnect a port with all of the Java stuff out of the way.

Answer 4 · 2023-06-30T14:25:39.000Z

Yes for sure I could do that! Just point me to them once you have them generated. Thanks!

Answer 5 · 2023-06-30T14:38:17.000Z

No problem! Can you provide me with the values you are using for the following config parameters so I can try to match exactly what you're doing as closely as possible:

Baud Rate
Num Data Bits
Num Stop Bits
Parity
Is DTR Enabled
Is RTS Enabled
Is CTS Enabled
Is RS485 Mode Enabled

Answer 6 · 2023-06-30T14:54:53.000Z

Surely,

{
     "baud_rate": 115200,
     "data_bits": 8,
     "stop_bits": 1,
     "parity": 0,
     "dtr_enabled": true,
     "rts_enabled": true,
     "cts_enabled": false
}

And we do not set anything withRS485ModeParameters .

Answer 7 · 2023-06-30T16:47:50.000Z

Perfect, here's a test binary: https://www.dropbox.com/t/YH8RFDpM32UdkGr1

What I need for you to do is:

Plug the device with the problem into your Mac
Start the attached binary and select your desired device when prompted
At some point with the binary running, unplug your device
Report back here with the last few console output lines that you saw when you unplugged the device
You can stop the program at any time using Ctrl+C

Thanks!

Answer 8 · 2023-06-30T17:09:37.000Z

Ok, got that done.

It definitely responded when the device was unplugged; here are the results:

Select the index of the serial device to connect to:
	[0]: /dev/cu.wlan-debug (Description = wlan-debug)
	[1]: /dev/tty.wlan-debug (Description = wlan-debug (Dial-In))
	[2]: /dev/cu.Bluetooth-Incoming-Port (Description = Bluetooth-Incoming-Port)
	[3]: /dev/tty.Bluetooth-Incoming-Port (Description = Bluetooth-Incoming-Port (Dial-In))
	[4]: /dev/cu.usbmodem00000000001 (Description = CR1100)
	[5]: /dev/tty.usbmodem00000000001 (Description = CR1100 (Dial-In))
Target device index: 5
Poll Result: 1, Revents: 1, Codes: 128
Received event: Available
Poll Result: 1, Revents: 1, Codes: 128
Received event: Available
Poll Result: 1, Revents: 1, Codes: 128
Received event: Available
Poll Result: 0, Revents: 0, Codes: 0
Poll Result: 0, Revents: 0, Codes: 0
Poll Result: 0, Revents: 0, Codes: 0
Poll Result: 0, Revents: 0, Codes: 0
Poll Result: 0, Revents: 0, Codes: 0
Poll Result: 0, Revents: 0, Codes: 0
Poll Result: 0, Revents: 0, Codes: 0
Poll Result: 0, Revents: 0, Codes: 0
Poll Result: 1, Revents: 17, Codes: 384
Received event: Disconnected
Saving session...
...copying shared history...
...saving history...truncating history files...
...completed.
Deleting expired sessions...none found.
[Process completed]

Answer 9 · 2023-06-30T17:31:11.000Z

Ha, well that's not the result I expected! There were only 3 configuration options (in native code) that are in the real library that I didn't include here. Let's incrementally add those options and see when it fails. Can you test each of the following and just let me know which ones work and which don't (and if they don't work, also post the console log if it shows anything other than "Revents: 1, Codes: 128" or "Revents: 0, Codes: 0":

Test Setting DTR: https://www.dropbox.com/t/ZueieK33hG8hE1wq
Test Setting RTS: https://www.dropbox.com/t/Gs4kUkNmHjEFYKUC
Test Setting Both: https://www.dropbox.com/t/06bHnDAkQhk1Rjyd
Test Flow Control: https://www.dropbox.com/t/IfmAX4dkFGDot4SW

Answer 10 · 2023-06-30T17:58:31.000Z

So all 4 tests passed. The final log line of each was [Process Completed]. I can give you the output of each if needed.

Answer 11 · 2023-06-30T18:15:10.000Z

So weird! There's only one more native-side thing I can test: https://www.dropbox.com/t/acENosbQNQ48d91S

If this passes too, I'll need to move into the Java side of the code.

Answer 12 · 2023-06-30T18:24:51.000Z

That one passed as well. Let me know how I can help fight this from the Java side and I'll do whatever is needed.

Answer 13 · 2023-06-30T18:31:58.000Z

Okay, could you test the following Java application: https://www.dropbox.com/t/G3wSlJLQPknzdxkq

Should be able to run from console using java -jar jSerialComm-2.10.2-test.jar

This is roughly the same as the native binary tests from earlier, except I'm calling everything from the Java-side. It will scan for incoming data for 10 seconds unless a disconnect event is detected.

Answer 14 · 2023-06-30T19:13:53.000Z

Ok, so I ran that one.

196 bytes were read; when I unplugged the device the program output Port disconnected! and then closed the port.

Answer 15 · 2023-06-30T19:32:13.000Z

Okay, so that means this application (and therefore the underlying jSerialComm library) is working on your Mac in this configuration. In that case, I'm going to punt debugging to your side. I will attach the source code I used to generate that JAR application, and I need you to slowly mutate it until it's doing the same thing as your own application. Whenever you make a change that causes the port-disconnect functionality to stop working, we can go from there to figure out why exactly it's happening.

Source code: https://www.dropbox.com/t/h3YML0OAHfR25QXw

Answer 16 · 2023-06-30T19:34:36.000Z

Awesome, will do! I will be able to get to this later this evening. I will let you know the results as soon as I reach the breaking point and how so.

Thank you very much for your help in this matter!

Answer 17 · 2023-06-30T19:38:45.000Z

Thanks for helping to test and debug! It's nearly impossible to address issues like this when I can't recreate them on my side, so you're helping everyone!

Answer 18 · 2023-07-03T14:37:58.000Z

Glad to help! Sorry for the delay, it was a busy weekend for me.

I was able to dig into your code and figure out what is happening and can hopefully give you a solid repro case now.

In order to reproduce you only need to change one line of code:

public int getListeningEvents() { return SerialPort.LISTENING_EVENT_DATA_AVAILABLE | SerialPort.LISTENING_EVENT_PORT_DISCONNECTED; }

to

public int getListeningEvents() { return SerialPort.LISTENING_EVENT_PORT_DISCONNECTED; }

When I step into the serialEvent() override with your unchanged code, the LISTENING_EVENT_PORT_DISCONNECTED is received when the device is physically unplugged and the else logic is hit which closes the port.

When I step into the serialEvent override with my change, the event is never received and the debugger never stops in serialEvent().

It would seem to me that the bug is actually the fact that you can never receive/catch the LISTENING_EVENT_PORT_DISCONNECTED event when it is the only event registered in getListeningEvents().

I have tested this in our code base and by adding the LISTENING_EVENT_DATA_AVAILABLE event to getListeningEvents() I now receive the LISTENING_EVENT_PORT_DISCONNECTED event and am able to close the port explicitly when the correct event is received.

We created our own DataListener for reading data from the ports that is based on Kotlin Coroutines, which is why we do not use the LISTENING_EVENT_DATA_AVAILABLE event in our SerialPortDataListener.

I am not certain what would cause the LISTENING_EVENT_PORT_DISCONNECTED to never be received in the serialEvent() override when that is the only event registered with getListeningEvents() but I hope this helps you reproduce the issue and identify where the bug lies.

As always, please let me know if there is anything else that you need from me on the fighting front!

Cheers!

Answer 19 · 2023-07-03T15:15:09.000Z

I have done a bit more testing to ensure the workaround I have in place to get this working in our code and thought this info might be helpful as well.

The only time I am ever able to step into serialEvent and "catch" the LISTENING_EVENT_PORT_DISCONNECTED event is when I have registered both LISTENING_EVENT_PORT_DISCONNECTED and LISTENING_EVENT_DATA_AVAILABLE in the getListeningEvents() override.

I swapped LISTENING_EVENT_DATA_AVAILABLE for all other events one by one and was never able to get the desired behavior with any of the other events.

Hope this helps!

Answer 20 · 2023-07-06T17:25:16.000Z

@hedgecrw I know the holiday this week is throwing everything off, but I was just curious if you had a chance to look at this at all yet and if you have any ideas as to what could be causing this behavior.

Answer 21 · 2023-07-06T20:14:08.000Z

I finally just now figured out the issue (thanks to your help narrowing down a reproducible test-case). It actually isn't a bug in this library, it's a bug in the MacOS kernel. Specifically, their documentation page for the OS function call poll() states that status events (like POLLERR, POLLHUP, etc.) should not be passed into the function but they will always cause it to return if detected. This is exactly how it works on every other OS; however, I found that I do, in fact, have to pass POLLHUP as an event of interest in MacOS in order for it to detect when a serial port has become disconnected, unless I am already listening for a read event POLLIN from that device. So definitely a bug on their end, but luckily, there's an easy workaround.

Could you please test the latest 2.10.2-SNAPSHOT version and see if it resolves your issue:

SNAPSHOT Version: 2.10.2-SNAPSHOT
SNAPSHOT Direct Download Link
SNAPSHOT Instructions

Answer 22 · 2023-07-10T04:10:58.000Z

This has been resolved with release v2.10.2

Answer 23 · 2023-07-10T18:54:39.000Z

@hedgecrw my apologies for not getting back sooner, other issues took precedence. I was able to test this in the release and the issue is in fact resolved. I now get the LISTENING_EVENT_PORT_DISCONNECTED on macOS when that is the only event registered with the getListeningEvents() override method. Thank you very much for all your support on this issue!