asyncio.dispatcher: exceptions are not reported

[eddybl]

While working on a SoftIOC I ran into a situation where an exception which happened in a function was not reported on the IOC shell.

Here is a minimum working example:

import asyncio
from softioc import asyncio_dispatcher, softioc
import logging

log = logging.getLogger("")

async def other_function():
    log.warning("In function call")
    await asyncio.sleep(2)
    raise RuntimeError("This exception is not visible")

dispatcher = asyncio_dispatcher.AsyncioDispatcher()

softioc.iocInit(dispatcher)

log.warning("Before function call")
asyncio.run_coroutine_threadsafe(other_function(), dispatcher.loop)

log.warning("After function call")

softioc.interactive_ioc()

SoftIOC continues to run and reports all log messages, but since the exception is not reported it took me a while to realize that there even is a problem. Is there anyway how I can get the exception to show up?

[AlexanderWells-diamond]

The issue is that asyncio.run_coroutine_threadsafe() by will put exceptions into the object that that function call returns, expecting the caller to interrogate it for the outcome of the coroutine. See the docs for more information.

You can get the exception printed as you want by re-using the existing dispatcher like so:

...
log.warning("Before function call")
# asyncio.run_coroutine_threadsafe(other_function(), dispatcher.loop)
dispatcher(other_function)

log.warning("After function call")
...

This will piggyback on PythonSoftIOC's existing logging and print the exception into your terminal.

I think the only caveat to this approach is that the function you pass in will run in the same event loop as all other async work in your IOC, and so therefore could inadvertently delay/block processing if the function ever blocks. But this is an edge case and probably won't apply to you.

[eddybl]

That was how I actually originally started: using dispatcher(other_function).

But then I was starting to run into issues with more complex other_functions with multiple arguments (including a list as one argument, which was unintentionally expanded). To circumvent that I was adding wrapper functions around other_function().. but at some point and after looking at the dispatcher.__call__() it seemed to me that with this setup the code was creating wrappers around wrappers around wrappers, while as with run_coroutine_threadsafe() I was able to pass other_functions with any set of arguments.

But at that point I was not aware of the exception handling. Looking at dispatcher.__call__() again it seems like I would still not get the exception, but just the log output, that there was an exception without any specifics or tracebacks?

[AlexanderWells-diamond]

Yes, we currently never return the exception object back to the user. We do a call to logging.exception(), which should print any message and stack trace present in the exception into wherever the logging has been configured to point to - by default I think it's stderr.

I'm not quite sure what you are asking. Do you need access to the Exception object itself, as you have stored special data there that the logging.exception() call doesn't recognise and so doesn't print?

As you've probably seen, we currently have no mechanism to pass data back from the result of the dispatched function - we throw away the returned object from our call to asyncio.run_coroutine_threadsafe(). In principal it's possible to save that object, and add a method to query for it at some later time. The major hiccup is that we also support Cothread as a second asynchronous provider and do our best to keep the APIs for the two as close as possible.

Overall it sounds like you may want to do as you were originally doing in your sample code, but keep track of the returned object yourself and interrogate the result for whether it succeeded or threw an exception?

EDIT: Just to check, what version of PythonSoftIOC are you running? In version 4.1.0 I added the ability to pass arbitrary arguments into the asyncio_dispatcher, something like this:

dispatcher(my_func, func_args=[["ABC", "DEF"], 123])

That may alleviate your issue with multiple layers of wrapper functions?

[eddybl]

EDIT: Just to check, what version of PythonSoftIOC are you running? In version 4.1.0 I added the ability to pass arbitrary arguments into the asyncio_dispatcher, something like this:

dispatcher(my_func, func_args=[["ABC", "DEF"], 123])

That may alleviate your issue with multiple layers of wrapper functions?

Yes! That actually solves my original issue of not using dispatcher(other_function) to begin with. Now the exception is also properly printed to the shell, thanks a lot!