Better support for non-async scenarios
Opened this issue · 4 comments
If you are using this within a ThreadPoolExecutor you may get the following error:
RuntimeError: There is no current event loop in thread 'ThreadPoolExecutor-0_0'.A workaround for this is to use asyncio.run instead. However, this has some overhead and it would be better if we provided a synchronous mechanism. @metric_scope can check whether or not an event loop exists, and if not, it can fall back to using a synchronous logger implementation.
This will likely be superseded by #21
Just curious when you think this will be resolved
Workaround :
In file aws_embedded_metrics/metric_scope/init.py
Change in line 49
from : loop = asyncio.get_event_loop()
to : loop = asyncio.new_event_loop()
So instead of looking for current event loop metrics create its own.
It's not a solution, just workaround.
If you're hitting this scenario then you are likely doing some explicit multi-threading since get_event_loop will create a new thread unless you're already running on a background thread (see here).
The solution above (creating a new event loop per logger) can run you into race conditions if you try to create multiple loggers on the same thread. It also has a lot of overhead by creating an event loop for each logger. Rather than doing that, you probably would want to create a single event loop for the app and run your tasks inside of that event loop rather than doing explicit multi-threading (see this example; alternatively you can use loop.run_in_executor() or asyncio.to_thread()). If you really need to do explicit multi-threading outside of an async runtime then you could create a loop per thread, though this has some overhead and would need to be done exactly once per thread.
Is this still planned? It's causing me all kinds of pain trying to use this package with Flask, Gunicorn and Gevent where for reasons I don't fully understand I cannot use Flask's async support. I don't want to be creating event loops per thread, so I'm using this hack to make MetricsLogger synchronous.
Fundamentally I don't think it's really okay for a core library like this to require async support, as in Python it's possible to be in execution contexts where you don't have access to the event loop on the main thread.