Asynchronous event dispatching to allow Flask to respond immediately

Question

Asynchronous event dispatching to allow Flask to respond immediately

CameronAavik opened this issue 7 years ago · 37 comments

Description

As per the Events API documentation, we should send a valid response in 3 seconds, when using this API I found it quite difficult to figure out how to achieve this when I tried to implement something which lasted longer than 3 seconds.

We are still trying to get it working in our bot, but it got somewhat messy and I feel like that it would be much nicer if it were in this library rather than having to make our own.

The way I'd see this being implemented is that the event emitter would run each handler in a background process by default (or maybe a way to opt-in to this behaviour?). I understand this might cause issues for bots which are already dependent on the current behaviour of the API though, but I would be curious to know your thoughts about whether this is something you'd like to include in some form.

What type of issue is this? (place an `x` in one of the `[ ]`)

Requirements

I've read and understood the Contributing guidelines and have done my best effort to follow them.
I've read and agree to the Code of Conduct.
I've searched for any related issues and avoided creating a duplicate issue.

Answer 1 · 2018-01-28T06:48:09.000Z

It would be very easy to put incoming events on a queue, then have a separate thread processing those events by taking them off the queue and calling emit there

Answer 2 · 2018-01-29T17:45:56.000Z

Pyee accepts coroutines as callbacks, but I’ve yet to figure out a way to build it in and seamlessly support Python2 and 3…

https://github.com/jfhbrook/pyee/blob/master/pyee/__init__.py#L48-L51

    For interoperation with asyncio, one can specify the scheduler and
    the event loop. The scheduler defaults to ``asyncio.ensure_future``,
    and the loop defaults to ``None``. When used with the default scheduler,
    this will schedule the coroutine onto asyncio's default loop.

Answer 3 · 2018-01-30T00:56:09.000Z

pyee doesn't support coroutines in python 2 because, afaik, python 2 doesn't have the concept of a coroutine as implemented in asyncio. I'm very much open to being proven wrong! I would love to be proven wrong.

In the meantime, pyee still supports synchronous hooks (much like node.js's eventemitter), which can then call asynchronous code. Prior to me adding support it would look like this in asyncio:

@ee.on('some_event')
def sync_hook(payload):
    f = asyncio.ensure_future(async_hook(payload), loop=event_loop)
    # f is an asyncio Future

or in twisted, more like:

@ee.on('some_event')
def sync_hook(payload):
    d = defer.ensureDeferred(async_hook(payload))
    # f is a twisted Deferred

Is this helpful? Or just telling you what you already know? :)

Answer 4 · 2018-01-30T01:03:19.000Z

It sounds like OP might be wanting to run this in a threaded context--in this case, you can call thread-related code in the sync callback--for instance, you can push those events onto a queue to communicate the events to another thread.

I'm very hesitant to explicitly add threading support to pyee, though if you want to try to convince me on a particular behavior change you're welcome to try.

Answer 5 · 2018-01-30T01:18:57.000Z

@jfhbrook the first example seems to be what I'll need for Python 3. Have any recs on how I would toggle between asyncio for Python3 or normal sync callback for Python 2? 🤔

Answer 6 · 2018-01-30T01:19:23.000Z

@Roach Check out Trollius

(writing a more detailed comment atm)

Answer 7 · 2018-01-30T01:26:51.000Z

Hi all, I'm working on the same project with @CameronAavik, we've actually ended up hacking our own support together.

As was said in the OP, the docs ask for prompt 200 responses to event hooks. If we're running code that takes more than 3sec in our event handlers, this obviously delays the response. While this can definitely be handled in the event handler itself, I personally believe that longer-running event handlers are common enough that this should be supported by the framework's default behaviour.

pyee calls the event listeners inside the call to emitter.emit(blah, ...), which means that these are called within the Flask request handler. It would be fairly trivial to change this behaviour so that the request handler instead puts events on a queue, and then the events themselves are emitted in another thread. I can put together a pull request including this if you'd like, it wouldn't be much effort.

The only behaviour change here is a dramatic speedup in returning from the response (as we're not waiting for synchronous execution), and handlers executing outside the Flask request context (I can include execution within a clone of the request context in the same PR if you'd like).

Regarding asyncio support, in our bot code we've retained the SlackServer flask subclass but replaced SlackEventEmitter with our own command parsing wrapper + async handler. It runs all async event handlers with asyncio directly, and all non-async handlers using a concurrent.futures.ThreadPoolExecutor. If this functionality looks good, I'd be happy to write it in an entirely separate PR.

Answer 8 · 2018-01-30T01:39:20.000Z

@jfhbrook the first example seems to be what I'll need for Python 3. Have any recs on how I would toggle between asyncio for Python3 or normal sync callback for Python 2? 🤔

The behavior toggles on whether the decorated function is a def or an async def, so the synchronous callback will Just Work in python 2. It will still, of course be blocking, meaning you would have to do something similar to what @TRManderson has suggested--that is, immediately pushing the event onto a queue.

Regarding asyncio support,

I mentioned this on twitter, but you should be able to port the SlackEventsAdapter internals to use https://github.com/twisted/klein; given the size of the flask implementation, I'd suggest using this.

I can put together a pull request including this if you'd like, it wouldn't be much effort.

Want to reiterate that I'm open to behavior switching for threaded code -- perhaps your function returns a concurrent.futures.Future and I detect on that? Anyway, open to a suggestion that doesn't break (much) existing code.

Answer 9 · 2018-01-30T01:39:58.000Z

@TRManderson I'd love to take a look at your solution and see if it points us in the right direction. My concern is that I want to fail back to something to works in Python 2, while supporting concurrency in Python 3. I'm exploring our options before I commit to either building something weird, importing a larger library like celery or twisted or forking it into a Python 3 limited version if it's not necessary.

@jfhbrook @TRManderson @CameronAavik I'm not super familiar with Python 3's concurrency, so I greatly appreciate your feedback. You folks are awesome 😎

Answer 10 · 2018-01-30T01:42:17.000Z

@Roach you may want to look into whether it's possible to make @inlineCallbacks decorated generators work with pyee, since that would get async/await style behavior from python 2

Answer 11 · 2018-01-30T02:14:35.000Z

I'm going to immediately recommend against looking at using celery for concurrency. That would require uses running a celery worker separately to the event handler process, but given the framework is just passing dicts you at least wouldn't have to deal with the ridiculous number of constraints Celery puts on function arguments.

Feel free to get in touch via me@trm.io if I can help in any way

Answer 12 · 2018-01-30T02:43:12.000Z

I'd say celery is in general a reasonable idea for use with fire-and-forget tasks and flask, though.

Answer 13 · 2018-01-31T03:52:10.000Z

fwiw, I've been digging into this problem further, and I think it actually makes sense for me to rewrite pyee to have a subclass of EE which uses an Executor and propagates exceptions from concurrent.futures Futures as the current EE implementation does for asyncio Futures (and using a similar pattern for asyncio and twisted support). Dunno when I'll get around to this, but this conversation is pushing me in that direction.

Answer 14 · 2018-04-17T17:26:25.000Z

Are we not over engineering the solution? The issue is the default flask built in server is a dev service and shouldn't be used in production.

Use something like tornado to start the server and you'll be able to handle parallel requests.

Code example

from tornado.wsgi import WSGIContainer
from tornado.httpserver import HTTPServer
from tornado.ioloop import IOLoop
from yourapplication import app

http_server = HTTPServer(WSGIContainer(app))
http_server.listen(5000)
IOLoop.instance().start()

Answer 15 · 2018-04-17T17:30:11.000Z

kinda, but also pyee could do better. jfhbrook/pyee#46 this is one possible way to improve things.

Answer 16 · 2018-10-14T08:18:37.000Z

https://python-rq.org/ makes this very easy if you're prepared to use Redis as the queue provider.

Answer 17 · 2018-10-15T03:11:57.000Z

There’s also this new project from Kenneth Reitz which looks pretty interesting

https://github.com/kennethreitz/responder

Answer 18 · 2018-10-15T05:58:17.000Z

gevent also makes this easy. You need to use their WSGI server instead of the native Flask server, and gevent.spawn to run a background thread. No broker, one process. Only downside is that it's not taking advantage of multi-core. For that I think you'd need to add https://gehrcke.de/gipc/ (I'm still experimenting with this).

Answer 19 · 2018-10-15T16:11:19.000Z

We got another Python dev on the team (Welcome, @RodneyU215 🎉), so we should be picking this back up soon.

Thanks a ton for all of your input, guys. it's been super helpful!

Answer 20 · 2018-10-15T16:43:11.000Z

cool, lmk if that changeset interests you and I can probably merge/release it

Answer 21 · 2018-11-28T01:03:29.000Z

Hello, is there any progress on this?
My bot often gets multiple events for the same message because of this (probably).

Answer 22 · 2018-12-14T02:03:55.000Z

As of now, there's no active development on this issue. We've been thinking about doing some sample code to try and show developers how they might do asynchronous event dispatch in their application.

It would be useful to know if this interests you (please 👍 this comment). Also, there are many reactive frameworks we could use, but we can't build a sample for all of them (examples mentioned above are gevent, responder, python-rq, twisted, etc). It would be really helpful to let us know if there are one or many of those that you are interested in.

Answer 23 · 2018-12-14T10:26:38.000Z

For those of you plagued by repeated request from Slack API because of this, I've added this to my endpoint to halt the flow. Not the best solution but it helps for now.

if 'X-Slack-Retry-Num' in request.headers:
    return 'OK'

Answer 24 · 2019-04-13T18:10:18.000Z

I wanted to give an update on this issue from my end.

I ended up merging jfhbrook/pyee#46 after reworking it a little. It's backwards compatible with v5 but raises a deprecation warning.

One complaint that people had in this thread was support for python 2. py2 support is starting to not be as big a deal as it used to be, but either way the twisted and executor based subclasses of BaseEventEmitter should work mostly fine in python 2.

I don't think it actually addresses OP's original issue though, at least not all the way. Because flask/gunicorn etc run each request in a separate thread, the ExecutorEventEmitter I whipped up won't work for this use case. Rewriting this hook to run on twisted or aiohttp would take care of this somewhat, but I feel like there's value in doing it with flask if only for the familiarity with other frameworks*.

I can say with some certainty that the "right way" to do async stuff given the constraint of flask is in fact either celery or RQ. I have a sketch for what that might look like at jfhbrook/pyee#49 and some day I might even implement it! But for now I think it's safe to use the BaseEventEmitter as this library already does and manually push the task onto your queue of choice.

* It could be cool if this library had the capability of creating an appropriate event emitter for multiple frameworks by detecting the routes library on setup and choosing an appropriate underlying event mitter to return. You could even allow passing a custom cls parameter and doing instance checks to see whether it's "compatible" with the detected framework. Could be cool!

Answer 25 · 2020-05-02T12:13:32.000Z

Hi guys,

I have built, in the last 2 days, my first slackbot and since I like async, I wanted to use it for the slackbot as well. I found that slack provides an Async version of the WebClient but, sadly, I realised that slackeventsapi does not support Async, so I created a package heavily based on slackeventsapi and adapted it to use Async.

To whomever would like to check it out, here it is:
https://github.com/tropxy/Asyncslackevent

I think it would be possible to use the tools I used and do a few modifications in the original slackeventsapi to allow it to use async methods. I wanted to build my bot as fast as possible so I didnt think yet in a good way to do this and that is why I am sharing my repo instead of doing a PR to this nice project.

Answer 26 · 2020-06-12T07:40:55.000Z

We're not planning to add async event dispatching support in this library.

As mentioned here, we've started working on Bolt for Python. Bolt will be supporting async in some ways.

Answer 27 · 2020-06-12T14:06:22.000Z

@seratch is there any where to see the work in progress for the bolt for python project?

Answer 28 · 2020-06-12T14:26:58.000Z

@psykzz
No, there isn't at this moment. We'll be making the initial alpha version public soon (hopefully by the end of this month).

We're aspiring to build the core part of Bolt as a framework-agnostic one. I'm already done with the initial sync version, which works with any WSGI, Flask, Django, and so on. The core part is not tightly coupled with those. If a developer wants to use it with Flask, the developer can enable an optional adapter module for it.

As for the async version of it, I'm still thinking about the reasonable design. In an ideal world, all a developer does is just define async functions (coroutine) as listeners. They won't be bothered by any compatibility issues with framework/runtime. That said, it's easy to say but it's a bit challenging.

If you already have great ideas on it, I'd love to know them.

Answer 29 · 2020-06-12T14:38:43.000Z

Decorators is an approach i've taken before myself,

@listen_to('who(?:s|\'s| is) on call(?:\?)?$')
def show_on_call(msg, *args, **kwargs):
    on_call_data = pagerduty.get_on_call()
    if on_call_data is None or isinstance(on_call_data, str):
        error = on_call_data or "No data, there could be an error, or a pagerduty token needs to be setup"
        return msg.reply(error)
    attachments = _generate_on_call_attachment(on_call_data)
    msg.raw_attachment(attachments)
    msg.react('robot_face')


@menu_callback('pagerduty-page-service')
def pagerduty_services(data):
    services = pagerduty.services
    return [{"value": key, "text": value['name']} for key, value in services.items()]


@interactive_callback('pagerduty-page-service')
def pagerduty_page_service(data):
    choice = '*error*'
    if data.get('actions', [{}])[0].get('type') == 'select':
        choice = data.get("actions", [{}])[0].get("selected_options", [{}])[0].get('value')
    else:
        choice = data.get("actions", [{}])[0].get("value", "")

    if choice == 'cancel':
        return "Request cancelled"

    username = data.get('user', {}).get('name')
    paged = pagerduty.page(username, choice)
    if paged:
        return "Page was sent for service: %s." % (choice,)
    return "Unable to find service: %s. No page was sent." % (choice,)

I didn't spend the time to create full class support for all the responses, but generally this worked really well for me, the one thing i missed was an easy way to create blocks.

(This was done before blocks were available)

Answer 30 · 2020-06-12T14:40:27.000Z

This obviously wasn't async, but ideally i dont care under the hood, I'd just want my app to be snappy.

Answer 31 · 2020-06-12T15:16:38.000Z

Thanks for sharing this! Bolt's approach is quite similar and it is very straightforward. The details may be changed later but the code using Bolt would be as below.

app = App(
    signing_secret = os.environ["SLACK_SIGNING_SECRET"],
    token= os.environ["SLACK_BOT_TOKEN"],
)

@app.shortcut("callback-id-here")
def handle_global_shortcut(ack, client, payload):
    ack()
    api_response = client.views_open( ... )

@app.view("view-id")
def view_submission(ack, payload, logger):
    ack()
    logger.info(payload)

@app.event("app_mention")
def event_test(ack, say):
    ack()
    say("What's up?")

if __name__ == '__main__':
    app.start(3000) # POST http://localhost:3000/slack/events

Refer to https://slack.dev/bolt-js/ to know the basic concept of Bolt. I have been trying to minimize the difference from the JavaScript version of Bolt. But, more importantly, we want to realize the same philosophy in a more pythonic way.

If a developer goes with a completely async (non-blocking) app design, it's inevitable to use async/await for all. Strictly speaking, it's not allowed to have even a single blocking operation. I don't think such use cases are in the mainstream as of today. It's almost unrealistic to avoid using libraries that may block internally.

Anyways, we'll be finalizing the sync version first. Then, we'll come up with the async version of it. I'll let you know here once I publish the initial alpha version.

Answer 32 · 2020-09-07T10:56:40.000Z

I found Async version of this for FastAPI it's called slackers https://github.com/uhavin/slackers
It aims to send response to slack asap and then process the payload.

Answer 33 · 2020-09-07T11:11:38.000Z

@1oglop1 Thanks for sharing the library!

I'll let you know here once I publish the initial alpha version.

Oops, I haven't posted the message here. Bolt for Python's alpha release is already available.

You can find many samples (the SDK supports most of the major Python frameworks already) here: https://github.com/slackapi/bolt-python/tree/main/samples
If you're interested in the new library, please try it out and give us feedback 🙇

Answer 34 · 2020-09-12T14:08:28.000Z

@seratch does slack maintain and support this library? is slack dropping support for this library in favor of https://github.com/slackapi/bolt-python?

Answer 35 · 2020-09-12T14:09:24.000Z

@psykzz how do you access the request headers within the event handler function?
I noticed that there is a PR from 2018 to add this feature, but it looks like it is currently not possible.

Answer 36 · 2020-09-12T23:22:19.000Z

@Lobosque We don't stop maintaining this library. That said, slackeventsapi supports only Events API and we don't have any plans to add other features to this library. So, if you start a new Slack app project, we recommend Bolt for Python in general.

Answer 37 · 2021-04-29T22:03:36.000Z

Bolt for Python v1 was released in November 2020 and we have been actively updating the library. Bolt resolves this issue with not only Flask but also many major web frameworks in Python. Check the list of its adapters here: https://github.com/slackapi/bolt-python/tree/main/slack_bolt/adapter

As I mentioned above, we don't have plans to resolve this issue in this library. Let me close this issue and I hope you will like Bolt too 🙇

Description

What type of issue is this? (place an x in one of the [ ])

Requirements

What type of issue is this? (place an `x` in one of the `[ ]`)