Asynchronous Query Support
lab176 opened this issue · 4 comments
Feature Request for Async Query Support
Hi!
I have a use case that would greatly benefit from async query support, perhaps added as additional functionality to the connection interface.
For example, introducing a function that will kick off a query and return an object containing QueryExecutionId, then another function that may use that object to poll for query completion, and another for cancelling that query, etc.
If any thought has been given to this I'd love to know.
Thanks!
Thank @lab176 for the great question.
You can refer to https://github.com/uber/athenadriver/blob/master/examples/querycancel.go for how a query can be canceled in database/sql framework. The poll for query completion has been handled by athenadriver so you don't have to write the code yourself. You can just pass the cancel function to the other goroutine/thread and cancel the query there.
Hi @henrywu2019 ,
Thanks for the response!
I need to clarify our use case, as you have provided a solution for what I asked but what I asked was insufficient.
Spinning up a go routine and cancelling the request via context is insufficient for our use case.
We have an app that will send an http request to an intermediate component, called query-gateway, that handles requests to Athena by leveraging this athenadriver. Because we often observe Athena query times of a few minutes, we'd like to request query-gateway to initiate the Athena query then simply return to the app, and enable the app to poll any query-gateway instance for that query's completion + results. This saves us from having to maintain a connection to a single query-gateway instance for the length of the Athena query or, alternatively, maintain state.
Ideally, this could be an interface containing the following: a function that will kick off a query and return an object containing QueryExecutionId, another function that may use that object to poll for query completion, and another for cancelling that query, etc.
Huge thanks, @lab176. The pseudo commands could be used to support Asynchronous Query you described above: https://github.com/uber/athenadriver#pseudo-commands Please refer to the sample code in the README and let me know if there is any issue/question.
@henrywu2019 Amazing turnaround! Thank you!