Add support of async client in sdk
kaustuk opened this issue · 6 comments
Currently, the version of SDK doesn't have the support for creating the async client. It would be great to have this support which can help in optimizing the code since all calls are IO-bound.
PS: I would like to contribute a feature.
@kaustuk really cool idea!
How would you solve it? threads? how to retrieve and save responses?
I was also wondering if that would be possible with aiohttp or similar. However, we noticed that Label Studio itself is the biggest bottleneck for us. Our Label Studio instance does not seem to handle any requests in parallel.
If you use docker-compose from the latest LS version, it should use uswgi by default, so LS shouldn't be a bottleneck anymore.
What exact SDK calls do you want to call asynchronously?
@makseq You're correct. Apparently, I did my benchmark on an older version of Label Studio 3 weeks ago. I just retried with the latest version from GitHub using uWSGI and the results show a speedup with parallel requests as expected. I attached my benchmark results below.
As sou can see with 1 connection I get about 20 req/sec (see second last line). If I increase that to 2 connections I get about 43 req/sec which is a speedup by about 2x as expected.
Great findings!
However, it's still unclear for me how we can turn sdk to async and why do we need it exactly?
In our case, we are requesting meta data for several datasets to show them on a dashboard. That is like 10 requests to /api/projects/X which are handled sequentially right now. Therefore, loading this data takes ~500ms. If the SDK would support asynchronous calls we could get the data for all of them concurrently, reducing the time to 50ms (in theory).
To achieve this without async support from the SDK we would have to reimplement the specific endpoints with async or use something like multitreading/processing.