Why do you need to separate the last batch of the output

[vkaul11]

https://github.com/hsiehjackson/RULER/blob/main/scripts/pred/call_api.py#L270 Why do you need to separate the last batch of output here and why do you need the threads above?

[hsiehjackson]

why do you need the threads above?

We can send multiple requests to our server in parallel. In server side, we may deal with multiple requests at the same time depends on your framework.

why do you need to separate the last batch of output here?

We may still remain some samples because we have this condition to join our threads.