Handle concurrent calls
cdevienne opened this issue · 1 comments
cdevienne commented
The current implementation of nrpc serializes all the calls server-side for each service.
Aside from the lack of concurrency, it also means that a call may be handled even if the client already had a timeout.
A solution to this would solve the following problems:
- A server should be able to authorize parallel handling of requests, especially with the streamed replies that can be very long.
- When parallel handling is enabled, a limit should be set on how many parallel calls are permitted
- If a call cannot be processed soon enough (ie, too many calls already processing and no room for a new one after a given timeout), the server should gracefully fail and inform the client that the server is overloaded (so it can try again later).
- Streamed replies methods should start the keep-alive as soon as the request arrives, and send an error if the handling cannot be started within a given time.
- In any case, if the number of pending requests is too big, an error should be returned immediately
I'm starting to work on a solution with those elements in mind.
If any idea pops out, please share it!