Client receive response with delay/timeout when there are high concurrent request to a grain

[zeinali-ali]

we have a non-reentrance grain which receive 10 to 20 request at same time from same client.
while each request on grain side took at max 300ms to complete it took 20000ms to 30000ms(timeout) for client to receive response for all request(s).

It seems when the number of concurrent request from same client to a grain is high, the client receive all response at same time but with a high latency.
Does Orleans trying to batch response(s)?

[ReubenBond]

Orleans doesn't batch responses: they are sent immediately. Could you provide more info about the workload?

[zeinali-ali]

Here are some of my code:

I have this grain

public class ContactAggregateActor : AggregateActor<ContactState, ContactEvent>, IContactAggregateActor
{
    public async Task<List<Contact>> ImportPhoneBook(ImportPhoneBookCommand command)
    {
       //some other codes
          .
          .
          .
        await EmitAsync(events);

        //hole method took around 300ms to execute
        return [];
    }
}

Here my client code which would be called concurrently for same grain-id (like 20 request at same time).

public async Task MyRequest()
{
    ///some other code
       .
       .
       .
    var reponse = await contactGrain.ImportPhoneBook(command);
}

I expect client receive first response in around 350ms
the second in around 700ms
the third in around 1050ms
the fourth in around 1400ms

but what actually happen is, all request(s) receive response in around 20000ms or more (get higher when number of concurrent request is more).

using Orleans 8.1.0

[ReubenBond]

That's strange. How are you measuring those times? Do you have logs you could share?

[zeinali-ali]

I'm Sorry, I did a mistake in my measurement.
there was some other call to that grain in my workload which all those call sat in grain queue before the second call for each request and that cause all request(s) took too much to get done.