InvokeCallbackAsync latency
meisamhasani opened this issue · 8 comments
Hello, good time
I have a problem , when i call a third-party in timer of Grain
I recive latency a lot is created to fix this latency
What way do you suggest?
Are you able to provide more information? What do your callbacks do, which version of Orleans, etc
my version is 8.2.0.
my timer is :
this.RegisterGrainTimer(SendMessage, this.State, new()
{
DueTime = TimeSpan.FromSeconds(2),
Period = TimeSpan.FromSeconds(9),
Interleave = true,
KeepAlive = true
});
and SendMessage method call this :
var t = await telegramBot.SendTextMessage("", "text message");
What is the latency of the Telegram API call?
Telegram delay is as much as the picture shows
However, I am looking for a solution to remove this delay from the cluster
Do we have a solution?
If the number of these timers and delays is thousands, won't these delays cause problems for my cluster?
These delays shouldn't cause any issues for your application or Orleans as long as the call is asynchronous (which it appears to be). Anything we do here to reduce the API latency displayed on the graph would just be hiding it, which is likely not useful.
Your system appears to be working well :)
@meisamhasani I want to clarify: if you have thousands of timers on one grain, that might end up causing some issue for that particular grain. Your timers are interleaved, so they will run concurrently - which is good for end-to-end latency and allows a single grain to scale further. Each grain is single-threaded, they never use more than one CPU core at a time. If you have a very large number of timers, it's possible to have more timers firing concurrently than a single thread can keep up with.
I doubt this will be an issue for you: you would likely need thousands of timers on a single grain to encounter an issue here, assuming your CPU's single-threaded performance is decent. Still, I recommend not having thousands of timers on a single grain. Instead, use more grains. You are likely doing this already, but I want to be explicit.
I only have one timer per grain.
I did not see any problem.
orleans dashboard showd me latency.
I wanted to make sure that this delay does not cause a disturbance to the cluster or grain.
thank you @ReubenBond
Happy to help any time, @meisamhasani