Performance overhead with a large number of services
claustres opened this issue · 3 comments
When reaching a large number of services (> 100) you can experience some performance issues like CPU load for message processing or network bandwidth consumption. Now we mitigate this by increasing the different intervals (hello/check messages) in the cote configuration. However this might not work with a large number of services. Refer to related cote issue.
We should probably manage all services methods using a single or limited set of requesters/responders and similarly service events using a single or limited set of publishers/subscribers.
If @dashersw pulls my PR to node-discover in once its merged at wankdanker/node-discover#39 - this should at least greatly help reduce CPU cost in cote.
@aikar have you run a benchmark on how much this saves performance? If we have concrete results, I'd be happy to review it and merge it to my branch, without waiting for the merge to the original repo.
benchmarks no, but avoiding synchronous cpu intensive encryption and synchronous cpu intensive serialization logic blocking the node.js event loop is overall pretty clear it will be an improvement.
@claustres identified this code is the problem here: https://cotejs.slack.com/archives/C06MER4NT/p1570175857015100
So this PR simply skips even running this code all together.