Increasing number of threads also polls Core more often

Question

Increasing number of threads also polls Core more often

Opened this issue 11 years ago · 4 comments

If you set numberOfThreads = 4 in a stage, not only will you have 4 worker threads for processing, you will also have 4 threads actively polling the database. Is this intended behavior?

Answer 1 · 2013-12-27T14:36:49.000Z

Sounds like a problem with how Core creates StageRunners for each remote stage. Hm, or it might be due to the stages polling at the same time and a thread being spawned from their HTTP request? I'm not familiar with the logic there, need to check what's going on to be sure.

Answer 2 · 2013-12-27T14:56:20.000Z

It's due to numberOfThreads controlling how many Stage instances to create, and each instance does its own polling. I haven't changed this behavior in my refactoring since I didn't know whether it was intended or not.

Answer 3 · 2013-12-27T14:56:49.000Z

By "active polling the database", I mean "polling the core".

Answer 4 · 2013-12-27T14:58:02.000Z

So whether it will poll the database or not, I don't know. May not be worth looking at, depending on whether my continued work moving over to a queue-based system bears any fruit 😄