ndo2db doesn't fork more than 2 process

Question

ndo2db doesn't fork more than 2 process

ranjithkodumbu opened this issue 8 years ago · 7 comments

Hello Team,

I am facing the issue with ndo2db processing, it doesn't write fast enough. I am currently running 222000 monitoring services, and it creates huge backlog. Looks like ndo2db doesn't fork more than 2 process. If we can increase the process dynamically that will clear huge backlog. Something like mod-gearman does, where in we can increase number of workers according to the load of the monitoring service.

It would be great help if you can look into this.

I would appreciate your efforts that team have contributed for developing such a great application.

Thanks,
Ranjith Kumar R

Answer 1 · 2017-06-16T20:52:51.000Z

This is actually a great idea. I'm going to assign it to myself with an enhancement label and review the work that would need to be done, and then I'll assign a milestone once we have that figured out.

Answer 2 · 2017-06-17T09:05:15.000Z

Great. Thanks a lot Bryan for considering my idea. On Jun 17, 2017 02:23, "Bryan Heden" <notifications@github.com> wrote: This is actually a great idea. I'm going to assign it to myself with an enhancement label and review the work that would need to be done, and then I'll assign a milestone once we have that figured out. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#38 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AZrB8s_A9tnfus5HRO8ifkNzRcXvJ2oVks5sEuslgaJpZM4N1GMr> . On Jun 17, 2017 02:23, "Bryan Heden" <notifications@github.com> wrote: This is actually a great idea. I'm going to assign it to myself with an enhancement label and review the work that would need to be done, and then I'll assign a milestone once we have that figured out. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#38 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AZrB8s_A9tnfus5HRO8ifkNzRcXvJ2oVks5sEuslgaJpZM4N1GMr> . On Jun 17, 2017 02:23, "Bryan Heden" <notifications@github.com> wrote: This is actually a great idea. I'm going to assign it to myself with an enhancement label and review the work that would need to be done, and then I'll assign a milestone once we have that figured out. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#38 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AZrB8s_A9tnfus5HRO8ifkNzRcXvJ2oVks5sEuslgaJpZM4N1GMr> . On Jun 17, 2017 02:23, "Bryan Heden" <notifications@github.com> wrote: This is actually a great idea. I'm going to assign it to myself with an enhancement label and review the work that would need to be done, and then I'll assign a milestone once we have that figured out. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#38 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AZrB8s_A9tnfus5HRO8ifkNzRcXvJ2oVks5sEuslgaJpZM4N1GMr> .

Answer 3 · 2017-08-07T13:14:37.000Z

I am curious about the progress, could you please update me the status.

Answer 4 · 2018-06-29T23:00:23.000Z

Sorry it's been so long on an update @ranjithkodumbu . I actually got a prototype working for this back in March or so. I'm likely going to work on getting a 2.1.4 bugfix release out, and then start integrating the prototype into ndo3. NDO3 (and subsequent bugfix releases) will likely be the last major release of NDO (as we plan on incorporating it directly into Core5). But NDO3 should have enough performance improvements (because of the true multithreading we'll be bringing) to keep people who don't want to upgrade to Core5 happy for a long while.

Answer 5 · 2018-06-29T23:04:20.000Z

Hi Bryan, Thank you for the update.

…

On Sat, Jun 30, 2018, 04:30 Bryan Heden ***@***.***> wrote: Sorry it's been so long on an update @ranjithkodumbu <https://github.com/ranjithkodumbu> . I actually got a prototype working for this back in March or so. I'm likely going to work on getting a 2.1.4 bugfix release out, and then start integrating the prototype into ndo3. NDO3 (and subsequent bugfix releases) will likely be the last major release of NDO (as we plan on incorporating it directly into Core5). But NDO3 should have enough performance improvements (because of the true multithreading we'll be bringing) to keep people who don't want to upgrade to Core5 happy for a long while. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#38 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AZrB8gwvuY1pwrObYUKdiL3s2BdMs83cks5uBrGJgaJpZM4N1GMr> .

Answer 6 · 2019-09-17T01:32:41.000Z

We actually went a different direction with this, and now there is no forking at all (checkout ndo-3 branch). If you're interested in helping us with testing, please let me know so we can facilitate that. Thanks!

Answer 7 · 2020-06-29T21:25:42.000Z

Sorry hedenface, I couldn't find ndo-3 branch. Could you please publish it.