Hydra 0.20.0 not always finishing it's run
Opened this issue · 4 comments
Hey guys,
Maybe this has been solved (which I'm hoping) but I was setting up hydra as our CI runner and testing locally on my laptop. I find that from time to time and much more often when there are test failures that the runners never seem to report back at the very end of the test run, generally one or two don't report back. The bigger issue with this in mind is that the test run never finishes running and gets stuck waiting on the all of the files to get run while the Master reports that it has no more files to send to any workers. As far as I can tell the runners are hitting a seg fault or anything (running on rvm which is awesome but I've had issues with 1.8.7 on certain machines). As an added note I'm using the multi db listener modified slightly from the Weplay example. Thanks for your guys' time up front.
I have noticed this as well. Certain exceptions and crashes are not getting caught by hydra, which kills the runner.
Hydra's worker is not properly detecting a runner crash and removing it from its pool, so it doesn't finish.
This isn't a priority when you're just testing, because you see the crash and cancel the process. But I can see how this could be an issue in CI.
I will look into this, but please give it a shot yourself. I'll merge your changes pretty quick.
If you could create a test (or a sample project) that consistently crashes hydra, that would be amazing and I could probably patch it pretty fast.
Thanks,
Nick
Thanks for the quick response Nick, I'm glad to hear this is a known issue because now we get to contribute back some work :-). Incidentally I'd like to talk more about contributing since I've been tracking this project for a while and decided to get some weight behind it after the Windy City talk. With regards to the patch do you still accept patches for 0.20.0 (we're stuck on rspec 1.9 for the moment)? Bonus followup question will future Hydra versions support rspec 1 or is it just rspec2 from here on out?
I'd accept a patch on 0.20.0 sure. We'd just branch on it and make patches. I'd also want to add it on the current version and release a patch.
As for version support, I need to rewrite the runner to use modules based on available versions. So, I'd detect if you're on rspec or rspec 2 and handle them appropriately. I haven't undertaken that project because it's pretty big :-)
This would involve making special test runner classes for each version (Hydra::Runner:Rspec1, Hydra:Runner:Rspec2, etc) then in Hydra::Runner, detect the version and invoke the runner.
Once that is decoupled, we could support all versions of the popular gems on master.
Good call, I'll look into it. Got some free time coming up and Chicago is cold so have computer will hack.