Failed regression test for Coevolution
SigmaX opened this issue · 2 comments
As off commit 09a2aee (Sean's work toward fixing elitism and NSGA-II), the behavior of app/coevolve2/coevolve2.params and app/gui/coevolve2.params differs from ECJ 24 in a statistically significant way.
Actually, this probably isn't the result of recent modifications to the codebase.
In the last regression test I did (before Sean's edits), the p-value on both of these apps was around 0.03 (i.e. a 3% chance that the deviation from ECJ 24 is due to chance).
By itself, that would have counted as a failed test. I treated it as passing because our suite runs 80 tests simultaneously, and it didn't fail after the Bonferroni correction.
Closing because the regression tests don't seem to be working properly anyway.