Known issue: Not 7! koan not detecting failures on some code path during some runs
jletroui opened this issue · 1 comments
Given the random nature of Not 7!, there might be some runs where we only detect a single path of the student's code, and validate the code, even though some unexecuted path in the game is still not correct.
Consequence: the student think their code is fine, when it is not complete. Upon next runs, it will show assertion errors on what the student might have thought was a 100% correct work.
This happens rather rarely and is minor. Fix would be to control the random seed, but this would make the dice thrown always returning the same results, which is a bit less interesting.
Fixed by allowing to specify some seed in certain tests. The seed is not mandatory, so for tests which assertions are not sensitive to randomness, we can let true random outcomes happen, thereby mitigating the above mentionned issue of fixing seeds.