martijndwars/grading-tools

Incorrect use of passed languages

Opened this issue · 0 comments

The TestResult#finishedLanguage is called for every finished language. If the test detected the language, variable passed is incremented by 1. In the report, this variable is used in the following sentence:

of your valid tests detect erroneous language definitions

However, the finishedLanguage method is called for the correct implementation as well. If I have 1 test which passes the correct implementation but does not catch an erroneous language I get:

0 of your valid tests detect 1 erroneous language definitions

which does not make sense.

Proposed solution: check if description == null or points == 0 in #finishedLanguage as an indicator that this is a 'pass' for the reference implementation and should not increment the passed variable. I'm not sure how this will affect the rest of the system..