Float comparison in questionnaire freetext questions should use a configurable accepted range
markkuriekkinen opened this issue · 0 comments
The grading of questionnaire freetext questions may use float comparison which compares the submitted value as a floating-point number to the model solution. The comparison accepts values that are "close enough" to the model solution. The relative tolerance currently used in the comparison is hardcoded to 2%. The accepted tolerance should be configurable because 2% is not sensible in all cases. With the hardcoded value, the float comparison has been basically unusable on courses in which 2% includes far too wide a range of solutions.
The float
comparison is documented here:
- https://github.com/apluslms/a-plus-rst-tools/blob/master/README.md#2-graded-questionnaire
- https://plus.cs.aalto.fi/aplus-manual/master/questionnaires/questionnaires/#freetext-questions
The current implementation uses the (Python built-in) math.isclose()
function. It already includes suitable parameters for this issue.
https://docs.python.org/3/library/math.html#math.isclose
Implementation in the mooc-grader:
mooc-grader/access/types/forms.py
Line 471 in e6fdfe8
The assignment configuration should be able to:
- set the relative tolerance as they wish, e.g.,
0.05
. (math.isclose(rel_tol=...)
) - set the minimum absolute tolerance (
math.isclose(abs_tol=...)
) - the options are optional: both may be set or only one or none
If the assignment configuration does not set any tolerance parameters, we could use the defaults from math.isclose()
. Its defaults are different than the currently used hardcoded rel_tol=0.02
, but those defaults seem more useful in a wider range of use cases.
In RST source, the tolerance parameters should be added as new options to the freetext
directive in a-plus-rst-tools.
For example,
.. freetext:: 5 float
:float-rel-tol: 0.05
:float-abs-tol: 0.001
Internal ticket:
https://rt.cs.aalto.fi/Ticket/Display.html?id=22687