apluslms/mooc-grader

Float comparison in questionnaire freetext questions should use a configurable accepted range

markkuriekkinen opened this issue · 0 comments

The grading of questionnaire freetext questions may use float comparison which compares the submitted value as a floating-point number to the model solution. The comparison accepts values that are "close enough" to the model solution. The relative tolerance currently used in the comparison is hardcoded to 2%. The accepted tolerance should be configurable because 2% is not sensible in all cases. With the hardcoded value, the float comparison has been basically unusable on courses in which 2% includes far too wide a range of solutions.

The float comparison is documented here:

The current implementation uses the (Python built-in) math.isclose() function. It already includes suitable parameters for this issue.
https://docs.python.org/3/library/math.html#math.isclose

Implementation in the mooc-grader:

return math.isclose(float(val), float(cmp), rel_tol=0.02)

The assignment configuration should be able to:

  • set the relative tolerance as they wish, e.g., 0.05. (math.isclose(rel_tol=...))
  • set the minimum absolute tolerance (math.isclose(abs_tol=...))
  • the options are optional: both may be set or only one or none

If the assignment configuration does not set any tolerance parameters, we could use the defaults from math.isclose(). Its defaults are different than the currently used hardcoded rel_tol=0.02, but those defaults seem more useful in a wider range of use cases.

In RST source, the tolerance parameters should be added as new options to the freetext directive in a-plus-rst-tools.
For example,

.. freetext:: 5 float
  :float-rel-tol: 0.05
  :float-abs-tol: 0.001

https://github.com/apluslms/a-plus-rst-tools/blob/85ec08fc22aef4116a84850eacea7de09bc45976/directives/questionnaire.py#L576

Internal ticket:
https://rt.cs.aalto.fi/Ticket/Display.html?id=22687