The answers of the questionnaires are revealed
jaguarfi opened this issue · 3 comments
This happens in a very specific case. If we set submissions equal to 1 and set reveal-model-at-max-submissions equals to True, students can get the answers before submitting the questionnaire.
The student can submit the questionnaire without checking any of the answers, then Aplus will provide the students with the feedback that includes the correct answers, then students can select the correct answers and get all the points.
Test Case
.. questionnaire:: Testing 20
:title: Getting answers
:submissions: 1
:reveal-model-at-max-submissions: True
.. pick-one:: 10
:required:
**Modifiers used in this question:** ``required``.
When :math:`(x + 1)^3 = 27`, what is :math:`x`?
a. 9
*b. 2
c. 3
a § Not quite. Remember the cube root.
b § Correct!
c § Rather close. Remember that you can add or subtract the same number to both sides of the equation.
.. pick-any:: 10
:required:
**Modifiers used in this question:** ``required``.
When :math:`(x + 1)^2 = 16`, what is :math:`x`?
a. 4
*b. an integer
*c. 3
d. an irrational number
e. -3
*f. -5
a § Rather close. Remember that you can add or subtract the same number to both sides of the equation.
b § Correct!
c § Correct!
d § No. This equation has a nice and easy solution.
e § Remember that :math:`x^2 = q \leftrightarrow x = \pm \sqrt{q}`
f § Correct!Check the following video
bug.mp4
It is important to highlight that the first submission was rejected. It gets the status "rejected" in A+ and it does not count as a submission. This happens when the grading fails due to validation, basically when the required fields are left empty.
This must be fixed in such a way that the feedback of rejected submissions never reveals anything about the correctness of the submission. It must only explain why the validation has failed, that is, which required fields were empty. Rejected submission feedback must never reveal anything about the correctness!
This is really a bug in the MOOC-Grader, not a-plus-rst-tools. I will now transfer the issue to the correct repo.
Based on the video, the data-aplus-once checkmarks have been rendered into the rejected feedback. This must be fixed.
I actually can't reproduce this issue directly because the browser refuses to submit the form without all required fields filled in. Both Firefox and Chrome behave like this.
However, if i use inspect element to remove the required attributes from the form elements, I can submit the form and get the correct answers like in the video. That means the system can still be exploited if the user knows about this issue (e.g. by reading this comment). I'll go ahead and fix this.