Rating deployed instances

[hgeorgsch]

CAPQuiz rates questions. For question types such as STACK the question is parameterised, and a question is really a class of random instances. This has a couple of potentially negative consequences.

1. All instances of the same question have the same rating, even if they may have different difficulty.
2. When parameterised questions are used together with unique questions, instances of the parameterised question come up less frequently than instances of the unique question.

There is no obvious and clear-cut solution to this problem, and every solution is likely to have negative side-effects, but alternatives should be identified and discussed. In particular,

1. is it possible to rate instances instead of questions? Could this be made optional when setting up the question set?
2. is it possible to take the number of instances into account in the matchmaking routine, so that questions with many instances come up with higher probability?

[hgeorgsch]

This will take some research and won't easily be fixed.
A feasibility analysis would be useful though.

[hgeorgsch]

This only affects a narrow use case. With CAPQuiz we normally want a large number of variants, which creates performance issues with STACK. We have therefore resorted to instantiating questions on the fly, with nothing pregenerated, and in this case instances cannot be graded.

Therefore, we won't fix.