results of the quiz

[dmiser]

Colleagues,
How can we interpret results of users?
For instance, we have English Placement Test with levels
1 - Elementary
2 - Pre-Intermediate
3 - Intermediate
4 - Upper-Intermediate
5 - Advanced
Every level has 20 questions: adpq_1, adpq_2, adpq_3, adpq_4 and adpq_5
After taking the quiz user gets 4.50 (87.55 %)
Which level it fits?
How can we grade results?

[adamfranco]

Hi @dmiser , for the theory behind this adaptive testing system and how to effectively utilize it I recommend that you read John Linacre's "Computer-Adaptive Testing: A Methodology Whose Time Has Come." MESA Memorandum No. 69 (2000). This adaptive testing module implements the algorithm discussed in that paper and detailed at: "Practical Adaptive Testing CAT Algorithm" by B.D. Wright published in Rasch Measurement Transactions, 1988, 2:2 p.24. In the article above Linacre goes through the theory of adaptive testing and details how to construct item banks and interpret results.

While I am a developer of this Moodle module and can verify that it is an implementation of Wright's algorithm (to the best of our ability), I must defer to the literature on questions related to the interpretation of results, constructing and weighting of item banks, and validations of the testing system.

All that said, my own [implementation-focused] understanding of the algorithm is that a measure of 4.50 (to use your example) would indicate that the test taker is estimated to be equally likely to provide a right or wrong answer to a theoretical question at level 4.50, whether or not such a question actually exists in the item bank.

For our own use of an adaptive placement test at Middlebury, our program validated their test in two ways: (1) by having instructors take the test answering questions as if they were a student of a certain knowledge level and (2) by having a number of students well known to the instructors take the adaptive test, allowing the instructors to verify that the test reported score for these students that matched their known proficiency. The results of this validation exercise were sufficient for our program to rely on their adaptive test and its item-bank, though I do not have a report on their analysis.

Best,
Adam

[dmiser]

Thank you @adamfranco , but it's not clear enough. We need some life examples. Theory is only theory.