Question on output
mloning opened this issue · 2 comments
mloning commented
Hi Hassan,
I've used your code for creating a critical difference diagram and got the following output. Am I misunderstanding something or should that extra line between M4 winner and M4 runner-up not be there?
hfawaz commented
Hi Markus,
Thanks for sharing your thoughts.
Indeed if we were applying Nemenyi test, we should not have such overlapping segments.
But when using Wilcoxon signed rank test for the post-hoc analysis, this sort of segments could appear since the Wilcoxon method does not take into consideration the average rank.
In this case, your diagram means that:
- RF-s and M4 runner-up are not significantly different.
- M4 winner and M4 runner-up are not significantly different.
- RF-s and M4 winner are significantly different.
This sort of behavior is described clearly in this paper, section "HESCA Accuracies on Test Data".
Hope this helps.