This Shiny visualization might give us a clue! Here binary classification is used for simple illustration, you may toggle classifier, or sampling method and see how 1)decision boundary and 2) prediction measurements are changed.
Have fun!
-
add penalty, different to each class (e.g. consider two scenarios - fraud detection or cancer identification)
-
consider add more classes, make it interactive