rois-codh/kmnist

Kuzushiji with SVM(support vector machine)

TomZephire opened this issue · 6 comments

TomZephire commented

Kuzushiji_by_SVM.zip

I believe inferior counterpart is necessary to feature sophisticated DNNs more.

SVMs with 'RBF' Kernel with chosen parameters C and gamma by a random search give the following results.

Accuracy:
KMNIST 92.82% (test data), 99.98% (training data)
K49 85.61% (test data), 97.94% (training data) <--correct value(class averaged)

hardmaru commented
TomZephire commented

Thank you for your advise. It is my first posting in GitHub. I have to learn and figure out how to do it...

tkasasagi commented

Thank you for your analysis. :)

mxbi commented

Hi @TomZephire, thanks for your analysis! I am running it now on the original MNIST dataset to get a number for that, and I will add it to our leaderboard :)

One thing I noticed is that you calculate K49 accuracy incorrectly - as we suggest using balanced accuracy (accuracy averaged over classes) to combat the inbalanced dataset. I notice you mention it can take over a day to run the SVM... So if you still have it could you send test predictions for K49?

Thanks

TomZephire commented

Thank you for your comment.
Fortunately, I saved the predictions in pickles. So, I could recalculate the accuracy quickly. See the original message and code in it.

mxbi commented

Added these! (apologies for the long delay)
It would be appreciated if you could rerun on the new version of the dataset with minor image processing fixes, as it can lead to slightly different results