thuml/CDAN

SVHN -> MNIST Accuracy

jongchyisu opened this issue · 3 comments

jongchyisu commented

Hi, I am trying to run the code for SVHN to MNIST.
However, with CDAN I only got ~84% accuracy (88.5% reported in the paper). Is there anything I should tune? or is the variance of accuracy very high?
Also, there is svhn_balanced.txt, how is it generated from svhn.txt? The class distributions are not uniform in both versions. Does the balanced version makes difference?
Thank you!

caozhangjie commented

svhn balanced is just balanced the number data in different classes. For classes with less data, we duplicate the data in that class to make it compatible with other classes.

jongchyisu commented

Thanks for your reply! But the class distribution is still not uniform in the balanced version? Also it seems like the performance has large variance, how did you deal with this issue?

jongchyisu commented

Ah I got it, you duplicate the class for 2 or 3 times. Seem like this matters a lot, with this version I got 87.86% (reported 88.5% ), and using the original svhn I got 83.46% accuracy.

  • 👍1