Questions about Clothing1M
Newbeeer opened this issue · 3 comments
Hi, I wonder about what the auxiliary noise transition is in your Clothing1M experiments (the paper you cite doesn't explicitly mention it). Another question is do you train the neural network in (K+1) class setting in LCCN*, then estimate the confusion matrix of (K+1) by K?
Thank you.
You mean the work in my Journal extension? The auxiliary noise transition for warming-up is the transition matrix manually provided by [1] (see their Figure 6).
In LCCN*, you can train CE directly on Clothing1M (add one outlier dimension for reasoning although there are no samples to supervise it), and then estimate a noise transition for warming-up like the M-step in [2].
A gentle tip: On real-world dataset, one interesting thing is the noise transition can occur diagonal degeneration during training due to the complex noise. I think this phenomena is because of the decoupling bias when two submodules (one strong NN and one weak simplex module) sequentially cooperate to reason noisy data. To solve such a problem, you can decrease the update frequency of noise transition to force its reasoning in LCCN, e.g., update the noise transition at the fixed epoch point. Another way is applying structure regularization on the noise transition, e.g., [3].
[1] T. Xiao, T. Xia, Y. Yang, C. Huang, and X. Wang, “Learning from massive noisy labeled data for image classification,” in CVPR, 2015.
[2] J. Goldberger and E. Ben-Reuven, “Training deep neuralnetworks using a noise adaptation layer,” ICLR, 2017.
[3] B. Han, J. Yao, G. Niu, M. Zhou, I. W. Tsang, Y. Zhang, and M. Sugiyama, “Masking: A new perspective of noisy supervision,” in NIPS, 2018.
I see. Thank you. But I found that on Clothing1M dataset, when I use the pre-train the model by CE, and estimate the initial noise transition matrix, then immediately test the accuracy before any further fine-tuning, there is only very little improvement in accuracy. What's your opinion on that?
I am not sure. I upload the Clothing1M code into github as a reference for you (see the bottom in README.md), which is similar to the CIFAR code. But I have not organized it or removed some intermediate validations. You may need to slightly modify the parameters yourself in LCCN. I will systematically format the code and the parameter after the Journal is accepted.