about what accuracy should be record
Closed this issue · 5 comments
Hellow!
I am now doing my ERM experiment. I would like to ask what an ideal curve should look like after the experiment (whether stable or both are acceptable). As for the results presented in the paper, should the top one be from a stable curve? Or if it's acceptable to choose the top one from an unstable curve, which one should I record as the result in the paper (point 1 or 2 or something else)? There should be two figures below if everything is okay.
Best wishes for your kind help!
Hi YuGuilliman,
An ideal curve should look like figure 1. In our paper, we use the last epoch's accuracy as the model's performance. To stabilize your training, you could consider reducing the lambda_gan parameter, which will decrease the impact of adversarial learning on the overall model.
Best,
Zihao
Thank you for your help!
Thank you for your kind help again!
I would like to ask another question that does your tgt_acc curve always get higher when both the epoch_num and the src_acc increase, which means the tgt_acc curve does not fall down like the curve in figure 1(stable curve)?
No, it not always get higher, but good hyperparameters can make it more stable. For convenience and fairness, we always report the final epoch's accuarcy.
Best,
Zihao
I think i do understand, thank you！