low accuracy on DG-15
xyy-ict opened this issue · 3 comments
After downloading the code, I follow the instructions provided and run the code on the DG-15 dataset without any modification. However, the accuracy is very low on DG-15 (64% total accuracy). Do you know the possible reasons behind this unexpected outcome?
The last two lines of the log are:
"epoch 499: {'E_pred': -8.388452115468681e-05, 'E_gan': -10.397211074829102, 'D': 10.397211074829102}
[Test][499] Accuracy: total average 64.0, test average 40.0, in each domain [100. 0. 49. 100. 100. 100. 37. 0. 100. 5. 18. 96. 100. 55. 100.]"
Hi xyy-ict,
Thank you for using our code! Based on the experiment result, it seems like the inherent unstability of adversarial training caused the issue. Here are some suggestions that might help:
- Lower the weight of adversarial training loss. You can modify the parameter "opt.lambda_gan" to 0.5 in "GRDA/configs/config_random_15_new.py". Feel free to try values between 0.1 to 0.6.
- If (1) does not work, you can further reduce the learning rate of encoder/decoder. You can modify the parameter "opt.lr_d" and "opt.lr_e" to 3e-5 in "GRDA/configs/config_random_15_new.py". Values between 1e-5 to 1e-4 should also be worth testing.
If you still encounter the same issue after these adjustments, please provide me with the full log and specify your PyTorch version. Reach out to me if you have any additional questions!
I lower the weight of adversarial training loss to 0.1 and the issue was successfully resolved. Thanks a lot for your help.
You are welcome!